Improved complex words simplification in non-annotated text using a deep-learning-based lexical complexity prediction model

Replacing complex words with simpler expressions is an effective way to improve readability and language understanding. We propose combining multiple levels of distributional semantics to measure word complexity beyond superficial lexical features. Without relying on predefined complexity lexicons, our lexical complexity prediction (LCP) model achieves performance comparable to human annotations and supports substitution retrieval for text simplification (TS). Unlike machine-translation-driven TS models trained on annotated corpora, our framework contains two modules: a supervised LCP module and an unsupervised simplification module. In the 2024 Multilingual Lexical Simplification Pipeline English Shared Task, the proposed LCP model is competitive with other state-of-the-art models. Additionally, experiments on TurkCorpus show that our model performs competitively with other popular supervised models while satisfying the requirements of TS in terms of adequacy, fluency, and simplicity.