  This paper presents an approach based on supervised machine learning methods
to build a classifier that can identify text complexity in order to present
Arabic language learners with texts suitable to their levels. The approach is
based on machine learning classification methods to discriminate between the
different levels of difficulty in reading and understanding a text. Several
models were trained on a large corpus mined from online Arabic websites and
manually annotated. The model uses both Count and TF-IDF representations and
applies five machine learning algorithms; Multinomial Naive Bayes, Bernoulli
Naive Bayes, Logistic Regression, Support Vector Machine and Random Forest,
using unigrams and bigrams features. With the goal of extracting the text
complexity, the problem is usually addressed by formulating the level
identification as a classification task. Experimental results showed that
n-gram features could be indicative of the reading level of a text and could
substantially improve performance, and showed that SVM and Multinomial Naive
Bayes are the most accurate in predicting the complexity level. Best results
were achieved using TF-IDF Vectors trained by a combination of word-based
unigrams and bigrams with an overall accuracy of 87.14% over four classes of
complexity.
