Previous work on readability assessment, applications of readability assessment and research work done by Lejun Feng towards readability assessment

Previous work on readability assessment, applications of readability assessment and research work done by Lejun Feng towards readability assessment

Are you fed up reading the text which is not of your choice or level????

For this we require the readability assessment tool to select the text of our choice :) :)

Let me start with-what is Readability and what all we need to consider while comprehending or understanding a given text.Let me explain:

Readability is defined as a measure of ease with which a written text can be understood.

Now the second question arises What makes a text easy or difficult to understand? For this let us go through the previous work done on readability-assessment.

Relevant Literature & Previous Work on Readability Assessment

Let us first see the characteristics and limitations of traditional readability metrics and recent statistical development in the field of readability.

Traditional readability metrics are given below:

1.Flesch Reading Ease and the FleschKincaid grade level formulas (Flesch, 1979) use average sentence length and average syllables per word to calculate the grade level of a text.

2.Gunning FOG (Gunning, 1952) and the SMOG (McLaughlin, 1969) index use average sentence length and the percentage of words with at least three syllable as parameters

3.Automated Readability Index (Senter and Smith, 1967) counts the number of characters per word instead to determine word difficulty.

4.Dale-Chall formula uses the percentage of difficult words (words that do not appear in the list) and average sentence length to predict the grade level of a text.

5.Stenner et al. (1983) had analyzed more than 50 lexical variables and did extensive correlation tests to find out that word frequency and sentence length have the most predictive power in ranking the reading difficulty of texts contained in their experiment data.

Advantages of traditional readability metric is explained as follows:

>>These traditional metrics are widely used, especially in educational settings, because they are simple and easy to calculate.

>>Grade levels that are calculated by the above methods indicate the number of years of education generally required to understand the text. It is generally understood that reading difficulty increases with grade level. They are a commonly accepted index for reading difficulty of a text, especially in educational settings, because the scale of grade levels make it easier for teachers, parents, librarians, and others to judge the readability level of various books and texts. Another reason to look at grade levels is that they have been widely used in previous research.

Drawbacks of traditional readability metrics:

>>>They ignored syntactic constituents, the structure of the text, local and global discourse coherence across the text(using the coherent basis for discourse i.e., familiarity of the discourse topic to the reader, readers’ prior knowledge and motivation to read.

>>>The traditional metrics cannot capture content information and often misjudge the reading difficulty of scientific web documents.

Statistical approaches towards readability metrics

Si and Callan (2001) used unigram language models to capture content information from scientific web pages. A linear model was built combining language models with sentence length.

CollinsThompson and Callan (2004) adopted Smoothed Unigram model to capture vocabulary variation across all grade levels contained in the corpus,their Smoothed Unigram model is purely vocabulary-based and does not contain any syntactic features.Although vocabulary-based unigram language models help capture important content information and variation of word usage, they do not capture syntactic information.

Schwarm and Ostendorf (2005)

used Charniak’s parser (Charniak, 2000) and higher order n-gram (n = 3) models over a combination of word and part-of-speech (POS) sequences to capture syntactic and semantic features.But it was limited to the study of lexical and syntactic features with regard to text comprehensibility

Heilman et al. (2007)

The readability measurement was motivated by pedagogical differences in first language (L1) and second language (L2) learning. They argue that grammatical features play a more important role in L2 texts than in L1 texts because, unlike L1 learners who learn grammar through natural interaction, L2 learners learn grammatical patterns explicitly from L2 textbooks.

But it was limited to the study of lexical and syntactic features with regard to text comprehensibility.

Barzilay and Lapata (2008)The first work on discourse relation was done by Barzilay and Lapata, designed and implemented an entity-grid model to capture the distribution of entity transition patterns at sentence to sentence level.

The cognitive science reveals that the most important process during reading comprehension lie in discourse comprehension, which entails making appropriate inferences from concepts and propositions, connecting and/or integrating related information to construct a coherent memory representation.

Their work was not motivated by text readability, but rather by other NLP tasks related to text generation, such as text ordering and summary coherence rating.

Pitler and Nenkova (2008) for the first time looked at readability factors at all three linguistic levels: lexical, syntactic and discourse.In the PDTB(Penn Discourse tree bank), all discourse connectives and the relations between two adjacent sentences of a text were manually annotated.Among all individual factors analyzed at all three linguistic levels, the likelihood of discourse relations with text length taken into account shows the strongest correlation with human readability ratings (r = .4835).Their work is novel and inspiring, because it touched the core of text comprehension and showed a new direction in readability study that has been long overdue

Limitation of Petler and Nenkova work

1.It cannot be adopted for any corpus other than the PDTB.

2.they mainly focussed on text style rather on text readability i.e. how well a text is written rather than3. how difficult or easy a text is to read.

3.they experiment conducted was only on 30 articles and because they relied only on limited subjective human ratings,their study lacks any objective measure..

After reading all the previous work done on readability.Let me conclude in a simple way that the readability cannot solely judged by

1. l >>lexical tokenisation( which looks at three factors: the number of syllables a word contains, the number of characters a word contains, and word frequency)

2. >>syntactic representation(the complexity of sentences is solely judge by their average length in words).

3.3>>. Sentence processing

But also on .....

>>> discourse relation to buid coherent memory representation of text by the reader. The discourse relation in totally means the amount of prior knowledge the reader needs to apply, the inferences the reader need to make, relate the text with the accumulated knowledge, the references required to understand the text, searching and retrieving the relevant information for comprehending.

1. >>> It also depends on the working memory capacity. If the text is not related to the main topic of discussion that means the text is not present under current working memory then the reader has to search the long term memory for understanding .

Now coming to the major contribution towards readability assessment done by Lejun Feng :

1.The readability from a text comprehension point of view; in particular, paid special attention to discourse processes that are crucial for constructing and maintaining local and global memory coherence of a text(we can say it as short term and long term memory), which is key to successful text comprehension.

2.The processes that occur in discourse comprehension, which contains the activities such as resolving entities, inferring meaning from words and phrases, assessing and evaluating semantic relations among concepts and propositions and making connections among them, using background knowledge to generate appropriate inferences to fill in gaps, and integrating new information into existing semantic structure to achieve and maintain coherent memory representation of a text.

3.The thesis propose to apply advanced NLP techniques to implement three classes of novel discourse features that have not been studied by any of the previous research.i.e.density of entities, lexical chains, coreferential inference features .

4.It focused not only on intrinsic text properties, but viewed text comprehensibility as the result of the interaction between the text and the reader’s prose processing ability, the characteristics of a given reader was taken into the readability study by addressing constraints of working memory capacity placed on the reader’s comprehension effort.The constraints on working memory is highlighted here because the individuals with ID(intellectual disability) do not have the same memory capacity as the one without ID.

5.Working memory while extracting the discourse features was taken under consideration. Working memory has great impact on various language comprehension activities, because it provides temporary storage and simultaneous manipulation of information and coordinates resources that are necessary for comprehension processes during reading. Since individuals with ID(intellectual disabilities) donot have the same working capacity as the individuals without ID(intellectual disabilities) which accounts in variation of comprehension performance.

6.The thesis proposes the development of the automatic readability assessment tool which consists of four major parts: data collection, feature extraction and implementation, building and evaluating the tool on labeled corpora, and test and evaluating the tool on unlabeled texts from different domain.

7.The study on readability combines various proxies, such as paired original/simplified corpora, grade levels, subjective ratings by experts and users, and objective observations in our user studies, to get all those underlying text properties that are associated with reading difficulties.

Let me conclude by telling the applications of buiding the automatic-readability-assessment tool.

1. >>> in educational settings, school children, second language learners, adults with low literacy can use the tool to select reading material that is of their interest and tailored to their varying reading proficiency.

2. >>> language instructors can use this tool to select teaching material effectively that is at appropriate level of reading difficulty for target readers.

3. >>>It can be used to rank the documents by reading difficulty for automated systems such as text simplification, text summarization, machine translation and other text generation systems for example tool can be used to select documents that are at appropriate level of reading difficulty among those on similar topic for the target system to begin with.

4. >>> A reliable tool that can accurately assess the change of reduction in reading difficulty before and after simplification process can be provided by this tool.

5. >>> we can use the tool to check the quality of text generated by systems such as text summarization, machine translation and text ordering system. Comparing the reading difficulty before and after the change of simplification process is required to check the coherence(as coherent text are easier to read).

I hope the above information will help us before getting delved into the more understanding of thesis .Finally I could say that after reading the thesis for three times made me to grade myself to the same level of understanding J