Automatic evaluation of aspects of document quality

ABSTRACT

Coh-Metrix is a web-based application currently in development that automatically evaluates text. It uses two central concepts from discourse processing: text-based cohesion and situation-model based coherence. Cohesion is the degree to which components of the text are linked. Coherence is the representation of the world that the text conveys. Our intention is for Coh-Metrix to eventually map the cohesion of a text to the background knowledge and reading skills of the reader. Coh-Metrix will then be able to give feedback to a writer about which aspects of the text are cohesive and which lack cohesion. This will enable the writer to determine which aspects of the text need to be improved. Applications of Coh-Metrix on document quality as well as other future directions for the development of Coh-Metrix are discussed.

References

1. Axelrod, R. B., & Cooper, C. R. (2001). The St Martins Guide to Writing. Bedford/St Martins: Boston, MA.
2. Burstein, J., Kukich, K., Wolff, S., Lu, C., Chodorow, M., Braden-Harder, L., & Harris, M. D. (1998). Automated Scoring Using A Hybrid Feature Identification Technique. Proceedings of the Annual Meeting of the Association of Computational Linguistics, August 1998. Montreal, Canada.
3. Flower, L. (1993). Problem-Solving Strategies for Writers. (4th Ed). Fort Worth: Harcourt Brace Jovanovich.
4. Foltz, P., Kintsch, W., & Landauer, T. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25, 285--307.
5. Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The Intelligent Essay Assessor: Applications to Educational Technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2).
6. Graesser, A.C., McNamara, D.S.,& Louwerse, M.M. (2003). What do readers need to learn in order to process coherence relations in narrative and expository text. In A.P. Sweet and C.E. Snow (Eds.), Rethinking reading comprehension (pp. 82--98). New York: Guilford Press.
7. Graesser, A.C., McNamara, D., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers. In press.
8. Kintsch, W. (1998) Comprehension: A paradigm for cognition. New York: Cambridge University Press.
9. Landauer, T., and S. Dumais. (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211--240.
10. Louwerse, M.M. (2002). An analytic and cognitive parameterization of coherence relations. Cognitive Linguistics, 12, 291--315.
11. Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3), 243--281.
12. McNamara, D. S., Louwerse, M. M., & Graesser, A. C. (2002). Coh-Metrix: Automated cohesion and coherence scores to predict readability and facilitate comprehension. Unpublished technical report: University of Memphis.
13. McNamara, D. S., & Kintsch, W. (1996). Learning from Text: Effects of prior knowledge and text coherence. Discourse Processes, 22, 247--287.
14. McNamara, D.S., Kintsch, E., Songer, N.B., & Kintsch, W. (1996). Are good texts always better? Text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1--43.
15. Reid, S. (1997). The Prentice-Hall guide for college writers: a customized edition. Pearson Custom Publishing: Needham Heights, MA.
16. Schreiner, M., Rehder, B., Landauer, T., & Laham, D. (1997). How latent semantic analysis (LSA) represents essay semantic content: Technical issues and analysis. In M. Shafto and P. Langley (Eds.), Proceedings of the19th Annual Meeting of the Cognitive Science Society (pp 1041). Mawhwah, NJ: Erlbaum.
17. Stenner, A. J. (1998). Measuring reading comprehension with the lexile framework. 4th North American Conference on Adolescent/Adult Literacy: Washington, D.C.
18. Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123, 162--185.

Index Terms

• Applied computing

  • Law, social and behavioral sciences

    • Psychology