Ryo Nagata
ABSTRACT
This paper proposes a method for evaluating grammatical error detection methods to maximize the learning effect obtained by grammatical error detection. To achieve this, this paper sets out the following two hypotheses --- imperfect, rather than perfect, error detection maximizes learning effect; and precision-oriented error detection is better than a recall-oriented one in terms of learning effect. Experiments reveal that (i) precision-oriented error detection has a learning effect comparable to that of feedback by a human tutor, although the first hypothesis is not supported; (ii) precision-oriented error detection is better than recall-oriented in terms of learning effect; (iii) F-measure is not always the best way of evaluating error detection methods.
- Bitchener, John, Stuart Young, and Denise Cameron. 2005. The effect of different types of corrective feedback on ESL student writing. Journal of Second Language Writing, 14(3):191--205.Google Scholar
Cross Ref
- Brockett, Chris, William B. Dolan, and Michael Gamon. 2006. Correcting ESL errors using phrasal SMT techniques. In Proc. of 21th International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 249--256, Sydney, Australia, July. Google ScholarDigital Library
- Burstein, Jill, Karen Kukich, Susanne Wolff, Chi Lu, Martin Chodorow, Lisa Braden-Harder, and Mary D. Harris. 1998. Automated scoring using a hybrid feature identification technique. In Proc. of 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, pages 206--210. Google ScholarDigital Library
- Chodorow, Martin and Claudia Leacock. 2000. An unsupervised method for detecting grammatical errors. In Proc. of 1st Meeting of the North America Chapter of ACL, pages 140--147. Google ScholarDigital Library
- Chodorow, Martin and Claudia Leacock. 2002. Techniques for detecting syntactic errors in text. In IEICE Technical Report (TL2002-39), pages 37--41.Google Scholar
- Chodorow, Martin, Joel R. Tetreault, and Na-Rae Han. 2007. Detection of grammatical errors involving prepositions. In Proc. of 4th ACL-SIGSEM Workshop on Prepositions, pages 25--30. Google ScholarDigital Library
- Ferris, Dana and Barrie Roberts. 2001. Error feedback in L2 writing classes: How explicit does it need to be? Journal of Second Language Writing, 10(3):161--184.Google ScholarCross Ref
- Han, Na-Rae, Martin Chodorow, and Claudia Leacock. 2004. Detecting errors in English article usage with a maximum entropy classifier trained on a large, diverse corpus. In Proc. of 4th International Conference on Language Resources and Evaluation, pages 1625--1628.Google Scholar
- Han, Na-Rae, Martin Chodorow, and Claudia Leacock. 2006. Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12(2):115--129. Google ScholarDigital Library
- Hirota, Shigeaki. 1992. Mastery (in Japanese). Kirihara Shoten, Tokyo.Google Scholar
- Iizuka, Shigeru and Satoshi Hagino. 1997. Prestige. Buneido, Tokyo.Google Scholar
- Izumi, Emi, Kiyotaka Uchimoto, Toyomi Saiga, Thepchai Supnithi, and Hitoshi Isahara. 2003. Automatic error detection in the Japanese learners' English spoken data. In Proc. of 41st Annual Meeting of ACL, pages 145--148. Google ScholarDigital Library
- Konishi, Tomoshichi and Kosei Minamide. 2007. Genious English-Japanese dictionary, 4th ed. Taishukan, Tokyo.Google Scholar
- Nagata, Ryo, Fumito Masui, Atsuo Kawai, and Naoki Isu. 2004. Recognizing article errors based on the three head words. In Proc. of Cognition and Exploratory Learning in Digital Age, pages 184--191.Google Scholar
- Nagata, Ryo, Takahiro Wakana, Fumito Masui, Atsuo Kawai, and Naoki Isu. 2005. Detecting article errors based on the mass count distinction. In Proc. of 2nd International Joint Conference on Natural Language Processing, pages 815--826. Google ScholarDigital Library
- Nagata, Ryo, Astuo Kawai, Koichiro Morihiro, and Naoki Isu. 2006. A feedback-augmented method for detecting errors in the writing of learners of English. In Proc. of 44th Annual Meeting of ACL, pages 241--248. Google ScholarDigital Library
- Okihara, Katsuaki. 1985. English writing (in Japanese). Taishukan, Tokyo.Google Scholar
- Robb, Thomas, Steven Ross, and Ian Shortreed. 1986. Salience of feedback on error and its effect on EFL writing quality. TESOL QUARTERY, 20(l):83--93.Google Scholar
- Yarowsky, David. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proc. of 33rd Annual Meeting of ACL, pages 189--196. Google ScholarDigital Library
Index Terms
- Evaluating performance of grammatical error detection to maximize learning effect
Recommendations
Weaken Grammatical Error Influence in Chinese Grammatical Error Correction
Natural Language Processing and Chinese ComputingAbstractChinese grammatical error correction (CGEC), a task of correcting grammatical errors in text, is treated as a translation task, where error sentences are “translated” to correct sentences. However, some grammatical errors in the training data can ...
Dynamic Negative Example Construction for Grammatical Error Correction Using Contrastive Learning
Chinese Computational LinguisticsAbstractGrammatical error correction (GEC) aims at correcting texts with different types of grammatical errors into natural and correct forms. Due to the difference of error type distribution and error density, current grammatical error correction systems ...
Grammatical error detection for corrective feedback provision in oral conversations
AAAI'11: Proceedings of the Twenty-Fifth AAAI Conference on Artificial IntelligenceThe demand for computer-assisted language learning systems that can provide corrective feedback on language learners' speaking has increased. However, it is not a trivial task to detect grammatical errors in oral conversations because of the unavoidable ...
Comments