Lee Becker
ABSTRACT
A key challenge for dialogue-based intelligent tutoring systems lies in selecting follow-up questions that are not only context relevant but also encourage self-expression and stimulate learning. This paper presents an approach to ranking candidate questions for a given dialogue context and introduces an evaluation framework for this task. We learn to rank using judgments collected from expert human tutors, and we show that adding features derived from a rich, multi-layer dialogue act representation improves system performance over baseline lexical and syntactic features to a level in agreement with the judges. The experimental results highlight the important factors in modeling the questioning process. This work provides a framework for future work in automatic question generation and it represents a step toward the larger goal of directly learning tutorial dialogue policies directly from human examples.
- Manish Agarwal and Prashanth Mannem. 2011. Automatic gap-fill question generation from text books automatic gap-fill question generation from text books automatic gap-fill questions from text books. In Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications. Google Scholar
Digital Library
- I. L. Beck, M. G. McKeown, J. Worthy, C. A. Sandora, and L. Kucan. 1996. Questioning the author: A year-long classroom implementation to engage students with text. The Elementary School Journal, 96(4):387--416.Google ScholarCross Ref
- L. Becker, R. D. Nielsen, and W. Ward. 2009. What a pilot study says about running a question generation challenge. In Proceedings of the Second Workshop on Question Generation, Brighton, England, July.Google Scholar
- L. Becker, W. Ward, S. van Vuuren, and M. Palmer. 2011. Discuss: A dialogue move taxonomy layered over semantic representations. In In Proceedings of the International Conference on Computational Semantics (IWCS) 2011, Oxford, England, January 12--14. Google ScholarDigital Library
- K. E. Boyer, E. Y. Ha, M. Wallis, R. Phillips, M. A. Vouk, and J. C. Lester. 2009a. Discovering tutorial dialogue strategies with hidden markov models. In Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED '09), pages 141--148, Brighton, U.K. Google ScholarDigital Library
- K. E. Boyer, W. J. Lahti, R. Phillips, M. D. Wallis, M. A. Vouk, and J. C. Lester. 2009b. An empirically derived question taxonomy for task-oriented tutorial dialogue. In Proceedings of the Second Workshop on Question Generation, pages 9--16, Brighton, U.K.Google Scholar
- M. Buckley and M. Wolska. 2008. A classification of dialogue actions in tutorial dialogue. In Proceedings of COLING 2008, pages 73--80. ACL. Google ScholarDigital Library
- H. C. Bunt. 2009. The DIT++ taxonomy for functional dialogue markup. In Proc. EDAML 2009.Google Scholar
- Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2): pp. 249--254. Google ScholarDigital Library
- M. Chi, P. Jordan, K. VanLehn, and M. Hall. 2008. Reinforcement learning-based feature selection for developing pedagogically effective tutorial dialogue tactics. In Ryan S. Baker, Tiffany Barnes, and Joseph Becker, editors, Proceedings of the 1st International Conference on Educational Data Mining, pages pp 258--265.Google Scholar
- M. Chi, P. W. Jordan, K. VanLehn, and D. J. Litman. 2009. To elicit or to tell: Does it matter? In Artificial Intelligence in Education, pages 197--204. Google ScholarDigital Library
- M. Chi, K. VanLehn, and D. Litman. 2010. Do microlevel tutorial decisions matter: Applying reinforcement learning to induce do microlevel tutorial decisions matter. In Vincent Aleven, Judy Kay, and Jack Mostow, editors, Preceedings of the 10th Internation Confernce on Intelligent Tutoring Systems (ITS 2010). Google ScholarDigital Library
- William W. Cohen, Robert E. Schapire, and Yoram Singer. 1998. Learning to order things. In Advances in Neural Information Processing Systems 10 (NIPS 1998). Google ScholarDigital Library
- A. Collins and A. Stevens. 1982. Goals and methods for inquiry teachers. Advances in Instructional Psychology, 2.Google Scholar
- M. G. Core and J. F. Allen. 1997. Coding dialogs with the DAMSL annotation scheme. In AAAI Fall Symposium, pages 28--35.Google Scholar
- A. C. Graesser and N. K. Person. 1994. Question asking during tutoring. American Educational Research Journal, 31:104--137.Google ScholarCross Ref
- A. C. Graesser, P. Chipman, B. C Haynes, and A. Olney. 2005. Autotutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions in Education, 48:612--618. Google ScholarDigital Library
- M. Heilman and N. A. Smith. 2010. Good question! statistical ranking for question generation. In Proceedings of NAACL/HLT 2010. Google ScholarDigital Library
- T. Joachims. 1999. Making large-scale svm learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT-Press. Google ScholarDigital Library
- J. H. Kim, M. Glass, R. Freedman, and M. W. Evens. 2000. Learning the use of discourse markers in tutorial dialogue learning the use of discourse markers in tutorial dialogue. In Proceedings of the 22nd Annual Conference of the Cognitive Science Society.Google Scholar
- D. Litman and S. Silliman. 2004. Itspoke: An intelligent tutoring spoken dialogue system. In Companion Proceedings of the Human Language Technology Conference: 4th Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL). Google ScholarDigital Library
- W. C. Mann and S. A Thompson. 1986. Rhetorical structure theory: Description and construction of text structures. In In Proceedings of the Third International Workshop on Text Generation, August.Google Scholar
- A. K. McCallum, 2002. MALLET: A Machine Learning for Language Toolkit. http://mallet.cs.umass.edu.Google Scholar
- R. D. Nielsen, J. Buckingham, G. Knoll, B. Marsh, and L. Palen. 2008. A taxonomy of questions for question generation. In Proceedings of the Workshop on the Question Generation Shared Task and Evaluation Challenge, September.Google Scholar
- R. M. Pilkington. 1999. Analysing educational discourse: The discount scheme. Technical Report 99/2, Computer Based Learning Unit, University of Leeds.Google Scholar
- Owen Rambow, Monica Rogati, and Marilyn A. Walker. 2001. Evaluating a trainable sentence planner for a spoken dialogue system evaluating a trainable sentence planner for a spoken dialogue system. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (ACL 2001). Google ScholarDigital Library
- C. P. Rose, P. Jordan, M. Ringenberg, S. Siler, K. VanLehn, and A. Weinstein. 2001. A comparative evaluation of socratic versus didactic tutoring. In Proceedings of Cognitive Sciences Society.Google Scholar
- C. P. Rose, D. Bhembe, S. Siler, R. Srivastava, and K. VanLehn. 2003. The role of why questions in effective human tutoring. In Proceedings of Artificial Intelligence in Education (AIED 2003).Google Scholar
- D. Tsovaltzi and E. Karagjosova. 2004. A view on dialogue move taxonomies for tutorial dialogues. In Proceedings of SIGDIAL 2004, pages 35--38. ACL.Google Scholar
- D. Tsovaltzi and C. Matheson. 2001. Formalising hinting in tutorial dialogues. In In EDILOG: 6th workshop on the semantics and pragmatics of dialogue, pages 185--192.Google Scholar
- K. VanLehn, A. C. Graesser, G. T. Jackson, P. Jordan, A. Olney, and C. P. Rose. 2007. When are tutorial dialogues more effective than reading? Cognitive Science, 31(1):3--62.Google ScholarCross Ref
- Marilyn A. Walker, Owen Rambow, and Monica Rogati. 2001. SPOT: A trainable sentence planner. In Proceedings of the North American Meeting of the Association for Computational Linguistics (NAACL). Google ScholarDigital Library
- A. Ward, D. Litman, and M. Eskenazi. 2011a. Predicting change in student motivation by measuring cohesion between predicting change in student motivation by measuring cohesion between tutor and student. In Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications, pages 136--141. Google ScholarDigital Library
- W. Ward, R. Cole, D. Bolaños, C. Buchenroth-Martin, E. Svirsky, S. van Vuuren, T. Weston, J. Zheng, and L. Becker. 2011b. My science tutor: A conversational multi-media virtual tutor for elementary school science. ACM Transactions on Speech and Language Processing (TSLP), 7(4), August. Google ScholarDigital Library
- W. Ward. 1994. Extracting information from spontaneous speech. In Proceedings of the International Conference on Speech and Language Processing (ICSLP).Google Scholar
- Xuchen Yao. 2010. Question generation with minimal recursion semantics. Master's thesis, Saarland University.Google Scholar
- Question ranking and selection in tutorial dialogues
Recommendations
Distilling dialogues: a method using natural dialogue corpora for dialogue systems development
ANLC '00: Proceedings of the sixth conference on Applied natural language processingWe report on a method for utilising corpora collected in natural settings. It is based on distilling (re-writing) natural dialogues to elicit the type of dialogue that would occur if one the dialogue participants was a computer instead of a human. The ...
(Semi-)Automatic Analysis of Dialogues
ICAART 2014: Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1We study human-human dialogues and human-computer dialogues with the aim to determine which dialogue acts and communicative strategies do the participants of interaction use, and which structural parts does a dialogue include. In order to simplify the ...
Acoustic-Prosodic Entrainment and Rapport in Collaborative Learning Dialogues
MLA '14: Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics Workshop and Grand ChallengeIn spoken dialogue analysis, the speech signal is a rich source of information. We explore in this paper how low level features of the speech signal, such as pitch, loudness, and speaking rate, can inform a model of student interaction in collaborative ...
Comments