A Framework for Dynamic Knowledge Modeling in Textbook-Based Learning

ABSTRACT

Various e-learning systems that provide electronic textbooks are gathering data on large numbers of student reading interactions. This data can potentially be used to model student knowledge acquisition. However, reading activity is often overlooked in canonical student modeling. Prior studies modeling learning from reading either estimate student knowledge at the end of all reading activities, or use quiz performance data with expert-crafted knowledge components (KCs). In this work, we demonstrate that the dynamic modeling of student knowledge is feasible and that automatic text analysis can be applied to save expert effort. We propose a data-driven approach for dynamic student modeling in textbook-based learning. We formulate the problem of modeling learning from reading as a reading-time prediction problem, reconstruct existing popular student models (such as Knowledge Tracing) and explore two automatic text analysis approaches (bag-of-words-based and latent semantic-based) to build the KC model. We evaluate the proposed framework using a dataset collected from a Human-Computer Interaction course. Results show that our approach for reading modeling is plausible; the proposed Knowledge Tracing-based student model reliably outperforms baselines and the latent semantic-based approach can be a promising way to construct a KC model. Serving as the first step to model dynamic knowledge in textbook-based learning, our framework can be applied to a broader context of open-corpus personalized learning.

References

1. S. Bechhofer, C. Goble, L. Carr, W. Hall, S. Kampa, and D. De Roure. Cohse: Conceptual open hypermedia service. In Annotation for the Semantic Web, pages 193--210. IOS Press, 2003.
2. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993--1022, 2003.
3. P. Brusilovsky and J. Eklund. A study of user-model based link annotation in educational hypermedia. Journal of Universal Computer Science, 4(4):429--448, 1998.
4. P. Brusilovsky, C. Karagiannidis, and D. Sampson. Layered evaluation of adaptive learning systems. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4/5):402 -- 421, 2004.
5. P. Brusilovsky and J. Kim. Enhancing electronic books with spatial annotation and social navigation support. In Proc. 5th Int. Conf. Universal Digital Library (ICUDL), 2009.
6. P. Brusilovsky and E. Millán. User models for adaptive hypermedia and adaptive educational systems. In The adaptive web, pages 3--53. Springer-Verlag, 2007.
7. P. Brusilovsky and L. Pesin. Adaptive navigation support in educational hypermedia: An evaluation of the isis-tutor. Journal of Computing and Information Technology, 6(1):27--38, 1998.
8. P. Brusilovsky, S. Sosnovsky, and M. Yudelson. Addictive links: The motivational value of adaptive link annotation. New Review of Hypermedia and Multimedia, 15(1):97--118, 2009.
9. K. R. Canini, L. Shi, and T. L. Griffiths. Online inference of topics with latent dirichlet allocation. In Int. Conf. Artificial Intelligence and Statistics, pages 65--72, 2009.
10. R. P. Carver. The causes of high and low reading achievement. Routledge, 2000.
11. H. Cen, K. R. Koedinger, and B. Junker. Comparing two irt models for conjunctive skills. In Proc. 10th Int. Conf. Intelligent Tutoring Systems, pages 796--798, Berlin/Heidelberg, 2008. Springer-Verlag.
12. D. Chin. Empirical evaluations of user models and user-adapted systems. User Modeling and User-Adapted Interaction, 11(1--2):181--194, 2001.
13. A. T. Corbett and J. R. Anderson. Knowledge tracing: Modelling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4):253--278, 1995.
14. A. Davidovic, J. Warren, and E. Trichina. Learning benefits of structural example-based adaptive tutoring systems. IEEE Transactions on Education, 46(2):241--251, 2003.
15. R. Farzan and P. Brusilovsky. Social navigation support in e-learning: What are real footprints. In IJCAI'05 Workshop on Intelligent Techniques for Web Personalization, pages 49--56, 2005.
16. Y. Gong, J. E. Beck, and N. T. Heffernan. Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In Proc. 10th Int. Conf. Intelligent Tutoring Systems, pages 35--44. Springer, 2010.
17. J. P. González-Brenes, Y. Huang, and P. Brusilovsky. General features in knowledge tracing: Applications to multiple subskills, temporal item response theory, and expert knowledge. In Proc. of the 7th Int. Conf. on Educational Data Mining, pages 84--91, 2014.
18. J. Guerra, D. Parra, and P. Brusilovsky. Encouraging online student reading with social visualization. In The 2nd Workshop on Intelligent Support for Learning in Groups at the 16th Conference on Artificial Intelligence in Education, pages 47--50, 2013.
19. J. Guerra, S. Sosnovsky, and P. Brusilovsky. When one textbook is not enough: Linking multiple textbooks using probabilistic topic models. In Scaling up Learning for Sustained Impact, pages 125--138. Springer, 2013.
20. N. Henze and W. Nejdl. Adaptation in open corpus hypermedia. International Journal of Artificial Intelligence in Education, 12(4):325--350, 2001.
21. I.-H. Hsiao, S. Sosnovsky, and P. Brusilovsky. Guiding students to the right questions: adaptive navigation support in an e-learning system for java programming. Journal of Computer Assisted Learning, 26(4):270--283, 2010.
22. Y. Huang, J. P. González-Brenes, R. Kumar, and P. Brusilovsky. A framework for multifaceted evaluation of student models. In Proceedings of the 8th International Conference on Educational Data Mining, pages 203--210, 2015.
23. M. A. Just and P. A. Carpenter. The psychology of reading and language comprehension. Allyn & Bacon, 1987.
24. A. Kavcic. Fuzzy user modeling for adaptation in educational hypermedia. IEEE Transactions on Systems, Man, and Cybernetics, 34(4):439--449, 2004.
25. K. Koedinger, J. Stamper, E. McLaughlin, and T. Nixon. Using data-driven discovery of better student models to improve student learning. In Proceedings of the 16th International Conference on Artificial Intelligence in Education., pages 412--430, 2013.
26. H. P. Luhn. The automatic creation of literature abstracts. IBM Journal of research and development, 2(2):159--165, 1958.
27. B. Martin, A. Mitrovic, K. Koedinger, and S. Mathan. Evaluating and improving adaptive educational systems with learning curves. User Modeling and User-Adapted Interaction, 21(3):249--283, 2011.
28. J. T. Mayes, M. R. Kibby, and H. Watson. Strathtutor: The development and evaluation of a learning-by-browsing on the macintosh. Computers and Education, 12(1):221--229, 1988.
29. A. Newell and P. S. Rosenbloom. Mechanisms of skill acquisition and the law of practice. Cognitive skills and their acquisition, 1:1--55, 1981.
30. K. O'Hara. Towards a typology of reading goals. Technical Report EPC-1996--107, Rank Xerox Research Centre Cambridge Laboratory, 1996.
31. K. A. Papanikolaou, M. Grigoriadou, H. Kornilakis, and G. D. Magoulas. Personalising the interaction in a web-based educational hypermedia system: the case of inspire. User Modeling and User Adapted Interaction, 13(3):213--267, 2003.
32. A. Paramythis and S. Weibelzahl. A decomposition model for the layered evaluation of interactive adaptive systems. In L. Ardissono, P. Brna, and A. Mitrovic, editors, 10th International User Modeling Conference, volume 3538 of Lecture Notes in Artificial Intelligence, pages 438--442. Springer Verlag, 2005.
33. P. I. Pavlik Jr., H. Cen, and K. R. Koedinger. Performance factors analysis -- a new alternative to knowledge tracing. In Proceedings of the 14th International Conference on Artificial Intelligence in Education, pages 531--538. IOS Press, 2009.
34. L. Rabiner and B. Juang. An introduction to Hidden Markov Models. ASSP Magazine, IEEE, 3(1):4--16, 1986.
35. M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth. The author-topic model for authors and documents. In Proceedings of the 20th conference on Uncertainty in artificial intelligence, pages 487--494. AUAI Press, 2004.
36. G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513--523, 1988.
37. F. Sebastiani. Machine learning in automated text categorization. ACM computing surveys (CSUR), 34(1):1--47, 2002.
38. S. Sosnovsky and P. Brusilovsky. Evaluation of topic-based adaptation and student modeling in quizguide. User Modeling and User-Adapted Interaction, 25(4):371--424, 2015.
39. W. J. van der Linden and R. K. Hambleton. Handbook of Modern Item Response Theory. Springer Verlag, New York, NY, 1997.
40. H. M. Wallach. Topic modeling: beyond bag-of-words. In Proceedings of the 23rd international conference on Machine learning, pages 977--984. ACM, 2006.
41. G. Weber and P. Brusilovsky. Elm-art: An adaptive versatile system for web-based instruction. International Journal of Artificial Intelligence in Education, 12(4):351--384, 2001.
42. Y. Xu and J. Mostow. Comparison of methods to trace multiple subskills: Is LR-DBN best? In Proceedings of the 5th International Conference on Educational Data Mining, pages 41--48, Chania, Greece, 2012.
43. M. Yudelson, K. Koedinger, and G. Gordon. Individualized bayesian knowledge tracing models. In Proceedings of 16th International Conference on Artificial Intelligence in Education (AIED 2013)., pages 171--180, Berlin/Heidelberg, 2013. Springer-Verlag.
44. M. Yudelson, P. Pavlik, and K. Koedinger. User modeling -- a notoriously black art. In Proceedings of the 19th International Conference on User Modeling Adaptation and Personalization (UMAP 2011), pages 317--328, 2011.

Index Terms

• Applied computing

  • Education

    • E-learning

    • Interactive learning environments

• Computing methodologies

  • Artificial intelligence

    • Knowledge representation and reasoning

    • Natural language processing

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User models

    • Interaction paradigms

      • Hypertext / hypermedia

• Information systems

  • Information retrieval

    • Document representation

  • Information systems applications

    • Data mining

    • Decision support systems

      • Data analytics

  • World Wide Web

    • Web searching and information discovery

      • Personalization

• Mathematics of computing

  • Probability and statistics

    • Probabilistic algorithms

    • Probabilistic reasoning algorithms

      • Kalman filters and hidden Markov models

    • Statistical paradigms

      • Regression analysis

      • Time series analysis

• Social and professional topics

  • Professional topics

    • Computing education

      • Computing education programs

        • Information science education