Longitudinal trends in sentiment polarity and readability of an online masters of computer science course

ABSTRACT

In four years, the Georgia Tech Online MS in CS (OMSCS) program has grown from 200 students to over 6000. Despite early evidence of success, there is a need to evaluate the program's effectiveness. In this paper, we focus on trends from Fall 2014 to Fall 2017 in the on-campus and online sections of one OMSCS course, Knowledge-Based Artificial-Intelligence (KBAI). We leverage sentiment analysis and readability assessments to quantify the evolving quality of discourse on the online forum discussions of the various sections. The research was conducted as a longitudinal study, and aims to evaluate the success of the KBAI course by comparing trends between residential and online sections. Despite slight downward trends in online discourse quality and sentiment polarity, our results suggest that the growing OMSCS program has been successful in replicating the quality of learning experienced by on-campus students in the KBAI course.

References

1. Alpert, W. T., Couch, K. A., & Harmon, O. R. (2016). A randomized assessment of online learning. The American Economic Review, 106(5), 378--382.
2. Brindley, J., Blaschke, L. M., & Walti, C. (2009). Creating effective collaborative learning groups in an online environment. The International Review of Research in Open and Distributed Learning, 10(3).
3. Goel, A., & Joyner, D. (2016). An Experiment in Teaching Artificial Intelligence Online. International Journal for the Scholarship of Technology-Enhanced Learning (1), 1.
4. Goodman, J., Melkers, J., & Pallais, A. (2016). Can Online Delivery Increase Access to Education? (No. w22754). National Bureau of Economic Research.
5. Hu, N., Bose, I., Koh, N. S., & Liu, L. (2012). Manipulation of online reviews: An analysis of ratings, readability, and sentiments. Decision Support Systems, 52(3), 674--684.
6. Hutto, C. J., & Gilbert, E. (2014, May). Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Eighth international AAAI conference on weblogs and social media.
7. Joyner, D., Goel, A., & Isbell, C. (2016) The Unexpected Pedagogical Benefits of Making Higher Education Accessible. In Procs. Third Conference on Learning at Scale, Edinburgh, UK, April 2016.
8. Kagklis, V., Karatrantou, A., Tantoula, M., Panagiotakopoulos, C. T., & Verykios, V. S. (2015). A learning analytics methodology for detecting sentiment in student for a: A case study in distance education. European Journal of Open, Distance and E-learning, 18(2), 74--94. Chicago
9. Kirsch, I. S. (1993). Adult literacy in America: A first look at the results of the National Adult Literacy Survey. US Government Printing Office, Superintendent of Documents, Washington, DC 20402 (Stock No. 065-00000588-3).
10. Kiritchenko, S., Zhu, X., & Mohammad, S. M. (2014). Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research, 50, 723--762.
11. Ortigosa, A., Martín, J. M., & Carro, R. M. (2014). Sentiment analysis in Facebook and its application to e-learning. Computers in Human Behavior, 31, 527--541.
12. Štajner, S., Evans, R., Orasan, C., & Mitkov, R. (2012). What can readability measures really tell us about text complexity. In Proceedings of the the Workshop on Natural Language Processing for Improving Textual Accessibility (NLP4ITA) (pp. 14--21).