Investigating the Usage Patterns of Algebra Nation Tutoring Platform

ABSTRACT

We study the usage of a self-guided online tutoring platform called Algebra Nation, which is widely by middle school and high school students who take the End-of-Course Algebra I exam at the end of the school year. This article aims to study how the platform contributes to increasing students' exam scores by examining users' logs over a three year period. The platform under consideration was used by more than 36,000 students in the first year, to nearly 67,000 by the third year, thus enabling us to examine how usage patterns evolved and influenced students' performance at scale. We first identify which Algebra Nation usage factors in conjunction with math overall preparation and socioeconomic factors contribute to the students' exam performance. Subsequently, we investigate the effect of increased teacher familiarity level with the Algebra Nation on students' scores across different grades through mediation analysis. The results show that the indirect effect of teacher's familiarity with the platform through increasing student's usage dosage is more significant in higher grades.

References

1. Dhammika Amaratunga and Javier Cabrera. 2001. Analysis of Data From Viral DNA Microchips. J. Amer. Statist. Assoc. 96 (2001), 1161--1170. Issue 456.
2. Reuben M. Barron and David A. Kenny. 1986. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology 51, 6 (1986), 1173--1182.
3. Andrew U Dane and Barry H. Schneider. 1998. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clinical Psychology Review 18, 1 (1998), 23--45.
4. Mingyu Feng, Jeremy Roschelle, Neil Heffernan, Janet Fairman, and Robert Murphy. 2014. Implementation of an Intelligent Tutoring System for Online Homework Support in an Efficacy Trial. In Intelligent Tutoring Systems. Springer International Publishing, Cham, 561--566.
5. Leo A. Goodman. 2012. On the exact variance of products. J. Amer. Statist. Assoc. 55, 292 (2012), 708--713.
6. Heewon Lee Gray, Isobel R. Contento, and Pamela A. Koch. 2015. Linking implementation process to intervention outcomes in a middle school obesity prevention curriculum: Choice, Control and Change. Health Education Research 30 (2015), 248--261. Issue 2.
7. Kosuke Imai, Luke Keele, and Teppei Yamamoto. 2010. Identification, Inference and Sensitivity Analysis for Causal Mediation Effects. Statist. Sci. 25, 1 (2010), 51--71.
8. Rita Karam, John F. Pane, Beth Ann Griffin, Abby Robyn, Andrea Phillips, and Lindsay Daugherty. 2017. Examining the implementation of technology-based blended algebra I curriculum at scale. Education Technology Research and Development 65 (2017), 399--425. Issue 2.
9. Jennifer L. Krull and David P. MacKinnon. 1999. Multilevel Mediation Modeling in Group-based Intervention Studies. Evaluation Review 23, 4 (1999), 418--444.
10. Jennifer L. Krull and David P. MacKinnon. 2001. Multilevel Modeling of Individual and Group Level Mediated Effects. Multivariate Behavioral Research 36 (2001), 249--277. Issue 2.
11. Heewon Lee, Isobel R. Contento, and Pamela Koch. 2013. Using a Systematic Conceptual Model for a Process Evaluation of a Middle School Obesity Risk-Reduction Nutrition Curriculum Intervention: Choice, Control and Change. Journal of Nutrition Education and Behavior 45 (2013), 126--136. Issue 2.
12. Lindsay Clare Matsumura, Helen E. Garnier, and Lauren B. Resnick. 2010. Implementing Literacy Coaching: The Role of School Social Resources. Educational Evaluation and Policy Analysis 32 (2010), 249--272. Issue 2.
13. R. Murphy, L. Gallagher, A. Krumm, J. Mislevy, and A. Hafter. 2014. Research on the Use of Khan Academy in Schools.
14. Sahba Akhavan Niaki, Clint P. George, George Michailidis, and Carole R. Beal. 2017. The Impact of an Online Tutoring Program for Algebra Readiness on Mathematics Achievements; Results of a Randomized Experiment. Technical report.
15. Florida Dept. of Education. 2017. Florida Standards Assessments: English Language Arts and Mathematics. http://www.fldoe.org/core/fileparse.php/5668/urlt/89FSAPacket.pdf
16. S. W. Raudenbush and A. S. Bryk. 2000. Sage Publictions, Thousand Oaks, CA.
17. Louise Ann Rohrbach, Melissa Gunning, Ping Sun, and Steve Sussman. 2010. The Project Towards No Drug Abuse (TND) dissemination trial: implementation fidelity and immediate outcomes. Prevention Science 11 (2010), 77--88. Issue 1.
18. Michael E. Sobel. 1982. Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models. Sociological Methodology 13 (1982), 290--312.

Index Terms

• Applied computing

  • Education

    • Computer-assisted instruction