Learning Loops: A Replication Study Illuminates Impact of HS Courses

ABSTRACT

A recent study about the effectiveness of subgoal labeling in an introductory computer science programming course both supported previous research and produced some puzzling results. In this study, we replicate the experiment with a different student population to determine if the results are repeatable. We also gave the experimental task to students in a follow-on course to explore if they had indeed mastered the programming concept. We found that the previous puzzling results were repeated. In addition, for the novice programmers, we found a statistically significant difference in performance based on whether the student had previous programming courses in high school. However, this performance difference disappears in a follow-on course after all students have taken an introductory computer science programming course. The results of this study have implications for how quickly students are evaluated for mastery of knowledge and how we group students in introductory programming courses.

References

1. Atkinson, R.K. et al. 2003. Aiding Transfer in Statistics: Examining the Use of Conceptually Oriented Equations and Elaborations During Subgoal Learning. Journal of Educational Psychology. 95, 4 (2003), 762.
2. Atkinson, R.K. et al. 2000. Learning from examples: Instructional principles from the worked examples research. Review of educational research. 70, 2 (2000), 181--214.
3. Atkinson, R.K. 2002. Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology. 94, 2 (2002), 416.
4. Atkinson, R.K. and Derry, S.J. 2000. Computer-based examples designed to encourage optimal example processing: A study examining the impact of sequentially presented, subgoal-oriented worked examples. Fourth International Conference of the Learning Sciences (2000).
5. Bennedsen, J. and Caspersen, M.E. 2005. An investigation of potential success factors for an introductory model-driven programming course. Proceedings of the first international workshop on Computing education research (2005), 155--163.
6. Bergin, S. and Reilly, R. 2005. Programming: factors that influence success. ACM SIGCSE Bulletin (2005), 411--415.
7. Bjork, R.A. 1994. Memory and metamemory considerations in the training of human beings. Metacognition: Knowing about Knowing. MIT Press.
8. Bransford, J. 2000. How people learn: Brain, mind, experience, and school. National Academies Press.
9. Campbell, V. and Johnstone, M. 2010. The significance of learning style with respect to achievement in first year programming students. Software Engineering Conference (ASWEC), 2010 21st Australian (2010), 165--170.
10. Catrambone, R. 1996. Generalizing solution procedures learned from examples. Journal of Experimental Psychology: Learning, Memory, and Cognition; Journal of Experimental Psychology: Learning, Memory, and Cognition. 22, 4 (1996), 1020.
11. Catrambone, R. 1994. Improving examples to improve transfer to novel problems. Memory & Cognition. 22, 5 (1994), 606--615.
12. Catrambone, R. 1998. The subgoal learning model: Creating better examples so that students can solve novel problems. Journal of Experimental Psychology: General. 127, 4 (1998), 355.
13. Chi, M. et al. 1989. Self-explanations: How students study and use examples in learning to solve problems. Cognitive science. 13, 2 (1989), 145--182.
14. Denny, P. et al. 2008. Evaluating a new exam question: Parsons problems. Proceeding of the Fourth international Workshop on Computing Education Research (Sydney, Australia, 2008), 113--124.
15. Eiriksdottir, E. and Catrambone, R. 2011. Procedural instructions, principles, and examples how to structure instructions for procedural tasks to enhance performance, learning, and transfer. Human Factors: The Journal of the Human Factors and Ergonomics Society. 53, 6 (2011), 749--770.
16. Evans, G.E. and Simkin, M.G. 1989. What best predicts computer proficiency? Communications of the ACM. 32, 11 (1989), 1322--1327.
17. Hagan, D. and Markham, S. 2000. Does it help to have some programming experience before beginning a computing degree program? ACM SIGCSE Bulletin (2000), 25--28.
18. Johnson, W.L. and Soloway, E. 1985. PROUST: Knowledge-based program understanding. Software Engineering, IEEE Transactions on. 3 (1985), 267--275.
19. Kalyuga, S. 2007. Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review. 19, 4 (2007), 509--539.
20. Leeper, R.R. and Silver, J.L. 1982. Predicting success in a first programming course. ACM SIGCSE Bulletin. 14, 1 (1982), 147--150.
21. Leppink, J. et al. 2013. Development of an instrument for measuring different types of cognitive load. Behavior research methods. 45, 4 (2013), 1058--1072.
22. Lister, R. et al. 2004. A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin (2004), 119--150.
23. Lopez, M. et al. 2008. Relationships between reading, tracing and writing skills in introductory programming. Proceedings of the fourth international workshop on computing education research (2008), 101--112.
24. Margulieux, L.E. et al. 2012. Subgoal-labeled instructional material improves performance and transfer in learning to develop mobile applications. Proceedings of the ninth annual international conference on International computing education research (2012), 71--78.
25. Margulieux, L.E. and Catrambone, R. 2014. Improving problem solving performance in computer-based learning environments through subgoal labels. Proceedings of the first ACM conference on Learning@ scale conference (2014), 149--150.
26. McCracken, M. et al. 2001. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students. Working group reports from ITiCSE on Innovation and technology in computer science education (Canterbury, UK, 2001), 125--180.
27. van Merriënboer, J.J. and Sweller, J. 2005. Cognitive load theory and complex learning: Recent developments and future directions. Educational psychology review. 17, 2 (2005), 147--177.
28. Morrison, Briana B. et al. 2015. Subgoals, Context, and Worked Examples in Learning Computing Problem Solving. ICER 2015 (Aug. 2015).
29. Palmiter, S. and Elkerton, J. 1993. Animated demonstrations for learning procedural computer-based tasks. Human-Computer Interaction. 8, 3 (1993), 193--216.
30. Parsons, D. and Haden, P. 2006. Parson's Programming Puzzles: A Fun and Effective Learning Tool for First Programming Courses. Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Darlinghurst, Australia, Australia, 2006), 157--163.
31. Plass, J.L. et al. 2010. Cognitive load theory. Cambridge University Press.
32. Renkl, A. and Atkinson, R.K. 2002. Learning from examples: Fostering self-explanations in computer-based learning environments. Interactive learning environments. 10, 2 (2002), 105--119.
33. Rochester Institute of Technology 2014. Undergraduate Course Descriptions.
34. Rountree, N. et al. 2004. Interacting factors that predict success and failure in a CS1 course. ACM SIGCSE Bulletin (2004), 101--104.
35. Simon et al. 2006. Predictors of success in a first programming course. Proceedings of the 8th Australasian Conference on Computing Education-Volume 52 (2006), 189--196.
36. Soloway, E. and Ehrlich, K. 1984. Empirical studies of programming knowledge. Software Engineering, IEEE Transactions on. 5 (1984), 595--609.
37. Sweller, J. et al. 1998. Cognitive architecture and instructional design. Educational psychology review. 10, 3 (1998), 251--296.
38. Sweller, J. et al. 2011. Cognitive load theory. Springer.
39. Sweller, J. 2010. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational psychology review. 22, 2 (2010), 123--138.
40. van Gog, Tamara and Paas, Fred 2012. Cognitive Load Measurement. Encyclopedia of the Sciences of Learning. Springer.
41. Ventura Jr, P.R. 2005. Identifying predictors of success for an objects-first CS1. (2005).
42. Watson, C. et al. 2014. No tests required: comparing traditional and dynamic predictors of programming success. Proceedings of the 45th ACM technical symposium on Computer science education (2014), 469--474.
43. White, G. and Sivitanides, M. 2003. An empirical investigation of the relationship between success in mathematics and visual programming courses. Journal of Information Systems Education. 14, 4 (2003), 409.
44. Wiedenbeck, S. 2005. Factors affecting the success of non-majors in learning to program. Proceedings of the first international workshop on Computing education research (2005), 13--24.
45. Wilson, B.C. and Shrock, S. 2001. Contributing to success in an introductory computer science course: a study of twelve factors. ACM SIGCSE Bulletin (2001), 184--188.

Index Terms

• Social and professional topics

  • Professional topics

    • Computing education

      • Computing education programs

        • Computer science education

          • CS1