Teaching the art of functional programming using automated grading (experience report)

Abstract

Online programming platforms have immense potential to improve students' educational experience. They make programming more accessible, as no installation is required; and automatic grading facilities provide students with immediate feedback on their code, allowing them to to fix bugs and address errors in their understanding right away. However, these graders tend to focus heavily on the functional correctness of a solution, neglecting other aspects of students' code and thereby causing students to miss out on a significant amount of valuable feedback.

In this paper, we recount our experience in using the Learn-OCaml online programming platform to teach functional programming in a second-year university course on programming languages and paradigms. Moreover, we explore how to leverage Learn-OCaml's automated grading infrastructure to make it easy to write more expressive graders that give students feedback on properties of their code beyond simple input/output correctness, in order to effectively teach elements of functional programming style. In particular, we describe our extensions to the Learn-OCaml platform that evaluate students on test quality and code style.

By providing these tools and a suite of our own homework problems and associated graders, we aim to promote functional programming education, enhance students' educational experience, and make teaching and learning typed functional programming more accessible to instructors and students alike, in our community and beyond.

References

1. Kirsti M Ala-Mutka. 2005. A Survey of Automated Assessment Approaches for Programming Assignments. Computer Science Education 15, 2 (2005), 83–102.
2. Steve D Benford, Edmund K Burke, Eric Foxley, and Christopher A Higgins. 1995. The Ceilidh system for the automatic grading of students on programming courses. In Proceedings of the 33rd annual on Southeast regional conference. ACM, 176–182.
3. Benjamin Canou, Roberto Di Cosmo, and Grégoire Henry. 2017. Scaling up functional programming education: under the hood of the OCaml MOOC. Proceedings of the ACM on Programming Languages 1, ICFP (aug 2017), 1–25.
4. Benjamin Canou, Grégoire Henry, Çagdas Bozman, and Fabrice Le Fessant. 2016. Learn OCaml, An Online Learning Center for OCaml. In OCaml Users and Developers Workshop 2016.
5. Peter M Chen. 2004. An Automated Feedback System for Computer Organization Projects. IEEE Transactions on Education 47, 2 (May 2004), 232–240.
6. Youyou Cong and Kenichi Asai. 2016. Implementing a stepper using delimited continuations. contract 1 (2016), r1.
7. Richard A DeMillo, Richard J Lipton, and Frederick G Sayward. 1978. Hints on test data selection: Help for the practicing programmer. Computer 11, 4 (1978), 34–41.
8. Roberto Di Cosmo, Yann Regis-Gianas, and Ralf Treinen. 2015. Introduction to Functional Programming in OCaml. (October 2015). https://www.fun-mooc.fr/courses/parisdiderot/56002/session01/about
9. Stephen H Edwards. 2003. Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In International Conference on Education and Information Systems: Technologies and Applications (EISTA’03), Vol. 3.
10. Stephen H Edwards and Manuel A Perez-Quinones. 2008. Web-CAT: automatically grading programming assignments. In ACM SIGCSE Bulletin, Vol. 40. ACM, 328–328.
11. Alex Gerdes, Bastiaan Heeren, Johan Jeuring, and L Thomas van Binsbergen. 2017. Ask-Elle: an adaptable programming tutor for Haskell giving automated feedback. International Journal of Artificial Intelligence in Education 27, 1 (2017), 65–100.
12. Vincent Gramoli, Michael Charleston, Bryn Jeffries, Irena Koprinska, Martin McGrane, Alex Radu, Anastasios Viglas, and Kalina Yacef. 2016. Mining Autograding Data in Computer Science Education. In Proceedings of the Australasian Computer Science Week Multiconference (ACSW ’16). ACM, New York, NY, USA, Article 1, 10 pages.
13. Robert W. Harper. 2013. Programming in Standard ML. (draft available at https://www.cs.cmu.edu/~rwh/isml/book.pdf ).
14. Jan Hoffmann, Klaus Aehlig, and Martin Hofmann. 2012. Resource Aware ML. Lecture Notes in Computer Science (2012), 781âĂŞ786.
15. Jan Hoffmann, Ankush Das, and Shu-Chun Weng. 2017. Towards automatic resource bound analysis for OCaml. ACM SIGPLAN Notices 52, 1 (Jan 2017), 359âĂŞ373.
16. Jack Hollingsworth. 1960. Automatic graders for programming classes. Commun. ACM 3, 10 (1960), 528–529.
17. David Jackson and Michelle Usher. 1997. Grading Student Programs Using Assyst. ACM SIGCSE Bulletin 29, 1 (1997), 335–339.
18. Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (Sep. 2011), 649–678.
19. Mike Joy, Nathan Griffiths, and Russell Boyatt. 2005. The boss online submission and assessment system. Journal on Educational Resources in Computing (JERIC) 5, 3 (2005), 2.
20. Greg Michaelson. 1996. Automatic analysis of functional program style. In Australian Software Engineering Conference. 38–46.
21. Cyrus Omar, Ian Voysey, Michael Hilton, Jonathan Aldrich, and Matthew A Hammer. 2017. Hazelnut: a bidirectionally typed structure editor calculus. ACM SIGPLAN Notices 52, 1 (2017), 86–99.
22. Raymond Pettit, John Homer, Roger Gee, Susan Mengel, and Adam Starbuck. 2015. An Empirical Study of Iterative Improvement in Programming Assignments. In 46th ACM Technical Symposium on Computer Science Education (SIGCSE ’15). ACM, 410–415.
23. Konstantinos Sagonas and Thanassis Avgerinos. 2009. Automatic refactoring of Erlang programs. Proceedings of the 11th ACM SIGPLAN conference on Principles and practice of declarative programming - PPDP âĂŹ09 (2009).
24. Saul Schleimer, Daniel S. Wilkerson, and Alex Aiken. 2003. Winnowing. Proceedings of the 2003 ACM SIGMOD international conference on on Management of data - SIGMOD âĂŹ03 (2003).
25. Mark Sherman, Sarita Bassil, Derrell Lipman, Nat Tuck, and Fred Martin. 2013. Impact of Auto-grading on an Introductory Computing Course. J. Comput. Sci. Coll. 28, 6 (June 2013), 69–75. http://dl.acm.org/citation.cfm?id=2460156.2460171
26. Zach Sims and Ryan Bubinski. 2011. Codecademy. (2011). http://www.codecademy.com
27. Chris Wilcox. 2015. The role of automation in undergraduate computer science education. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. ACM, 90–95.

Index Terms

• Social and professional topics

  • Professional topics

    • Computing education

      • Computing education programs

        • Computer science education

      • Student assessment

• Software and its engineering

  • Software notations and tools

    • General programming languages

      • Language types

        • Functional languages