Rebecca Vivian
Skip Abstract Section
Abstract
Although teamwork has been identified as an essential skill for Computer Science (CS) graduates, these skills are identified as lacking by industry employers, which suggests a need for more proactive measures to teach and assess teamwork. In one CS course, students worked in teams to create a wiki solution to problem-based questions. Through a case-study approach, we test a developed teamwork framework, using manual content analysis and sentiment analysis, to determine if the framework can provide insight into students’ teamwork behavior and to determine if the wiki task encouraged students to collaborate, share knowledge, and self-adopt teamwork roles. Analysis revealed the identification of both active and cohesive teams, disengaged students, and particular roles and behaviors that were lacking. Furthermore, sentiment analysis revealed that teams moved through positive and negative emotions over the course of developing their solution, toward satisfaction. The findings demonstrate the value of the detailed analysis of online teamwork. However, we propose the need for automated measures that provide real-time feedback to assist educators in the fair and efficient assessment of teamwork. We present a prototype system and recommendations, based on our analysis, for automated teamwork analysis tools.
- W. Archer and J. Davison. 2008. Graduate employability: What do employers think and want? In Graduate Employability: The Views of Employers, R. Brown and K. Herrmann (Eds.). The Council for Industry and Higher Education, London, 1--18.Google Scholar
- K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, and D. Thomas. 2001. Manifesto for Agile Software Development. Retrieved from http://agilemanifesto.org/.Google Scholar
- H. Beetham and R. Sharpe. 2007. Rethinking Pedagogy for a Digital Age: Designing and Delivering e-Learning. Routledge, New York. Google ScholarDigital Library
- A. Begel and B. Simon. 2008a. Novice software developers, all over again. In Proceedings of the Workshop on Computing Education Research. Google ScholarDigital Library
- A. Begel and B. Simon. 2008b. Struggles of new college graduates in their first software development job. In Proceedings of the SIGCSE Technical Symposium on Computer Science Education. Google ScholarDigital Library
- L. Bender, G. Walia, K. Kambhampaty, K. Nygard, and T. Nygard. 2012. Social sensitivity correlations with the effectiveness of team process performance: An empirical study. In Proceedings of the International Computing Education Research Conference. Google ScholarDigital Library
- S. Bull and J. Kay. 2010. Open learner models. In Advances in Intelligent Tutoring Systems, R. Nkambou, J. Bourdeau, and R. Mizoguchi (Eds.). Springer-Verlag, Berlin, 301--322.Google Scholar
- L.-J. ChanLin and K.-C. Chan. 2010. Group learning strategies for online course. Procedia - Social and Behavioral Sciences 22, 397--401. DOI: 10.1016/j.sbspro.2010.03.033Google ScholarCross Ref
- Z. Chen, B. Liu, M. Hsu, M. Castellanos, and R. Ghosh. 2013. Identifying intention posts in discussion forums. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.Google Scholar
- J. Creswell. 2008. Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 3rd ed. Pearson, Upper Saddle River, NJ.Google Scholar
- J. Creswell. 2013. Qualitative Inquiry and Research Design: Choosing Among Five Approaches 3rd ed. Sage, Thousand Oaks, CA.Google Scholar
- S. Crowe, K. Cresswell, A. Robertson, G. Huby, A. Avery, and A. Sheikh. 2011. The case study approach. BMC Medical Research Methodology 111, 100.Google ScholarCross Ref
- S. Dawson, A. Bakharia, and E. Heathcote. 2010. SNAPP: Realising the affordances of real-time SNA within networked learning environments. In Proceedings of the 7th International Conference on Networked Learning.Google Scholar
- A. De Liddo, S. Buckingham Shum, I. Quinto, M. Bachler, and L. Cannavacciuolo. 2011. Discourse-centric learning analytics. In Proceedings of the International Conference on Learning Analytics and Knowledge (LAK). Google ScholarDigital Library
- T. Dickinson and R. McIntyre. 2009. A conceptual framework for teamwork measurement. In Team Performance Assessment and Measurement: Theory, Methods, and Applications, M. Bannick, E. Salas, and C. Prince (Eds.). Lawrence Erlbaum, Mawah, NJ.Google Scholar
- T. Dingsoyr and T. Dyba. 2012. Team effectiveness in software development: Human and cooperative aspects in team effectiveness models and priorities for future studies. In Proceedings of the International Workshop on Cooperative and Human Aspects of Software Engineering. Google ScholarDigital Library
- A. El-Abbassy, R. Muawad, and A. Gaber. 2010. Evaluating agile principles in CS Education. International Journal of Computer Science and Network Security 10, 10, 19--28.Google Scholar
- K. Falkner and N. Falkner. 2012. Supporting and structuring “contributing student pedagogy” in computer science curricula. Computer Science Education 224, 413--443. DOI: 10.1080/08993408.2012.727713Google ScholarCross Ref
- K. Falkner, N. Falkner, and R. Vivian. 2013. Collaborative learning and anxiety: A phenomenographic study of collaborative learning activities. In Proceedings of the Technical Symposium on Computer Science Education. Google ScholarDigital Library
- A. Forte and A. Bruckman. 2007. Constructing Text: Wiki as a Toolkit for Collaborative Learning? Paper presented at the WikiSym. Google ScholarDigital Library
- H. Fournier, R. Kop, and H. Sitlia. 2011. The value of learning analytics to networked learning on a Personal Learning Environment. In Proceedings of the Learning Analytics and Knowledge Conference. Google ScholarDigital Library
- S. Govaerts, K. Verbert, J. Klerkx, and E. Duval. 2010. Visualizing activities for self-reflection and awareness. In Proceedings of the 9th International Conference on Web-based Learning.Google Scholar
- G. Gweon, S. Jun, J. Lee, S. Finger, and C. Penstein Rosé. 2011. A framework for assessment of student project groups on-line and off-line. In Analyzing Interactions in CSCL, S. Puntambekar, G. Erkens, and C. Hmelo-Silver (Eds.),Vol. 12. Springer, New York, 293--317.Google Scholar
- A. Hadwin, S. Järvelä, and M. Miller. 2011. Self-regulated, co-regulated, and socially shared regulation. In Handbook of Self-Regulation of Learning and Performance, B. Zimmerman and D. Schunk (Eds.). Taylor & Francis, New York, 65--63.Google Scholar
- T. Heafner and A. Friedman. 2008. Wikis and constructivism in secondary social studies: Fostering a deeper understanding. Computers in the Schools 253--254, 288--302.Google Scholar
- C. Hmelo-Silver. 2003. Analyzing collaborative knowledge construction: Multiple methods for integrated understanding. Computers & Education 414, 397--420. DOI: 10.1016/j.compedu.2003.07.001 Google ScholarDigital Library
- C. Hmelo-Silver and E. Chemobilsky. 2004. Understanding collaborative activity systems: The relation of tools and discourse in mediating learning. In Proceedings of the 6th International Conference on Learning Sciences. Google ScholarDigital Library
- C. Hmelo-Silver, E. Chernobilsky, and R. Jordan. 2008. Understanding collaborative learning processes in new learning environments. Instructional Science 365, 409--430. DOI: 10.1007/s11251-008-9063-8Google ScholarCross Ref
- C. Hmelo-Silver and C. Eberbach. 2012. Learning Theories and Problem-Based Learning: Problem-Based Learning in Clinical Education. In S. Bridges, C. McGrath, and T. Whitehill (Eds.), Vol. 8. Springer, 3--17.Google Scholar
- H.-F. Hsieh and S. Shannon. 2005. Three approaches to qualitative content analysis. Qualitative Health Research 159, 1277--1288. DOI: 10.1177/1049732305276687Google ScholarCross Ref
- R. Hughes and S. Jones. 2011. Developing and assessing college student teamwork skills. New Directions for Institutional Research 149, 53--64. DOI: 10.1002/ir.380Google ScholarCross Ref
- K. Krippendorff. 2004. Content Analysis: An Introduction to its Methodology 2nd ed. Sage, Thousand Oaks, CA.Google Scholar
- N. Law, J. Yuen, W. Wong, and J. Leng. 2011. Understanding learners’ knowledge building trajectory through visualizations of multiple automated analyses. In Analyzing Interactions in CSCL, S. Puntambekar, G. Erkens, and C. Hmelo-Silver (Eds.),Vol. 12. New York: Springer, New York, 47--82.Google Scholar
- R. Lingard and S. Barkataki. 2011. Teaching teamwork in engineering and computer science. In Proceedings of the Frontiers in Education Conference. Google ScholarDigital Library
- C. Loftus, L. Thomas, and C. Zander. 2011. Can graduating students design: Revisited. In Proceedings of the Technical Symposium on Computer Science Education. Google ScholarDigital Library
- R. Mazza and V. Dimitrova. 2004. Visualising student tracking data to support instructors. In Proceedings of the 13th International World Wide Web Conference. Google ScholarDigital Library
- S. Minocha and P. Thomas. 2007. Collaborative learning in a wiki environment: experiences from a software engineering course. New Review of Hypermedia and Multimedia 132, 187--209. DOI: 10.1080/13614560701712667 Google ScholarDigital Library
- M. Munezero, C. Calkin Suero Montero, M. Mozgovoy, and E. Sutinen. 2013. Exploiting sentiment analysis to track emotions in students’ learning diaries. In Proceedings of the 13th Koli Calling International Conference on Computing Education Research. Google ScholarDigital Library
- B. Oakley, D. Hanna, Z. Kuzmyn, and R. Felder. 2007. Best practices involving teamwork in the classroom: Results from a survey of 6435 Engineering student respondents. IEEE Transactions on Education 503, 266--272. Google ScholarDigital Library
- B. Pang and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval 21--22, 1--135. Google ScholarDigital Library
- L. Polanya and A. Zaenen. 2004. Contextual valence shifters Working Notes -- Exploring Attitude and Affect in Text. AAAI Spring Symposium Series.Google Scholar
- H. Prakken, C. Reed, and D. Walton. 2003. Argumentation schemes and generalisations in reasoning about evidence. In Proceedings of the 9th International Conference on Artificial Intelligence and Law (ICAL). Google ScholarDigital Library
- A. Radermacher and G. Walia. 2013. Gaps between industry expectations and the abilities of graduates. In Proceedings of the Technical Symposium on Computer Science Education. Google ScholarDigital Library
- M. Robles. 2012. Executive perceptions of the top 10 soft skills needed in today's workplace. Business Communication Quarterly. DOI: 10.1177/1080569912460400Google Scholar
- S. Ruff and M. Carter. 2009. Communication learning outcomes from software engineering professionals: A basis for teaching communication in the engineering curriculum. In Proceedings of the Frontiers in Education Conference. Google ScholarDigital Library
- M. Sandelowski. 1995. Sample size in qualitative research. Research in Nursing & Health 182, 179--183.Google ScholarCross Ref
- J. Santos, K. Verbert, S. Govaerts, and E. Duval. 2011. Visualizing PLE usage. In Proceedings of the 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments.Google Scholar
- H. Schmidt, J. Rotgans, and E. Yew. 2011. The process of problem-based learning: What works and why. Medical Education 458, 792--806. DOI: 10.1111/j.1365-2923.2011.04035.xGoogle ScholarCross Ref
- H. Schmitz, M. Scheffel, M. Friedrich, M. Jahn, K. Niemann, and M. Wolpers. 2009. Learning in the synergy of multiple disciplines. In Lecture Notes in Computer Science, U. Cress, V. Dimitrova, and M. Specht (Eds.). Springer. 507--520Google Scholar
- D. Schunk. 2008. Learning Theories: An Educational Perspective 5th ed. Pearson, Upper Saddle River, NJ.Google Scholar
- J.-W. Strijbos, M. De Laat, R. Martens, and W. Jochems. 2005. Functional versus spontaneous roles during CSCL. In Proceedings of the Computer Support for Collaborative Learning Conference. Google ScholarDigital Library
- G. Trentin. 2009. Using a wiki to evaluate individual contribution to a collaborative learning project. Journal of Computer Assisted Learning 251, 43--55. DOI: 10.1111/j.1365-2729.2008.00276.xGoogle ScholarCross Ref
- R. Vatrapu, C. Teplovs, N. Fujita, and S. Bull. 2011. Towards visual analytics for teachers’ dynamic diagnostic pedagogical decision-making. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK). Google ScholarDigital Library
- R. Vivian, K. Falkner, and K. Falkner. 2013a. Analysing computer science students’ teamwork role adoption in an online self-organised teamwork activity. In Proceedings of the Koli Calling Conference. Google ScholarDigital Library
- R. Vivian, K. Falkner, and N. Falkner. 2013b. Building consensus: students’ cognitive and metacognitive behaviours during wiki construction. In Proceedings of the Learning and Teaching in Computer Engineering (LaTiCE) Conference. Google ScholarDigital Library
- R. Weber. 1990. Basic Content Analysis. Sage, Newbury Park, CA.Google Scholar
- A. Weinberger and F. Fischer. 2006. A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education 46, 1, 71--95. Google ScholarDigital Library
- M. Wells and R. Clougherty. 2008. Use of wikis in chemistry instruction for problem-based learning assignments: an example in instrumental analysis. Journal of Chemical Education 85, 10, 1446. DOI: 10.1021/ed085p1446Google ScholarCross Ref
- R. Wilson Lundin. 2008. Teaching with wikis: Toward a networked pedagogy. Computers and Composition 254, 432--448. DOI: 10.1016/j.compcom.2008.06.001Google ScholarCross Ref
- T. Wilson, J. Wiebe, and P. Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Human Language Technology and Empirical Methods in Natural Language Processing Conference. Google ScholarDigital Library
- T. Yasseri, R. Sumi, A. Rung, A. Kornai, and J. Kertész. 2012. Dynamics of conflicts in wikipedia. PLoS ONE 76, e38869. DOI: 10.1371/journal.pone.0038869Google ScholarCross Ref
- Z. Zanial. 2007. Case study as a research method. Jurnal Kemanusiaan 9, 1--6.Google Scholar
- B. Zimmerman. 1986. Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology 114, 307--313.Google ScholarCross Ref
- B. Zimmerman and M. Pons. 1986. Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal 234, 614--628.Google ScholarCross Ref
Index Terms
- A Method to Analyze Computer Science Students’ Teamwork in Online Collaborative Learning Environments
Recommendations
Analysing computer science students' teamwork role adoption in an online self-organised teamwork activity
Koli Calling '13: Proceedings of the 13th Koli Calling International Conference on Computing Education ResearchComputer Science (CS) professionals are regularly required to work in teams to solve complex problems. Agile software development processes are increasingly popular in organisations as a method for teamwork but the self-organising nature of the method ...
Relating Collaborative Technology Use to Teamwork Quality and Performance: An Empirical Analysis
Although team-based work systems are pervasive in the workplace, the use of collaborative systems designed to facilitate and support ongoing teamwork is a relatively recent development. An understanding of how teams embrace and use such collaborative ...
Gender Effects on Collaborative Online Brainstorming Teamwork
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsIt is common for individuals with diverse demographic backgrounds to collaborate through computer-mediated communication (CMC) technologies. Groups with internal diversity are typically considered to be advantageous to group performance due to the ...
Comments