Fruitful Feedback: Positive Affective Language and Source Anonymity Improve Critique Reception and Work Outcomes

ABSTRACT

Feedback is information that can improve task performance. Online communities, educational forums, and crowd-based feedback platforms all support feedback exchange among a more diverse set of sources than ever before, with greater control over how to moderate this exchange. In this work, we study how the power relationship between the source and receiver and the tone of language influence the recep-tivity, effort, and work performance resulting from online feedback exchange. We conducted an online experiment manipulating affective language and source of feedback on a writing task. We found that critiques with positive affec-tive language increased positive emotions and reduced participants' annoyance and frustration, which led to an increase in work quality, compared to critiques without positive language. Feedback without positive affective language led to more edits, but not better work outcomes. Participants reacted more positively to feedback from an anonymous source than from a peer or an authority. Our findings provide design implications for platforms to support more fruitful feedback exchange.

References

1. Chris Argyris. 1991. Teaching Smart People How to Learn. Harvard Business Review 69, 99--109.
2. Frank D. Belschak and Deanne N. Den Hartog. 2009. Consequences of positive and negative feedback: The impact on emotions and extra-role behaviors. Applied Psychology 58, 2: 274--303.
3. Crystal Project Inc. 2014. Crystal. Email Application. Retrieved Sep 4th 2015 from https://www.crystalknows.com/
4. Susan J. Ashford, and Anne S. Tsui. 1991. SelfRegulation for Managerial Effectiveness: The Role of Active Feedback Seeking. Academy of Management Journal, 34, 2: 251--80.
5. Jennifer N. Belding, Karen Z. Naufel, and Kentaro Fujita. 2015. Using High-Level Construal and Perceptions of Changeability to Promote Self-Change Over Self-Protection Motives in Response to Negative Feedback. Personality & Social Psychology Bulletin 41, 822--838.
6. Gilad Chen, Stanley M. Gully, and Dov Eden. 2001. Validation of a New General Self-Efficacy Scale. Organizational Research Methods 4, 1: 62--83.
7. Steven P. Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel L. Schwartz, and Scott R. Klemmer. 2010. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Transactions on Computer-Human Interaction 17, 4, Article 18, 24 pages.
8. Steven P. Dow, Anand Kulkarni, Scott Klemmer, & Bjorn Hartmann. 2012. Shepherding the crowd yields better work. In Proceedings of CSCW '12, 1013--1022.
9. Carol Sorich Dweck & Ellen L. Leggett. 1998. A Social Cognitive Approach to Motivation and Personality. Psychological Review 95, 256--73.
10. Donald B Fedor, Walter D Davis, John M Maslyn, and Kieran Mathieson. 2001. Performance improvement efforts in response to negative feedback: the roles of source power and recipient self-esteem. Journal of Management 27, 1: 79--97.
11. Donald B. Fedor, Robert W. Eder, Ronald M. Buckley. 1989. The contributory effects of supervisor intentions on subordinate feedback responses. Organizational Behavior and Human Decision Processes 44, 3: 396414.
12. Stacey R. Finkelstein and Ayelet Fishbach. 2012. Tell Me What I Did Wrong: Experts Seek and Respond to Negative Feedback. Journal of Consumer Research 39, 1: 22--38.
13. Steve Graham, Charles MacArthur, and Shirley Schwartz.1995. Effects of goal setting and procedural facilitation on the revising behavior and writing performance of students with writing and learning problems. Journal of Educational Psychology 87, 2: 230240.
14. John Hattie and Helen Timperley. 2007. The power of feedback. Review of Educational Research 77, 1: 81--112.
15. Andrew F. Hayes and Klaus Krippendorff (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1, 1: 77--89.
16. Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, NY, NY, USA, 453--456.
17. Avraham Kluger and Angelo DeNisi. 1998. Feedback interventions: toward the understanding of a doubleedged sword. Current Directions in Psychological Science 7, 3: 67--72.
18. Stavros Konstantinidis. 2005. Computing the Levenshtein distance of a regular language. In Proceedings of IEEE ISOC Information Theory Workshop 2005 on Coding and Complexity, 113--116.
19. Minjung Koo and Ayelet Fishbach. 2008. Dynamics of Self-Regulation: How (Un)Accomplished Goal Actions Affect Motivation. Journal of Personality and Social Psychology 194, 2: 83--95.
20. Chinmay E. Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15), 75--84.
21. Kwok Leung, Steven Su, and Michael W. Morris. 2001. When is Criticism Not Constructive? The Roles of Fairness Perceptions and Dispositional Attributions in Employee Acceptance of Critical Supervisory Feedback. Human Relations 54, 1155--1187.
22. Ruiling Lu and Linda Bol. 2007. A Comparison of Anonymous Versus Identifiable e-Peer Review on College Student Writing Performance and the Extent of Critical Feedback. Journal of Interactive Online Learning 6, 2: 100--115.
23. Kurt Luther, Amy Pavel, Wei Wu, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, and Steven P. Dow. 2014. CrowdCrit: crowdsourcing and aggregating visual design critique. In Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW Companion '14), 21--24.
24. Laura MacLeod. 1999. Computer-aided peer review of writing. Business Communication Quarterly 62, 3: 8795.
25. Jennifer Marlow and Laura Dabbish. 2014. From rookie to all-star: professional development in a graphic design social networking site. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14), 922--933.
26. Jennifer Marlow and Laura Dabbish. 2013. Activity traces and signals in software developer recruitment and hiring. In Proceedings of the 16th ACM conference on Computer supported cooperative work (CSCW '13), 145--156.
27. Winter Mason and Siddharth Suri. Conducting behavioral research on Amazon's Mechanical Turk. 2012. Behavior research methods 44, 1: 1--23.
28. Bruce S. McEwen and Robert M. Sapolsky. 1995. Stress and cognitive function. Current Opinion in Neurobiology 5, 2: 205--216.
29. Melissa M. Nelson and Christian D. Schunn. 2008. The Nature of Feedback: How Different Types of Peer Feedback Affect Writing Performance. Instructional Science 37, 4: 375--401.
30. Christine M. Neuwirth, Ravinder Chandhok, David Charney, Patricia Wojahn, and Loel Kim. 1994. Distributed collaborative writing: a comparison of spoken and written modalities for reviewing and revising documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94), 5157.
31. Duyen T. Nguyen, Laura A. Dabbish, and Sara Kiesler. 2015. The Perverse Effects of Social Transparency on Online Advice Taking. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 207--217.
32. Gabriele Paolacci, Jesse Chandler, and Panagiotis G. Ipeirotis. Running experiments on amazon mechanical turk. 2010. Judgment and Decision making 5, 5: 411419.
33. Reinhard Pekrun. 1992. The impact of emotions on learning and achievement: Towards a theory of cognitive/motivational mediators. Applied Psychology 41, 4: 359--376.
34. Kristopher J. Preacher and Andrew F. Hayes. 2008. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior research methods 40, 3:79--891.
35. Sheng-Chau Tseng and Chin-Chung Tsai. 2007. Online peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education 49, 4: 1161--1174.
36. R. William Soukoreff and I. Scott MacKenzie. 2001. Measuring errors in text entry tasks: an application of the Levenshtein string distance statistic. In CHI '01 Extended Abstracts, 319--320.
37. Marie Stevenson, Rob Schoonen, & Kees de Glopper. 2006. Revising in two languages: A multi-dimensional comparison of online writing revisions in L1 and FL. Journal of Second Language Writing 15, 3: 201--233.
38. Jakko van der Pol, Ineke van den Berg, Wilfried F. Admiraal, Robert Jan Simons. 2008. The nature, reception, and use of online peer feedback in higher education. Computers & Education 51, 4: 1804--1817.
39. Anna C. K. van Duijvenvoorde, Kiki Zanolie, Serge R. A. B. Rombouts, Maartje E. J. Raijmakers, and Eveline A. Crone. 2008. Evaluating the negative or valuing the positive? Neural mechanisms supporting feedbackbased learning across development. The Journal of Neuroscience 28, 38: 9495--9503.
40. C. W. Von Bergen, Martin S. Bressler, and Kitty Campbell. 2014. The sandwich feedback method: not very tasty. Journal of Behavioral Studies in Business 7, 1--13.
41. David Watson, Lee A. Clark, and Auke Tellegen. 1988. Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of personality and social psychology 54, 6: 1063 - 1070.
42. Anbang Xu, Huaming Rao, Steven P. Dow, and Brian P. Bailey. 2015. A Classroom Study of Using Crowd Feedback in the Iterative Design Process. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 1637--1648.
43. Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14), 14331444.
44. Haiyi Zhu, Amy Zhang, Jiping He, Robert E. Kraut, and Aniket Kittur. 2013. Effects of peer feedback on contribution: a field experiment in Wikipedia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 2253--2262.

Index Terms

• Human-centered computing

  • Collaborative and social computing