Aaron Halfaker
Aniket Kittur
Robert Kraut
John Riedl
ABSTRACT
Wikipedia is a highly successful example of what mass collaboration in an informal peer review system can accomplish. In this paper, we examine the role that the quality of the contributions, the experience of the contributors and the ownership of the content play in the decisions over which contributions become part of Wikipedia and which ones are rejected by the community. We introduce and justify a versatile metric for automatically measuring the quality of a contribution. We find little evidence that experience helps contributors avoid rejection. In fact, as they gain experience, contributors are even more likely to have their work rejected. We also find strong evidence of ownership behaviors in practice despite the fact that ownership of content is discouraged within Wikipedia.
- B. T. Adler and L. de Alfaro. A content-driven reputation system for the Wikipedia. In WWW'07. ACM, 2007. Google Scholar
Digital Library
- Ahmed, Elsheikh, Stratton, Page, Adams, and Wass. Outcome of transphenoidal surgery for acromegaly and its relationship to surgical experience. Clinical Endocrinology, 50:561--567, May 1999.Google ScholarCross Ref
- L. Argote. Organizational Learning: Creating, Retaining, and Transferring Knowledge. Kluwer Academic Publishers, Norwell, MA, USA, 1999. Google ScholarDigital Library
- I. Beschastnikh, T. Kriplean, and D. W. McDonald. Wikipedian self-governance in action: Motivating the policy lens. In AAAI International Conference on Weblogs and Social Media, 2008.Google Scholar
- U. Brandes and J. Lerner. Visual analysis of controversy in user-generated encyclopedia. Information Visualization, 7:34--48, 2008. Google ScholarDigital Library
- S. L. Bryant, A. Forte, and A. Bruckman. Becoming Wikipedian: Transformation of participation in a collaborative online encyclopedia. In GROUP'05. ACM, 2005. Google ScholarDigital Library
- M. T. H. Chi, R. Glaser, and E. Rees. Expertise in problem solving. Technical report, Pittsburgh Univ., PA. Learning Research and Development Center, 1981.Google Scholar
- S. Cole, J. R. Cole, and G. A. Simon. Chance and consensus in peer review. Science, 214(4523):881--886, 1981.Google ScholarCross Ref
- A. C. Justice, M. K. Cho, M. A. Winker, J. A. Berlin, and D. Rennie. Does masking author identity improve peer review quality? JAMA, 280(3):240--242, July 1998.Google ScholarCross Ref
- A. Kittur and R. E. Kraut. Harnessing the wisdom of crowds in Wikipedia: Quality through coordination. In CSCW'08. ACM, 2008. Google ScholarDigital Library
- A. Kittur, B. Suh, B. A. Pendleton, and E. H. Chi. He says, she says: Conflict and coordination in Wikipedia. In CHI'07. ACM, 2007. Google ScholarDigital Library
- T. Kriplean, I. Beschastnikh, and D. W. McDonald. Articulations of WikiWork: Uncovering valued work in wikipedia through barnstars. 2008. Google ScholarDigital Library
- T. Kriplean, I. Beschastnikh, D. W. McDonald, and S. A. Golder. Community, consensus, coercion, control: CS*W or how policy mediates mass participation. In GROUP'07. ACM, 2007. Google ScholarDigital Library
- A. Mockus, R. T. Fielding, and J. Herbsleb. A case study of open source software development: The Apache server. In ICSE'00. ACM, 2000. Google ScholarDigital Library
- R. Priedhorsky, J. Chen, S. Lam, K. Panciera, L. Terveen, and J. Riedl. Creating, destroying, and restoring value in Wikipedia. In GROUP'07, Sanibel Island, FLorida, USA, 2007. Google ScholarDigital Library
- B. Stvilia, M. B. Twidale, L. C. Smith, and L. Gasser. Assessing information quality of a community-based encyclopedia. ACM Press, 2005.Google Scholar
- J. Thom-Santelli, D. R. Cosley, and G. Gay. What's mine is mine: Territoriality in collaborative authoring. In CHI'09. ACM, 2009. Google ScholarDigital Library
- S. van Rooyen, F. Godlee, S. Evans, R. Smith, and N. Black. Effect of blinding and unmasking on the quality of peer review: A randomized trial. JAMA, 280(3):234--237, July 1998.Google ScholarCross Ref
- F. B. Viégas, M. Wattenberg, and K. Dave. Studying cooperation and conflict between authors with history flow visualizations. In CHI'04. ACM, 2004. Google ScholarDigital Library
- F. B. Viégas, M. Wattenberg, J. Kriss, and F. van Ham. Talk before you type: Coordination in Wikipedia. In HICSS '07, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- B.-Q. Vuong, E.-P. Lim, A. Sun, M.-T. Le, and H. W. Lauw. On ranking controversies in Wikipedia: Models and evaluation. In WSDM'08. ACM, 2008. Google ScholarDigital Library
- J. Wales. Wikipedia sociographics. 21st Chaos Communication Congress http://ccc.de/congress/2004/fahrplan/event/59.en.html, December 2004.Google Scholar
- X. Zhang and F. Zhu. Intrinsic motivation of open content contributors: The case of Wikipedia. Workshop on Information Systems and Economics, 2006.Google Scholar
Index Terms
- A jury of your peers: quality, experience and ownership in Wikipedia
Recommendations
Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work
WikiSym '11: Proceedings of the 7th International Symposium on Wikis and Open CollaborationReverts are important to maintaining the quality of Wikipedia. They fix mistakes, repair vandalism, and help enforce policy. However, reverts can also be damaging, especially to the aspiring editor whose work they destroy. In this research we analyze ...
Measuring the Quality of Edits to Wikipedia
OpenSym '14: Proceedings of The International Symposium on Open CollaborationWikipedia is unique among reference works both in its scale and in the openness of its editing interface. The question of how it can achieve and maintain high-quality encyclopedic articles is an area of active research. In order to address this question,...
Assessing quality score of Wikipedia article using mutual evaluation of editors and texts
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementIn this paper, we propose a method for assessing quality scores of Wikipedia articles by mutually evaluating editors and texts. Survival ratio based approach is a major approach to assessing article quality. In this approach, when a text survives beyond ...
Reviews
Gordon B. Davis
Halfaker et al.'s paper presents interesting research on quality control in Wikipedia. Since Wikipedia is often the first place one looks for information, this is an important topic to many of us. The content is prepared (and corrected) by millions of volunteers who work with little formal organization. Many of us who use Wikipedia wonder about the quality of the entries, including how they are corrected and improved. When a Wikipedia article is written and stored by a registered editor, edits can be made and stored by both the original writer and other registered editors. If any edit is considered incorrect, a special kind of edit is made by any registered editor to revert the article to its original reading. Halfaker et al. use a random sample of approximately 1.4 million revisions attributed to registered editors, to examine two categories of factors expected to predict the revisions that will be reverted: measures of quality and factors unrelated to quality. They carefully explain their reasoning for the design and procedures of their research. Certain factors strongly predict whether an edit will be reverted: "edits that remove established words," edits from "editors with a history of high quality contributions," edits from "editors who have been reverted recently," and "edits that remove the words of active editors." One factor-"stepping on toes"- appears to be an example of ownership behavior, even though Wikipedia discourages this. The paper also explores two other factors: editor experience and "editors who cite policy." The results presented in this paper are useful for understanding how quality in Wikipedia is maintained. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments