Tracy Riggs
ABSTRACT
The current system for scholarly information dissemination may be amen able to significant improvement. In particular, going from the current system of journal publication to one of self-distributed documents offers significant cost and timeliness advantages. A major concern with such alternatives is how to provide the value currently afforded by the peer review system.
Here we propose a mechanism that could plausibly supply such value. In the peer review system, papers are judged meritorious if good reviewers give them good reviews. In its place, we propose a collaborative filtering algorithm which automatically rates reviewers, and incorporates the quality of the reviewer into the metric of merit for the paper. Such a system seems to provide all the benefits of the current peer review system, while at the same time being much more flexible.
We have implemented a number of parameterized variations of this algorithm, and tested them on data available from a quite different application. Our initial experiments suggest that the algorithm is in fact ranking reviewers reasonably.
- 1.ACM-SIGIR 1999 Workshopon Recommender Systems: Algorithms and Evaluation. http://www.csee.umbc.edu/ian/sigir99- rec/summary.html.Google Scholar
- 2.The Berkeley Electronic Press. http://www.bepress.com/.Google Scholar
- 3.CiteSeer.http://citeseer.nj.nec.com.Google Scholar
- 4.Collaborative filtering. http://www.sims.ber eley.edu/resources/collab/.Google Scholar
- 5.The UC Berkeley Digital Library Project. http://elib.cs.berkeley.edu.Google Scholar
- 6.C.Avery, P. Resnick, and R. Zeckhauser. The Market for Evaluations. American Economic Review 89(3):564-584,1999.Google ScholarCross Ref
- 7.M. Balabanovic and Y. Shoham. Fab: Content-Based Collaborative Recommendation. Communications of the ACM, 40(3): 66-72, March 1999. Google ScholarDigital Library
- 8.J. Canny. Personal communication with the authors.Google Scholar
- 9.D. Goldberg, D. Nichols, B. M. Oki, and D.Terry. Using Collaborative Filtering to Weave an Information Tapestry. Communications of the ACM 35(12):61-70, December 1992. Google ScholarDigital Library
- 10.J. L. Herlocer, J. A. Konstant, A. Brochers, and J. Riedl. An Algorithmic Framework for Performing Collaborative Filtering.In Proceedings of the 1999 Conference on Research and Development in Information Retrieval ACM-SIGIR, August 1999. Google ScholarDigital Library
- 11.J. M. Kleinberg. Authoritative Sources in a Hyperlin ed Environment. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms 1998. Google ScholarDigital Library
- 12.G. Z. A. Moukas and P. Maes. Collaborative Reputation Mechanisms in Electronic Mar etplaces. In Proceedings of the 32nd Hawaii International Conference on System Sciences 1999. Google ScholarDigital Library
- 13.T. A. Phelps and R. Wilensky. Multivalent Documents: Anywhere, Anytime, Any Type, Every Way User-Improvable Digital Documents. Communications of the ACM 43(6), June 2000. Google ScholarDigital Library
- 14.J. Rucker and M. J. Polanco. Siteseer: Personalized Navigation for the Web. Communications of the ACM 40(3): 73-75, March 1997. Google ScholarDigital Library
- 15.J. B. Schafer, J. Konstan, and J. Riedl. Recommender Systems in E-Commerce. In Proceedings of the ACM Conference on Electronic Commerce November 1999. Google ScholarDigital Library
- 16.R. Smith. Opening up BMJ peer review. BMJ, 318:23-27, 1999.Google ScholarCross Ref
- 17.C. Tenopir and D. W. King. Trends in scientific scholarly journal publishing in the United States. Journal of Scholarly Publishing 28:135-170, 1997.Google ScholarCross Ref
- 18.L. Terveen, W. Hill, B. Amento, D. McDonald, and J. Creter. PHOAKS: A System for Sharing Recommendations. Communications of the ACM 40(3):59-62, March 1997. Google ScholarDigital Library
- 19.H. R. Varian. The Future of Electronic Journals. Technology and Scholarly Communication 1999.Google Scholar
Index Terms
- An algorithm for automated rating of reviewers
Recommendations
Acknowledgment to Guest Editors and Reviewers (2011)
Manufacturing & Service Operations Management (M&SOM) formally thanks the guest editors and reviewers, who provided expert counsel and guidance on a voluntary basis. Through their efforts, the journal was able to provide submitting authors with timely, ...
Acknowledgment to Guest Editors and Reviewers (2013)
<P>Manufacturing & Service Operations Management (M&SOM) formally thanks the guest editors and reviewers, who provided expert counsel and guidance on a voluntary basis. Through their efforts, the journal was able to provide submitting authors with ...
Acknowledgment to Guest Editors and Reviewers (2010)
Manufacturing & Service Operations Management (M&SOM) formally thanks the guest editors and reviewers, who provided expert counsel and guidance on a voluntary basis. Through their efforts, the journal was able to provide submitting authors with timely, ...
Comments