Mihai Lupu
ABSTRACT
This paper is a result of the experience of the past three years in organizing evaluation campaigns for the patent domain. However, its aim is not really to give an account of what has been done in such evaluation campaigns, but rather to explore what has not been done. It also aims to convince all participants in this process, that we must have a more rigorous way of testing systems and system components. The efforts of the scientists, while being very narrow in the scope of each individual evaluation exercise, are still the most reliable way of comparing system components. Yet they fail to translate into an informative and comprehensible message for the users. I therefore propose a Patent Retrieval Tool Evaluation Map (PRET-EVAL map), to help all participants visualise the focus of each evaluation exercise.
- http://www.wallwisher.com/wall/patentEvaluation.Google Scholar
- S. Adams. The text, the full text and nothing but the text: Part 1 - standards for creating textual information in patent documents and general search implications. World Patent Information, 32(1):22--29, 2010.Google ScholarCross Ref
- S. Adams. The text, the full text and nothing but the text: Part 2 - standards for creating textual information in patent documents and general search implications. World Patent Information, 32(2), 2010.Google Scholar
- D. Alberts, C. B. Yang, D. Fobare-DePonio, K. Koubek, S. Robins, M. Rodgers, E. Simmons, and D. DeMarco. Current Challenges in Patent Information Retrieval, chapter 1 : Introduction to Patent Searching - Practical Experience and Requirements for Searching the Patent Space. Springer Verlag, 2011.Google Scholar
- M. Annies. Full-text prior art and chemical structure searching in e-journals and on the internet - a patent information professional's perspective. World Patent Information, 31(4):278--284, 2009.Google ScholarCross Ref
- C. Buckley, M. Smucker, and M. Lease. Relevance feedback track at trec 2009 http://trec.nist.gov/data/relevance.feedback09.html.Google Scholar
- B. Carterette, E. Kanoulas, M. Sanderson, and P. Clough. Trec 2011 session track http://ir.cis.udel.edu/sessions/index.html.Google Scholar
- CERN. Worldwide lhc computing grid http://public.web.cern.ch/public/en/lhc/computing-en.html.Google Scholar
- Y. Chang, C. Cirillo, and J. Razon. Evaluation of feedback retrieval using modified freezing, residual collection & test and control groups. In G.Salton, editor, The SMART retrieval system - experiments in automatic document processing. Prentice Hall, 1971.Google Scholar
- C. Emmerich. Comparing first level patent data with value-added patent information: A case study in the pharmaceutical field. World Patent Information, 31(2):117--122, 2009.Google ScholarCross Ref
- N. Kando and M.-K. Leong. Workshop on patent retrieval (sigir 2000 workshop report). SIGIR Forum, 34(1):28--30, 2000. Google ScholarDigital Library
- M. Lupu, J. Huang, J. Zhu, and J. Tait. TREC Chemical Information Retrieval - An Evaluation Effort for Chemical IR Systems. WPI Journal, 2011.Google Scholar
- OCLC. Dewey decimal classification. http://www.oclc.org/dewey/default.htm, 2011.Google Scholar
- F. Piroi and J. Tait. Clef-ip 2010: Retrieval experiments in the intellectual property domain. In Proc. of CLEF, 2010.Google Scholar
- M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do User Preferences and Evaluation Measures Line Up? In Proc. of SIGIR, 2010. Google ScholarDigital Library
- J. Tait, M. Lupu, H. Berger, G. Roda, M. Dittenbach, A. Pesenhofer, E. Graf, and K. van Rijsbergen. Patent Search: An Important New Test Bed for IR. In Proc. of the Dutch-Belgian Information Retrieval Workshop, 2009.Google Scholar
- A. Trippe and I. Ruthven. Current Challenges in Patent Information Retrieval, chapter 6 : Evaluating Real Patent Retrieval Effectiveness. Springer Verlag, 2011.Google Scholar
- E. Voorhees and D. Harman, editors. TREC Experiment and Evaluation in Information Retrieval. MIT Press, 2005. Google ScholarDigital Library
Index Terms
- The status of retrieval evaluation in the patent domain
Recommendations
Patent image retrieval: a survey
PaIR '11: Proceedings of the 4th workshop on Patent information retrievalDrawings are an important component of patents, and many search tasks in the intellectual property domain rely on the comparison of patent drawings. In this paper, we begin with a review of algorithms developed for the automated retrieval of similar ...
Patent information retrieval: an instance of domain-specific search
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrievalThe tutorial aims to provide the IR researchers with an understanding of how the patent system works, the challenges that patent searchers face in using the existing tools and in adopting new methods developed in academia.
At the same time, the tutorial ...
Patent overlay mapping: Visualizing technological distance
This paper presents a new global patent map that represents all technological categories and a method to locate patent data of individual organizations and technological fields on the global map. This overlay map technique may support competitive ...
Comments