Jesús Tomás
ABSTRACT
Accurate evaluation of machine translation (MT) is an open problem. A brief survey of the current approach to tackle this problem is presented and a new proposal is introduced. This proposal attempts to measure the percentage of words, which should be modified at the output of an automatic translator in order to obtain a correct translation. To show the feasibility of the method we have assessed the most important Spanish-Catalan translators in comparing the results obtained by the various methods.
- Brown, P. F., J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R. Mercer,&P. Roossin. 1990. A Statistical Approach to Machine Translation. Computational Linguistics 16(2). Google Scholar
Digital Library
- Canals-Marote, R., A. Esteve-Guillén, A. Garrido-Alenda, M. I. Guardiola-Savall, A. Iturraspe-Bellver, S. Montserrat-Buendia, S. Ortiz-Rojas, H. Pastor-Pina, P. M. Pérez-Antón, M. L. Forcada. 2001. The Spanish-Catalan machine translation system interNOSTRUM. In Proceedings of the Machine Translation Summit VIII. Santiago de Compostela, Spain.Google Scholar
- Niessen, S., F. J. Och, G. Leusch, and H. Ney. 2000. An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation, Athens, Greece.Google Scholar
- Papineni, K. A., S. Roukos, T. Ward, W. J. Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia. Google ScholarDigital Library
- Tomás, J., F. Casacuberta. 2001. Monotone Statistical Translation using Word Groups. In Proceedings of the Machine Translation Summit VIII. Santiago de Compostela, Spain.Google Scholar
- Tillmann, C., S. Vogel, H. Ney, H. Sawaf, and A. Zubiaga. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of the 5th European Conference on Speech Communication and Technology, Rhodes, Greece.Google Scholar
- Vidal, E. 1997. Finite-State Speech-to-Speech Translation. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Munich, Germany. Google ScholarDigital Library
- White, J., T. O'Connell, F. O'Mara. 1994. The DARPA Machine Translation Evaluation Methodologies: Evolution, Lessons and Future Approaches. In Proceedings of the first Conference of the Association for Machine Translation in the Americas. Columbia, USA.Google Scholar
- A quantitative method for machine translation evaluation
Recommendations
Evaluation of machine translation
ICWET '11: Proceedings of the International Conference & Workshop on Emerging Trends in TechnologyMachine Translation (MT) refers to the use of a machine for performing translation task which converts text or speech from one Natural Language (NL) into another Natural Language. Machine Translation is an important technology for localization, and is ...
Dependency-based automatic evaluation for machine translation
SSST '07: Proceedings of the NAACL-HLT 2007/AMTA Workshop on Syntax and Structure in Statistical TranslationWe present a novel method for evaluating the output of Machine Translation (MT), based on comparing the dependency structures of the translation and reference rather than their surface string forms. Our method uses a treebank-based, widecoverage, ...
Quantitative fine-grained human evaluation of machine translation systems: a case study on English to Croatian
This paper presents a quantitative fine-grained manual evaluation approach to comparing the performance of different machine translation (MT) systems. We build upon the well-established multidimensional quality metrics (MQM) error taxonomy and implement ...
Comments