Inducing domain-specific semantic class taggers from (almost) nothing

ABSTRACT

This research explores the idea of inducing domain-specific semantic class taggers using only a domain-specific text collection and seed words. The learning process begins by inducing a classifier that only has access to contextual features, forcing it to generalize beyond the seeds. The contextual classifier then labels new instances, to expand and diversify the training set. Next, a cross-category bootstrapping process simultaneously trains a suite of classifiers for multiple semantic classes. The positive instances for one class are used as negative instances for the others in an iterative bootstrapping cycle. We also explore a one-semantic-class-per-discourse heuristic, and use the classifiers to dynamically create semantic features. We evaluate our approach by inducing six semantic taggers from a collection of veterinary medicine message board posts.

References

1. }}ACE. 2005. NIST ACE evaluation website. In http://www.nist.gov/speech/tests/ace/2005.
2. }}ACE. 2007. NIST ACE evaluation website. In http://www.nist.gov/speech/tests/ace/2007.
3. }}ACE. 2008. NIST ACE evaluation website. In http://www.nist.gov/speech/tests/ace/2008.
4. }}Daniel M. Bikel, Scott Miller, Richard Schwartz, and Ralph Weischedel. 1997. Nymble: a high-performance learning name-finder. In Proceedings of ANLP-97, pages 194--201.
5. }}A. Blum and T. Mitchell. 1998. Combining Labeled and Unlabeled Data with Co-Training. In Proceedings of the 11th Annual Conference on Computational Learning Theory (COLT-98).
6. }}Andrew Carlson, Justin Betteridge, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2009. Coupling semi-supervised learning of categories and relations. In HLT-NAACL 2009 Workshop on Semi-Supervised Learning for NLP.
7. }}M. Collins and Y. Singer. 1999. Unsupervised Models for Named Entity Classification. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP/VLC-99).
8. }}S. Cucerzan and D. Yarowsky. 1999. Language Independent Named Entity Recognition Combining Morphologi cal and Contextual Evidence. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP/VLC-99).
9. }}O. Etzioni, M. Cafarella, D. Downey, A. Popescu, T. Shaked, S. Soderland, D. Weld, and A. Yates. 2005. Unsupervised named-entity extraction from the web: an experimental study. Artificial Intelligence, 165(1):91--134, June.
10. }}M. B. Fleischman and E. H. Hovy. 2002. Fine grained classification of named entities. In Proceedings of the COLING conference, August.
11. }}T. Joachims. 1999. Making Large-Scale Support Vector Machine Learning Practical. In A. Smola B. Schölkopf, C. Burges, editor, Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA.
12. }}S. Keerthi and D. DeCoste. 2005. A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs. Journal of Machine Learning Research.
13. }}Mamoru Komachi, Taku Kudo, Masashi Shimbo, and Yuji Matsumoto. 2008. Graph-based analysis of semantic drift in espresso-like bootstrapping algorithms. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing.
14. }}Z. Kozareva, E. Riloff, and E. Hovy. 2008. Semantic Class Learning from the Web with Hyponym Pattern Linkage Graphs. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-08).
15. }}D. McClosky, E. Charniak, and M Johnson. 2006. Effective self-training for parsing. In HLT-NAACL-2006.
16. }}T. McIntosh and J. Curran. 2009. Reducing Semantic Drift with Bagging and Distributional Similarity. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics.
17. }}R. Mihalcea. 2004. Co-training and Self-training for Word Sense Disambiguation. In CoNLL-2004.
18. }}G. Miller. 1990. Wordnet: An On-line Lexical Database. International Journal of Lexicography, 3(4).
19. }}C. Mueller, S. Rapp, and M. Strube. 2002. Applying co-training to reference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
20. }}V. Ng and C. Cardie. 2003. Weakly supervised natural language learning without redundant views. In HLT-NAACL-2003.
21. }}V. Ng. 2007. Semantic Class Induction and Coreference Resolution. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics.
22. }}Cheng Niu, Wei Li, Jihong Ding, and Rohini K. Srihari. 2003. A bootstrapping approach to named entity classification using successive learners. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics (ACL-03), pages 335--342.
23. }}M. Paşca. 2004. Acquisition of categorized named entities for web search. In Proc. of the Thirteenth ACM International Conference on Information and Knowledge Management, pages 137--145.
24. }}W. Phillips and E. Riloff. 2002. Exploiting Strong Syntactic Heuristics and Co-Training to Learn Semantic Lexicons. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 125--132.
25. }}E. Riloff and R. Jones. 1999. Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping. In Proceedings of the Sixteenth National Conference on Artificial Intelligence.
26. }}E. Riloff and J. Shepherd. 1997. A Corpus-Based Approach for Building Semantic Lexicons. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 117--124.
27. }}B. Roark and E. Charniak. 1998. Noun-phrase Co-occurrence Statistics for Semi-automatic Semantic Lexicon Construction. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics, pages 1110--1116.
28. }}M. Thelen and E. Riloff. 2002. A Bootstrapping Method for Learning Semantic Lexicons Using Extraction Pa ttern Contexts. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 214--221.
29. }}K. Toutanova, D. Klein, C. Manning, and Y. Singer. 2003. Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network. In Proceedings of HLT-NAACL 2003.
30. }}R. Yangarber. 2003. Counter-training in the discovery of semantic patterns. In Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics.
31. }}D. Yarowsky. 1995. Unsupervised Word Sense Disambiguation Rivaling Supervised Methods. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics.
32. }}Imed Zitouni and Radu Florian. 2009. Cross-language information propagation for arabic mention detection. ACM Transactions on Asian Language Information Processing (TALIP), 8(4):1--21.

Index Terms

• Applied computing

  • Arts and humanities

    • Language translation

• Computing methodologies

  • Artificial intelligence

    • Natural language processing

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Machine learning approaches

      • Classification and regression trees