A structured vector space model for word meaning in context

ABSTRACT

We address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing models for this task do not take syntactic structure sufficiently into account.

We present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words' argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.

References

1. C. Brockmann, M. Lapata. 2003. Evaluating and combining approaches to selectional preference acquisition. In Proceedings of EACL, 27--34.
2. I. Dagan, O. Glickman, B. Magnini. 2006. The PASCAL Recognising Textual Entailment Challenge. In Machine Learning Challenges, Lecture Notes in Computer Science, 177--190. Springer.
3. K. Erk. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of ACL, 216--223.
4. T. Ferretti, C. Gagné, K. McRae. 2003. Thematic role focusing by participle inflections: evidence form conceptual combination. Journal of Experimental Psychology, 29(1):118--127.
5. D. Gildea, D. Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245--288.
6. D. Hindle, M. Rooth. 1993. Structural ambiguity and lexical relations. Computational Linguistics, 19(1):103--120.
7. M. Jones, D. Mewhort. 2007. Representing word meaning and order information in a composite holographic lexicon. Psychological review, 114:1--37.
8. J. J. Katz, J. A. Fodor. 1964. The structure of a semantic theory. In The Structure of Language. Prentice-Hall.
9. A. Kilgarriff. 1997. I don't believe in word senses. Computers and the Humanities, 31(2):91--113.
10. W. Kintsch. 2001. Predication. Cognitive Science, 25:173--202.
11. T. Landauer, S. Dumais. 1997. A solution to Platos problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211--240.
12. D. Lin. 1993. Principle-based parsing without overgeneration. In Proceedings of ACL, 112--120.
13. D. Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of COLING-ACL, 768--774.
14. K. Lund, C. Burgess. 1996. Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, and Computers, 28:203--208.
15. C. D. Manning, P. Raghavan, H. Schütze. 2008. Introduction to Information Retrieval. CUP.
16. D. McCarthy, J. Carroll. 2003. Disambiguating nouns, verbs, and adjectives using automatically acquired selectional preferences. Computational Linguistics, 29(4):639--654.
17. D. McCarthy, R. Navigli. 2007. SemEval-2007 Task 10: English Lexical Substitution Task. In Proceedings of SemEval, 48--53.
18. S. McDonald, C. Brew. 2004. A distributional model of semantic context effects in lexical processing. In Proceedings of ACL, 17--24.
19. S. McDonald, M. Ramscar. 2001. Testing the distributional hypothesis: The influence of context on judgements of semantic similarity. In Proceedings of CogSci, 611--616.
20. K. McRae, M. Spivey-Knowlton, M. Tanenhaus. 1998. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38:283--312.
21. K. McRae, M. Hare, J. Elman, T. Ferretti. 2005. A basis for generating expectancies for verbs from nouns. Memory and Cognition, 33(7):1174--1184.
22. J. Mitchell, M. Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL, 236--244.
23. A. Moschitti, S. Quarteroni. 2008. Kernels on linguistic structures for answer extraction. In Proceedings of ACL, 113--116, Columbus, OH.
24. S. Narayanan, D. Jurafsky. 2002. A Bayesian model predicts human parse preference and reading time in sentence processing. In Proceedings of NIPS, 59--65.
25. S. Padó, M. Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161--199.
26. U. Padó, F. Keller, M. W. Crocker. 2006. Combining syntax and thematic fit in a probabilistic model of sentence processing. In Proceedings of CogSci, 657--662.
27. S. Padó, U. Padó, K. Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of EMNLP/CoNLL, 400--409.
28. P. Resnik. 1996. Selectional constraints: An information-theoretic model and its computational realization. Cognition, 61:127--159.
29. G. Salton, A. Wang, C. Yang. 1975. A vector-space model for information retrieval. Journal of the American Society for Information Science, 18:613--620.
30. H. Schütze. 1998. Automatic word sense discrimination. Computational Linguistics, 24(1):97--124.
31. S. Sharoff. 2006. Open-source corpora: Using the net to fish for linguistic data. International Journal of Corpus Linguistics, 11(4):435--462.
32. P. Smolensky. 1990. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46:159--216.
33. I. Szpektor, I. Dagan, R. Bar-Haim, J. Goldberger. 2008. Contextual preferences. In Proceedings of ACL, 683--691, Columbus, OH.
34. Y. Wilks. 1975. Preference semantics. In Formal Semantics of Natural Language. CUP.
35. A. Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceeedings of COLING, 947--953.