Table of Contents Author Guidelines Submit a Manuscript
Computational Intelligence and Neuroscience
Volume 2015 (2015), Article ID 986574, 18 pages
http://dx.doi.org/10.1155/2015/986574
Research Article

Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation

1University of Cambridge, Cambridge CB2 1TN, UK
2Swedish Institute of Computer Science, 164 29 Kista, Sweden
3Redwood Center for Theoretical Neuroscience, University of California, Berkeley, Berkeley, CA 94720, USA
4Indiana University, Bloomington, IN 47405, USA

Received 14 December 2014; Accepted 26 February 2015

Academic Editor: Carlos M. Travieso-González

Copyright © 2015 Gabriel Recchia et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. M. N. Jones, J. Willits, and S. Dennis, “Models of semantic memory,” in Oxford Handbook of Mathematical and Computational Psychology, J. R. Busemeyer and J. T. Townsend, Eds., 2015. View at Google Scholar
  2. T. K. Landauer and S. T. Dumais, “A solution to Plato's problem: the latent semantic analysis theory of acquisition, induction and representation of knowledge,” Psychological Review, vol. 104, no. 2, pp. 211–240, 1997. View at Publisher · View at Google Scholar · View at Scopus
  3. K. Lund and C. Burgess, “Producing high-dimensional semantic spaces from lexical co-occurrence,” Behavior Research Methods, Instruments, and Computers, vol. 28, no. 2, pp. 203–208, 1996. View at Publisher · View at Google Scholar · View at Scopus
  4. M. Hare, M. N. Jones, C. Thomson, S. Kelly, and K. McRae, “Activating event knowledge,” Cognition, vol. 111, no. 2, pp. 151–167, 2009. View at Publisher · View at Google Scholar · View at Scopus
  5. L. L. Jones, “Pure mediated priming: a retrospective semantic matching model,” Journal of Experimental Psychology: Learning Memory and Cognition, vol. 36, no. 1, pp. 135–146, 2010. View at Publisher · View at Google Scholar · View at Scopus
  6. T. Warren, K. McConnell, and K. Rayner, “Effects of context on eye movements when reading about possible and impossible events,” Journal of Experimental Psychology: Learning Memory and Cognition, vol. 34, no. 4, pp. 1001–1010, 2008. View at Publisher · View at Google Scholar · View at Scopus
  7. B. T. Johns and M. N. Jones, “Predicting word-naming and lexical decision times from a semantic space model,” in Proceedings of the 30th Annual Meeting of the Cognitive Science Society, B. C. Love, K. McRae, and V. M. Sloutsky, Eds., pp. 279–284, Cognitive Science Society, Austin, Tex, USA, 2008.
  8. B. T. Johns and M. N. Jones, “False recognition through semantic amplification,” in Proceedings of the 31st Annual Meeting of the Cognitive Science Society, N. A. Taatgen and H. van Rijn, Eds., pp. 2795–2800, Cognitive Science Society, Austin, Tex, USA, 2009.
  9. W. Kintsch and P. Mangalath, “The construction of meaning,” Topics in Cognitive Science, vol. 3, no. 2, pp. 346–370, 2011. View at Publisher · View at Google Scholar
  10. T. Landauer, D. McNamara, S. Dennis, and W. Kintsch, Handbook of Latent Semantic Analysis, Erlbaum, Mahwah, NJ, USA, 2006.
  11. J. Mitchell and M. Lapata, “Composition in distributional models of semantics,” Cognitive Science, vol. 34, no. 8, pp. 1388–1429, 2010. View at Publisher · View at Google Scholar
  12. A. Utsumi and M. Sakamoto, “Predicative metaphor comprehension as indirect categorization,” in Proceedings of the 32nd Annual Meeting of the Cognitive Science Society, S. Ohlsson and R. Catrambone, Eds., pp. 1034–1039, Cognitive Science Society, Austin, Tex, USA, 2010.
  13. C. A. Perfetti, “The limits of cooccurrence: tools and theories in language research,” Discourse Processes, vol. 25, no. 2-3, pp. 363–377, 1998. View at Publisher · View at Google Scholar
  14. B. Riordan and M. N. Jones, “Comparing semantic space models using child-directed speech,” in Proceedings of the 29th Annual Cognitive Science Society, D. S. MacNamara and J. G. Trafton, Eds., pp. 599–604, Cognitive Science Society, Austin, Tex, USA, 2007. View at Google Scholar
  15. B. Riordan and M. N. Jones, “Redundancy in perceptual and linguistic experience: comparing feature-based and distributional models of semantic representation,” Topics in Cognitive Science, vol. 3, no. 2, pp. 303–345, 2011. View at Publisher · View at Google Scholar · View at Scopus
  16. J. A. Bullinaria and J. P. Levy, “Extracting semantic representations from word co-occurrence statistics: a computational study,” Behavior Research Methods, vol. 39, no. 3, pp. 510–526, 2007. View at Publisher · View at Google Scholar · View at Scopus
  17. M. Sahlgren, A. Holst, and P. Kanerva, “Permutations as a means to encode order in word space,” in Proceedings of the 30th Annual Meeting of the Cognitive Science Society, B. C. Love, K. McRae, and V. M. Sloutsky, Eds., pp. 1300–1305, Cognitive Science Society, Austin, Tex, USA, 2008. View at Google Scholar
  18. T. L. Griffiths, M. Steyvers, and J. B. Tenenbaum, “Topics in semantic representation,” Psychological Review, vol. 114, no. 2, pp. 211–244, 2007. View at Publisher · View at Google Scholar · View at Scopus
  19. M. N. Jones and D. J. K. Mewhort, “Representing word meaning and order information in a composite holographic lexicon,” Psychological Review, vol. 114, no. 1, pp. 1–37, 2007. View at Google Scholar · View at Scopus
  20. S. Dennis, “A memory-based theory of verbal cognition,” Cognitive Science, vol. 29, no. 2, pp. 145–193, 2005. View at Publisher · View at Google Scholar
  21. P. Kanerva, J. Kristoferson, and A. Holst, “Random indexing of text samples for latent semantic analysis,” in Proceedings of the 22nd Annual Meeting of the Cognitive Science Society, L. R. Gleitman and A. K. Joshi, Eds., pp. 103–106, Cognitive Science Society, Philadelphia, Pa, USA, 2000.
  22. P. J. Kwantes, “Using context to build semantics,” Psychonomic Bulletin & Review, vol. 12, no. 4, pp. 703–710, 2005. View at Publisher · View at Google Scholar · View at Scopus
  23. J. R. Firth, “A synopsis of linguistic theory, 1930–1955,” in Studies in Linguistic Analysis, J. R. Firth, Ed., pp. 1–32, Blackwell, Oxford, UK, 1957. View at Google Scholar
  24. Z. Harris, Distributional Structure, Humanities Press, New York, NY, USA, 1970.
  25. M. de Vega, A. M. Glenberg, and A. C. Graesser, Symbols and Embodiment: Debates on Meaning and Cognition, Oxford University Press, Oxford, UK, 2008.
  26. A. M. Glenberg and D. A. Robertson, “Symbol grounding and meaning: a comparison of high-dimensional and embodied theories of meaning,” Journal of Memory and Language, vol. 43, no. 3, pp. 379–401, 2000. View at Publisher · View at Google Scholar · View at Scopus
  27. M. Andrews, G. Vigliocco, and D. Vinson, “Integrating experiential and distributional data to learn semantic representations,” Psychological Review, vol. 116, no. 3, pp. 463–498, 2009. View at Publisher · View at Google Scholar · View at Scopus
  28. K. Durda, L. Buchanan, and R. Caron, “Grounding co-occurrence: identifying features in a lexical co-occurrence model of semantic memory,” Behavior Research Methods, vol. 41, no. 4, pp. 1210–1223, 2009. View at Publisher · View at Google Scholar · View at Scopus
  29. M. N. Jones and G. L. Recchia, “You can't wear a coat rack: a binding framework to avoid illusory feature migrations in perceptually grounded semantic models,” in Proceedings of the 32nd Annual Meeting of the Cognitive Science Society, S. Ohisson and R. Catrambone, Eds., pp. 877–882, Cognitive Science Society, Austin, Tex, USA, 2010. View at Google Scholar
  30. M. Steyvers, “Combining feature norms and text data with topic models,” Acta Psychologica, vol. 133, no. 3, pp. 234–243, 2010. View at Publisher · View at Google Scholar · View at Scopus
  31. R. L. Gómez and L. A. Gerken, “Infant artificial language learning and language acquisition,” Trends in Cognitive Sciences, vol. 4, no. 5, pp. 178–186, 2000. View at Publisher · View at Google Scholar · View at Scopus
  32. J. R. Saffran, “Statistical language learning: mechanisms and constraints,” Current Directions in Psychological Science, vol. 12, no. 4, pp. 110–114, 2003. View at Publisher · View at Google Scholar · View at Scopus
  33. C. M. Conway, A. Bauernschmidt, S. S. Huang, and D. B. Pisoni, “Implicit statistical learning in language processing: word predictability is the key,” Cognition, vol. 114, no. 3, pp. 356–371, 2010. View at Publisher · View at Google Scholar · View at Scopus
  34. D. J. Foss, “A discourse on semantic priming,” Cognitive Psychology, vol. 14, no. 4, pp. 590–607, 1982. View at Publisher · View at Google Scholar · View at Scopus
  35. M. Hare, K. McRae, and J. L. Elman, “Sense and structure: meaning as a determinant of verb subcategorization preferences,” Journal of Memory and Language, vol. 48, no. 2, pp. 281–303, 2003. View at Publisher · View at Google Scholar · View at Scopus
  36. M. Hare, K. McRae, and J. L. Elman, “Admitting that admitting verb sense into corpus analyses makes sense,” Language and Cognitive Processes, vol. 19, no. 2, pp. 181–224, 2004. View at Publisher · View at Google Scholar · View at Scopus
  37. M. C. Macdonald, “The interaction of lexical and syntactic ambiguity,” Journal of Memory and Language, vol. 32, no. 5, pp. 692–715, 1993. View at Publisher · View at Google Scholar · View at Scopus
  38. M. C. MacDonald, “Lexical representations and sentence processing: an introduction,” Language and Cognitive Processes, vol. 12, no. 2-3, pp. 121–136, 1997. View at Publisher · View at Google Scholar
  39. K. McRae, M. Hare, J. L. Elman, and T. Ferretti, “A basis for generating expectancies for verbs from nouns,” Memory and Cognition, vol. 33, no. 7, pp. 1174–1184, 2005. View at Publisher · View at Google Scholar · View at Scopus
  40. R. Taraban and J. L. McClelland, “Constituent attachment and thematic role assignment in sentence processing: influences of content-based expectations,” Journal of Memory & Language, vol. 27, no. 6, pp. 597–632, 1988. View at Publisher · View at Google Scholar · View at Scopus
  41. T. K. Landauer, D. Laham, B. Rehder, and M. E. Schreiner, “How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans,” in Proceedings of the 19th Annual Meeting of the Cognitive Science Society, M. G. Shafto and P. Langley, Eds., pp. 412–417, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 1997. View at Google Scholar
  42. C. Burgess, “From simple associations to the building blocks of language: modeling meaning in memory with the HAL model,” Behavior Research Methods, Instruments, and Computers, vol. 30, no. 2, pp. 188–198, 1998. View at Publisher · View at Google Scholar · View at Scopus
  43. W. Kintsch, “An overview of top-down and bottom-up effects in comprehension: the CI perspective,” Discourse Processes, vol. 39, no. 2-3, pp. 125–129, 2005. View at Google Scholar
  44. J. L. Elman, “On the meaning of words and dinosaur bones: lexical knowledge without a lexicon,” Cognitive Science, vol. 33, no. 4, pp. 547–582, 2009. View at Publisher · View at Google Scholar · View at Scopus
  45. G. McKoon and R. Ratcliff, “Meaning through syntax: language comprehension and the reduced relative clause construction,” Psychological Review, vol. 110, no. 3, pp. 490–525, 2003. View at Publisher · View at Google Scholar · View at Scopus
  46. R. Jackendoff, Foundations of Language: Brain, Meaning, Grammar, Evolution, Oxford University Press, Oxford, UK, 2002.
  47. P. G. O'Seaghdha, “The dependence of lexical relatedness effects on syntactic connectedness,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 15, no. 1, pp. 73–87, 1989. View at Publisher · View at Google Scholar · View at Scopus
  48. M. Andrews and G. Vigliocco, “The hidden Markov topic model: a probabilistic model of semantic representation,” Topics in Cognitive Science, vol. 2, no. 1, pp. 101–113, 2010. View at Publisher · View at Google Scholar · View at Scopus
  49. M. N. Jones, W. Kintsch, and D. J. K. Mewhort, “High-dimensional semantic space accounts of priming,” Journal of Memory and Language, vol. 55, no. 4, pp. 534–552, 2006. View at Publisher · View at Google Scholar · View at Scopus
  50. C. Shaoul and C. Westbury, “Exploring lexical co-occurrence space using HiDEx,” Behavior Research Methods, vol. 42, no. 2, pp. 393–413, 2010. View at Publisher · View at Google Scholar · View at Scopus
  51. M. M. Louwerse, “Symbol interdependency in symbolic and embodied cognition,” Topics in Cognitive Science, vol. 3, no. 2, pp. 273–302, 2011. View at Publisher · View at Google Scholar · View at Scopus
  52. G. L. Recchia and M. N. Jones, “More data trumps smarter algorithms: comparing pointwise mutual information with latent semantic analysis,” Behavior Research Methods, vol. 41, no. 3, pp. 647–656, 2009. View at Publisher · View at Google Scholar · View at Scopus
  53. T. R. Risley and B. Hart, “Promoting early language development,” in The Crisis in Youth Mental Health: Critical Issues and Effective Programs, Volume 4, Early Intervention Programs and Policies, N. F. Watt, C. Ayoub, R. H. Bradley, J. E. Puma, and W. A. LeBoeuf, Eds., pp. 83–88, Praeger, Westport, Conn, USA, 2006. View at Google Scholar
  54. M. Brand, “Fast low-rank modifications of the thin singular value decomposition,” Linear Algebra and its Applications, vol. 415, no. 1, pp. 20–30, 2006. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  55. G. Gorrell, “Generalized Hebbian algorithm for incremental singular value decomposition in natural language processing,” in Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, S. Pado, J. Read, and V. Seretan, Eds., pp. 97–104, April 2006. View at Scopus
  56. M. Louwerse and L. Connell, “A taste of words: linguistic context and perceptual simulation predict the modality of words,” Cognitive Science, vol. 35, no. 2, pp. 381–398, 2011. View at Publisher · View at Google Scholar · View at Scopus
  57. J. L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, no. 2, pp. 179–211, 1990. View at Publisher · View at Google Scholar
  58. T. T. Rogers and J. L. McClelland, Semantic Cognition: A Parallel Distributed Processing Approach, MIT Press, Cambridge, UK, 2004.
  59. L. Onnis and M. H. Christiansen, “Lexical categories at the edge of the word,” Cognitive Science, vol. 32, no. 1, pp. 184–221, 2008. View at Publisher · View at Google Scholar
  60. J. A. Legate and C. Yang, “Assessing child and adult grammar,” in Rich Languages from Poor Inputs, M. Piattelli-Palmarini and R. C. Berwick, Eds., pp. 68–182, Oxford University Press, Oxford, UK, 2013. View at Google Scholar
  61. H. A. Simon, “The architecture of complexity: hierarchic systems,” in The Sciences of the Artificial, H. A. Simon, Ed., pp. 183–216, MIT Press, Cambridge, Mass, USA, 2nd edition, 1996. View at Google Scholar
  62. J. L. Elman, “Learning and development in neural networks: the importance of starting small,” Cognition, vol. 48, no. 1, pp. 71–99, 1993. View at Publisher · View at Google Scholar · View at Scopus
  63. D. Servan-Schreiber, A. Cleeremans, and J. L. Mcclelland, “Graded state machines: the representation of temporal contingencies in simple recurrent networks,” Machine Learning, vol. 7, no. 2-3, pp. 161–193, 1991. View at Publisher · View at Google Scholar · View at Scopus
  64. M. F. St. John and J. L. McClelland, “Learning and applying contextual constraints in sentence comprehension,” Artificial Intelligence, vol. 46, no. 1-2, pp. 217–257, 1990. View at Publisher · View at Google Scholar · View at Scopus
  65. M. W. Howard, “Neurobiology can provide meaningful constraints to cognitive modeling: the temporal context model as a description of medial temporal lobe function,” in Proceedings of the 37th Annual Meeting of the Society for Mathematical Psychology, Ann Arbor, Mich, USA, July 2004.
  66. M. W. Howard and M. J. Kahana, “A distributed representation of temporal context,” Journal of Mathematical Psychology, vol. 46, no. 3, pp. 269–299, 2002. View at Publisher · View at Google Scholar · View at Scopus
  67. M. W. Howard, K. H. Shankar, and U. K. K. Jagadisan, “Constructing semantic representations from a gradually changing representation of temporal context,” Topics in Cognitive Science, vol. 3, no. 1, pp. 48–73, 2011. View at Publisher · View at Google Scholar · View at Scopus
  68. V. A. Rao and M. W. Howard, “Retrieved context and the discovery of semantic structure,” Advances in Neural Information Processing Systems, vol. 20, pp. 1193–1200, 2008. View at Google Scholar
  69. M. H. Tong, A. D. Bickett, E. M. Christiansen, and G. W. Cottrell, “Learning grammatical structure with echo state networks,” Neural Networks, vol. 20, no. 3, pp. 424–432, 2007. View at Publisher · View at Google Scholar · View at Scopus
  70. J. L. Elman, “Distributed representations, simple recurrent networks, and grammatical structure,” Machine Learning, vol. 7, no. 2-3, pp. 195–225, 1991. View at Publisher · View at Google Scholar · View at Scopus
  71. R. Veale and M. Scheutz, “Neural circuits for any-time phrase recognition with applications in cognitive models and human-robot interaction,” in Proceedings of the 34th Annual Conference of the Cognitive Science Society, N. Miyake, D. Peebles, and R. P. Cooper, Eds., pp. 1072–1077, Cognitive Science Society, Austin, Tex, USA, 2012. View at Google Scholar
  72. G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  73. Y. Bengio, “Learning deep architectures for AI,” Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1–127, 2009. View at Publisher · View at Google Scholar · View at Scopus
  74. Y. Bengio, “Deep learning of representations: looking forward,” in Statistical Language and Speech Processing, vol. 7978 of Lecture Notes in Computer Science, pp. 1–37, Springer, Berlin, Germany, 2013. View at Publisher · View at Google Scholar
  75. G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, 2006. View at Publisher · View at Google Scholar · View at Scopus
  76. J. Martens and I. Sutskever, “Learning recurrent neural networks with Hessian-free optimization,” in Proceedings of the 28th International Conference on Machine Learning (ICML '11), pp. 1033–1040, Bellevue, Wash, USA, July 2011. View at Scopus
  77. R. Collobert and J. Weston, “A unified architecture for natural language processing: deep neural networks with multitask learning,” in Proceedings of the 25th International Conference on Machine Learning (ICML '08), pp. 160–167, ACM, Helsinki, Finland, July 2008. View at Publisher · View at Google Scholar · View at Scopus
  78. A. Mnih and G. E. Hinton, “A scalable hierarchical distributed language model,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '09), pp. 1081–1088, 2009. View at Scopus
  79. J. Weston, F. Ratle, H. Mobahi, and R. Collobert, “Deep learning via semi-supervised embedding,” in Neural Networks: Tricks of the Trade, vol. 7700 of Lecture Notes in Computer Science, pp. 639–655, Springer, Berlin, Germany, 2012. View at Publisher · View at Google Scholar
  80. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems 26, pp. 3111–3119, 2013. View at Google Scholar
  81. I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance of initialization and momentum in deep learning,” in Proceedings of the 30th International Conference on Machine Learning (ICML '13), pp. 1139–1147, 2013.
  82. I. Sutskever, J. Martens, and G. E. Hinton, “Generating text with recurrent neural networks,” in Proceedings of the 28th International Conference on Machine Learning (ICML '11), pp. 1017–1024, New York, NY, USA, July 2011. View at Scopus
  83. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, “Multimodal deep learning,” in Proceedings of the 28th International Conference on Machine Learning (ICML '11), pp. 689–696, Bellevue, Wash, USA, July 2011. View at Scopus
  84. C. Burgess and K. Lund, “The dynamics of meaning in memory,” in Cognitive Dynamics: Conceptual and Representational Change in Humans and Machines, E. Dietrich, Eric, and A. B. Markman, Eds., pp. 117–156, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, USA, 2000. View at Publisher · View at Google Scholar
  85. K. Durda and L. Buchanan, “WINDSORS: windsor improved norms of distance and similarity of representations of semantics,” Behavior Research Methods, vol. 40, no. 3, pp. 705–712, 2008. View at Publisher · View at Google Scholar · View at Scopus
  86. D. Rohde, L. Gonnerman, and D. Plaut, “An improved model of semantic similarity based on lexical co-occurence,” in Proceedings of the Annual Meeting of the Cognitive Science Society, 2006.
  87. L. Buchanan, C. Burgess, and K. Lund, “Overcrowding in semantic neighborhoods: modeling deep dyslexia,” Brain and Cognition, vol. 32, no. 2, pp. 111–114, 1996. View at Google Scholar · View at Scopus
  88. L. Buchanan, C. Westbury, and C. Burgess, “Characterizing semantic space: neighborhood effects in word recognition,” Psychonomic Bulletin and Review, vol. 8, no. 3, pp. 531–544, 2001. View at Publisher · View at Google Scholar · View at Scopus
  89. P. D. Siakaluk, C. Westbury, and L. Buchanan, “The effects of filler type and semantic distance in naming: evidence for change in time criterion rather than change in routes,” in Proceedings of the 13th Annual Meeting of the Canadian Society for Brain, Behaviour, and Cognitive Science, Hamilton, Canada, May 2003.
  90. D. Song and P. Bruza, “Discovering information flow using a high dimensional conceptual space,” in Proceedings of the 24th Annual International Conference on Research and Development in Information Retrieval, W. B. Croft, D. J. Harper, D. H. Kraft, and J. Zobel, Eds., pp. 327–333, Association for Computing Machinery, New York, NY, USA, 2001.
  91. T. L. Griffiths, M. Steyvers, D. M. Blei, and J. B. Tenenbaum, “Integrating topics and syntax,” Advances in Neural Information Processing Systems, vol. 17, pp. 537–544, 2005. View at Google Scholar
  92. P. Kanerva, “Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors,” Cognitive Computation, vol. 1, no. 2, pp. 139–159, 2009. View at Publisher · View at Google Scholar
  93. G. E. Cox, G. Kachergis, G. Recchia, and M. N. Jones, “Toward a scalable holographic word-form representation,” Behavior Research Methods, vol. 43, no. 3, pp. 602–615, 2011. View at Publisher · View at Google Scholar · View at Scopus
  94. M. Sahlgren, “An introduction to random indexing,” in Proceedings of the Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering, Copenhagen, Denmark, August 2005.
  95. T. Plate, Holographic Reduced Representation: Distributed Representation for Cognitive Structures, CSLI Publications, Stanford, Calif, USA, 2003.
  96. T. K. Landauer and S. Dumais, “Latent semantic analysis,” Scholarpedia, vol. 3, no. 11, p. 4356, 2013. View at Publisher · View at Google Scholar
  97. D. C. van Leijenhorst and T. P. van der Weide, “A formal derivation of Heaps' Law,” Information Sciences, vol. 170, no. 2–4, pp. 263–272, 2005. View at Publisher · View at Google Scholar · View at Scopus
  98. M. N. Jones, Learning semantics and word order from statistical redundancies in language: a computational approach [Ph.D. dissertation], Queen's University, 2005.
  99. J. Karlgren and M. Sahlgren, “From words to understanding,” in Foundations of Real-World Intelligence, Y. Uesaka, P. Kanerva, and H. Asoh, Eds., CSLI Publications, Stanford, Calif, USA, 2001. View at Google Scholar
  100. T. A. Plate, “Holographic reduced representations,” IEEE Transactions on Neural Networks, vol. 6, no. 3, pp. 623–641, 1995. View at Publisher · View at Google Scholar · View at Scopus
  101. A. Borsellino and T. Poggio, “Convolution and correlation algebras,” Kybernetik, vol. 13, no. 2, pp. 113–122, 1973. View at Publisher · View at Google Scholar · View at Scopus
  102. J. M. Eich, “A composite holographic associative recall model,” Psychological Review, vol. 89, no. 6, pp. 627–661, 1982. View at Publisher · View at Google Scholar · View at Scopus
  103. J. M. Eich, “Levels of processing, encoding specificity, elaboration, and CHARM,” Psychological Review, vol. 92, no. 1, pp. 1–38, 1985. View at Google Scholar · View at Scopus
  104. P. Liepa, Models of Content Addressable Distributed Associative Memory (CADAM), University of Toronto, 1977.
  105. B. B. Murdock, “A theory for the storage and retrieval of item and associative information,” Psychological Review, vol. 89, no. 6, pp. 609–626, 1982. View at Publisher · View at Google Scholar · View at Scopus
  106. B. B. Murdock, “Item and associative information in a distributed memory model,” Journal of Mathematical Psychology, vol. 36, no. 1, pp. 68–99, 1992. View at Publisher · View at Google Scholar · View at Scopus
  107. J. C. R. Licklider, “Three auditory theories,” in Psychology, A Study of a Science, S. Koch, Ed., vol. 1, pp. 41–144, McGraw-Hill, New York, NY, USA, 1959. View at Google Scholar
  108. K. H. Pribram, “Convolution and matrix systems as content addressible distributed brain processes in perception and memory,” Journal of Neurolinguistics, vol. 2, no. 1-2, pp. 349–364, 1986. View at Publisher · View at Google Scholar · View at Scopus
  109. W. E. Reichardt, “Cybernetics of the insect optomotor response,” in Cerebral Correlates of Conscious Experience, P. Buser, Ed., North Holland, Amsterdam, The Netherlands, 1978. View at Google Scholar
  110. C. Eliasmith, “Cognition with neurons: a large-scale, biologically realistic model of the Wason task,” in Proceedings of the 27th Annual Meeting of the Cognitive Science Society, B. G. Bara, L. Barsalou, and M. Bucciarelli, Eds., pp. 624–629, Erlbaum, Mahwah, NJ, USA, 2005.
  111. P. Kanerva, Sparse Distributed Memory, MIT Press, Cambridge, Mass, USA, 1988.
  112. P. Kanerva, “Binary spatter-coding of ordered k-tuples,” in Proceedings of the International Conference on Artificial Neural Networks, C. von der Malsburg, W. von Seelen, J. Vorbrüggen, and B. Sendhoff, Eds., vol. 1112 of Lecture Notes in Computer Science, pp. 869–873, Springer, Bochum, Germany, 1996. View at Publisher · View at Google Scholar
  113. G. Recchia, M. Jones, M. Sahlgren, and P. Kanerva, “Encoding sequential information in vector space models of semantics: Comparing holographic reduced representation and random permutation,” in Proceedings of the 32nd Cognitive Science Society, S. Ohlsson and R. Catrambone, Eds., pp. 865–870, Cognitive Science Society, Austin, Tex, USA, 2010.
  114. J. A. Willits, S. K. D'Mello, N. D. Duran, and A. Olney, “Distributional statistics and thematic role relationships,” in Proceedings of the 29th Annual Meeting of the Cognitive Science Society, pp. 707–712, Nashville, Tenn, USA, 2007.
  115. H. Rubenstein and J. Goodenough, “Contextual correlates of synonymy,” Communications of the ACM, vol. 8, no. 10, pp. 627–633, 1965. View at Publisher · View at Google Scholar
  116. G. A. Miller and W. G. Charles, “Contextual correlates of semantic similarity,” Language and Cognitive Processes, vol. 6, no. 1, pp. 1–28, 1991. View at Publisher · View at Google Scholar
  117. P. Resnik, “Using information content to evaluate semantic similarity,” in Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI '95), C. S. Mellish, Ed., pp. 448–453, Morgan Kaufmann, Montréal, Canada, 1995.
  118. L. Finkelstein, E. Gabrilovich, Y. Matias et al., “Placing search in context: the concept revisited,” ACM Transactions on Information Systems, vol. 20, no. 1, pp. 116–131, 2002. View at Publisher · View at Google Scholar · View at Scopus
  119. M. Sahlgren, The Word-Space model: using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces [Ph.D. dissertation], Stockholm University, 2006.
  120. M. Lukoševičius and H. Jaeger, “Reservoir computing approaches to recurrent neural network training,” Computer Science Review, vol. 3, no. 3, pp. 127–149, 2009. View at Publisher · View at Google Scholar · View at Scopus
  121. T. A. Plate, “Randomly connected sigma-pi neurons can form associator networks,” Network: Computation in Neural Systems, vol. 11, no. 4, pp. 321–332, 2000. View at Google Scholar · View at Scopus
  122. T. Yamazaki and S. Tanaka, “The cerebellum as a liquid state machine,” Neural Networks, vol. 20, no. 3, pp. 290–297, 2007. View at Publisher · View at Google Scholar · View at Scopus
  123. J. Garson, “Connectionism,” in The Stanford Encyclopedia of Philosophy (Winter 2012 Edition), E. N. Zalta, Ed., 2012, http://plato.stanford.edu/archives/win2012/entries/connectionism/. View at Google Scholar
  124. K. Murakoshi and K. Suganuma, “A neural circuit model forming semantic network with exception using spike-timing-dependent plasticity of inhibitory synapses,” BioSystems, vol. 90, no. 3, pp. 903–910, 2007. View at Publisher · View at Google Scholar · View at Scopus
  125. C. Cuppini, E. Magosso, and M. Ursino, “A neural network model of semantic memory linking feature-based object representation and words,” BioSystems, vol. 96, no. 3, pp. 195–205, 2009. View at Publisher · View at Google Scholar · View at Scopus
  126. R. N. Shepard, “Neural nets for generalization and classification: comment on Staddon and Reid,” Psychological Review, vol. 97, no. 4, pp. 579–580, 1990. View at Publisher · View at Google Scholar · View at Scopus
  127. K. McRae, V. R. de Sa, and M. S. Seidenberg, “On the nature and scope of featural representations of word meaning,” Journal of Experimental Psychology: General, vol. 126, no. 2, pp. 99–130, 1997. View at Publisher · View at Google Scholar · View at Scopus
  128. G. Vigliocco, D. P. Vinson, W. Lewis, and M. F. Garrett, “Representing the meanings of object and action words: the featural and unitary semantic space hypothesis,” Cognitive Psychology, vol. 48, no. 4, pp. 422–488, 2004. View at Publisher · View at Google Scholar · View at Scopus
  129. M. Baroni and A. Lenci, “Concepts and properties in word spaces,” Italian Journal of Linguistics, vol. 20, no. 1, pp. 53–86, 2008. View at Google Scholar
  130. T. N. Rubin, J. A. Willits, B. Kievit-Kylar, and M. N. Jones, “Organizing the space and behavior of semantic models,” in Proceedings of the 35th Annual Conference of the Cognitive Science Society, Cognitive Science Society, Austin, Tex, USA, 2014.
  131. D. J. Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature, vol. 393, no. 6684, pp. 440–442, 1998. View at Publisher · View at Google Scholar · View at Scopus
  132. H. Schütze, Dimensions of Meaning, IEEE Computer Society Press, Washington, DC, USA, 1992.
  133. P. Gärdenfors, The Geometry of Meaning: Semantics Based on Conceptual Spaces, MIT Press, Cambridge, Mass, USA, 2014. View at MathSciNet
  134. W. Lowe, “Towards a theory of semantic space,” in Proceedings of the 23rd Annual Conference of the Cognitive Science Society, J. D. Moore and K. Stenning, Eds., pp. 576–581, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 2001.
  135. D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” The Journal of Machine Learning Research, vol. 3, no. 4-5, pp. 993–1022, 2003. View at Google Scholar · View at Scopus
  136. C. Eliasmith and C. H. Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems, MIT Press, Cambridge, Mass, USA, 2003.
  137. J. G. Sutherland, “A holographic model of memory, learning and expression,” International Journal of Neural Systems, vol. 1, no. 3, pp. 259–267, 1990. View at Publisher · View at Google Scholar
  138. F. Choo and C. Eliasmith, “A spiking neuron model of serial-order recall,” in Proceedings of the 32nd Annual Meeting of the Cognitive Science Society, S. Ohisson and S. Catrambone, Eds., pp. 2188–2193, Cognitive Science Society, Austin, TeX,USA, 2010. View at Google Scholar
  139. C. Eliasmith, T. C. Stewart, X. Choo et al., “A large-scale model of the functioning brain,” Science, vol. 338, no. 6111, pp. 1202–1205, 2012. View at Publisher · View at Google Scholar · View at Scopus
  140. D. Rasmussen and C. Eliasmith, “A neural model of rule generation in inductive reasoning,” Topics in Cognitive Science, vol. 3, no. 1, pp. 140–153, 2011. View at Publisher · View at Google Scholar · View at Scopus
  141. T. C. Stewart, X. Choo, and C. Eliasmith, “Dynamic behaviour of a spiking model of action selection in the basal ganglia,” in Proceedings of the 10th International Conference on Cognitive Modeling, D. D. Salvucci and G. Gunzelmann, Eds., pp. 235–240, Drexel University, Austin, Tex, USA, 2010.
  142. K. K. de Valois, R. L. de Valois, and E. W. Yund, “Responses of striate cortex cells to grating and checkerboard patterns,” Journal of Physiology, vol. 291, no. 1, pp. 483–505, 1979. View at Publisher · View at Google Scholar · View at Scopus
  143. M. S. Gazzaniga, R. B. Ivry, G. R. Mangun, and E. A. Phelps, Cognitive Neuroscience: The Biology of the Mind, W. W. Norton & Company, New York, NY, USA, 2002.
  144. K. H. Pribram, “The primate frontal cortex—executive of the brain,” in Psychophysiology of the Frontal Lobes, K. H. Pribram and A. R. Luria, Eds., pp. 293–314, 1973. View at Google Scholar
  145. P. Földiák and D. Endres, “Sparse coding,” Scholarpedia, vol. 3, no. 1, article 2984, 2008. View at Publisher · View at Google Scholar
  146. B. A. Olshausen and D. J. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V1?” Vision Research, vol. 37, no. 23, pp. 3311–3325, 1997. View at Publisher · View at Google Scholar