Abstract
Network science provides a set of quantitative methods to investigate complex systems, including human cognition. Although cognitive theories in different domains are strongly based on a network perspective, the application of network science methodologies to quantitatively study cognition has so far been limited in scope. This review demonstrates how network science approaches have been applied to the study of human cognition and how network science can uniquely address and provide novel insight on important questions related to the complexity of cognitive systems and the processes that occur within those systems. Drawing on the literature in cognitive network science, with a focus on semantic and lexical networks, we argue three key points. (i) Network science provides a powerful quantitative approach to represent cognitive systems. (ii) The network science approach enables cognitive scientists to achieve a deeper understanding of human cognition by capturing how the structure, i.e., the underlying network, and processes operating on a network structure interact to produce behavioral phenomena. (iii) Network science provides a quantitative framework to model the dynamics of cognitive systems, operationalized as structural changes in cognitive systems on different timescales and resolutions. Finally, we highlight key milestones that the field of cognitive network science needs to achieve as it matures in order to provide continued insights into the nature of cognitive structures and processes.
1. Introduction
Networks are everywhere. The friends you interact with in real life and on social media form your social network. Webpages form a network that you navigate through when you browse the World Wide Web. The same holds for roads, train tracks, or waterways for navigation in the real world. Over the past two decades, an increasing number of studies have applied network science methodologies across diverse scientific fields to study complex systems (e.g., [1–3]). Complex systems involve multiple components that interact with each other to give rise to complex behavior. This includes the human brain and the cognitive processes it gives rise to, such as memory and language (e.g., [4–8]). Network science is based on mathematical graph theory and provides a set of powerful quantitative methods to investigate these systems as networks (e.g., [9]).
In recent years, network science has become a popular tool in the study of structures and dynamics at the neural level of the brain [3, 10]. Despite the rich potential of the methods, research in cognitive phenomena has applied these tools to a lesser extent. This review aims to discuss how network science approaches have been applied to the study of human cognition and how network science can uniquely address and shed novel light on important questions related to cognitive systems and the processes that occur within those systems. In particular, we aim to establish the following three points by an in-depth discussion of the extant literature on cognitive network science. This review is organized into three sections, each of these addressing one of the three issues.
(1) Network Science Provides a Quantitative Approach to Represent Cognitive Systems. One important goal of cognitive science is to model cognitive structures, for instance, semantic memory—our memory for facts or events (e.g., [5, 11, 12]), and the mental lexicon—the part of long-term memory where lexical representations are stored (e.g., [8, 13, 14]). This goal of formalizing cognitive representations is reflected in the diversity of approaches that have been employed including symbolic approaches (e.g., [15]), connectionist or neural network approaches (e.g., [16–18]), and combinations of the two (e.g., [19, 20]). We argue that a network science approach can provide a powerful alternative framework for modeling and quantifying cognitive representations in diverse domains. Network science provides a suite of computational tools that allows the cognitive scientist to explicitly examine the structural properties of cognitive systems—something that can be difficult to achieve with, for instance, connectionist approaches where the structure of a cognitive system is obfuscated within a black box of “hidden” layers [21]. Section 2 focuses on cognitive representations and architectures, demonstrating how network science approaches can be used to represent and describe the structural properties of cognitive systems. This section also introduces the reader to some basic terminology of networks.
(2) Network Science Facilitates a Deeper Understanding of Human Cognition by Allowing the Researcher to Consider How Network Structure and the Processes Operating on the Network Structure Interact to Produce Behavioral Phenomena. Another strength of the network science approach is the ability to not only quantify aspects related to the structure of cognitive systems, but also to model the processes that operate within these systems. For instance, a model of memory retrieval typically requires two core components—a representation of memories and a process to retrieve them. Within a network approach, these components might be modeled, for instance, as a network representation that depicts semantic memory as a network of similar concepts and a random walk process that walks randomly across the semantic network, emitting a sequence of nodes that approximates the outputs of the retrieval process as implemented by the random walk that traverses the underlying network structure. Generally, the joint consideration of structure and process emerges naturally from a network approach and provides a parsimonious account of human behavior and cognition in domains such as semantic memory and lexical retrieval. Section 3 focuses on the processes that occur in cognitive and lexical networks. We highlight how a thorough understanding of cognitive processes requires close consideration of how the structure of the cognitive system interacts with processes to account for complex human behavior.
(3) Network Science Provides a Framework to Model Structural Changes in Cognitive Systems on Multiple Scales. Another area of research that cognitive scientists are deeply involved in concerns the development and decline of cognitive systems [8]. Research in areas such as language acquisition and cognitive aging has revealed that cognitive systems are dynamic and are sensitive to changes in the linguistic environment [22], accumulation of experience over time [23, 24], and deficits or other age-related decline in sensory processing [25]. Dynamic changes may occur over longer timescales, reflecting the long-term accumulation of experiences, and smaller timescales, reflecting the dynamic nature of semantic memory in response to different contexts and experimental tasks [26]. We show how network science methods can provide new ways of quantifying and modeling the dynamics in these areas. Section 4 focuses on the dynamics of cognitive networks and specifically discusses research focusing on the factors that lead to structural changes in the network, and thus in cognition and behavior, on multiple timescales.
Cognitive science has largely employed network science methodologies to study the relationships between words and concepts (e.g., [5, 11, 27, 28]), aside from applications to social relationships (e.g., [29]). While a wide range of cognitive constructs can be represented as a network, the review will primarily focus on research studying memory and language-related phenomena using networks. Nevertheless, we note that network science can be a valuable tool far beyond the study of words and concepts (see Table 1) and encourage researchers to consider how network science methods can be used to address a broad spectrum of research questions in the cognitive sciences.
2. Cognitive Constructs as Networks
Networks are composed of two elements: nodes that represent the conceptual entities of interest (e.g., persons, websites, or words) and edges that represent the relationships among those units (e.g., friendship, hyperlinks, or semantic similarity). While additional aspects can be considered as is done in bipartite and multiplex networks (defined below), identifying these two basic elements in the system of study is sufficient to formalize the system as a network and to employ the powerful tools provided by network science. Network science approaches often capitalize on the fact that relationships between nodes (i.e., edges) can be as important as the nodes themselves, if not more important. A first challenge in studying cognitive systems as networks is to represent these systems in a meaningful way in terms of nodes and edges.
2.1. Network Representations of Cognitive Systems
Cognitive science traditionally has a strong interest in words and concepts as the basis for thought, reasoning, and communication (e.g., [37]). Much research has been dedicated to studying the properties of words, such as their frequency in natural language, their valence, or their concreteness (e.g., [38]). While these efforts have been instrumental for predicting the behavior of human memory and lexical retrieval, researchers have also found that much can be gained by considering the relationships between words within a network representation. Consider, for instance, the free association task that requires participants to freely generate associative responses to a given cue, i.e., responding with the words dog, purr, and kitten to the cue word cat [31, 39, 40]. These associative responses can be represented as a semantic network, for example, by defining nodes as words and edges as the associative strength between words, i.e., the likelihood that one word is named as an association to another word. Representing the data in this way has allowed research to observe that a word’s degree, a popular node metric derived from a network (defined below), predicts how well words can be learned [28, 41].
Associative strength is one of many options to construct networks of words. Edges between words can also be constructed based on the number of shared features (e.g., the concepts banana and cheese are connected as they are both yellow in color; [42], see also [43]), their semantic relations, such as synonymy (e.g., happy and joy share similar meanings), hypernymy (e.g., maple is a tree), and meronymy (e.g., a bird has a beak; see [44]), their phonological similarity (i.e., words that sound similar are connected to each other; [32, 45, 46]), their orthographic similarity (i.e., words that have similar spellings are connected to each other; [47, 48]), their cooccurrences in naturalistic speech [49], language corpora statistics [50], and manually annotated syntactic dependency relationships ([51, 52]; see [53], for a review). Likewise, research has studied different linguistic units other than words by mapping letters [54, 55], syllables or segments [56], or entire documents [57] onto nodes. Furthermore, research has applied network science methods to examine the semantic [58], stylistic [59, 60], typological [61–63], and phonological [45, 64] structure of languages other than English. Figure 1 shows examples of cognitive networks.
The minimum requirement for representing a cognitive system as a network is to identify nodes and edges. However, there is much more information that can be represented. For instance, networks can be specified with multiple types of nodes to distinguish between cues and responses in an associative network, creating a bipartite network [36]. Similarly, networks can be specified with multiple types of edges, for instance, to represent both phonological and semantic similarity between words, creating a multiplex network [65, 66]. Finally, edges can be weighted and directed in order to reflect the strength and directionality of a relationship, respectively. Directionality and strengths have been used to account, for instance, for the fact that a cue word triggers a response at a higher rate than the other way around, such as the case with dog frequently cuing the response bone but bone only infrequently cuing dog, with higher associates to words like skeleton or body instead [31, 36].
The construction of networks leaves much freedom to the researcher and renders it possible to represent a wide variety of cognitive systems as networks. For instance, the emerging area of network psychometrics represents statistical relationships between personality traits or items in a symptom questionnaire as a network, seeking to better understand the causal structure of human personality and psychological disorders (see [67, 68] for reviews). This approach is rapidly being established in personality and clinical research as a fruitful alternative to traditional approaches that use latent variable modeling, which assumes the presence of a latent variable that accounts for psychological and personality disorders, whereas the network approach emphasizes the relationships (i.e., edges) between symptoms and the importance of considering the causal pathways that can lead to the emergence of a disorder (e.g., [69–72]). Moreover, networks have been used to represent the external environment that people are embedded in, such as their social network or the informational space that learners are exposed to. Emerging work is showing that quantifying such external structures could lead to new insights into a number of topics of deep interest to cognitive scientists, including the influence of a speaker’s social network size on language processing [73], language evolution [74, 75], how students represent their conceptual understanding of a body of knowledge [76–78], problem solving and decision making in groups [79, 80], and how learners are able to extract the external structure of the world via statistical learning [81, 82]. See Table 1 for a summary of different types of cognitive networks and relevant cognitive science topics.
In summary, representing cognitive structures in terms of a network offers high degrees of flexibility to researchers investigating various cognitive phenomena (see Table 1). The nodes and edges in any cognitive network should represent theoretically motivated constructs, with nodes depicting an appropriate and relevant scale of representation and edges defining a meaningful relationship between nodes [83]. Choosing a network representation can be likened to choosing a measurement instrument. Different network representations will reveal different aspects of the underlying cognitive system. Ultimately, it is up to the researcher to decide which aspects to place the focus on.
2.2. How Can Network Structures Be Characterized?
A key strength of studying cognitive systems as networks is the accessibility of reliable, well-established quantitative measures and tools, reflecting the long history of graph theory and its mathematical foundations [84], as well as its continual refinement and development (for instance, in the area of multiplex networks, [85]). In this section, we present common measures used to quantify aspects of networks at three scales of structure in network representations: (i) the microscopic structure, i.e., a “node’s eye view” of structural properties of individual nodes and edges, (ii) the mesoscale, involving a subset of nodes and the substructures that they form, and (iii) the macroscale, i.e., a “network’s eye view” summarizing the entire network structure. To highlight how these three different scales provide novel opportunities for the study of cognition, we review measures on each of these scales and how they have been employed to improve our understanding of cognition.
2.2.1. Microscopic Network Measures
At the microscopic level, network analysis examines different properties of nodes and edges, most commonly focusing on quantifying the “importance” of a node in the graph representation via measures of centrality [86–88].
One popular measure of node centrality is the node’s degree, , i.e., the number of edges connected to a node. Nodes of higher degrees are directly connected to a higher number of nodes in the network and can have an important role in, for instance, exchanging information across the network [4]. A node’s degree also defines a node’s “neighborhood size”, a property that has often been used in the context of phonological and orthographic networks of word forms to predict lexical retrieval times, where edges commonly represent an edit distance of one phoneme [89] or one letter [90]. Here, the degree of a node defines the level of similarity of a word form to other word forms.
Another property of nodes derived from its immediate neighborhood is local clustering coefficient, , which characterizes the extent to which the neighbors of a node are interconnected. Specifically, clustering coefficient measures the extent to which a node’s neighbors are also neighbors of each other (similar to the notion of transitivity in social networks, i.e., are your friends also friends of each other; [91]). As shown in Figure 2, it is possible for a word with the same degree to have different values of , reflecting differences in the internal structure of their neighborhoods. Similar to a node’s degree, a node’s clustering coefficient will influence the flow of information through and around a node. Along these lines, recent psycholinguistic research has shown that people are sensitive to nuances in the similarity structure of words as operationalized by . The clustering coefficient of words has small but measurable influences on how people recognize spoken [30] and written words [48, 92], produce speech [93], learn new words [94], and recall words in memory tasks [95].
Both node degree and local clustering coefficient consider only the immediate neighborhood of the node of interest. To characterize the importance of a node beyond its immediate neighborhood, measures such as closeness centrality or PageRank centrality can be used. Closeness centrality is computed as the inverse of the average shortest path length (defined in more detail below) to all other nodes in the network [96]. Closeness centrality of words computed from phonological and orthographic similarity networks has been shown to influence spoken [97] and visual word recognition [48], picture naming performance among people with aphasia [98], and performance in a mental navigation task [99]. PageRank centrality is widely known as an algorithm used to rank websites in Google search results [100]. PageRank centrality can be thought of in terms of a “fluid” that flows throughout the network and pools at the most important nodes. The general idea is that more important nodes (websites) receive more “fluid” (endorsement) from nodes (websites) that are themselves important in a recursive fashion. Although PageRank centrality was developed for the purpose of optimizing web search, Griffiths, Steyvers, and Firl [41] showed that an implementation of the PageRank algorithm on language networks was better able to account for people’s responses in a fluency task as compared to traditional predictors such as word frequency or associate frequency—suggesting strong parallels between the mechanisms underlying successful information retrieval in search engines and in human memory.
Finally, the shortest path, or shortest distance between two nodes, , can be used to reveal something about the relationship of (nonneighboring) nodes rather than the nodes themselves. For instance, path length between two nodes in cognitive and language networks influence the misperception of spoken words [101], judgments of semantic relatedness [27, 102], and picture naming performance in people with aphasia [98].
Many other network measures that we do not discuss here in detail for the sake of brevity have been shown to have measurable effects on human behavior. These measures include assortative mixing by degree [103], key players [104, 105], whether a node resides in the largest connected component of the network or in smaller connected components [106, 107], network connectivity of a node’s broader neighborhood that included its immediate neighbors and nonneighboring words [108], and many others that have yet to be thoroughly explored in the cognitive sciences, such as betweenness centrality (see [109], for a review detailing the influence of various network metrics on language processing, and [110], for a review of various centrality measures).
2.2.2. Mesoscopic Network Measures
At the mesoscopic level, research has focused on network community structure. Communities refer to the grouping of nodes into subnetworks based on their connectivity, i.e., how strongly they are interconnected. To identify communities in networks, existing algorithms aim to maximize connectivity within clusters while minimizing connectivity between clusters [111]. The identification of communities and the nodes they include can provide interesting insights regarding a cognitive system. In the domain of language, for instance, community detection can be used to identify semantic fields or categories [11]. The ability of community detection algorithms to describe a network in terms of a set of communities is typically captured using a measure known as modularity [111, 112]. Investigations at the neural level in the brain [113–115], for instance, have consistently shown how the community structure of neural networks changes with the progression of several different psychopathologies [116, 117].
Recent studies have also highlighted the significance of modularity in cognitive networks in both healthy and clinical populations [46, 118–120]. For example, the semantic network of individuals with high functioning autism (Asperger’s syndrome) has been found to exhibit higher modularity than matched controls, which offers one possible account of their rigidity in processing language [118]. Moreover, high modularity observed in the phonological network was suggested to constrain the spreading of activation in lexical retrieval [46] and higher modularity observed in semantic networks was negatively related to individual differences in creative ability [34, 121].
2.2.3. Macroscopic Network Measures
Measures of the macroscopic structure of networks speak to the overall organization of networks as opposed to node level or community level structures. These measures can reveal emergent properties of a system visible only when considering the network as a whole, which can play an important role in the system’s behavior. Below we describe macroscopic network measures that have been used to study cognitive systems.
Average Node Measures. Networks are regularly characterized with the averages of local, node-based measures described above, such as average degree, average shortest path length, and average (local) clustering coefficient (e.g., [11, 28]). One pattern that frequently emerges in a variety of systems is known as a small world structure, characterized by high local clustering and moderate average shortest path lengths, relative to similarly sized, density matched, randomly drawn networks [91]. This small world property may be important in the domain of cognition for two reasons. First, small world structures have been found to be an almost universal property of real-world networks across diverse domains including biological networks (e.g., [122, 123]), social networks (e.g., [124]), and information networks (e.g., [125]). Second, small world structures may emerge from systematic growth processes that may adapt to environmental constraints to give rise to a beneficial structure. For instance, the small world structure of brain networks has been said to reflect the trade-off between short neuronal distances between brain regions and the costs associated with creating these connections, with the conclusion that small world structure may provide a means to optimize the organizational structure of neurons [113]. The idea of optimized organization in brain networks is related to the idea of network efficiency, , referring to a network’s ability to quickly exchange information (Table 2; [126]).
“Small world-ness” is also a ubiquitous feature of many types of language networks, including semantic networks [28], phonological networks of various languages [32, 45], the orthographic network of English [48], and syntactic networks [52]; although we note that some people debate the usefulness of measuring small world structure as Watts and Strogatz [91] showed that even an extremely structured lattice will exhibit small world structure when a small amount of random rewiring of edges is introduced. Similar to the argument concerning brain networks, small world properties in language networks might arise due to two competing aspects of language learning and use—distinctiveness (e.g., each object having a unique word mapping) and memory constraints (e.g., the easiest language to learn is one where a single word refers to everything). These two competing features of language may result in the emergence of local clusters of similar meaning and form but a low average path length due to the influence of memory constraints resulting in the reuse of words and sublexical segments [50, 127]. Finally, the small world structure in semantic and lexical networks could provide important clues into how the structure of such cognitive systems might be exploited in order to maximize the efficiency of search processes within semantic memory or prevent catastrophic failures in lexical structures [11, 128].
Degree Distribution. The degree distribution, P(k), of a network indicates how many nodes have a given number of connections in the network (i.e., its degree). In many naturally occurring networks many nodes have low degree (few connections) and a few nodes have very high degree (many connections). The degree distribution of some networks is often best approximated by a power law (see Clauset et al. [129] for a counter argument to the idea that power laws accurately capture degree distributions in networks.) such that and is typically referred to as scale-free networks when the exponent, γ, is between 2 and 3 [129, 130]. The term scale-free refers to the fact that the second and higher order moments tend to go to infinity, implying that the degree distribution has infinite variance. Scale-free degree distributions include nodes with degrees much larger than the average degree of the network, which are often referred to as hubs. The scale-free property of networks has been linked to a network’s resilience to random node failure. That is, studies of percolation processes have found that the connectivity in scale-free networks withstand a continued, random deletion of edges longer than networks with other degree distributions [131].
The degree distributions of semantic networks consistently follow an approximate power law distribution, with the exponents of the best fitting power laws converging at ~3 (see Figure 3, left; [28]). This has been demonstrated across semantic networks constructed from free associations [31] and from more complex semantic relationships (e.g., WordNet; [44]), suggesting commonalities in the semantic organization of word meanings ([132]; but see [133]). On the other hand, the degree distributions of phonological networks of various languages appear to be best fit by a truncated power law ([45]; see Figure 3, right), which have implications for candidate models of network growth.
To account for the ubiquity of scale-free degree distributions observed in naturally occurring networks, Barabási and Albert [134] proposed an influential model of network growth known as preferential attachment where, as new nodes are added to the network, these nodes are more likely to be connected to nodes with higher degree (i.e., more connections). Therefore, highly connected words are more likely to acquire new connections, resulting in a “rich-get-richer” effect. Steyvers and Tenenbaum [28] suggested that the growth of semantic networks could have occurred in a similar manner, by conceptualizing preferential attachment as a process of semantic differentiation in which words are likely to be learned if they connect to other words with many varied meanings, increasing the child’s vocabulary and helping the child learn the various meanings of words. The truncated power law of phonological networks may instead suggest that a different underlying process operates on the growth of the phonological network.
2.3. Methodological Tools and Resources for Cognitive Network Analysis
In this section, we showcase a selection of resources that are available for cognitive scientists who are interested in estimating and analyzing cognitive networks. The number of available open source toolboxes to conduct cognitive network analysis is growing rapidly; below we cover a nonexhaustive list of the most relevant resources that enable the analysis, modeling, and visualization of cognitive and language networks.
NetworkToolbox: Methods and Measures for Brain, Cognitive, and Psychometric Network Analysis [135]: this R toolbox comprehensively implements network analysis and graph theory measures used in neuroscience, cognitive science, and psychology. NetworkToolbox includes various filtering methods for psychometric networks and efficiency-cost optimization for brain networks (https://cran.r-project.org/web/packages/NetworkToolbox/index.html).
SemNetCleaner: An Automated Cleaning Tool for Semantic and Linguistic Data [136]: this R package implements several functions that automatize the cleaning, binarizing, and spell-checking of text data. It also removes plurals and continuous strings, converges, and finalizes text data for semantic network analysis (https://cran.r-project.org/web/packages/SemNetCleaner/index.html).
memnet: Network Tools for Memory Research [24]: memnet provides a set of network science tools efficiently implemented for research in human memory. The memnet R package contains methods for inferring networks from verbal fluency data, implementing network growth models and diverse (switcher-) random walk processes, and tools to analyze and visualize networks (https://cran.r-project.org/web/packages/memnet/index.html).
spreadr: Simulating Spreading Activation in a Network [137]: spreadr enables cognitive scientists and psychologists to conduct computer simulations that implement spreading activation in a network representation (https://cran.r-project.org/web/packages/spreadr/index.html).
SNAFU: The Semantic Network and Fluency Utility [33]: SNAFU provides psychologists with the tools to generate semantic networks from category/verbal fluency data (e.g., name as many as animals as possible in 1 minute) and to compare the semantic networks of different groups or individuals (https://alab.psych.wisc.edu/projects/2017/12/08/snafu.html).
The Aging Lexicon Project [8]: the Aging Lexicon Project contains resources related to the study of the mental lexicon across the lifespan, including resources to measure and represent the linguistic environment, tools to quantify and model the mental lexicon over the lifespan, as well as a comprehensive list of open-access linguistic norms, natural language corpora, and behavioral megastudy data (https://aginglexicon.github.io/).
The Brain Connectivity Toolbox [138]: the brain connectivity toolbox is a comprehensive toolbox of MATLAB scripts, dedicated to analyzing networks. While several of the scripts compiled under this toolbox were primarily developed to analyze brain networks, they are well-suited to analyze cognitive networks as well (https://sites.google.com/site/bctnet/).
Psychometric network analysis: while outside the scope of this review, it is important to note that several R toolboxes have been developed to analyze psychometric networks (i.e., the analysis of psychometric questionnaires as networks to study psychopathology and personality). Such toolboxes include qgraph (https://cran.r-project.org/web/packages/qgraph/index.html), bootnet (https://cran.r-project.org/web/packages/bootnet/index.html), and mlVAR (https://cran.r-project.org/web/packages/mlVAR/index.html). For further detail, the reader is referred to Epskamp, Borsboom, and Fried [139].
2.4. Summary
This section provided an overview of various network measures at the micro-, meso-, and macrolevels and highlighted cases where these network measures were predictive of human behavior or provided novel insight into human cognition, predominantly drawn from the domain of language processing and semantic memory. Furthermore, we summarize a noncomprehensive list of main toolboxes that can allow the cognitive scientistto estimate and analyze cognitive networks.
3. Processes in Cognitive Networks
Now we turn to models that may account for processes that occur in the kinds of cognitive networks discussed above. In this section we begin with an overview of the classic spreading activation theory in cognitive psychology and discuss extensions inspired by random walk models that are popular in network science research. We then discuss how these processes can account for behavioral findings implemented in a network representation and provide insight into latent cognitive processes from various domains in human cognition, including lexical retrieval, creativity, and cognitive search and navigation.
3.1. Cognitive Processes in Networks
A key advantage of the network science approach relevant for cognitive science is that it is possible to formalize processes that operate on the structure of a network. One of the earliest attempts to conceptualize cognitive processes on a network of nodes and their relationships focused on human semantic memory to explain an intricate taxonomy of human reasoning [140, 141]. When asked to verify statements such as a robin is an animal or a robin is a bird, participants typically take longer to verify the former statement as compared to the latter [142]. To account for this response time discrepancy, human semantic memory was assumed to exhibit a tree-like network organization, in which nodes represented concepts and edges represented whether words were elements of a higher order concept. In such a network, the triad, robin, bird, and animal, forms a line with robin being connected to bird and bird being connected to animal. Any reasoning process seeking to identify whether two concepts are, at least, indirectly connected via “is-contained-in” relationships would consequently have to traverse two edges to verify the first statement (i.e., the edges depicting a robin is a bird and birds are animals) but only one edge to verify the second (i.e., the edge that depicts a robin is a bird). This model of network processing represents one of the classic examples explaining behavioral phenomena as a combination of an underlying network representation and a process operating on the network.
Later, Collins and Loftus [27] generalized the insight that network structure in tandem with a retrieval process can account for human behavior and developed the theory of spreading activation in semantic memory (see also [143]). The key contribution of this theory was to make explicit the process operating on a semantic network when executing a cognitive task and to link the outputs of that process to participant performance and response times. Specifically, in the spreading activation theory it was assumed that reading or thinking about a concept would activate the concept and that this activation would spread to neighboring concepts in the network, priming related concepts and making them easier to retrieve. The process of spreading activation proposed by Collins and Loftus [27] implicitly assumes the presence of some cognitive resource (i.e., activation) that can be assigned to specific nodes, spread among connected nodes in a predefined network, and decay over time (as formally implemented in [143, 144]; see also [137]). This spread of activation quickly decays over time and distance in the semantic network [145]. Overall, the success of the spreading activation account demonstrates the significance of formalizing a process that captures search within a cognitive representation.
Network science provides new ways of expanding the original conceptualization of spreading activation by Collins and Loftus [27] by formalizing diffusion models over network representations. Diffusion processes on a network have been independently used to extensively study and predict epidemics of disease spread and of contagious ideas in a population (e.g., [146, 147]). The models of diffusion developed in these domains can be similarly used and adapted to study how information or activation “spreads” in a cognitive network. Although different implementations of network diffusion models exist, one core idea that illustrates the power of such network process models is the notion of a random walk on a network. A random walk model is a naïve search process that moves from node to node as function of a set of transition probabilities specifying the probability of moving from one node to any of its directly connected neighboring nodes. Note that when combined with a decay parameter, a random walk model becomes similar to a model of spreading activation [6]. However, despite their similarity, there are noteworthy differences in the implementations and goals of spreading activation in the tradition of Collin and Loftus [27] and random walk models by network scientists. Specifically, random walk models produce individual paths taken by the walk (e.g., an ordered list of words; see Figure 4), whereas a process of spreading activation produces a pattern of activation levels among nodes in the network and how they change over time. In contrast to random walks, spreading activation represents the aggregate, long-run behavior arising from an underlying basic process which could be modeled as a random walk. Diffusion models provide a means of exploring existing cognitive theories and insights such as spreading activation and have the ability to account for individual decision processes as well as describe the functioning of various cognitive systems. In recent years, various empirical studies have demonstrated how memory search can be modeled as a random walk process over semantic memory and have shown predictive power in accounting for human behavior [33, 148–151].
In the rest of this section, we showcase in greater detail recent empirical work showing how behavioral data from experiments can be used to provide a deeper understanding of the interaction of structure and processes that occur in cognitive and lexical networks. Each subsequent section focuses on a different cognitive domain—specifically, lexical retrieval, creative processes, and search and navigation in cognitive networks—that draw on either the process of random walks or the theoretical construct of spreading activation to account for relevant behavioral findings. The final section focuses on a key debate in the cognitive science regarding the complexities of disentangling structure and process in the domain of cognitive search and attempts to show how network science methods can contribute to this important debate.
3.2. Lexical Retrieval
The theory of spreading activation as proposed by Collins and Loftus [27] offers a blueprint for a mechanistic explanation to several psycholinguistic studies examining similarity effects on language processing. As mentioned in Section 2.2.1 (Microscopic Network Measures), network measures such as degree, local clustering coefficient, closeness centrality, and many others can be calculated for individual words. Critically these features may determine how activation spreads throughout a network and, thus, influences behavioral performance in psycholinguistic and memory tasks. Studies that have investigated such behavioral tasks have found that words with more clustered neighborhoods (words with high values) are more slowly responded to in spoken [30] and visual word recognition tasks [48, 92], and more slowly produced [93] than words in less clustered neighborhoods. In these psycholinguistic tasks, participants typically are presented with, and respond to, words that have either high or low values on a particular network measure (e.g., higher or lower clustering coefficients, or higher or lower closeness centralities) in order to detect its influence on the efficiency of lexical retrieval that is typically indicated by faster reaction times and higher accuracy. Examples of such psycholinguistic tasks include lexical decision, where participants decide whether the presented stimulus represented a real word or a nonsense word, and speeded naming, where participants read out loud the presented word. Fast reaction times and high accuracy rates on these tasks indicate greater efficiency of lexical retrieval (i.e., a processing benefit or advantage). To account for clustering coefficient effects in word recognition of words versus nonwords, Chan and Vitevitch [30] provide a theoretical account that assumes a spreading activation process operating on a phonological network, where a word’s structural characteristics affect how activation spreads through the network. Words with low clustering coefficients are hypothesized to receive more activation (and hence a processing advantage) from its neighbors as compared to those with high clustering coefficients because activation in the latter case is more likely to be shared among neighbors resulting in lower levels of activation for the target word. This account of a possible mechanism for word confusability within a theoretical spreading activation framework was further formalized and validated in computer simulations conducted by Vitevitch, Ercal and Adagarla ([152]; see also [137]).
Although spreading activation process as discussed above is mainly related to the local structural properties of words, the spreading activation process can also be used to account for findings showing that global structural aspects of words in the network influence lexical retrieval. Such findings suggest that closeness centrality of words influence spoken and visual word recognition [48, 97], that the size of the component that words reside in (i.e., whether the target word is in the largest connected component or an isolate) influences spoken word recognition, serial recall, and picture naming [106, 107], and that assortative mixing by degree, the tendency for nodes with similar degrees to be connected to each other, influences failures in lexical retrieval [103]. Among these results, the finding of a processing advantage for high closeness centrality words in spoken word recognition [97] is especially interesting as this finding goes against the general finding that greater similarity does not tend to help recognition (e.g., degree and local clustering effects result in poorer recognition; [30, 89]). To account for the processing advantage for high closeness centrality words, however, we can again draw on spreading activation theory. Goldstein and Vitevitch [97] suggest that, as a result of the activation of lexical representations spreading over time, high closeness centrality words will accumulate more activation than less central words due to their topologically advantageous location in the network. This higher accumulation of long-term activation provides an explanation for the processing benefit of words with high closeness centrality that counteract the more immediate, negative effects of neighborhood size and clustering—namely, that words that reside in structurally important locations in the network are suggested to have higher resting activation levels due to residual activation from other words in the network.
3.3. Creative Processes
Another cognitive domain where spreading activation has proven to be a powerful framework is creativity. Theories of the creative process attribute a key role to spreading activation over memory to account for individual differences in creativity [153–155]. Such theories propose that creativity is related to the ability to combine together weakly related (i.e., distant) concepts into new concepts that are both novel and appropriate. On average, the further away the combined components are in a semantic network, the more novel the new concept will be [154]. A number of studies have applied network science methodologies to examine the idea that creativity involves the combination of distant concepts. These studies have shown how differences in semantic memory structure relate to individual differences in creativity, both at the group level (Figure 5; [34]) and at the individual level [121]. In group-level studies, semantic networks of low and high creative individuals were measured using a continuous free association task, which required participants to generate as many associations they can think of in one minute to a list of cue words. These responses were then used to determine the overlap of associations for all pairs of cue words (cf. [27]), which gave rise to separate semantic networks of low and high creative individuals [34, 40]. Individual-level networks were measured directly using relatedness judgments for all possible pairs of a list of target words. This approach allowed linking the resulting semantic networks to individual-level measures of creative ability [121].
Focusing on the macro- and mesolevel structure, these studies found that higher levels of creativity were associated with semantic networks that exhibit higher clustering coefficients, smaller average shortest path lengths, and lower modularity in community structure. The authors argue that the structural macroscopic properties of the semantic network of higher creative individuals facilitate the spread of activation across network communities in their semantic memory, leading to the generation of more novel ideas as activation reaches nodes in the network that are further apart from each other. Indeed, simulated random walks over the semantic networks of individuals of low and high creative ability revealed that the simulated search processes over the semantic network of highly creative individuals reached distant nodes and nodes with weaker relations ([156]; see [157], for further empirical support). These findings demonstrate how network science can quantitatively investigate how the structure of a cognitive system constrains processing on the representation [153, 158].
3.4. Mental Navigation and Cognitive Search
A large body of research has shown that people search their internal cognitive spaces in similar ways as they would in an external, physical space (e.g., [159–161]), implying that spatial and cognitive search processes have similar evolutionary roots [162]. Representing semantic memory as networks essentially produces a map of semantic space that allows one to mathematically trace search processes as paths in a network and make predictions about how structural properties of such network maps influence cognitive search behavior.
In one study, Iyengar et al. [99] had participants play a word-morph game that required them to transform one word into another by only changing one letter at the time (e.g., ball- -tall- -tale- -take). This word game is analogous to finding a path in a lexical network of word forms. The results showed that over time participants began to actively utilize network landmarks, hub words, and words of high closeness centrality, to drastically improve their performance both in terms of time and minimal number of word forms used. In another study, individuals were able to successfully identify the shortest path between a start word to a target word within a predefined semantic network based on free association norms [150, 163]. This result indicates that people are able to actively exploit both global and local structure of semantic memory in order to estimate the distance between two words and to use that information to guide their search (see also, [164–166]).
Similar conclusions were drawn from more open-ended search tasks requiring individuals to actively explore their mental representations, such as category and letter fluency tasks. In verbal fluency tasks, individuals are required to retrieve from memory, in a fixed amount of time, as many elements belonging to a semantic category (e.g., name all the animals that you can think of) or words beginning with a specific letter (e.g., name all the words beginning with the letter 'S') as they can [167, 168]. Previous work has used behavioral data obtained from the fluency task to infer the structure of semantic networks via various estimation or inferential techniques (e.g., [33, 128, 169]) and to analyze and compare macro- and/or mesolevel network structure. This has led to important insights into the structural differences in semantic networks between younger and older adults [24, 170], less creative and highly creative individuals [118, 171], the first and second languages of bilinguals [172], and individuals with low or high openness to experience, a personality trait related to intellectual curiosity and an active imagination [173].
The sequences that individuals produce in verbal fluency tasks (in particular when listing items from a semantic category) can, however, also be used to study the search processes involved when individuals forage their mental representations. The finding that items with high semantic relatedness and many shared features tend to cluster together (e.g., [174]) was proposed to be related to an active search process that dynamically switches between retrieval cues [160, 161, 175, 176]. Other work has shown that micro (node-level) information such as PageRank centrality was most predictive of word recall [41] and that process models such as random walks could account for verbal fluency data ([33, 148]; Goñi et al., 2010). A prevailing question in this line of research is whether such search processes are better accounted for by a foraging process or a random walk remains open to debate and research ([177]; see next section).
Collectively, the studies discussed above provide important insights into how people navigate and retrieve information from their semantic and linguistic networks. Strong behavioral evidence and model-based approaches show that people are able to successfully search their mental representations in flexible ways to accomplish a variety of cognitive tasks, such as converting a word to another word, searching the semantic space as quickly and efficiently as possible, or generating creative ideas. The ability of network processes to capture behavioral differences suggests that the network science perspective provides ways to formulate testable hypotheses with regard to how individuals are accessing and navigating these cognitive structures.
3.5. Disentangling Structure and Process
A critical and open debate in cognitive network science is whether insights into the network structure and representations can be obtained independently from the retrieval processes operating upon it. This debate arises from the fact that both the underlying structure and the process operating on the representation are flexible enough to produce a wide array of behavior. Consider, for example, a verbal fluency task on the country category (i.e., name as many countries as you can think of), where participants can use various retrieval cues to help them complete the task. For a participant who uses the geographic relationships between countries as a retrieval cue, France and Spain would be very close, whereas France and French Polynesia would be very distant. On the other hand, for a participant who uses the phonological (sound-based) relationships among country names as a retrieval cue, France and Spain are now distant, whereas France and French Polynesia would be close as they share the same first sound. Research has found that individuals can flexibly switch between such retrieval strategies, essentially creating “wormholes” in memory, where shortcuts are (momentarily) created in the memory space to connect previously distant concepts ([161]; see also, [178]).
The indeterminacy problem, which is associated with the notion that both representation and retrieval processes can be powerful explanations of human behavior, has been the focus of a recent debate on models of the verbal fluency task. In modeling verbal fluency using a semantic space extracted from a text corpus, Hills et al. found evidence for an active search process that dynamically switches between subcategories of the semantic space [160, 179]. Shortly after, Abbott, Austerweil, and Griffiths [148] argued that a simple random walk model operating on a semantic network constructed from free associations—a model that does not require a switching process—is equally plausible as a mechanism of search in verbal fluency tasks, suggesting that a simpler model could reproduce the original results found by Hills, Jones, and Todd [160]. However, Abbott et al. evaluated the verbal fluency search on a network estimated from free association data, which—as subsequently argued by Jones, Hills, and Todd [180]—may already contain traces of the underlying search processes in semantic memory, rendering it unnecessary to account for such traces using an elaborate search model.
There have been a few attempts to disentangle process and structure (e.g., [177, 181]). One approach to address this issue is by developing research designs that directly compare representation-based hypotheses against process-based hypotheses on the same type of behavioral data. One such attempt was recently conducted by Kenett et al. [171], who examined the relationships between semantic network structure, creative ability, and intelligence in a large sample of individuals. The sample of participants were divided according to two dimensions – low/high creativity and low/high intelligence, and the animal category networks of all groups were estimated and compared. The authors found that creative ability and intelligence were associated with different structural aspects of the semantic network as estimated from the verbal fluency task. Specifically, intelligence was related to higher average shortest path length and modularity, whereas creativity was related to higher “small worldness” properties of the semantic network. Taking intelligence as a proxy for the contribution of cognitive control processes such as attention and working memory, and creativity as an emergent property of the underlying network structure, these results suggest different contributions for process and representations and demonstrate one possible path for disentangling the two.
The bottom line of this open debate of the influence of structure and process on cognitive representations is that unless one of the two is clearly identified, it is difficult, if not impossible, to make strong inferences from data about the other. Currently, a growing body of work focuses on examining the reliability and reproducibility of estimating networks in both cognitive and psychological networks [24, 33, 69, 170, 173] and this task of network estimation remains an important challenge for network science approaches to understand the details of a given cognitive system. Another area of avid development and a promising route to addressing the indeterminacy problem is the employment of plausible learning models to learn a semantic structure from digitized text and images rather than responses in behavioral tasks. Structure created in this way will contain little trace of the processes operating on it, and may allow for cleaner separation of the contribution of structure and process [180]. Moreover, many of these learning models, such as the BEAGLE model [182], process the environmental input incrementally, principally allowing for the modeling of developmental changes in the network structure.
3.6. Summary
In this section, we discussed recent empirical work that provided a deeper understanding of the interaction of structure and processes that occur in cognitive and lexical networks. We focused on the cognitive domains of lexical retrieval, cognitive search, and creativity to illustrate how the process of random walks or spreading activation can be implemented in a cognitive network representation to account for a variety of behavioral phenomena and offer novel insights and testable predictions of how individuals might be accomplishing these tasks. These studies not only emphasize the importance of considering how the structure of the underlying network interacts with processes operating in it, but also the complexities of disentangling structure and processes, particularly in the domain of cognitive search. We conclude that the consideration of structure and process emerges naturally from a network science approach by compelling researchers to explicitly define and model the relationship between structure and process in order to account for human behavior and cognition. Formalizing the relationship between process and structure enriches our theoretical understanding of the interplay between cognitive processes and cognitive structures in various domains.
4. Network Dynamics across Multiple Timescales
Conceptualizing cognitive systems in terms of a network representation not only motivates cognitive scientists to think more explicitly about the structure of cognitive systems (Section 2: Network Representations of Cognitive Systems) and how cognition might be captured by processes operating on a network (Section 3: Processes in Cognitive Networks), but also stimulates the question of how a particular structure arose and how this structure develops and changes across time.
In this section, we posit that semantic and lexical networks are inherently dynamic—the structure of such cognitive systems changes at multiple timescales and in response to (i) linguistic input and experiences that reflect the long-term accumulation of knowledge and (ii) exposure to experimental tasks and manipulations that trigger more immediate functional changes in semantic memory [26, 183]. The first part of this section focuses on developing semantic networks to capture the process of language acquisition. The second part focuses on semantic networks of older adults. Finally, the third part focuses on individual differences in semantic networks. Although it is possible to examine the network dynamics of the language system itself, i.e., how the structure of language has evolved over time (see [184, 185], for examples), we focus our discussion of network dynamics on cognitive network structures that are relevant for semantic memory and lexical access.
4.1. Developmental Networks
Rather than focusing on a single snapshot of a network in time, some recent investigations include a temporal dimension to quantify and elucidate how network representations of individuals change across the lifespan. For instance, Hills et al. [35, 186] used normative data from the MacArthur-Bates Communicative Development Inventory (CDI; [187]), a vocabulary checklist completed by parents to indicate the words produced by their child across development, to empirically study normative language development using networks. These data can be used to create semantic networks that develop over the course of language acquisition by placing edges between words that an average child knows at a given time point. Here edges can represent semantic associations that are based on adult free association norms (e.g., [31]) or cooccurrences in child-directed speech [188]. Semantic networks created in this way conceptualize a normative child’s semantic knowledge at various time points during development (see [188–190], for modeling of individual children’s semantic network acquisition). To capture development over time, Hills and colleagues performed a statistical comparison of three different models of network growth (see Figure 6): preferential attachment, preferential acquisition, and lure of the associates. In the spirit of Barabási & Albert’s original model [134], the preferential attachment model predicts that words that connect to well-connected words already known by the (normative) child will be learned earlier. The preferential acquisition model predicts that words are learned earlier if they are highly connected in the learning environment as approximated by the full (adult) semantic network. Finally, the lure of associates model predicts that new words are more likely to be learned if they result in the addition of more connections to the network of words already known by the child.
Comparing these models, Hills et al. found that the preferential attachment model was in fact not a good fit to the normative CDI data and that the preferential acquisition model was able to best account for vocabulary growth in early semantic networks [35, 186, 191]. These modeling results highlight that the learning environment plays an important role in language acquisition—words that occur in many different contexts (and hence are well-connected) in the learning environment are more likely to be acquired before less connected words in the learning environment. Additionally, these results also demonstrate that growth models other than preferential attachment (and other related variations) can lead to scale-free degree distributions of semantic networks.
The growth mechanisms underlying semantic and language networks remain, however, imperfectly understood (for an overview, see [192]). For instance, Hills et al. [186] found that the preferential acquisition model accounted for growth of semantic networks with edges constructed from free associations, but not for semantic networks with edges constructed from shared features, suggesting that different growth processes may govern different aspects of language. In phonological networks, psycholinguistic evidence seems to support the lure of associates model [193, 194], although this model remains to be empirically tested against other growth models such as preferential attachment and preferential acquisition. The difference between semantic and phonological networks could provide unique insight into the acquisition process of children—for example, phonological information may be more constrained by phonological form characteristics of the child’s current vocabulary, whereas semantic knowledge may be readily observed in the physical environment, accounting for the difference in the contribution of the child’s current vocabulary in phonological and semantic domains (e.g., see [66]).
Studies have recently pursued new approaches by examining language acquisition in terms of feature networks [195, 196], or multiplex networks representing both semantic and phonological information [65, 66]. One particularly informative approach has been to study atypical acquisition processes. Using the previously described approach of converting vocabulary checklist data into semantic networks and then further analyzing its network structure, Beckage, Smith, and Hills [188] examined the semantic networks of children who were classified as late talkers and found that the semantic networks of late talkers had, on average, higher average path length and lower clustering as compared to the semantic networks of typically developing children, even after controlling for differences in network size or the age of the child. These differences between typically developing children and those with risk of language impairment suggested a maladaptive tendency in late talkers to acquire “odd” words that were less connected in the semantic network (see also [49]). Studying the structure of semantic networks of children with cochlear implants, Kenett et al. [169] also found differences in the semantic network of children with cochlear implants as compared to the semantic network of typically developing children. Specifically, they observed shorter average path lengths for children with cochlear implants relative to typically developing children, suggesting an underdevelopment of the semantic network due to impoverished input.
4.2. Aging Networks
The development of semantic networks does not halt with onset of adulthood (see [8], for a review). Language learning and change continues throughout the lifespan. Wulff and colleagues [24, 170] examined the semantic networks of younger and older adults (as estimated from semantic fluency data and similarity ratings obtained from both populations) and found that both the aggregate and individual older adults’ networks exhibited lower entropy in the degree distribution, larger average shortest path lengths, and smaller clustering coefficients relative to the aggregate and individual networks of younger adults. Using free associations obtained from a cross-sectional sample across the lifespan to estimate semantic networks for groups of young, middle-aged, and older adults, Dubossarsky, De Deyne, and Hills [36] similarly found that structure of the semantic network in early life reverses, in parts, in later life (see Figure 7; for similar results see [197]). Behavioral research on cognitive aging usually finds that older adults take more time and perform worse on variety of cognitive tasks involving memory, concentration, and reasoning than younger adults, which is commonly attributed to cognitive slowing [198, 199]. However, the findings of Dubossarsky et al. [36] and Wulff et al. [170] fuel an interesting hypothesis regarding age-related decline in cognitive function. Based on the observed differences in the structure of semantic networks, changes in the underlying representation of knowledge might be held partly accountable for behavioral changes in older adulthood [8, 23, 24, 200]. Older adults have access to a larger vocabulary, implying a larger network representation with more nodes and edges. Their larger network may exact higher search costs in accessing and navigating the representation which may account for the observed cognitive slowing. Assuming a process akin to spreading activation, it is further conceivable that other changes in structure affecting, for instance, the average path length or clustering coefficient, of older adults’ networks may play an additional role. Thus, in the case of age-related cognitive decline, the application of a network approach has led to a promising rival explanation to established theories, paving the way for a new perspective on an important issue considering our aging society.
In light of the growing importance of age-related degenerative diseases, such as dementia and Alzheimer’s disease, it is especially worthwhile to study models that may capture possible mechanisms underlying these changes in behavior and provide a link to memory representations. To understand the cognitive mechanisms underlying pathological decline, studies have focused on comparing healthy individuals to clinical patients. In one study, Borge-Holthoefer, Moreno, and Arenas [201] conducted simulations based on a network model of language degradation in order to account for hyperpriming among people with Alzheimer’s disease. Hyperpriming refers to increased priming effects observed in patients with Alzheimer’s as compared to healthy controls in naming and lexical decision tasks [202], despite the fact that Alzheimer’s patients have more difficulties in other memory related tasks. Their simulations showed that a process of edge degradation provided a possible mechanism for why individuals with Alzheimer’s disease show evidence of hyperpriming. Specifically, the effect appeared to be driven by the fact that weak (distinctive) associations were lost earlier than stronger (common) associations, leading to a loss of distinctiveness between primes and targets and an increase in priming effects for related words. This result suggests that changes in aging semantic networks are not simply the “inverse” of early development.
4.3. Individual Differences in Networks
The structural differences observed in semantic networks of younger and older adults, as discussed above, indicate that mental representations are shaped, to some extent, by the specific environmental input that individuals are exposed to [8]. This has broad implications beyond research on development and aging and specifically in the domain of personality and individual differences. If we accept the premise that the experiences of individuals can influence their mental representations, this naturally provides a plausible mechanistic explanation that could help us understand individual differences observed in the structure of mental representations—differences that reflect the accumulation of experiences over an individual’s life. In line with this idea, research has demonstrated links between semantic network structure and personality. Recent work by Christensen, Kenett, Cotter, Beaty, and Silvia [173] found that the semantic networks of people with higher openness to experience (where semantic networks were inferred from verbal fluency responses and levels of openness were measured using an independently administered personality questionnaire) were more interconnected and better organized as compared to the semantic networks of people with lower openness to experience. One possible explanation for this result is that individuals who are open to experiences are drawn to, and subsequently exposed to, more diverse environments, which may encourage the formation of diverse associations and may thus account for the higher interconnectivity observed in their semantic networks.
The link between experience and the underlying network structure may also have implications for other domains, such as creativity and problem solving. As discussed earlier, higher creative people exhibit less modular semantic networks than lower creative people [34, 153], which is accompanied by a reduction in path length and may facilitate insight problem solving [203]. These results suggest a codependent relationship between personality and mental representations, where individuals with different personalities seek out different experiences, which shapes their mental representations and, in turn, their behavior and personality.
To date, almost all cognitive network studies have estimated cognitive networks of groups. To truly examine individual differences in cognitive networks and how they may relate to other cognitive and psychological constructs, methods must be developed to estimate cognitive networks at the level of the individual. Only a few studies have attempted to estimate an individual’s semantic network. Morais, Olsson, and Schooler [133] collected free associations from participants based on an associative “snow-ball” sampling approach, where participants provided associations to a set of “seed” cue words in the first iteration, and in the second iteration, associations to their own associative responses in the previous iteration, and so on. These data were used to estimate semantic networks of each individual. Austerweil and colleagues [33, 204] developed a computational approach to estimate individual semantic networks from semantic fluency data, based on a censored random walk model of memory retrieval (see also [24]).
Other studies estimated individual semantic networks based on semantic relatedness ratings [24, 121] and related the structural properties of these individual semantic networks to individual differences with respect to the individual’s age ([24]; see Section 4.2), or the individual’s creative ability and intelligence measures [121]. The latter study replicated the group-based network analysis of Kenett et al. [34] described above, by finding a positive correlation with CC, a negative correlation with ASPL, and a trending negative correlation with Q and creative abilities. Furthermore, the authors showed how the ASPL of each participant’s network and their measure of fluid intelligence predicted creative thinking [121], providing further evidence demonstrating how process and structure of memory might be independently measured.
Although additional methodological development is required, these methods provide researchers with the means to estimate individual cognitive networks. Current theories of semantic memory view semantic memory as a dynamic system that is contingent on different contexts (e.g., environment, age) and individual differences (e.g., creativity, personality traits), with short- and long-term effects on semantic memory reflecting the cognitive processes operating on semantic representations [8, 26, 183]. Therefore, quantifying the properties of cognitive systems, such as semantic memory, at the individual level can greatly advance our understanding of their complex structure and dynamics.
4.4. Summary
The tools of network science have revealed interesting differences in the network structure of various populations (e.g., younger vs. older adults, healthy vs. clinical populations, individuals with low openness to experience vs. individuals with high openness to experience). Such findings support the idea that network representations and processes are inherently dynamic, compelling cognitive scientists to formalize theoretical and algorithmic explanations for how these differences emerge in the first place. By combining process or growth models provided by network science with empirical data from the cognitive sciences, researchers can formalize relationships between (network) representation and processes that operate within the representation and construct models that take into account the contributions of external factors, such as the linguistic environment of young language learners, transient changes that are due to creative abilities, or the accumulative effects of semantic knowledge acquired in a person’s lifetime. The framework of process and structure can offer explanations and predictions of changes in the structure of cognitive representation, which in turn affects our understanding of language and cognitive processes in a continual, interacting cycle.
5. Summary and Conclusions
In this review, we demonstrate the usefulness of the network science approach to the study of cognition in at least three ways.
(1) Network Science Provides a Quantitative Approach to Represent Cognitive Systems. To demonstrate the quantitative power of network science in describing cognitive systems, Section 2 discussed how networks can represent a variety of cognitive systems, including language, semantic memory, personality traits, and the language environment of individuals (see Table 1). Furthermore, we highlighted a host of network measures that are available to the researcher when he or she commits to the theoretical decision of representing the cognitive system of interest as a network. We reviewed previous research using these tools to characterize the structure and behavior of networks on the micro-, meso-, and macroscopic levels in order to derive novel insights.
(2) Network Science Facilitates a Deeper Understanding of Human Cognition by Allowing Researchers to Consider How Network Structure and the Processes Operating on the Network Structure Interact to Produce Behavioral Phenomena. Section 3 focused on processes operating on networks. Network representations of cognitive systems, particularly in the area of language and semantic memory, are often used to represent a latent mental structure that requires the assumption of some process to link the network to observable behavior. Adopting a network science approach naturally compels researchers to consider the interaction between structure and process, which has been especially useful in the three research domains discussed in Section 3 (lexical retrieval, creativity, and cognitive search). Finally, we briefly discuss the difficulties in dissociating structure and process, particularly as it relates to the modeling of behavioral outputs in retrieval tasks from semantic memory and suggest ways in which network science methods can enrich the investigation of such cognitive phenomena.
(3) Network Science Provides a Framework to Model Structural Changes in Cognitive Systems at Multiple Timescales. In Section 4, we discussed how network science can be used to study the development of cognitive systems, enabling a better understanding of cognition at both the early and late stages of human life, as well as structural changes that occur at more immediate timescales, as related to higher order cognitive processes such as creative insight and problem solving. Such cognitive systems can be modeled as a dynamic network representation that changes in response to the accumulation of experiences and linguistic input. The research discussed in this section demonstrates how network science approaches can be used to quantify structural changes and the dynamics of cognitive systems across different timescales.
5.1. Future Directions
It is clear from this review that network science approaches have contributed much to the study of human cognition. Indeed, this is reflected in the recent growth in methodological and computational tools designed specifically for the cognitive scientist to model, analyze, and visualize cognitive and language networks (see Section 2.3: Methodological Tools and Resources for Cognitive Network Analysis, for a compilation of methodological tools commonly used in the field of cognitive network science and by many of the studies described in this review). But it is important to emphasize that cognitive network science is a relatively young field and many methodological and theoretical challenges remain to be addressed. For example, as previously discussed in Section 3.5, to what extent can specific aspects of the network structure and the processes operating in the network be disentangled? Below we briefly highlight three milestones that cognitive network science needs to achieve in order to become a mature research paradigm in the cognitive sciences and in the network science community.
One critical milestone needed in cognitive network science is the development of inferential methodologies to analyze empirical networks—that is, networks that are inferred from behavioral data. Currently, statistical models that allow for hypothesis testing when comparing empirical networks remain a major challenge. This challenge is mainly due to difficulties in estimating or collecting a large sample of empirical networks and the lack of accessible statistical methods to test differences in observed networks [205]. In these cases, bootstrapping methods over comparable networks might be a solution [206]. Similarly, issues in how to minimize spurious connections in psychological and cognitive networks are currently debated and require further methodological development [69].
Another crucial milestone is the development of methods to represent psychological and cognitive networks at the individual level (see Section 4.3). Psychological and cognitive constructs vary across individuals and aggregating across participants in group-based cognitive network analysis may conceal nuanced differences across individuals. While there has been some attempts at representing individual semantic networks [24, 33, 121, 133, 204], developing a reliable and easy-to-apply methodology to represent an individual’s semantic network will allow researchers to design studies that relate individual representations to other cognitive and neural measures.
Finally, a third milestone is the development of new network science methodologies to quantitatively study specific theoretical issues across different cognitive domains. Two such examples were briefly discussed in the review: the application of multiplex network analysis to examine how different cognitive domains interact (e.g., [65, 66, 207]) and the application of percolation theory to study cognitive phenomena such as memory decline or flexibility of thought (e.g., [119, 201]). Cognitive network science could also benefit from adopting and contributing to state-of-the-art network methodologies used to study neural systems and brain dynamics, such as network dynamic analysis [208] and network control theory [209–211]. Network dynamic analysis examines time varying community assignments over brain functional connectivity networks and has attributed state flexibility—variation of assignment of a brain region to a specific community across time—to capacities such as motor-skill learning and language comprehension [208]. Network control theory quantifies the extent that different nodes in a network drive the dynamics over the network. Recent studies have applied network control theory to the analysis of white-matter connectivity networks to examine the roles of different brain regions in driving neural dynamics [209–211]. Importing such state-of-the-art methods to the cognitive domain could greatly advance the study of dynamics in cognitive networks.
5.2. Cognitive Network Science: A New Frontier
The aim of this review was to highlight and emphasize the feasibility and significance of applying network science methodologies to study cognition. Such applications allow the quantification of theoretical cognitive constructs and direct examination of cognitive theories in domains such as language and memory. The cognitive network framework also provides a means to model and formalize theoretical and mathematical descriptions of dynamics operating in cognitive systems. The work conducted in this new field of cognitive network science has already provided many novel insights on cognitive issues such as lexical retrieval, language acquisition, memory search and retrieval, bilingualism, learning, creativity, personality traits, and clinical populations.
In this review, we focused on lexical and semantic networks that capture various types of relationships between words. However, as alluded to where relevant research was available, the usefulness of network science in cognitive science is by no means limited to the domains of language and memory. Network science is a general-purpose toolbox that can be used to study many types of cognitive systems provided that it is indeed theoretically meaningful for these systems to be represented as a network. This further implies that the usefulness of network science depends on both the problem at hand and, of course, the researcher, who will make theoretically motivated decisions. These decisions include, specifically, (i) what aspects of the problem can and should be represented as a network, (ii) what tools and measures should be applied to analyze the network representation, and (iii) how to link network structure and process to offer meaningful insights into empirical and behavioral data. Network science is no panacea for the challenges faced by cognitive research, but when used appropriately, network science can produce important and novel insights for cognitive research, as demonstrated by the vast array of research on cognitive network science covered in this review.
Our understanding of any cognitive or language-related process is necessarily incomplete if we do not consider the structural properties of the cognitive system that the process is occurring in. Network science provides cognitive scientists with a well-studied and formal language to quantify and study the structure of these cognitive systems. On the other hand, the cognitive science community has developed a suite of experimental tasks that can provide crucial behavioral evidence that constrains and informs network models of cognition. The judicious combination of these two approaches will lead to continued insights into a variety of behavioral and cognitive phenomena and strengthen psychological theories of lexical access, memory retrieval, cognitive search, language acquisition, cognitive decline, and creativity, as well as many other related domains of cognitive science that will benefit from the application of network science methods.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
Cynthia S. Q. Siew is supported by the Overseas Postdoctoral Fellowship from the National University of Singapore.