Abstract
The brain is the center of intelligence that biological systems have acquired during their evolutionary history. In unpredictably changing environments, animals use it to recognize the external world and to make appropriate behavioral decisions. Understanding the mechanisms underlying biological intelligence is important for the development of artificial intelligence. Olfaction is one of the sensory modalities that animals use to locate distant objects. Because of its relative simplicity compared with other sensory modalities and the wealth of knowledge at cellular, network, system, and psychophysical levels, it is possible that the biological olfactory system would be understood comprehensively. This paper reviews our biological and computational works with a focus on the temporal aspects of olfactory information processing. In addition, the paper highlights that the “time” dimension is essential for the functioning of the olfactory information processing system in the real world.
1. Introduction
Consider animals, such as honeybees, butterflies and so forth, collecting flower nectar in fields. For simplicity, all flowers have the same visual cues (e.g., shape, size, color, etc.), but each of them has a distinct odor. Some flowers can have the same odor but are spatially distributed randomly in the field. The quantity of flower nectar is fixed according to the type of odor; for example, flowers with odor A have 10 mL nectar, odor have 5 mL, and odor have 0 mL. There might be some dangerous flowers, such as insectivorous flowers, giving off specific odors. It is important to be aware of the cognitive abilities that an animal needs to possess in order to collect nectar efficiently and avoid danger.
An animal can sense an odor at various distances from its source, implying that an odor can be sensed at different concentrations. To approach or avoid flowers with a specific odor, the animal should be able to judge the odor at different concentrations as an odor with the same quality and should be able to determine its direction based on whether the odor signal becomes stronger or weaker. Thus, the animal must have the ability to identify odors in a concentration-invariant manner.
Suppose that the animal has visited many flowers in a certain field and has experienced the relationship of odors with nectar quantities or dangers and then moves to a different field. How will the animal behave in the new field where there are no flowers with odors identical to those experienced in the previous field? Based on the experience in the previous field, the animal should visit flowers with odors similar to those associated with a large amount of nectar and avoid flowers with odors similar to those of dangerous flowers. Thus, the animal must detect similarities and differences between odors and make appropriate behavioral decisions. These examples demonstrate that, at large, olfactory information processing is a typical example of pattern recognition. Problems of pattern recognition are further complicated in the real world where unpredictable noises or perturbations occur. Even in such situations, animals use the olfactory information to make appropriate behavior decisions in order to survive in the real world.
Olfaction is the sensory modality that many animals use to locate distant objects. Structures of the olfactory system in the brain, especially those from peripheral olfactory receptors to the primary olfactory network, have common characteristics among vertebrates, insects, terrestrial gastropods (slugs/snails), and so forth [1]. Odorant molecules are received by olfactory receptor neurons (ORNs) that are distributed in the peripheral organs (e.g., the nasal cavity in vertebrates, the antennae in insects, and the tentacles in slugs/snails) [1–4]. ORNs project axons to round-shaped structures called glomeruli in the primary olfactory network [1, 5]. Axons of ORNs that express the same odor receptor type converge in each glomerulus [6–8]. The primary olfactory network (the olfactory bulb in vertebrates and the antennal lobe, AL, in insects) consists of output neurons and local inhibitory interneurons [2]. These neurons extend their dendrites into glomeruli and receive inputs from ORNs. The glomeruli are distributed as a glomerular layer along the surface of the olfactory bulb in vertebrates [2, 3] and botryoidally arranged in the insect antennal lobe [2, 4]. Physiological studies using an optical recording technique revealed that even monomolecular odorants evoke broadly distributed activation across glomeruli [9–12], so that an odor input is first represented as a spatial pattern of glomerular activation at the primary olfactory network.
The primary olfactory network transforms the input spatial pattern into the spatiotemporal activity pattern of output neurons [2] and sends it to various brain regions [5]. In general, brain functions emerge by cooperative information processing in various brain regions. To understand how the biological olfactory system solves pattern recognition problems in the real world, it is important to clarify the following two issues.
(i)How do odors spatiotemporally activate the olfactory regions in the brain, and how does the odor information flow within the brain as a whole?(ii)To solve the problems of pattern recognition, what kind of computational algorithm is needed? How does the biological network implement it? To perform the algorithm, how is the spatiotemporal odor representation in the primary olfactory network useful?By understanding these issues comprehensively, the intelligence that the biological olfactory system has acquired evolutionarily to solve pattern recognition problems in the real world can be explained.
This paper reviews physiological and computational studies on olfaction from these perspectives. Section 2 describes an experimental approach to explain the flow of olfactory information in the slug’s brain. Section 3 describes the computational coding scheme of olfactory information using the time dimension. Section 4 summarizes the results and provides directions for future work.
2. Information Flows in the Whole Brain
Understanding the flow of information within the brain as a whole is one of the goals that neuroscience research addresses, and for this purpose, several techniques have been developed to measure brain activities. However, it is still hard to observe the flow of information in the whole brain because of the following difficulties: there is a tradeoff between spatial and temporal resolutions in any method, and the brain is too large and complex to be easily studied. So, it is important to select appropriate animals for measurement. In cases where the object to be measured is small, the optical imaging technique has a good spatiotemporal resolution. If we can use an animal that has a good learning ability, can discriminate objects, and has a small brain that is simple in structure, the flow of information can be visualized in such a brain, and it will be possible to explain the information processing principle in the brain as a whole.
From this perspective, a slug/snail is a useful animal model. In these animals, olfaction is a dominant sensory modality for recognizing external objects (visual or auditory systems have not been developed in their brain) [1, 13, 14]. The slug/snail has a good odor learning ability [15–20]. It has a small and simple brain; so whole brain activity can be measured with good spatiotemporal resolutions [21–26]. Furthermore, there are several experimental advantages of their olfactory system. Noses of these animals are located on the tips of two pairs of tentacles (superior and inferior tentacles, STs and ITs). Several behavioral and physiological studies have revealed that there are functional differences between STs and ITs [27–31]; ST is involved in olfactory orientation [28], whereas IT is involved in learning [29] or retrieving odors [27, 31]. Their brain can be dissected and isolated as a whole without a lose in the function of these olfactory organs [23, 24, 27, 32, 33]. These features are useful for explaining the relationship between the flow of information and the emergence of brain functions.
Figure 1(a) shows the slug we used, the Japanese slug Incilaria fruhstorferi, and Figure 1(b) schematically illustrates the slug’s brain, its cerebral ganglia (CG). The STs and ITs are connected to CG via superior and inferior tentacle nerves (STNs and ITNs), respectively. CG comprises three lobes: the procerebrum (PC), mesocerebrum (MsC), and metacerebrum (MtC). Anatomical studies indicate that after entering the body of CG, afferent fibers of the tentacle nerves segregate into several bundles and terminate in PC and MtC [1, 14, 35]. The PC is regarded as the olfactory center, and many studies have investigated its role in the processing of olfactory information [22–24, 27, 30, 36–40]. It is suggested that PC is involved in acquiring and retrieving odor memory [27, 30, 38].
(a)
(b)
(c)
(d)
In contrast, few studies have investigated olfactory information processing in MtC. Anatomically, afferent fibers from STN and ITN converge into the medial region of MtC (mMtC) (Figures 1(c) and 1(d)) [26]. The MtC is thought to collect olfactory, taste, and other sensory information and to command motor actions when appropriate signals are received [14]. The relationships between PC and MtC in the slug’s brain might correspond to those between cortical and subcortical structures in the vertebrate’s brain. Hence, the slug’s brain is suitable for our research purpose to observe the flow of information within the whole brain.
We optically recorded the brain activity evoked by electrical stimulations of STN and ITN and obtained the following results [26].
(1)STN and ITN stimulations activate both PC and mMtC.(2)Regardless of STN or ITN stimulations, the mMtC response is about 50 milliseconds earlier than the PC response. (3)STN and ITN stimulations evoked different activation patterns of mMtC: the ITN stimulation activated the lateral half of mMtC more strongly than its medial half, whereas the STN stimulation activated both halves evenly. In contrast, there seems to be no difference between the activation patterns in PC evoked by STN and ITN stimulations.It is interesting that PC responses to STN and ITN stimulations are the same, in spite of memory functions of PC [27, 30, 38, 40] and functional differences between STs (orientation) and ITs (memory) [27–31]. This implies that direct olfactory inputs to PC would not explain the differences in memory functions between STs and ITs.
Since mMtC responds faster than PC to the nerve stimulation, there is a possibility that mMtC activations evoked by STN and ITN stimulations affect PC differently, and as a result, a functional difference between ST and IT for memory functions might emerge. We hypothesized that monoamine- (such as serotonin, dopamine, etc.) containing neurons might mediate the transfer of information from mMtC to PC because, as neuromodulators, monoamines are known to have important roles in memory functions [41–43] and in changing the functional modes of neural networks [44–46]. Therefore, we stained the dopamine-containing neurons in the slug’s brain [34]. Main results are summarized in Figure 2. The dopamine-containing neuropils are intensively distributed in the central and lateral regions of MtC (Figure 2(a), dotted area), and this central MtC region seems to spatially overlap with lateral area of the mMtC region that was strongly activated by the ITN stimulation (Figure 2(a)). The dopamine containing output fibers from the central MtC region seem to project into PC. Gelperin and his colleagues physiologically suggested that dopamine might modulate the network function of the PC [47, 48]. So, the revealed morphological features suggest that the dopamine-containing neurons might act as a bridge between mMtC and PC, especially when the mMtC region is activated through the ITs [Note 1]. Dopamine-containing fibers project densely into the STN and ITN, and these come at the tentacle ganglion of ST and IT, suggesting another possibility that dopaminergic modulation of olfactory information processing might occur more peripherally through the tentacle ganglia.
(a)
(b)
Olfactory functions in the slug brain can be explained on the basis of the flow of olfactory information (Figure 3). Based on the mMtC activity, ST and IT would have different roles in olfactory functions. The medial half of mMtC (that is strongly activated by the STN stimulation) might be related to olfactory orientation. Indeed, several motor neurons related to olfactory orientation movements are known in regions close to mMtC [49–51]. The lateral half of mMtC (that is strongly activated by the ITN stimulation) might activate the monoamine system, and this system would modulate functional modes of PC to acquire or retrieve odor memory.
(a)
(b)
It is suggested that the brain (cerebral ganglia) can evaluate important odors and can command motor actions without PC, even though the odor discriminating ability of the brain without PC is not good compared to that with PC [49]. Based on this observation and data from our study [26], it can be hypothesized that mMtC might contribute to the rough evaluation of important odors. The quick response of mMtC to STN and ITN stimulations might result in quick responses of the slug/snail to important odors and limit the odors that are processed by PC for fine discrimination [26]. Such a temporal process would be ecologically advantageous, since various odors exist in the real world.
3. Information Representation Using Time Dimension Solves Pattern-Recognition Problems
There is growing evidence that biological systems use time as a dimension for sensory information coding [52–57]. Computationally, a central aspect of pattern recognition is achieving a “good” representation in which the main features of an object are simply and naturally revealed [58, 59]. Neurobiological information representation is crucial in facilitating such a computation [58, 60].
Regardless of biological or artificial systems, recognition systems must extract invariant features of objects from varying input signals depending on situations and must detect and judge similarities and differences between objects appropriately. These requirements are for a system that can identify objects and make appropriate behavioral decisions in the real world. Thus, the temporal representation of objects should satisfy certain computational criteria for it to be useful for recognition. Criterion 1 is invariance. For instance, the size or strength of sensory signals varies when the same object is sensed from different distances. Even in this situation, object representation should be invariant. Criterion 2 is similarity. The degree of similarity of objects must be reflected in their representation while also expressing subtle differences. Regardless of sensory modalities, biological recognition shows a speed/accuracy tradeoff, which is the relationship between sampling time and accuracy in object discrimination [61–63]. This psychophysical principle constrains the temporal representation format of the biological system by imposing Criterion 3, which is coarse-to-fine nature. The coarse similarity or difference of objects should be encoded in the early epoch of the temporal representation, and the information encoded in the late epoch should help in detecting subtle differences among objects.
The question arises as to what kind of information representation format would satisfy the three representation criteria. Suppose that there is a set of feature dimensions each of which describes a specific object feature. By preprocessing of sensory signals from an object, the intensity of each feature is obtained, and consequently, the input representation of the object is an analog vector of n-dimensional feature space. Changing the signal intensity or size from the same object would only cause changes in the coefficients or bias of input analog vectors. In this case, the order of feature dimensions (in other words, the ranking of feature dimensions) determined by elemental analog values of the input vector would be scale invariant. So, if a computational algorithm transforms the order of analog values of the input vector into a temporal sequence of spike activity of neurons (or neural assemblies), each of which controls the respective feature dimension (Figure 4(a)), then the emergent temporal representation has scale invariance, satisfying Criterion 1. A temporal sequence of neural activities represents the hierarchical relationship of feature dimensions: as time advances, an object is coarsely classified on the basis of a feature represented by the first activated neuron, further subclassified by subsequent ones, and finally identified by taking all activated neurons into account (Figure 4(a), bottom), implying that the representation satisfies Criteria 2 and 3.
(a)
(b)
Using the -dimensional feature space, a system adopting such a coding scheme can theoretically represent and discriminate n! objects (“!” is the factorial operator). If the system can set a temporal range for object representation, it can control the clustering levels of objects and make appropriate behavioral decisions that are flexible according to situations.
The proposed coding scheme is highly consistent with information transformation in the primary olfactory network of biological systems, especially in that of insects and vertebrate zebra fish [52–55]. As mentioned in Section 1, odor input representation is the spatial pattern of glomerular activation [9–12]. Even monomolecular odorants evoke a broadly distributed activation across glomeruli, such that each glomerulus is regarded as a functional module detecting a particular feature of odorant molecular structure [3, 9–12, 64]. Odors at very low concentrations activate a few glomeruli above the background level. As the concentration increases, several additional glomeruli are recruited, and the glomerular activity is a logarithmic function of concentration [9, 11, 65–67]. Furthermore, similarity in the molecular structures of different odor molecules is correlated with similarity in spatial patterns of glomerular activation [10, 11, 68, 69] and with perceptual similarity [69–71]. Spatiotemporal activity patterns of the insect primary olfactory network are robust to odor concentration changes [54], whereas those of the zebra fish primary olfactory network are suggested to reflect odor similarity in a temporally coarse-to-fine manner [53]. These imply that the input spatial pattern of odor-evoked glomerular activation can be represented as an analog vector with a concentration bias [67]; the glomerular ranking that can be determined by the odor-evoked glomerular activation would be concentration invariant [72–74]; if the glomerular ranking can be transformed into temporal sequence of neural activities according to the proposed coding algorithm, the emergent spatiotemporal patterns would reflect the odor similarity as physiologically observed [72–74].
By means of biologically plausible neural components and their interactions, we constructed a primary olfactory network model that can implement the coding algorithm shown in Figure 4(a) for odor information. The model consists of glomerular modules, each consisting of one output neuron and two types (intraglomrular and interglomerular types) of inhibitory local interneurons. In the single glomerular module model, the intraglomerular inhibition helps to cause a phasic activity of the output neuron in response to a constant glomerular input, and onset latency of the phasic response of the output neuron decreases as the input strength increases. In the network model, the glomerular modules are connected by the interglomerular inhibitory interneurons. Through these interglomerular connections, the activated output neuron inhibits other glomerular output neurons and delays their phasic activation, and consequently phasic activities of the output neurons are ordered according to the strength order of the glomerular inputs (Figure 4(b)). The network model successfully implements this spatiotemporal transformation regardless of absolute input analog values or concentration biases [72–74].
In the insect higher center network, that is, the mushroom body (MB), odors are represented by spikes sparsely distributed in space and time [53]. In our primary olfactory network model, the odor information is represented as the temporal sequence of the output neurons’ activities, in which the information about the activity sequence is intensively-coded around transient overlaps of the neural activities (see Figure 4(b)). So, we further constructed a higher center model that evaluates the transient overlaps of the neural activities of the output neurons of the primal olfactory network model. The higher center model can extract odor information as a temporal sequence of sparse spikes with a coarse-to-fine nature and can store the odor information as network connectivity with the help of simple Hebbian connections [72–74].
These indicate that the essential problems of olfactory recognition can be solved by information representation using the time dimension. In the proposed coding scheme, significant features characterizing objects are temporally placed first, followed by more subtle features. This strategy is crucial for the system to survive in an unpredictably changing environment because it allows the system to respond quickly to vitally important object features. Even though it may be just one dimension mathematically, “time” is an important dimension for systems that live and survive in the real world. Thus, the information representation using the time dimension is a basis for pattern recognition that warrants further investigation in biological systems.
4. Towards an Understanding of Biological System Intelligence
In Section 2, we have shown that the olfactory system in the slug’s brain consists of two pathways: one is the mMtC-pathway that probably has a role for quickly and roughly discriminating important odors; the other is the PC-pathway that is rather slow but important for finely discriminating odors. With respect to the computational model shown in Section 3, the physiological data about the spatiotemporal neural activities are mainly based on those obtained from the insect AL-MB pathway [52, 54, 55]. The insect MB is suggested to have important roles in fine discrimination of odors [75, 76] and odor learning and memory [77–79], as the slug/snail PC is suggested to have such roles [33, 38, 40]. So, the computational model in Section 3 is thought to correspond to the PC-pathway in Section 2, and we can suggest that the olfactory system of the brain uses “time” dimension doubly (i.e., the anatomically differentiated two pathways that have different temporal responses to odors, and the coarse-to-fine odor representation using the temporal sequence of neural activity in the fine discrimination pathway) to quickly and dominantly process important odors or odor-features. These temporal aspects of olfactory information processing might be essential for systems working in the real world.
To comprehensively understand the mechanisms involved in olfactory information processing, we must explain how global information flow and computational information processing are related and integrated on the systems level. In the information representation proposed in Section 3, odor is described as the permutation of odor features, each of which works like letters of an alphabet. So, information represented by the temporal sequence of neuronal activities is regarded as symbolic information. Symbolic information is useful for identifying objects and detecting their similarities and differences. However, symbolic information of objects itself cannot help the system make a behavioral decision because it does not represent information on the values of objects. We call this “value information.” Value information cannot be defined by attributes of objects alone. Value information of an object should be evaluated and determined by the system in real time on the basis of experience with the object and current conditions of the environment and the system. From invertebrates (including the slugs and snails mentioned in Section 2) to vertebrates (including primates), monoamine systems in the brain are known to be strongly related to internal and motivational states of biological systems and to play important roles in learning and memory [32, 41–43, 80–86], so that monoamine systems might be involved in determining the value information of objects. As neuromodulators, monoamines would affect the brain activity in a wide spatial and temporal range, and their spatiotemporal dynamics in the whole brain would be critical for neural network functions [84]. The challenge is to explain biological and computational mechanisms relating temporally represented symbolic information with value information that might be determined by global information flows within the brain. This approach would clarify the design principles of an intelligent system that works well in the unpredictably changing environment of the real world.
Note. This hypothesis predicts that the PC should respond differently between the STN and ITN stimulation by the indirect input through the mMtC. However, we could not observe such difference. In the experiment, both STN and ITN stimulations evoked transient depolarization followed by the strong and long lasting (about 10 seconds) hyperpolarization in the PC [26]. The transient depolarization of the PC is caused by the direct input from the STN and ITN, whereas the following PC hyperpolarization is probably caused by inhibitory effects of the intrinsic PC neurons [87]. In the experiment of [26], this intrinsic hyperporlaization in the PC might mask the PC response caused by the mMtC pathway.
Acknowledgments
This study was supported by a Grant no. 21500294 for Y.Makino from the Japan Society for the Promotion of Science, and by a Grant no. 17075003 for Y.Makino and M.Yano from the Japanese Ministry of Education, Culture, Sports, Science, and Technology.