Abstract
Obstacle perception using sound is the ability to detect silent objects, such as walls and poles. It is very important for blind people to recognize their environment using acoustic information through their auditory sense when walking or conducting various daily activities. In this paper, to develop an objective method for evaluating the degree of obstacle perception acquisition in the education and rehabilitation of the blind, the authors measured the spontaneous body movements evoked by the approach of an acoustic virtual wall. Ten blind persons who have experienced obstacle perception in their daily life, and seven sighted persons with no such experience participated in the experiment. The reciprocal (approach and receding) movements of the virtual wall were presented using simulated reflected sound, and the spontaneous body movements of the subjects were measured. As the results indicate, eight of the ten blind participants showed large maximum values for the correlation function between the wall and their body movements, whereas six of the seven sighted participants showed small maximum values. These results indicate that body movements can be used for an objective evaluation of obstacle perception. In particular, it was determined that the maximum value of the correlation function is the most appropriate for such an evaluation, because it does not depend on the subject’s physique.
1. Introduction
Obstacle perception using sound is the ability to detect silent objects, such as walls and poles. Cues helping with the detection of silent objects include sound reflection and/or insulation (obstacle interrupts the sound that comes from behind the obstacle) of the objects. This ability seems superficially similar to the echolocation of bats or dolphins and is therefore sometimes called “human echolocation” [1].
Obstacle perception is a very important tool for blind people to recognize their environment using acoustic information through their auditory sense while walking or conducting various daily activities. Well-trained blind persons can avoid walls or poles without touching such objects with a long cane while they walk. The training needed to acquire this ability is provided through the orientation and mobility training applied during the education and rehabilitation of the blind [2].
Many studies [3–29] on the effective use of obstacle perception in blind education and rehabilitation have been conducted. Supa et al. [3] reported the first systematic research revealing the acoustical mechanism of obstacle perception through the focusing of a reflected sound. Since their study was first conducted, many researchers have investigated this mechanism. For example, Bassett and Eastmond [4] reported that the pitch caused by phase interference between the direct and reflected sounds is important cue to detect the distance of the obstacle. Seki [5] reported that the important sounds for obstacle perception are not only self-generated sounds, such as footsteps or cane-tapping sounds, but also ambient sounds, i.e., nonself-generated sounds that surround us. Therefore, its mechanism is slightly different from the echolocation of bats or dolphins, which use self-generated ultrasonic waves. Ashmead et al. [6] also reported that ambient sound could be used to provide distance information about silent objects.
Recently, some study tried to reveal its mechanism on the brain-scientific point of view [7–14]. The various aspects of obstacle perception have also been investigated in psychophysical studies [15–29]. Based on these studies, PC-based audio modules for testing this ability through the use of artificial reflected sound have also been proposed [30]. Furthermore, an acoustical virtual-reality system for training this ability was developed and has been used for blind education and rehabilitation [31].
However, despite the large number of such studies, an evaluation method for obstacle perception has yet to be proposed, and no methods for measuring the degree of acquisition of this ability in an objective manner have been developed. Some previous studies [32–35] proposed the assessment methods, but their studies were conducted using the actual or binaural-recorded sound fields of real environment where the real ambient noises and reverberation exists. This type of assessment is subject to be influenced by the conditions of real sound environment and has poor replication. Recently, an evaluation was conducted through a subjective assessment of the orientation and mobility instruction applied during the education and rehabilitation of the visually impaired [2]. In the future, objective and highly reproductive evaluation methods for obstacle perception will be required in order to measure achievement of the training. This measurement is useful to determine whether the conducted training was effective or not. Thus, the authors are now trying to develop a new evaluation method that does not require a subjective assessment of the instructors.
On the other hand, it is well known that dynamic changes in sound, such as sound image movements, may influence human posture [36–43]. Certain kinds of sound movements evoke human body movements, a phenomenon that has occasionally been called “auditory kinesthesis.” Herein, we can suppose that when an approaching wall is reproduced virtually using simulated reflected sound and the dynamic sound change is presented to a person who has acquired obstacle perception, the person may demonstrate spontaneous body movements to avoid the wall. In this case, we can measure the degree of acquisition of obstacle perception objectively by measuring their body movements. It was anticipated that the blind participants would do better, because it was estimated that the many blind persons have acquired the obstacle perception.
In this paper, for the development of an objective evaluation method of obstacle perception, the authors measured the spontaneous body movements evoked through the approach (or receding) of an acoustic virtual wall and discuss the possibility of developing an evaluation method using body movements.
2. Methods
2.1. Principle
In general, when a wall is present in front of a listener and a sound comes from behind, the sound arrives at the listener from behind first, and the reflected sound comes from the wall in front with a time delay. Thus, if a sound is presented from behind and a delayed sound is presented from the front, the sound field where the wall exists in front of the listener can be reproduced.
During our experiment, an acoustically simulated “virtual wall” was projected in front of each participant by presenting a “direct sound (the first sound)” from behind and a “reflected sound (the delayed sound)” from the front using two loudspeakers. Movement of the wall in the front-back direction was reproduced through a control of the time delay, and the participant’s front-to-back body movements (i.e., slow and small front-to-back sway of the upper body) evoked by the movement of the wall were measured by a precision position sensor.
It must be noted that, in our experiment, the reflected sound was reproduced by delay time, and an attenuation or a spectrum change of the reflected sound that caused disturbance at the surface of the obstacle was not reproduced, because they are not always important to detect the presence and distance. The sound pressure or the spectrum change is generally used to detect a texture or surface structure of the obstacle, and it is not focused on in our experiment. The obstacle was supposed to have a flat surface and to reflect sounds completely in all frequency. The sounds were supposed to have a flat wave front, and they did not reduce their sound pressure level depending on distance. This situation was simulating that the wall was ideally hard, heavy, large, and flat. The ambient sounds come from infinite distance.
It must be also noted that our experiment was simulating the “passive” obstacle perception where the listener did not generate any sound and used only ambient sound.
2.2. Subjects
The subjects were recruited from acquaintance of the authors. We could recruit ten blind persons (four females and six males) and seven sighted persons (six females and one male) participated for our experiment. The average ages of the blind and sighted participants were 28.5 () and 35.2 (), respectively. There was no significant difference between the ages of two groups (-test, , ).
All ten blind participants were totally blind. Eight of the ten blind participants were congenitally blind, and two were adventitiously blind.
Through self-assessments of the participants, it was noted that all ten blind participants had experience with obstacle perception in their daily life (they have acquired the skill of obstacle perception using sound in their orientation and mobility training and/or their daily life but were not trained echolocators), whereas none of the seven sighted participants had such experience. It was also noted that none of the seventeen participants had hearing or other physical disabilities by the medical examination in their schools or offices.
2.3. Setup
Figure 1 shows the experiment setup. The experiment was conducted in an anechoic room; the size of which was 4,350 mm in width, 6,000 mm in depth, and 2,950 mm in height. The room demonstrated anechoic characteristics for frequencies of 125 Hz and above and quasianechoic characteristics for frequencies below 125 Hz. The background noise level inside the room was less than 20 dBA. The floor of the anechoic room was a wire mesh, but the participant was standing on a rigid beam of the wire mesh.
The participant stood still in the middle of the anechoic room, and two loudspeakers (BOSE 111 AD) were set in front of and behind the participant’s head at a distance of 2.0 m at the same height. The front and back loudspeakers were used as sound sources of the simulated “reflected sound” and “direct sound,” respectively. The sound signal was generated using a digital sound player (YAMAHA MD4S). The direct sound was simulated using a sound signal directly emitted from the loudspeaker through an amplifier (BOSE 1702). The reflected sound was simulated using a digital delay device (Roland SDE-330) and was also emitted from the loudspeaker through an amplifier. The digital sound player and digital delay device were controlled using a computer (NEC PC9821Ap3) connected through MIDI. The control frequency of the delay time through the MIDI device was 40 Hz. The sound signals were processed digitally by the digital sound player and digital delay device, with a 16-bit resolution and a sampling frequency of 44.1 kHz.
To measure the body movements, a magnetic field position sensor (3SPACE FASTRAK, Polhemus) was used. The source coil of the position sensor used to generate the magnetic field was set above the participant’s head with a pole. The sensor coil of the position sensor used to receive the magnetic field was set behind the participant’s neck using a fabric belt. The position of the sensor coil was measured to be accurate to within 1 mm with a 10 Hz sampling frequency. The measured position was recorded in a computer using an RS-232C.
2.4. Stimulus
A pink noise was used as the stimulation sound signal, because it had the higher power at the lower frequency; therefore, it was similar to an ambient noise. The frequency band of this sound was 40 Hz to 8 kHz. The sound pressure level of the direct sound was 56 dBA at the center position of the participant’s head, and the reflected sound was presented using the same sound pressure level.
The distance from the virtual wall was controlled based on the time delay of the reflected sound with respect to the direct sound. The distance was found using where is the time delay and is the sound velocity in air. The sound pressure level of the reflected sound remained constant even when the distance changed.
In our experiment, as described in Procedure, the range of the distance was between 0.2 and 0.8. The velocity of the wall movement was 0.06 m/s. This means that the frequency of the movement was 0.05 Hz.
By this stimulus, only the distance was presented, and the size of the wall was not presented. The subject could not estimate the size of wall because there was no information to know it.
2.5. Procedure
Figure 2 shows the wall movements used. Linear (constant velocity) movement was used because it was most simple movement. During the experiment, reciprocal (approach and receding) movements of the virtual wall were presented to the participant. In one trial of the experiment, wall movements were generated reciprocally six times at distances between 0.2 and 0.8 m linearly in front of the participant within a 20 s period. The distance range 0.2-0.8 m was decided because an appropriate distance for obstacle perception is generally less than 1 m. The body movements of the participant were measured for the last five of the six wall movements. Time duration of one trial was 120 s ( periods). The first body movement was omitted because it occurred just after the start of the wall movement and was not steady.
During the experiment, the body movements were measured only in a front-back direction. Up-down and right-left movement data were omitted because the wall movements were only in a front-back direction.
Ten trials were conducted for each participant, i.e., a total of 170 trials () were used. Total time of the experiment for one subject was 1,200 s ( s). Total time of the experiments was 20,400 s ().
The subjects were just instructed “Please do not move. Please keep standing still during the sounds are presented.” The subjects were not instructed that the virtual wall was presented. The subjects were also informed that there was nothing except loudspeakers and position sensor in the anechoic room. The subjects knew that this experiment was safe.
The subjects were not blindfolded for safe emergency evacuation from unexpected disaster during the experiment, but they were instructed to close their eyes during the trial. In addition, the anechoic room was dark during the experiment.
The start and end of the trial were notified by the voice announcement of the experimenter that was presented from the loudspeakers. During the experiment, the experimenter was in the control room that was outside of the anechoic room.
Neither training nor practice was given to the subject in advance because the task of the subjects was just to keep standing still and not to do anything.
It must be noted that the body movement did not occur when the wall movement was not presented.
2.6. Analysis
The measured data were analyzed as follows for each participant.
The final result of body movement for one subject was obtained by smoothing the ten sets of measured data using where is the number of trials (i.e., ) and is the measured body movement data during trial for one subject. It must be noted that, in our experiment, the body movement did not show remarkable phase difference among the trials, and therefore, the averaging did not reduce the amplitude of the measured body movement.
The standard deviation (SD) of body movement for one subject was calculated using (3) and (4). The SD is proportional to the square root of power of body movement, and therefore, it can be used as the index of magnitude.
The correlation function between the wall and body movements for one subject was calculated using where is the time difference, is the time duration of the measurement (i.e., ), and and are the wall and body movements as a function of time, respectively. For easier calculation, it was supposed that is an infinite periodical function of a triangle wave within the range of . The range of was because the period of was 20 s.
The maximum value of the correlation function and its time difference () were also found.
3. Results
3.1. Perception of Wall
After the experiment, the subjects were interviewed, and they answered whether they perceived the wall existence or not. The 10 blind subjects reported that they perceived the wall existence by the stimulus of our experiment, and 7 sighted subjects reported that they heard the sound change but did not perceive the wall existence during the experiment.
All 17 subjects also reported that they kept standing still as instructed during the experiment, and they did not become aware of their body movements.
3.2. Body Movement and Correlation Function
Figure 3 shows the wall movement , the measured body movement , and the correlation function between the wall and body movements . The results show that five (Blind No. 1–No. 5 in Figure 3) of the ten blind participants showed large body movements. The amplitudes of their body movements were about 0.03 to 0.06 m, and the maximum values of the correlation functions were over 0.8.
The results also show that three (Blind No. 6–No. 8 in Figure 3) of the ten blind participants and one (Sighted No. 1 in Figure 3) of the seven sighted participants showed large maximum values of the correlation functions, although their body movements were small.
On the other hand, two (Blind No. 9 and No. 10 in Figure 3) of the ten blind participants and six (Sighted No. 2–No. 7 in Figure 3) of the seven sighted participants did not show any clear responses to the corresponding wall movements.
3.3. SD of Body Movement and Maximum Value of Correlation Function
Figure 4 shows the distribution of SDs of body movement and the maximum value of the correlation function . The results illustrate that the SD of the body movements of the blind participants was significantly larger than that of the sighted participants (-test, , ). The results also indicate that the maximum value of the correlation function of the blind participants was significantly larger than that of the sighted participants (-test, , ). It was also noted that, when the significances of the difference (i.e., values of ) were compared, the maximum value of the correlation function was superior to the SD of the body movements.
As the distribution graph illustrates, some of the participants showed high maximum values of the correlation function despite the small SD of their body movements (i.e., three of the blind and one of the sighted participants had an SD of higher than 0.5 despite a movement of less than 0.01 m). It should also be noted that the participants who showed a low maximum value of the correlation function did not show large SDs of their body movements (i.e., two of the blind and six of the sighted participants had SDs of lower than 0.5 despite body movements of no larger than 0.005 m).
3.4. Time Difference for Maximum Correlation Function
Figure 5 shows the distribution of the time difference for the maximum correlation functions and . The results illustrate that, when comparing the time differences between these maximum values of the correlation functions, there was no significant difference between the blind and sighted participants (-test, , n.s.).
When the time difference for the maximum value was negative (i.e., ), it means that the phase of the body movement was delayed with respect to that of the periodical wall movement. When it was positive (i.e., ), it means that the phase was preceded. When it was zero (i.e., ), it means that the phase was identical. In addition, when the absolute value was a quarter of the period or less (i.e., ), it means that the phase was nearly synchronized (i.e., when the wall approached the participant, the participant’s body moved away from the wall). When the phase was not synchronized (i.e., or ), it means that the phase was nearly inversed (i.e., when the wall approached the participant, the participant’s body approached the wall).
As the results show, eight (6 blind and 2 sighted) of the seventeen participants showed a “delayed body movement” (), and the other nine showed a “preceded body movement” (). Nine showed a “synchronized” () phase, and the other eight showed an “inverse” ( or ) phase. When only the nine participants who showed a higher maximum value than 0.5 were focused upon, five of them showed a “delayed” body movement, and the other four showed a “preceded” body movement. Six of the nine showed a “synchronized” phase, and the other three showed an “inverse” phase. It was also noted that there were two “synchronized” participants who has “preceded” movements, and their time differences for the maximum value were less than 1 s.
4. Discussion
As the results of the experiment indicate, eight of the ten blind participants who had experience with obstacle perception showed large (over 0.5) maximum values of the correlation function, whereas six of the seven sighted participants who had no experience showed small (less than 0.5) maximum values of the correlation function. In addition, some of the participants showed high maximum values of the correlation function despite the small SDs of their body movements. This means that the maximum correlation functions can be used as one of the index to determine whether or not the participants acquired obstacle perception capability. The threshold of the determination can be assumed to be about 0.4 to 0.5, according to the results shown in Figure 4. The two blind subjects who did not show remarkable responses were congenitally blind. The reason why the two blind subjects did not show remarkable responses was estimated that their acquisition of obstacle perception was poor, though they reported that they had the experience of obstacle perception.
During the experiment, the SD of the body movements was also examined, and the maximum correlation function was shown to be superior to the SD of the body movement when finding the significant difference between the blind and sighted participants. Even when the maximum values of the correlation function of the blind participants were large, the SDs of the participants’ body movements were not always remarkable. It was estimated that the amplitude of the body movement depends on the participant’s physique, such as their height and weight. Thus, the value of the SD, which is related to the amplitude, was also influenced by their physique and was not steady. The threshold of the determination could also not be found.
The “phase” issues should be discussed here. Why did three of the nine participants who were at higher than 0.5 show an “inverse” phase? The one of the potential reasons was estimated to be because they confused the front-back direction in their median plane localization. Median plane localization is the ability to find the position of a sound source on the median plane, including the frontward, backward, upward, and downward directions. This ability is based on spectral cues of the head-related transfer function (HRTF). In this experiment, a pink noise of 40 Hz to 8 kHz was used as the stimulus sound, although this frequency band is slightly narrow for the median plane localization for the human auditory system (ideally, 8 kHz and upper-frequency components are needed [44]). Thus, there was a possibility that some of the participants may have mistakenly perceived that the reflected sound came from behind them and that the virtual wall also approached from behind. To avoid such confusion, a wider frequency band noise must be used as the stimulus sound.
The “delayed” and “preceded” body movement issues should also be discussed here. During the experiment, the body movements were evoked through the wall movements. Thus, a body movement will show a delay with respect to the wall movement. However, two of the nine participants who were at higher than 0.5 and showed a “synchronized” phase had a “preceded” body movement. As the results indicate, the preceded time was less than 1 s. The reason for this was estimated to be because the two participants were able to predict the wall movement because it was periodical, and they spontaneously moved slightly before the wall movement. On the other hand, other 17 subjects could not show the preceded body movement. This means the absence of “task-learning” for the 17 subjects. Had a non-periodical or speed-varying wall movement been presented, this “preceded” body movement would not have occurred.
In addition, we tested only one period (i.e., 20 s) and only one movement (i.e., 0.8-0.2 m linearly) in this paper. The body movement was observed although the virtual wall did not collide with the subject because the distance range was 0.8-0.2 m. It was estimated that the subjects show the body movement because they tried to keep plenty safe distance between the wall even if it did not collide. There was neither significant background noise nor reverberation because the experiment was conducted in an anechoic room. The wall size, shape, and material also could not be changed. As future works, other kinds of periods, distance range, movements, and conditions must be tested.
It is also noted that the equipment for this objective assessment is expensive recently, but in future, the cost of the equipment can be reduced, because this will be able to be conducted by the sensors and audio function of a smart phone. If the smart phone will be able to provide 3D sound image through a close-type stereophonic headphones, an anechoic room will not be needed, either.
5. Conclusions
In this paper, to develop an objective evaluation method for obstacle perception, the authors measured the spontaneous body movements evoked by the approach of an acoustic virtual wall and discussed the possibility of developing such a body movement-based evaluation method.
The results indicate that body movements can be used for an objective evaluation. In particular, it was found that the maximum correlation function between the wall and body movements is the most appropriate for such an evaluation.
In the future, this finding will be applied to the objective assessment of achievement of the training in the field of education and rehabilitation of the persons with visual impairment.
6. Limitations of the Research
This research has potential limitation. (1) The wall movement that was used in our experiments was only one combination of distance (0.2-0.8 m), period (20 s), and direction (front). (2) Only the front-to-back body movement was measured. The movements of other directions were not measured. (3) Only one kind of sound (pink-noise, 56 dBA) was tested. (4) Only the reflected sound that had the same sound pressure level and spectrum as the direct sound was tested. (5) The participants were only sighted persons and persons with total blindness. The persons with low vision were not included.
7. Recommendations for Future Research
As mentioned in Limitations of the Research, this research tested the limited condition. In order to build up more knowledge, the following conditions must be tested in future: (1) Various wall movements are presented. (2) Three directions of body movements are measured. (3) Various kinds of sounds are presented. (4) Various sound pressure levels and spectrums of reflected sounds are presented. (5) Various kinds of persons including sighted, blind, and low vision participate.
Data Availability
All relevant data are within the manuscript.
Ethical Approval
This research was ethically approved by Bioethics and Biosafety Management Office, Safety Management Division, National Institute of Advanced Industrial Science and Technology (AIST).
Consent
All the participants were consented to attend the experiments of this research. Our experiments were conducted with the subjects’ understanding and consent.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This study was supported by the “Basic Study on Role of Echolocation in Posture Control by the Blind,” Grant-in-Aid for Scientific Research, from the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan, and by the Japan Society for the Promotion of Science (JSPS), Grant No. 15500373 (2003-2005).