User Assessment in Serious Games and Technology-Enhanced LearningView this Special Issue
Research Article | Open Access
Thomas D. Parsons, Albert A. Rizzo, Christopher G. Courtney, Michael E. Dawson, "Psychophysiology to Assess Impact of Varying Levels of Simulation Fidelity in a Threat Environment", Advances in Human-Computer Interaction, vol. 2012, Article ID 831959, 9 pages, 2012. https://doi.org/10.1155/2012/831959
Psychophysiology to Assess Impact of Varying Levels of Simulation Fidelity in a Threat Environment
There are many virtual environments found in the serious game community that simulate real world scenarios. There is a broad range of fidelity and experimental controls among these serious games. An important component to most evaluations is the extent to which level of fidelity impacts the persons immersed in the serious game. While a great deal of virtual environment and serious game research has assessed the subjective state or feeling of the participant (e.g., the participant’s sense of presence) through the use of questionnaires, the current study examines participant experience by examining psychophysiological responses of participants to their surroundings. The primary goal in this study was evaluative: will a virtual environment with arousing contents result in increased sensory arousal if it is presented in a highly immersive configuration? A secondary goal of this study was to investigate the utility of our environment to offer varying levels of stimulus threat to impact the user’s experience of the virtual environment. Increased simulation fidelity in an arousing environment resulted in faster heart rates and increased startle eyeblink amplitudes, suggesting that higher fidelity scenarios had great efficacy related to sensory arousal.
1. Psychophysiology to Assess Impact of Varying Levels of Simulation Fidelity in a Threat Environment
Virtual environments (VEs) and serious games offer the potential to stimulate and measure changes in the users’ emotion, neurocognition, and motivation processes. The value in using simulation technology to produce serious games targeting such processes has been acknowledged by an encouraging body of research. Some of the work in this area has addressed affective processes: anxiety disorders, pain distraction, and posttraumatic stress disorder [1–3]. Other work has assessed neuropsychological processes [4, 5]. Further, psychophysiology is increasingly being incorporated into research using virtual reality environments [6–8]. The use of psychophysiological measures in affective and neurocognitive studies of persons immersed in VE scenarios offers the potential to develop current physiological computing approaches  into affective computing  scenarios.
The incorporation of simulation technology into neuroergonomic and psychophysiological research is advancing at a steady rate . New discoveries and techniques are demanding a more rapid and advanced paradigm. In response to the demands, a wide variety of simulations have been developed. The range and depth of these simulations cover a large domain, from simple low fidelity task environments to complex high fidelity full immersion simulators. All of these simulators rely on some type of representation of the real world. An important issue for research into simulation for social and behavioral sciences is the determination of how advanced the simulator needs to be to adequately assess and/or train a particular individual or team. While high-end simulations can train a variety of user types, the cost associated with these devices can be difficult to justify .
In this paper, we attempt to build on earlier work that used psychophysiology to assess the propensity of users to respond to virtually generated sensory data as if they were real . We aim to assess the propensity of users to respond to virtually generated sensory data as if they were real . In the same way people experience physiological responses to stimuli in the real world, researchers seek to quantify participant experience by measuring responses evoked by stimuli in a VE. A low fidelity VE may be preferable in studies where a maximal amount of experimental control is desired because such environments may increase psychometric rigor through limiting the number of sensory variables available to the user . Contrariwise, high fidelity environments are preferable for studies desiring increased ecological validity because they recreate more of the real world environment—better capture the participant’s performance as it would occur in a real world setting .
Discussions of the level of fidelity and experimental control needed for a VE often go beyond simple discussions of the “immersive” qualities of the environment to discussions of the impact upon the perceived feeling of “presence” of the individual while immersed in the environment . A number of discussions of the distinction between the terms “immersion” and “presence” can be found in the literature [18–22]. This distinction is important for the current study because issues of fidelity tend to reflect levels of immersion, while levels of presence reflect the user’s experience relative to the level of fidelity/immersion. For the current study, we focus on fidelity and levels of immersion. We also feel that it is important to differentiate between immersion and presence. By immersion, we follow Slater and Wilburs  delineation, in which immersion is seen as an objective description of aspects of the system—that which the overall VE can deliver (e.g., the level of fidelity in representing the real world; the field of view, the number of sensory systems it simulates, the frame-rate, and latency ). Hence, the level of immersion is an objective property of a VE that in principle can be measured independently of the human experience that it engenders. We view presence as a subjective phenomenon (e.g., sensation of being in a VE).
Knowledge of the user-state during exposure to the VE is imperative for development and assessment of VE design. A number of presence studies have researched such issues using questionnaires [23–25]. Subjective measures tend to rely on post-test assessments of the user’s feelings during the exposure to the VE, which is dependent on memory of the event . Self-report data, when used in isolation, are highly susceptible to influences outside the subject’s own targeted attitudes . The item’s wording, context, and format are all factors that may affect self-report responses. A limitation of questionnaire measures is that they can only be administered following a participant’s immersion in a VE, but in order to assess participant experience during the actual immersion in a VE, researchers have sought a more objective measure. Online assessment of participant experience is difficult when using subjective measures, in that the very existence of subjective questions during immersion serves to break the continuity of the participant’s conscious awareness currently being experienced. As a result, a quite different view seems to be emerging, in which presence is treated as something rooted in physiological and behavioral activity [14, 22] and there is a growing emphasis upon physiological and behavioral assessment , as well as the relation between immersion and emotion [28, 29]. Further, there has been increased use of neuroscience techniques for presence measurement, such as EEG , transcranial Doppler , and fMRI .
Up until this point, VE studies have typically relied on self-report and behavioral measures to assess levels of fear and arousal [33, 34]. Some studies however, have moved toward using more objective psychophysiological measures. Jang et al.  measured psychophysiological responses including skin resistance and heart rate variability to assess arousal levels in normal subjects exposed to fear of driving and fear of flying VEs. Subjects showed lowered levels of skin resistance compared to baseline, indicating higher levels of arousal, especially during the first 7 to 8 minutes of exposure to the VE. Other studies have also found VEs to be physiologically arousing [36–38].
Meehan et al.  sought to uncover a more objective, valid, and reliable measure of presence through psychophysiological metrics. Because psychophysiological responses can be made without consciousness of the response, the experience of the virtual environment, and feelings of presence, need not be interrupted. Meehan and colleagues found that heart rate and skin conductance increased along with increased feelings of presence. They conclude that psychophysiological measures may be utilized as an objective and reliable measure of presence, though they note that additional research using different environments and stressors is necessary to further elucidate these findings.
Psychophysiological metrics proffer the advantage of an objective measure of response that can be recorded in real-time as the environment is experienced, providing a continuous measure of presence. Indeed, highly immersive presentations are thought to not only increase subjective ratings, but also result in increased physiological responses . As such, researchers may study the impact of VEs on participants by looking at the psychophysiological responses of participants to their surroundings [6, 39, 40]. The recording of psychophysiological variables while participants operate within VEs has produced useful results in studies examining presence and immersion [41–44]. As such, the VE assets that allow for precise stimulus delivery within ecologically enhanced scenarios appears well matched for this research.
1.1. Current Study
In the current study, we aimed to look at the psychophysiological responses of participants experiencing “high” versus “low” levels of immersion into a virtual Iraqi scenario that had varying levels of stimulus intensity. Further, these psychophysiological responses may aid researchers in their development of VEs that balance fidelity and experimental control.
The primary goal in this study was evaluative: will a virtual environment with arousing contents result in increased sensory arousal if it is presented in a highly immersive configuration? To assess this, we looked at subjects immersed in a VE on two separate experimental runs consisting of both a “high” immersion condition and a “low” immersion condition. A secondary goal of this study was to investigate the utility of our environment to offer varying levels of stimulus threat to impact the user’s experience of the VE. Within each of the immersion conditions (high and low), arousal was manipulated by presenting participants with differing “safe” and “ambush” zones. Safe (low threat) zones consisted of little activity aside from driving down a desert road, while the more stressful ambush (high threat) zones included gunfire, explosions, and shouting amongst other stressors.
In the current study, startle eyeblink and heart rate were measured to assess psychophysiological differences in response to varying degrees of immersion and levels of arousal in a virtual Iraqi environment. Participants encountered a highly immersive VE while wearing a head mounted display (HMD) that used two OLED microdisplays with on-board 3D frame sequential video processors to deliver flicker-free motion. Together with the integrated , , and -axis head-tracker, the participant was able to look around the VE by turning his or her head left or right, up or down, and leaning forward or back. Also in the high immersion condition, to increase the potential for sensory immersion, the participant was seated on a tactile transducer. Although participants experienced the same VE content in the low immersion condition, the VE was presented on a computer screen. It was expected that the highly immersive condition would lead to an increased sensory arousal, thus resulting in augmented levels of psychophysiological responding.
We hypothesized that the highly immersive condition would elicit a more intense physiological response to the stressful high threat zones due to the higher-fidelity environment. It is important to note that the volume levels in both the high and low immersion conditions were held constant in order to increase experimental control of the arousal manipulation in this study and to limit changes in arousal to changes brought on by increased levels of fidelity, rather than changes in volume level. It is our hope that this research will (1) proffer a greater understanding of the psychophysiological correlates of immersion in an arousing VE and (2) act as an initial validation (ecological validation) of the intended impact of varying degrees of stimulus intensity programmed into our virtual Iraqi city.
1.3. Serious Games and Clinical Psychology
Serious game researchers are increasingly interested in working with clinicians to better understand a military service member’s ability to return to active duty. Recent conflicts have increased the prevalence of blast injuries to the head. Many of these brain injuries may have no external marker of injury. As a result, there is need for the serious games community to research innovative assessment methods. Currently, clinicians make “Return-to-Duty” assessments that are based upon the “Return-to-Play” guidelines found in Sports Medicine. Both have incorporated two dimensional cognitive assessments to aid in decisions related to resuming activities following a concussion. Unfortunately, these two dimensional computerized assessments were not developed with the intention of tapping into everyday behaviors like driving through a Middle Eastern city.
Serious gaming environments can increase the ecological validity of neurocognitive batteries through the use of simulation technologies for assessment and treatment planning. The success of such serious games may lead to a psychophysiological computing approach, in which such data gleaned from persons interacting within a military relevant simulation may be used to develop adaptive virtual environments for training and rehabilitation. A beginning step is the identification of the level of immersion needed for a serious game to proffer the appropriate level of arousal. This is the overarching goal of this study.
A total of 50 healthy college aged students (males: , mean age = 20.4, mean years of education = 14.6; females: , mean age = 19.8, mean years of education = 14.1) participated in this experiment. An interview with a psychologist and a mental health history form were completed with each participant in order to allow for the following of strict exclusion criteria to minimize the possible confounding effects of additional factors known to adversely impact a person’s ability to process information, including psychiatric (e.g., mental retardation, psychotic disorders, diagnosed learning disabilities, attention-deficit/hyperactivity disorder, and bipolar disorders, as well as substance-related disorders within two years of evaluation) and neurologic (e.g., seizure disorders, closed head injuries with loss of consciousness greater than 15 minutes, and neoplastic diseases) conditions.
The apparatus used for the virtual humvee (i.e., a high mobility multipurpose wheeled vehicle) included a Pentium 4 desktop computer with a 3 GHz Processor; 3 GB of RAM; and an nVidia GeForce 6800. Two monitors were used: (1) one for displaying the Launcher application which is used by the Examiner and (2) another for displaying the participant’s view of the VE in the HMD. Participants wore an eMagin Z800 head mounted display, and an InterSense InteriaCube 2+ attached for enhanced tracking. A Logitech Driving Force steering wheel was clamped on to the edge of a table in front of the monitors. A separate module consisting of the gas and brake pedals was positioned under the table. To increase the potential for sensory immersion, we built a tactile transducer using a three foot square platform with six Aura bass shaker speakers (AST-2B-04, 4 Ω 50 W Bass Shaker) attached. The tactile transducer was powered by a Sherwood RX-4105 amplifier with 100 Watts per Channel ×2 in Stereo Mode.
2.2.2. Virtual Environment
The software was designed using Virtual Battle Space 2 (VBS2). The VBS2 engine was used due to its robust fidelity simulation, ease of modification, and the fact that many military forces have adopted it. The VBS2 engine offers enhanced capability for interoperability and compatibility with existing standards for simulation. We designed the scenarios using a visual scenario editor and VBS2’s own scripting language. To implement the scenario we used VBS2-engine specific script language and the built-in Finite State Machine (FSM) functionality.
2.2.3. Neuroscience and Simulation Interface
The application uses the Neuroscience and Simulation Interface (NSI) developed in the Neuroscience and Simulation Laboratory (NeuroSim) at the University of Southern California . The NSI was used for data acquisition, stimulus presentation, psychophysiological monitoring, and communication between the psychophysiological system and the VE. Parameters were saved to files using the NSI and automatically loaded through its control module. The NSI allowed our system to switch between parameter files, or executables modules, in order to perform specific experimental sequences. The NSI enabled the sending of event markers from the stimulus presentation computer to a recording device. Matlab scripts were executed in real-time from within NSI and filters were compiled to execute as stand-alone programs. The software runs on Windows XP 32-bit, and requires 5 Gb of free hard drive space for installation and storage of user data.
2.3. Stimuli and Design
The University of Southern California’s Institutional Review Board approved the study. After informed consent was obtained, basic demographic information was obtained. Next, participants were immersed in a VE on two separate experimental runs consisting of both a “high” immersion condition and a “low” immersion condition. In the high immersion condition, participants wore a head mounted display (HMD) with full tracking capabilities and were free to explore their environment visually. The high immersion condition also made use of headphones and a tactile transducer floor to simulate the experience of a large vehicle. The low immersion condition consisted of the same virtual Iraqi scenario presented on a 17 inch laptop screen while wearing headphones. Stimuli within the virtual environment experienced in both immersion conditions were identical. The only differences between conditions were due to the inclusion of the enhanced presentation quality of the high immersion condition. The presentation order of high and low immersion conditions was counterbalanced across subjects.
The VE used in both immersion conditions was comprised of a series of low threat and high threat zones in a virtual Iraqi city. In both the high immersion and low immersion conditions, participants experienced the VE from the perspective of the driver of a Humvee. The speed of the vehicle was kept constant as it followed a predefined trajectory to control for time spent in each zone of the VE and to keep that time consistent across participants. Participants were given a basic 10° steering wheel to limit the trajectory, though they were instructed to stay on the road. This allowed for some level of control of the environment without sacrificing experimental control of the stimuli experienced. Low threat zones consisted mainly of a road surrounded by a desert landscape and were free of gunfire and other loud noises (see Figure 1). The high threat zones included improvised explosive devices (IEDs), gunfire, insurgents, and screaming voices (see Figure 2). The auditory background levels associated with the low threat and high threat zones were identical in both the high and low immersion conditions. Participants passed through three low threat and three high threat zones in an alternating sequence in both immersion conditions. Low threat zones were always experienced first and were used to allow the participant to habituate to the novelty of the virtual environment. High and low threat zones also varied in length, with low threat zones consistently lasting longer than high threat zones. High threat zones averaged 20 seconds in duration, while low threat zones averaged 50 seconds. This was to ensure that zone lengths were not predictable, and so that participants had ample time to return to low levels of responding after experiencing the highly arousing high threat zones. The total length of each run was 210 seconds.
An acoustic startle stimulus was used to elicit startle eyeblink responses. Following accepted guidelines for human startle eyeblink electromyographic studies , the startle stimulus was a 110 dB white noise burst 50 ms in duration with a near instantaneous rise/fall time presented binaurally through Telephonics TDH-50P headphones. Decibel levels were measured with a Realistic sound level meter using a Quest Electronics earphone coupler. Startle stimuli were experienced intermittently throughout the experimental runs. A total of four startle stimuli were experienced in both the low threat and high threat zones in each run.
2.4. Dependent Variables
Psychophysiological assessment included: startle eyeblink amplitude and heart rate, which were recorded simultaneously throughout the experiment using Contact Precision Instruments equipment and a computer running SAM1 software.
2.4.1. Startle Eyeblink Response
One psychophysiological measure employed in the current study, and that is widely used as an index of valence (e.g., emotional positive or negative reactions), is electromyographic (EMG) recording of the startle eyeblink reflex. This reflex is often elicited by a burst of loud white noise with a nearly immediate rise and fall time presented at very high decibel levels (e.g., 110 dB) for a brief duration (e.g., 50 ms). Vrana et al.  found that startle responses are facilitated when startle stimuli are presented in conjunction with a negative stimulus, and inhibited when presented with a positive stimulus relative to startle presentations with neutral stimuli. It is important to note that the positive and negative stimuli used in the Vrana et al. study were matched on subjective ratings of arousal, meaning the startle reflex can be a sensitive measure of valence. Relevant to the measurement of eyeblink responses during exposure to VEs, these findings have been replicated with moving film clips [48, 49].
Startle eyeblink responses were recorded as electromyographic activity using two small (4 mm in diameter) silver-silver chloride electrodes placed over the orbicularis oculi muscle of the left eye and an 8 mm silver-silver chloride electrode placed behind the left ear to serve as a ground. One 4 mm electrode was placed directly below the pupil in forward gaze while the other was placed about 1 cm lateral to the first. The electrodes were placed as close to the eye as possible while still allowing the participant to open and close his or her eyes comfortably. Impedance between the two electrodes was measured and deemed acceptable if below 10 kΩ.
2.4.2. Heart Rate
A second psychophysiological measure employed in the current study was the electrocardiographic (ECG) recording. Heart rate is a psychophysiological measure that is useful in differentiating between orienting and defensive responses. A person’s heart rate will accelerate during a defensive response and decelerate when orienting occurs . Numerous studies have found that phobic participants will exhibit an accelerated heart rate when viewing feared images, while control participants will respond with a deceleration to the same images . Thus, cardiovascular responding can be an informative measure when differentiating between the effectiveness of fear elicitation in VEs. An increase in heart rate during the high threat zones would thus likely be indicative of a defensive response caused by increased levels of fear experienced by the participant.
ECG was recorded with use of a Lead 1 electrode placement, with one 8 mm silver-silver chloride electrode placed on the right inner forearm about 2 cm below the elbow and another placed in the same position on the left inner forearm. Electrode sites were cleaned with rubbing alcohol in order to improve contact.
2.5. Data Analytics
2.5.1. Startle Eyeblink Response
The raw EMG signal was recorded at a rate of 1000 Hz throughout the experiment using a 10 Hz high pass and 200 Hz low pass filter. Raw signals were stored and exported for analysis in microvolt (μV) values. The raw EMG signal was rectified and integrated for analysis. In order to qualify for scoring, the eyeblink response had to begin within a window of 20 to 100 ms following the offset of the startle stimulus and reach peak activity within a window of 20 to 150 ms following the startle stimulus . Blinks occurring at longer latencies were not considered to be the result of the startle stimulus. Amplitudes were recorded as the difference between the peak activity value and the baseline level present immediately preceding onset of the blink response. If the participant was blinking during the onset of the startle stimulus, that blink response was removed from further analysis due to artifact. Participants who failed to reach 1 V amplitudes on greater than 50% of startled trials were considered nonresponders and were dropped from further EMG analyses. One participant reached this criterion, leaving 49 participants to be included in EMG analyses.
Due to the high levels of variability between participants in EMG responses, all blink amplitude values were standardized by taking the difference between each participant’s raw EMG amplitude value on each trial and that participant’s mean value across all trials and dividing by the standard deviation of all values. Scores were then subjected to a linear transformation resulting in a mean of 50 and a standard deviation of 10 for display purposes. This helped to ensure that all participants contributed to group means equally, minimizing the influence that one participant could have on the outcome of the subsequent analyses.
2.5.2. Heart Rate
Interbeat intervals (IBIs) were scored as the time difference in milliseconds between successive R waves in the ECG signal. IBIs across a period of 5 seconds during each high threat and low threat zone were analyzed. The 5 second period occurred at least 10 seconds following any startle stimulus or large explosion, and no startle stimuli or explosions occurred during the period. A mean IBI score was recorded for each 5 second period and analyzed.
For each dependent variable, a 2 (immersion level) by 2 (zone type) repeated measures analysis of variance (ANOVA) was utilized to determine whether the high immersion setting was effective in increasing psychophysiological responding in general and whether it affected participants differently in low threat versus high threat zones.
All significant main effects and interactions were followed with paired samples -tests in order to identify the precise nature of these effects. All reported significant -test results are corrected using a sequentially rejective test procedure based on a modified Bonferroni inequality to prevent inflation of type 1 error rates .
3.1. Startle Eyeblink Results
A significant immersion level main effect was uncovered, and was the result of increased blink amplitudes when participants were in the high immersion setting, F(1,48) = 16.34, = 0.49, = 0.38, = 0.33, . Zone type did not yield a significant main effect. The interaction between zone type and immersion level also failed to reach significance. However, a post hoc paired samples t-test revealed that the difference between the high and low immersion was only significant while participants experienced the high threat zones, t(48) = 3.82, . As can be seen in Figure 3 and Table 1, eyeblink amplitudes increased in the high threat zone only in the high immersion setting. Responses in the high threat zone of the high immersion setting were large enough to account for most of the immersion level main effect as differences between high and low immersion in the low threat zones were not significant after Rom correction.
|Quartile and range data are given for the entire sample, while separate median values are given for both the safe and ambush zones.|
3.2. Heart Rate Results
In general, ECG results were in agreement with EMG results. Again, a significant main effect of immersion level was found, F(1, 49) = 10.78, = 0.06, = 0.03, = 0.03, . This immersion level effect was the result of faster heart rates when participants were in the high immersion setting. As can be seen in Figure 4 and Table 2, participants were again evidencing increased responding during the high threat zones only when in the high immersion setting, although this increase in heart rate was not significant. No significant zone type main effect or interaction between immersion level and zone type existed. There was again a significant difference between the high and low immersion presentations only during the high threat zones, t(49) = 3.42, , as was the case in regards to the eyeblink results.
|Quartile and range data are given for the entire sample, while separate median values are given for both the safe and ambush zones.|
4.1. Primary Analysis: Effects of Immersion Level
For our primary analysis in this study we sought to evaluate whether a highly immersive environment results in increased sensory arousal as measured by psychophysiological responses. Immersion effects were consistent with each measure. Participants consistently had faster heart rates when in the high immersion setting, suggesting that highly immersive VEs are more arousing than experiencing the same presentation on a computer screen. Participants also had larger startle eyeblinks when highly immersed, especially during the high threat zones, which suggests that the high immersion format facilitated startle eyeblinks.
Although on first reading these results appear to reflect the possibility that highly immersive VEs are more effective for eliciting increased arousal and producing fear responses than are low immersion VEs, this conclusion cannot be generalized given that there are restorative virtual environments that decrease arousal [53, 54]. The fact is that this VE was a warzone simulation with varying levels of threat stimuli. Both of the dependent measures were shown to mainly vary with immersion, not threat.
Another area that may put our results at odds with those reported by others is the issue that our study was for neuroscientific assessment of varying levels of fidelity and threat in a nonclinical sample of healthy college age students. Clinical populations tend to have significantly greater responding to threat stimuli presented in VEs when compared to nonclinical populations. For example, virtual stimuli that are relevant to a given phobia (e.g., phobics respond with more anxiety to phobogenic stimuli) will have more robust reactions to threatening stimuli. Further, it also seems intuitively clear that participants in the current study would react less to the threatening zones than would persons sensitive to the content of the virtual Iraq (e.g., soldiers returning from a rotation in Iraq, suffering from PTSD, or having been in a war zone) .
4.2. Secondary Analysis: Effects of Zone Type
A secondary goal of this study was to investigate the utility of our environment to offer varying levels of stimulus threat to impact the user’s experience of the VE. Our analysis revealed that high threat zones were ineffective in creating statistically significant increases in arousal levels compared to the low threat zones, according to eyeblink and heart rate responses. However, participants appeared to show the appropriate directional trend toward increased heart rate and eyeblink responding in the high immersion setting, lending credence to the notion that the high immersion setting may be more effective in creating differential responding between the two zone types. However, these trends in response did not lead to significant interactions between immersion level and zone type.
4.3. Enhancing the Virtual Environment
The lack of differential responding in the high threat and low threat zones may have been due to the fixed order of presentation. While the presentation of the low and high immersion settings was counterbalanced across participants, the order of the zones was not. This meant that in each pair of low threat and high threat zones, the low threat zone was experienced first. While it is impossible to know what the exact effects of a counterbalanced presentation order would have on psychophysiological response, one possible explanation for the lack of differential responding may have been caused by habituation that led to a general decrease in responding during the high threat zones in comparison to the low threat zones that always preceded them. Had the high threat zones occurred prior to the low threat zones, a greater difference between the different types of zones may have been revealed, especially in the high immersion setting.
Additionally, the low threat zones were generally longer in duration than the high threat zones. This may have led to greater habituation taking place during the low threat zones, and created an additional confound that is difficult to account for in participant responses. Moreover, the low threat zones would transition into the high threat zones unpredictably and without warning, making the low threat zones potentially threatening.
The presentation of startle stimuli may also have added to the lack of differential responding in the low threat and high threat zones. In order to make the startle stimuli stand out from the background noise in the environment enough to elicit a startle response, the maximum capacity of the environmental noises were reduced to ten percent of the startle stimulus volume, greatly lowering the potentially arousing effects of gunshots and explosions experienced in the high threat zones.
It is important to note that there is parallel research on the restorative effects of nature that has explored the relationship between presence/immersion, psychophysiological measurements, and virtual reality. Previous research examining whether immersion in a VE simulated nature setting could produce restorative effects found that immersion in virtual nature settings has similar beneficial effects as exposure to surrogate nature. These results also suggest that VR can be used as a tool to study and understand restorative effects [53, 54].
4.4. Future Directions
Future studies using this VE may be enhanced through counterbalancing of the order of zones experienced in the VE. Counterbalancing across participants to allow for half to experience low threat zones first and half to experience high threat zones first should help to alleviate the possible order effects that occurred in the present study. In order to better understand which particular zone is the most effective in increasing arousal, it is important that the high and low immersion conditions can begin with any zone. We can then counterbalance whether a low threat zone or a high threat zone is experienced first, and which particular low threat or high threat zone is experienced first. A uniform amount of time spent in each zone will also help to control the effects of habituation from zone to zone. Furthermore, in order to make the low threat zones more clearly perceived as being safe, a cue could be given to warn the user of the impending high threat zone. This way, the low threat zones are clearly separated from the high threat zones.
The removal of startle stimuli to allow background environmental noises to be played at one hundred percent capacity may also be beneficial in creating more arousing high threat zones. Eyeblink responses will no longer be an option as a psychophysiological measure of valence in a noisy background environment, but facial corrugator EMG recording can be used as an index of perceived valence in its stead. Other metrics such as electrodermal activity, respiration, and blood pressure may also be useful measures of arousal, and responses would most likely be enhanced by the increased volume levels.
A further enhancement for future studies would be the addition of subjective evaluations. Having both subjective and objective information would strengthen the validity of the results and allow combining them for the conclusions . The inclusion of the subjective data may have aided this study through greater explanatory power for the nonsignificant but apparent difference between low/high threat in the eyeblink responses found in low immersion.
One of the main goals of the present research was to assess whether a VE with arousing contents would result in increased sensory arousal if it is presented in a highly immersive configuration. A secondary goal of this study was to investigate the utility of our environment to offer varying levels of stimulus threat to impact the user’s experience of the VE. Increased simulation fidelity in an arousing VE resulted in faster heart rates and increased startle eyeblink amplitudes, suggesting that higher fidelity scenarios with threatening contents were related to sensory arousal. Hence, highly immersive VEs appear to be more effective for eliciting increased arousal and producing fear responses than are low immersion VEs.
Conflict of Interests
No financial Conflict of interests exist for any of the authors of this paper.
This research is partially supported by the US Army Research Laboratory, Human Research & Engineering Directorate, Translational Neuroscience Branch (Aberdeen Proving Ground, MD).
- A. Gorini and G. Riva, “Virtual reality in anxiety disorders: the past and the future,” Expert Review of Neurotherapeutics, vol. 8, no. 2, pp. 215–233, 2008.
- T. D. Parsons and A. A. Rizzo, “Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis,” Journal of Behavior Therapy and Experimental Psychiatry, vol. 39, no. 3, pp. 250–261, 2008.
- M. B. Powers and P. M. G. Emmelkamp, “Virtual reality exposure therapy for anxiety disorders: a meta-analysis,” Journal of Anxiety Disorders, vol. 22, no. 3, pp. 561–569, 2008.
- T. D. Parsons, “Neuropsychological assessment using virtual environments: enhanced assessment technology for improved ecological validity,” in Advanced Computational Intelligence Paradigms in Healthcare: Virtual Reality in Psychotherapy, Rehabilitation, and Assessment, S. Brahnam, Ed., pp. 271–289, Springer, Germany, 2011.
- T. D. Parsons, A. A. Rizzo, S. Rogers, and P. York, “Virtual reality in paediatric rehabilitation: a review,” Developmental Neurorehabilitation, vol. 12, no. 4, pp. 224–238, 2009.
- L. Pugnetti, M. Meehan, and L. Mendozzi, “Psychophysiological correlates of virtual reality: a review,” Presence: Teleoperators and Virtual Environments, vol. 10, no. 4, pp. 384–400, 2001.
- C. G. Courtney, M. E. Dawson, A. M. Schell, A. Iyer, and T. D. Parsons, “Better than the real thing: eliciting fear with moving and static computer-generated stimuli,” International Journal of Psychophysiology, vol. 78, no. 2, pp. 107–114, 2010.
- T. D. Parsons and J. Reinebold, “Adaptive virtual environments for neuropsychological assessment in serious games,” IEEE Transactions on Consumer Electronics, vol. 58, pp. 197–204, 2012.
- J. Allanson and S. H. Fairclough, “A research agenda for physiological computing,” Interacting with Computers, vol. 16, no. 5, pp. 857–878, 2004.
- R. W. Picard, Affective Computing, MIT Press, Cambridge, Mass, USA, 1997.
- R. Parasuraman and G. F. Wilson, “Putting the brain to work: neuroergonomics past, present, and future,” Human Factors, vol. 50, no. 3, pp. 468–474, 2008.
- T. S. Langhan, “Simulation training for emergency medicine residents: time to move forward,” Canadian Journal of Emergency Medicine, vol. 10, no. 5, pp. 467–469, 2008.
- M. Slater, P. Khanna, J. Mortensen, and I. Yu, “Visual realism enhances realistic response in an immersive virtual environment,” IEEE Computer Graphics and Applications, vol. 29, no. 3, pp. 76–84, 2009.
- M. V. Sanchez-Vives and M. Slater, “From presence to consciousness through virtual reality,” Nature Reviews Neuroscience, vol. 6, no. 4, pp. 332–339, 2005.
- M. R. Banaji and R. G. Crowder, “The bankruptcy of everyday memory,” American Psychologist, vol. 44, no. 9, pp. 1185–1193, 1989.
- T. D. Parsons, T. Bowerly, J. G. Buckwalter, and A. A. Rizzo, “A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods,” Child Neuropsychology, vol. 13, no. 4, pp. 363–381, 2007.
- M. Slater, “Presence 2005,” in Proceedings of the 8th International Workshop on Presence, Department of Computer Science, University College London, London, UK, 2005.
- C. Dillon, E. Keough, J. Freeman, and J. Davidoff, “Aroused and immersed: the psychophysiology of presence,” in Proceedings of the 3rd International Workshop on Presence, pp. 27–28, Delft University of Technology, Delft, The Netherlands, 2000.
- J. V. Draper, D. B. Kaber, and J. M. Usher, “Telepresence,” Human Factors, vol. 40, no. 3, pp. 354–375, 1998.
- N. Schwarz, “How the questions shape the answers,” American Psychologist, vol. 54, no. 2, pp. 93–105, 1999.
- M. Slater and S. Wilbur, “A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments,” Presence: Teleoperators and Virtual Environments, vol. 6, no. 6, pp. 603–616, 1997.
- M. Slater, “Measuring presence: a response to the Witmer and Singer presence questionnaire,” Presence-Teleoperators and Virtual Environments, vol. 8, pp. 560–565, 1999.
- R. M. Baños, C. Botella, A. Garcia-Palacios, H. Villa, C. Perpiña, and M. Alcañiz, “Presence and reality judgment in virtual environments: a unitary construct?” Cyberpsychology and Behavior, vol. 3, no. 3, pp. 327–335, 2000.
- J. Lessiter, J. Freeman, E. Keogh, and J. Davidoff, “A cross-media presence questionnaire: the ITC-sense of presence inventory,” Presence: Teleoperators and Virtual Environments, vol. 10, no. 3, pp. 282–297, 2001.
- M. Usoh, E. Catena, S. Arman, and M. Slater, “Using presence questionnaires in reality,” Presence: Teleoperators and Virtual Environments, vol. 9, no. 5, pp. 497–503, 2000.
- B. G. Witmer and M. J. Singer, “Measuring presence in virtual environments: a presence questionnaire,” Presence: Teleoperators and Virtual Environments, vol. 7, no. 3, pp. 225–240, 1998.
- M. Gordon, R. A. Barkley, and B. J. Lovett, “Tests and observational measures,” in Attention-Deficit Hyperactivity Disorder: A Handbook for Diagnosis and Treatment, R. A. Barkley, Ed., pp. 369–388, Guilford, New York, NY, USA, 3rd edition, 2006.
- R. M. Baños, C. Botella, M. Alcañiz, V. Liaño, B. Guerrero, and B. Rey, “Immersion and emotion: their impact on the sense of presence,” Cyberpsychology and Behavior, vol. 7, no. 6, pp. 734–741, 2004.
- M. Meehan, S. Razzaque, B. Insko, M. Whitton, and F. P. Brooks, “Review of four studies on the use of physiological reaction as a measure of presence in stressful virtual environments,” Applied Psychophysiology Biofeedback, vol. 30, no. 3, pp. 239–258, 2005.
- T. Baumgartner, L. Valko, M. Esslen, and L. Jäncke, “Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study,” Cyberpsychology and Behavior, vol. 9, no. 1, pp. 30–45, 2006.
- M. Alcañiz, B. Rey, J. Tembl, and V. Parkhutik, “A neuroscience approach to virtual reality experience using transcranial Doppler monitoring,” Presence: Teleoperators and Virtual Environments, vol. 18, no. 2, pp. 97–111, 2009.
- T. Baumgartner, D. Speck, D. Wettstein, O. Masnari, G. Beeli, and L. Jäncke, “Feeling present in arousing virtual reality worlds: Prefrontal brain regions differentially orchestrate presence experience in adults and children,” Frontiers in Human Neuroscience, vol. 2, article 8, 2008.
- S. L. Calvert and S. L. Tan, “Impact of virtual reality on young adults' physiological arousal and aggressive thoughts: interaction versus observation,” Journal of Applied Developmental Psychology, vol. 15, no. 1, pp. 125–139, 1994.
- B. R. Cornwell, L. Johnson, L. Berardi, and C. Grillon, “Anticipation of public speaking in virtual reality reveals a relationship between trait social anxiety and startle reactivity,” Biological Psychiatry, vol. 59, no. 7, pp. 664–666, 2006.
- D. P. Jang, I. Y. Kim, S. W. Nam, B. K. Wiederhold, M. D. Wiederhold, and S. I. Kim, “Analysis of physiological response to two virtual environments: driving and flying simulation,” Cyberpsychology and Behavior, vol. 5, no. 1, pp. 11–18, 2002.
- K. Elsesser, I. Heuschen, I. Pundt, and G. Sartory, “Attentional bias and evoked heart-rate response in specific phobia,” Cognition and Emotion, vol. 20, no. 8, pp. 1092–1107, 2006.
- F. K. Graham and R. K. Clifton, “Heart-rate change as a component of the orienting response,” Psychological Bulletin, vol. 65, no. 5, pp. 305–320, 1966.
- H. Kaviani, J. A. Gray, S. A. Checkley, Veena Kumari, and G. D. Wilson, “Modulation of the acoustic startle reflex by emotionally-toned film- clips,” International Journal of Psychophysiology, vol. 32, no. 1, pp. 47–54, 1999.
- M. Meehan, B. Insko, M. Whitton, and F. P. Brooks, “Physiological measures of presence in stressful virtual environments,” Acm Transactions on Graphics, vol. 21, pp. 645–652, 2002.
- J. M. Flach and J. G. Holden, “The reality of experience: Gibson's way,” Presence: Teleoperators and Virtual Environments, vol. 7, no. 1, pp. 90–95, 1998.
- L. W. Jerome and P. J. Jordan, “Psychophysiological perspective on presence: the implications of mediated environments on relationships, behavioral health and social construction,” Psychological Services, vol. 4, no. 2, pp. 75–84, 2007.
- M. F. Macedonio, T. D. Parsons, R. A. Digiuseppe, B. K. Weiderhold, and A. A. Rizzo, “Immersiveness and physiological arousal within panoramic video-based virtual reality,” Cyberpsychology and Behavior, vol. 10, no. 4, pp. 508–515, 2007.
- T. D. Parsons, A. Iyer, L. Cosand, C. Courtney, and A. A. Rizzo, “Neurocognitive and psychophysiological analysis of human performance within virtual reality environments,” Studies in Health Technology and Informatics, vol. 142, pp. 247–252, 2009.
- B. K. Wiederhold and A. Rizzo, “Virtual reality and applied psychophysiology,” Applied Psychophysiology Biofeedback, vol. 30, no. 3, pp. 183–185, 2005.
- T. D. Parsons and C. Courtney, “Neurocognitive and Psychophysiological Interfaces for Adaptive Virtual Environments,” in Human Centered Design of E-Health Technologies, C. Röcker T and M. Ziefle, Eds., pp. 208–233, IGI Global, Hershey, Pa, USA, 2011.
- T. D. Blumenthal, B. N. Cuthbert, D. L. Filion, S. Hackley, O. V. Lipp, and A. Van Boxtel, “Committee report: guidelines for human startle eyeblink electromyographic studies,” Psychophysiology, vol. 42, no. 1, pp. 1–15, 2005.
- S. R. Vrana, E. L. Spence, and P. J. Lang, “The startle probe response: a new measure of emotion?” Journal of Abnormal Psychology, vol. 97, no. 4, pp. 487–491, 1988.
- P. M. G. Emmelkamp, M. Krijn, A. M. Hulsbosch, S. De Vries, M. J. Schuemie, and C. A. P. G. Van der Mast, “Virtual reality treatment versus exposure in vivo: a comparative evaluation in acrophobia,” Behaviour Research and Therapy, vol. 40, no. 5, pp. 509–516, 2002.
- D. M. Jansen and N. H. Frijda, “Modulation of the acoustic startle response by film-induced fear and sexual arousal,” Psychophysiology, vol. 31, no. 6, pp. 565–571, 1994.
- A. S. Carlin, H. G. Hoffman, and S. Weghorst, “Virtual reality and tactile augmentation in the treatment of spider phobia: a case report,” Behaviour Research and Therapy, vol. 35, no. 2, pp. 153–158, 1997.
- L. F. Hodges, B. A. Watson, G. D. Kessler, B. O. Rothbaum, and D. Opdyke, “Virtually conquering fear of flying,” IEEE Computer Graphics and Applications, vol. 16, no. 6, pp. 42–49, 1996.
- D. M. Rom, “A sequentially rejective test procedure based on a modified bonferroni inequality,” Biometrika, vol. 77, no. 3, pp. 663–665, 1990.
- Y. A. W. de Kort, A. L. Meijnders, A. A. G. Sponselee, and W. A. IJsselsteijn, “What's wrong with virtual trees? Restoring from stress in a mediated environment,” Journal of Environmental Psychology, vol. 26, no. 4, pp. 309–320, 2006.
- D. Valtchanov, K. R. Barton, and C. Ellard, “Restorative effects of virtual nature settings,” Cyberpsychology, Behavior, and Social Networking, vol. 13, no. 5, pp. 503–512, 2010.
- A. A. Rizzo, K. Graap, K. Perlman et al., “Virtual Iraq: initial results from a VR exposure therapy application for combat-related PTSD,” Studies in health technology and informatics, vol. 132, pp. 420–425, 2008.
- L. L. Di Stasi, L. L. Di Stasi, R. Renner et al., “Saccadic peak velocity sensitivity to variations in mental workload,” Aviation Space and Environmental Medicine, vol. 81, no. 4, pp. 413–417, 2010.
Copyright © 2012 Thomas D. Parsons et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.