Immersion in Computer Games: The Role of Spatial Presence and Flow

Weibel, David; Wissmath, Bartholomäus

doi:https://doi.org/10.1155/2011/282345

International Journal of Computer Games Technology

On this page

Abstract Introduction Discussion Conclusion References Copyright Related Articles

Research Article | Open Access

Volume 2011 | Article ID 282345 | https://doi.org/10.1155/2011/282345

Immersion in Computer Games: The Role of Spatial Presence and Flow

David Weibel¹and Bartholomäus Wissmath¹

Academic Editor: Daniel Thalmann

Received24 Aug 2011

Revised30 Nov 2011

Accepted12 Dec 2011

Published05 Jan 2012

Abstract

A main reason to play computer games is the pleasure of being immersed in a mediated world. Spatial presence and flow are considered key concepts to explain such immersive experiences. However, little attention has been paid to the connection between the two concepts. Thus, we empirically examined the relationship between presence and flow in the context of a computer role-playing game (), a racing game (), and a jump and run game (). In all three studies, factor analysis revealed that presence and flow are distinct constructs, which do hardly share common variance. We conclude that presence refers to the sensation of being there in the mediated world, whereas flow rather refers to the sensation of being involved in the gaming action. Further analyses showed that flow and presence depend on motivation and immersive tendency. In addition, flow and presence enhanced performance as well as enjoyment.

1. Introduction

In a broad survey, Yee [1] assessed motivations and experiences of 30,000 game users. He found that many users play computer games because they like to be immersed in a fantasy world. Immersion into mediated environments in general and computer games in particular has previously been explained through (spatial) presence (e.g., [2, 3]) and flow (e.g., [4–6]). Flow and presence share conceptual similarities such as an immersive component and intense feelings of involvement (cf. [7, 8]), but there are clear differences (cf. [6, 9]) whereas flow can be defined as immersion or involvement in an activity (i.e., the gaming action), presence rather refers to a sense of spatial immersion in a mediated environment. In the last decades, a plethora of studies have been conducted to investigate presence and flow in various media contexts. Surprisingly, however, there is not one single comprehensive empirical study that investigates the relationship between the two concepts. Within the study in hand, we therefore aim to empirically analyse the relationship between flow and (spatial) presence. In order to validate differences and similarities, we accomplished three large-scale experiments using three different computer games.

Since presence as well as flow can be described as immersive experiences, Draper et al. [7] suggest that presence is a particular type of flow experience that occurs during teleoperations. This is in line with Bystrom et al. [10] who assume that presence—just like flow—occurs due to a feedback loop between task characteristics and attention allocation. Alike, Hoffman and Novak [8] propose that attention and involvement are essential components of the sensation of presence as well as the experience of flow. Despite these conceptual similarities, both concepts are used independently to describe and explain immersive experiences in the context of media use.

Furthermore, we are interested whether and to what extent certain personal characteristics play a crucial role in both, presence and flow. Previous studies suggest that the users’ motivation and immersive tendency—as proposed by Witmer and Singer [11]—predict immersive experiences. The influence of immersive tendency on presence has been shown by Weibel et al. [12], the influence of motivation on flow was found by Engeser et al. [13]. However, it is not yet clear whether these two variables also exert a strong influence in the context of computer games. Additionally, the relationships between motivation and presence as well as between immersive tendency and flow have not yet been examined.

To provide even deeper insights into the role of presence and flow in the context of computer games, we intend to shed light on the question whether presence and flow influence enjoyment and performance. Existing literature speculates that these two variables depend on both, flow and presence (e.g., [2–4, 14]). To date, however, there still is a lack of compelling empirical evidence. To our knowledge, there is no empirical study investigating the influence of flow and presence on enjoyment and performance in the context of computer games. Also, the relation between presence and performance is so far mere speculation and has not yet been investigated. We feel that investigating all variables together may help understanding the bigger picture of the processes involved in computer gaming and to identify possible mediating effects.

As aforementioned, we conducted three studies using different computer game settings. Study 1 was accomplished in the context of a computer role-playing game. In order to replicate the findings, we conducted a second study using a racing game running on Sony Playstation 3. In order to further enhance external validity and to investigate whether the relation between presence and flow depends on the vividness of the virtual environment, we conducted a third study within the jump and run game Sonic the Hedgehog. This game lacks of vividness and portrays the elements of the virtual world rather cartoon-like and less realistically. Thus, it seems less ideal to induce presence compared to the racing game and the role-playing game.

2. Theoretical Considerations

2.1. Spatial Presence

In the recent decades, technological development formed the basis for a completely new experience. The sensation of being spatially present at remote places displayed by technical interfaces. Presence, also referred to as telepresence or spatial presence, describes a state of consciousness that gives the impression of being physically present in a mediated world. According to Steuer [15], (tele)presence is the extent to which an individual feels present in the mediated environment, rather than in the immediate physical environment. Mediated contents become real and one’s self-awareness is immersing into another world [7]. Slater and Wilbur [16] define this experience as a “state of consciousness, the (psychological) sense of being in the virtual environment” (page 604). According to Lombard and Ditton [2], presence is a perceptual illusion of nonmediation. A more recent approach was proposed by Wirth et al. [3]. Their two-level process model of spatial presence suggests attention allocation and the establishment of a mental model of the mediated environment to be prerequisite conditions for the sensation of presence. Once these preconditions are met, the individual-mediated environment will define the primary subjective frame of reference.

Due to enhanced vividness and new ways of interaction, virtual reality (VR) technology including computer games are assumed to elicit stronger presence experiences than more traditional media offerings such as TV [17]. In accordance, Tamborini [18] points out the capability of computer games to induce feelings of immersion and involvement, which are both essential components of presence [11].

2.2. Flow

The holistic experience of flow was first proposed by Csikszentmihalyi in 1975 [19]. Csikszentmihalyi describes flow as a mental state of operation in which a person is fully immersed in what he/she is doing. This gratifying mental state is characterized by a feeling of energized focus, full involvement, and success in the process of the activity. The concept suggests that the psychological experiences of various leisure activities such as rock climbing, dancing, or chess playing have several common dimensions. The characteristics of such intrinsically rewarding flow experiences are intense involvement, clarity of goals and feedback, concentrating and focusing, lack of self-consciousness, distorted sense of time, balance between the challenge and the skills required to meet it, and finally the feeling of full control over the activity [14].

An issue of recent research is the investigation of flow experience in the context of computer use (e.g., [4, 5, 20–23]). An adaptation of Csikszentmihalyi’s concept of flow with regard to the specific experiences of human-computer interactions was assessed by Rheinberg et al. [24, 25]. In a factor analytical approach, they found that the construct contains the two dimensions: (1) smooth and automatic running and (2) absorption. The first factor refers to the feeling of utmost concentration and focusing, control over the activity, clarity of the operations, and smooth and automatic cogitations. The second factor refers to the feeling of full involvement, distorted sense of time, optimal challenge, and absent mindedness.

Preliminary research on flow and computer games suggests that the psychological experience of gaming is consistent with the dimensions of flow experiences as outlined by Csikszentmihalyi and Rheinberg and Engeser. Thus, the concept of flow “form(s) the basis of the psychological presence of gamers within the game and provides a useful framework” (paragraph 3, Psychological Presence section, [20]). The importance of flow in the context of media consumption was also pointed out by Sherry [4], who argues that media enjoyment is after all the result of flow experiences. According to Sherry, flow experiences especially occur while playing computer and video games: “video games possess ideal characteristics to create and maintain flow experiences in that the flow experience of video games is brought on when the skills of the player match the difficulty of the game” (page 328). In line with these assumptions, Voiskounsky et al. [5] as well as Rheinberg and Vollmeyer [22] experimentally demonstrated that players of Multi-User Dungeons experience high levels of flow. Voiskounsky et al. as well as Klimmt [26] propose flow to be one the main sources of the attractiveness of computer games.

2.3. Presence and Flow as Process Variables in Media Use

Presence and flow describe some sort of immersive experiences and both are process variables referring to the actual media use. According to Draper et al. [7], presence can be defined as a special type of flow experience that occurs during teleoperations. Bystrom et al. [10] assume a feedback loop between task characteristics, attention allocation, and presence. They consider this loop to be similar to the flow concept. In 1996, Hoffman and Novak [27] hypothesized that within computer-mediated environments, presence leads to more attention and more flow. Novak et al. [28] supported this assumption. In a compelling online environment, they found that presence and flow correspond. A positive correlation between presence and flow was also found by Weibel et al. [6] in the context of a computer game. The similarities of flow and presence are also pointed out by Fontaine [9] who states that the flow experience produces peaks of involvement that seem to be similar to the “vividness” of presence.

Yet, not only similarities but also differences between the two concepts are discussed (e.g., [6, 9]). As aforementioned, presence has often been described as immersion into a virtual environment; in contrast, flow rather refers to an experience of being involved in an action (cf. [6]). The concept of flow focuses more on the task characteristics, while the concept of presence is more focused on technological characteristics of a medium. Flow could thus be described as immersion into an activity (i.e., the gaming action), whereas presence rather refers to a sense of spatial immersion in a mediated world. Fontaine describes this distinction as follows: “flow involves a narrow focus on a limited range of task characteristics, whereas presence involves a broader awareness of the task ecology” (p. 485). Furthermore, Fontaine states that flow is associated with feelings of control, whereas presence is rather associated with novel ecologies that lack of predictability. According to Fontaine, it is these differences that make flow a state of consciousness most suitable for performance in familiar ecologies and presence rather in unfamiliar ones.

As mentioned above, theoretical articles suggest differences as well as similarities between presence and flow. To date, there is a lack of empirical evidence concerning the connection between the two concepts. Therefore, our first objective is to answer the question whether and how the presence and flow are related to each other in the context of computer games.

2.4. The Influence of Motivation and Immersive Tendency on Flow and Presence

Lombard and Ditton [2] assume that certain traits facilitate the occurrence of presence experiences. Accordingly, Witmer and Singer [11] propose a disposition of individuals—immersive tendency—which influences whether someone experiences presence. According to Witmer and Singer, someone who scores high on immersive tendency has the capability to become involved in situations, shows a tendency to maintain focus on current activities, and generally likes playing video games. Furthermore, they propose that this trait relates to involvement in tasks. Thus, immersive tendency might not only influence presence, but also flow.

A further personal factor discussed in the literature is the user’s motivation. According to Lombard and Ditton [2] and Wirth et al. [3], motivation is a prerequisite to experience presence. To date, convincing empirical evidence that proves these considerations is missing. Csikszenthimalyi believes flow and motivational factors to be connected. Accordingly, Engeser et al. [13] showed that students with higher motivation reported stronger flow experiences during statistic lectures. Weibel et al. [29] replicated these findings within an e-learning environment. Thus, the motivation of an individual seems—at least to a certain amount—to determine the level of presence and flow.

Taken together, we assume that personal traits in terms of immersive tendency and motivation at least partly determine the degree of presence and flow.

2.5. The Influence of Flow and Presence on Enjoyment and Performance

There seems to be a connection between presence and enjoyment (e.g., [6, 30]). Wirth et al. [3] assume that “spatial presence can intensify existing media effects such as enjoyment” (page 495). Accordingly, Teng [31] proposes that immersion in games is pleasurable and does satisfying the user’s need. Since flow is a gratifying experience, flow and enjoyment should also be related. Sherry [4] as well as Weber et al. [32] suggest that media enjoyment after all results from flow experiences. Correspondingly, Ghani and Deshpande [21] could experimentally demonstrate the positive connection between flow and enjoyment.

Performance is yet another factor that is assumed to be influenced by presence and flow. Witmer and Singer [11] as well as Lombard and Ditton [2] assume a positive correlation between presence and task performance. In an experimental setting, Sheridan [33] showed that participants scoring high on presence indeed performed better. Csikszentmihalyi [14] states that the balance between challenges and required skills is a central aspect of flow experiences. He assumes that flow rather occurs if this balance is reached on a superior performance level. Thus, flow should lead to higher performance. Engeser et al. [13] could prove this assumption in the context of a statistic lecture, Weibel et al. [29] within an e-learning environment. In both studies, students scoring higher on flow during the lectures performed better in the subsequent test. Thus, we expect that presence and flow positively influence enjoyment and performance.

2.6. Summary of the Expected Relationships

To sum up, existing literature suggests that the actual media use in terms of presence and flow is determined by the personal characteristics immersive tendency and motivation. In turn, presence and flow are assumed to enhance enjoyment as well as performance. Our second aim—besides testing the relation between presence and flow—is to empirically investigate this relationship in three studies within three different computer games. The expected relations are shown in Figure 1.

3. Study 1

3.1. Method

3.1.1. Participants

A total of 70 students of a large public university in Switzerland participated. All of them were undergraduate students enrolled in psychology. Thirty-seven females and 33 males took part in the experiment. The average age was 23.9 years (SD = 5.42). All participants received a credit in their introductory courses for participating. All participants were treated according to the declaration of Helsinki [34].

3.1.2. Design and Statistical Methods

Before starting the experiment, we assessed the participants’ motivation and their immersive tendency. The participants’ spatial presence, flow, and enjoyment were captured expost. Furthermore, we measured the gaming performance.

We examined the dimensionality of presence and flow by means of factor analysis. To investigate the relationship between personal characteristics, actual media use, and outcome variables, a path analysis was conducted.

3.1.3. Stimulus Material

The computer role-playing game Neverwinter Nights produced by BioWare [35] was used. It is setting is a medieval fantasy world, a player is traveling through. There, an adventurous mission has to be completed. We divided the game into two parts. In the first part, the participants had to travel through the virtual world with the mission to find as many so-called rubies as possible. These rubies were said to provide power. The duration of this part of the game was exactly 15 minutes. After the expiration of the time the avatars were automatically transported into an arena where they had to compete against an opponent over five rounds. For reasons of standardization, the combat itself was programmed in a way that all players lost the final combat in the fifth round.

The game was played on a desktop system. Thereby, we used a 46′′ LCD large-screen television.

3.1.4. Measurements

All data besides performance were captured by means of self-report. The subjects rated all items on five-point Likert scales.

Immersive Tendency
The immersive tendency was assessed before the experiment started. We used the immersive tendency questionnaire by Witmer and Singer [11]. The questionnaire consists of 29 items (example item: “how frequently do you find yourself closely identifying with the characters in a story line?”). According to Witmer and Singer the reliability of the scale is between 0.75 and 0.81.

(Pre)Motivation
The motivation was measured with one single item. The participants were asked the following question before playing the computer game: “How motivated are you to play the game?” (1 = not at all; 5 = very much).

Presence
We used the presence scale by Kim and Biocca [36]. Kim and Biocca define the term presence in line with Lombard and Ditton as a sense of (spatially) being there in a mediated environment. The questionnaire was originally designed to measure presence in the context of televised media contents and was also used to assess presence in the context of computer games. The scale had been used in previous studies and turned out to be valid and reliable (e.g., [6, 28, 37]). The scale consists of eight items (example item: “the computer game came to me and created a new world for me, and the world suddenly disappeared when the game ended”). The authors do not mention the scale’s reliability. In our study, the scale was reliable (Cronbach’s alpha = .75).

Flow
Flow was assessed using the flow short scale by Rheinberg et al. [25] (example item: “I was totally absorbed in what I am doing”). This scale was previously used in the context of computer games (e.g., [6, 22]) and turned out to be a valid and reliable. According to the authors, the consistency of the scale (Cronbach’s alpha) is around 0.90.

Enjoyment
In line with various other studies [6, 38, 39], we measured enjoyment with one single item: “Did you enjoy the game?” (1 = not at all; 5 = very much).

Performance
The participants had to travel through the virtual world in the first part of the game. Their mission was to find as many rubies as possible. The performance measure was the amount of rubies found.

3.2. Results

3.2.1. Reliabilities

Sufficient reliabilities were revealed for presence (Cronbach’s alpha = .75), immersive tendency (Cronbach’s alpha = .74), and the flow scale (Cronbach’s alpha = .84).

3.2.2. Descriptives

Table 1 shows the intercorrelations between the measured variables including means and standard deviations. Individuals rated all items on five-point Likert scales. A value of 1 indicates the lowest, and 5 the highest possible value. The mean performance reflects the average amount of collected rubies. These findings show that overall presence, flow, and enjoyment ratings were medium.

3.2.3. Dimensionality of Flow and Presence: Factor Analysis

Presence correlates positively with flow, , , but how are these concepts related to each other? To examine the dimensionality of presence and flow, we additionally conducted a factor analysis. Our focus lied on identifying the underlying variables, concepts, and items. Therefore, a principal axis factor analysis was conducted. First, we tested if the output variables belong together. The measure of sampling adequacy (MSA) criterion is acceptable (MSA = .78) (cf. [40]). The scree test suggests a two-factor-solution. The two-factor solution suggested by the scree-test explains 38.7 percent of variance. The varimax rotation extracts a first factor, which explains 22 percent of variance. It consists of eight out of ten flow-items, which load higher than .4 on the particular factor. The second factor which explains 16.7 percent of the variance mainly consists of four out of eight items of the presence scale. No items of different constructs load high on the same factor. Table 2 shows these findings.

In order to further test construct validity, we additionally computed a confirmatory factor analysis. Thereby, we tested the solution as suggested by the exploratory factor analysis. The findings show that the data fitted the model well (; ; ; ; ; ). Furthermore all path coefficients turned out to be significant. Thus, the results of confirmatory factor analysis confirm the two-factor structure.

3.2.4. Relationship between Personal Characteristics, Actual Media Use, and Outcome Variables: Path Analysis

To test the relationship between personal characteristics (immersive tendency and motivation), actual media use (presence and flow), and outcome variables (enjoyment and performance), a path analysis was conducted. Thereby, we tested the model proposed in the introduction section. Figure 2 shows the resulting model. Nonsignificant paths were removed from this model. The indices suggest a good fit (; ; ; ; ; ).

All paths are significant on the 0.01 level except the path between presence and flow which is significant on the 0.05 level. The two exogenous variables immersive tendency and motivation are uncorrelated, , . As expected, the results reveal that the actual media experience depends on personal characteristics. The process variables presence and flow—which both refer to the actual use—in turn influence both, enjoyment and performance.

The exogenous variable immersive tendency influences the endogenous variable presence and thereby explains 13 percent of variance. Immersive tendency also affects flow directly as well as indirectly over presence. Thus, presence partly mediates the relationship between immersive tendency and flow. Motivation only influences flow, but not presence. Together, motivation, immersive tendency, and presence explain over one third of variance in the endogenous variable flow (36%). Flow in turn influences enjoyment as well as performance, whereby 24 percent (enjoyment), respectively 8 percent (performance) of variance is explained. In contrast, presence does not directly influence enjoyment and performance, but indirectly via flow. Additional analyses using the Baron and Kenny steps [41] show that flow fully mediates the relationship between presence and enjoyment as well as the relationship between presence and performance.

3.2.5. Summary of the Results of Study 1

Flow and presence positively correlate. Still, the results suggest that flow and presence are distinct constructs. The factor analysis revealed a two-factorial solution with one factor representing flow and the other representing presence. Path analysis showed that presence and flow are influenced by immersive tendency. Motivation, in contrast, only influenced flow experiences. Furthermore, flow influenced enjoyment as well as performance, whereby the latter influence was not very strong. Presence, on the other hand, only influenced the two outcome variable indirectly over flow. Thus flow was identified as an important mediator.

4. Study 2

To replicate the findings of study 1 and to enhance external validity, we conducted a second study in the context of a different computer game.

4.1. Method

4.1.1. Participants

120 participants took part in the second study (80 women and 40 men). All of them were students of a large public university in Switzerland. The average age was 22.50 years (SD = 3.51). The sample consists of undergraduate students enrolled in psychology, who obtained a credit for participating. All participants were treated according to the declaration of Helsinki [34].

4.1.2. Design and Statistical Methods

The same variables as in study 1 were assessed (motivation, immersive tendency, spatial presence, flow, enjoyment, and performance) and the same analyses were conducted.

4.1.3. Stimulus Material

The Game Formula 1
Championship Edition [42] for Playstation 3 was used, whereby the circuit “Autodroma Nazionale Monza” was chosen. Participants drove the car using a steering-wheel and a foot pedal. To exercise the handling, there was a two minutes training phase. Then, the testing phase began. Participants were instructed to drive as fast as possible without crashing. The duration of the testing phase was ten minutes.
The game was played on a desktop system by using a 46′′ LCD large-screen television.

4.1.4. Measurements

To measure immersive tendency, motivation, flow, and enjoyment, the same instruments as in study 1 were used.

Since the presence scale we used in study 1 was originally designed to measure presence in the context of television, we chose to use another scale in study 2, namely, the I group Presence Questionnaire [43]. According to the authors, this scale builds upon other existing presence questionnaires and is more commonly used than Kim and Biocca’s scale (e.g., in the following studies: [44, 45]). We used the spatial presence subscale, which consists of five items on a five-point scale (example item: “somehow I felt that the virtual world surrounded me.”). Since another computer game was used, the performance was also measured differently.

The number of rounds completed during the 10-minute testing phase served as performance measure.

4.2. Results

4.2.1. Reliabilities

Reliabilities for the presence scale (Cronbach’s alpha = .74) and the immersive tendency questionnaire (Cronbach’s alpha = .77) were acceptable. The flow scale turned out to be reliable as well (Cronbach’s alpha = .83).

4.2.2. Descriptives

Table 3 shows the means and standard deviations of the variables, we captured, as well as their intercorrelations. The mean performance reflects the average number of rounds that were completed in the racing game. The findings show that presence, flow, and enjoyment ratings were above the midpoint of the scales and thus slightly higher than in study 1.

4.2.3. Dimensionality of Flow and Presence: Factor Analysis

The MSA criterion is acceptable (MSA = .77) and the scree test suggests a two-factor solution. The varimax rotation extracts a first factor which explains about one-fourth of the variance. It consists of six flow items. The second factor explains 18 percent of the variance and mainly consists of four out of five presence items. Like in study 1, no items of different constructs load high on the same factor. Table 4 shows these results.

The result of a confirmatory factor analysis shows that the data fits the model well (; ; ; ; ; ), whereby all path coefficients were significant. Hence, the results of confirmatory factor analysis confirm the two-factorial structure of the data.

4.2.4. Relationship between Personal Characteristics, Actual Media Use, and Outcome Variables: Path Analysis

The relationship between personal characteristics (immersive tendency and motivation), actual media use (presence and flow), and outcome variables (enjoyment and performance) was again tested with a path analysis. Figure 3 shows the resulting model. The fit of the model turned out to be good (; ; ; ; ; ).

All paths are significant on the 0.01 level. All nonsignificant paths were removed from the model as well. In contrast to the first study, the two exogenous variables immersive tendency and motivation are correlated. Like in study 1, the findings show that personal characteristics can predict the actual media use in terms of presence and flow. Actual media use in terms of presence and flow in turn affects enjoyment as well as performance.

The exogenous variable immersive tendency influences the endogenous variable presence, whereby 11 percent of variance is explained. In contrast to study 1, immersive tendency does not influence flow directly. Only an indirect influence via presence occurs. Thus, presence totally mediates the relationship between immersive tendency and flow. This mediation was tested by using the steps by Baron and Kenny [41]. Motivation only influences flow, but not presence. Immersive tendency, motivation, and presence together explain about one-third of variance in the endogenous variable flow. Flow in turn influences enjoyment as well as performance and thereby explains 22 percent (enjoyment) and 7 percent (performance) of variance. Like in the first study, presence does not directly influence enjoyment and performance, but indirectly via flow. Again, the Baron and Kenny steps show that flow fully mediates the relationship between presence and enjoyment and between presence and performance.

4.2.5. Summary of the Results of Study 2

Generally, the results were quite similar to those of study 1. Flow and presence are positively related, but the results of the factor analysis again suggest that flow and presence are distinct constructs and do hardly share common variance. The findings of the path analysis suggest that presence is influenced by immersive tendency, whereas flow is influenced by motivation. Flow in turn low influenced enjoyment as well as performance, whereby more variance was explained in the variable enjoyment. Presence influenced enjoyment and flow indirectly. Alike study 1, these relations were mediated by flow.

5. Study 3

In the games we used for study 1 and study 2, the elements of the virtual environments were rather vivid and realistic and thus ideal to induce spatial presence. We therefore aimed to further replicate the findings of study 1 and 2 in the context of a computer game that portrays the elements of the environment less realistically.

5.1. Method

5.1.1. Participants

78 participants took part in the third study (37 women and 41 men). All of them were students of a large public university in Switzerland. The average age was 23.82 years (SD = 8.12). All participants were treated according to the declaration of Helsinki [34].

5.1.2. Design and Statistical Methods

The same variables as in studies 1 and 2 were assessed (motivation, immersive tendency, spatial presence, flow, enjoyment, and performance) and the same analyses were conducted.

5.1.3. Stimulus Material

The game Ultimate Flash Sonic was used (FlashGames.de, n.d.). It is a flash version of the Sonic the Hedgehog Game [46]. Ultimate Flash Sonic is a jump and run game. Compared to the games used for study 1 and 2, the design and the graphics are rather poor as well as cartoon-like. Participants had to control their avatar by using the keyboard. First, there was a two-minute training phase. Then, the participants were instructed to complete the first level as fast as possible.

The game was played on a desktop system. Like in the previous studies, we used a 46′′ LCD large-screen television.

5.1.4. Measurements

To measure immersive tendency, motivation, flow, and enjoyment, the same instruments as in studies 1 and 2 were used.

To further enhance external validity, we again chose to use another scale to assess spatial presence, namely, the MEC Spatial Presence Questionnaire [47]. The questionnaire was tested with different types of media stimuli and was found reliable and valid [48]. As we were particularly interested in the spatial dimension of presence, we restricted ourselves to the subscale Spatial Presence: Self Location. This subscale consists of 8 items, which have to be judged on a five-point Likert scale (1 = I do not agree at all; 5 = I fully agree).

Since another computer game was used, the performance was also measured differently. The time to complete the first level of the game comprised the performance measure.

5.2. Results

5.2.1. Reliabilities

Reliabilities for the presence scale (Cronbach’s alpha = .93), the flow scale (Cronbach’s alpha = .75), and the immersive tendency questionnaire (Cronbach’s alpha = .84) were acceptable.

5.2.2. Descriptives

Table 5 shows the means and standard deviations of the variables, we captured, as well as their intercorrelations. The mean performance reflects the average amount of time needed to complete the first level (in seconds). The findings show that flow and enjoyment ratings were above the midpoint of the scales and thus comparable to the values of studies 1 and 2. The presence ratings were lower compared to the other studies. This may be due to the poor graphics of the game used for study 3.

5.2.3. Dimensionality of Flow and Presence: Factor Analysis

An acceptable MSA criterion resulted (MSA = .79). The scree test suggests a three-factor solution. The varimax rotation extracts a first factor which explains 28.8% percent of the variance. It consists of all eight presence items. The second and the third factor consist of all flow items and together explain slightly more than the presence items (32 percent of the variance). The second factor mainly consists of six flow items. In accordance with the classification of Rheinberg et al. [25], these items cover the flow aspect smooth and automatic running and reflect the sensation of focusing and control over the activity as well as smooth and automatic cogitations. The third factor consists of the remaining four flow items. In line with Rheinberg’s classification, these items cover the flow aspect absorption, which includes full involvement, distorted sense of time, optimal challenge, as well as absent-mindedness. Like in studies 1 and 2, no items of different constructs load high on the same factor. Table 6 shows these results.

To confirm the factorial structure, we additionally conducted a confirmatory factor analysis. The results show an acceptable model fit (; ; ; ; ; ) and all path coefficients turned out to be significant. Thus, the three-factorial structure of the data could be confirmed.

5.2.4. Relationship between Personal Characteristics, Actual Media Use, and Outcome Variables: Path Analysis

The relationship between personal characteristics (immersive tendency and motivation), actual media use (presence and flow), and outcome variables (enjoyment and performance) was tested with path analysis. The results of the factor analysis suggest flow to be a two-factorial construct. Therefore, and in contrast to studies 1 and 2, the two flow subdimensions absorption and smooth and automatic running were both included. Figure 4 shows the resulting model. The fit of the model turned out to be acceptable (; ; ; ; ; ).

All nonsignificant paths were removed from the model. Like in the previous studies, personal characteristics influence presence and flow. Presence and flow in turn affect the outcome variables enjoyment and performance.

Immersive tendency influences presence, whereby 17 percent of variance is explained. Immersive tendency also indirectly affects the flow subdimension absorption. Thus, presence mediates the relationship between immersive tendency and absorption. This mediation was tested using the procedure described by Baron and Kenny [41]. Motivation neither influences presence nor absorption. However, the user’s motivation affects the second flow subdimension smooth and automatic running, explaining 19 percent of variance. Smooth and automatic running in turn influences performance. Performance is also influenced by absorption. Thereby, 12 percent of variance is explained. No mediation was found between presence and performance. Thus, in contrast to studies 1 and 2, presence has neither a direct nor an indirect influence on the participants’ performance. Enjoyment is only affected by absorption. Similar to studies 1 and 2, we also observed an indirect effect of presence on enjoyment. The steps proposed by Baron and Kenny [41] show that absorption fully mediates between presence and enjoyment. In total, 7 percent of variance of the enjoyment ratings is explained.

5.2.5. Summary of the Results of Study 3

The correlation between flow and presence is also significant in study 3, but the findings again suggest that flow and presence differ from each other. The factor analysis revealed a three-factorial solution with one factor representing presence and the second and third factor representing flow. Thus, in contrast to studies 1 and 2, flow turned out to be a two-factorial construct with one factor representing the aspect absorption and the other representing the aspect smooth and automatic running (cf. [25]). Hence, both flow dimensions were included in the path analysis. The results revealed that presence is dependent on immersive tendency. Motivation influenced smooth and automatic running, but not absorption. Furthermore, both flow dimensions influenced performance. In contrast to studies 1 and 2, presence had neither a direct nor an indirect effect on performance. Enjoyment was only directly affected by absorption, but there was also an indirect effect of presence on enjoyment. Thus, flow was again identified as a mediator. However, the influenced on enjoyment was rather small compared to studies 1 and 2.

6. Discussion

The aim of the present study was to analyse the relationship between flow and (spatial) presence. Furthermore, we investigated whether these two variables depend on the users’ motivation and immersive tendency and whether presence and flow influence enjoyment and performance. We first analysed these questions within the computer role-playing game Neverwinter Nights and then replicated the findings in the context of the racing game Formula 1 for Playstation 3. To examine whether the relation between presence and flow depends on the vividness of the virtual environment, we conducted a third study within the jump and run game Sonic the Hedgehog. In contrast to the other games we used, Sonic the Hedgehog lacks of vividness and portrays the elements of the virtual world less realistically, but rather cartoon-like. Thus, it seems less ideal to induce sensations of presence. All studies revealed quite similar results even though different games and different presence scales were used and the handling of the games was different (keyboard versus steering wheel).

Presence and flow are positively correlated in all studies. However, the three principal axis factor analyses provide evidence that presence and flow are two distinct constructs, which do hardly share common variance. The results of all three studies show that there is not even one single item of different constructs that loads high on the same factor. In the first two studies, two-factor solutions were suggested by the screen tests. These solutions go in line with the prior classification. One factor represents presence, the other factor represents flow. In the third study, a three-factorial solution resulted. One factor represents presence, whereas flow is represented by two factors. In line with the conceptualisation of Rheinberg et al. [25], one factor is referring to absorption—the sensation of involvement, absent mindedness, and distorted sense of time—the other is referring to smooth and automatic running, the experience of utmost focus and control over the activity. The findings of the three studies suggest that even though the two constructs correlate and are thus related, presence and flow represent different aspects of immersive experiences. It is by all means plausible that spatial presence refers to the feeling of being involved or present in a virtual reality. A user experiencing presence is highly immersed in the virtual world portrayed by the computer game. Thus, the concept seems to cover spatial aspects. In contrast, we assume that flow rather describes a mental state in which a person is fully immersed in a task. Thus, flow seems to be related to the gaming action. This in line with Weibel et al. [6] Fontaine [9] who considers that flow involves a narrow focus on task characteristics, whereas presence involves a broader awareness of the task ecology. Taken together, the experience of flow seems to refer to the sensation of being highly involved in the gaming action, but not to the sensation of being spatially immersed in the world portrayed by the game. Therefore, we conclude that immersive experiences can be divided into (1) spatial immersion—the sensation of being there (presence)—and (2) immersion in the task—the sensation of being involved in the gaming action (flow). As the flow factor explained more variance than the presence factor within all three computer games, we suggest that flow is more important for immersive experiences in computer games.

Our second objective was the investigation of the relationship between motivation and immersive tendency, actual media use in terms of presence and flow, and the outcome variables enjoyment and performance. The results of the three studies support the predicted relations. Presence and flow are positively affected by motivation and immersive tendency and in turn influence enjoyment and performance.

Witmer and Singer [11] suggest that immersive tendency is a disposition which can predict the amount of immersion a person is able to experience. The findings of all three studies support this assumption. We can therefore conclude that mediated environments designed to absorb and immerse their users are most effective for users scoring high on immersive tendency. Furthermore, our study shows that motivational factors play a crucial role and enhance flow experiences. Users rather immerse in the gaming action if they are motivated before interacting with a medium. As a consequence, designers of e-learning environments should try to evoke the users’ motivation. To sum up, the results are in line with Wirth et al. [3], who state that user characteristics may support the processes that lead to immersive experiences.

The results furthermore show that flow experiences positively affect enjoyment and performance. Presence, in contrast, does only indirectly influence enjoyment (in all studies) and performance (in studies 1 and 2) via flow, which was identified a mediator. This again suggests that presence is an ancestor of flow. In line with Weibel et al. [6], the two sensations seem not to occur simultaneous, but rather sequential. It might be that the sensation of being there in the computer game facilitates smooth and automatic running of the gaming task because the primary frame of reference is in the world provided by the game and no longer in the physical world, where the task is actually accomplished. This may explain the autotelic nature of flow during gaming.

The influence of flow on performance was not too strong. Nevertheless, the effect seems to be solid since the influence of flow on performance was found in three different gaming settings with three completely different performance measurements. As expected, flow also influences enjoyment. This could be shown in all studies. Whereas, the influence was not too strong in study 3, about one-quarter percent of variance was explained in studies 1 and 2. This refers to a strong effect. In line with previous assumption and findings (e.g., [3, 49, 50]), we found that gaming is more fun when a player is experiencing presence and flow during the actual media use. Thus, flow strongly contributes to gaming enjoyment. Voiskounsky et al. [5] as well as Klimmt [26] seem to be right when suggesting that flow is an important source of the attractiveness of computer games.

The relationship between presence, flow, enjoyment, and performance also seems to be dependent on the type of game. Even though, the results were quite similar, the influence of presence and flow on enjoyment was smaller within the jump and run game compared to the computer role-playing game and the racing game. In study 3, less than 10 percent of variance was explained. This might be because this game provided less vivid and elaborate spatial information what in turn may have led to lower presence ratings. Furthermore and in contrast to study 1 and 2, no indirect effect of presence on performance was observed in the jump and run game. Therefore, we conclude that presence plays a more important role in vivid and realistic games than in cartoon-like games with less vividness. This is in line with the assumptions of Tamborini and Skalski [50]. Since the effect of flow on performance was even stronger in the jump and run game compared to the other games, we assume that flow experiences are less depending on spatial aspects like the vividness of the game environment. Thus, the type of game rather seems to be of more importance for presence experiences than for flow experiences. However, further studies should be conducted to draw final conclusions about the role of presence in different gaming settings.

Our findings are of practical and especially of theoretical importance because they can—at least to a certain amount—help explaining the popularity of computer games. Enjoyment is a main gratification for playing games [51]. The studies, we accomplished, are of theoretical significance since they help explaining the sources of performance and—after all—enjoyment. Our results suggest that game players after all enjoy the gaming action when they experience presence and flow. Thus, game developers should design games in a way they evoke presence and especially flow. Our study also shows the importance of individual characteristics. Therefore, we conclude that prior assessment of a users’ personality and motivation might effectively help in tailoring the most suitable environment for each user. This in turn could help to facilitate immersion and to enhance sensations of presence and flow. As a consequence, game designers could then increase the level of enjoyment, what is one of their main objectives (cf. [4]). Furthermore, our findings suggest that future research using computer games could consider presence and flow as relevant covariates. This could help to partial out the possibly confounding influences of presence or flow on the observed variables.

Since we used three different computer games for our studies, the external validity of our study is good and generalizations for other computer games seem reasonable. Further research could additionally investigate these issues for other media. Furthermore, since gamers are mostly male (cf. [1]), future studies are suggested to replicate this study using a sample with male participants as the majority. Even though no gender differences occurred within our studies, gender may still be a significant factor influencing gaming experience. Also, performance measures were assessed within nonspatial tasks. The results may have been different with performance measures that rather refer to spatial tasks. Future studies should therefore consider capturing spatial as well as nonspatial task performances.

7. Conclusion

We attempted to examine the relation between presence and flow. The results of exploratory as well as confirmatory factor provides empirical evidence that flow and presence are distinct constructs, the first referring to the sensation of being involved in the gaming action, the latter referring to the sensation of being there. Furthermore, we could show within three different computer games that immersive tendency and the (pre)motivation contribute to presence and flow. Flow in turn influences enjoyment and performance. In addition, flow mediates the relationship between presence and enjoyment. In two of the three studies, flow also mediates between presence and performance.

Our study shows that flow is a central construct and may explain the popularity of computer games. Computer Games seem to be ideal to induce flow experiences. This might be because the difficulty level of a computer game is usually varying. As a consequence, it is likely that the balance between challenge and skills is rather given compared to other applications. However, this a mere speculation which should be tested in future studies.

References

N. Yee, “The demographics, motivations, and derived experiences of users of massively multi-user online graphical environments,” Presence: Teleoperators and Virtual Environments, vol. 15, no. 3, pp. 309–329, 2006.
View at: Google Scholar
M. Lombard and T. Ditton, “At the heart of it all: the concept of presence,” Journal of Computer-Mediated Communication, vol. 3, no. 2, 1997.
View at: Google Scholar
W. Wirth, T. Hartmann, S. Böcking et al., “A process model of the formation of spatial presence experiences,” Media Psychology, vol. 9, no. 3, pp. 493–525, 2007.
View at: Google Scholar
J. L. Sherry, “Flow and media enjoyment,” Communication Theory, vol. 14, no. 4, pp. 328–347, 2004.
View at: Google Scholar
A. E. Voiskounsky, O. V. Mitina, and A. A. Avetisova, “Playing online games: flow experience,” PsychoNology Journal, vol. 2, no. 3, pp. 259–281, 2004.
View at: Google Scholar
D. Weibel, B. Wissmath, S. Habegger, Y. Steiner, and R. Groner, “Playing online games against computer- vs. human-controlled opponents: effects on presence, flow, and enjoyment,” Computers in Human Behavior, vol. 24, no. 5, pp. 2274–2291, 2008.
View at: Publisher Site | Google Scholar
J. V. Draper, D. B. Kaber, and J. M. Usher, “Telepresence,” Human Factors, vol. 40, no. 3, pp. 354–375, 1998.
View at: Google Scholar
D. L. Hoffman and T. P. Novak, “Flow online: lessons learned and future prospects,” Journal of Interactive Marketing, vol. 23, no. 1, pp. 23–34, 2009.
View at: Publisher Site | Google Scholar
G. Fontaine, “The experience of a sense of presence in intercultural and international encounters,” Presence: Teleoperators and Virtual Environments, vol. 1, no. 4, pp. 482–490, 1992.
View at: Google Scholar
K. E. Bystrom, W. Barfield, and C. Hendrix, “A conceptual model of the sense of presence in virtual environments,” Presence: Teleoperators and Virtual Environments, vol. 8, no. 2, pp. 241–244, 1999.
View at: Google Scholar
B. G. Witmer and M. J. Singer, “Measuring presence in virtual environments: a presence questionnaire,” Presence: Teleoperators and Virtual Environments, vol. 7, no. 3, pp. 225–240, 1998.
View at: Google Scholar
D. Weibel, B. Wissmath, and F. W. Mast, “Immersion in mediated environments: the role of personality traits,” Cyberpsychology, Behavior, and Social Networking, vol. 13, no. 3, pp. 251–256, 2010.
View at: Publisher Site | Google Scholar
S. Engeser, F. Rheinberg, R. Vollmeyer, and J. Bischoff, “Motivation, flow-experience, and performance in learning settings at universities,” Zeitschrift fur Padagogische Psychologie, vol. 19, no. 3, pp. 159–172, 2005.
View at: Publisher Site | Google Scholar
M. Csikszentmihalyi, “The flow experience and its significance for human psychology,” in Optimal Experience: Psychological Studies of Flow in Consciousness, M. Csikszentmihalyi and I. Csikszentmihalyi, Eds., pp. 15–35, Cambridge University Press, Cambridge, UK, 1988.
View at: Google Scholar
J. Steuer, “Defining virtual reality: dimensions determining telepresence,” Journal of Communication, vol. 42, pp. 72–92, 1992.
View at: Google Scholar
M. Slater and S. Wilbur, “A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments,” Presence: Teleoperators and Virtual Environments, vol. 6, no. 6, pp. 603–616, 1997.
View at: Google Scholar
A. Van Dam, A. S. Forsberg, D. H. Laidlaw, J. J. LaViola, and R. M. Simpson, “Immersive VR for scientific visualization: a progress report,” IEEE Computer Graphics and Applications, vol. 20, no. 6, pp. 26–52, 2000.
View at: Google Scholar
R. Tamborini, “The experience of telepresence in violent games,” in Proceedings of the Annual Conference of the National Communication Assosiation, Seattle, Wash, USA, 2000.
View at: Google Scholar
M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play, Jossey-Bass, San Fransisco, Calif, USA, 1975.
J. Bryce and J. Rutter, “In the game—in the flow: presence in public computer gaming,” Poster presented at the computer games and digital textualities, Copenhagen, Denmark, March 2001, http://www.cric.ac.uk/cric/staff/Jason_Rutter/presence.htm.
View at: Google Scholar
J. A. Ghani and S. P. Deshpande, “Task characteristics and the experience of optimal flow in human computer interaction,” Journal of Psychology, vol. 128, no. 4, pp. 381–391, 1994.
View at: Google Scholar
F. Rheinberg and R. Vollmeyer, “Flow experience in a computer game under experimentally controlled conditions,” Zeitschrift fur Psychologie, vol. 211, no. 4, pp. 161–170, 2003.
View at: Publisher Site | Google Scholar
J. Webster, L. K. Trevino, and L. Ryan, “The dimensionality and correlates of flow in human-computer interactions,” Computers in Human Behavior, vol. 9, no. 4, pp. 411–426, 1993.
View at: Publisher Site | Google Scholar
F. Rheinberg, S. Engeser, and R. Vollmeyer, “Measuring components of flow: the Flow-Short-Scale,” in Proceedings of the 1st International Positive Psychology Summit, Washington, DC, USA, October 2002.
View at: Google Scholar
F. Rheinberg, R. Vollmeyer, and S. Engeser, “Die Erfassung des Flow-Erlebens [Measuring flow experiences],” in Diagnostik von Motivation und Selbstkonzept. Tests und Trends, J. Stiensmeier-Pelster and F. Rheinberg, Eds., vol. 2, pp. 261–279, Hogrefe, Göttingen, Germany, 2003.
View at: Google Scholar
C. Klimmt, “Computer-Spiel: Interaktive Unterhaltungsangebote als Synthese aus Medium und Spielzeug,” Zeitschrift für Medienpsychologie, vol. 13, no. 1, pp. 22–32, 2001.
View at: Google Scholar
D. L. Hoffman and T. P. Novak, “Marketing in hypermedia computer-mediated environments: conceptual foundations,” Journal of Marketing, vol. 60, no. 3, pp. 50–68, 1996.
View at: Google Scholar
T. P. Novak, D. L. Hoffman, and Y. F. Yung, “Measuring the customer experience in online environments: a structural modeling approach,” Marketing Science, vol. 19, no. 1, pp. 22–42, 2000.
View at: Google Scholar
D. Weibel, D. Stricker, and B. Wissmath, “The use of a virtual learning centre in the context of a university lecture: factors influencing satisfaction and performance,” Interactive Learning Environments. In press.
View at: Google Scholar
C. Heeter, “Being There: the subjective experience of presence,” Presence, vol. 1, pp. 262–271, 1992.
View at: Google Scholar
C. I. Teng, “Customization, immersion satisfaction, and online gamer loyalty,” Computers in Human Behavior, vol. 26, no. 6, pp. 1547–1554, 2010.
View at: Publisher Site | Google Scholar
R. Weber, R. Tamborini, A. Westcott-Baker, and B. Kantor, “Theorizing flow and media enjoyment as cognitive synchronization of attentional and reward networks,” Communication Theory, vol. 19, no. 4, pp. 397–422, 2009.
View at: Publisher Site | Google Scholar
T. B. Sheridan, “Musing on telepresence and virtual presence,” Presence, vol. 1, pp. 120–126, 1992.
View at: Google Scholar
World Medical Association, “Declaration of Helsinki,” Law, Medicine & Health Care, vol. 19, pp. 264–265, 1991.
View at: Google Scholar
BioWare Corp., “Neverwinter Nights [Computersoftware und Manual],” 2002, BioWare Corp., Edmonton, Canada.
View at: Google Scholar
T. Kim and F. Biocca, “Telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion,” Journal of Computer-Mediated Communication, vol. 3, no. 2, 1997.
View at: Google Scholar
S. G. Nicovich, G. W. Boller, and T. B. Cornwell, “Experienced presence within computer-mediated communications: initial explorations on the effects of gender with respect to empathy and immersion,” Journal of Computer-Mediated Communication, vol. 10, no. 2, 2005.
View at: Google Scholar
M. C. Green, T. C. Brock, and G. F. Kaufman, “Understanding media enjoyment: the role of transportation into narrative worlds,” Communication Theory, vol. 14, no. 4, pp. 311–327, 2004.
View at: Google Scholar
S. Knobloch and D. Zillmann, “Mood management via the digital jukebox,” Journal of Communication, vol. 52, no. 2, pp. 351–366, 2002.
View at: Publisher Site | Google Scholar
H. F. Kaiser and J. Rice, “Little jiffy mark IV,” Educational and Psychological Measurement, vol. 34, pp. 111–117, 1974.
View at: Google Scholar
R. M. Baron and D. A. Kenny, “The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations,” Journal of Personality and Social Psychology, vol. 51, no. 6, pp. 1173–1182, 1986.
View at: Publisher Site | Google Scholar
Sony Computer Entertainment Europe, “Formula 1: Championship Edition (Version 1.0) [Computer Software],” London, UK, 2006.
View at: Google Scholar
T. Schubert, F. Friedmann, and H. Regenbrecht, “The experience of presence: factor analytic insights,” Presence: Teleoperators and Virtual Environments, vol. 10, no. 3, pp. 266–281, 2001.
View at: Publisher Site | Google Scholar
N. J. J. M. Smets, M. S. Abbing, M. A. Neerincx, J. Lindenberg, and H. van Oostendorp, “Game-based versus storyboard-based evaluations of crew support prototypes for long duration missions,” Acta Astronautica, vol. 66, no. 5-6, pp. 810–820, 2010.
View at: Publisher Site | Google Scholar
R. C. Freire, M. R. De Carvalho, M. Joffily, W. A. Zin, and A. E. Nardi, “Anxiogenic properties of a computer simulation for panic disorder with agoraphobia,” Journal of Affective Disorders, vol. 125, no. 1–3, pp. 301–306, 2010.
View at: Publisher Site | Google Scholar
Sega Corporation, “Sonic the Hedgehog [computer software],” Tokyo, Japan, 1991.
View at: Google Scholar
P. Vorderer, W. Wirth, F. R. Gouveia et al., “MEC Spatial Presence Questionnaire (MEC-SPQ): short documentation and instructions for application,” Tech. Rep. MEC (IST-2001-37661), European Community, 2004.
View at: Google Scholar
A. Sacau, J. Laarni, and T. Hartmann, “Influence of individual factors on presence,” Computers in Human Behavior, vol. 24, no. 5, pp. 2255–2273, 2008.
View at: Publisher Site | Google Scholar
C. Klimmt and P. Vorderer, “Media Psychology “is not yet there”: introducing theories on media entertainment to the Presence debate,” Presence: Teleoperators and Virtual Environments, vol. 12, no. 4, pp. 346–359, 2003.
View at: Publisher Site | Google Scholar
R. Tamborini and P. Skalski, “The role of presence in the experience of electronic games,” in Playing Computer Games: Motives, Responses, and Consequences, P. Vorderer and J. Bryant, Eds., pp. 225–240, Erlbaum, Mahwah, NJ, USA, 2006.
View at: Google Scholar
J. H. Wu, S. C. Wang, and H. H. Tsai, “Falling in love with online games: the uses and gratifications perspective,” Computers in Human Behavior, vol. 26, no. 6, pp. 1862–1871, 2010.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2011 David Weibel and Bartholomäus Wissmath. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

31552

Downloads

7012

Citations