Business Simulation Games in Higher Education: A Systematic Review of Empirical Research

Faisal, Nadia; Chadhar, Mehmood; Goriss-Hunter, Anitra; Stranieri, Andrew

doi:https://doi.org/10.1155/2022/1578791

Human Behavior and Emerging Technologies

On this page

Abstract Introduction Background Results Discussion Limitations Conclusion Data Availability Conflicts of Interest References Copyright Related Articles

Review Article | Open Access

Volume 2022 | Article ID 1578791 | https://doi.org/10.1155/2022/1578791

Business Simulation Games in Higher Education: A Systematic Review of Empirical Research

Nadia Faisal,¹Mehmood Chadhar,¹Anitra Goriss-Hunter,²and Andrew Stranieri³

Academic Editor: Zheng Yan

Received07 Aug 2022

Accepted01 Sept 2022

Published22 Nov 2022

Abstract

Over the last few years, business simulation games (BSGs) in higher education have attracted attention. BSGs tend to actively engage students with course material, promoting higher engagement and motivation and enabling learning outcomes. Increasingly, researchers are trying to explore the full potential of these games with an upsurge of research in the BSG field in recent years. There is a need to understand the current state of research and future research opportunities; however, there is a lack of recent systematic literature reviews in BSG literature. This study addresses this gap by systematically compiling online empirical research from January 2015 to April 2022. We followed PRISMA guidelines to identify fifty-seven (57) papers reporting empirical evidence of the effectiveness of BSGs in teaching and learning. Findings showed that BSGs improve learning outcomes such as knowledge acquisition, cognitive and interactive skills, and behaviour. The review also summarises different issues concerning the integration of BSGs into the curriculum, learning theories used in the selected studies, and assessment methods used to evaluate student achievement in learning outcomes. The findings of this review summarise the current research activities and indicate existing deficiencies and potential research directions that can be used as the basis for future research into the use of BSGs in higher education.

1. Introduction

The simulation and virtual training market was valued at $204.41 billion in 2019 and has been predicted to reach $579.44 billion by the end of 2027 according to the Global Opportunity and Industry Forecast 2020-2027 report [1]. For many years, simulations employing mixed reality have been a sign of the future since they merge several linked disciplines of art and science to generate improved interactive experiences [2]. The cross-border combination of education and gaming has gotten a lot of interest from many parts of society, including education and technology. By providing a fun and engaging learning environment and encouraging students to learn via hands-on experience, simulation games have the potential to enhance students’ advanced skills [3]. Business simulations were established in the 1950s, and the simulation industry has grown significantly since then, with a significant growth in usage by business schools in higher education institutions. With the availability of off-the-shelf and customised business simulation games (BSGs), universities started integrating them into their courses in the mid-60s to provide an active learning experience to the students [4]. Since then, business simulations have been increasingly used to enhance students’ learning. This development has resulted in an increase in research in this field. Academics have attempted to establish the advantages of these games via evidence-based research [5]. Research evidence is used to inform policymakers about the effectiveness of a specific educational approach. However, single studies present contextual and methodological limitations of research evidence as most empirical research are done within a confined context [6]. Therefore, there is a need to identify, evaluate, and synthesise research results from different empirical studies focusing on a single phenomenon to create a summary of current evidence. If done systematically, a literature review provides researchers and practitioners with a broad overview of the research area, shows what work has been done, helps identify research gaps, and directs future research in that particular subject area [7].

Past reviews summarising the impact of using BSGs as part of pedagogical approaches to meet learning outcomes show ambiguous results and lack empirical evidence. Furthermore, the extensive effort and timescale involved in carrying out and then publishing literature reviews make them obsolete relatively quickly. By the time they are completed, many new studies have been done on the same phenomena with different or sometimes conflicting results [8]. Fu et al. [9] conducted the last literature review on the reported empirical evidence of the positive impact of BSGs on learning, summarising studies from 2005 to 2014. However, a significant expansion has been observed in BSG research, as shown in Figure 1.

Some of the newer games, teaching and evaluation techniques, and positive effects of these games on different learning and behavioural outcomes were examined and explored during this period. There is a value in systematically compiling these studies to provide a sound basis for further endeavours in this field. Therefore, this review is aimed at presenting a systematic literature review (SLR) of empirical studies on BSGs published between January 2015 and February 2022. The review addresses the research question: “What empirical evidence exists concerning the use of BSGs in promoting learning and effective teaching?” During the literature review, four distinct themes emerged that were then used to create four subquestions to analyse the fifty-seven selected papers for this systematic literature review. Figure 2 represents the themes. (i)Integration of BSGs in higher education: this theme addresses the integration of business simulation games to teach IT and business courses. This section also describes the instructors’ role in facilitating the learning process by using BSGs and the barriers to integrating these games(ii)Learning theories/models and BSGs: different learning theories and models are identified under this theme in BSG research, and links between these theories/models and simulation games are established(iii)Evaluation of BSGs: this theme discusses frameworks and guidelines to efficiently evaluate the simulation games’ learning outcomes(iv)Learning outcomes of BSGs: this theme addresses the different learning outcomes that can be achieved using simulation games as a learning tool. The learning outcomes are categorised into skills, knowledge, and behavioural effects

Overall, this SLR provides an overview of empirical research, what is currently being investigated, and possible future directions in BSGs research.

The structure of the rest of the paper is as follows. Section 2 presents the background of BSGs and discusses the past reviews on BSGs. Section 3 describes the research methodology. The results are shown in Section 4. Section 5 discusses the findings of the study. The limitations and future recommendations of the selected papers are discussed in Section 6, and Section 7 concludes the paper with a brief discussion of future research agendas.

2. Background

In this section, a brief introduction to BSGs, previous reviews, and their limitations are presented.

2.1. Business Simulation Games (BSGs)

BSGs are experiential learning tools where students learn business processes by running a simulated firm in a risk-free, interactive, and realistic environment. Students make all strategic decisions and compete either individually or in teams. These active learning experiences improve student engagement and develop collaboration, decision-making, problem-solving, and critical thinking [4]. BSGs have been very popular in business education since their inception. With the technological developments in operations research, war games, computer technology, and education theory, educators started integrating these games into business courses in the late 1950s [10]. Games such as the Beer Distribution Game [11] and the Markstrat Simulation Game [12] were released in the following years and gained popularity among educators.

Technology developments such as interactive features, advanced graphics processing, artificial intelligence, and cloud computing have influenced business simulations [13, 14]). Furthermore, BSGs are sometimes used as incentives to make crowdsourcing (crowd-based online work) more valuable and attractive [15]. Global enterprises like Google, Microsoft, American Express, and Caterpillar are using these simulation games to train their employees and managers [16]. Currently, many business courses include one or more management simulations. They are more commonly used at the undergraduate level to provide an active learning experience to students [17]. In addition to encouraging higher engagement levels, simulations in the business context can improve soft skills such as teamwork, decision-making, leadership, and other technical skills used in strategic management, marketing, finance, and project management.

In contrast to passive learning experiences that can occur in traditional teaching techniques, such as lectures and tutorials, BSGs bridge the gap between academia and industry through experiential learning techniques. In BSGs, students run a simulated company to learn business processes and strategies. They get the opportunity to integrate and experiment with what they have learned through simulations, solve complex problems, get involved in active decision-making, experience the consequences of their decisions, and learn from their mistakes [18]. BSGs are an effective teaching technique to help students develop managerial and generic skills in demand in the industry [19].

2.2. Past Reviews on Simulation Games

Several studies summarised the literature on simulation games in the past ten years, but most reviews target a specific and limited feature of simulation games. Table 1 presents some of the reviews and meta-analyses published between 2009 and 2022 on game-based learning and their respective scopes:

Although the past reviews focused on the simulation games’ effectiveness in building knowledge and skills, there are limitations in these studies. For example, the latest SLR by Ferreira et al. [20] only focused on researchers that used electroencephalogram (EEG) or eye tracking (ET) signals as data collection methods. Another limitation of this review is the limited amount of research linking ET and EEG devices to help study the BSG user experience which is a drawback of this work. These devices have employed marketing, human behaviour, and applied neuroscience research. However, they are integrated with other methodologies, comparing and contrasting findings. This paper lacks methodological integration, leading to skewed outcomes. In another review, Sitzmann [30] statistically summarised 52 studies on simulation games’ effectiveness in enhancing industry-related knowledge and skills. The review focused on adult work competency requirements in organisational settings and did not discuss simulation games in educational settings. The study findings showed that technology could improve the learning experience and stressed the importance of multiple factors (integration, training, and debriefing) which can contribute to rich learning experiences through gamified learning. A review by Connolly et al. [28] addressed the impact of games on overall learning outcomes. That review had a broad scope, and instead of just focusing on simulation games, it considered all computer games (entertainment games, serious games, video games, simulation games, etc.). It concluded that computer games positively impact students’ cognitive, affective, behavioural, and motivational outcomes. The most significant gains were in knowledge acquisition and affective and motivational outcomes. In 2016, the same researchers [23] presented an updated version of their previous review, including entertainment and educational computer games. This review again demonstrated positive impact on learning outcomes but did not specifically focus on computer simulation games. Lopes et al. [26] summarised the effect of using simulation games as a means of meeting learning outcomes, but again, the review only covered leadership development through simulation games. Scholtz and Hughes [32] conducted a systematic literature review to explore the innovative and new pedagogical methods instructors use to integrate simulation games into business courses. The scope of that review was limited to exploring the educators’ role in the learning process.

3. Research Methodology

A systematic literature review (SLR) helps to compile relevant research on a particular phenomenon of interest or topic [33]. In line with Keele [34] guidelines, this study used a systematic process to conduct a comprehensive literature review. This process involves developing protocols for the literature search, identifying and selecting relevant primary studies, extracting and synthesising data, and reporting results.

The broad objective of this SLR was to answer the following research question.

RQ: “What empirical evidence exists concerning using BSGs to promote learning and effective teaching?”

The subquestions that emerged from the coding of the literature were as follows: (i)What are the learning outcomes of BSGs?(ii)How are BSGs integrated into existing curricula to support learning outcomes?(iii)What were the learning theories used in BSG literature?(iv)What assessment methods are used in BSG literature to evaluate learning outcomes and game performance?

Learning outcomes are classified into behavioural, knowledge, and skill outcomes [35].

3.1. Data Sources and Search Strings

Databases used to search for the relevant open access research papers were EBSCOhost, Emerald insight, IEEE, Informit, JSTOR, Oxford, Sage, ScienceDirect, Scopus, SpringerLink, Tylor & Francis, Web of Science, Wiley Online Library, and Google Scholar. These databases are the best and most supportive resources for computing game research.

Initially, we conducted Boolean searches using the “AND” operator between the keywords business simulation games, learning outcomes, and higher education. To avoid losing relevant data, researchers also used alternative words by conducting a Boolean search again, but they used “OR” as an operator this time. Table 2 shows the main key terms and alternative search terms for data search.

The snowballing method [36] was applied to exhaust the included papers’ relevant sources to minimise any data loss. This process was considered complete when no pertinent new papers were found. The literature search was last updated on 28^th February 2022.

3.2. Selection Process

This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines. Frequently, systematic reviews indicate a lack of comprehension of common principles that enable them to be replicable and scientifically competent. PRISMA is a common peer-reviewed methodology that utilises a checklist of guidelines, which was carefully adhered to in this work. It adds to the revision process’s quality assurance and reproducibility. We created a protocol outlining the article selection criteria, search technique, data extraction, and data analysis methods. The researchers followed the Dybå and Dingsøyr [37] citation management process to sort out the relevant citations with the help of EndNote 20. The complete selection process is shown in Figure 3.

3.2.1. Inclusion Criteria

Papers that fulfil the following criteria were included: (i)Only peer-reviewed journal and conference papers were included(ii)Only empirical papers that reported on research using data collected from experiments or observation designs were included(iii)The participants should be studying in higher education institutes (graduate and postgraduate business courses)(iv)The review included only open access articles published between 2015 and 2022(v)The papers were available in English(vi)The papers explicitly report the learning outcomes of BSGs

3.2.2. Exclusion Criteria

(i)Nonempirical studies are not included in this review(ii)Studies conducted with participants younger than 18 are not included as the focus is on higher education institutions(iii)Studies on nondigital and entertainment games are not included in this review(iv)Book chapters are not included as they are difficult to search on databases and hard to access as full texts. The peer-review process for journal and conference papers does not always apply to books. Other sources were also excluded for the same reason. For example, dissertations, theses, editorials, book reviews, and reports are also excluded for similar reasons(v)Papers that are not open access are not included(vi)Papers that do not adhere to the research objectives are not included(vii)Papers published before January 2015 are also excluded

Coauthors of this SLR reviewed the selected papers to ensure they met the inclusion criteria. In cases of doubt, all four reviewers discussed and determined the inclusion or exclusion decision on the paper.

3.3. Quality Assessment

In the third stage, a quality assessment of the included papers was conducted. Extra discussions among researchers were undertaken to ensure the objectivity and quality of the selected papers. The quality assessment criteria gave each of the 57 selected papers on a five-dimensional scale inspired by Connolly et al. [28]. Five dimensions were scored from 1 to 3 across each dimension, where 3 represented high, 2 medium, and 1 low quality of the paper. In this paper, we adopted the following five dimensions in Table 3.

The total quality weight of each paper was determined by adding scores for all five dimensions ranging from 5 to 15. The mean rating was 8.9, and the mode rating was 9. 37 papers reaching 9 or above scores on the five dimensions scale were considered methodologically more critical papers, providing strong evidence of the impact of BSGs on learning outcomes. These papers are shown in Table 3 with their respective methodological designs.

3.4. Data Analysis and Synthesis

The 57 selected papers were analysed using a qualitative content analysis, which involved coding for the emergence of themes. The researchers examined the papers from a range of viewpoints. Firstly, they conducted a bibliographic overview which included demographics, research design, year of publication, database of publications, and citation counts of the selected papers. Secondly, the papers were examined to determine the learning outcomes, learning theory, integration techniques, and assessment methods that were discussed in BSG research. The categorisation is aimed at helping instructors select the most highly ranked games according to the required learning outcomes, the evaluation methods used to assess these outcomes, integration of the games into the curriculum, and the role of instructors in facilitating the learning.

4. Results

This section presents results from the selected papers ().

4.1. Bibliographic Overview of the Papers

The included papers are journal publications except two [38, 39]. The included papers presented multiple study designs, with most papers falling into survey design. Most papers are retrieved from the SCOPUS database, followed by ScienceDirect. The number of papers per year and per data base is shown in Figures 4 and 5, respectively. The papers have an average citation count of (till February 2022).

Due to cultural settings and available resources, demographic differences can cause varying attitudes and expectations toward learning [40]. The number of papers published in the region also shows its willingness to adapt and experiment with innovative knowledge and teaching methods and tools. Most of the studies on simulation games (Figure 6) were conducted in Europe () followed by North America (). Asia () comes after them, followed by Australia, Africa, and the Middle East.

4.2. Methodologies of Selected Studies

There are different types of constructs and variables used in the selected papers. Most empirical studies on BSGs evaluated how playing the game might assist students in meeting learning outcomes. Entrepreneurial skills and attitudes are other commonly used constructs in the BSG literature. A range of study designs (quantitative, qualitative, and mixed methods) was used in the selected papers depending on the research objectives. Figure 7 presents an overview of the study design.

Most of the papers used postgame surveys (questionnaires) to collect data (28 papers), and the participants were students in the selected studies except three papers. Mainly, informal methods were used to develop these questionnaires without defining a measurement model or theoretical construct. 13 studies used tests to assess the students’ learning outcomes and skills acquisition, typically in experimental studies and by applying them before and after playing the game. However, a few studies conducted surveys just after the game session (8 studies). Five studies reported the use of challenge or exercise for data collection. Interviews with students on how playing the BSGs helped them to achieve learning outcomes were also used, typically in qualitative design or mixed-method approaches. These interviews were done most of the time after playing the game. Only 3 studies included interviews with instructors. In five studies, researchers conducted focus group meetings to collect opinions on the effectiveness of BSGs in enhancing students’ learning experiences. Figure 8 shows the number of papers as per the data collection method used in the study.

BSGs are usually played in teams, and most studies conducted on BSGs collected data over large samples (100-500), usually in postgame surveys or game results. Three selected studies used extensive samples, with one analysing 3681 messages posted by students on an online discussion forum [41], one analysing 3129 game results [39], and one longitudinal study surveying more than 12000 students over three years [42]. The number of papers as per sample size range is shown in Figure 9.

Quantitative data were mostly analysed through statistical analysis using different software. Regression analysis, -test, and one-way ANOVA are the most commonly employed tests on the quantitative data sets in selected studies. A comprehensive bibliographic review of the selected papers is given in Tables 4 and 5.

4.3. Results of the Quality Assessment

The researchers selected the papers based on quality ratings. The minimum acceptable number was 9, and the maximum number given on the paper was 15. The average quality rating of the selected papers is 11. A standard deviation of 1.7 was found in the quality ratings for the total mean score. As shown in Figure 10, there has been a steady increase in the quality of empirical papers on BSGs over time.

5. Discussion and Findings

In this literature review on the use of BSGs for teaching and learning, we found that the research on BSGs in teaching and learning has grown in the last five years. The quality rating of the selected papers is also increasing, as shown in Figure 10. Most empirical studies explored the learning outcomes of incorporating these games in coursework. Using the key terms, the initial literature search retrieved 523 relevant papers, which were later reduced to the final sample of high-quality forty-nine open access empirical studies. The factor analysis of the selected studies provided a framework for organising the diverse research on BSGs and summarising trends in literature. This section presents the selected papers’ findings, limitations, and future recommendations of the selected papers. Four distinctive trends emerged from the literature review, which was used to analyse the selected studies: (1)Integration of BSGs in teaching and learning processes(2)Learning outcomes achieved by using BSGs as a learning tool(3)Learning theories used in BSG literature(4)Evaluation methods used to assess the learning outcomes of BSGs and the game performance

5.1. Integration of BSGs in Teaching and Learning

The way simulation games are integrated into teaching and learning makes a difference in the success of achieving desired learning outcomes [95]. In a literature review, Oliveira et al. [96] addressed seven emerging technologies utilised in education, one of which is simulations. According to the research, simulations have been shown to improve students’ learning experiences, particularly in terms of problem-solving and creative thinking. An important enabling component for the successful use of simulation games in education is the role of the instructor/facilitator. Seven studies discussed the instructor’s part in achieving desired learning outcomes using BSGs.

BSGs are often a supplementary tool to assist educators on regular courses. They are mainly conducted after lectures and tutorials, usually at the end of the semester as a final test or integrational activity. They are also taught as separate courses in a degree program to complement the other courses [97]. In most selected studies, off-the-shelf BSGs are incorporated as an add-on or run as a test activity. Research on BSGs has proven that these games can help instructors facilitate students’ engagement, learning, and employability if they are appropriately integrated [4]. Since a BSG consists of many decision-making elements (e.g., finance, marketing, human resources, distribution, and logistics), it is essential for instructors first to identify the learning outcomes they want to achieve and then use BSGs as a mean to achieve those outcomes [98].

Vos [43] explained the three important roles instructors have to play in integrating and supporting these learning activities: instructors have to plan appropriate learning and teaching strategies, support students during the whole process, and then design appropriate assessment tasks to assess whether the game is achieving the required learning outcomes. Ellahi et al. [65] reinforced the instructors’ role in incorporating BSGs into courses. The study used instructor support as a moderating factor to examine the relationship between simulation games and learning outcomes. The study’s findings showed that although the role of the instructor is essential in the learning process using computer simulations, the research argued that students should mainly take control of their learning. On the contrary, Hernández-Lara and Serradell-López [41] suggested that instructors should play an active role at the pregame stage and then throughout the game while determining the complexity levels and other conditions of the game that include the economic environment, market conditions, production facilities, employee engagement and motivation, and other conditions. Loon et al. [81] conducted a mixed-methods study on the role of instructional design and the context of the study in strategic decision-making. He suggested that a lack of support during the simulation activity might result in demotivating the students from taking part in the activity. Even if they participate, the absence of instructor support can negate any possibility of learning from the process. He suggested three types of support for students during the activity: interpretative support (background information and relevant input knowledge, including elaborative and explanatory feedback), experimental support (in developing perspectives and propositions), and reflective support (inquiry process and knowledge gained from the simulation).

Another important factor concerning the instructors’ role in the successful integration of BSGs was discussed by Kriz and Auchter [42] in an empirical study where they explored the effect of simulation games on entrepreneurial competencies by conducting more than 12000 surveys with students who have played different BSGs. One factor that influenced students’ acceptance of the simulation activity was the role of the instructors. According to the students, instructors’ skill levels, enthusiasm, and teaching quality impacted how students perceived and adopted these simulations. Pando-Garcia et al. [84] explored the effect of perceived usefulness and ease of use of simulation on the attitude and intention to use simulation by students using the Technology Acceptance Model (TAM) (Urquidi-Martín, Tamarit-Aznar, & Sánchez-García) model. TAM is a theory of information systems that describes how people learn to accept and use technology. The study’s results showed a strong relationship between positive attitudes towards using simulation games and their perceived ease of use. This leads the authors to conclude that the instructors can influence students’ attitudes toward using simulation because they can help students understand the game’s technical aspects, which may make the game experience easier for students. Overall, the selected studies which explored instructors’ role in using BSGs established the instructor’s significant role in the integration or running phases of the BSGs.

Kriz and Auchter [42] explored the barriers to integrating BSGs in existing courses as part of their longitudinal study with multiple business schools all over Europe using BSGs. They identified the three most common barriers to adopting BSGs: the perceived risk of adopting new teaching techniques, the suitability of the available simulations, and lack of resources (financial and infrastructural). These three barriers are interlinked [99] and should be examined together. The risk perceptions of individual instructors vary and play an essential part in explaining the instructors’ attitude toward adopting the simulation. From an instructor’s perspective, as the simulation is a student-centred learning activity, there is the possibility that they may lose control over student learning.

Rogmans and Abaza [69] conducted a comparative analysis of the effectiveness of two teaching methods in management education. The results of students’ surveys showed that the average engagement levels of students were higher in the traditional case study compared to simulation games. The findings showed that the main reason for students’ preference for selecting the case study over simulation is the complexity level of the simulation. The results showed that complexity was crucial in determining students’ acceptance of simulation-based learning. Calabor et al. [93] used Delphi techniques to explore instructors’ perceptions concerning barriers to incorporating BSGs in classrooms. In the panellists’ view, the most significant obstacle to implementing BSGs in existing courses was the lack of information on the suitability of the game for a particular course, the lack of infrastructure and financial resources, and the lack of knowledge regarding expected learning outcomes from the use of a specific BSG. They proposed to train instructors for new technologies and teaching methods so that they can facilitate their students’ learning performance. The researchers concluded that there is a general reluctance to adopt new teaching techniques and a need to create an awareness of the benefits for students and instructors of adopting simulation systems.

The selected studies demonstrated the benefits of blended learning and incorporating BSGs into existing courses and traditional teaching methods (lectures and tutorials). For example, in one of the selected papers, Loon et al. [81] suggested the adoption of the guidelines provided by [100] and Salas et al. [101] in the selection and implementation of simulation games. The integrative aspect of blended learning was found to be effective as simulation games are not sufficient on their own to promote learning. They must be supported and supplemented by effective instructional designs and teaching approaches that involve regular intervention and instructor support through coaching and playing the role of a facilitator.

5.2. Learning Theories/Models and BSGs

Drawing on learning theory can assist educators in making well-informed decisions in selecting, integrating, and delivering specific simulation games. From the BSGs researchers’ point of view, learning theories were used to explain why certain aspects of simulations are successful or problematic and to develop a clear understanding of the research problems. Therefore, these researchers argued that BSGs provide a learning environment for students to bridge theoretical and conceptual knowledge taught in classrooms with authentic, real-world experiences [102].

Several theories/models have been proposed in the selected studies for understanding the role of BSGs in enhancing learning and teaching processes. Fourteen studies used learning theory to develop a well-defined research model out of forty-nine chosen papers. Kolb’s [103] experiential learning theory is the most commonly used approach, followed by the Technology Acceptance Model [104]. The list of these fourteen papers, learning theories/models, and the constructs used is presented in Table 6.

The BSG literature uses the learning paradigms of humanism, constructivism, cognitivism, and behaviourism to varying degrees. The fundamental theories that play a significant role in understanding learning during simulations are game-based, problem-based, and experiential learning.

Many studies combined multiple theories. For example, Tao et al. [64] conducted an experimental study where he combined the problem-based gaming model [108] and the input-process-outcome game model [109] to examine the perceived learning-cycle effects caused by playing multiple BSGs with different complexity levels to develop the research model of the study. He used the problem-based gaming model to create constructs related to complexity and the input-process-outcome game model to develop constructs of knowledge gain (declarative, procedural, and strategic). The study’s findings showed that a significant number of students perceived that the complexity level of the BSG affected their knowledge levels. Zulfiqar et al. [68] combined the theory of planned behaviour [105] and the Technology Acceptance Model [104] to explore how the use of simulation games influenced and developed entrepreneurial attitudes in students. The variables measuring perceived business value came from the TAM, and subjective norms, perceived behavioural controls, and attitudes were taken from the theory of planned behaviour. The results showed a significant increase in the perceived behavioural controls and attitude toward entrepreneurial activities after using BSGs. However, subjective norms did not have a substantial effect on entrepreneurial attitudes.

5.3. Learning Outcomes of Using BSGs

A significant part of empirical studies on BSGs focused on measuring learning outcomes. Identifying learning outcomes enables instructors and researchers to determine how much progress students have made in meeting learning outcomes while completing a course. These outcomes consist of students’ knowledge and skills to acquire for effective learning. Many studies discussed the methodologies and benefits of integrating simulation games in different business courses. For the effective integration of simulation games in education, the specificity of the selected game regarding the desired learning outcomes to be considered. We observed that most chosen papers explained clearly defined learning objectives using a specific simulation game. Faisal et al. [61], for example, conducted qualitative research to investigate teachers’ perceptions of utilising ERPsim games in a supply chain training. The ERPsim game is an innovative learning method which provides students a real-time, risk-free environment to improve their knowledge of business processes and ERP systems [115]. ERPsim game runs on SAP HANA, the latest version of SAP as it provides an in-memory database which facilitates rapid processing and analysis of big data in real time [13]. This in-memory database provides students instant results of the decisions they have made [116]. This business simulation game has been used in more than 250 universities worldwide and a few universities in Australia including University of Melbourne, RMIT University, Victoria University, and Federation University, to teach enterprise system concepts.

Figure 11 illustrates the scope of the manufacturing simulation game. Shaded rectangles represent SAP transactions automated by ERPsim, while students must perform other transactions to manage their company.

These games were also designed to provide a rich and economically representative simulation of a medium-sized organisation. The most measured skills in the selected studies are communication, decision-making, entrepreneurial skills, analytical thinking, and problem-solving skills. The most common knowledge outcome is business process knowledge discussed in 9 studies, followed by ICT and functional knowledge (5 studies) and supply chain knowledge (3 studies). The most common behavioural outcomes are engagement (12 studies), followed by motivation (7 studies) and self-efficacy (4 studies). Table 7 shows the commonly measured learning outcomes and the respective studies they are discussed in.

From the selected studies, the learning outcomes can be classified into three main categories: knowledge acquisition, skill development, and behavioural outcomes.

5.3.1. Knowledge Gains from Using BSGs

Many reviewed papers discussed the impact of simulation games on knowledge acquisition. Almost all the selected papers reported the beneficial implications of the use of simulation games on knowledge acquisition and conceptual understanding. Specific simulation games were used in different courses to improve the content knowledge of students. For example, the pre-postgame survey’ results of a study by Beranič and Heričko [62] showed a positive impact on students’ knowledge in the business process and ERP transactions and in the domain of technical knowledge for SAP ERP. Similarly, a quantitative study by Angolia and Pagliari [119] discussed an ERP simulation game to aid students’ conceptual learning of ERP systems in a supply chain course. ERPsim was also reported to increase students’ knowledge of business processes. The study revealed that the simulation enhanced the distribution supply chain management’s concepts, which increased the course content’s understanding. One of the selected studies [46] also revealed knowledge gains in business processes and basic business concepts using ERPsim. Other studies confirmed the positive effect of using ERPsim games on business processes and supply chain knowledge [83, 86]. The ENTRExplorer game had a positive effect on students’ accounting knowledge and business processes knowledge (Fernando [52]). Our review of selected papers revealed that simulations are directly linked to the course content, and their use allows applying a better understanding of theoretical concepts [41, 45, 81].

5.3.2. Skill Outcomes of Using BSGs

Cognitive abilities enable students to perform complex tasks such as logical critical thinking, problem-solving, and strategic decision-making [117]. These abilities are also sought after by the employers [120]. Twenty-eight papers were identified that reported the positive impact of using simulation games on students’ cognitive skills. For example, Levant et al. [73] conducted a pre-postsurvey to determine the effect of a business simulation game on students’ cognitive and interactive skills. This case study suggested positive results in the skills related to information management, communication, time management, project management, work-load management, team work, and problem-solving, which are considered higher-order cognitive skills [51]. The studies reporting the positive effects of using BSGs on the development and enhancement of cognitive skills are as follows: (i)Decision-making [39, 41, 46, 52, 67, 68, 73, 89, 117, 121](ii)Critical thinking and scientific reasoning [19, 38, 43, 66, 81, 90](iii)Problem-solving skills [19, 38, 43, 52, 54, 67, 73, 81, 90]

Simulation games are often perceived as powerful learning tools to enhance students’ interactive or social skills as they are often played in teams. Most of the papers reported the positive impact of simulation games on different interactive skills. (i)Teamwork [4, 19, 38, 41, 42, 46, 50, 73, 99, 122](ii)Collaboration [43, 47, 80](iii)Social and emotional skills [123](iv)Learner-learner interaction [41]; the paper examined the impact of a simulation game called ERPsim by analysing the messages posted by the students on an online discussion forum during the game. They observed students’ active participation and social interaction and reported an overall positive impact on interactive skills. Other studies reported that the interactive skills enhanced by playing BSGs are communication, persuasion, and conflict resolution

5.3.3. Behavioural Outcomes of BSGs

Many papers highlighted the behavioural and affective outcomes of computer simulation games. (i)Positive effects were reported concerning students’ engagement with the simulation activity [43, 46, 62, 65, 67, 68, 76, 83](ii)Increased motivation toward playing simulation [4, 41, 42, 64, 65, 69, 84](iii)Higher satisfaction levels with the course content [4, 42, 47, 65, 84](iv)Increased self-efficacy [39, 50, 64, 66, 68, 76](v)Increased self-awareness and confidence [38, 39, 68, 91]

To conclude, most of the selected papers reveal positive effects regarding the use of computer simulation games on learning outcomes. Figure 12 shows the most common learning outcomes of BSGs identified in the selected papers of this review.

5.4. Evaluation Methods Used in BSG Literature

Analysis of the selected studies shows the absence of a uniform agreed evaluation framework to assess the impact of using simulation games in learning [124]. Instructors use different assessment methods to evaluate the simulation activity performance of students and other evaluation methods used by the researchers to assess the impact of simulation activity on the learning outcomes in the selected studies. (i)Instructors used game results (profit gains) to indicate better game performance [39, 42, 79, 80, 83](ii)Reflection reports submitted by the students act as a formative assessment for the game activity [19, 66]. Reflection reports usually contain questions related to students’ opinions on the simulation activities, their perceived gains from participating in the activity, their intention to use it again, the decisions they made, and the mistakes they wanted to rectify in the next session(iii)Exam grades are used as performance indicators as objective measures to assess students’ knowledge gains [43, 79]

During the pregame stage, questionnaires are the most frequently used assessment tools. Researchers can use this tool to collect data from many respondents regarding their previous knowledge, demographic information, perceptions of innovative pedagogical tools, and prior learning styles. This information can help educators set goals and the pace and pattern of the game as they can indicate students’ current knowledge and skill levels. Pregame surveys are rarely stand-alone. They are always followed by postgame surveys, which help researchers compare and analyse students’ knowledge gains and achievement of the desired learning outcomes. Out of forty-nine selected studies, six [47, 65, 66, 73, 86, 89] used pregame surveys followed by postgame surveys.

The in-game assessments include observations of players while playing the game, in-group discussion, and reflection reports. Classroom observation is helpful to obtain instant feedback on game experience and enable a cognitive analysis of students as players to verbalise their thinking and decision-making process during discussions. In addition, observations provide insight into students’ interaction with the interface and other game designs [85]. Reflection reports are also a part of in-game assessment where students qualitatively assess their learning outcomes, experience, and knowledge gain. Reflection reports can be conducted once or multiple times during the game. Overall, in-game assessments measure students’ mood, understanding, learning, and performance. The literature search revealed the importance of in-game assessment as only this can provide instant feedback about overall learning processes and other behavioural changes resulting from this process [125]. However, it is the least adopted method in BSG literature because of its complexity, time constraints, and issues regarding permission to access participants.

Postgame assessments are the most used evaluation methods in the selected studies (45) and assess students’ actual knowledge gain and improvement in skills. In this assessment, games are evaluated considering their usefulness in achieving desired learning outcomes. Questionnaires are the most common tool for postgame assessment, followed by interviews and focus groups. Although most studies used quantitative surveys to conduct postgame assessments, a few used qualitative methods. For example, Loon et al. [81] conducted interviews with 36 students to explore their simulation activity views after they had taken part in the simulations. Goi [4] conducted focus group discussions with 17 students who played Total Enterprise Simulation as part of an undergraduate course to explore the role of authentic team-based learning in enhancing students’ learning outcomes and satisfaction. In several studies, game results (financial and nonfinancial indicators) are also used as postgame assessments [39, 66, 79, 92]).

The most popular assessment tool is questionnaires due to their ease of use with larger sample sizes. However, qualitative assessments such as reflection reports, classroom observation, and interviews give more profound insight into the learning experience [126]. In the research, there was not an assessment framework used to evaluate BSGs, as most studies used self-developed frameworks. This lack of a univocal assessment framework raises questions about the validity of the results. The analysis of the 57 selected papers shows that the evaluations are done at different phases of BSGs. Researchers used other qualitative and quantitative measurements to assess the impact of BSGs on learning outcomes. Based on the analysis, researchers presented a comprehensive evaluation framework (Figure 13) to evaluate the learning outcomes of simulation games. Instead of focusing on just a single point of time. Different learning outcomes are gained at various game stages; this framework can help educators assess them all.

6. Limitations and Future Research Directions in Selected Studies

This literature review of the BSG literature found three significant limitations that point to future research recommendations. (i)The most common limitations in the selected studies are the limited context of the study [54, 64, 79, 81] and the small convenience sample size which created issues in generalizing the results [19, 43, 64–66, 78, 85, 89]

Future recommendations: different studies proposed different recommendations to overcome these limitations. Vos [43] suggested future studies with a broader cross section of BSG users from business and other fields. The diverse sample will include participants with different characteristics, which may increase the chances of generalizability. Tao et al. [64] advised the expansion of the sample sizes in future studies to cover more classes in different universities for more in-depth and extensive analyses. Loon et al. [81] recommended collecting data from various sources (e.g. students’ academic records) to establish the relationship between learning through simulation and academic performance. Urquidi-Martín et al. [54] conducted postgame surveys from 58 students from a single university in Romania. She suggested future research comparing the impact of using BSGs in different cultures and countries. (ii)Another limitation discussed in selected studies is either including limited variables or ignoring moderating and meditating factors that can influence the effectiveness of using BSGs to enhance learning outcomes

Future recommendations: Chen et al. [44] only examined two factors (enjoyment and cognitive appraisal). He recommended that more elements be included in the learning outcomes model to understand learning behaviour and outcomes, for example, students’ concentration, curiosity, innovative attitudes, personal skills in IT, and understanding of business processes. [67] compared the learning outcomes of three teaching methods (lectures, case studies, and simulation) by conducting surveys with students and instructors. The study did not consider the wide range of individual and group-level variables affecting BSGs’ performance and students’ learning outcomes. To overcome this limitation, he suggested investigating the factors that help predict or influence the use of BSGs may impact students’ attainment of learning outcomes. Additional variables that can be analysed in the future were presented by Buil et al. [76]. The study focused on personal factors but not on contextual factors. For future studies, the study suggested that contextual variables should be analysed. Other learning components, such as retention or transfer learning, can be assessed objectively (e.g., students’ academic grades and application tests). (iii)Students’ perceptions and reports about learning outcomes were also a limitation in a few studies

Future recommendations: Burdon and Munro [45] suggested further research to focus on students’ objective performance instead of just including students’ perceptions. Similarly, [47] only used students’ self-reported learning outcomes to measure simulation activity success. This can present a bias in the results as many factors can influence students’ responses. To reduce this bias, in future studies, researchers suggested using objective measures such as student grades and application tests to measure learning. Another study [65] advised conducting interviews with instructors in addition to surveys with students to determine teachers’ viewpoints concerning the use of digital games to facilitate learning. The researchers also suggested investigating factors such as students’ self-efficacy, population demographic characteristics, curriculum, and instructors’ experience with BSGs in future studies focusing on the impact of game play on learning efficiency. The results in another study [4] were only based on students’ self-reported measures obtained by conducting postgame surveys with students. However, other factors like differences in student demographic characteristics and teaching approaches used by instructors, which may impact learning outcomes, were not considered. To overcome these limitations, researchers suggested a future study where objective measures of learning gains, for example, grades, alongside self-reported measures, for instance, postgame surveys, can be used to reduce biases in studies. Another suggestion was to explore instructors’ views regarding teamwork facilitating the acquisition and development of learning outcomes when using online BSGs.

The selected studies in this review presented myriad future research recommendations in the growing field of BSG research. These recommendations can further expand researchers’ and instructors’ knowledge to take advantage of the full potential of using BSGs in teaching and learning.

7. Conclusion

This paper has presented a systematic literature review of the empirical research on BSGs published between January 2015 and February 2022. The analysis of the fifty-seven selected studies, fulfilling the inclusion and exclusion criteria set by researchers, presented four significant themes within BSG research. These themes are techniques for the integration of BSGs in the existing curriculum, the use of BSGs to achieve learning outcomes, evaluation methods used in BSGs by instructors and researchers, and learning theories used to explain learning processes in BSG use. A significant part of BSG research is concerned with the use of BSGs to attain learning outcomes, the assessment of those outcomes, and integration of BSGs to achieve those leering outcomes. Figure 14 presents an overview of the findings of this review. As presented, BSGs follow the experiential learning cycle and by doing so can be used to attain verity of learning outcomes at different stages of games. Educators usually use one or mix of assessment methods to evaluate the learning outcomes.

All empirical studies included in this review focused on the impact that the use of BSGs might have on the achievement of different learning outcomes: skill development, knowledge acquisition, and/or behavioural outcomes. Three conclusions can be drawn from this review: (i)The use of BSGs in higher education courses positively affects students’ achievement of learning outcomes(ii)Three critical factors should be kept in mind while integrating BSGs in existing courses (integration process, the specificity of the game, and the instructors’ role)(iii)There is a need for a univocal multistage evaluation framework (from both students’ and instructors’ perspectives) to assess how using BSGs might enhance learning outcomes

Like all SLRs, this research has a few limitations that should be considered when evaluating the reported findings: limited search terms, period of the paper published, and the databases included. However, the articles in this SLR provide an overview of the recent empirical research on BSGs. Given the growing number of studies in this field, several papers that discuss the impact of computer simulation games on learning outcomes may have been omitted in this study. However, it would not have been a deliberate, systematic omission. The review excluded theoretical and opinion papers and only included empirical studies as it was necessary to ground our understanding of the effects of the use of BSGs on students’ attainment of learning outcomes in research evidence rather than suppositions.

Another limitation is that the review only included open access papers. The choice regarding the inclusion of open access papers is motivated by their worldwide visibility without barriers. These open access publications reach nonscholarly, industry, and academic audiences in less developed areas of the world, where there is minimal access to commercial databases and journals [127]. The sample excluded dissertations, books, and grey literature. The significance of this is that the quality standards of peer-reviewed publications do not have to be fulfilled by grey literature. Articles reviewed by academics must conform to the standards of the peer reviewer and the editorial board responsible for ensuring the journal’s content and quality [128]. Therefore, many studies were excluded from the review, which might otherwise have provided additional insight to this review. It is important to note that, although not considered a literary form of publication, grey literature is often produced by experts in the field [129]. In future studies, the grey literature can be used to minimise publication bias, improve the comprehensiveness of the review, and offer a more informed image of the available evidence [130].

Our review will contribute to the BSG literature by consolidating the latest evidence to help researchers to explore the effective use of BSGs to facilitate learning processes. The study has many methodological and applied implications. From a methodological standpoint, we suggest that researchers consider the concordance between learning outcomes and the evaluation measures used to assess them. While most of the papers used surveys to determine knowledge and skill gains, combining different learning measures could be more appropriate. We suggest a multistage multimethod evaluation strategy to capture the full potential of using the BSGs to enhance student attainment of learning outcomes as each stage presents an opportunity for students to gain different skills and knowledge. An authentic and embedded assessment should include multiple measurement procedures to cover various aspects of games [125], reduce bias, and increase the results’ validity. Another piece of advice for the researchers is to consider both instructors’ and students’ perspectives while evaluating the impact of using BSGs on student achievement of learning outcomes as it would reduce biases in results. From an applied standpoint, this review may help instructors choose the best possible BSG according to their needs. It may also allow universities to integrate these games effectively into existing graduate and undergraduate courses. Our findings present an extensive avenue for future research in the BSG field, especially regarding the effects of BSGs on employability and transferrable skills of graduates.

Data Availability

Since this paper is a systematic review of published articles, it does not involve primary data collection, and there were no data to make available for public access. Similarly, no ethics approval was necessary as this paper was a review paper.

Conflicts of Interest

The authors declare no conflict of interest in the work reported here.

References

P. B. Shahab Khan and V. Kumar, “Virtual training and simulation market by component (hardware and software) and end user (defense & security, civil aviation, education, entertainment, and others): global opportunity analysis and industry forecast,” no. A01425, pp. 2020–2027, 2020, https://www.alliedmarketresearch.com/virtual-training-and-simulation-market.
View at: Google Scholar
C. Kerdvibulvech, “Geo-based mixed reality gaming market analysis,” Human Behavior and Emerging Technologies, vol. 2022, article 1139475, pp. 1–9, 2022.
View at: Publisher Site | Google Scholar
J. Zeng, S. Parks, and J. Shang, “To learn scientifically, effectively, and enjoyably: a review of educational games,” Human Behavior and Emerging Technologies, vol. 2, no. 2, pp. 186–195, 2020.
View at: Publisher Site | Google Scholar
C.-L. Goi, “The use of business simulation games in teaching and learning,” Journal of Education for Business, vol. 94, no. 5, pp. 342–349, 2019.
View at: Publisher Site | Google Scholar
M. L. Joseph, “Why the jury is still out on business simulation games,” In Handbook of Teaching with Technology in Management, Leadership, and Business, Edward Elgar Publishing, 2020.
View at: Publisher Site | Google Scholar
M. Newman and D. Gough, “Systematic reviews in educational research: methodology, perspectives and application,” Systematic Reviews in Educational Research, pp. 3–22, 2020.
View at: Publisher Site | Google Scholar
O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, and K. Buntins, Systematic Reviews in Educational Research: Methodology, Perspectives and Application, Springer Nature, 2020.
View at: Publisher Site
A. Booth, A. Sutton, and D. Papaioannou, Systematic Approaches to a Successful Literature Review, Sage, 2016.
K. Fu, T. Hainey, and G. Baxter, “A systematic literature review to identify empirical evidence on the use of computer games in business education and training,” in European Conference on Games Based Learning. Academic Conferences International Limited, United Kingdom, 2016.
View at: Google Scholar
F. M. Ricciardi and E. Marting, Top management decision simulation:The AMA approach, American Management Association, 1957.
P. Kaminsky and D. Simchi-Levi, “A new computerized beer game: a tool for teaching the value of integrated supply chain management,” Global Supply Chain Technology Management, vol. 1, no. 1, pp. 216–225, 1998.
View at: Google Scholar
J.-C. Larreche and H. Gatignon, “MARKSTRAT: a marketing strategy game,” Course Technology, vol. 1, 1977.
View at: Google Scholar
S. Malik, M. Chetty, and M. Chadhar, “Information technology and organizational learning interplay: a survey,” in Proceedings of the Australasian Conference on Information Systems Malik, Chetty & Mehmood, Sydney, Australasian, 2018.
View at: Publisher Site | Google Scholar
G. J. Summers, “Today’s business simulation industry,” Simulation Gaming, vol. 35, no. 2, pp. 208–241, 2004.
View at: Publisher Site | Google Scholar
M. Rokicki, S. Chelaru, S. Zerr, and S. Siersdorfer, “Competitive game designs for improving the cost effectiveness of crowdsourcing,” in Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1469–1478, Shanghai China, 2014.
View at: Publisher Site | Google Scholar
A. Uskov and B. Sekar, “Serious games, gamification and game engines to support framework activities in engineering: case studies, analysis, classifications and outcomes,” in IEEE International Conference on Electro/Information Technology, Milwaukee, WI, USA, June 2014.
View at: Publisher Site | Google Scholar
M. P. Bach, J. Zoroja, and M. Fašnik, “Teaching business simulation games: preliminary current practice overview,” in Proceedings of the International Conference: Theory and Applications in the Knowledge Economy, Lisbon: Eduardo Tome, 2017.
View at: Google Scholar
B. Coffey and S. Anderson, “The students’ view of a business simulation: perceived value of the learning experience,” Journal of Strategic Management Education, vol. 3, no. 1, pp. 151–168, 2006.
View at: Google Scholar
Y. Costin, M. P. O’Brien, and B. Hynes, “Developing cognitive and non-cognitive entrepreneurial competences through business simulation games,” Journal of Enterprising Culture, vol. 27, no. 4, pp. 471–498, 2019.
View at: Publisher Site | Google Scholar
C. P. Ferreira, C. S. González-González, and D. F. Adamatti, “Business simulation games analysis supported by human-computer interfaces: a systematic review,” Sensors, vol. 21, no. 14, p. 4810, 2021.
View at: Publisher Site | Google Scholar
M. Stanitsas, K. Kirytopoulos, and E. Vareilles, “Facilitating sustainability transition through serious games: a systematic literature review,” vol. 208, pp. 924–936, 2019.
View at: Publisher Site | Google Scholar
S. Subhash and E. A. Cudney, “Gamified learning in higher education: a systematic review of the literature,” Computers in Human Behavior, vol. 87, pp. 192–206, 2018.
View at: Publisher Site | Google Scholar
E. A. Boyle, T. Hainey, T. M. Connolly, G. Gray, J. Earp, and M. Ott, “An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games,” Computers & Education, vol. 94, pp. 178–192, 2016.
View at: Publisher Site | Google Scholar
H. W. Giessen, “Serious games effects: an overview,” Procedia-Social and Behavioral Sciences, vol. 174, pp. 2240–2244, 2015.
View at: Publisher Site | Google Scholar
J. Hamari, J. Koivisto, and H. Sarsa, “Does gamification work?-a literature review of empirical studies on gamification,” in 2014 47th Hawaii international conference on system sciences, Waikoloa, HI, USA, January 2014.
View at: Publisher Site | Google Scholar
M. C. Lopes, F. A. Fialho, C. J. Cunha, S. I. J. S. Niveiros, and Gaming, “Business games for leadership development,” A Systematic Review, vol. 44, no. 4, pp. 523–543, 2013.
View at: Publisher Site | Google Scholar
P. Wouters, C. Van Nimwegen, H. Van Oostendorp, and E. D. Van Der Spek, “A meta-analysis of the cognitive and motivational effects of serious games,” Journal of Educational Psychology, vol. 105, no. 2, pp. 249–265, 2013.
View at: Publisher Site | Google Scholar
T. M. Connolly, E. A. Boyle, E. MacArthur, T. Hainey, and J. M. Boyle, “A systematic literature review of empirical evidence on computer games and serious games,” Cmputers Education, vol. 59, no. 2, pp. 661–686, 2012.
View at: Publisher Site | Google Scholar
S. Tobias and J. D. Fletcher, “Reflections on “a review of trends in serious gaming”,” Review of Educational Research, vol. 82, no. 2, pp. 233–237, 2012.
View at: Publisher Site | Google Scholar
T. J. P. Sitzmann, “A meta-analytic examination of the instructional effectiveness of computer-based simulation games,” Personnel Psychology, vol. 64, no. 2, pp. 489–528, 2011.
View at: Publisher Site | Google Scholar
M. Jahangirian, T. Eldabi, A. Naseer, L. K. Stergioulas, and T. Young, “Simulation in manufacturing and business: a review,” A Review, vol. 203, no. 1, pp. 1–13, 2010.
View at: Publisher Site | Google Scholar
F. Scholtz and S. Hughes, “A systematic review of educator interventions in facilitating simulation based learning,” Journal of Applied Research in Higher Education, vol. 13, no. 5, 2019.
View at: Google Scholar
C. Okoli and K. Schabram, “A guide to conducting a systematic literature review of information systems Research,” Sprouts: Working Papers on Information Systems, vol. 10, no. 26, 2010, http://sprouts.aisnet.org/10-26.
View at: Google Scholar
S. Keele, Guidelines for performing systematic literature reviews in software engineering, Technical report, ver. 2.3 ebse, 2007.
T. Connolly, M. Stansfield, and T. Hainey, “Development of a general framework for evaluating games-based learning,” in Proceedings of the 2nd European conference on games-based learning, Universitat Oberta de Catalunya, 2008.
View at: Google Scholar
J. D. Lecy and K. E. Beatty, “Representative literature reviews using constrained snowball sampling and citation network analysis,” 2012, Available at SSRN 1992601.
View at: Publisher Site | Google Scholar
T. Dybå and T. Dingsøyr, “Strength of evidence in systematic reviews in software engineering,” in Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pp. 178–187, Kaiserslautern Germany, October 2008.
View at: Publisher Site | Google Scholar
M. Severengiz, G. Seliger, and J. Krüger, “Serious game on factory planning for higher education,” Procedia Manufacturing, vol. 43, pp. 239–246, 2020.
View at: Publisher Site | Google Scholar
N. Tawil, R. Hassan, S. Ramlee, and Z. K-Batcha, “Enhancing entrepreneurship skill among university’s students by online business simulation,” Journal of Engineering Science and Technology, vol. 6, pp. 71–80, 2015.
View at: Google Scholar
M. Islam, A. Abdul Rahim, C. L. Tan, and H. Momtaz, “Effect of demographic factors on e-learning effectiveness in a higher learning institution in Malaysia,” International Education Studies, vol. 4, no. 1, pp. 112–121, 2011.
View at: Publisher Site | Google Scholar
A. B. Hernández-Lara and E. Serradell-López, “Student interactions in online discussion forums: their perception on learning with business simulation games,” Behaviour Information Technology, vol. 37, no. 4, pp. 419–429, 2018.
View at: Publisher Site | Google Scholar
W. C. Kriz and E. Auchter, “10 years of evaluation research into gaming simulation for German entrepreneurship and a new study on its long-term effects,” Simulation Gaming, vol. 47, no. 2, pp. 179–205, 2016.
View at: Publisher Site | Google Scholar
L. Vos, “Simulation games in business and marketing education: how educators assess student learning from simulations,” The International Journal of Management Education, vol. 13, no. 1, pp. 57–74, 2015.
View at: Publisher Site | Google Scholar
L. Chen, A. Keys, and D. Gaber, “How does ERPsim influence student’s perceived learning outcomes in an information systems course? An empirical study,” Journal of Information Systems Education, vol. 26, no. 2, p. 135, 2015.
View at: Google Scholar
W. M. Burdon and K. Munro, “Simulation–is it all worth it? The impact of simulation from the perspective of accounting students,” The International Journal of Management Education, vol. 15, no. 3, pp. 429–448, 2017.
View at: Publisher Site | Google Scholar
L. B. Eder, Y. L. Antonucci, and E. Monk, “Developing a framework to understand student engagement, team dynamics, and learning outcomes using ERPsim,” Journal of Information Systems Education, vol. 30, no. 2, p. 127, 2019.
View at: Google Scholar
P. Bitrián, I. Buil, and S. Catalán, “Flow and business simulation games: a typology of students,” The International Journal of Management Education, vol. 18, no. 1, p. 100365, 2020.
View at: Publisher Site | Google Scholar
M. G. A. Ghani and N. Mohammad, “The integration of logic model in business plan simulation approach for effective entrepreneurial learning,” Jurnal Pengurusan, vol. 62, 2021.
View at: Publisher Site | Google Scholar
M. Thanasi-Boçe, “Enhancing students’ entrepreneurial capacity through marketing simulation games,” Education+ Training, vol. 62, no. 9, pp. 999–1013, 2020.
View at: Publisher Site | Google Scholar
K. Õun, M. Mägi, and A. Noppel, “Learning business through simulation games. Survey among students who played developed games,” Perspectives on Computer Gaming in Higher Education, p. 89, 2016.
View at: Google Scholar
K. J. Lovelace, F. Eggers, and L. R. Dyck, “I do and I understand: assessing the utility of web-based management simulations to develop critical thinking skills,” Academy of Management Learning Education, vol. 15, no. 1, pp. 100–121, 2016.
View at: Publisher Site | Google Scholar
F. Almeida, “Learning entrepreneurship with serious games-a classroom approach,” 2017, http://arxiv.org/abs/04118.
View at: Google Scholar
K. Mohsen, S. Abdollahi, and S. Omar, “Evaluating the educational value of simulation games: Learners’ perspective,” Innovations in Education Teaching International, vol. 56, no. 4, pp. 517–528, 2019.
View at: Publisher Site | Google Scholar
A. C. Urquidi-Martín, C. Tamarit-Aznar, and J. Sánchez-García, “Determinants of the effectiveness of using renewable resource management-based simulations in the development of critical thinking: an application of the experiential learning theory,” Sustainability, vol. 11, no. 19, p. 5469, 2019.
View at: Publisher Site | Google Scholar
F. Almeida and Z. Buzady, “Assessment of entrepreneurship competencies through the use of FLIGBY,” Digital Education Review, vol. 35, pp. 151–169, 2019.
View at: Publisher Site | Google Scholar
Z. Buzady and F. Almeida, “FLIGBY—a serious game tool to enhance motivation and competencies in entrepreneurship,” Informatics, vol. 6, no. 3, p. 27, 2019.
View at: Publisher Site | Google Scholar
S. Zulfiqar, H. A. Al-reshidi, M. A. Al Moteri, H. M. B. Feroz, N. Yahya, and W. M. Al-Rahmi, “Understanding and predicting students’ entrepreneurial intention through business simulation games: a perspective of COVID-19,” Sustainability, vol. 13, no. 4, p. 1838, 2021.
View at: Publisher Site | Google Scholar
M. S. Yusof, “Business simulation as key motivation factor in stimulating business intention amongst university students in Malaysia,” Advanced International Journal of Business, Entrepreneurship and SME's, vol. 3, no. 10, pp. 69–74, 2021.
View at: Publisher Site | Google Scholar
D. Lovin, M. Raducan, A. Capatina, and N. Cristache, “Sustainable knowledge transfer from business simulations to working environments: correlational vs. configurational approach,” Sustainability, vol. 13, no. 4, p. 2154, 2021.
View at: Publisher Site | Google Scholar
C. F. Dharmastuti, S. Darmoyo, R. A. Gunawan, and M. N. Duka, “Business simulation, student competency, and learning outcomes,” Jurnal Manajemen, vol. 25, no. 1, pp. 160–174, 2021.
View at: Publisher Site | Google Scholar
N. Faisal, M. Chadhar, A. Stranieri, and A. Gorris-Hunter, “Effects of simulation games on IS students’ work-readiness: instructors’ perspectives,” in American Conference on Information Systems, 2022.
View at: Google Scholar
T. Beranič and M. Heričko, “The impact of serious games in economic and business education: a case of ERP business simulation,” Sustainability, vol. 14, no. 2, p. 683, 2022.
View at: Publisher Site | Google Scholar
R. Hishiyama and Y. Nakajima, “Business game-based experimental active learning using a multiagent approach for management education,” in 2015 3rd International Conference on Applied Computing and Information Technology/2nd International Conference on Computational Science and Intelligence, Okayama, Japan, July 2015.
View at: Publisher Site | Google Scholar
Y.-H. Tao, C.-C. R. Yeh, and K. C. Hung, “Validating the learning cycle models of business simulation games via student perceived gains in skills and knowledge,” Educational Technology Society, vol. 18, no. 1, pp. 77–90, 2015.
View at: Google Scholar
A. Ellahi, B. Zaka, and F. Sultan, “A study of supplementing conventional business education with digital games,” Journal of Educational Technology & Society, vol. 20, no. 3, pp. 195–206, 2017.
View at: Google Scholar
D. Williams, “The impact of SimVenture on the development of entrepreneurial skills in management students,” Industry Higher Education, vol. 29, no. 5, pp. 379–395, 2015.
View at: Publisher Site | Google Scholar
F. Beuk, “Sales simulation games,” Journal of Marketing Education, vol. 38, no. 3, pp. 170–182, 2016.
View at: Publisher Site | Google Scholar
S. Zulfiqar, B. Sarwar, S. Aziz, K. Ejaz Chandia, and M. K. Khan, “An analysis of influence of business simulation games on business school students’ attitude and intention toward entrepreneurial activities,” Journal of Educational Computing Research, vol. 57, no. 1, pp. 106–130, 2019.
View at: Publisher Site | Google Scholar
T. Rogmans and W. Abaza, “The impact of international business strategy simulation games on student engagement,” Simulation Gaming, vol. 50, no. 3, pp. 393–407, 2019.
View at: Publisher Site | Google Scholar
Y.-Y. Wang, Y.-S. Wang, and S.-E. Jian, “Investigating the determinants of students’ intention to use business simulation games,” Journal of Educational Computing Research, vol. 58, no. 2, pp. 433–458, 2020.
View at: Publisher Site | Google Scholar
T. Kiss and R. Schmuck, “A longitudinal study of the skills and attitudes conveyed by two business simulation games in Pécs, Hungary,” Simulation & Gaming, vol. 52, no. 4, pp. 435–464, 2021.
View at: Publisher Site | Google Scholar
T. M. Kuang, R. W. Adler, and R. Pandey, “Creating a modified monopoly game for promoting students’ higher-order thinking skills and knowledge retention,” Issues in Accounting Education, vol. 36, no. 3, pp. 49–74, 2021.
View at: Publisher Site | Google Scholar
Y. Levant, M. Coulmont, and R. Sandu, “Business simulation as an active learning activity for developing soft skills,” Accounting Education, vol. 25, no. 4, pp. 368–395, 2016.
View at: Publisher Site | Google Scholar
S. L. Humpherys, N. Bakir, and J. Babb, “Experiential learning to foster tacit knowledge through a role play, business simulation,” Journal of Education for Business, vol. 97, no. 2, pp. 119–125, 2022.
View at: Publisher Site | Google Scholar
B. I. N. Obi, T. I. Eze, and N. F. Chibuzo, “Experiential learning activities in business education for developing 21st century competencies,” Journal of Education for Business, vol. 97, no. 1, pp. 36–42, 2022.
View at: Publisher Site | Google Scholar
I. Buil, S. Catalán, and E. Martínez, “Engagement in business simulation games: a self-system model of motivational development,” British Journal of Educational Technology, vol. 51, no. 1, pp. 297–311, 2020.
View at: Publisher Site | Google Scholar
A. Olive, M. Carre, J. Cuadros, L. Gonzalez-Sabate, and V. Serrano, “Tracking the behavior of players in a finance simulation and identifying work patterns,” Procedia-Social Behavioral Sciences, vol. 182, pp. 197–202, 2015.
View at: Publisher Site | Google Scholar
J. Carenys, S. Moya, and J. Perramon, “Is it worth it to consider videogames in accounting education? A comparison of a simulation and a videogame in attributes, motivation and learning outcomes,” Revista de Contabilidad-Spanish Accounting Review, vol. 20, no. 2, pp. 118–130, 2017.
View at: Publisher Site | Google Scholar
A. L. Leal-Rodriguez and G. Albort-Morant, “Promoting innovative experiential learning practices to improve academic performance: empirical evidence from a Spanish business school,” Journal of Innovation Knowledge, vol. 4, no. 2, pp. 97–103, 2019.
View at: Publisher Site | Google Scholar
I. A. Ștefan, J. B. Hauge, F. Hasse, and A. Ștefan, “Using serious games and simulations for teaching co-operative decision-making,” Procedia Computer Science, vol. 162, pp. 745–753, 2019.
View at: Publisher Site | Google Scholar
M. Loon, J. Evans, and C. Kerridge, “Learning with a strategic management simulation game: a case study,” The International Journal of Management Education, vol. 13, no. 3, pp. 227–236, 2015.
View at: Publisher Site | Google Scholar
R. Bell and M. Loon, “The impact of critical thinking disposition on learning using business simulations,” The International Journal of Management Education, vol. 13, no. 2, pp. 119–127, 2015.
View at: Publisher Site | Google Scholar
H.-M. Lee, J. Long, and L. Visinescu, “The relationship between a business simulator, constructivist practices, and motivation toward developing business intelligence skills,” Journal of Information Technology Education: Research, vol. 15, pp. 593–609, 2016.
View at: Publisher Site | Google Scholar
J. Pando-Garcia, I. Periañez-Cañadillas, and J. Charterina, “Business simulation games with and without supervision: an analysis based on the TAM model,” Journal of Business Research, vol. 69, no. 5, pp. 1731–1736, 2016.
View at: Publisher Site | Google Scholar
H.-H. Lin, W.-C. Yen, and Y.-S. Wang, “Investigating the effect of learning method and motivation on learning performance in a business simulation system context: an experimental study,” Computers Education, vol. 127, pp. 30–40, 2018.
View at: Publisher Site | Google Scholar
M. Hwang and K. Cruthirds, “Impact of an ERP simulation game on online learning,” The International Journal of Management Education, vol. 15, no. 1, pp. 60–66, 2017.
View at: Publisher Site | Google Scholar
A. C. Urquidi Martín and C. Tamarit Aznar, “Meaningful learning in business through serious games,” Intangible Capital, vol. 13, no. 4, pp. 805–823, 2017.
View at: Publisher Site | Google Scholar
I. Mustata, C. Alexe, and C. Alexe, “Developing competencies with the general management II business simulation game,” International journal of simulation modelling, vol. 16, no. 3, pp. 412–421, 2017.
View at: Publisher Site | Google Scholar
R. Newbery, J. Lean, J. Moizer, and M. Haddoud, “Entrepreneurial identity formation during the initial entrepreneurial experience: the influence of simulation feedback and existing identity,” Journal of Business Research, vol. 85, pp. 51–59, 2018.
View at: Publisher Site | Google Scholar
P. Torres and M. Augusto, “The impact of experiential learning on managers’ strategic competencies and decision style,” Journal of Innovation Knowledge, vol. 2, no. 1, pp. 10–14, 2017.
View at: Publisher Site | Google Scholar
M. Farashahi and M. Tajeddin, “Effectiveness of teaching methods in business education: a comparison study on the learning outcomes of lectures, case studies and simulations,” The International Journal of Management Education, vol. 16, no. 1, pp. 131–142, 2018.
View at: Publisher Site | Google Scholar
R. Alas, P. Kross, A. Liivat, and J. Saar, “Success factors for teams in business game Dynama,” Problems Perspectives in Management, vol. 16, no. 1, pp. 309–319, 2018.
View at: Publisher Site | Google Scholar
M. S. Calabor, A. Mora, and S. Moya, “The future of ‘serious games’ in accounting education: a Delphi study,” Journal of Accounting Education, vol. 46, pp. 43–52, 2019.
View at: Publisher Site | Google Scholar
S. A. Samaras, C. L. Adkins, and C. D. White, “Developing critical thinking skills: simulationsvs.cases,” Journal of Education for Business, vol. 97, no. 4, pp. 270–276, 2022.
View at: Publisher Site | Google Scholar
A. Calderón, M. Ruiz, and E. Orta, “Integrating serious games as learning resources in a software project management course: the case of ProDec,” in 2017 IEEE/ACM 1st International Workshop on Software Engineering Curricula for Millennials (SECM), Buenos Aires, Argentina, May 2017.
View at: Publisher Site | Google Scholar
A. Oliveira, R. Feyzi Behnagh, L. Ni, A. A. Mohsinah, K. J. Burgess, and L. Guo, “Emerging technologies as pedagogical tools for teaching and learning science: a literature review,” Human Behavior and Emerging Technologies, vol. 1, no. 2, pp. 149–160, 2019.
View at: Publisher Site | Google Scholar
E. Weenk, Mastering the Supply Chain: Principles, Practice and Real-life Applications, Kogan Page Publishers, 2019.
S. R. Tiwari, L. Nafees, and O. Krishnan, “Simulation as a pedagogical tool: measurement of impact on perceived effective learning,” The International Journal of Management Education, vol. 12, no. 3, pp. 260–270, 2014.
View at: Publisher Site | Google Scholar
J. Moizer, J. Lean, M. Towler, and C. Abbey, “Simulations and games,” Active Learning in Higher Education, vol. 10, no. 3, pp. 207–224, 2009.
View at: Publisher Site | Google Scholar
L. Proserpio and D. A. Gioia, “Teaching the virtual generation,” Academy of Management Learning Education, vol. 6, no. 1, pp. 69–80, 2007.
View at: Publisher Site | Google Scholar
E. Salas, J. L. Wildman, and R. F. Piccolo, “Using simulation-based training to enhance management education,” Academy of Management Learning Education, vol. 8, no. 4, pp. 559–573, 2009.
View at: Google Scholar
M.-L. Wang, R.-Y. Chang, and C.-H. D. Hsu, Molding Simulation: Theory and Practice, Carl Hanser Verlag GmbH Co KG, 2018.
View at: Publisher Site
D. Kolb, “Experiential learning experience as the source of learning and development,” Tech. Rep., Upper Sadle River: Prentice Hall, 1984.
View at: Google Scholar
F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS Quarterly, vol. 13, pp. 319–340, 1989.
View at: Publisher Site | Google Scholar
I. Ajzen, “From intentions to actions: a theory of planned behavior,” Action Control, Springer, pp. 11–39, 1985.
View at: Publisher Site | Google Scholar
M. Csikszentmihalyi, “The flow experience and its significance for human psychology,” Optimal Experience: Psychological Studies of Flow in Consciousness, vol. 2, pp. 15–35, 1988.
View at: Publisher Site | Google Scholar
F. Liñán, J. C. Rodríguez-Cohard, and J. M. Rueda-Cantuche, “Factors affecting entrepreneurial intention levels: a role for education,” International entrepreneurship management Journal, vol. 7, no. 2, pp. 195–218, 2011.
View at: Publisher Site | Google Scholar
K. Kiili, “Foundation for problem-based gaming,” British Journal of Educational Technology, vol. 38, no. 3, pp. 394–404, 2007.
View at: Publisher Site | Google Scholar
R. Garris, R. Ahlers, and J. E. Driskell, “Games, motivation, and learning: a research and practice model,” Simulation gaming, vol. 33, no. 4, pp. 441–467, 2002.
View at: Publisher Site | Google Scholar
A. Leontiev, “Activity, consciousness, and personality,” Englewood Cliffs, NJ Prentice Hall, 1978.
View at: Google Scholar
J. P. Connell and J. G. Wellborn, “Competence, Autonomy, and Relatedness: A Motivational Analysis of Self-System Processes,” Self processes and development, Lawrence Erlbaum Associates, Inc., pp. 43–77, 1991.
View at: Google Scholar
J. Eccles, Expectancies, values and academic behaviors, Achievement achievement motives, 1983.
D. Shepherd and J. M. Haynie, “Family business, identity conflict, and an expedited entrepreneurial process: a process of resolving identity conflict,” Entrepreneurship theory practice, vol. 33, no. 6, pp. 1245–1264, 2009.
View at: Publisher Site | Google Scholar
V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User acceptance of information technology: toward a unified view,” MIS Quarterly, vol. 27, pp. 425–478, 2003.
View at: Publisher Site | Google Scholar
P.-M. Léger, D. Lyle, G. Babin, P. Charland, and R. Pellerin, “Scope management: a core information system implementation project pedagogy,” International Education Studies, vol. 6, no. 3, pp. 55–65, 2013.
View at: Publisher Site | Google Scholar
B. Welz and A. Rosenberg, SAP Next-Gen: Innovation with Purpose, Springer, 2018.
View at: Publisher Site
M. Loon and R. Bell, “The moderating effects of emotions on cognitive skills,” Journal of Further Higher Education, vol. 42, no. 5, pp. 694–707, 2018.
View at: Publisher Site | Google Scholar
P. Léger, P. Charland, J. Robert, P. Cronan, and G. Babin, “Assessing the dimensionality of an objective measurement of ERP knowledge,” 2009, No. 09-05. Working paper.
View at: Google Scholar
M. G. Angolia and L. R. Pagliari, “Experiential learning for logistics and supply chain management using an SAP ERP software simulation,” Learning for Logistics and Supply Chain Management Using an SAP ERP Software Simulation, vol. 16, no. 2, pp. 104–125, 2018.
View at: Publisher Site | Google Scholar
N. Faisal, M. Chadhar, A. Stranieri, and A. Gorris-Hunter, “Rethinking IS graduates work-readiness: employers’ perspectives,” in America Conference on Information Systems, Virtual conference, 2021.
View at: Google Scholar
P. M. Léger, F. D. Davis, J. Perret, and M. Dunaway, “Psychophysiological measures of cognitive absorption,” in SIGHCI 2010 Proceedings. 9, 2010, , http://aisel.aisnet.org/sighci2010/9.
View at: Google Scholar
W. C. McGaghie and I. B. Harris, “Learning theory foundations of simulation-based mastery learning,” Simulation in Healthcare, vol. 13, no. 3S, pp. S15–S20, 2018.
View at: Publisher Site | Google Scholar
D. Vlachopoulos and A. Makri, “The use of games and simulations in higher education can improve students’ cognitive and behavioural skills,” Impact of Social Sciences Blog, 2017.
View at: Google Scholar
S. Usherwood, “Assessment strategies in simulation games,” Simulations of Decision-Making as Active Learning Tools, Springer, pp. 109–118, 2018.
View at: Publisher Site | Google Scholar
D. Ifenthaler, D. Eseryel, and X. Ge, “Assessment for game-based learning,” In Assessment in game-based learning, Springer, pp. 1–8, 2012.
View at: Google Scholar
D. M. Mertens, “Research and Evaluation in Education and Psychology: Integrating Diversity with Quantitative, Qualitative, and Mixed Methods,” Tech. Rep., Sage Publications, 2019.
View at: Google Scholar
Y. Gargouri, C. Hajjem, V. Larivière et al., “Self-selected or mandated, open access increases citation impact for higher quality research,” PLoS One, vol. 5, no. 10, p. e13636, 2010.
View at: Publisher Site | Google Scholar
C. Pappas and I. Williams, “Grey literature: its emerging importance,” Journal of Hospital Librarianship, vol. 11, no. 3, pp. 228–234, 2011.
View at: Publisher Site | Google Scholar
J. Schöpfel and D. J. Farace, “Grey literature,” Encyclopedia of library and information sciences, pp. 2029–2039, 2010.
View at: Google Scholar
A. Paez, “Gray literature: an important resource in systematic reviews,” Journal of Evidence-Based Medicine, vol. 10, no. 3, pp. 233–240, 2017.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Nadia Faisal et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

3033

Downloads

861

Citations