Abstract
This study presents a hybrid methodology for solving the serious game design evaluation in which evaluation criteria are based on meaningful learning, ARCS motivation, cognitive load, and flow theory (MACF) by rough set theory (RST) and experts’ selection. The purpose of this study tends to develop an evaluation model with RST based fuzzy Delphi-AHP-TOPSIS for MACF characteristics. Fuzzy Delphi method is utilized for selecting the evaluation criteria, Fuzzy AHP is used for analyzing the criteria structure and determining the evaluation weight of criteria, and Fuzzy TOPSIS is applied to determine the sequence of the evaluations. A real case is also used for evaluating the selection of MACF criteria design for four serious games, and both the practice and evaluation of the case could be explained. The results show that the playfulness (C24), skills (C22), attention (C11), and personalized (C35) are determined as the four most important criteria in the MACF selection process. And evaluation results of case study point out that Game 1 has the best score overall (Game 1 > Game 3 > Game 2 > Game 4). Finally, proposed evaluation framework tends to evaluate the effectiveness and the feasibility of the evaluation model and provide design criteria for relevant multimedia game design educators.
1. Introduction
This study aims to provide a system evaluation model for multimedia game design educators selecting the most suitable MACF design from a series of evaluation criteria. The selection of MACF design criteria is regarded as multicriteria problem, including the subjectivity, uncertainty, and fuzziness in the evaluation process.
Based on the above factors, six reasons are proposed for the evaluation. (a) The confirmation of criteria is determined by group experts in which they present subjective and objective considerations. (b) TOPSIS logic is rational and understandable. (c) The judgment rules close to human thinking could be acquired through fuzzy linguistic evaluation. (d) The alternatives could be compared with the objectives and criteria decision-making process. (e) The weights of importance between criteria are taken into account and included in the comparison. (f) The calculation process is simple and easy to understand [1].
Furthermore, fuzzy Delphi method is utilized for screening the criteria indicators form the literature. The entire fuzzy AHP is used for pairwise comparison of weights, as there are plenty of evaluation criteria in the literatures. It is expected to reduce the large amount of comparisons with AHP and achieve the final sequence. Fuzzy TOPSIS therefore simplifies the process of AHP and rapidly calculates the ideal solution and positive ideal solution. The alternatives are compared with ideal solution and positive ideal solution for the sequence. A project is regarded as the best one when it is close to ideal solution and far away from positive ideal solution. The optimal projects are further sequenced. The remainder of this study is structured as follows Section 2 briefly describes the proposed methods. In Section 3, proposed model for meaningful serious-game flow selection is presented and the stages of the proposed approach are explained in detail. How the proposed model is used on a real world example is explained in Section 4. In Section 5, conclusions and suggestions are discussed.
2. Related Works
2.1. Rough Set Theory
Rough set theory proposed by Pawlak [2] has been shown as a useful mathematical tool for exploring data patterns [3] and a tool for decision support systems, especially dealing with imprecise, uncertain, and vague information in the decision process. It is used for distinctive classification and recognition, and it has achieved many goals and has been used for machine learning, researching, expert systems, and decision support system and has been widely applied in diverse domains, including finance, manufacturing, medicine, and image processing. In RST [4], a dataset can be described as an information system containing decision rules in the form “if the conditional attributes apply, then the decisional attributes apply.” Chen and Cheng [5] construct hybrid models by using Rough Set (RS) classifiers to provide meaningful decision rules as knowledge-based systems from an intelligent perspective, and offers an alternative method for forecasting credit ratings and assessing the quality of RS classification systems in the global banking industry. Therefore, by using RS to execute selecting important attribute and then classify selecting attributes via intelligent technique.
2.2. Fuzzy Multiple Criteria Decision-Making
2.2.1. Fuzzy Set
Fuzzy set theory is a mathematical theory pioneered by Zadeh [6], which is designed to model the vagueness or imprecision of human cognitive processes. The key idea of fuzzy set theory is that an element has a degree of membership in a fuzzy set [7, 8]. A fuzzy set is defined by a membership function that maps elements to degrees of membership within a certain interval, which is usually the value . If the value assigned is zero, the element does not belong to the set (it has no membership). If the value assigned is one, the element belongs completely to the set (it has total membership). Finally, if the value lies within the interval, the element has a certain degree of membership (it belongs partially to the fuzzy set) [9]. Table 1 shows the structure of triangular fuzzy numbers that are used in this paper.
2.2.2. Fuzzy Delphi Method
Fuzzy Delphi Method was proposed by Garris et al. [10], and it was derived from the traditional Delphi technique and fuzzy set theory. Noorderhaben [11, 12] indicated that applying the Fuzzy Delphi Method to group decision can solve the fuzziness of common understanding of expert opinions. van Laarhoven and Pedrycz [13] proposed the FAHP, which is to show that many concepts in the real world have fuzziness. Therefore, the opinions of decision-makers are converted from previous definite values to fuzzy numbers and membership numbers in FAHP.
2.2.3. Fuzzy AHP
Satty [14] proposed the analytic hierarchy process (AHP) methodology which was a systematic method developed. It is to solve complex and multicriteria decision problems powerfully. Cheng et al. [15] improve the AHP by Fuzzy theory. Hsieh et al. [16] employed fuzzy analytic hierarchy process (FAHP) method to solve the problem of planning and design tenders selection in public office building. And FAHP method was also applied in the research of Chen et al. [17] to evaluate expatriate assignments. Thus, in this study, due to the fuzziness existed in the part of evaluation criteria, we decide to adopt the RST and FDM to form the primary evaluation criteria of MACF selection, and employ the FAHP to calculate the weight of individual criteria so as to establish the fuzzy multicriteria model of MACF selection criteria.
2.2.4. Fuzzy TOPSIS
Chen and Hwang [18] proposed TOPSIS multiple criteria method to identify solutions from a finite set of alternatives and initially proposed. Hwang and Yoon [19] define the ideal solution and negative ideal solution. The optimal solution should have the shortest distance from the positive ideal solution and the farthest from the negative ideal solution. In recent years, several researchers adopt fuzzy TOPSIS methods and applications to solve the problem and conflict [20–24]. Fuzzy TOPSIS methodology requires preliminarily information about the relative importance of the criteria. This importance is expressed by attributing a weight to each considered criterion. Chen adapted the methodology to calculate the weight of each criterion and to evaluate it by fuzzy AHP [25].
2.3. Learning Theory in Serious Game Design
Serious games have been some attempts to bring in learning effectiveness evaluation models. Garris et al. [10] presented a far-reaching input-process-output model of instructional games and learning that has implications for the design and implementation of effective instructional games. Prensky [26] proposed digital game-based learning (DGBL) which includes activities that involve learning through solving problems or overcoming challenges posed in games. Specifically, learning arises as a result of the game’s tasks; knowledge is enhanced through the game’s content, and skills are developed while playing the game [27]. The design of digital games is critical in learning. A successful digital game must involve challenge, curiosity and fantasy to increase interest and intrinsic motivation for learning [28], and Added practice and exercise in the game, which can help students to retain information more easily [29], provide immediate feedback, and activate prior knowledge by requiring players to use previously learned skills in order to advance to higher levels of the game [30].
2.3.1. ARCS Learning Motivation Model
The ARCS model is a problem solving approach to designing the motivational aspects of learning environments to stimulate and sustain students’ motivation to learn [31]. There are two major parts to the model. The first is a set of categories representing the components of motivation. The second part of the model is a systematic design process that assists in creating motivational enhancements that are appropriate for a given set of learners. To accurately measure the change in learner motivation, Karoulis and Demetriadis [32] indicated that the ARCS model can be the standard of how much the learning motivation is increased by the game [33]. The four dimensions of ARCS are the following. Attention: attention which increases the learner’s curiosity, relevance: establishment of the relevance of the learning content to learners, confidence-feedback to the learner, through the effort and the learning process of self-control, and satisfaction: the satisfaction or reward the learner can gain.
2.3.2. Flow Theory
Csikszentmihalyi [34] proposed the original definition of flow and he defined it as “the holistic experience that people feel when they act with total involvement.” Flow describes a state of complete absorption or engagement in an activity and refers to the optimal experience [35]. During the state, people are extremely involved with activity that nothing seems to matter. Csikszentmihalyi [34, 35] summarized the most commonly exhibited factors of flow into nine characteristic dimensions, including clear goals, immediate feedback, potential control, the merger of action and awareness, personal skills well suited to given challenges, concentration, loss of self-consciousness, time distortion, and autotelic experience. The concept has been broad applied in studies such as sports, work, shopping, rock climbing, dancing, games, and others [35].
It is important that the challenge that the player faces in the game matches the player’s skill. If challenge is significantly higher than player’s skill, the player will fell anxiety. In contrast, if the challenge is significantly lower than player’s skill, the player will feel bored. The three-channel model of flow explains the above situation in Figure 1. Therefore, for keeping a player in a flow state, game designers should ensure that while a player’s skill increases, the challenges also should become more difficult.
2.3.3. Meaningful Learning
Ausubel [36] proposed meaningful learning strategy with research of cognition and learning. Meaningful learning importance lies in enabling students’ acquisition of new knowledge and it is relevant to previous experiences in the personal’s information and unique understandings [37]. In recent years, several researches have employed mobile technologies to support the achievement of meaningful learning [38, 39]. Huang et al. [40] design and implement a meaningful learning-based evaluation model for ubiquitous which based on previous study are active, authentic, constructive, cooperative and personal. Several characteristics of u-learning are also linked to attributes of meaningful learning. Therefore, this study adopted these five characteristics of meaningful learning as the evaluation criteria of MACF selection.
2.3.4. Cognitive Load
Cognitive load theory [41] was utilized to measure the cognitive load items. Extraneous cognitive load is the source for the cognitive load when delivering textbook information or presenting to spend learning memory resources for learners. Cognitive load is based on the load when learners understand how to deal with materials [42, 43]. Therefore, cognitive load depends on learners having prior knowledge and learning memory resources. In serious game design, it is usually used as supplementary learning material for enhancing learning achievement. There are three criteria to intrinsic cognitive load from the structure and complexity of the materials for cognitive load. The criteria items are shown in Table 2.
3. The Proposed MACF Evaluation Model
A MACF design evaluation model (Figure 2) is proposed in this study. By integrating group experts’ wisdom with AHP and multicriteria decision-making (MCDM) in RST based Fuzzy TOPSIS, three stages are included: Criteria selected by RST and FDM identification of necessary criteria for MACF design by RST and fuzzy Delphi method; Calculating the weights of criteria by Fuzzy AHP; Evaluating and determining the ranking by Fuzzy TOPSIS.
Stage 1 (criteria selected by RST and FDM). MACF design evaluation criteria are first collected through literature, selecting attribute by RST and the hierarchy model identified by experts are screened with Fuzzy Delphi Method.
Step 1 (collect the criteria of decision group from related work and RST). RST addresses the continuing problem of vagueness, uncertainty, and incompleteness by applying equivalence classes to partition training instances using lower and upper approximations [50].
Let and be an information system. The set is approximated using information from and then by constructing the lower and upper approximation sets, which are, respectively, The lower approximation requires the classification of elements in as members of set using knowledge in . The upper approximation requires the classification of elements in as possible members of set using knowledge in which is the -boundary region of set , comprising those objects that cannot be clearly classified as members of set using knowledge in . Set is considered “rough” with respect to knowledge in when the boundary region is not empty. The quality of classification of by expresses the percentage of objects that can be correctly assigned to set using attribute , as follows: If , then the decision table is consistent; otherwise, it is inconsistent.
Reduct and core are the two key RST concepts [4, 51]. Given and , the reduct is the minimal nonredundant subset of attributes of the original dataset for IND() = IND(), and the reduct can classify the universe with the same quality. The core comprises the most relevant attributes in the original dataset and cannot be removed from an information system. Let RED() denote all reducts of . The intersection of RED() is a core of , as it is shown by (3). Pawlak described RST in detail [4, 51] as follows:
Step 2 (establish the fuzzy positive reciprocal matrix ). The core set of RST is a basic attribute of evaluation for experts. Linguistic variables in a questionnaire are first utilized for finding out the evaluation score of each alternate factor’s significance given by the experts. Triangular fuzzy numbers are then used for integrating the experts’ opinions to show the fuzziness of significance between factors. The triangular fuzzy number is established with the expert’s evaluation values acquired from the questionnaire following the equation , , , and .
The calculation equations are shown as follows.
Assuming the evaluation of the th factor’s significance given by the th expert, among experts, being
the fuzzy weight of the th factor appears, where
Step 3 (calculation of triangular fuzzy number of each criterion significance). The evaluation of each criterion’s triangular fuzzy number given by the experts is calculated for the triangular fuzzy number of the alternate factor’s significance.
Assuming the evaluation of the th initial indicator’s significance given by the th expert being , the fuzzy weight of the th initial indicator is calculated as below: where
: fuzzy weight, : evaluation of the th initial indicator’s significance given by the th expert, : number of indicator, : number of expert.
The total value of the triangular fuzzy number of indicators is organized with defuzzification, proposed by Chen and Hwang [18], to present the consensus of the Delphi members to the indicator evaluation scale.
Step 4 (defuzzification). Center-of-gravity (COG) tends to perform defuzzification the fuzzy weight of each alternate factor being the actual value . The equation is show as below.
Step 5 (screening evaluation indicators and establishing evaluation criteria). Finally, a proper factor could be screened from numerous factors by setting the threshold (determined by Scree test criterion). The screening follows the principles as below.
Situation 1. When , accept the th factor as the evaluation indicator.
Situation 2. When , remove the th factor.
Practically, the value of threshold could be determined by Scree test criterion, in which fuzzy numbers are first ordered then Linear Graph is drawn, and the Maximum turning point of linear graph scree test is the threshold .
Stage 2 (calculate the weights of criteria by FAHP). Criteria confirmed by (Fuzzy Delphi Method) FDM are evaluated the hierarchic weight with AHP. The evaluation criteria weights of MACF design formed the matrix through pairwise comparisons.
Step 1 (set up Fuzzy Paired Comparison Matrices). is the weight of the pairwise comparison matrix in each row where is the number of criteria; then
Step 2 (defuzzification). and are as follows which could be used for calculation with Center-of-gravity where
Step 3 (calculate the relative weight). The calculation of weight is show as
Step 4 (validity test of weight). Substitute to acquire the eigenvector of the weight. The left of the equation assumed that
Substitute with known and for the maximum eigenvalue
Calculate the consistency index CI with known
Calculate CR with known CI
Consistency test.
Situation 1. When (the pairwise comparison data is consistent), the relative weight outputs .
Situation 2. When (pairwise comparison data is not consistent), the pairwise comparison empirical analysis is re-proceeded.
Stage 3 (evaluating and determining the ranking by TOPSIS). The decision team would distribute the evaluation criteria weights and determine the optimal alternatives for sequence.
Our overall survey instrument was based on both past literature published surveys (ARCS, flow theory, and meaningful learning and cognitive load) and serious game-based learning. To consider the meaningful serious game flow selection practices in game design evaluation, we built on the MACF selection criteria. We gathered and developed the instruments of serious game design selection criteria from these different sources. All instruments were distributed in 15 critical constructs; all the instruments were represented in Table 2.
A MACF design evaluation with “” alternatives and “” criteria can be expressed in matrix format as given below: where , , , and are feasible alternatives, , , , and are evaluation criteria, is the performance rating given by the decision-makers to alternative against criterion , and is the weight of criterion .
An algorithm of the fuzzy decision-making method for dealing with the selection of reverse logistics provider is given as follows.
Step 1 (adopts evaluation criteria). Form Stage 1, committee of decision-makers adopts FDM results to identify the evaluation criteria.
Step 2 (adopts weight of evaluation criteria). Form Stage 2, choose the appropriate linguistic variables for the importance weight of the criteria by Fuzzy AHP.
Step 3 (evaluate the ratings of criterion). Use linguistic rating variables to evaluate the ratings of alternatives with respect.
Step 4 (normalized decision matrix). Construct the normalized decision matrix. The normalized value is calculated as,
Step 5 (construct weighted normalized decision matrix). The weighted normalized is calculated as.
Step 6 (determine evaluation solution). Determine positive ideal solution (maximum value on each criterion) and negative ideal solution (minimum value on each criterion) from the weighted normalized decision matrix. In the below equation, is the set of benefit criteria and is the set of cost criteria:
Step 7 (calculate the euclidean distance). Equations (21), (22) between positive ideal solution and negative ideal solution for each alternative:
Step 8 (calculate ranking order of all alternatives). Calculate the closeness coefficient of each alternative. According to the closeness coefficient, we can understand the assessment status of each alternative and determine the ranking order of all alternatives as follows:
4. Implementation of MACF Evaluation Model to Game Design
4.1. Evaluated Game Description
Game 1. Energy education game functions as follows.
Green city is a serious game which was designed for achieving energy education for elementary school. It takes the form of mobile game played within android pad. Each of the game levels and eleven different building locations around Taiwan is instantiated as a team in the game. Each game element has been equipped with real-world energy sensors to measure energy use and this monitoring is the main source for scoring of each team. Additionally, players play simple; fast ecoaction based minigames, take quizzes and learn about energy efficiency from sources on cloud. The actions of a team of players combine to win awards that improve and upgrade the virtual representation of their team building. Figure 3(b) shows that teams cooperate on virtual environment to build the energy city.
(a)
(b)
Game 2. Software engineering project management game functions as follows [52].
The game situation: the construction of the game, besides the design of the game screen, also includes the situation and character design. The story is set in a computer and internet service company whose clients and complicated equipment are getting more and more. This company therefore wants to develop systems that can answer questions of clients and increase the efficiency. The player must help the company evaluate and develop software, act as different roles in the developing process, and complete different tasks as different roles to complete the software development.
The interface design: the game in this study takes the story background, environment, and age of players into consideration. The game provides five different roles to be chosen. Figure 4(b) shows that every role corresponds to different situations and tasks, and the player can go through the different roles to learn all different tasks of various positions. In the requirement analysis, this study uses the maze game, which will show the problem sign and player position. When passing a problem sign, the character must stop, and the player must solve the current problem in order to keep going forward. In this task, the multiple choice questions are designed by the meeting record from the game. Besides solving all problems in the maze, Figure 4(a) shows that the player must find a way out in order to increase his interest and keep the player’s attention on game-based learning. In this task, the player must distinguish the requirements into functional and nonfunctional. The player has to take an evaluation then he will get the score which will be provided to the teacher for reference.
(a)
(b)
Game 3. 3D CCGBLS game functions as follows. The learners can understand the operation procedure through game-based simulation of clinical path. With the game-based simulation of Clinical Path, this system is expected to achieve the three objectives. The simulation of various operations allows the player to be familiar with the operation process. The operation simulation allows the player to understand the complication in the operation process. The healthcare information offered in the game could assist the player in acquiring knowledge.
The game provides four different game situation levels to be chosen. Figure 3(a) a doctor that visits patient and talks. In the game of cardiac catheterization, this study uses the first person view control of the game, which will show the situation of view Cardiac Catheterization training. The player must solve all the problem in the game order to keep going forward. In this task, the multiple choice questions are designed by meeting the record from the game. Besides solving all problems in the game, Figure 5(b) shows that the player must find a way out in order to increase his interest and keep the player’s attention on game-based learning. In this task, the player must distinguish the emergencies that will be happen occasionally in the 3D-CCGBLS. If the answer is wrong, the health points will decrease by one and the question will reappear and the countdown will be reset in order to give the player the chance to correct the mistake. The player must answer in limited time, to increase the challenge of the game. At the end of the learning phase, the player has to take a learning evaluation; then the learner’s learning data will be collected into learner’s portfolio by mobile intelligent agent and then he will get the score which will be provided to the teacher for reference.
(a)
(b)
Game 4. Blood circulation system serious game is called red adventure which functions as follows.
Aiming at the function of blood circulation in human bodies, the learners could log in to the system from the menu in Figure 6(a) and get into the unit practice through the level menu in Figure 6(b) with the main menu and the role selection to enter the unit learning of the circulation system of heart and blood, the circulation system of liver, the circulation system of spleen, and the circulation system of lung.
(a)
(b)
The game starts on right atrium, going through left lung, left atrium, spleen, liver, right atrium, right lung, and left atrium, where the circulation of blood refers to lung circulation and systemic circulation, returning to the circulation of lung. In addition to the collection of oxygen and carbon dioxide, oxygen or carbon dioxide are released at specific locations in the process, where there are attacks of bacteria and wounds for recovery. Before leaving the learning level of each organ, correctly answering the questions in the games is requested for learning feedback.
4.2. Identification of Necessary Criteria
In this stage, we focused on the analysis of evaluation criteria of MACF selection. Thus, the experts chosen were the professionals in the area related to our study with the experience of serious game design experts. Besides, they should be of rich working experience with the serious game designs and their positions where at least the rank of department managers should be over 10 years of working experience. In general, the numbers of expert were from three to fifteen [11, 12]. This study was sent out to eleven serious game design experts as questionnaire subjects.
After that we designed the questionnaire in a 9-point fuzzy semantic differential scale; see Table 1. And, we asked the selected experts to answer instrument survey. The selected experts assigned a relative importance to every collected variable with respect to three dimensions of ARCS, flow, and meaningful learning in order to confirm critical constructs as the evaluation criteria of MACF selection. The expert questionnaires were collected, and triangular fuzzy function with respect to every potential variable was established as represented in Table 2.
When selecting the evaluation criteria, it was generally considered important if the relative importance is greater than 65%. According to the above filtering treatment, we obtained from the collected experts’ questionnaires, there are 15 important criteria commonly agreed by 11 experts. And, totally 15 instrument items were eliminated. They were listed as shown in Table 2.
According to the experts’ decision, evaluation hierarchy of MACF model was built. There are four levels in the decision hierarchy structured for MACF design evaluated criteria (Figure 7). The overall goal of the MACF decision process determined is in the first level of the hierarchy. The criteria are on the second level, subcriteria are on the third level, and evaluated MACF model is on the fourth level of the hierarchy.
4.3. Calculate the Weights of Criteria
This stage was employed to calculate the fuzzy weights. The next three steps were shown as follows.
Step 1 (collection). This method was based on the experts’ precise value and the synthesized experts’ opinions with the geometric mean instead of the fuzzy numbers input directly by experts.
Step 2 (defuzzification). Since the weights of all evaluation criteria were fuzzy values, it was necessary to compute a nonfuzzy value by the process of defuzzification. Through formulas, the defuzzification weight Wi can be obtained.
Step 3 (normalization). In order to effectively compare the relative importance among evaluation criteria, we normalized the obtained weights using formula. They were listed as shown in Table 2.
In the finalization of AHP steps, results are shown in Table 3. From these obtained results, the specialization, interactivity, and the accuracy of MACF selection can be concluded. The results obtained from the computations based on the pairwise comparison matrix are provided in Table 3.
Result shows the playfulness (C24), skills (C22), attention (C11), and personalized (C35) which are determined as the four most important criteria in the MACF selection process by fuzzy AHP. Consistency ratio of the pairwise comparison matrix is calculated as 0.068 < 0.1. So, the weights are shown to be consistent and they are used in the selection process.
4.4. Evaluating and Determining the Final Rank of Alternative
In the following step, decision-makers assessed the quality of the alternative hospital web sites. The same fuzzy scale is used for evaluation as in fuzzy AHP and the decision matrix with alternatives and criteria can be seen with linguistic terms in Table 1. In the case study, there are three game alternatives. After constructing the fuzzy decision matrix and normalized matrix, TOPSIS method expression was used to calculate [25].
The last step of the methodology consists of ranking the selected game project to the ideal solution. The performance indices are computed to rank the alternatives, and the obtained results are given in Table 4. The evaluation results point out that Game 1 has the best score overall (Game 1 > Game 3 > Game 2 > Game 4).
5. Conclusion and Suggestions
Multimedia game design education criteria evaluation refers to the selection of instructional strategies, which could affect the learning effectiveness of game design education. A lot of alternatives should be taken into account evaluating the factors in different game design criteria. In this case, an efficient decision evaluation method is necessary for reinforcing the decision evaluation quality for MACF design.
A system evaluation process for MACF design is proposed in this study, which applies triangular fuzzy numbers to expressing the evaluation linguistics and considering the subjective judgment and objective analyses. A mixed fuzzy multicriteria decision-making is further applied to completing the group decision evaluation model. The evaluation mode completed the MACF design evaluation based on literature and experts’ definitions. The criteria comparisons of the case are preceded by the optimal design sequence.
The criteria weights are acquired by Fuzzy AHP, which provides the calculations of ideal solution and positive ideal solution of Fuzzy TOPSIS in the criteria decision process. Meanwhile, weighted decision evaluation is further calculated according to such weights to generate alternatives and determine the sequence.
The proposed MACF design evaluation decision model has largely enhanced the working efficiency of MACF design education. Fuzzy TOPSIS simplifies the solution of AHP calculating the evaluation process and rapidly generates the results and sequence of decision evaluations. Moreover, the calculation of indicator weights through Fuzzy TOPSIS is important for the evaluation comparison in the case. Different weights could generate the priority sequence of the evaluation results. It shows that the weights are determined through experts’ group decision to avoid prejudice, reduce bias in the decision process, and benefit the correctness of criteria evaluation [53].
Most decisions are complex and conflict that decision-makers should consider solving problems with scientific methods. The development of MACF design evaluation decision model could assist game design educators in proposing teaching strategies.
A mixed multicriteria decision-making evaluation model is combined in this study. Although the research model is compared with MACF design, the future research could revise the criteria and could be utilized for the selection from distinct design course evaluation. Besides, distinct multicriteria decision-making evaluations are used for comparing the priority sequence of decision projects. For instance, the comparison between TPOSIS, VIKOR, and ANP could have evaluation decisions be more valuable.