Mobile Information Systems

Volume 2019, Article ID 4260196, 11 pages

https://doi.org/10.1155/2019/4260196

## Multicriteria Model of Students’ Knowledge Diagnostics Based on the Doubt Measuring Level Method for E&M Learning

^{1}International Information Technology University, Almaty, Kazakhstan^{2}Al-Farabi Kazakh National University, Almaty, Kazakhstan

Correspondence should be addressed to Y. S. Maulenov; moc.liamg@smnahzle

Received 20 December 2018; Revised 4 April 2019; Accepted 23 April 2019; Published 2 June 2019

Academic Editor: Filippo Sciarrone

Copyright © 2019 V. V. Serbin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This research is devoted to the development of a multicriteria model, valid only for testing, which takes into account the number of responses, the level of question complexity depending on the time spent on their response, time of the whole testing, doubts, number of passes, and the use of additional programs. Also the developed method of measuring the level of the user doubts is described, which gives a clearer and “transparent situation picture” for a more objective decision-making. The method makes it possible to reduce the probability of guessing the correct answer, which increases the objectivity of assessing the knowledge level in the diagnostic systems for E&M learning.

#### 1. Introduction

Distance education and e-learning have laid the foundation for a new global phenomenon, Smart Education, which is not so much about technology as it is about the new philosophy of education.

Developers of distance learning systems cannot always fully take into account all the laws of the learning process and therefore use not quite correct models. And even the most successful solutions in the field of online training rarely get widespread.

Undoubtedly, distance learning is by no means a complete substitute for traditional training, because it is difficult to replace live communication with a teacher or the educational atmosphere that develops in the classroom between subjects of the learning process. At the same time, modern ICT (information and communication technologies) can minimize the “narrow” places of distance learning. Modern pedagogy is also changing, and nowadays transfer of part of the training load to the online mode is quite an acknowledged educational scenario. This can be done when developing educational content on a modern IT basis, for example, using MOOCs (a mass open educational course).

Today, we can confidently state the existence of a new digital Z-generation of people for whom a mobile phone, a computer, and the Internet are natural elements of their living space. In modern conditions, effective education is education without reference to time and place. This education teaches through everyday life. The transition to this mobile learning technology involves the application of new methods, approaches, and principles of the learning process organization. When developing content for mobile devices, it is necessary to take into account that it is intended for young people of the Z-generation, as the method of preparing training content for them differs from the traditional learning content.

So, smart technologies in education, such as mobile applications, are of great importance as they allow to optimize the costs of the university logistics and also to raise the quality of educational services and products to a new level. It is smart technology that allows to develop revolutionary teaching materials, as well as to form individual trajectories of training.

In 2015, UNESCO published Recommendations on Mobile Learning Policies [1], which fully justifies the need for the introduction of mobile technologies in the educational process. According to these recommendations, “In a world in which dependence on means of communication and access to information is growing, mobile devices will not be a transient phenomenon. As the capacity and capabilities of mobile devices are constantly growing, they can be used more widely as educational tools and take a central place, both in formal and informal education.”

History shows that the competitiveness of the national economy as a whole depends on the development of information technologies. According to the World Economic Forum, the competitiveness index of the economies of states has a high level of correlation with the countries’ information and communication technologies. According to the 2012 rating of the competitiveness of 142 countries of the world by the World Economic Forum, countries that are actively developing information technologies are ahead of Kazakhstan which ranks 51^{st} in terms of creating demand for information technology (the USA-13, Germany-19, India-63, and Egypt-96) and occupies the 104^{th} place in the information technology business conditions place (USA-21, Germany-38, and India-72).

Scientific novelty consists in developing a method of objective measurement of students’ knowledge on the basis of a multicriteria decision-making model.

#### 2. Development of a Multicriteria Model for Diagnosing Knowledge

To solve the problem, an intellectual multicriteria model for assessing students’ knowledge has been developed, which makes it possible to take into account the following parameters: (1) time spent on the answer, (2) the number of correct answers, (3) time of all testing, (4) the doubts of the test person, (5) omission of questions, (6) use of additional programs, and (7) psychological characteristics of the tested person.

Given in the following sections are the coefficients of the model and the methods for measuring the coefficients.

##### 2.1. Coefficient of Accounting for Time Spent on the Answer

Tests contain questions of different levels of complexity (50% of them are easy (level *a*), 30% are average (level *b*), and 20% are difficult (level *c*)). Therefore, the score for each correct answer should depend on the complexity of the question, as well as on the time spent on it.

The maximum score *G*_{max} (subject to a quick response in less than 15 s) can be calculated from the formulawhere is the number of questions on levels of complexity , is the maximum score for answering easy questions (level *a*), is the maximum score for answering average questions (level *b*), and is the maximum score for answering difficult questions (level *c*).where is the number of questions in the test.

The maximum score can be typed with a different number of questions in the test.

Points scored for the elapsed time can be calculated by the formulawhere refers to the points for the correct *i*-question level *а*, refers to the points for the correct *i*-question level *b*, and refers to the points for the correct *i*-question level *c.*

The coefficient characterizing the share of knowledge at score “5” under the condition of a quick response (for a time not exceeding 15 seconds) is determined by the formulawhere is the number of points scored for the elapsed time, subject to the choice of the correct answer.

Provided that all the answers are correct and the time for making a decision of each answer will not exceed 15 seconds, the knowledge coefficient of the tested “excellent” is 1.

##### 2.2. Coefficient of the Number of Correct Answers

where is the coefficient that characterizes knowledge of all issues regardless of time, is the number of correct answers, and is the total number of questions.

##### 2.3. Coefficient for Time of All Testing

To take account of the time spent on the test (regardless of anything), we need to use the following formula:where is the coefficient characterizing the passage of the test, is the time spent on testing, and is the maximum allowable time that can be spent on testing.

##### 2.4. Coefficient of Doubt

The mathematical description of doubt as a significant component of the rational choice model, based on the axioms of independence, transitivity, convexity, and monotony of individual preferences, is represented in some way in the study, as discussed by Vinogradov and Kuznetsov [2]. A person builds a sequence of conclusions about the adequacy of his ideas in accordance with his subjective level of conviction. He is convinced of the adequacy of his views in the situation of choosing the type of Ω with respect to goal G, if he believes that the choice of the mode of action C on their basis will allow it to be achieved. At the same time, he perceives some of the characteristics of the X situation Ω; with respect to the other part, he makes assumptions and shows an intention to prove (to verify their plausibility).

The assumption is the default value of the observed characteristic or a description of the cause-effect relationship between the observed characteristics.

Representations of a person are characterized by a level of conviction. The level of his belief in his ideas about the situation of choosing type Ω with respect to goal G is determined by the frequency of its achievement when choosing the mode of action C based on them. The assessment of the level of conviction changes from zero to one. If the number of unsuccessful attempts to reach goal G when choosing the mode of action C based on human representations increases, then the level of conviction of a person decreases (and vice versa), which becomes an incentive for him to apply efforts for their modification or complete reconstruction due to the growing doubt in the plausibility of the assumptions made. The desire to verify the correctness of assumptions is a measure of a person’s doubts.

Thus, the efforts that a person expends to prove (refute) assumptions characterize the degree of his doubts about the ideas about the situation of choosing type Ω as he strives for goal G. According to the provisions of the theory of behavioral psychology, if the level of conviction that depends on the number of confirmations of the correctness of the choice based on representations increases, then the person’s desire for verification falls, since he does not see the point in this. The increase in the degree of doubt is an incentive for finding additional arguments (counterarguments).

A parameter that takes into account these two characteristics is the degree of conviction considered aswhere is the degree of conviction; is the level of conviction (past experience); is the degree of a person’s doubt about the correctness of his ideas about the situation of choice; and are the coefficients of significance that a person gives to his experience and the need to find evidence. Thus, from the preceding expression, one can derive a formula expressing the degree of doubt.

The degree of doubt in modern tests, in our opinion, is a latent parameter and can be measured only indirectly. If the degree of doubt and the degree of confidence *U* is estimated on a 100-point scale and expressed in %, then the formula for the degree of doubt will acquire the following form:

Quantitative characteristics are needed to measure latent parameters. The quantitative parameters for measuring the degree of doubt of the user are the following variables: the number of missed operations, the amount of unconfirmed information, the amount of heterogeneous information (ambiguous from the first time), and the state of the logical chain (the sequence of actions, levels of complexity, etc.).

To determine the level of doubt, we used a superposition of *n* models, each of which determines the latent parameter of doubt on the quantitative parameters.

Thus, we can postulate the following:(1)At the initial level of complexity, the measured level should be the lowest level of doubt.(2)On the average level of complexity, the level of doubt should be greater for the advanced level, but less for the initial level.

The quantitative parameters for measuring the level of doubt of the user are the following:(i)The number of missed operations(ii)The amount of unconfirmed information(iii)The amount of heterogeneous information (ambiguous from the first time)(iv)The state of the logical chain (the sequence of actions, levels of complexity, etc.)

To take into account the user’s level of doubt, the following method is suggested:(1)A test is organized, containing *i*-questions, under the condition *i* > 0.(2)Each *i*-question contains *j* variants of answers under condition 2 < *j* ≤ 5.(3)The testing questions are divided into training elements. Each training element contains questions of *S* different levels of complexity. The question contains only one correct variant *G*_{i} for the *i*-question, *G*_{i} > 0, *G*_{i} ≥ *n*_{ij}*.* Each *j* variant can be preselected and subsequently confirmed with the only accepted variant of the answer *k*_{ij} under the condition *j* = 1. Each variant of the question has the ability to switch *m* times. Each question contains a ability to skip an *i*-question only once, that is, . Then, the amount of heterogeneous information (ambiguous from the first time) can be determined by the number of switching options .

Consequently,where is the coefficient per doubt of *i*-question, is the number of variants of answers in the question, is the coefficient that characterizes the weight of switching from the correct to the wrong answer, is the coefficient that characterizes the weight of switching from wrong to correct answer option, is the coefficient that characterizes the weight of switching from wrong to wrong answer option, is the number of switches from right to wrong answer, is the number of switching from wrong to the correct answer, and is the number of switching from wrong to wrong answer option.

Let us find the average coefficients of doubt to questions of level , respectively,where is the number of difficulty levels, is the number of level *a* questions, is the number of level *b* questions, and is the number of level questions.

The final coefficient, which characterizes the average level of doubt of the learning element, is calculated by the formulawhere is the number of difficulty levels.

When S = 3 (3 levels: initial, intermediate, and advanced) of each educational element and states 2 (1, answered; 0, did not answer), there are 2^{3} = 8 various states of the logical chain, each of which has its own level of doubts in the matrix states of the level of complexity logical chain. The level of doubt in the matrix depends on the subject area, compiled by the expert.

The weight of the doubt level at *s* = 3 for this model corresponds to *A*-50%, *B*-30%, and *C*-20%. The user is given a portion of information of *A*, *B*, and *C* levels of this training element. After that at Table 1, the truth table is analyzed, and the decision is made.