Abstract
We show how to optimize English diagnostic Q matrix based on cognitive diagnostic model fitting method. Firstly, attribute annotation verifies the reliability of existing Q matrix and fitting analysis, as researchers found that they still have the original Q matrix optimization space; secondly, this paper proposes a classification algorithm based on organization evolution and the information entropy of English in the diagnosis of intelligent evaluation algorithm, the running mechanism of the existing evolutionary algorithm, and the evolution of its direct effects on operation data rather than the rule. After the end of evolution, rules can be extracted from each organization to avoid meaningless rules in the process of evolution. According to the characteristics of the classification problem, we put forward three kinds of evolutionary operators and a selection mechanism, which is presented based on the information office of the evolution of the way of attribute importance. Based on this definition, the organizational fitness function, and finally the algorithm used in six test data sets and compared with the existing two classification methods, the experimental results show that the method obtained the higher forecast accuracy, and smaller rule sets are produced. Finally, a matching combination and quantitative fitting screening based on G-DINA measurement model were decomposed and analyzed, and a better fitting model was optimized based on the original Q matrix model. The results show that, first, the optimized new model is better than the original model in relative data fitting value and interpretation and diagnosis of fractional variation; second, the new model has a higher correlation with the results of self-evaluation, indicating that the probability of the new model is closer to the results of self-evaluation.
1. Introduction
Cognitive Diagnostic Assessment (CDA) is a combination of cognitive psychology and psychometrics, which can explore the potential characteristics (cognitive attributes) of knowledge, skills, strategies, and processes involved in the cognitive processing of the subjects. Diagnostic analysis and explanation of individual difference information hidden behind students' performance are provided in [1], and strong evidence support for individualized teaching and remedial learning is also provided. In recent years, CDA has attracted extensive attention from researchers and practitioners and has shown great advantages and application potential. An important content of CDA is to use Cognitive Diagnosis Model (CDM) to estimate subjects' knowledge of cognitive attributes, and the key is to select appropriate CDM, because incorrect model selection will lead to inaccurate diagnosis results [2].
CDM can be divided into compensatory and noncompensatory models according to different assumptions about the relationship between cognitive attributes. Compensation CDM refers to the probability of mastering the correct part of the cognitive attributes, rather than trying to let participants master the correct probability [3]. According to the complexity of attribute questioning relationship, CDM can be divided into contracted model (containing only single-attribute parameters but not interactive parameters of multiattribute questions) and saturated model (containing not only all single-attribute parameters but also interactive parameters of multiattribute questions) [4]. Many studies [5–7] have proved the feasibility of CDM in English testing, but the coexistence of compensatory and uncompensatory, minimalistic, and saturated CDM in English cognitive diagnostic literature and the current understanding of the relationship between English attributes are still incomplete [8]. Whether CDM is more compatible with English testing remains to be further explored.
As an important subject in the field of intelligent computing, evolutionary algorithms have been applied in many fields. However, most of the existing evolutionary algorithms adopt the top-down search strategy and thus tend to produce useless rules in the process of evolution [9]. Data mining is an interdisciplinary field of machine learning, statistics, database, and other disciplines [10]. Different from classical statistics, data mining aims to discover useful knowledge that is easy to understand by users [11]. The tasks of data mining include classification, regression, clustering, and dependency model [12]. Classification is an important data mining problem, which can be described as follows: input data or training set consists of exanp. Each record contains several attributes, which constitute a feature vector. Each record in the training set also has a specific class name corresponding to it. The purpose of classification is to analyze the input data and find an accurate description or model for each class based on the characteristics of the data in the training set. The resulting class description is used to classify future test data. Among the four abilities of English listening, speaking, reading, and writing, the research on English intelligent diagnosis is relatively least [13], and there has been controversy on whether English intelligent diagnosis comprehension ability is separable. Some studies believe that English intelligent diagnosis is a kind of integral and indivisible understanding ability [14], but more studies support the multiple separability of English intelligent diagnosis understanding ability [13], which provides a theoretical basis for cognitive diagnosis research of English intelligent diagnosis. Regular space model [14] is used to conduct cognitive diagnosis studies on English using fusion model and G-DINA model [15], but it is still not clear which model is more appropriate for English. By comparing the fitting ability of various CDMs with data, it is speculated that there is a compensation relationship between English attributes. By using similar methods, it is inferred that English attribute questions have strong compensatory relations [16], but there are also noncompensatory relations. Therefore, researchers have not yet reached a clear conclusion on the relationship between English attribute questions. In the field of psychological measurement, the Wald test is used to select the optimal contracted model for multiattribute questions without losing the model-data fitting degree [17]. Multiattribute questions here refer to questions that examine two or more attributes. In line with the principle of simplicity, the reduced model can provide more intuitive and meaningful explanations and can provide more accurate attribute master pattern classification and parameter estimation for small samples (N < 1000) [18]. Based on the topic level, the method of selecting simplified models for multiattribute questions and using G-DINA for single-attribute questions is called mixed-CDMs selection method [19]. Mixed models not only make the selection of CDM more flexible, but also have the advantages of saturated and simplified models, which is conducive to giving full play to the advantages of different CDMs [20]. However, there is little research on whether hybrid model can be used to solve the model selection and optimization of language diagnostic assessment. Based on this, this study intends to apply the hybrid model selection method in the English test and examine whether this method has applicability and advantages in the English cognitive diagnosis assessment, so as to provide reference for the model selection of the English cognitive diagnosis assessment.
The current English intelligent diagnostic assessment method generally only provides students with a macro overall score report, but does not further distinguish the microinformation such as individual's different cognitive structure or processing skills behind the same score. Cognitive Diagnostic Assessment (CDA) is based on cognitive psychology and uses psychometrics as a tool to estimate latent variables such as knowledge, skills, and cognitive processing involved in the learning process. In order to achieve the diagnosis analysis and explanation of the individual difference information hidden behind the students' scores, it is beneficial to provide students with detailed diagnosis information. To achieve cognitive diagnosis, building quality Q matrix is the key to generate a detailed diagnostic report. This paper discussed in detail the organizational evolution and the information entropy of English in the diagnosis of intelligent evaluation algorithm, namely, the definition of organization, based on the information entropy of fitness function structure and organization evolution operator and group selection mechanism, specific classification algorithm, rules extraction algorithm, and the pruning strategy. Finally, classification rules are used to classify the new samples.
2. English Diagnostic Intelligence Evaluation Algorithm Based on Organizational Evolution and Information Entropy
2.1. Tissue Evolution Fitness Function
The English intelligent diagnostic evaluation algorithm based on organizational evolution and information entropy uses a completely different search strategy: the task of classification data mining is to generate classification rules covering a certain number of similar samples. Based on this, first of all the samples with the same attribute value cluster to form groups, and then according to the different values of attribute importance they will directly impact on organization evolution operator; at the end of the evolutionary process, classification rules will be extracted from the organization, thus avoiding the useless rules in the process of evolution. Therefore, it takes a bottom-up search strategy. It uses data to drive evolution; it is data-driven.
The following factors must be considered when designing the organizational fitness function.
The number of members (objects) in an organization: the more objects the organization contains, the more reliable the extracted rules are. Therefore, the fitness value of the tissue is proportional to the number of objects in the tissue.
The number of useful attributes: because rules are generated through the set of useful attributes, the larger the number of rules is, the more conditions the rules contain, and the more conditions the rules have, the fewer objects the rules cover. However, if the conditions are too few, this will cause the overfitting of rules.
Considering the influence of the second factor, parameters are introduced to describe the importance of attribute a in classification. For convenience of discussion, Wa is defined as follows:
Attribute importance is defined as
The evaluation model of English diagnostic intelligence based on organizational evolutionary information entropy is established as shown in Figure 1. The model combines the hierarchical structure with the alliance structure, which can represent both the hierarchical nature of information transfer and the group nature among agents.
The organizational fitness function related to classification rules is defined as
2.2. Organizational Evolutionary Selection Algorithm
An individual is an organization, and the object of evolutionary operation is also an organization. The traditional genetic mechanism and genetic operators (such as crossover, mutation, and selection) cannot be applied directly, and the evolutionary mechanism and corresponding evolutionary operators suitable for the organization must be adopted. Therefore, three evolutionary operators and a selection mechanism are designed to act on the tissue.
Migration operator: randomly select 2 organizations from a population as the parent, and then randomly select N objects to be added to the organization to form 2 child organizations.
Exchange operator: randomly select 2 organizations from a population as the parent, and then randomly select N objects to be added to the organization, and then randomly select N objects to be added to the second organization, forming 2 child organizations.
Merging operator: randomly select 2 organizations O from a population as the parent and merge them into a child organization.
The purpose of organization selection algorithm is to encourage offspring organization to compete with parent organization. Through the evolution operator to produce a pair of progeny tissues consistent with the operator to produce only one progeny tissue, the progeny tissue and its parent compete. The two tissues with the highest fitness were selected as the parent tissues into the next generation, and the remaining tissues were removed and deleted. Selection algorithms must prevent abnormal organizations from entering the next generation, because evolution operators may produce abnormal organizations, and rules extracted from abnormal organizations have no effect. When the abnormal organization enters the next generation, the abnormal organization is discarded and its member object is added to the next generation as a common organization. If the offspring organization and the parent organization have the same number of objects, the number of objects in the population remains unchanged. If there is only one organization in the group, it goes straight to the next generation.
The English diagnostic intelligence evaluation algorithm of organizational evolutionary information entropy is shown in the following algorithm:
|
In order to further reduce the number of rules, the following metrics are adopted: calculate the relative support of each rule (denoted as RS); RS is the ratio of the number of positive samples covered by a rule to the total number of samples. All rules are sorted based on relative support. To avoid the classification rules for classes with small samples being put behind, the rules are sorted by proportion rather than the absolute number of positive samples. After sorting all rules, if the number of samples covered by a rule is a subset of the union of the number of samples covered by an existing rule, the rule is deleted.
After all the rules are gathered together, there may be a situation where the uniform sample is covered by different rules. In order to ensure the classification accuracy, it is necessary to eliminate the conflicts between different rules. Regular pruning is a common solution in data mining. The main purpose of rule pruning is to eliminate the irrelevance of condition terms so as to avoid the overfitting of rules to training samples. Rule pruning can also reduce the complexity of rules, because short rules are usually easier to understand than long rules. In this paper, rules are pruned based on fitness values.
In order to classify the new test samples that have not appeared in the training process, relevant rules are selected in the sorted rule list according to the discovered rules to act on the new samples. The rule that covers the samples first is the classification rule, and the class name of the sample is the class represented by the afterword of the selected rule. If the sample is not covered by any rule in the rule table, the sample is divided into the decision class of the default rule, which is the rule that can simply predict the majority of the classes in the training sample set that are not covered by any rule.
2.3. English Diagnostic Agent Inference Based on Information Entropy
English diagnostic intelligent reasoning knowledge base mainly includes three parts: knowledge acquisition, knowledge representation, and rule base establishment. The main task of knowledge acquisition is to obtain the above knowledge through interfaces and to establish and improve the effective knowledge base to meet the needs of the task. Knowledge mainly includes relevant professional and technical literature, expert experience, and case summary.
Knowledge representation consists of static attribute, rule item, and relation item. The rule item of each piece of knowledge in the knowledge base consists of four parts of information: rule number, node type, subrule set, and the call after the rule is accessed. Each nonleaf node in the library corresponds to a rule of the knowledge base. The result of invoking the previous rule is the basis on which the knowledge base is used to determine the selection of subrules. The root node is the entrance of the whole knowledge base, as shown in Table 1.
Input instance data to establish knowledge base, as shown in Figure 2.
As shown in Figure 2, input sample data and extract data features, mainly extracting corresponding rule features, to determine whether rules already exist in the knowledge base. If not, input rules directly into the knowledge base and then determine whether to terminate instance data. If not, continue to input instance data. If rules already exist in the knowledge base, then determine whether knowledge base has its own knowledge; for their own knowledge, choose not to refresh the existing knowledge; if not their own knowledge, the size of the comparison is needed, mainly to determine whether a value is the same; if the same, do not refresh the knowledge base of the corresponding knowledge; if not the same, then refresh the original value; after averaging entry knowledge base, record the times of input, and finally judge whether to end the instance data; if not, continue to input the instance data.
3. Intelligent Evaluation Model of English Diagnosis
Based on the characteristics of English subjective questions, combined with natural language processing technology, dynamic programming algorithm, regular expression, etc., we integrated the use of organization evolution of information entropy, the shortest edit distance algorithm, natural language processing technology of regular matching algorithm, and cosine vector space vector algorithm on students' spelling, word collocation, and lexical and grammatical structure to grasp the essentials of basic English language knowledge ability to conduct a comprehensive diagnosis, so the basic diagnostic procedure can be defined as follows.
Build a corpus: a corpus of syllabus grammar and vocabulary will be established, as well as a rule base of composition core sentences and key words.
Initialize the rule engine; that is, unify all the rules of each key sentence and its similar sentences in each composition.
Sentences with large differences are filtered through text similarity.
The filtered sentences are regularly matched with the corresponding rule base, the minimum editing distance is obtained, whether the wrong words can be deformed is judged, and a set of rules with the maximum total score is obtained.
Record the knowledge points of the maximum rule pairs of total scores, and repeat the next sentence until all the rules are used up.
According to the above ideas, the basic flow chart can be obtained as shown in Figure 3.
If the current knowledge point has no parent knowledge point, although the knowledge point reflects the one-dimensional data model in the database, it can finally reflect the multidimensional tree structure through the association of the parent knowledge point. Table 2 is an example of the subjunctive mood.
For sentences, the difference between the response sentence string and the matching string in corpus is the editing distance. The similarity D of two sentences can be preliminarily obtained by calculating the minimum editing distance R of two sentences. For words that constitute sentences, first of all, it is necessary to determine whether they are deformed forms in the lexical corpus. If they do not exist, the minimum editing distance P is calculated, and the threshold is set according to the word length to define whether they are misspelled. If the spelling is wrong, it will be recorded in the wrong word bank, so that students can find their own shortcomings in time.
The stronger the ability of a word to convey its main idea, the greater the weight , and vice versa (the lower the weight). The weight of words to be deleted should be zero.
The specific calculation method is as follows: n is the number of words appearing in the current sentence, M is the total number of words appearing in all other sentences except this sentence, and m is the total number of text sentences; then
3.1. Experimental Design
DDCAOEE performance was tested against six public data sets provided by Irvine LABS, the main characteristics of which are described in Table 3. The first column is the name of the data set, and the other columns are sample number, discrete attribute number, continuous attribute number, and category number, respectively. As mentioned above, the discovery of classification rules is based on the characteristics of discrete attributes. If continuous attributes exist in the data set, they need to be discretized in advance.
The ordered rule table is also generated after the evolution process, that is, the two algorithms and our algorithm have the same representation of rules, and the difference in performance is mainly caused by different search strategies. Therefore, AntM iZer and CN2 were selected for comparison experiments to test the performance of DDCAOEE through comparison experiments on the selected six data sets. All the experiments were designed using MATLAB design language, and the operating environment was Lenovo Ben 4 PC with CPU frequency of 18 GH and memory of 512 MB.
4. Results and Analysis
The algorithm performance is compared by two criteria: prediction accuracy and simplicity of classification rules. The prediction accuracy of the three methods is shown in Table 4. It can be seen from Table 4 that the prediction accuracy of classification rules discovered by DDCAOEE is higher than that of the other two algorithms in all data sets. Meanwhile, the prediction accuracy of DDCAOEE is the highest among the three algorithms. In this way, DDCAOEE has better performance than AntMine.
Sample size is an important factor affecting the diagnostic accuracy of CDM model. G-DINA is a saturated model and requires a higher sample size. Therefore, Q-H3 verified by a large sample (N = 1000) should be more reliable than H3 (N = 534). In addition, the study shows that, under the saturation model (G-DINA), when the sample size is 1000, the minimum value in the alternative Q matrix has more than 78% probability of screening the overlabeled Q matrix, and the minimum BIC value is the optimal matrix selection. Using the data of this study (N = 1000) and Q-H3 and H3, it is found that the BIC of Q-H3 (29003.85) is less than the BIC of H3 (29228.21), and H3 is less than the -2LL of Q-H3 matrix (27670.65). It indicates that Q-H3 obtained in this study is superior to H3, and H3 may be an overlabeling matrix. Figure 4 shows the comparison of the quantity distribution of each attribute corresponding to the optimized Q-H3 and H3. Except for A1 and A3, the quantity distribution of each attribute in H3 is higher than that in Q-H3.
The English diagnostic intelligence assessment results of 675 subjects on their attribute mastery were compared with the attribute mastery probability obtained by Q-H3 and H3, respectively. For a clear demonstration, the English diagnostic intelligence assessment data (level 1–6 measures) were converted into the 0–1 range in Figure 4. It can be seen from Figure 5 that the probability distribution of attribute mastery estimated by 675 samples based on Q-H3 is consistent with the 1000 population (the broken lines almost coincide with each other) and is very similar to the result of English diagnostic intelligence assessment: A5 is located at the vertex of the broken line and is the attribute with the best mastery; A3 is better mastered than A2; A1 mastered it better than A2. However, the probability of owning H3 attributes is far different from the result of self-assessment, in which A3 is the highest probability of owning attributes, and A5 is only higher than A2. In comparison, the attribute estimation results of Q-H3 were closer to the test of English diagnostic intelligence.
The experiment compares the differences of LLE (Locally Linear Embedding) algorithm in the use of various distance measures and compares it with information entropy measures. The experimental results are shown in Figure 6. As can be seen from Figure 6, when Manhattan distance and Chebyshev distance measures are used, the clustering and classification effects of data are poor, and the data are scattered and chaotic. When Euclidean distance is used, although the clustering is improved, the classification is not very good, and there are many overlaps. Using information entropy as a measure, the classification and clustering are better than other algorithms, which is suitable for feature extraction, proving the effectiveness of the proposed algorithm.
(a)
(b)
The situation of mastery has changed significantly accordingly. The scatter diagram of mastery probability is shown in Figure 7. 0.2 is taken as the critical value of attribute mastery (mastery probability >0.2 indicates that the attribute has been mastered). It can be seen that, in mixed-CDMs, the mastery of English diagnostic intelligence assessment has a more obvious positive correlation with score and better differentiation, and the subjects with an overall score below 1 can hardly master this attribute. Only the subjects with more than 0.5 points were sure of the English diagnostic intelligence assessment, so the score range of the subjects near the critical value was 0.5–1.5 points. However, in the G-DINA model, the mastery of English diagnostic intelligence assessment is scattered, and the score span near the critical value is large (0.5–1.5 points), indicating that the degree of differentiation is slightly poor.
(a)
(b)
5. Conclusion
English diagnosis intelligent evaluation model based on information entropy to further research multiagent interaction and collaboration between individuals in the multiagent system is through mutual cooperation and mutual sharing, does not consider agent with multiple agent group subordinate relations between English diagnosis intelligent evaluations, and is given based on the information entropy of multiagent reasoning algorithm. On this basis, the membership relationship between agent and group is further considered, and the evaluation of English diagnostic intelligence of multiple agents is improved. The existing genetic algorithm to solve the multiagent situation reasoning problem based on information state still has the problem of long running time. For the subsequent large-scale multiagent massive data, we plan to continue to study the real time and stability of solving using genetic algorithm on the basis of the existing work.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by School of Foreign Languages, Xinyang College.