Abstract
One advantage of an adaptive learning system is the ability to personalize learning to the needs of individual users. Realizing this personalization requires first a precise diagnosis of individual users’ relevant attributes and characteristics and the provision of adaptability-enabling resources and pathways for feedback. In this paper, a preconcept system is constructed to diagnose users' cognitive status of specific learning content, including learning progress, specific preconcept viewpoint, preconcept source, and learning disability. The “Force and Movement” topic from junior high school physics is used as a case study to describe the method for constructing a preconception system. Based on the preconception system, a method and application process for diagnosing user cognition is introduced. This diagnosis method is used in three ways: firstly, as a diagnostic dimension for an adaptive learning system, improving the ability of highly-adaptive learning systems to support learning activities, such as through visualization of the cognition states of students; secondly, for an attribution analysis of preconceptions to provide a basis for adaptive learning organizations; and finally, for predicting the obstacles users may face in the learning process, in order to provide a basis for adaptive learning pathways.
1. Introducing the Problem
Diagnosing user attributes and providing appropriate feedback based on these results is fundamental to realizing personalized learning support functions in data-driven learning systems. Traditional cognitive diagnostic methods utilize discrete representation methods, such as “mastered/not mastered” to express a learner’s cognitive status in relation to pieces of knowledge, and then use this data as a foundation for estimating the user’s skill or ability level. This approach to diagnosis inevitably weakens diagnostic precision; the learning outcomes from a data-driven environment will decline correspondingly.
A few issues inhibit the effectiveness of the traditional approach. One is that this type of measurement does not accurately reflect the user’s cognitive patterns. The learning process cannot be reduced simply to “yes or no.” This discrete descriptive method ignores the actual cognitive processes of the user as well as the problems the user needs to resolve in the learning process. From the perspective of cognitive development patterns, the process of user gaining new knowledge is in fact a process of conceptual transformation. Reductive quantifiable diagnostic results do not capture why a user has not yet grasped an aspect of knowledge, meaning that the system cannot provide precise adaptive feedback. Second, in terms of methodology, this approach represents a “qualitative-quantitative-qualitative” approach. As such, information loss during transmission is inevitable, reducing the accuracy of diagnosis. Reduced accuracy of the diagnosis system means that the quality of feedback provided, including visualization of the user’s knowledge and skills, will be reduced as the effectiveness of adaptive support mechanisms built on diagnostic results is reduced. Third, in terms of practical pedagogy, conducting transformative instruction based on a single preconceptual perspective leads to poor learning results. Since users have their own naïve theory framework [1–8] to explain everyday life, naïve concepts are not separated from one another in their minds. If transformation is applied to only one conceptual point in the framework, cognitive dissonance will emerge between the conceptual point and the overall framework. When users experience cognitive dissonance, they tend to adjust the concept to be consistent with the overall naïve theory framework, rendering prior conceptual transformation useless. Another possibility is that users will classify concepts according to different application scenarios, so that the same knowledge points can sometimes only be used to correctly solve related problems. Effective pedagogy needs not only the conceptual transformation of individual preconceptions, but also the conceptual transformation of preconceptions related to the relevant naïve theoretical framework, so as to attribute them from the perspective of specific concepts.
Based on the three issues described above, the authors of this paper analyze users’ true cognitive processes based on preconception research, designing a cognitive diagnosis method for data-driven learning environments. The method described enhances the precision of system diagnosis to meet the needs of users in learning and teaching activities. The topic of “Force and Movement” from junior high school physics is used as a topic to explain this diagnostic method.
2. Preconception Definition Differentiation and Analysis
The concept of “preconception” can be traced back to Piaget (The Child’s Conception of the World, 1929) and Ausubel (Meaningful Learning Theory, 1960s). These scholars suggested that, prior to formal learning, users produce their own concepts, perspectives, and ideas regarding occurrences in life. These prior knowledge structures play an important role in later learning. From the 1970s, research on the conceptual transformation from preconception to scientific conception started to emerge [9–13]. Based on four different perspectives, researchers emphasized the following: this conceptual transformation begins prior to formal learning; it is different from scientific concepts; and it has its origins in everyday life. Research was primarily conducted with children of ages 5–17. Details are shown in Table 1.
This paper focuses on diagnosing the cognitive states of users. The authors believe that using the term “preconception” to describe the characteristics of users is most suitable. For the purposes of this paper, the term is characterized as follows:(1)Preconceptions Occur in Daily Life and Typically Deviate from Scientific Concepts. For a user, engaging in learning involves a process of transforming preconceptions into scientific conceptions. In other words, learning involves transforming daily experience into standard scientific knowledge and abstract symbols.(2)Differences between Eastern and Western Definitions of “Preconception” May Lead to Deviations in Understanding. Regarding the English term “preconception,” the root of the word “conception” has an objective meaning, but is also associated with subjective scholarly perspectives and ideas.(3)The Difficulty of Transforming Preconceptions into Scientific Conceptions Is Higher Due to the Cognitive Patterns of Users. Previously, the author investigated 185 students from two schools in J province. The topics considered include thermodynamics, electricity, and force. It is found that prejudice constitutes an obvious obstacle to learning in the process of physics learning. The author also finds that bias still exists even after classroom teaching. For example, question 3 in the survey requires students to compare the brightness of two small bulbs in a closed series circuit. Students show two classic biased views in their answers. One bias is that students believe that a current, similar to water flow, will lose strength when reaching an obstacle, resulting in a brighter bulb closer to the positive pole. Another view is that the second bulb will collect the power supply and the current from the first bulb, which means that the second bulb will receive more current than the first bulb, so it will be brighter. See Table 2.(4)The “Pre” in Preconception Does Not Refer to the Time Dimension, but Rather the User’s Current Conceptual Processing and the State of the User prior to Learning Scientific Concepts. Some preconceptions appear in the learning process, such as when instructors explain “concepts in electrical currents” using water currents as an analogy to aid student understanding. In such a scenario, preconceptions will emerge when students consider problems related to electrical currents. At the same time, acceptance of cognitive approaches similar to using “water currents analogous to electrical currents” to learn new scientific concepts is also a preconception.
Only by constructing a preconception system and establishing the connections between preconceptions can preconceptions be properly attributed and predicted, enabling personalized learning. Below, the authors will describe the methodology used and a case study for the construction of a preconception system using the topic of “Force and Movement.”
3. Process and Methodology for Constructing a Preconception System
Constructing a system requires analyzing the components of the system and then exploring the connections between those components. The specific preconception perspectives of users are the components of preconception systems, while the connections between those preconception perspectives comprise the structure of preconception systems.
3.1. Methods for Extracting Preconception System Components
Preconceptions emerge from the experiences and perceptual structures of users, representing their understanding, perspectives, and explanations regarding scientific knowledge. Ultimately, preconceptions represent the mapping of objective knowledge by subjective individuals. Extracting the preconception perspectives of users requires diagnosing how users understand scientific knowledge and explain scientific phenomena when learning specific content. Common methodologies include interviews, the think-aloud method, and two-stage diagnostic surveys. As this study required the calculation and analysis of a large sample set, a two-stage objective diagnostic survey was used.
The two-stage diagnostic questionnaire is an effective method for diagnosing the preconceptions of respondents. Unlike regular questionnaires, two-stage questionnaires include two parts in each question. The first part asks respondents to make a judgment on the question; the second part asks them to explain the reasoning behind their judgment. Using a two-stage diagnostic questionnaire enables researchers to obtain the users’ perspectives on explanations of the questions. Developing a two-stage objective diagnostic questionnaire enables the diagnosis of large numbers of users, while simplifying data compilation and analysis [14–19].
3.2. Method for Examining Preconception System Structure
Understanding scientific content is a concept-level reason for the emergence of preconception viewpoints. Explaining scientific phenomena involves the application of concepts in understanding phenomena; in other words, it is the “dependent” variable. As such, preconception systems are necessarily hierarchical structures.
As a causal relationship exists between the comprehension layer of the concepts and cognitive phenomena, in situations with sufficient sample sizes, the significant correlations between preconception errors in the understanding of scientific concepts and other preconceptions will be more numerous than those among other preconceptions. Accordingly, a hierarchical structure for preconceptions can be constructed. Bayesian probability can be used to derive correlations between various preconceptions and to map internal relationships within the preconception system.
3.3. Process of Constructing a Preconception System
Preconception systems are an important component of the user cognition diagnosis method introduced in this paper. Users’ preconception systems must be constructed based on specific learning content based on the following stages: steps and methods. The construction process can be divided into three stages: preconception perspective extraction; preconception hierarchy division; and preconception correlation calculation. The process, stages, and methods for constructing a preconception system are described below using the “force and movement” unit from junior high school as a case study.
3.3.1. Preconception Perspective Extraction
Extracting the preconception viewpoints of students on specific learning content is the most fundamental and important stage for constructing a preconception system. The objective is to extract users’ preconceptions of specific knowledge points, in other words specific preconception viewpoints, using a two-stage diagnostic questionnaire. This stage consists of four steps: analysis of knowledge points for the tested content; design and distribution of a two-stage subjective diagnostic questionnaire; analysis and summary of the two-stage subjective diagnostic questionnaire; and analysis of specific preconception viewpoints.
(1) Analyzing Knowledge Points for the Tested Content. Using People’s Education Publishing curriculum, the authors selected knowledge points from the “Force and Movement” unit for analysis [18]. Concepts in “Force and Movement” include movement [20], force, Newton’s First Law of Motion, gravity, and friction. Knowledge points were further divided into specific content and coded as Movement–M, Force–F, Newton’s First Law of Motion–N, Gravity–G, Friction–C. Specific knowledge points were arranged in order, e.g., F1—definitions of the three elements of force and F2—definitions of interactions among forces.
(2) Two-Stage Subjective Diagnostic Questionnaire(1)Questionnaire Design. Preconceived information about the subject content was gathered through a review of relevant literature and interviews with faculty and students. Then this information was used to design subjective questions. Then the questions are compared with the knowledge points of the test to ensure that the questions designed cover all the knowledge points of the test content. At this stage, all questions in the questionnaire are subjective and open-ended.(2)Questionnaire Testing. The questionnaire was then distributed for testing. Teachers and students provided feedback for editing. The most common issues were that question phrasing was unclear, that questions and answers lacked tight logic, and that diagrams were incomprehensible.(3)Formal Survey Testing. The updated edition of the survey was distributed and retrieved in person.
(3) Analysis and Summary of the Two-Stage Subjective Diagnostic Survey(1)Questionnaire Analysis. Qualitative and quantitative analysis were conducted for the retrieved surveys. The main answers for each question were summarized, and students were interviewed to identify the reasoning behind their answers. The results of summary and analysis are shown in Table 3 below, ensuring that all possible responses are included.(2)Questionnaire Summary.(a)The results of some questions were not comprehensive enough; additional questions need to be added to those knowledge points to obtain more detailed conclusions(b)The answers of respondents to these two questions will form the backbone of objective questionnaire answers in the next step(c)New preconception viewpoints were obtained through the summary, such as respondents believing that a taller object is subject to greater gravity(d)Design and distribution of the two-stage objective diagnostic questionnaire(3)Questionnaire Design The analysis results of subjective questionnaires are used as the basis for the answer choices of objective questions. The second part of each question contains an open-ended option that allows students to provide new answers or explanations. Questions in subjective questionnaires with poor measurement results were redesigned or decomposed into new detailed questions.(4)Questionnaire Distribution Questionnaires were distributed and then retrieved in person.
(4) Preconception Viewpoint Analysis. As shown in Table 4, respondents’ answers represent their judgment of the questions as well as their reasons for making those judgments, i.e., preconception viewpoints in their minds. Each answer combination was analyzed. A total of 41 specific preconception viewpoints (L1-L41) from the “Force and Movement” unit are summarized below.
Specific preconceptual views are associated with corresponding knowledge points. For example, for the interviewee who chose A for the two subquestions in question 1, A indicates that the interviewee has A preconceived view L12: inanimate objects cannot apply force. This preconceived view is the knowledge point related to Newton’s first law of knowledge, indicating that interviewees have a wrong understanding of F2’s fourth preconceived concrete content “F2—the definition of the three elements of force.” Preconceptions and how they relate to knowledge points are shown in Figure 1.
3.3.2. Dividing the Preconception Hierarchy into Stages
Dividing the preconception hierarchy into stages primarily involves using sample data to establish connections between the various preconception viewpoints and to build the preconception system structure. There are different connections between various knowledge content and preconceptions, potentially leading to different structures. Defining the structure of preconcept is the basis of designing preconcept attribution algorithm.
(1) Obtaining Connections between Preconception Viewpoints through Sample Data. Regression analysis for the various preconception viewpoints related to the “force and movement” topic was conducted using SPSS software, yielding significant correlations for preconception viewpoints. The table below lists significant correlations for all preconception viewpoints, represented by constructing vectors. For example, specific preconception viewpoint L1 is significantly correlated with L2, L4, L12, and L33, establishing correlation vector R1 (L1, L2, L4, L12, L33), as shown in Table 5. It can be seen that significant correlation among some preconception viewpoints is significantly higher than the average.
(2) Analyzing Reasons for Data Results and Establishing a Preconception System Structure. A number of commonalities and attributes can be observed from preconception viewpoints with a greater number of significant correlations amongst themselves. These preconception viewpoints show that students waver in their understanding of scientific concepts; these misunderstandings are the root cause of preconceptions. The other preconception viewpoints represent the preconceptions of users in explaining specific physical phenomena. At the same time, preconceptual views that can be analyzed can be divided into two levels. The top layer is the ontology layer, which refers to the user’s preconceptions about the specific content of scientific concepts, which may be because the user has not yet learned the scientific concepts or has difficulty remembering or understanding them. The bottom layer is the cognition level, referring to users’ preconceptions in understanding questions and understanding phenomena, which can arise for many different reasons. See Table 6 below for specifics.
It should be pointed out that preconceptions also exist in mathematics, chemistry, and biology. Some preconceptions therein also necessarily deal with concept ontology and application of concept ontology. As these fields differ in their knowledge content and attributes, however, they will exhibit different correlations and hierarchical structures.
3.3.3. Calculation of Preconception Correlation
Calculation of preconception correlation was conducted using SPSS analysis of sample data to obtain the correlation coefficients for preconception viewpoints with significant correlation, thereby realizing the attribution and prediction functions of this diagnosis method.
As can be seen from Table 5, there is a significant correlation between some preconceptions. Therefore, statistical methods can be used to calculate correlation coefficients of preconcepts with significant correlation using Bayesian probability formulas.
The formula for Bayesian probability is as follows:
In this formula, P(AB) is the joint probability of event AB; P(A) is the probability of event A; P(B) is the probability of event B; P(A|B) represents conditional probability, indicating the probability A under conditions of B; and P(B|A) represents conditional probability, indicating the probability of B under conditions of A.
It can be seen that, under conditions of sufficient samples,
Therein, P′ is the sampling probability {Lxi, Lxj ∈ S(o) ∪ S(f), 1 ≤ xi ≤ 41, 1 ≤ xj ≤ 41}.
When Lxi, Lxj ∈ Si, the two preconception viewpoints are significantly correlated:
Therein, the correlation coefficient of Lxi and ′(LxiLxj) can be observed. The Bayesian probability function can be used to predict other preconception viewpoints and to guide the adjustment of pedagogical strategies.
In summary, by conducting statistical analysis and calculation of preconception viewpoints, a preconception system with significant correlations between preconceptions, consisting of specific preconception viewpoints, can be made and an ontology level and cognition level can be constructed.
4. Method for Conducting Cognition Diagnosis for Users Using Preconception System
In a quantified learning system, the preconception system can be used to describe the preconception states and cognitive states of users, enabling attribution and prediction of learning obstacles. This information also provides a basis for system feedback and improvement of pedagogical quality. The algorithms are described as follows.
4.1. Preconception Attribution Algorithms
Attributing cause to preconceptions involves two dimensions: first, preconception viewpoints generated by errors in understanding scientific concept ontologies; second, preconception viewpoints generated due to application of preconceptions, rather than scientific concepts in explaining scientific phenomena.
4.1.1. Concept Ontology Dimension
As shown in Table 7, students responded to a two-stage objective diagnostic survey. Each answer corresponded to a preconception viewpoint Lxi. If Lxi ∈ S(o), then the preconception viewpoint is rooted in the ontology level; in other words, the student does not have an adequate understanding of the scientific concept itself, which can be directly traced to the knowledge point corresponding to Lxi. For example, in diagnosing a student as having preconception viewpoint L6, an instructor can use Figure 1 and Table 3 to figure out that the student does not have a proper understanding of “Knowledge point N2: when an object is subject to a nonzero combined force, its state of movement changes.”
4.1.2. Conceptual Cognition Dimension
If Lxi ∈ S(f), then the preconception viewpoint is rooted in the cognition level. In other words, the student’s preconception viewpoint is generated in the cognitive process, influenced by one or more preconceptions. The naïve theoretical framework in the student’s mind is reflected in this field. Attribution can be conducted by backtracking to derive the relevant preconception viewpoints.
As shown in Table 5, Rxi is defined as a significant one-dimensional correlation vector that exists between Lxi and other preconceptions. There exists r[xi][0] = Lxi, r[xi] [1] = L(xi + t1), r[xi] [2] = L(xi + t2)……r[xi][n] = L(xi + tn), where 1 ≤ xi + t1, xi + t2, xi + tn ≤ 41, t1, t2, tn ≠ 0, indicating the various preconceptions viewpoints significantly related to Lxi. n represents the number of preconception viewpoints related to Lxi. When the diagnostic result LXi ∈ S(f), the naïve theoretical framework generated by Lxi should be attributed.(1)In determining Rxi, test for whether L(xi + tm) ∈ S(o), where 1 ≤ xi + tm ≤ 41, 1 ≤ tm ≤ n. If it exists, L(xi + tm) is a preconception viewpoint in the naïve framework, Frame = { L(xi + tm)}.(2)If L(xi + tm) ∉ S(o), the various r in R(xi + tm) are determined.(3)As in (1), determine whether L[xi + tm][tm] ∈ S(o) in R(xi + t1), R(xi + t2) ……R(xi + tn). If yes, L[xi + tm][tm] is a preconception viewpoint in the naïve framework. Frame = {L(xi + tm), L[xi + tm][tm]}. If no, then repeat Step (2).(4)Based on iterations, there must be L[xi + tm][tm]n ∈ S(o), as specific preconception viewpoints are based on a misunderstanding of the knowledge point. As a result, attribution procedures will ultimately converge on one or more preconception viewpoints at the ontology level. As such, Lxi is attributed to the naïve theoretical framework:
Taking a real example, in diagnosing student Xiaoming as holding the preconception viewpoint L41, L41 ∈ S(f), then diagnosis can begin from R41.(1)R41 = (L41, L17, L39), L17 ∈ S(f), L39 ∈ S(f)(2)R17 = (L17, L10, L27, L39, L41), R39 = (L39, L17, L27, L41), L10 ∈ S(o), L27 ∈ S(o), then Frame = {L10, L27}
It can be seen that the student holds the preconception viewpoint L41, as they hold the naïve cognition framework Frame = {L10, L27}.
4.2. Learning Obstacle Prediction Algorithm
Once the user’s current preconceptions are known, the user’s future learning can be predicted.
It can be seen from the previously derived formulas that when Lxi, Lxj ∈ Si, the two preconception viewpoints are significantly related:
Sampling P′(LxiLxj), P′(Lxi), P′(Lxj) can be known. When the diagnosed student holds preconception, the probability of each specific preconception viewpoint significantly related to Lxi can be obtained for Rxi:
These calculation results indicate that the probability P0(Lxj|Lxi) of each related preconception viewpoint occurring can be used to predict learning obstacles emerging in the learning process. Performing ranking based on likelihood enables instructors to select appropriate learning paths.
4.3. Example of Personal Cognition Diagnosis Output Results
In summary, by constructing a preconception system to conduct preconception attribution and prediction, individuals can be diagnosed. The following information can be visualized: basic information, student ID, name, learning path, codes for preconception viewpoints currently held by the student, the level of the preconception, the reasons the student holds these preconception viewpoints, latent preconceptions the student may hold, the possibility those preconceptions exist, and the reasons those latent preconceptions may exist. The process of the cognitive diagnosis method is illustrated in Figure 2, and one example of method results is illustrated in Table 7.
5. Conclusions and Vision
By developing a preconception system for diagnosing users, the user cognition diagnostic method developed in this study produces outputs, including a description of the cognitive states of users, and also describes and attributes current preconception viewpoints, enabling the prediction of potential preconceptions in the learning process.
5.1. Satisfying E-Learning Requirements for Actual Pedagogy and Learning
The persistence and invisibility of preconceptions have consistently been a troubling issue in physics education. The diagnostic method in this study was developed directly based on pedagogical strategies, enabling visualization of the user’s preconceptions and their origins, as well as potential future learning difficulties. The method, offered to both instructors and teachers, facilitates the design of pedagogical activities effective in driving conceptual transformation.
The outputs of the diagnostic method developed in this study do not only represent how well a student does or does not understand a knowledge point, guessing at the level of familiarity or confidence for that knowledge point. The diagnostic method in this study was developed based on the design and application of students’ actual cognitive development processes. By taking into consideration the actual cognitive development of students during the learning process, as well as specific difficulties faced in the learning process, this method enables the visualization of users’ specific cognitive states, their difficulties, and the origins of those difficulties.
5.2. Low Design Difficulty, Easy Development
The design thinking and process for the cognitive diagnosis method introduced in this study were based on specific questions, allowing for easy understanding by developers. The design approach herein represents a commonplace research method that is also easy for instructors to learn and apply. At the same time, the research carrier in this diagnosis method is based on specific learning content, requiring less content to be analyzed and reducing the required sample sizes and workload. The difficulty of development is decreased. Correspondingly, satisfying the needs of users for more learning activities requires higher development costs. The application of this diagnosis method requires the designers of quantified learning systems to make wise, nimble choices based on actual needs.
5.3. Increasing the Precision of Adaptive Learning System Diagnosis
The purpose of adaptive learning systems is to meet the individualized learning needs of users in an adaptive way, which includes the following: supporting learning automation; diagnosing personal characteristics, pushing learning resources and increasing the precision of learning path selection; and supporting the personalization of learning for different users.
5.3.1. Increasing Diagnosis and Feedback Precision through Smart Design
By constructing a preconception system, the diagnosis method introduced in this study can accurately and realistically describe the current cognitive state of users, as well as trace the origins of current learning obstacles and preconceptions. The system can also predict future learning obstacles and visualize the results of automated diagnosis on the cognitive states of users. At the same time, the diagnosis method produces individualized diagnosis results based on the answers of different users. Based on these diagnostic results and in accordance with conceptual transformation theory, instructors can design learning activities and develop targeted learning resources, offering feedback rules for the adaptability of the adaptive learning system, guaranteeing the precision and accuracy of learning support. As such, this diagnosis method can provide information for developers to apply smart design in increasing the performance of the adaptive learning system.
5.3.2. Diagnostic Precision Increases with Sample Sizes
This study draws on data mining and machine learning concepts. Using statistical and probabilistic methods, one can view the preconceptual system as a structured training model. In the process of development and application, with the increase of sample size and the enrichment of training, the accuracy is also improved. The authors intend to increase sample sizes in future studies. There are two primary routes to increasing sample sizes. One is to increase sample sizes during the process of designing diagnosis methods, thus increasing accuracy. The second is to increase sample sizes during pedagogical processes applying this diagnosis method, increasing the precision of the correlation coefficients of preconception systems.
5.4. New Perspective on User Diagnosis
Currently, adaptive learning systems primarily diagnose the cognitive levels and capabilities of users. In other words, they focus on what the user is capable of doing and how the user’s recent development has progressed. This study was based on a different set of issues: what the user is not capable of and where learning obstacles lie; which knowledge points have not been grasped or where a user’s understanding is actually tenuous; and which knowledge points the user fails to grasp due to misunderstandings arising from preconceptions. This approach represents a new paradigm for developing adaptive learning systems. Its design, based on the diagnostic dimension of preconceptions, is an augmentation of existing adaptive diagnosis.
Data Availability
The labeled datasets used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare no conflicts of interest.
Acknowledgments
This paper was funded by Research on the Construction of Precision Learner Model in Intelligent Learning Environment; a project supported by the Ministry of Education of Humanities and Social Science Project (grant no. 18YJCZH169); Research on the Construction of Precision Learner Model in Intelligent Learning Environment Based on Cognitive Development Theory; a project supported by the Youth Fund Project of Northeast Normal University in Philosophy and Social Sciences (Fundamental Research Funds for the Central Universities) (grant no. 18XQ005); Research on the Construction and Representation Method of Precision Learner Model in Intelligent Learning Environment; and a project funded by China Postdoctoral Science Foundation (grant no. 2019M651189).