Abstract
We propose a learner profiling system that uses the multidimensional characteristic analysis of learners. For this purpose, a real-time monitoring system for learners was developed and then used to extract information of the characteristics of learners that appear in learning environments. The extracted information on the characteristics of learners is automatically constructed into personalized learner profiles through the learner profiling system. The contents of learner profiles consist of the cognitive ability of learners, the learning condition, and teacher assessment. To verify the effectiveness of the proposed study method, profiles of actual learners were constructed, with 210 elementary school students as subjects. As a result, the group that learned using the learner profiling showed a significant level of satisfaction in comparison to the group that did not use learner profiling. From this, it was demonstrated that the learner profiling system proposed in this paper could enhance learning effectiveness.
1. Introduction
Recently, Information and Communications Technologies (ICT) have been used in various fields and have been changing many areas of our daily lives. In particular, various teaching and learning methods, learning tools, and educational contents that apply ICT are being developed in the field of education. Wired and wireless networks and learning tools are used in education to improve learning effectiveness in learners. A representative case of applying ICT to education is e-learning. E-learning is a form of education based on electronic technology. Since the introduction of high-speed internet, e-learning has continued to develop through the introduction of cyber universities, specialized firms that confer degrees through the completion of online lectures only, thus generating a social issue [1]. Recently, various learning methods have been studied, where the concept of e-learning has expanded to include mobile learning, ubiquitous learning, and smart learning. Meanwhile, e-learning is being increasingly applied not only in school education, but also in corporate education, social education, and continuing education. The final goal of learning methods that use these ICT is to provide effective and efficient education that learners will find satisfactory, without being limited by time or space.
Despite these many advantages of e-learning, various problems have surfaced. The most significant of these is the lack of mutual interaction between the teacher and the learner who use IT devices as a medium. That is, the one-sided provision of learning contents, where the learner is not considered in the learning process, can cause the reduction of learning effectiveness and the loss of interest in learning. Learning is achieved by founding itself on the experience, culture, sex, cognitive ability, characteristics, and so forth of the learners [2]. However, learning that uses existing IT devices does not take the level of understanding about learning and the situation of the learners into consideration, so it may proceed as one-sided learning. Recently, in order to address these shortcomings, research was conducted on teaching-learning models and intelligent tutoring systems that take the characteristics of learners into consideration. A teaching-learning model that takes the characteristics of learners into consideration refers to an online learning method that considers the learners’ environment and the characteristics of individuals, exactly as one would in traditional learning methods [3, 4]. In [5, 6], the learners were classified according to their cognitive characteristics and, in [7], the characteristics of learners were explored based on the definitive characteristics of learners. In [8], learners’ learning methods were analyzed by classifying the learners according to their information perception and information processing methods. However, these studies mostly consist of those that try to produce an online learning model by using a teacher-centered learning model or a system-centered learning model [9, 10]. Thus, these studies did not take into consideration the diversity of types of learners that derives from the peculiarity of virtual space [11]. An intelligent tutoring system refers to a supplementary educational tool that manages learning and possesses functions such as information management and questions and answers (Q&A). Also, research on intelligent tutoring systems based on learners in an online environment has been conducted in diverse ways. They could be largely classified into research on providing learning functions and research on Q&A. A representative example of studies in providing learning functions is ITTs [12]. ITTs were developed through the Byzantium project among 6 universities. The Byzantium project provides efficient learning functions such as opening diverse courses and sharing information. However, since it is not a system developed on an education foundation, it does not provide the function necessary from the students’ or the teacher’s perspective. A representative example of the systems proposed based on Q&A is Atlas [13]. Atlas is a system that can automatically respond to the questions asked by the learners. Moreover, some studies are being conducted on learning systems that replace teachers, although Q&A only focuses on the functional aspects and most of the research does not consider the characteristics of the learners [14–16]. To provide learning in consideration of the characteristics of learners, we must obtain accurate information about the learners and provide a customized learning service using this information. To provide customized learning, we need learners’ profiles that can save and manage the learners’ information systematically. Learners’ profiles would need to include not only basic information such as learning information and learning history, but also information from analyzing the characteristics of the learners. In [17], Index of Learning Styles (ILS), a model for learners based on learning methods for analyzing learners, is proposed. The ILS model separates the learning methods of learners into 4 dimensions and measures them from an educational engineering perspective; learning methods are then classified based on this. In [18], an intelligent tutoring system is developed using learners’ profiles by using the ILS model. However, these studies are founded on the results measured from the general information of the learners, so it is difficult to categorize the learners specifically. In [19], the characteristics of learners are analyzed by measuring the willingness to participate in learning and the level of participation. In [20], the characteristics of learners are classified in the online environment based on 3 factors: the learners’ information, environment information, and learning content information. In [21], the characteristics of learners are classified into 5 elements (learner, environment, operation, data, and mutual interaction), and learners are classified based on the level of satisfaction of learners per element. In [22], the decisive elements of the level of participation and satisfaction of learners in the online environment that affect learning are researched, and an attempt is made to construct an online learning environment using this.
In this study, we propose a system that can extract the characteristics of learners through a real-time learner monitoring system and that can automatically construct learners’ profiles through multidimensional analysis. In order to correctly understand the characteristics of learners, observation and analysis on learners during the learning process is needed. The proposed learner profiling system is able to automatically generate profiles by automatically extracting and analyzing the characteristics of learners through real-time learner monitoring. Factors that need to be considered to extract the characteristics of learners are as follows. First, the learning ability of learners should be considered. Learning that does not consider the learning ability of learners reduces the learners’ interest in learning and decreases the level of understanding. Thus, providing learning by measuring the learning ability of learners plays an important role in increasing the effectiveness of learning. Second, the situational factors of learners should be considered. The levels of concentration and understanding of learning can differ greatly according to environmental factors. Third, the assessment of teachers should be considered. Teachers are the most sensitive to the various responses that arise in learning situations. Accordingly, judgment and assessment of teachers on learning are important factors in understanding the learning situation.
2. Related Work
The method with the greatest educational effectiveness among the teaching-learning models is the individualized instruction method [23]. The individually customized learning system method combines the individualized instruction method with ICT technology. Various types of individually customized learning systems include intelligent tutoring, personalized learning, and customized learning. An individually customized learning system takes into consideration the learning level, attitude, method, and motive of learners to recompose the learning material accordingly and thus provide a service to the learners. An individually customized learning system combines various ICT technologies. This chapter will explain the main technologies, namely, learner profiles and personalized learning.
2.1. Learner Profiles
Learner profiles are important sources of information that not only contain basic information such as the name, age, and gender of learners, but also reveal the learning ability, characteristics, and condition of the learners. Research related to learner profiles is not limited to studies that aim to manage the learners’ information but is expanding to include studies that provide customized learning that considers the characteristics of learners. In [24], a system is proposed that converts learners’ information into a Resource Description Framework (RDF), translates this information into learner profiles, and recommends contents that match the users’ preference. However, only simple profiles were used that employed basic learning information in order to analyze individualized characteristics, and a method to extract the characteristics of learners was not able to be proposed. In [25], the preference grade of learners was predicted by extending the learning time into a fuzzy hierarchy, converting it into fuzzy numbers, and giving grades to the learning time. However, the learning time of learners could not explain the deep connection between user preference and tendencies.
2.2. Individualized Instruction
Individual learning is a method that aims at providing customized learning to learners. The learning preference of learners was analyzed in [26], and the results were made available to the learners, which facilitated their application. The learner profiling technique was used to calculate and apply the learning preference. For the analysis of preference, the preference vector values of the topic map were used. Through this, the learners were allowed to choose the learning contents they prefer and proceed with learning, even with the same learning contents. However, the disadvantage is that the teachers would need to create various contents with the same material when creating learning contents. As such, many studies are underway that attempt to increase learning effectiveness and provide customized learning; however, there is inadequate amount of research on analyzing the characteristics of learners and on learning services that apply this analysis. In general, the characteristics of learners that are related to learning occur during the learning process and the learning situation. Despite this, learner profiles that do not consider this are being constructed and applied. Hence, this study aims at addressing these issues by developing a real-time learner monitoring system to extract and analyze the characteristics of learners that occur in learning situations. Also, this study aims to use this to construct solid learner profiles.
3. The Learner Profiling System
3.1. An Overview of the System
In this study, we proposes a system that can extract the characteristics of learners through a real-time learner monitoring system and that can automatically construct learners’ profiles through multidimensional analysis. Figure 1 shows an organized diagram of a real-time monitoring and learner profiling system as proposed in this study. The proposed method uses the real-time monitoring system to extract information about the learners in order to automatically extract the characteristics of learners that are observed during learning situations. The real-time monitoring system allows the teachers to monitor events and situations that occur during the learning process in real time and saves the assessment information of the learners. Also, the system saves the assessment information of teachers (student, class, and event), which is an important element for specific analysis of learners. The saved information on learner assessment is used to construct learner profiles, and these can be applied to the reconstruction of learning and customized learning systems, such as intelligent tutoring systems.
3.2. Real-Time Monitoring System
The real-time monitoring system proposed in this study is composed of functions including real-time learner monitoring, assessment, and extraction of learners’ characteristics. The real-time monitoring system monitors the learning situation of learners and extracts the information of learners that occurs during learning situations in real time and provides this to the teachers. In this process, the information on the characteristics of learners is automatically extracted and transmitted to the profiling system.
3.2.1. Real-Time Learner Monitoring and Assessment
Real-time learner monitoring and assessment are composed of the monitoring module that can monitor various events and the activity of learners in learning situations and of the assessment module for teachers in which the teachers can assess the learners on their learning. The monitoring module for learners provides a function in which teachers can check the results of the events occurring during learning situations in real time. At the same time, the module saves the measurement results from the events. The saved results are used to extract the characteristics of learners. The function of the assessment module for teachers is that of extracting the assessment information of teachers during learning situations. The assessment information of teachers regarding learning is as follows.(1)Level of understanding: information on the level of understanding of the learner regarding learning as assessed by the teacher subjectively.(2)Level of concentration: information on the level of concentration of the learner regarding learning as assessed by the teacher objectively.(3)Learning attitude: information on the attitude of the learner regarding learning as assessed by the teacher objectively.The assessment information of teachers is a subjective assessment of learners carried out while teachers are teaching. The Likert scale was used for assessment [27]. The results of the assessment are used in analyzing the characteristics of learners and in the learner profiling system.
3.2.2. Real-Time Learner Characteristic Extraction
The real-time monitoring system extracted the characteristics of learners by categorizing the cognitive ability of learners, the information on the learning condition, and the information on teacher assessment. This is an accurate reflection, since the information on the characteristics of learners has been extracted from learning situations in real time. The criteria for extracting the characteristics of learners are largely similarities, differences, fixed elements, and variable elements [28]. Within these criteria, the background, experience, aptitude, motive, learning, cognitive ability, and so forth of the learners can be extracted and analyzed in order to provide an optimized learning service to learners.
We extracted the characteristics of learners by only considering those that have a high level of correlation with the type of learning that occurs during learning situations. The extracted characteristics of learners are as follows. First is the information on the cognitive ability of learners; as one of the fixed characteristics of learners, cognitive ability can provide information on the differences between individuals from the cognitive perspective and the perspective of information processing. Second is the information on the learning condition of learners. The learning condition is the measured value of the level of development that occurred during the learning process. The variable characteristics of learners include intellectual ability, linguistic ability, psychological growth, and general/specific preparative learning. The intellectual ability refers to the ability to think abstractly, and the linguistic ability is the level of linguistic development. Psychological growth refers to the desires related to learning, such as the sense of unease regarding learning, the concept of self, and interest. Preparative learning refers to the knowledge or skill that the learners are expected to have already learned and is a common term used to describe age, culture, and educational curriculum. Third is the information on teacher assessment. The assessment of teachers provides important information that can help to determine learning situations.
From the log data, we extracted the results of events that occurred during learning situations from the monitoring system and the learners’ devices in order to extract the characteristics of the learners. The characteristics are classified into level of understanding and level of concentration. The level of understanding refers to how the learners understand the learning and is the change of intellectual and linguistic ability following learning. The level of understanding refers to the extent to which the learners conduct events related to the learning that occurs during learning situations. Equation (1) is used to calculate the level of understanding. Learnerund is used to indicate the score for the level of understanding of learners, while is the assessment score for the th event, and is the total number of events that occurred during a learning situation:The level of concentration refers to the data on interest and desire towards learning and is a measure of how much learners are interested in learning during learning situations and how much they concentrate on learning. The following equation is used to calculate the level of concentration:Learnercnt is used to indicate the learner’s level of concentration, while Learnerund is the score for the level of understanding, and Learneratt is the score for the learner’s attentiveness in class. Devicecount indicates how many times the learners’ learning devices were operated, and Averagecount is the average number of times the learning devices of learners were operated. is a value that compares the frequency of the devices of a learner operated in learning situations and the average frequency of the devices operated; this can be used to make judgments on the scores for the level of understanding and attitude as well as the degree of interest in learners. For example, if the scores for the level of understanding and attitude are high, and the frequency of which the learner device is operated is higher than the average frequency of operation, the learner has a high level of interest in learning. Conversely, if the scores for the level of understanding and attitude are low, and the frequency of the device of a learner operated is lower than the average frequency of operation, the learner has a low level of interest in learning. Also, this study has set up critical values for each value so that values that can express the level of interest in learners that are as close to reality as possible could be found.
3.3. Learner’s Profiling System
The learner profiling system automatically saves and constructs information on the characteristics of learners through a real-time monitoring system. The learner’s profiles are important information in analyzing the characteristics of learners and can be used in customized learning, reconstruction of learning, and e-portfolios. The learners’ profiles, as proposed in this study, are shown in Table 1.
Learners’ basic information is composed of their name, grade, and class and refers to the basic personal information of the learners. Learners’ cognitive ability information is determined by analyzing the learners’ cognitive ability that influences their learning. We composed cognitive ability information by measuring 3 elements: memory, concentration, and visual cognition perception. Memory was measured by measuring working memory, which is closely connected to learning. The measuring method used word recognition task, where the learners are presented with 15 words and are told to find the presented words among 50 words. Concentration was measured by using a visual continuous performance test (CPT) [29]. The learners were shown pictures of certain fruits and told to click when the computer monitor displayed pictures of fruits that were shown to the learners beforehand. Through this, the level of concentration that learners could hold for a certain time in response to stimulation could be measured. Visual cognition perception was measured by using a stroop task [30]. The learners were presented with letters written in different colors (red, yellow, blue, and black) and the response time and the numbers of errors were measured while the learners were reading the color of the letters. The learners’ learning situation information refers to the level of understanding and concentration measured from the real-time monitoring system. Teacher assessment information refers to the assessment information of teachers measured from the real-time monitoring system.
We used a self-organizing map (SOM), a clustering method, to classify the learners based on the information of their characteristics [31]. SOM is a type of unsupervised learning and is a method that expresses input vectors in a multidimensional space on a low-dimension grid space. Learning in SOM repeats the process of finding the weight vector closest to the input vector and moving it towards the input vector. In this process, the weight vectors surrounding the closest vectors are also moved towards the direction of the input vector. Consequently, SOM is an algorithm that makes possible the grouping of learners for their categorization, which is the aim of this study. Through the SOM algorithm, learners are classified into groups that possess similar characteristics. A detailed description of the process of classifying learners using the SOM algorithm is shown below.
SOM AlgorithmStep 1.Initialize strength of connections .Step 2.New input vector is characteristics of learners.Step 3.Calculate the distance between input vector and all competing units. The distance between input and output is Here, is the th input vector, and is the strength of connection between th input vector and th output.Step 4.Select closest output. Select output , the smallest distance .Step 5.Update the strength of connection between output and its neighbors.Step 6.Go back to step 2 and repeat.
4. Results and Discussion
4.1. The Environment for Constructing Learner Profiles
A behavioral experiment was conducted in this study on students attending an elementary school in Gyeonggi province, Korea, for 4 weeks, as shown in Table 2. It was conducted on a total of 210 students (Grade 4), and all teachers and learners were introduced to the system and educated on how to use it before the experiment began. The environment was prepared for the application of the proposed system and included an electronic blackboard, an electronic teaching desk, and 20 smart learning devices. The experiment was conducted 16 times for 4 weeks during class. For the experiment, the teachers were provided with the measured results and the learner profile results measured from the learner monitoring system for every class. The learners were also given the opportunity to check on their learner profile results using the system.
4.2. Contents of Constructing Learner Profiles
4.2.1. Real-Time Learner Characteristic Extraction
Information on cognitive ability was collected by measuring concentration, memory, and visual cognition perception, measured at the first stage of the experiment. Changes in information on cognitive ability were analyzed by measuring once a week for a total of 4 times.
As shown in Figure 2, the average change in the concentration of learners, 0.95, was not a significant improvement, probably due to the following. First, concentration is one of the many cognitive abilities that reflect the inherent characteristics of a learner; however, it is difficult to show significant improvement within a short period of time. Second, the measured initial values are high on average. Despite this, the average change in Class 5, to which the system was applied, was 0.95, which was higher than that in the control group, which was 0.275; it could be said that the attempt made by the system had a positive impact on learning.
Figure 3 shows the result of the measurements of visual cognition perception of learners. Although the change is small, this is acceptable, since it is the result that reflects the inherent characteristics of learners. However, since the change was higher than the change in the control group, the system could be said to have had a positive impact on learning.
Figure 4 shows the result of the measurements of visual intelligence of learners. Although the change is small, this is acceptable, since it is the result that reflects the inherent characteristics of learners. However, since the change was higher than the change in the control group, the system could be said to have had a positive impact on learning.
4.2.2. Information on the Learning Situation of Learners
The level of understanding was measured in each learning session for the information on the learning situation by using a real-time monitoring system. The experiment measured the levels of understanding and concentration for all the classes that participated in the experiment, and the role that the system played in influencing learning was analyzed by making comparisons with the control group.
Figure 5 shows an expression of the measured values of the level of understanding. The classes in which the proposed system was applied displayed an improvement of 20.8 points on average, while the classes in which the proposed system was not applied displayed an improvement of only 12.8 points on average. This is because as learning progressed, the understanding of the learning generally improved. It is interesting to note that Class 5 showed an improvement of 26.6 points on average from its initial value, but Class 4 showed only 15.9 points on average, which was similar to Class 6, the control group. This is most likely due to the difference between the abilities of the teachers to apply the system. It could therefore be said that the level of understanding has a positive impact on learning through the proposed system.
Figure 6 shows an expression of the measured values of concentration. The classes in which the proposed system was applied displayed an improvement of 28 points on average, while the classes in which the proposed system was not applied displayed an improvement of only 12.2 points on average. This is because as learning progressed, the concentration towards learning improved on the whole. In particular, Class 4 showed an improvement of 15.9 points on average, which was similar to that of Class 6, the control group. This is assumed to be due to the difference between the abilities of the teachers to apply the system. It could thus be said that the level of understanding also has a positive impact on learning through the proposed system.
4.2.3. Information on Teacher Assessment
The results of teacher assessment were collected through a questionnaire. The questionnaire’s purpose was to verify the level of satisfaction, since satisfaction has a strong connection to learning effectiveness. Since the final goal of constructing learner profiles is the promotion of learning effectiveness, it is possible to modify learning methods by verifying the level of satisfaction. The items on the questionnaire were verified for their appropriateness through a content validity ratio (CVR) analysis. This research utilized the opinion of 10 experts (professionals with 5 years or more of experience in the related field or currently a teacher) to assess the appropriateness of the contents of the questionnaire. As the result of expert assessment, the items scored 0.75 on average (0.2 as minimum, 1 as maximum), and the item with the lowest value was deleted. The contents of the questionnaire are shown in Table 3.
The questionnaire for teacher assessment was separated into that for the teacher and that for learners. The level of satisfaction for teachers was satisfactory at 4.178 on average. It could be said that the learner profiling that used a real-time monitoring system had a positive impact on learning. The level of satisfaction for learners was satisfactory at 4.114. This is also thought to mean that learner profiling was helpful in motivating learners to learn and to improve their learning in general.
5. Conclusion
We proposed a learner profiling system for providing customized learning by analyzing the characteristics of learners. To do this, the characteristics of learners that occur during learning situations were extracted by using a real-time monitoring system, and learner profiles were constructed. The contents of learner profiles are the cognitive ability, learning condition, and teacher assessment of learners. For the specific elements, the information on cognitive ability refers to the inherent learning ability of learners and is classified as memory, concentration, and visual cognition perception. Information on learning condition includes the levels of understanding and concentration towards the learning process. The information on teacher assessment refers to the level of satisfaction of learning. In order to verify the utility of the proposed research method, we constructed actual learner profiles with 210 elementary school students as subjects. The result of the experiment showed that the learners who used the proposed method showed a more positive impact in learning effectiveness than those who did not use it.
The contributions of this study are as follows. First, the result of this study supports the theory that the characteristics of learners that occur during learning situations have a significant relationship with learning effectiveness. This means that learning which considers the learner’s learning ability that occurs during learning situations and environmental factors could improve learning effectiveness. Second, learning which examines the learning condition of learners could promote the motivation of learners. Learners who used the learner profiles proposed in this study showed a higher average level of concentration than those who did not use the profiles. Similar results were found in tests for the level of satisfaction. Third, the system could positively affect learning from the aspect of learning convenience. The information on learners’ characteristics is extracted automatically by the real-time monitoring system and so participates in, rather than hinders, the learning process.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research was supported by the ICT R&D program of MSIP/IITP [B0101-15-0340] and was supported by the Soonchunhyang University Research Fund (no. 20150000).