Mathematical Problems in Engineering

Mathematical Problems in Engineering / 2020 / Article

Research Article | Open Access

Volume 2020 |Article ID 3485846 | https://doi.org/10.1155/2020/3485846

Eugenio Roanes-Lozano, Angélica Martínez-Zarzuelo, María José Fernández-Díaz, "An Application of Knowledge Engineering to Mathematics Curricula Organization and Formal Verification", Mathematical Problems in Engineering, vol. 2020, Article ID 3485846, 12 pages, 2020. https://doi.org/10.1155/2020/3485846

An Application of Knowledge Engineering to Mathematics Curricula Organization and Formal Verification

Academic Editor: Libor Pekař
Received15 May 2020
Revised17 Jul 2020
Accepted25 Jul 2020
Published11 Sep 2020

Abstract

The authors present a theoretical proposal for the organization of mathematical contents, more precisely to curricula development formalization and formal verification, inspired by knowledge engineering techniques. The situation addressed is the following: the starting point is a mathematical “official curriculum” (or part of it), not necessarily completely detailed. In our proposal, a group of experts would have to first build a detailed formulation of this curriculum (including the “prerequisite” relation between contents), which we will denominate “preprocessed official curriculum.” We detail how any “official curriculum development” could then be rigorously formalized and formally verified in a way inspired by rule-based expert system formal verification. We have defined the following terms: “contents soundness,” “contents completeness,” “relation soundness,” “relation completeness,” and “absence of cycles.” We believe that this is a completely new formalization within mathematics teaching theory that, once computer is implemented, would be very helpful. That would be the case, for instance, in countries where government sets the “official curricula” for Primary and Secondary Education and textbook contents have to be manually checked and approved by academic authorities: evaluators would “only” have to extract the textbook contents and set the “prerequisite” relation among them and let the computer do the rest.

1. Introduction

1.1. Context

Secular education is nowadays considered a fundamental right. Nevertheless, the first efforts to rule access to teaching and somehow organize and unify the different educational levels took place during the Age of Enlightenment. For instance, if we exclude very wealthy families that could afford private teachers, during many centuries, Primary Education was restricted to a small part of the population, usually relying on literate people with basic notions on arithmetic, geometry, history, religion, and so forth, as well as having no nationwide content organization [1]. Meanwhile, nations worldwide fix nowadays detailed Primary and Secondary curricula. Similarly, universities, although with a high degree of independence, fix their curricula too.

In the past, the accuracy of numeric and symbolic calculations relied on human beings. For instance, Tycho Brahe’s excellent astronomic observations and calculations were key to Kepler’s discovery of the laws of planetary motion, but his star catalog contained systematic errors due to the application of an erroneous value of parallax (as well as errors due to the neglect of polestar refraction and incorrect transcriptions by scribes) [2]. Therefore it is not surprising that the two main users of the pioneer computer algebra systems (REDUCE (REDUCE is a trademark of the RAND Corp.) and Macsyma (Macsyma is a registered trademark of Macsyma, Inc.)) when launched in the sixties were astronomers and high energy physicists.

Regarding knowledge handling automation, verification is now an important branch of artificial intelligence, especially important when dealing with safety-critical issues (see Sections 2.3 and 2.4). Performed by human experts in the past, it now relies on computerized methods.

1.2. Motivation

Taking into account the previous considerations, we concluded that a computer tool that could verify that a proposed development of a curriculum does cover all the contents and that its organization is sound could be very useful. The theoretical approach presented here would be the first step (formalization of the theoretical aspects) of the design and development of such computational tool. We address mathematical education because of the hierarchical nature and organization of its contents. The formal method presented here is inspired by knowledge engineering, more precisely by Rule-Based Expert Systems (RBES) verification.

Three examples justifying the convenience of having such a computer tool available are indicated as follows:(i)In some countries, textbooks have to be approved by educational authorities, which have to manually check their compliance with the “official curriculum” requirements. A computer-based approach could be convenient, time-saving, and, more importantly, reliable (see, e.g., the medical RBES verification milestone described in Section 2.4, where the mathematicians and computer scientists in charge of the computer tool found out a strange case that had not been considered by the panel of experts).(ii)Even in the case where authorities are not in charge of checking whether a textbook complies with the “official curriculum,” book authors may desire to verify whether textbooks follow certain guidelines (that all the contents are covered and the chosen ordering is correct). With the approach proposed, the availability of a guideline (comparable to the “official curriculum”) is a must.(iii)In some countries, Project Based Learning (PBL) is allowed but must reasonably cover the “official curriculum.” Again, in this teaching method, where work is distributed among different teachers, the adequacy of the curriculum relies on the criterion of the set of teachers that are in charge of guiding the students.

We believe that the possibility to automate the verification of the compliance of textbooks and projects is an important and useful task.

1.3. The Need for This Work

Regarding the present state of the art, some authors are approaching the topic of knowledge engineering and curricula or university knowledge, such as [3, 4], but do not analyze curricular contents into such detail. The closest approaches we have found are [5, 6]. In [5], the authors describe a Declarative Curricula Model Language (DCML) that uses temporal logic in order to determine the potential acquisition of competencies (based on the relation among different skills), checking that there are no competency gaps and focusing on learning goals and the compliance with the goals set for a specific grade. Meanwhile, [6] is focused on sequencing the content of web-based education (computer based learning), performing prerequisite checking, and finding possible holes.

We wanted to address the problem from a different point of view: the theoretical approach presented here considers an “official curriculum development” as a guideline to be followed, is comprehensive, and admits the existence of multiple paths (these differences are detailed in Section 3: Results and Discussion).

As will be shown in Section 2.6, the size of the problems to be addressed can clearly deserve a computational approach.

We have started from scratch to tackle this challenge, using in fruitful synergy concepts and ideas borrowed from logic, artificial intelligence, graph theory, and curriculum development. Designing such a theoretical model was not trivial and took months. A milestone was the possibility to address multiple paths, which delayed the design for weeks. The theoretical model is not an intellectual exercise, as it lays the foundations for an ulterior design and development of a sound computer tool, which we believe could be very useful.

1.4. Structure of the Paper

This paper is organized in different sections. Section 2 (Materials and Methods) begins with a brief overview of RBES (Section 2.1) and their formal verification (Section 2.2), followed by some examples of RBES for different purposes, including education (Section 2.3), and an example of the importance of logic verification of RBES (Section 2.4). Sections 2.1 to 2.4 can be skipped by a reader familiar with this subject matter.

Section 2.5 includes the theoretical framework, comparing what we call the “official curriculum” with the different types of curricula, and offers an overview of the graphic representation of educational knowledge.

Section 2.6 details the kind of problems to be addressed with the proposed methodology. Examples are taken from real Secondary Education curricula.

Subsequently, Section 2.7 details the methodology proposal. Firstly, a proposal for curriculum development formalization is presented. Our starting point is a mathematics “official curriculum” (or certain parts of it), which has to be preprocessed and completely developed (expanded) as a digraph including all the possible contents (nodes) and all ordered pairs of the “prerequisite” relation (directed edges) that could be involved in that (part of a) curriculum. This way, this digraph is far more detailed than the baseline “official curriculum.” Note that this digraph will include nodes and edges that may or may not be used in a certain “official curriculum development.” We decided to focus on the mathematics subject because of the existing strong hierarchy among the contents involved.

Next, a possible formal verification method is proposed (which is an adaptation of the RBES formal verification introduced in Section 2.2 and exemplified in Section 2.4).

The article ends with Section 3 (Results and Discussion) and Section 4 (Conclusions and Future Research).

2. Materials and Methods

2.1. A Brief Overview of RBES

Roughly speaking, a RBES consists of “rules,” “facts,” and “integrity constraints” [7].

The “rules” are logical expressions of the following form:where , , and , respectively, represent the logic conjunction, the logic disjunction, and the logic implication. Meanwhile, symbols must be replaced in the expression above by the logic negation () or no symbol at all. are the logical variables in the antecedent of the rule and are the logical variables in the consequent of the rule. Rules translate the knowledge of the experts. A medium size RBES can have dozens of rules.

The so-called “facts” are the characteristics that are true at a certain moment for the patient, apparatus, vehicle, course, student, and so forth under consideration by the RBES. These facts allow to “fire” some of the rules (i.e., to extract the logical consequences from the given facts and rules). The rules that can be fired are those such that all the variables or negations of variables that appear in their antecedents do hold.

There exist many methods to perform knowledge extraction: logic programming, answer-set programming, algebraic inference engines, and so forth. We would not address them here, as we are just proposing a theoretical framework.

Integrity constraints consist of incompatible extra knowledge specified by the experts (e.g., male pregnant).

2.2. A Brief Overview of RBES Verification

The verification process of a rule-based expert system involves checking the absence of logical contradictions within the RBES (in other words, that its logical consistency is assured).

An example of logical inconsistency would be the following: on the basis of the RBES knowledge and the data available regarding patient , the RBES concludes the following:(i)Patient should take medicine and should...(ii)Patient should not take medicine and should...

Note that if, for instance, a medical RBES recommends a dehydrated patient not to drink at all (because that is what the experts have stated in the knowledge in the background—an obviously very dangerous mistake from the experts’ side), this would not represent a “logical inconsistency” but an error included in the medical knowledge collection. It is clear that this kind of errors cannot be debugged from a logical point of view.

In the past, rule-based expert systems verification relied mainly on human supervision. Experts in RBES development and verification checked the charts, diagrams, tables, and so forth, on which the knowledge behind the RBES was based, and took decisions on its safety.

Nowadays, formal verification of RBES (i.e., a debugging from the logical point of view) is mechanized and usually carried out through computer processes [8, 9].

A RBES can be understood as a directed graph, where nodes are well-formed logic formulae following the logic underlying the RBES and directed edges correspond to the implications reflecting the knowledge extraction.

2.3. Some RBES Applications

RBES have been used to face very different issues. We mention some of those addressed by the first author’s research group:(i)Medical diagnosis of different illnesses: anorexia, health-care problems, depression, and bronchogenic carcinoma [1013](ii)Coronary artery bypass surgery techniques recommendations [14](iii)Identifying car breakdowns from dashboard lights information and giving an advice [15](iv)Personalized menus generation [16]

RBES have been applied to teaching in very different fields, like the early work of Clancey and Letsinger [17] with NEOMYCIN. This is a medical consultation system in which the pioneer MYCIN [18] knowledge base is reconfigured for its use in teaching diagnosis strategies.

RBES have also been used in Mineral Sciences and Geology. Meech [19] developed a series of RBES for teaching and training of mineral industry undergraduate students. The work of Folorunso et al. [20], at the undergraduate level too, involves the development of a mineral identification expert system so as to support this knowledge field education. Another example by one of the authors is [21], where a RBES is used to train undergraduates in metamorphic rocks recognition and classification.

RBES have also been applied to the teaching of mathematics. For example, Salekhova et al. [22] developed an expert system that builds up an individual educational strategy for a student of algebra, concluding that the use of expert systems in education can simplify and improve the process of teaching mathematics.

Other authors present a web-based intelligent education system. They use adaptive hypermedia techniques and RBES to achieve adapting the teaching to student needs and the evaluation of the student knowledge level [23, 24].

RBES are also used in other aspects of the educational field, for example, to advise undergraduate students and postgraduate students on course selection [2527] and to suggest scholarships to undergraduate students [28]. Expert systems are also employed for other education-related goals such as student performance analysis [29].

Regarding the educational curriculum, there are studies that analyze the need to include concepts related to expert systems of areas such as accounting in the curriculum [30, 31]. However, these studies fall outside the purpose of the present study.

2.4. An Example of the Importance of Logic Verification of RBES

We can illustrate the importance of verification of RBES by summarizing article [14]. This paper describes the automatization and ulterior verification of a clinical practice guideline regarding coronary artery bypass surgery techniques recommendation. The guideline was developed by a panel of well-known international experts.

A RBES translating the knowledge of a medical practice guideline was designed and developed. Despite the fact that all the RBES first tests were correct, a mechanical checking of the different possible input data (coronary illness, age, gender, LVEF, etc.) found out an important anomaly: for an elderly patient with very delicate health and suffering two different coronary illnesses (left common trunk disease and two-vessel disease with anterior proximal descendent not affected), all the possible surgery techniques were discarded! But at least one of the coronary surgery techniques should be always recommended. From the logical point of view, at least one coronary surgery technique should always be reached by forward firing. As regards the design of the RBES, the disjunction of the propositional variables corresponding to different coronary surgery techniques is added as true. Therefore, in this case, a logical contradiction is reached.

This failure led to a complete redrafting of this medical practice guideline. It is remarkable that the anomaly was found by the computer system and explained by the computer scientists and mathematicians that designed and developed the RBES (that were not aware of the advantages and disadvantages of the different coronary artery surgery techniques).

2.5. Theoretical Framework: an Overview of Different Types of Curricula and a Graphic Representation of Educational Knowledge

It is a well-known fact that in education in general (and in mathematics teaching in particular) there exists a difference between the intended, perceived, and implemented curricula [32]. This study does not focus on these differences, nor does it focus on the alignment of the intended, implemented, and attained curricula.

We define “official curriculum” as the general baseline curriculum that is fixed by educational authorities in many countries. Note, for example, that in some countries textbooks have to be approved manually by academic authorities. This is the reason why we believe that the implementation of our proposal would be helpful in this field.

Considering different levels of the curriculum and according to the definitions provided in [3335], the closest curriculum to our “official curriculum” is the intended curriculum (which is associated with a macrolevel), because the intended curriculum is determined by the educational policy makers and curriculum developers along with textbooks [36]. Meanwhile, both the implemented curriculum, which is related to the contents, time allocations, instructional strategies, and so forth and is associated with a mesolevel, and the attained curriculum, which is related to the students’ abilities and attitudes and is associated with a microlevel, are not so close to our “official curriculum.”

We consider that curriculum development formalization and formal verification can be of interest in any educational area and especially in the area of mathematics. The hierarchical structure of this discipline allows its concepts to be organized coherently according to a prerequisite relation. Thus, using knowledge engineering techniques for mathematics organization could be useful for different members of the educational community.

Therefore, the general objective of this study is to provide a theoretical proposal for mathematics curricula development formalization and formal verification using knowledge engineering techniques such as RBES.

An author that organizes educational concepts using graphical tools is Joseph D. Novak. His concept maps [3739] are graphical tools for organizing and representing educational knowledge, including both concepts and their relationships.

In this context, another interesting technique for educational knowledge graphic representation is pathfinder associative networks [40]. Some research carried out applying this type of technique has led to the formulation of a new theory: the theory of nuclear concepts [41, 42]. According to Luengo and Casas [43], the associative pathfinder networks allow not only hypothesizing about the internal processes of acquisition and organization of educational knowledge but also obtaining visual representations of their status and evolution.

Likewise, Astrid Brinkmann proposes the graphical representation of different sorts of connections among mathematical objects. According to [44]:

“Mathematical knowledge has the character of a network, as mathematical objects, i.e. for example concepts, definitions, theorems, proofs, algorithms, rules, theories, are manifold interrelated but also connected with components of the external world. Thus, a mathematical network may be represented by a graph whose vertices represent mathematical objects and nonmathematical components, and whose edges represent a relation on them, each of the edges linking the vertices of two mathematical objects or the vertex of a mathematical object and the vertex of a non-mathematical component.”

Other techniques of educational knowledge graphic representation are mental maps [45], semantic maps (also known as semantic networks), semantic organizers or semantic chains [4649], cognitive maps [5052], thinking maps [53, 54], knowledge maps [55, 56], idea mapping [57], flow charts [58], conceptual networks [59, 60], and so forth.

We have also worked on organizing, structuring, grouping, and sequencing educational contents using graph theory [6163]. Moreover, we have worked on debugging curriculum developments using graph theory techniques rather than RBES [64]. Furthermore, we have analyzed the spiral character of a part of the curriculum as well. We are currently studying sequencing mathematical contents thoroughly, focusing on mathematical contents. However, it should be noted that these studies can be easily extended to other areas of educational knowledge.

Similarly, [65] uses a prerequisite network as a means to visualize the hidden structure of a particular subject academic curriculum. In that study, the author conceives the academic curriculum as a complex system where nodes (or vertices in graph theory) represent grades, and direct links (or arcs) between courses represent prerequisite requirements.

Joseph Eigenmann proposes the organization of curricula in sequences [66]. Certainly, the idea of sequencing instructional experiences following a logical order can already be found in the work of seventies’ authors such as Gagné [67] and Tennyson [68]. A survey on the topic (in Spanish) can be found in [69].

In [70], content relevance-based sequencing and the time required for its teaching are proposed. In all cases, the main educational objective is to reconcile teaching theory and practice [71, 72].

2.6. The Size of the Problems to Be Addressed: The Origins of This Work

As said in the previous section, the authors have a long experience in structuring, grouping, and sequencing mathematical contents using graph theory [6163] (the first and third authors are the Ph.D. advisors of the second author who started working in the field back in 2010). If mathematical contents are detailed, the data handled cannot be considered as Big Data but become rapidly unmanageable by hand and appropriate computer programs are needed.

For instance, in [6163], the mathematics curricula of the four years of Spanish Compulsory Secondary Education (corresponding to Grades 7–10 in the K-12 system) are analyzed in detail. The digraph obtained has 814 nodes and 2217 edges (Figure 1) and managing this information required the use of specialized graph software (Gephi (Gephi is a trademark of Gephi NGO (“Association Gephi”)) and Pajek (Pajek, Program for Analysis and Visualization of Very Large Networks, by Mrvar and Batagelj)).

Even if restricted to a more concrete scenario, such as the algebraic contents in the first year of the Spanish Compulsory Secondary Education (Grade 7), the digraph has 20 nodes and 40 edges (Figure 2). This one is clear but would be really hard to verify manually.

As said above, the previous work dealt with organizing, structuring, grouping, and sequencing mathematical contents, something that could be achieved with graph software. But the idea of going one step further and verifying the underlying logical structure arose (borrowing ideas from knowledge engineering, in a cross-disciplinary approach). For achieving this goal, a mathematical formalization is firstly required, which is the topic of this work (see Section 2.7).

We would like to remark that the following section is not devoted to detailing full-size examples but to set the theoretical background and to illustrate case-by-case the different situations that can arise (e.g., the existence of multiple paths to reach a content).

2.7. Methodology

We will describe and order each of the definitions and steps that must be considered for the theoretical proposal of this study, in particular for the curricula development formalization and formal verification.

For the purpose of this paper, the term “content” refers to concepts, properties, sets of properties, operations, sets of operations, theorems, processes, and algorithms (namely, conceptual and procedural contents).

2.7.1. A Proposal of Curriculum Development Formalization

The notation of this section is not exactly the same but is partially inspired by the previous work [61]. In this paper, we will adopt the definitions and notations below.

Throughout this subsection, the following steps will be addressed:(i)Formalization of the “official curriculum”(ii)Formalization of the “preprocessed official curriculum”(iii)Existence of multiple paths(iv)Formalization of a given “official curriculum development”

We therefore begin by formalizing the “official curriculum.”

Definition 1. The “official curriculum” consists in the official standard guidelines provided by the national educational authorities.

Notation 1. Let be the set of (identified) contents in the “official curriculum.”
There are no more steps at this stage, as “official curricula” usually describe the contents to be covered (they only list them) and do not usually (or, at least, do not always) detail how to treat them or in which order (as sequencing is not uniquely determined if independent topics exist).

Example 1. An example of the items comprised in an “official curriculum” can be found in Figure 3.
Next, we can formalize the “preprocessed official curriculum.”

Definition 2. The “preprocessed official curriculum” is a digraph obtained from the chosen “official curriculum” (or part of it) by the following preprocess:(i)Its nodes are all the possible contents required for detailing the “official curriculum”(ii)Its directed edges correspond to the “immediate prerequisite” relation (see Definition 3) between the contents (nodes)This should be made by mathematics teachers with extensive experience in the particular educational level. Their role is similar to that of the experts who develop a RBES.
Formally, we have the following.

Notation 2. Let be the set of contents in the “preprocessed official curriculum.”

Consequence 1. As the “preprocessed official curriculum” details the “official curriculum,” .

Definition 3. Let . We shall say that is an “immediate prerequisite” of , denoted as , if and only if needs to be understood or proved and such that .

Example 2. The “preprocessed official curriculum” corresponding to the “official curriculum” in Example 1 can be found in Figure 4 (division is supposed to be taught in previous grades). Observe that new contents (nodes) have been introduced with respect to the “official curriculum” and directed arcs have been declared.

Definition 4. Let . The fact that is a “prerequisite” of will be denoted as . The “prerequisite” relation is obtained by calculating the transitive closure (the transitive closure of a relation is the smallest relation , such that is transitive) of the “immediate prerequisite” relation ().

Consequence 2. If the “preprocessed official curriculum” has been correctly constructed, () is a partial ordering.

Remark 1. All the ulterior stages require that this “preprocessed official curriculum” be comprehensive, completely detailed, and correct and rely on the work of experts.
From our point of view, a very important contribution to this formalization is the possibility to not only consider multiple paths but also simultaneously contemplate more than one path. That is the case when one topic is taught two ways (e.g., it is common to teach linear system solving using two different techniques during the same course).

Example 3. In the example presented in Figure 5, the content located to the right has three prerequisites, which means that the three of them have to be taught beforehand (in this particular case, “addition” is also a prerequisite for “subtraction” and “multiplication”).
Nevertheless, sometimes it is possible to teach a particular content in two different ways (i.e., using different prerequisites). In such a case, all the prerequisites will appear as nodes in the digraph, but each author will choose which one or ones to use in order to introduce the content.
An example of this would be the possibility of teaching linear system solving using Gaussian elimination method versus using Cramer’s rule (see Example 4). Another example is to introduce real numbers using Dedekind cuts versus using Cauchy series (nested intervals). Yet another example is to teach mechanical theorem proving in geometry using Gröbner bases [73] versus using Wu’s method [74].

Example 4. The digraph in Figure 6 would be incorrect. In this case, a possible solution would be to split the content “Linear system solving” into two contents, “Lin. syst. solv. ” and “Lin. syst. solv. ,” meaning “Linear system solving using Gaussian elimination” and “Linear system solving using Cramer’s rule,” respectively, as in Figure 7. This means that if two paths are being considered simultaneously, then multiple labeling of the same content is required.
Therefore, the experts developing the “preprocessed official curriculum” must be very careful in order to distinguish this latter case from the case we mentioned before where there was more than one prerequisite.
Finally, the last proposed step is the formalization of an “official curriculum development.” In the current paper, we define the “official curriculum development” as a curricular design made, for instance, by a group of teachers or textbook authors in accordance with an “official curriculum.”

Notation 3. Let be the set of contents in a particular “official curriculum development” (or part of an “official curriculum”).

Definition 5. In the “official curriculum development” under consideration, the author(s) will have established or used a specific “immediate prerequisite” relation. We shall name this relation , meaning that if and is allocated in the “official curriculum development” as an “immediate prerequisite” of , we shall write .

Example 5. An example of a possible development of the part of the curriculum shown on Figure 7 can be found in Figure 8.
The designations of the sets of contents and the symbols used to show their relations throughout this paper are presented in Table 1.
Note that the codification of the “official curriculum development” as a digraph has to be made using the contents specified in the “preprocessed official curriculum.”


Sets of contentsImmediate prerequisite relationPrerequisite relation

“Official curriculum”
“Preprocessed official curriculum”
“Official curriculum development”

2.7.2. A Proposal of Curriculum Development Formal Verification

We will analyze below what could be done to formally verify a certain proposal of curriculum development (in a way inspired by RBES verification).

We will establish common sense guidelines for developing an “official curriculum,” based on the idea that “a new content should not be taught before teaching its prerequisites.” These guidelines are the following:(i)The proposed “official curriculum development” must not introduce contents that have not been included in the “preprocessed official curriculum”(ii)The proposed “official curriculum development” must cover all the contents in the “official curriculum”(iii)The “paths” necessary to reach any content should be correct and without “bypasses” of required contents(iv)A content should not be a prerequisite of itself

It is important to note that, sometimes, when the “official curriculum” offers the possibility to follow different approaches (e.g., Gaussian elimination method or Cramer’s rule), the contents enclosed in the “official curriculum development” will be only those strictly necessary.

The algorithm proposed for the comprehensive verification process is as follows:Step 1a: contents soundnessStep 1b: contents completenessStep 2a: relation soundnessStep 2b: relation completenessStep 3: absence of cycles

and is detailed afterwards.

Therefore, we will analyze contents first, starting with contents soundness.

Step 1a. It involves checking that all the contents contained in the proposed “official curriculum development” already appear in the “preprocessed official curriculum”; that is, .

Otherwise, experts should be contacted in order to determine whether this unexpected content that was missing in the “preprocessed official curriculum” should be included in it (in ) or whether it should be excluded from the proposed “official curriculum development” (i.e., excluded from ).

It should be remarked that this step is necessary in order to automate the verification. If a content not appearing in the “preprocessed official curriculum” was introduced in the “official curriculum development,” the “preprocessed official curriculum” could not be used as a correction guide.

Now, we will introduce contents completeness.

Step 1b. It involves checking whether all contents specified in the “official curriculum” are dealt with in its proposed development, that is, whether or not has to be checked.

Otherwise, the proposed “official curriculum development” would not cover the whole subject and should therefore be rejected.

As a second step within the comprehensive verification process, we propose to test the relation, starting with its soundness.

Please note that a relation is formally defined as a set of ordered pairs of elements and therefore a subset of the Cartesian product of the set by itself. That is why we can write an inclusion symbol between relation symbols.

Let us recall that is the transitive closure of .

Step 2a. It involves analyzing whether the ordered pairs of the “immediate prerequisite” relation considered at the proposed “official curriculum development” were already included in the “preprocessed official curriculum” as an immediate prerequisite or prerequisite, that is, whether has to be checked (note that if denotes the transitive closure of , checking is equivalent to checking , as is necessarily transitive (because it is the transitive closure of )).

If that is not the case, the experts should be contacted in order to determine whether the unexpected ordered pair detected on the prerequisite relation should be included in the corresponding relation of the “preprocessed official curriculum” or an incorrectness has shown up in the proposed “official curriculum development.”

Example 6. For instance, the “official curriculum development” proposal shown in Figure 9 would not satisfy the condition of Step 2a because there is an unexpected arrow that does not appear in the transitive closure of . In this case, the proposal would not be correct.
Next, we discuss in detail how the relation completeness is addressed.
Step 2b. It has to be checked whether all “immediate prerequisite” relation ordered pairs of the proposed “official curriculum development” appear in the “preprocessed official curriculum,” that is, whether or not.
Otherwise, this would mean that there is a prerequisite in the “preprocessed official curriculum” which the authors of the proposed “official curriculum development” have bypassed. The authors of the proposed “official curriculum development” should be informed of this anomaly. In case they insist on the correctness of their approach, the experts who built the “preprocessed official curriculum” should be contacted in order to check the authors’ claim.

Example 7. If the proposed “official curriculum development” in Example 4 was that of Figure 10, the authors would have bypassed the content “Equivalent systems,” that is, but . Therefore, this “official curriculum development” would need to be corrected.
Finally, the absence of cycles has to be checked in order to complete the part of the verification process concerning the relation.
Step 3. It involves checking that there are no cycles in the “prerequisite” relation. It can also be interpreted as checking that the corresponding digraph is acyclic.
The following theorem justifies this requirement.

Theorem 1. If there are cycles in , then there is a logical inconsistency.

Proof. If there is a cycle, certain concepts exist, such thatwhere concept requires of itself to be understood/explained, which is not admissible in mathematics.

Example 8. An obviously wrong example would bewhich arises if we define apiculture as beekeeping, beekeeping as the labour of a beekeeper, and a beekeeper as someone that works in apiculture.

Remark 2. If this takes place during the “official curriculum development,” a problem has been detected in the “preprocessed official curriculum” (as ). Consequently, the latter has to be revised and development has to be corrected accordingly.

3. Results and Discussion

As a first result, we have detailed how it is possible to deal with curricula formalization and verification using a cross-disciplinary approach, viewing pedagogy from a computer science or, more precisely, curriculum studies from knowledge engineering. We believe that this can be a very fruitful symbiosis.

Special attention has been paid to modeling the case when multiple paths exist. As a second result, our model admits multiple paths not only in the “official curriculum” but also in the “official curriculum development” (this means that it is possible to handle a real situation where a content can be taught in two different ways in a certain course; see Section 2.7.1).

As a third result, the reuse of well-known and intensively experimented techniques from other scientific fields (knowledge extraction and verification in RBES) will allow developing and implementing this approach following a well-known path, which would involve less effort.

Considering the current state of the art, and more precisely [5, 6], our proposal is similar but also very different. Firstly, ours focuses on comparing not only the correctness of a sequencing but also a proposed “official curriculum development” with a preestablished one (the “official curriculum”). Secondly, it takes into account the existence of multiple paths in the proposed curriculum (considering that more than a certain sequencing might be chosen among the possible ones), which requires special contents treatment, that is, multiple labeling. Thirdly, the verification we propose is comprehensive, including contents and relation soundness and completeness, as well as the absence of cycles.

On the one hand, as shown in the references related to RBES knowledge extraction and verification [1015], there are no problems in dealing with big expert systems, so there are no size limitations.

On the other hand, we found some limitations for this computer tool future applications. It is curriculum-independent, which means that, once designed and developed, it can address any case. But, as explained in Definition 2, the “preprocessed official curriculum” is obtained from the “official curriculum” by humans (we recommend that this is accomplished by mathematics teachers with long experience in the particular educational level). Thus, having such initial development available would rely on the interest of the academic authorities or a research group working in this field. The same situation holds, for instance, for RBES in medicine.

The second limitation is that any new educational law normally includes changes to the “official curriculum,” which implies modifications of the “preprocessed official curriculum.” Therefore these changes should be entered into the computer tool. This is also the case, for instance, when a new therapy or surgery technique is approved: the corresponding RBES has to be updated.

4. Conclusions and Future Research

The proposed verification process requires the work of human beings for preparing the “preprocessed official curriculum.” Nevertheless, that work has to be done only once. From there onwards, the proposed “official curriculum development” verification could be carried out by computers. Consequently, the computer tool that can be designed and developed from this theoretical study could be very useful for curricular developments verification (such as textbooks or PBL development).

This paper establishes a possible theoretical basis for the formal verification of a proposed “official curriculum development.” Our future research goals are listed as follows:(i)To design the software tool for developments of “official curricula” verification. It does not have to be developed from scratch, as the required processes are related to others known in graph theory and artificial intelligence. Initially, Boolean logic is considered to be the appropriate one, but it is necessary to decide which approach to use (logic programming, algebraic inference engine, etc.).(ii)To develop and implement the software tool.(iii)To develop a “preprocessed official curriculum” based on a certain existing “official curriculum” for the real testing of the software tool. An almost ready example can be found in [62].(iv)To test and refine the software tool.(v)To offer the computer tool to colleagues in the field of mathematics education for its debugging and improvement.

Data Availability

There are no experimental data to share. It is a theoretical proposal.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was partially supported by the Government of Spain under Grant PGC2018-096509-B-100.

References

  1. C. Solano Macías and A. Vivas Moreno, La cultura escrita en Extremadura a finales del siglo XVIII a través del interrogatorio de la Real Audiencia, Trea, Gijón, Spain, 2018.
  2. R. Pearson, The History of Astronomy, Astro Publication, Nottingham, UK, 2020.
  3. J.-M. Rubio, W. Palma, N. Rodriguez et al., “Solving the balanced academic curriculum problem using the ACO metaheuristic,” Mathematical Problems in Engineering, vol. 2013, Article ID 793671, 8 pages, 2013. View at: Publisher Site | Google Scholar
  4. Y.-Q. Zhao, “Agent-based supernetworks model of university knowledge system,” Mathematical Problems in Engineering, vol. 2015, Article ID 320601, 8 pages, 2015. View at: Publisher Site | Google Scholar
  5. M. Baldoni and E. Marengo, “Curriculum model checking: Declarative representation and verification of properties,” in Creating New Learning Experiences on a Global Scale. European Conference on Technology Enhanced Learning, E. Duval, R. Klamma, and M. Wolpers, Eds., pp. 432–437, Springer, Heidelberg, Germany, 2007. View at: Google Scholar
  6. P. Brusilovsky and J. Vassileva, “Course sequencing techniques for large-scale web-based education,” International Journal of Continuing Engineering Education and Life Long Learning, vol. 13, no. 1-2, pp. 75–94, 2003. View at: Publisher Site | Google Scholar
  7. E. Roanes-Lozano, L. M. Laita, A. Hernando, and E. Roanes-Macías, “An algebraic approach to rule based expert systems,” Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, vol. 104, no. 1, pp. 19–40, 2010. View at: Publisher Site | Google Scholar
  8. R. Drechsler, “Verification of multi-valued logic networks,” in Proceedings of the 26th International Symposium on Multiple–Valued Logic, 1996, R. Spencer, Ed., IEEE Computer Society Press, Santiago de Compostela, Spain, May 1996. View at: Publisher Site | Google Scholar
  9. L. M. Laita and L. de Ledesma, “Knowledge-based systems verification,” in Encyclopedia of Computer Science and Technology, A. Kent and J. G. Williams, Eds., pp. 253–280, Marcel Dekker, New York, NY, USA, 1997. View at: Google Scholar
  10. C. Pérez-Carretero, L. M. Laita, E. Roanes-Lozano, L. Lázaro, J. González-Cajal, and L. Laita, “A logic and computer algebra-based expert system for diagnosis of anorexia,” Mathematics and Computers in Simulation, vol. 58, no. 3, pp. 183–202, 2002. View at: Publisher Site | Google Scholar
  11. M. Jimenez, J. Santamaría, R. Barchino et al., “Knowledge representation for diagnosis of care problems through an expert system: model of the auto-care deficit situations,” Expert Systems with Applications, vol. 34, no. 4, pp. 2847–2857, 2008. View at: Publisher Site | Google Scholar
  12. C. Rodríguez-Solano, L. M. Laita, E. Roanes-Lozano, L. López-Corral, and L. Laita, “A computational system for diagnosis of depressive situations,” Expert Systems with Applications, vol. 31, no. 1, pp. 47–55, 2006. View at: Publisher Site | Google Scholar
  13. I. Rodriguez-Daza, L. M. Laita, E. Roanes-Lozano et al., “An outline of an expert system for diagnosis and treatment of bronchogenic carcinoma,” in Medical Data Analysis. International Symposium on Medical Data Analysis, P. Perner, R. Brause, and H. G. Holzhütter, Eds., pp. 118–126, Springer, Heidelberg, Germany, 2003. View at: Google Scholar
  14. L. M. Laita, E. Roanes-Lozano, V. Maojo, E. Roanes-Macías, L. De Ledesma, and L. Laita, “An expert system for managing medical appropriateness criteria based on computer algebra techniques,” Computers and Mathematics with Applications, vol. 42, no. 12, pp. 1505–1522, 2001. View at: Publisher Site | Google Scholar
  15. E. Roanes-Lozano, J. L. Galán-García, and G. Aguilera-Venegas, “A portable knowledge-based system for car breakdown evaluation,” Applied Mathematics and Computation, vol. 267, pp. 758–770, 2015. View at: Publisher Site | Google Scholar
  16. E. Roanes-Lozano, J. L. Galán-García, and G. Aguilera-Venegas, “A prototype of a RBES for personalized menus generation,” Applied Mathematics and Computation, vol. 315, pp. 615–624, 2017. View at: Publisher Site | Google Scholar
  17. W. J. Clancey and R. Letsinger, NEOMYCIN: Reconfiguring a Rule-Based Expert System for Application to Teaching, Department of Computer Science, Stanford University, Stanford, CA, USA, 1982.
  18. E. H. Shortliffe, Computer-based Medical Consultations: MYCIN, American Elsevier, New York, NY, USA, 1976.
  19. J. A. Meech, “Expert systems for teaching and training in the mineral industry,” Minerals Engineering, vol. 3, no. 1-2, pp. 129–136, 1990. View at: Publisher Site | Google Scholar
  20. I. O. Folorunso, O. C. Abikoye, R. G. Jimoh, and K. S. Raji, “A rule-based expert system for mineral identification,” Journal of Emerging Trends in Computing and Information Sciences, vol. 3, no. 2, pp. 205–210, 2012. View at: Google Scholar
  21. E. Roanes-Lozano, R. García, E. Roanes-Macías, A. Aparicio, and L. M. Laita, “Meta-petro: an expert system for training undergraduates in metamorphic rocks recognition and classification using photomicrographies,” in Computer Algebra in Scientific Computing. CASC 2005. Lecture Notes in Computer Science, V. G. Ganzha, E. W. Mayr, and E. V. Vorozhtsov, Eds., pp. 344–354, Springer, Heidelberg, Germany, 2005. View at: Google Scholar
  22. L Salekhova, A. Nurgaliev, R. Zaripova, and N. Khakimullina, “The Principles of Designing an Expert System in Teaching Mathematics,” Universal Journal of Educational Research, vol. 1, no. 2, pp. 42–47, 2013. View at: Google Scholar
  23. I. Hatzilygeroudis, C. Giannoulis, and C. Koutsojannis, “Combining expert systems and adaptive hypermedia technologies in a web based educational system,” in Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT’05), pp. 249–253, IEEE, Kaohsiung, Taiwan, July 2005. View at: Publisher Site | Google Scholar
  24. I. Hatzilygeroudis, P. Chountis, C. Giannoulis, and C. Koutsojannis, “Using expert systems technology for student evaluation in a web based educational system,” in Proceedings of the IASTED International Conference on Web-Based Education (WBE-2005), pp. 21–23, grindelwald, Switzerland, February 2005. View at: Google Scholar
  25. N. Dunstan, “ET: an enrolment tool to generate expert systems for university courses,” in Expert Systems, P. Vizureanu, Ed., pp. 35–46, IntechOpen, Croatia, Balkans, 2010. View at: Google Scholar
  26. M. A. Al Ahmar, “A prototype student advising expert system supported with an object-oriented database,” International Journal of Advanced Computer Science and Applications, Special Issue on Artificial Intelligence, vol. 1, no. 3, pp. 100–105, 2011. View at: Publisher Site | Google Scholar
  27. A. Al-Ghamdi, S. Al-Ghuribi, A. Fadel, and F. A. A. T. AL-Ruhaili, “An expert system for advising postgraduate students,” International Journal of Computer Science and Information Technologies, vol. 3, no. 3, pp. 4529–4532, 2012. View at: Google Scholar
  28. G. Engin, B. Aksoyer, M. Avdagic et al., “Rule-based expert systems for supporting university students,” Procedia Computer Science, vol. 31, pp. 22–31, 2014. View at: Publisher Site | Google Scholar
  29. S. Hossain, D. Sarma, F. Tuj-Johora, J. Bushra, S. Sen, and M. Taher, “A belief rule based expert system to predict student performance under uncertainty,” in Proceedings of the 2019 22nd International Conference on Computer and Information Technology (ICCIT), pp. 1–6, IEEE, Dhaka, Bangladesh, December 2019. View at: Publisher Site | Google Scholar
  30. L. M. Hye and E. S. Y. Nar, “An expert system for the teaching of income tax in the accounting curriculum,” Accounting Education, vol. 2, no. 1, pp. 43–52, 1993. View at: Publisher Site | Google Scholar
  31. A. Sangster, “The integration of expert systems within the accounting curriculum,” Accounting Education, vol. 4, no. 3, pp. 211–216, 1995. View at: Publisher Site | Google Scholar
  32. L. K. Smith and S. A. Southerland, “Reforming practice or modifying reforms?: elementary teachers' response to the tools of reform,” Journal of Research in Science Teaching, vol. 44, no. 3, pp. 396–423, 2007. View at: Publisher Site | Google Scholar
  33. A. Thijs and J. van den Akker, Curriculum in Development, SLO–Netherlands Institute for Curriculum Development, Enschede, Netherlands, 2009.
  34. J. Van den Akker, “The science curriculum: between ideals and outcomes,” in International Handbook of Science Education, B. J. Fraser and K. G. Tobin, Eds., pp. 421–449, Kluwer Academic Publishers, Dordrecht, Netherlands, 2003. View at: Google Scholar
  35. J. Van den Akker, “Curriculum perspectives: an introduction,” in Curriculum Landscapes and Trends, J. van den Akker, W. Kuiper, and U. Hameyer, Eds., pp. 1–10, Kluwer Academic, London, UK, 2010. View at: Google Scholar
  36. W. Kuiper, E. Folmer, and W. Ottevanger, “Aligning science curriculum renewal efforts and assessment practices,” in Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy, D. Corrigan, R. Gunstone, and A. Jones, Eds., pp. 101–118, Springer, Dordrecht, Netherlands, 2013. View at: Google Scholar
  37. J. D. Novak and D. B. Gowin, Learning How to Learn, Cambridge University Press, Cambridge, UK, 1984.
  38. J. D. Novak, “Learning, creating and using knowledge,” in Concept Maps as Facilitative Tools in Schools and Corporations, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 1998. View at: Google Scholar
  39. J. D. Novak and A. J. Cañas, The Theory Underlying Concept Maps and How to Construct and Use Them, Florida Institute for Human and Machine Cognition, Pensacola, FL, USA, 2008, Technical Report IHMC CmapTools 2006-01 (Rev 2008-01).
  40. R. W. Schvaneveldt, Pathfinder Associative Networks: Studies in Knowledge Organization, Ablex Publishing, Norwood, NJ, USA, 1990.
  41. L. M. Casas, “El estudio de la estructura cognitiva de alumnos a través de Redes Asociativas Pathfinder. Aplicaciones y posibilidades en Geometría,” Universidad de Extremadura, Badajoz, Spain, 2002, Tesis Doctoral. View at: Google Scholar
  42. L. M. Casas and R. Luengo, “Representación del conocimiento y aprendizaje. Teoría de los Conceptos Nucleares,” Revista Española de Pedagogía, vol. 227, pp. 59–84, 2004. View at: Google Scholar
  43. L. M. Casas and R. Luengo, “Redes Asociativas Pathfinder y Teoría de los Conceptos Nucleares. Aportaciones a la investigación en Didáctica de las Matemáticas,” in Investigación en Educación Matemática: séptimo Simposio de la Sociedad Española de Investigación en Educación Matemática (SEIEM), E. Castro, P. Flores, T. Ortega, L. Rico, and A. Vallecillos, Eds., pp. 179–188, Universidad de Granada, Granada, Spain, 2003. View at: Google Scholar
  44. A. Brinkmann, “Mathematical networks conceptual foundation and graphical representation,” in Current State of Research on Mathematical Beliefs, R. Soro, Ed., University of Turku, Turku, Finland, 2001. View at: Google Scholar
  45. T. Buzan and B. Buzan, El libro de los mapas mentales: cómo utilizar al máximo las capacidades de la mente, Ediciones Urano, Barcelona, Spain, 1996.
  46. P. D. Pearson and D. D. Johnson, Teaching Reading Comprehension, Harcourt School, San Diego, CA, USA, 1978.
  47. K. M. Fisher, “Semantic networking: the new kid on the block,” Journal of Research in Science Teaching, vol. 27, no. 10, pp. 1001–1018, 1990. View at: Publisher Site | Google Scholar
  48. J. E. Heimlich and S. D. Pittelman, Elaboración de Mapas Semánticos como Estrategia de Aprendizaje: Aplicaciones para el Salón de Clases, Trillas, México, 2001.
  49. J. F. Sowa, Principles of Semantic Networks: Explorations in the Representation of Knowledge, Morgan Kaufmann, Burlington, VT, USA, 2014.
  50. G. M. Diekhoff and K. B. Diekhoff, “Cognitive maps as a tool in communicating structural knowledge,” Educational Technology, vol. 22, no. 4, pp. 28–30, 1982. View at: Google Scholar
  51. R. M. Kitchin, “Cognitive maps: what are they and why study them?” Journal of Environmental Psychology, vol. 14, no. 1, pp. 1–19, 1994. View at: Publisher Site | Google Scholar
  52. C. de Castro, “Mapas cognitivos. qué son y cómo explorarlos,” Scripta Nova: Revista Electrónica de Geografía Y Ciencias Sociales, vol. 33, 1999. View at: Google Scholar
  53. D. Hyerle, “Thinking maps: seeing is understanding,” Educational Leadership, vol. 53, no. 4, pp. 85–89, 1996. View at: Google Scholar
  54. D. Hyerle and L. Alper, Student Successes with Thinking Maps: School-Based Research, Results, and Models for Achievement Using Visual Tools, Corwin Press, Thousand Oaks, CA, USA, 2011.
  55. R. A. Howard, “Knowledge maps,” Management Science, vol. 35, no. 8, pp. 903–922, 1989. View at: Publisher Site | Google Scholar
  56. A. M. O”Donnell, D. F. Dansereau, and R. H. Hall, “Knowledge maps as scaffolds for cognitive processing,” Educational Psychology Review, vol. 14, no. 1, pp. 71–86, 2002. View at: Google Scholar
  57. B. B. Armbruster and T. H. Anderson, “Idea-mapping: the technique and its use in the classroom or simulating the “ups” and “downs” of reading comprehension,” Reading Education Report, vol. 36, pp. 1–33, 1982. View at: Google Scholar
  58. E. Geva, “Mejora de la comprensión lectora mediante diagramas de flujo,” Infancia Y Aprendizaje, vol. 8, no. 31-32, pp. 45–66, 1985. View at: Publisher Site | Google Scholar
  59. L. R. Galagovsky, “Redes conceptuales: base teórica e implicaciones para el proceso de enseñanza-aprendizaje,” Ensenanza de las Ciencias, vol. 11, no. 3, pp. 301–307, 1993. View at: Google Scholar
  60. L. R. Galagovsky, Redes Conceptuales. Aprendizaje, Comunicación Y Memoria, Lugar, Buenos Aires, Argentina, 1996.
  61. A. Martínez-Zarzuelo, E. Roanes-Lozano, and M. J. Fernández-Díaz, “About organizing and structuring the contents of mathematical subjects using graph theory,” in Graph Theory: New Research, A. Cavalcante, Ed., pp. 185–203, Nova Science Publishers, Inc, New York, NY, USA, 2013. View at: Google Scholar
  62. A. Martínez-Zarzuelo, “Selección, organización y secuenciación del conocimiento matemático mediante teoría de grafos,” Universidad Complutense de Madrid, Madrid, Spain, 2015, Tesis Doctoral. View at: Google Scholar
  63. A. Martínez-Zarzuelo, E. Roanes-Lozano, and M. J. Fernández-Díaz, “Grouping mathematical contents using network analysis software. An application to the Spanish Secondary Education case,” The International Journal for Technology in Mathematics Education, vol. 24, no. 4, pp. 199–206, 2017. View at: Google Scholar
  64. A. Martínez-Zarzuelo, E. Roanes-Lozano, and M. J. Fernández-Díaz, “A computer approach to mathematics curriculum developments debugging,” Eurasia Journal of Mathematics, Science & Technology Education, vol. 12, no. 12, pp. 2961–2974, 2016. View at: Publisher Site | Google Scholar
  65. P. R. Aldrich, “The curriculum prerequisite network: modeling the curriculum as a complex system,” Biochemistry and Molecular Biology Education, vol. 43, no. 3, pp. 168–180, 2015. View at: Publisher Site | Google Scholar
  66. J. Eigenmannm, El desarrollo secuencial del curriculum, Anaya, Madrid, Spain, 1981.
  67. R. M. Gagné, “1: learning and instructional sequence,” Review of Research in Education, vol. 1, no. 1, pp. 3–33, 1973. View at: Publisher Site | Google Scholar
  68. R. D. Tennyson, “A review of experimental methodology in instructional task sequencing,” AV Communication Review, vol. 20, no. 2, pp. 147–159, 1972. View at: Google Scholar
  69. J. L. Rodríguez, “La estructura del mensaje en el acto didáctico: revisión del problema y propuesta metodológica,” Enseñanza & Teaching: Revista interuniversitaria de didáctica, vol. 1, pp. 57–76, 1983. View at: Google Scholar
  70. M. A. Zabalza, Diseño Y Desarrollo Curricular, Narcea Ediciones, Madrid, Spain, 2009.
  71. T. Leuders, B. Barzel, and S. Hußmann, “Outcome standards and core curricula: a new orientation for mathematics teachers in Germany,” Zentralblatt für Didaktik der Mathematik, vol. 37, no. 4, pp. 275–286, 2005. View at: Publisher Site | Google Scholar
  72. J. M. Goñi, “El currículo de Matemáticas en la educación secundaria obligatoria y el bachillerato,” in Matemáticas. Complementos de formación disciplinar, J. M. Goñi, Ed., pp. 9–28, Graó, Barcelona, Spain, 2011. View at: Google Scholar
  73. B. Buchberger, “Bruno Buchberger's PhD thesis 1965: an algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal,” Journal of Symbolic Computation, vol. 41, no. 3-4, pp. 475–511, 2006. View at: Publisher Site | Google Scholar
  74. W. T. Wu, Mechanical Theorem Proving in Geometries, Springer-Verlag, Wien, Switzerland, 1994.

Copyright © 2020 Eugenio Roanes-Lozano et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views236
Downloads240
Citations

Related articles

Article of the Year Award: Outstanding research contributions of 2020, as selected by our Chief Editors. Read the winning articles.