Abstract
Embedded systems have become increasingly connected and communicate with each other, forming large-scaled and complicated network systems. To make their design and testing more reliable and robust, this paper proposes a formal specification language called SENS and a SENS-based automatic test generation tool called TGSENS. Our approach is summarized as follows: (1) A user describes requirements of target embedded network systems by logical property-based constraints using SENS. (2) Given SENS specifications, test cases are automatically generated using a SAT-based solver. Filtering mechanisms to select efficient test cases are also available in our tool. (3) In addition, given a testing goal by the user, test sequences are automatically extracted from exhaustive test cases. We’ve implemented our approach and conducted several experiments on practical case studies. Through the experiments, we confirmed the efficiency of our approach in design and test generation of real embedded air-conditioning network systems.
1. Introduction
An embedded network system (ENS) is a system consisting of a number of embedded units that communicate with each other. Nowadays, ENSs are growing rapidly and playing an important role in our daily life, such as in-vehicle networking systems, home appliance networks, and building energy management systems. On the other hand, because of the increasing complexity of functionality and growing network scales, as well as the increasing numbers and types of embedded units in ENSs, development of the ENSs has been becoming more and more difficult. Particularly, design and test generation are ones of the most important and cost-consuming phases in reliable ENS development, but many companies still rely on the traditional techniques where system requirements are described in a natural language and test cases are generated based on human experience. However, design and test by manpower may not sufficiently cover enormous numbers of combination to be considered especially for complicated ENSs.
During the last decade, numerous formal methods have been proposed to improve software quality. Especially, test generation is an attractive application for mechanized formal methods; the importance of good test cases is universally recognized and so is the high cost of generating them by hand. One of these methods, the model-based test generation, has been extensively studied (e.g., see [1–4]). One of the advantages of this technique is that each behavior of the system model is described directly, and then test cases can be automatically generated from the models. Note, however, that if test cases are generated from incomplete specifications and models, they suffer from the problem of lacking and low quality of test cases. Therefore, formal specification based modeling and test generation for large-scale ENSs are still challenges.
This paper presents our works gained in a joint project of , , and . The ultimate goal of our project is to develop a robust and reliable automatic test generation tool suitable for a development process of air-conditioning network systems developed by . (We refer to the company as Company in the rest of this paper.) The target system consists of a number of embedded controllers and indoor/outdoor air-conditioning equipment for building management. For this project, we have taken an approach of automatic test generation based on a formal specification. In the following, we present three main contributions toward our overall goal.(i)Development of a Formal Specification Language. First, we have developed a specification language for ENSs called (Specification language for Embedded Network Systems). is designed under the concept of test-oriented, object-oriented, modular, and lightweight description of ENSs. In system, the requirements are specified using logical constraints. This logical property rigorously determines a set of state transitions that satisfy all constraints and thus helps to get the specification without lacking and inconsistency and clearly obtain an advantage over a manpower design of system and test.(ii)Development of a Test Case Generator. Second, we have developed a tool called (Test Generator based on SENS) which automatically generates high quality test cases from a given specification written in SENS. Our tool can find distinct and enormous test cases quickly that satisfy complex specifications using a SAT solver as a core engine. Our tool can also perform filtering to select efficient test cases. In a feasibility study using air-conditioning systems of Company D, given a specification for a certain function of the system, our tool took only about 12 seconds to generate 77,700 test cases. We expect that our tool will provide both quality improvement and cost saving for testing target ENSs.(iii)Development of a Test Sequence Generator. Third, we have also developed a tool in TGSENS to generate test sequences that satisfy a given testing goal by a user. In an experiment using components of the target air-conditioning systems, our tool generated 156 test sequences in 8 minutes for a given testing goal such that the sequence contains 2 milestone states whose distance is less than 3 state transitions. We confirmed that our tool is helpful for test engineers to design test scenarios efficiently.
The rest of this paper is organized as follows. In the next section, we briefly introduce the overview of our work and our tool. Section 3 describes the features, syntax, and semantics of the SENS language. In Section 4, we propose the method for generating test cases and sequences from SENS specifications. Section 5 presents experimental results when applying our tool to the air-conditioning systems of Company D. Section 6 discusses related works. Finally, we conclude our work and mention future works in Section 7.
2. Overview of Our Work
Our target system is an embedded network system (ENS). An ENS can be considered as a concurrent system consisting of multiple subsystems with global environments, for example, embedded indoor and outdoor equipment and controllers in an air-conditioning network system as illustrated in Figure 1. Each subsystem in an ENS is an event-triggered system whose events are communications with other subsystems and whose operations are constrained by conditions related to its environment and time.
The behaviors of an ENS are specified in a hierarchical way: an ENS consists of subsystems, a subsystem is constructed from its functions, and each function is constructed from a set of functional requirements, as described in Figure 1. To enhance the reliability and efficiency of design and testing ENSs, we first focus on designing and modeling a formal specification language for ENSs, called , which supports the hierarchy of ENSs and is convenient for our test generation purpose. We next achieve an automatic test generation approach based on , called . Figure 2 illustrates the overview of our tool that consists of the following 5 steps.(1)Specifying Requirements in SENS. A user describes system requirements of the target ENS using . For the air-conditioning systems of Company D, we have translated original specification documents written in a natural language to formal specifications in .(2)Translating SENS Specifications to CNF. The translator first performs the syntax and type checking for the given specification. Next, the translator automatically transforms logical requirements of the specification to CNF (conjunctive normal form) formulas according to our translation rules.(3)Solving CNF Formulas Using a SAT Solver. The CNF formulas obtained in Step are imported as an input of the solver. The solver automatically finds all possible assignments that satisfy the input CNF formulas by iteratively using a SAT solver.(4)Mapping from Assignments to Test Cases. The set of possible assignments obtained in Step are automatically mapped to the set of state transitions corresponding to one-step test cases.(5)Generating Test Sequences under a Testing Goal. When a user gives a testing goal including remarkable states and lengths of sequences, a set of test sequences are automatically derived from the test cases (state transitions) obtained in Step .
3. SENS Specification
In this section, we present a compact formal language called to specify requirements of embedded network systems (ENSs). In Section 3.1, we present features of . In Sections 3.2 and 3.3, we briefly describe the syntax and semantics of .
3.1. Features of SENS
The specification is basically a set of requirements represented by logical formulas. is based on a quantifier-free predicate modal logic with a next operator where predicate variables are classified into environments, systems, communication channels, and timer variables. has the following features.
3.1.1. Property-Based
Property-based logical formulas represent requirements which constrain system behaviors. Since a part of the specification less affects the entire specification, it is easy to modify and reuse specifications. A requirement is described in a traditional Hoare triple-like way: where (resp., ) denotes a logical constraint holding in a prestate (resp., poststate) and an (resp., ) represents a message receiving (resp., sending) between subsystems. This 3-tuple requirement needs to hold for all transitions of the system model, and we call this 3-tuple requirement a transition requirement.
3.1.2. Test-Oriented
For a testing purpose, specifying tested values and bounds of variables is available in the variable declaration. This supports a test-oriented design and an efficient test generation from specifications.
3.1.3. Object-Oriented
A group of subsystems is represented as a parameterized class, and each subsystem is defined as an instance of the class in the main class representing the whole system. This representation leads to an efficient description of a large number of subsystems in an ENS.
3.1.4. Modular
Specifications of an ENS, a subsystem, and a function are broken down into specifications of subsystems, functions, and transition requirements, respectively. This modular construction makes it easy to specify a large-scaled and complicated ENS piece by piece and provide a reusability of specifications. This also allows a step-wise test generation from specifications for unit testing, integration testing, and system testing.
3.1.5. Lightweight Description of Time Constraints
We consider a timer as a component of a subsystem and then specify time constraints using events and actions with the timer. provides a timer variable which is declared with a time-value, a time-unit, and a condition formula for counting up the timer.
3.2. Syntax of SENS
Here, we give the brief explanation of description using the following example embedded system.
Example 1. Consider an illustrating system in Figure 3. There are 2 kinds of equipment, namely, classes and . Let us consider a simple ENS consisting of 3 subsystems , , and , where is an instance of and and are instances of . A subsystem of class sends a message after 2 minutes from starting its operation. Each subsystem of class turns on its lamp when receiving message . This system can be described in the specification of Pseudocode 1.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The declarations consist of the following declarations (refer to Appendix A for the full syntax of SENS).
(i) Main Class Declaration. File <main.cls> specifies the main class. The main class represents the whole system consisting of subsystems and the environment. In the file <main.cls> of Pseudocode 1, the classes and are imported in line 1, and subsystems , , and are instantiated in lines 6-7. Environment variables are declared in the Var part. Data types, environment variables, and channels declared in the main class are referred in all class declarations. Invariants (logical expressions) for the entire system are declared in the main class. In the example <main.cls>, global data type onoff_status is declared in lines 3-4.
(ii) Class Declaration. Files <A.cls> and <B.cls> of Pseudocode 1 specify classes and , respectively. class other than the main class specifies a class of subsystems. Local data types, variables, channels, and timers for the subsystem are declared in the class declaration. In <A.cls>, local variable op and timer t are declared in lines 6–9, and a function of the subsystem is declared in lines 11–14.
For data types, variables, channels, and timers externally declared and internally used, the files to declare them are imported with the Import statement and their names are described in the Extern part. The main class file is imported to all class files by default. A class can have parameters used as constants in the class specification. In line 1 of <A.cls>, class is imported since class is used in the parameter declaration of . In lines 3-4, data type onoff_status is described in the Extern part.
(iii) Variable and Data Type Declarations. A variable is declared with its name, type, initial value, and test declaration (values and bounds for testing) in the part following to Var. Data types are int and double with ranges, and enumerated data types are defined in the Data part. In the Data part, a type is defined as a set of values. In , only finite and discrete values are used. In the example <main.cls>, a global data type, onoff_status, which has two elements on and off, is declared in lines 3-4. In <B.cls>, a local data type, m_type, which has the only element m, is declared in lines 5-6. Using the test declaration beginning with the keyword testedwith, one can define the specific values of variables to be considered in a testing phase. For example, consider the following variable declaration: Var v1 double -2, 5 {0} testedwith {-2, 0..5 (0.5)};
variable v1 is defined as follows: its type is double, its range is −2 ≤ v1 ≤ 5, its initial value 0, and its tested values are −2 and a number from 0 to 5 in increments of 0.5. In our semantics of , each variable of a subsystem takes one value, that is, an element of its domain. If the declaration of a variable has a test declaration, the domain is defined by the test declaration. Otherwise, the domain is defined by the data type of the variable.
(iv) Channel and Timer Declarations. Channels are used to describe communications between subsystems. A channel is declared with its name and message type. In lines 10-11 of <B.cls>, a channel ch for receiving message m is declared. A timer is declared with its name, time-value, unit, and conditions (logical formula) for counting up. When the condition is omitted, the condition is always true. For example, consider the following timer declarations: Timer t1 5 min; t2 5 hour [(v==dry) ∣∣ (v==cool)];
Timer t1 is declared to count up 5 minutes. Timer t2 is declared to count up 5 hours, but the counting is performed only when the value of v is dry or cool.
The event is a message receiving (channel-name?expr) or timeout (timer-name?ring). Similarly, the action is a message sending (channel-name!expr), timer starting (timer-name!start), or timer reset (timer-name!clear). When specifying a communication between subsystems, a message sending (action) is described using both a recipient's object name and a channel name, while a message receiving (event) is described using only a channel name.
(v) Function and Requirement Declarations. Functions are declared in the class declaration or in individual function declaration files. In the function declaration, constants, data types, variables, and timers can be defined locally. A function contains a set of requirements.
A requirement is an expression representing an invariant or a 3-tuple of a precondition, an event, and a postcondition or/with/without actions. (More precisely, the third part of the transition requirement is a postcondition or an action or actions or a postcondition with an action or a postcondition with actions.) We call the 3-tuple requirement a transition requirement. The BNF description for a transition requirement is given in Algorithm 1 (Refer to Appendix A.4).
| ||||||||||||||
The precondition and postcondition are logical expressions. In the transition requirement declaration, the event part can contain no more than one event (it can also be null which is a symbol showing that no event occurs), while the action part can contain multiple actions. We assume that a transition requirement contains no more than one event and actions related to the same timer.
Expressions are constructed with constants, variables, and operators. Expressions have logical comparison and arithmetic operators that general programming languages such as C have. In addition, expressions in SENS have the logical implication operator (–>) and the previous state operator (~). Especially, expression ~v represents the previous value of v.
3.3. Semantics of SENS
A SENS specification describes requirements of an ENS consisting of subsystems communicating each other. We model the system specified by SENS using a labeled transition system. Figure 4 illustrates our system modeling where time constraints in SENS are modeled using our timer models that will be shown later. First, each instance corresponding to a subsystem in the given specification is modeled. When the class corresponding to the subsystem contains timer variables, we compose the subsystem model and its timer models. Next, an entire ENS is modeled as a parallel composition of all subsystems. The transition system modeled from the SENS specification is used for generating test cases and test sequences.
In Section 3.3.1, we give the modeling of timers. In Section 3.3.2, we give the modeling of a subsystem and explain the composition of the subsystem and the timer. In Section 3.3.3, we explain the composition of subsystems.
3.3.1. Modeling of Timers
A timer is specified in the timer declaration with its timer name, number (time), unit, and conditions for counting up in SENS. We consider two kinds of models and for timers expressed in Figures 5(a) and 5(b). We can use simpler timer model for state elimination, while we can use timer model for considering the relation of a time length between timers (refer to Section 3.3.2.).
| (a) Model |
| (b) Model |
| (c) Model for timers with multiple conditions |
Definition 2 (timer model). We model the behavior of each timer as a labeled transition system with , where denotes a set of states, denotes a set of labels, and denotes a labeled transition for . for and are defined as follows: (i)ready, 0, 1, , max, over} in ;(ii)ready, run, over} in ;(iii)?start, ?clear, !ring, c} in , .
In models and , states ready and over, respectively, represent states in which timer is ready and counted over. In model , state “” (0 ≤ ≤ max) represents a state in which timer has counted up times. In model , we reduce states 0, …, max to state run representing a state in which timer is running, for the state elimination.
In both models, each state is changed to state ready when receiving message clear. State ready is changed to state 0 in (resp., state run in ) when receiving message start. State max in (resp., state run in ) is changed to state over when sending message ring. We define that transitions labeled with and c occur only if the condition for the timer holds. If the condition is not described in the timer declaration, the condition is considered to be “true”.
If there are multiple conditions , we define transitions as represented in Figure 5(c). In the figure, transitions labeled with ?clear are eliminated for simplicity. In the figure, the first, second, and third transitions labeled with c occur when conditions , , and , respectively, hold.
3.3.2. Modeling of Subsystems
A subsystem is specified in the class declaration in . For a given specification, we model the behavior of each subsystem as a labeled transition system in two steps. First, the behavior of a subsystem except timers is modeled as a labeled transition system . Next, the behavior of a subsystem including timers is modeled as which is a composition of and models of the timers.
Definition 3 (subsystem model). We model the behavior of a subsystem , except for its timers, as with . Given a SENS specification, the set of states , the set of labels , and the set of transitions are obtained by Definitions 15, 16, and 17 of Appendix B.1.
Example 4. Consider the example specification of subsystem in Pseudocode 1. Figure 6 illustrates the subsystem model for which satisfies the specification. Given SENS specification for class , the states are (~op=off, op=off), (~op=off, op=on), (~op=on,op=on), and (~op=on, op=off) from variable op and previous state variable ~op used in the specification by Definition 15. By Definition 16, the labels are t!start, t?ring, B[1]. ch!m, and B[2]. ch!m with the events and actions used in the specification. Each transition in satisfies the two conditions in Definition 17. For example, with (~op=off,op=off) and (~op=on,op=off) cannot be a transition of , since it does not hold the condition for previous state variables; that is, ⊨ (op=on), if and only if ⊨ (~op=on) by Definition 17.
We model the behavior of a subsystem with its timers as an interleaving composition of the subsystem model and timer models as follows: where denote the subsystem model explained in Definition 3 and denotes the model for timer () explained in Section 3.3.1. We inductively define the above model composition, that is, (1) in Definition 18,(2) in Definition 19, where with . And the composition rule is defined in Definitions 18 and 19 of Appendix B.2.
Example 5. Consider again the example specification of subsystem in Pseudocode 1. Figure 7 shows a part of the composed model of in Figure 6 and timer model for timer . The transitions labeled with start and ring correspond to rules (a1) and (a2) in Definition 18, that is, message synchronization. The transition labeled with c corresponds to rule (a4) in Definition 18.
3.3.3. System Model
Now we explain the model for the entire system. Assume that the entire system consists of subsystems. We define the model for the system as the parallel synchronized composition of labeled transition systems representing all subsystems (). The composition of two subsystems is defined in Definition 20 and the entire system model is obtained by an inductive composition. (When a transition of a subsystem has multiple message sending, we assume that its composition is synchronously performed in our modeling.) The composition rule is defined in the definition of Appendix B.3.
Example 6. Consider the example specifications of subsystems and in Pseudocode 1. Figure 8 shows an example composition of transitions in and .
4. Automatic Test Generation from the SENS Specification
In this section, we present the methods for a given specification to automatically generate test cases and test sequences in Sections 4.1 and 4.2, respectively.
4.1. Test Case Generation
4.1.1. Concept
In our approach, we find all test cases by using a SAT solver. A SAT solver is a tool to solve the SAT (satisfiability) problem which determines whether there is a value assignment to satisfy a given Boolean formula in CNF (conjunctive normal form). Although this problem is an NP-complete problem, there have been very high-speed SAT solvers (e.g., MiniSAT [5]) for practical sized problems. Using a SAT solver, we can find distinct and enormous test cases quickly that satisfy specifications.
4.1.2. Test Cases
Given a specification that describes a target system, every transition of the system model satisfies all requirements in the given specification, as mentioned in the previous section. Therefore, we can consider that each transition of the system model corresponds to a one-step test case. That is, the prestate (resp., events) of is a valid input state (resp., system inputs) of a test case and the poststate (resp., action) of is a valid output state (resp., system output) of the test case. We formally define a test case as follows.
Definition 7 (test case). Consider a given specification . A test case is a tuple, corresponding to a transition of the system model such that . In other words, a test case is an assignment that makes every transition requirement of hold.
Note that, if the given specification is for a subsystem module, subsystems, and an entire ENS, derived test cases are, respectively, used for unit testing, integration testing, and system testing.
4.1.3. Translation of SENS to CNF
For generating test cases, we first translate all the requirements in the given specification to a CNF formula which becomes an input of a SAT solver. Since the specification is a set of logical constraint formulas, we can translate it to CNF according to the semantics of .
There are 3 main steps of the translation procedure as follows. First, we introduce fresh propositional variables to construct the state space of the system model. Second, transition requirements and invariants in the specification are translated into logical formulas. Third, we collect all of the constraints presented in logical formulas. The constraints are connected with conjunction and the result is converted to CNF. (Every propositional logic formula can be converted into an equivalent formula, that is, in CNF. However, by naive algorithms where the transformation is based on rules of De Morgan's laws and the distributive law, the size of CNF can be exponentially increased. To get a compact CNF formula, we used the Tseitin-transformation [6] after translating SENS to a propositional logic formula.) Due to space limitation, the details of all translation rules are given in Appendix C.
4.1.4. Generating Test Cases
Once we obtain the CNF formula from the given specification, we apply a SAT solver to the CNF formula to find an assignment. As mentioned above, the assignment that makes requirements of the specification true corresponds to a test case.
Algorithm 2 describes our method for generating test cases. We use a SAT solver iteratively to find all possible assignments and convert them to test cases that cover all possible transitions of the system model.
| ||||||||||||||||||||||||||||||
4.1.5. Filtering
The proposed method can generate all possible test cases with 100% coverage for requirements of a given specification. On the other hand, the number of complete test cases can be very huge. In order to extract efficient test cases, we also give several filtering mechanisms in our method. In the current version of our tool, a user can select test cases satisfying a precondition of transition requirements, test cases containing a communication label, and test cases whose timer label is ring, which means the test cases related to a time constraint.
Example 8. Consider again the example specification of subsystem in Pseudocode 1. Each transition of the model for system in Figure 7 corresponds to a test case. Especially, ((~op=off,op=on), , {t!start}, (~op=on,op=on)) correspond to a test case satisfying a precondition of the transition requirement <~op==off && op==on:null:t!start> in line 13 of the specification. On the other hand, ((~op=on,op=on), {t?ring}, {C_B[1].ch!m,C_B[2].ch!m}, (~op=on,op=off)) correspond to a test case containing timer label ring and communication labels.
As shown in Table 3, the number of all test cases for is 176. Among them, test cases satisfying a precondition are 10, which is less than 6% of the entire test cases. Moreover, among them, 2 test cases can be extracted, respectively, with a communication label and timer label ring.
4.2. Test Sequence Generation
4.2.1. Concept
As mentioned in Section 4.1, our test generator can generate all possible test cases for a given specification. Since the number of complete test cases can be very huge, it is hard to run tests by inputting all the test cases separately. Therefore, it is practical to extract test sequences and focus on specific variables interested for testing. To deal with this problem, we propose a test sequence generator for appropriate testing goals given by users.
4.2.2. Test Sequences and Testing Goals
In our method, we generate test sequences from two inputs: given a specification and a testing goal. First, we define a test sequence as follows.
Definition 9 (test sequence). A test sequence is a sequence of test cases, whose adjacent output and input are the same. This means that a test sequence is a sequence, where , for .
Next, a “testing goal” is a key feature that helps to generate appropriate test sequences. This also allows us to reduce the size of space and time for searching test sequences. In the current version of our method, a user can specify the following 3 features as a testing goal: “key variables” for a testing purpose, a list of “milestone” states forming a backbone of the sequence, and a “maximum length” between two adjacent milestones. We formally define a testing goal as follows.
Definition 10 (testing goals). Given a specification, a testing goal , where is a set of key variables, is a list of milestone states, and is a list of maximum lengths such that (1) “key variable” is a variable that we focus on testing in the given specification;(2) “milestone” is a state that we want each test sequence to pass through;(3) “maximum length,” max of a run from a milestone to the next milestone , denoted by , is the maximum number of transitions needed for the run.
We say that a test sequence satisfies a testing goal , if and only if (1) the first input state and the last output state of the test sequence, respectively, are equal to the first milestone and the last milestone , (2) all other milestone states exist in the input states of the test sequence in order, and (3) the length between the milestone states in the test sequence is less than the maximum length defined in . In addition, value assignments only for the key variables appear in the generated test sequences satisfying the testing goal.
Example 11. If the list of milestones is , then we can specify the maximum length in total as the , where is the maximum of transitions which can be taken to go through from the milestone to the milestone , and is the maximum of transitions which can be taken to go through from the milestone to the milestone .
Let us consider the simple example specification of system in Pseudocode 1. Assume a testing goal whose key variables = {op,lamp}, milestones = 〈 = (op=off,lamp=off), = (op=on,lamp=on) 〉, and . Then, 〈((op=off,lamp=off), , ∅, (op=on,lamp=off)),((op=on,lamp=off), ch?m, ∅, (op=on,lamp=on))〉is a test sequence satisfying the testing goal.
4.2.3. Generating Test Sequences
Our approach for test sequence generation has three main steps. First, from a given specification, we first generate all test cases by our method in Algorithm 2. Then, the obtained test cases correspond to all transitions of the target system model.
Second, we synthesize all the transitions into a transition system (graph). Each state of this transition system is a tuple of variable assignments. Each edge is a transition relation from a prestate to a poststate, and a label of each edge is an event that makes the transition occur. The transition system of the specification is stored in a file, while the testing goal is stored in a separate file.
Third, given the transition system and the testing goal in two files, we compute all possible desired test sequences that satisfy the testing goal; that is, we search the set of sequences starting from a given milestone state to another milestone state whose distance is less than the given maximum length by tree searching algorithms including exhaustive BFS (breath first search) and DFS (depth first search) algorithms. Each resulting test sequence is treated as a test scenario. In our test sequence generation, we reduce the searching state space of the system model by an equivalence partitioning based on the key variables of the given testing goal and search the sequences containing milestone states.
In our , test sequences can be generated from both the original transition system and the filtered test cases. In the test sequence generation from the original transition system, the obtained test sequences obviously keep the transition relation of the original system model. In the test sequence generation from the filtered test cases, test sequences including transitions eliminated by filtering are not generated. However, since the filtering is defined by formal rules, the effect by filtering is clear for users who recognize that test sequences with filtering options will be obtained. For this reason, we believe that it does not affect the main results of our research, in other words, generations of reliable test cases/sequences.
5. Implementation and Experiments
5.1. Implementation
We implemented our approach of -based automatic test generation in the tool called . The tool consists of the following four parts as illustrated in Figure 2. (1) The first part translates the specification written in into a CNF formula. (2) The second part searches and collects all assignments that satisfy the CNF formula by using a SAT solver as its core engine. (3) The third part translates the obtained assignments into test cases which are expressed with variables used in the original specification. (4) The forth part searches the test sequences from the output file of the third part and given testing goals. The resulting test sequences can be viewed by two ways: a textual format or a graphical figure using a dot file viewer.
Currently, for the function level (corresponding to the function declaration in ) and the subsystem level (corresponding to the class declaration in ), we have completed the implementation of the first part in but not yet for the system level. That means that the implementation for the composition of subsystems for the entire system level is our future work. Hence, the current version of our tool can perform the test generation for unit testing.
Our tool was implemented on Windows XP for both ×86 and ×64 environments. The first, third, and forth parts were implemented with a Java language. The second part was implemented based on MiniSAT [5] with C++. The size of the implemented programs is totally about 20,000 lines, respectively, , , 500, and 340 lines for each part.
We also implemented the user interface (in Japanese) of given in Figure 9. Given an input specification, our tool has execution options to generate output files of exhaustive test cases, filtered test cases, and test sequences under a given testing goal. also has a function to check a reachability of states and generate path sequences among the input states.
(a) The main interface for generating test cases.
(b) The interface for generating test sequences.
5.2. Experiments: Specification in SENS
We applied to specify system requirements of real air-conditioning systems by Company D. In general, there are specifications with different levels. In Company D, specifications are divided into “instruction manuals,” “functional specifications,” and “detailed specifications,” and each specification is broken into functions. For example, as to specifications of certain indoor equipment, an instruction manual consists of about 10 functions with 30 pages, and a functional specification and a detailed specification, respectively, consist of about 80 functions with 400 pages.
We translated several specifications in a natural language with tables and figures to specifications. The target specifications were for two modules and in an instruction manual and one function in a detailed specification of an indoor equipment, one function in a functional specification of an air-cleaning system, and three functions , , and in a functional specification of a controller.
For all specifications, this translation needed 1 or 2 days by one person who is one of our developers. Almost all (70–80%) of the translation time was for understanding the real specifications and confirming whether the specifications are valid with industrial engineers, while the rest of the time (several hours) was for the actual description. In addition, we held a one-day seminar for industrial engineers to explain how to describe the specifications and we confirmed that after the one-day seminar, the engineers were easily able to use to specify embedded systems by themselves.
Example 12. We show a part of an example specification for function in Pseudocode 2. is a mode change function for an air-cleaning system by Company D. The original specification of this function was written in Japanese with sized 4 pages and includes 5 tables and 4 transition diagrams. We transformed this document into about 100 lines of a specification. The specification consists of 8 variables (including 6 global variables, 1 local variable, and 1 channel), 20 invariants, and 8 transition specifications. Lines 16–18 specify invariants and lines 20–22 specify transition specifications. All 8 variables have discrete values and the details of variables and their ranges are given in Table 1.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Table 2 shows the scale of the specifications for function : the number of lines except comments and blank lines, the number of declared variables and channels, and the number of requirements including invariants and transition requirements. As to the specification of , the size of requirements was relatively long since variables of the array type with a big size are used. In the future, our description could be more compact by preparing for-statement or while-statement descriptions. For comparison, we also show the number of pages of the original specifications in a natural language in Table 2.
Although we, here, showed the experimental results only for air-conditioning systems, we also have other case studies applying to specify embedded systems such as an electric pot and an automatic dispenser. For instance, we applied to describe a requirement specification for an electric pot (http://www.sessame.jp/workinggroup/WorkingGroup2/POT_Specification.htm), which has been openly used as an example embedded system by SESSAME (http://www.sessame.jp) (Society of Embedded Software Skill Acquisition for Managers and Engineers) of Japan. The original requirement specification was written in Japanese including state transition diagrams with 14 pages and consisting of 14 functions. We described approximately 1,100 lines of the SENS specifications for all the functions, which include 28 variables, 5 timers, and 7 channels in the class specification and totally 386 requirements for all function specifications. Through our case studies, we confirmed that is efficiently applicable to not only the air-conditioning systems but also other general embedded systems.
In addition, by describing the specifications in , we found several undefined and under-specified requirements in the original specifications for the target air-conditioning system. Through the feasibility study, we could confirm both the description ability and facility of . Engineers in Company D plan to use our tool in real development processes. We anticipate that using increases the quality of specifications by preventing the undefinition, the ambiguity, and the inconsistency in the specifications.
5.3. Experiments: Test Generation
In this section, we describe several case studies of generating test cases and test sequences from given example specifications <A.cls> and <B.cls> in Pseudocode 1 and , , , and , using our tool . is a specification whose variable range is reduced from that of for testing. The experiment was performed in the following environments: Intel Core 2 Duo P8600 2.4 GHz, 3.0 GB Memory, Microsoft Windows XP SP3, and Sun Java Runtime Environment 1.6.0_24.
Table 3 describes the result of applying to test case generation. In the table, we show the size of specifications, the size of CNF, the number of test cases, and the execution times to generate test cases for each part of . For the size of a SENS specification, we represent the number of states and the number of transition labels in the table. Here, the number of states indicates the one of the state space constructed by all possible variable assignments in the specification. The number of transition labels indicates the number of labels in the system model constructed from the specification.
Example 13. Consider the case of . The number of possible states (including states not satisfying the specification) for is 68,040 from Table 1. The number of transition labels is 2 as shown in Table 3. Then the number of all possible state transitions (those that do not satisfy the specification are included) is 68,048,04. From such a large number of transitions, figured out 77,700 transitions satisfying the given specification as test cases within 12 seconds. For instance, as for the filtering option of including a communication label, selected 8,100 test cases (≈10.5% of all the test cases) within 1 second.
For (resp., ), took only about 26 (resp., 1) minutes to generate approximately 900,000 (resp., 160,000) test cases. For example, the specification of has 13 variables including 3 previous state variables and those value ranges are 3, 5, 5, 4, 5, 5, 2, , , 3, 3, 3, and 3. Then, the number of states for is 1,74,535,0. On the other hand, the numbers of states for and are 45,000 and 68,040, respectively. has a feature of including variables whose range are large, like , and thus the number of states is excessively large compared to those of other specifications. The execution result of indicates that the specification has 1,744,535,0 states and 30 transition labels. Then, the number of all possible state transitions for is 1,744,535,01,744,535,0. Since the number of state transitions for is much larger than those for and , the CNF size for is much larger than those for and . Accordingly, needed much execution time to search test cases. Although, with such a huge amount of state space, it is very impossible to manually check all cases by human, can correctly do that and takes only about 26 minutes.
We next describe the result of applying to test sequence generation. Table 4 shows the results using the specification for function with 3 example testing goals, G1–G3. In the table, for each testing goal, we show the number of milestone states, the maximum lengths between milestones, the number of key variables of the given testing goal, the number of test sequences, and the execution time to generate test sequences.
Example 14. Pseudocode 3 shows the input file to specify the testing goal G3 in . We can also set the testing goal directly in the GUI interface of shown in Figure 9(b). The testing goal G3 specifies the following: (i)key variables (declared in lines 25–28): three variables Mode_A, Mode_B, and AV in the specification.(ii)The list of milestones (declared in lines 4–19): , where is a state in which Mode_A=Auto, is a state in which Mode_A=Off, and is a state in which Mode_A=Auto.(iii)maximum lengths between milestones (declared in lines 22-23): and .
When the specification for and the testing goal G3 are given, a transition system of this function is first obtained from the result of test case generation. Then, automatically searches desired test sequences satisfying the testing goal from the transition system. Pseudocode 4 shows a part of the generated test sequences.
The resulting transition system for has 1,800 states and 77,700 transitions. From the transition system, figured out 28 (resp., 32) sequences from to (resp., to ) and thus totally 896 (=28 32) test sequences satisfying the testing goal only in 3 minutes.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||
6. Related Work
Model-based testing technique has been an attractive research area in software engineering field recently [7–9]. Model-based testing is a technique that derives a desired test case from a model or a specification of a system. As modeling makes it possible to refine or check functionality of target systems, we can expect to obtain correct test cases generated from valid models in earlier phases of development. Thus model-based test generation has been attracting the attention of model-driven software development approaches. Model-based testing has two technical issues: modeling target systems appropriately and generating test cases effectively.
As for the modeling, many techniques have been proposed, even if we focus on formal modeling only. Models are described based on contracts, abstract data types, process algebras, labeled transition systems, data flow, and so on. These concepts are implemented using specification languages in tools like VDM [7], Z [10], B [11], Spec# [12], and JML [13]. With most modeling languages, a user has to specify the systems' behaviors individually.
Test case generations from models have been proposed in several ways. Here, test cases include arguments to execute specific functions, pairs of possible inputs and expected outputs, or event sequences. Test cases are searched by tracing executions of models, selecting random values with filtering, or generating candidates with some algorithms.
Here, we describe previous works on model-based testing. Spec Explorer [9] generates test cases by state exploration based on multiple strategies from contract-like specifications Spec# on a Microsoft. NET framework. In TGV [8] by INRIA, as a part of the CADP (Construction and Analysis of Distributed Processes) package, test graphs are generated from labeled transition systems. Uppaal-Tron [14] generates test sequences for real-time systems using the model checker UPPAAL, and an industrial case study of Uppaal-Tron targeted an electronic refrigerator controller. T-VEC [15] is a coverage-based test generation tool of Simulink/MathWorks which has been used to identify a fault in the Mars Polar Lander.
Some model-based test generators use constraints solving to generate test cases. In [16], a test generator based on autofocus is proposed. Autofocus [17] is a graphical tool for model-based development of distributed systems. By translating a specification into a constraint logic program, this generator can deal with all the possible execution traces of the model. TestEra [18] is a test generation framework for Java programs. By considering pre-/postconditions for executing methods as constraints, TestEra generates test cases with SAT-solvers. Exhaustive generation is available through certain options.
Most previous studies targeted the testing of programs or protocols. Although TGV has case studies for small-sized embedded applications, there are only a few studies on the large-scale embedded network systems targeted in our proposed project. Different from other works, we have proposed the original specification language, SENS, focusing on property-based specifications and the tool to automatically generate test cases and sequences focusing on testing goals for embedded network systems.
7. Conclusion
In this paper, we have proposed a method to automatically generate test cases and sequences based on the specification of embedded networked systems (ENSs). We first proposed a formal specification language SENS whose features are property-based, test-oriented, object-oriented, modular description, and lightweight description of time constraints. By the “property-based” feature such that requirements are property-based constraints holding in the entire ENS, logical properties for the system specification are clear and modifications and reuse of the specifications are easy since a part of specifications less affect the entire specifications. Thus, SENS has an advantage on the efficient description and reuse of specifications for large and complicated ENSs. The “object-oriented” and “modular” feature enables SENS to structurally describe requirement specifications of ENSs consisting of a number of embedded subsystems/equipment. The “lightweight” feature on description of time constraints leads to readily specify embedded systems sensitive to time requirements. In addition, the “test-oriented” feature is helpful for automatic test generation from SENS specifications. To the best of our knowledge, there has been no other formal specification language having all the above features for ENSs. Using SENS, we can efficiently describe the requirements of ENSs.
We next proposed the methods for automatic test generation in the tool whose features are summarized as follows: takes specifications of an ENS written in SENS as its input and automatically generates test cases and test sequences; the test case generation problem is translated into a satisfiability problem, and test cases are generated by iteratively using a SAT solver; the test sequences are also automatically obtained based on a testing goal, that is, key variables, milestone states, and a desired maximum length of test sequences, given by a user. We finally applied to generate tests for the real air-conditioning systems, and our tool rapidly succeeded in generating about 900,000 test cases in 30 minutes and extract 28 test sequences in 3 minutes.
We expect that using can improve the quality of test cases and thus enhance the quality of ENSs. Previously in industrial fields, test cases have often been designed in a manual way. Such the test design depending on testing engineers’ skills seldom covers all the requirements of the complicated ENSs. On the other hand, can automatically and mechanically generate test cases from given formal specifications. Both the description of formal specifications and the automatic test generation lead to improving the quality of test cases. Actually, we found several undefined and under-specified requirements by describing the SENS specifications for real industrial embedded systems as noted in Section 5.2. In addition, can generate test cases with 100% coverage for requirements of the given specification by an exhaustive search using a SAT-based solver. also supports feasible test generation by user-specified test filtering and test sequence generation.
The successive work of our project is the improvement of our test generator in order to deal with the composition of subsystems. One of the most important aspects of test generation based on formal specifications and models for industrial systems is “scalability.” To overcome this problem, we plan to tackle a higher-speed and more scalable test generation using parallel computing. We also plan to find a constructive way of generating test suites of the whole system from test suites of subsystems as a future work. We also plan to apply the model checking technique to generate more target-specific test cases and sequences.
Appendices
A. Syntax of SENS
A.1. Main Class Declaration
See Pseudocode 5.
| ||||||||||||||||||||||||||||||||||||||
A.2. Class Declaration
See Pseudocode 6.
| ||||||||||||||||||||||||||||
A.3. Variable, Channel, and Timer Declarations
See Pseudocode 7.
| ||||||||||||||||||||||||||||||||||
A.4. Function and Requirement Declarations
See Pseudocode 8.
| ||||||||||||||||||||||||||
A.5. Expression
See Pseudocode 9.
| ||||||||
B. Semantics of SENS
B.1. Modeling of Subsystems
States For . Let denote a set of all variables used in the SENS specification for the subsystem . That is, is a set of all variables declared in the variable declaration of the class declaration for , and the extern declaration of the class-file-declaration for . For each variable (), let denote a set of all values of declared in the specification. When considering a test model, let be a set of all values in the test declaration for .
In addition, we also define a set of variables with previous state operator “.” Let denote a set of every variable whose previous state variable is used in the specification.
Definition 15 (states of ). is a set of all states such that (i) is an -tuple of value assignments for all variables in and ; that is, , , ;(ii) satisfies all conditions specified as invariants in the requirement declaration of the class declaration for and the invariant declaration.
Hereafter we denote if the value for is assigned to at state . Otherwise and thus since each variable is assigned to one value at the same time.
Labels For . Let be a set of all events and actions declared in the class declaration and the class-file-declaration for subsystem . There are three kinds of labels: input labels corresponding events, output labels corresponding actions, and label .(i)Input labels are channel-name?expr and timer-name?ring and represent message receiving.(ii)Output labels are channel-name!expr, timer-name!start, and timer-name!clear and represent message sending.(iii)Label means an internal transition with no message receiving and sending.
Definition 16 (labels of ). is a set of labels such that
where denotes a set of input labels in .
Each is a set of labels in a transition, and we call a transition label. Note that each transition label can contain at most one input label and multiple output labels by the syntax of . Transition label in corresponds to .
Transitions For . Consider a transition requirement where is precondition, is event, is postcondition, and is actions. We say that a transition satisfies a transition requirement , if and only if the following holds: that is, if state satisfies the precondition and event is an input label of , then next state should satisfy the postcondition and actions should be output labels of . Hereafter denotes a labeled transition .
Definition 17 (transitions of ). is a set of all that satisfies the following two conditions (we say that such a transition satisfies the given specification): (i)for each state variable , , if and only if ;(ii) satisfies all the transition requirements of the requirement declaration in the class declaration for subsystem .
B.2. Modeling of Subsystems with Timers
Definition 18 (composition of a subsystem and a timer). is defined as follows. Hereafter, let and let denote the condition for timer : (i)states: ;(ii)labels: ;(iii)transitions: for each and , the following transitions exist:(a1), if and ;(a2), if and ;(a3), if and ;(a4), if , , and ; that is, the condition for timer holds at state .
The transitions of rules (a1) and (a2) represent the composition of message synchronization. The transitions of rules (a3) and (a4) represent the interleaving composition of transitions of subsystem and timer , respectively. By this interleaving composition with a subsystem and a timer, the composed model can cover all combinations of time length in the model.
Definition 19 (composition of a subsystem and timers). with is defined as follows. Note that and denotes the condition for timer : (i)states: ;(ii)labels: ;(iii)transitions: for each and , the following transitions exist:(b1), if and ;(b2), if and ;(b3), if and ;(b4), if , , , and ;(b5), if , , and .
The transitions of rules (b1) and (b2) represent the composition of message synchronization. The transitions of rules (b3) and (b4) represent the interleaving composition of transitions of subsystem and timer , respectively. Transition of rule (b5) represents the synchronization of counting up timers. Therefore, the length relation of timers can be modeled by using timer model in Section 3.3.1.
B.3. Composition of Subsystems
Definition 20 (composition of subsystem models). for subsystems and is defined as follows: (i)states: , where () is the set of states of ();(ii)labels: for each and , where and are, respectively, input and output labels representing receiving and sending massage with communication channel between system and . Here, denotes a label after composing input and output labels with the same name.(iii)transitions: let be a communication channel between and and is not a communication channel between and . and , respectively, denote a set of input labels and output labels in . The following transitions exist for each and :(c1), if ;(c2), if ;(c3), if ;(c4), if ;(c5), if ;(c6), if(i), ,(ii),(iii),(iv);(c7), if(i), ,(ii),(iii),(iv).
The transition of rule (c1) represents the composition of internal transitions labeled with and . The transitions of rules (c2) and (c3) represent the composition of internal transitions labeled with external communication channels. The transitions of rules (c4)–(c7) are for the composition of two transitions including message synchronization. In the composition, we interpret the meaning of a transition with input and output labels of subsystems as illustrated in Figure 10. In the figure, transition corresponds to rule (c4) where . Transition corresponds to rule (c6) where .
C. Translating SENS to Propositional Logic
Here, we explain how to translate given a specification to a propositional logic formula.
C.1. Variable Declaration
For each variable v and its value t in the specification, we introduce two auxiliary propositional variables representing the value of v in the current state as t (denoted by v#t) and the value of v in the next state as t (denoted by @v#t). In addition, if the variable v appears in an expression in the form ~v, then we also introduce two auxiliary propositional variables such as ~v#t and @~v#t. The value t of v is an element in the domain of v; the domain is defined by test declaration or the type of the variable.
All variables of the SENS specification have the following two constraints: (1) a variable must take a value in its domain at any time and (2) a variable cannot take two values at the same time. Moreover, if a variable with ~ operator appears in an expression, the variable has the following constraint: the value of a variable with ~ operator in the next state equals the value of the variable without ~operator in the current state.
In this translation, we need to represent these constraints as logical formulas by using auxiliary propositional variables. For example, consider variable op of <A.cls> in Pseudocode 1. The constraints to variable op are as follows:(i)(op#on ∨ op#off) ∧ (@op#on ∨ @op#off);(ii)(¬op#on ∨ ¬op#off) ∧ (¬@op#on ∨ ¬@op#off);(iii)(@~op#on⇔op#on)∧(@~op#off⇔op#off).
C.2. Timer Declaration
We translate the timer declaration step-by-step considering the three parts: a subsystem side, a timer side, and a composition of the subsystem and the timers.
First, on the subsystem side, we prepare four kinds of propositional variables as follows:(i)SubCountUp: a counting up event occurring in a subsystem is prepared; there exists only one variable of this kind for each subsystem;(ii)t?ring: a subsystem receives a counting over event from its timer t;(iii)t!clear: a subsystem does an action by sending a reset message to its timer t;(iv)t!start: a subsystem does an action by sending a startup message to its timer t.
We assume that no more than one event or action related to the same timer t occurs at the same time. We represent this constraint as follows:(i) .
Next, on the timer side, we introduce auxiliary propositional variables for two events, an action, a counting-up event, and the states of each timer. The propositional variables prepared for timer t are as follows:(i)t?clear/t?start: for reset/start-up event from the subsystem,(ii)t!ring: an action of sending the counting over message,(iii)t?CountUp: a counting-up event of the timer,(iv)t#ready/t#run/t#over: the current timer state,(v)@t#ready/@t#run/@t#over: the next timer state.
Timer t in the subsystem has the following constraints and we represent them using prepared propositional variables as follows:(i)the events and action of timer t do not occur at the same time:(a) ; (ii)at any time, the state of a timer is one of three states, ready, run, and over:(a) @t#ready ∨ @t#run ∨ @t#over), (b)∧(¬t#ready ∨ ¬t#over) ∧(¬t#run ∨ ¬t#over)∧(¬@t#ready∨¬@t#run) ∧(¬@t#ready∨¬@t#over)∧(¬@t#run∨¬@t#over);(iii)the transition relation of the timer models is the same as the one in Figure 5:(a) t?clear→@t#ready, (b) t?start⇔(t#ready ∧ @t#run),(c) t!ring⇔(t#run∧@t#over),(d) t?CountUp→(t#run ∧ @t#run), (e)t?start→@t#ready) ∧¬t?CountUp→@t#run) ∧(@t#over ∧ ¬t?clear→@t#over) ;(iv)when the timer declaration has a condition for counting up, that is, a logical formula cond, the following two formulas are added:(a), (b).
Finally, we describe constraints of composition between a subsystem and timers.(i)Events and actions of a subsystem are associated with actions and events of each timer t:(a); (b);(c).(ii)For timers of each subsystem, all the timers satisfying their counting up conditions count up at the same time:(a) ;(b) ;(c) , where condti is the counting up condition of timer ti.
(iii)The counting up of timers in a subsystem and a change of the values of variables in the subsystem do not occur at the same time. Consider variable op of <A.cls> in Pseudocode 1. This constraint is represented by the following formula:(a) .
C.3. C.3. Channel Declaration
For events through a channel, we prepare propositional variables. A variable denoted by c?m shows that the event of receiving message m through channel c occurs. In addition, for each subsystem, prepare one propositional variable representing that no event occurs (denoted by null#true).
No more than one event occurs at the same time in a subsystem where we consider the counting over of a timer and the counting up of a subsystem as events. We need to represent this constraint of event as logical formulas by using prepared variables. For example, consider the subsystem <A.cls> in Pseudocode 1. The subsystem has two events, a timer-over event and a counting up the timer. So, the constraint of events in <A.cls> is translated as follows:(i); (ii) .
For actions as well as events, we prepare propositional variables. A variable denoted by obj.c!m shows that a subsystem takes the action of sending message m to a district subsystem obj through a receiver's channel c.
There is a constraint that a subsystem cannot take any action, if its counting-up event occurs. For example, consider the subsystem <A.cls>. Since it has two actions, the constraint of actions is expressed by the following formula:(i)(¬arg1.ch!m∨¬SubCountUp) ∧(¬arg2.ch!m∨¬SubCountUp).
C.4. C.4. Requirements
Basically, in the translation of requirements of a specification, we only have to replace the requirements with logical formulas using the propositional variables prepared above. A requirement represented as a logical expression in should be satisfied both in the current state and in the next state. Therefore, we translate the requirement into two logical formulas: one for the current state and the other for the next state.
The transition requirement of SENS specifications, “P: e: (Q, a),” represents a logical implication: “(P∧e) → (Q∧a).” The postcondition Q is a constraint for the next state, so we translate the condition into a logical formula using propositional variables for the next state.
For example, consider the two requirements of <B.cls> in Pseudocode 1. They are translated as follows:(i)(op#on∧lamp#off∧ch?m)→(@lamp#on);(ii)op#off→lamp#off, @op#off→@lamp#off.
All the variables of including variables of numerical types take only a finite number of values. We start explaining the translation using the following examples,
Var a: int 0,2]; b: int 0,2];(i)comparison between a variable of and a numerical value:(a)the equation a==0 is denoted by a propositional variable a#0, and the inequation a!=0 is denoted by a literal ¬a#0;(b)the inequality a<2 is translated to a logical formula a#0 ∨ a#1.(ii)comparison between variables:(a)the equation a==b is translated to a formula (a#0 ∧ b#0)∨(a#1 ∧ b#1) ∨ (a#2 ∧ b#2);(b)the inequation a!=b is translated to a formula ¬((a#0 ∧ b#0) ∨ (a#1 ∧ b#1) ∨ (a#2∧b#2));(c)the inequality a<b is translated to a formula (a#0 ∧ (b#1 ∨ b#2)) ∨ (a#1 ∧ b#2).
For general numerical comparison, we translate the requirement inductively. In particular, let be a general numerical comparison in , where , and are expressions. We transform this requirement to logical formula as follows. First, we introduce two auxiliary variables and ; these variables represent and , respectively. The requirement is reduced to . Second, based on the domain of input variables, we can compute the domain (lower and upper values) of and . Assume that and . The requirement is translated to logical formula by a similar way as in the example above.
Conflict of Interests
The authors declare that they have no conflict of interests.
Acknowledgments
The authors thank Takao Sonoda, Shouichi Hasuike, Munekazu Huruichi, Keiichi Shimatani, and Hayato Shikata of for their great contribution in their joint project. This paper was partly supported by the JSPS-VAST bilateral research project.