Design and Lyapunov Stability Analysis of a Fuzzy Logic Controller for Autonomous Road Following

Abstract

Autonomous road following is one of the major goals in intelligent vehicle applications. The development of an autonomous road following embedded system for intelligent vehicles is the focus of this paper. A fuzzy logic controller (FLC) is designed for vision-based autonomous road following. The stability analysis of this control system is addressed. Lyapunov's direct method is utilized to formulate a class of control laws that guarantee the convergence of the steering error. Certain requirements for the control laws are presented for designers to choose a suitable rule base for the fuzzy controller in order to make the system stable. Stability of the proposed fuzzy controller is guaranteed theoretically and also demonstrated by simulation studies and experiments. Simulations using the model of the four degree of freedom nonholonomic robotic vehicle are conducted to investigate the performance of the fuzzy controller. The proposed fuzzy controller can achieve the desired steering angle and make the robotic vehicle follow the road successfully. Experiments show that the developed intelligent vehicle is able to follow a mocked road autonomously.

1. Introduction

Intelligent transportation systems (ITSs) are an emerging global phenomenon benefiting public and private sectors alike. ITSs and intelligent vehicles can relieve congestion, improve safety, and enhance productivity. The field of intelligent vehicles is rapidly growing worldwide. Technologies involved with intelligent vehicles include sensing and control technologies, communications, and computer informatics. Intelligent vehicles offer the potential to significantly enhance safety and operational efficiency. As one component of ITSs, intelligent vehicles use sensing and intelligent algorithms to understand the vehicle's immediate environment, either assisting the driver or fully controlling the vehicle [1]. Intelligent vehicles function at the control layer to enable the driver vehicle subsystem to operate more effectively. Intelligent vehicles can help drivers in various ways: (1) collision warning—to advise or warn the driver; (2) driver assistance—to partially control the vehicle as an emergency intervention to avoid a collision; (3) automated driving—to fully control the vehicle. Vehicles that can navigate autonomously in everyday traffic will become a reality in a few decades. In this paper, we will focus on the development of embedded systems to realize automated road following for intelligent vehicles.

A lot of highway accidents are caused by deviation from the road. Autonomous road following is one of the major research topics in the area of intelligent vehicles. Because of its potential in preventing accidents caused by human fatigue and performing tasks in environments unreachable by human beings, a lot of road following methods for intelligent vehicles have been proposed in the recent decades. Intelligent vehicles are robotic systems that perceive the driving environment to assist the driver in safe vehicle operation by providing driving assistance or full control of the vehicle. With the proposed research, we will be able to develop an autonomous control system that can be used on cars to assist driving or take the control of driving in order to avoid fatal accidents. Furthermore, the developed intelligent vehicles can be used to perform tasks in an environment that is unreachable by human beings.

Generally speaking, autonomous road following requires two major steps: road feature extraction, and speed/steering control of the vehicles. The success of autonomous road following requires the ability to continuously detect and extract useful road features, to analyze road features, and to perform steering and speed control based on road conditions.

Computer-vision-based road feature extraction has been applied in intelligent vehicles for road following widely because of its advantages in low-power consumption, compact size, availability as a commercial product, cost effectiveness, and robustness [2]. Numerous image processing and feature extraction methods can be found in [3–7].

Control algorithms should be considered as an important issue in road following to ensure safe and smooth rides. Although a lot of researches have been done on this topic, most of them are based on traditional control theories such as PID [2] and nonlinear controllers [8]. The kinematic behavior of autonomous road following is typically nonlinear. Therefore linear models usually fail to describe these systems efficiently. However, it is difficult to analyze nonlinear mathematical models for autonomous road following schemes. Other methods such as neural networks [9, 10] and reinforcement learning (RL) [11] approaches have also been used in road following, but these approaches require learning procedures which consume extra computation time.

Human drivers can drive a car smoothly with their driving expertise rather than knowledge on control theories. This fact leads us to the fuzzy logic solution. Fuzzy logic control is known to be an organized method to emulate human expertise in dealing with imprecise data. It attempts to apply a human-like way of thinking in the application areas and allows intermediate values to be defined with linguistic terms besides conventional evaluations. It has been proven to be an effective and active method in solving control problems during the past decades. Fuzzy logic is a logic much closer to human thinking than traditional logic. It is a precise logic of imprecision and approximate reasoning. Fuzzy logic controllers provide a means of converting a linguistic control strategy based on expert knowledge into an automatic control strategy [12]. In areas where conventional control methods have difficulties because of the lack of precise quantitative data regarding the system inputs and outputs, fuzzy logic controllers can process imprecise data and make rational decisions by emulating human thinking and decision making capabilities. Fuzzy logic control has been applied for intelligent vehicles in many areas such as car parking and vehicle guidance. Various applications of fuzzy logic control in the field of intelligent vehicles can be found in [13–17].

Safety and reliability are of great importance for intelligent vehicles especially if the vehicles are used for transportation. Therefore, before applying any dynamical systems to intelligent vehicles, the stability of these dynamical systems should be studied to make sure they are stable. Several well-known methods could be used to analyze the stability of a system, such as the Routh-Hurwitz stability criterion, the Lyapunov stability criterion, the Nyquist criterion, finding poles directly, the root locus method, or the Jury stability test for discrete-time control systems. Lyapunov stability theory is probably the most used tool for stability analysis. A dynamic system is Lyapunov stable if all solutions of the system which start near an equilibrium point remain near the point for all time. However, it is difficult to find the suitable Lyapunov functions. There is no general method to construct a Lyapunov function. Trial-and-error or mathematical/physical methods are often used. Some useful approaches for finding Lyapunov functions are Sum of Squares Decomposition [18, 19], convex search for storage functions [20], and variable gradient method [21]. However, these methods do not always lead to a desired Lyapunov function.

In this paper, we will design an autonomous road following embedded system for multiple intelligent vehicles. An intelligent vehicle which is capable of moving between two lines on the road will be designed. A fuzzy logic controller (FLC) will be developed to control the steering wheel of the vehicle for autonomous road following. An FLC on a nonlinear system requires less computational power compared to traditional nonlinear system applications. The resources required for building the embedded system will be significantly reduced. Lyapunov's direct method will be used to analyze the stability of the control system. The “variable gradient” approach proposed in [21] will be used to construct the Lyapunov function of this system. This control system will be implemented on a vision-based intelligent vehicle which is able to perform road following autonomously.

The rest of the paper is organized as follows. The kinematic model of the vehicle is studied in Section 2. Section 3 presents the design of the FLC. Section 4 provides the proof of the stability using Lyapunov's direct method. Experimental setup for the vision-based intelligent vehicle is described in Section 5. Section 6 discusses the simulation and experimental results. The last section of this paper concludes the research.

2. Kinematic Model of the Car

The road following problem of the intelligent vehicle with kinematic constraints in the two-dimensional workspace is studied. A nonholonomic constraint for a robotic vehicle is a nonintegrable equation involving the configuration parameters and their derivatives (velocity parameters) [22]. Such a constraint does not reduce the dimension of the space of configurations attainable by the robot but reduces the dimension of the space of possible motions at any given configuration. Considering the robotic vehicle modeled as Figure 1, the rear wheels are aligned with the vehicle while the front wheels are allowed to pivot about the axes. In a sufficiently large empty space, a robotic vehicle can be driven to any position with any orientation, hence the robot's configuration space has four dimensions, two for translation, one for rotation and one for the steering angle. Let (𝑥,𝑦,𝜙,𝜃) denote the configuration of the robot, parameterized by the location of the front wheels. The kinematic model of the robotic vehicle can be represented as 𝑥̇=𝑢₃̇cos𝜃,𝑦=𝑢₃̇𝑢sin𝜃,𝜃=³𝑙tan𝜙, (2.1) where 𝑢₃ corresponds to the forward velocity of the vehicle and the angle of the vehicle body with respect to the horizontal line is 𝜃, the steering angle with respect to the vehicle body is 𝜙, (𝑥,𝑦) is the location of the center point of the front wheels, 𝑙 is the length between the front and the rear wheels.

Figure 1 

Kinematic model of the car.

Since the turn radius of the robot is quite large compared with the radius of the wheels, referring to Figure 1, we have the following relations: 𝑆₁𝑤=(𝑅−2𝑆cos𝜙)𝜙,_𝑚𝑆=𝑅𝜙,₂𝑤=(𝑅+2cos𝜙)𝜙, (2.2) where 𝑆₁ and 𝑆₂ give the displacement (distance traveled) of the front left and front right wheel, respectively, 𝑅 is the turn radius of the center point of the front wheels, 𝑤 is the distance between wheels (from center-to-center along the length between the two font wheels or two back wheels), and 𝜙 is the angle of the turn in radians. 𝑆_𝑚 is the displacement at the center point of the front wheels. Once we have established the simple geometry for the robotic vehicle system, it is easy to develop algorithms for controlling the robot's steering angle 𝜙, thus controlling the robot's orientation 𝜃. As the robot is considered as having a rigid body, to develop a forward kinematic equation for the steering system, we start by specifying a frame of reference in which an arbitrarily chosen point is treated as stationary. All other points in the system are treated as moving relative to the reference point. Here we treat the center point of the front wheels as the origin of the simulated robot's frame of reference.

For the robotic vehicle that we are modeling, the turn angle of the front wheels has a range of [−𝜋/6,𝜋/6], that is, −𝜋/6≤𝜙≤𝜋/6. By adjusting the steering angle 𝜙, we can control the velocity of both the front wheels

𝑆̇₁=𝜙̇(𝑅−𝑤2𝑤cos𝜙)+2𝜙̇𝑆̇𝜙sin𝜙,₂=̇𝑤𝜙(𝑅+2𝑤cos𝜙)−2𝜙𝜙̇sin𝜙, (2.3) so that

𝑢₂−𝑢₁=𝜙̇𝑤(cos𝜙−𝜙sin𝜙), (2.4) where 𝑢₁ and 𝑢₂ correspond to the forward velocity of the front left wheel and the front right wheel, respectively.

The real-time path of the vehicle can be obtained by integrating (2.1). The angle of the vehicle body with respect to the horizontal line at time (𝑡+Δ𝑡) could be derived first. If 𝜙̇≠0, that is, 𝑢₂≠𝑢₁, the angle of the vehicle body with respect to the horizontal line at time (𝑡+Δ𝑡) is given as

𝑢𝜃(𝑡+Δ𝑡)=𝜃(𝑡)+³𝑙_𝑡^𝑡+Δ𝑡𝑢tan𝜙𝑑𝑡=𝜃(𝑡)+³𝑙𝜙̇(−ln|cos𝜙|)|_𝑡^𝑡+Δ𝑡𝑢=𝜃(𝑡)+³𝑙𝜙̇(−ln|cos𝜙(𝑡+Δ𝑡)|+ln|cos𝜙(𝑡)|), (2.5) and if 𝜙̇=0, that is, 𝑢₂=𝑢₁, the angle of the vehicle body at time (𝑡+Δ𝑡) is given as

𝑢𝜃(𝑡+Δ𝑡)=𝜃(𝑡)+³𝑙tan𝜙Δ𝑡. (2.6)

Next the position of the vehicle at time (𝑡+Δ𝑡) could be derived. If 𝜃̇≠0, that is, 𝜙≠0, the position of the moving vehicle at time (𝑡+Δ𝑡) is given as

𝑢𝑥(𝑡+Δ𝑡)=𝑥(𝑡)+³𝜃̇𝑢(sin𝜃(𝑡+Δ𝑡)−sin𝜃(𝑡)),𝑦(𝑡+Δ𝑡)=𝑦(𝑡)−³𝜃̇(cos𝜃(𝑡+Δ𝑡)−cos𝜃(𝑡)), (2.7) and if 𝜃̇=0, that is, 𝜙=0, the position of the vehicle at time (𝑡+Δ𝑡) is given as

𝑥(𝑡+Δ𝑡)=𝑥(𝑡)+𝑢₃cos𝜃Δ𝑡,𝑦(𝑡+Δ𝑡)=𝑦(𝑡)+𝑢₃sin𝜃Δ𝑡. (2.8)

3. Design of the FLC

In this section, the general structure of the proposed FLC is presented first. Then based on simulation and experiments, the FLC is designed to perform the road following task. The developed FLC not only enhances performance for road following but also saves computational resources.

3.1. The Structure of the FLC

Figure 2 shows the structure of the proposed FLC which has four layers: input processing layer, fuzzification layer, decision-making layer, and defuzzification layer. These layers form an FLC following the Mamdani model. There are 𝑀 inputs and 𝑁 outputs for the FLC. Each rule in the system has the following form: Rule 𝑘: IF 𝑥₁ is 𝑆₁^(𝑘) AND … AND 𝑥_𝑚 is 𝑆_𝑚^(𝑘)… AND 𝑥_𝑀 is 𝑆_𝑀^(𝑘), THEN 𝑦₁ is 𝑅₁^(𝑘) AND …𝑦_𝑛 is 𝑅_𝑛^(𝑘)… AND 𝑦_𝑁 is 𝑅_𝑁^(𝑘)  (𝑘=1,2,…,𝐾),where 𝑥_𝑚 is the 𝑚th input, 𝑚=1,2,…,𝑀, 𝑦_𝑛 is the 𝑛th output, 𝑛=1,2,…,𝑁, 𝑆_𝑚^(𝑘) is the fuzzy linguistic sets for input 𝑥_𝑚, 𝑅_𝑛^(𝑘) is the fuzzy rule sets for output 𝑦_𝑚, and 𝐾 is the number of rules in the rule base. The design of each layer is presented as follows.

Figure 2 

Structure of the FLC.

Input Processing Layer
The inputs of the FCL are crisp values. The input vector can be presented as 𝑥₁𝑥₂⋯𝑥_𝑀. (3.1) Each input is adjusted by a weight factor. Then the input vector can be presented as 𝑤₁𝑥₁𝑤₂𝑥₂⋯𝑤_𝑀𝑥_𝑀, (3.2) where 𝑤₁,𝑤₂,…,𝑤_𝑚 are weight factors.

Fuzzification Layer
It converts the weighted crisp inputs to fuzzy variables. This layer has three nodes for each input: one node defines the fuzzy membership functions (MFs) for the input; the other two nodes represent the two triggered MFs for the input. Triangle MFs are used for fuzzy sets except for the leftmost and the rightmost fuzzy set. The leftmost and rightmost MFs are trapezoidal. Without loss of generosity, MFs for different fuzzy sets can have different triangle or trapezoidal shapes. We constrain the maximum overlapping degree of two MFs to 50%. Therefore the maximum number of triggered MFs is 2 and these two MFs must be adjacent. An example of the MFs with five fuzzy sets is shown in Figure 3. We define MF⁽¹⁾ and MF⁽²⁾ as the first and second triggered MF, and 𝜇_𝑚⁽¹⁾ and 𝜇_𝑚⁽²⁾ as the membership value of the first and second triggered MF of the input 𝑥_𝑚, respectively, then 𝜇_𝑚⁽¹⁾ and 𝜇_𝑚⁽²⁾ can be calculated as 𝜇_𝑚⁽¹⁾=1,𝜇_𝑚⁽²⁾=0,𝑥_𝑚≤𝐶_𝑙,𝜇_𝑚⁽¹⁾=0,𝜇_𝑚⁽²⁾=1,𝑥_𝑚≥𝐶_𝑟,𝜇_𝑚⁽¹⁾𝑤=max(0,1−^𝑚𝑥_𝑚−𝐶_𝑚⁽¹⁾𝑏_𝑚⁽¹⁾𝜇),_𝑚(2)𝐶=max(0,1−^𝑚(2)−𝑤_𝑚𝑥_𝑚𝑏_𝑚⁽²⁾),𝐶_𝑚⁽¹⁾≤𝑥_𝑚≤𝐶_𝑚⁽²⁾, (3.3) where 𝐶_𝑚⁽¹⁾ and 𝐶_𝑚⁽²⁾ are the centers of the two triggered MFs and 𝐶_𝑚⁽¹⁾<𝐶_𝑚⁽²⁾, 𝐶_𝑙 and 𝐶_𝑟 are the leftmost and the rightmost limits, respectively, 𝑏_𝑚⁽¹⁾ is the width of the right-half base of MF⁽¹⁾, and 𝑏_𝑚⁽²⁾ is the width of the left half base of MF⁽²⁾. If one of the membership values is zero, the MF with respect to this membership value is not triggered, then there is only one triggered MF. The outputs of this layer are membership values of triggered MFs for all inputs.

Figure 3 

Input MFs.

Decision-Making Layer
For every output, each node in this layer selects a fuzzy control rule from the rule base of this output and obtains the antecedent value of this rule based on triggered input MFs. Each output has one rule base, which means in the case of 𝑁 outputs there are 𝑁 different rule bases in total. For each output, rules are selected according to the predefined rule base corresponding to this output. Thus, each node in this layer selects 𝑁 rules from 𝑁 different rule bases. Each control rule is selected based on a combination of MFs. Each of these MFs is one of the triggered MFs of an input. For example, if there are three inputs, two of which have two triggered MFs while the other input has only one triggered MF, the number of selected rules for each output is 4(2×2×1). The antecedent value of the selected rule 𝑘 is computed using 𝐴𝑁𝐷 fuzzy logic. Therefore, “min” operation is considered for composition of the FLC: 𝐹^(𝑘)=min(𝜇₁,𝜇₂,…,𝜇_𝑚,…,𝜇_𝑀), (3.4) where 𝜇_𝑚 is one of the triggered input MFs (either 𝜇_𝑚⁽¹⁾ or 𝜇_𝑚⁽²⁾) of 𝑥_𝑚,𝑘=1,2,…,𝐾^∗, and 𝐾^∗ is the number of selected rules.

Defuzzification Layer
This layer converts fuzzy values to crisp values and send them out as control outputs of the FLC. Each node in this layer performs defuzzification for an output. Triangles are used for the output MFs. Figure 4 shows an example of MFs for an output with three fuzzy sets. The center of gravity (COG) algorithm is used to compute output crisp values. Let 𝐶_𝑛^(𝑘) be the center of the 𝑘th triangle. The output 𝑦_𝑛 can be computed as 𝑦_𝑛=∑_𝑘𝐶_𝑛^(𝑘)∫𝐹_𝑛^(𝑘)𝑑𝑡∑_𝑘∫𝐹_𝑛^(𝑘)𝑑𝑡, (3.5) where 𝐹_𝑛^(𝑘) is the antecedent value of the 𝑘th selected fuzzy rule for 𝑦_𝑛. As all the output MFs are triangles, ∫𝐹_𝑛^(𝑘)𝑑𝑡 can be obtained by computing the area of the trapezoid under the line 𝑓=𝐹_𝑛^(𝑘). Thus, (3.5) can be modified as 𝑦_𝑛=∑_𝑘𝐶_𝑛^(𝑘)𝑏_𝑛^(𝑘)(𝐹_𝑛^(𝑘)−(𝐹_𝑛^(𝑘))²/2)∑_𝑘𝑏_𝑛^(𝑘)(𝐹_𝑛^(𝑘)−(𝐹_𝑛^(𝑘))²/2) (3.6) where 𝑏_𝑛^(𝑘) is the base length of the 𝑘th output MF. Equation (3.6) simplifies the defuzzification calculation by replacing derivatives with the computation of trapezoid areas.

Figure 4 

An example of output MFs.

3.2. The Developed FLC for Road following

A block diagram of the specified FLC for the road following task is shown in Figure 5. The desired orientation of the center line of the car should be aligned with the road centroid. The error is the angle between the desired orientation of the center line and the actual center line of the car. The error is represented by 𝑒=𝜃_𝑑−𝜃. To reduce the error to zero, the steering angle should be equal to 𝜙. 𝜙 is determined by the FLC. The error and the change-in-error are calculated and fed into the FLC. The FLC is designed to output control signals corresponding to the control torque to the front steering motor to control the front wheels' steering angle 𝜙.

Figure 5 

Block diagram of the specified FLC.

The fuzzification procedure maps the crisp input values to the linguistic fuzzy terms with the membership values between 0 and 1. In this study, we use five MFs for both error 𝑒=𝜃_𝑑−𝜃 and change-in-error 𝑒̇=𝜃̇_𝑑−𝜃̇ because five MFs can balance between the performance and the complexity of the FLC. The isosceles triangle membership function (MF) is used except for the leftmost and the rightmost input MF which are trapezoidal. Each MF has a 50% overlapping with its neighboring MFs. Figure 6 illustrates the input MFs for 𝑒 and 𝑒̇, respectively. Note that the scale for the 𝑋 coordinate can be changed easily in the program.

(a)

(b)

Figure 6 

Input Membership functions. (a) Error MF. (b) Change-in-error MF.

The inference engine is responsible for decision making in the control system using approximate reasoning. The fuzzy control rules are designed based on expert knowledge and testing. Furthermore, the control rules also meet the stability requirements derived from Lyapunov's direct method. Table 1 represents abstract knowledge that an expert uses to control the steering angle given information about the error and its derivative. For example, if 𝑒 is positive large “PL” and is increasing rapidly (𝑒̇ is positive large), then the vehicle should turn left sharply, that is, 𝜙 should be left large (LL). The rule base stores rules which define the relation between the inputs and the output. Based on the membership functions of the error and the change in error, twenty-five fuzzy rules are obtained.

Table 1 

Fuzzy rule base.

Figure 7 shows output MFs. All MFs for the output are isosceles triangles with 50% overlap between two neighboring MFs.

Figure 7 

Steering control output MF.

The defuzzification procedure maps the fuzzy output from the inference mechanism to a crisp signal. We use the COG defuzzification method to combine the recommendations represented by the implied fuzzy sets from all the rules. The crisp value is computed according to (3.6).

4. Stability Analysis Using Lyapunov's Direct Method

The autonomous road following system can be represented as

𝑒=𝜃−𝜃_𝑑=𝑥₁,𝑒̇=𝜃̇−𝜃̇_𝑑=𝑥₂, (4.1) where 𝜃_𝑑 is the desired angle, and 𝜃 is the actual angle. From (2.1), we know that we need to design the FLC to control the steering wheel so that 𝜃 can approach 𝜃_𝑑. The output of the fuzzy controller Φ(⋅) is a function of 𝑥₁ and 𝑥₂ that can be represented by Φ(𝑥₁,𝑥₂). On the intelligent vehicle, the control signal Φ(𝑥₁,𝑥₂) is used to steer the front wheels. We assume that the low-level tracking controller can output the motor torque to steer the front wheels accordingly. Recall (2.1), we have

𝑥̇₁=𝑥₂=𝜃̇−𝜃̇_𝑑=𝑢₃𝑙tan𝜙−𝜃̇_𝑑,𝑥̇₂=𝜃̈−𝜃̈_𝑑=𝑢₃𝑙1cos²𝜙𝜙̇−𝜃̈_𝑑. (4.2)

Assume that at the equilibrium point 𝑒=0 and 𝑒̇=0, so that the equilibrium is preserved. The main shortcoming of Lyapunov theory is the difficulty associated with the construction of Lyapunov functions [21]. To overcome this problem, we use the “variable gradient” approach proposed in [21] to construct the Lyapunov function. The essence of this method is to assume the gradient of the unknown Lyapunov function 𝑉(⋅) is known up to some adjustable parameters. Then finding 𝑉(⋅) becomes integrating the assumed gradient.

The system can be represented as

𝑥̇=𝑓(𝑥)=𝑥̇₁𝑥̇₂^𝑇=𝑥₂𝑥̇₂^𝑇. (4.3) We proceed to find a suitable Lyapunov function for this system as follows.

Step 1. Assume that ∇𝑉(𝑥)=𝑔(𝑥) has the form 𝑔(𝑥)=[ℎ¹₁𝑥₁+ℎ²₁𝑥₂ℎ¹₂𝑥₁+ℎ²₂𝑥₂]. (4.4)

Step 2. Impose the symmetry conditions 𝜕²𝑉𝜕𝑥_𝑖𝑥_𝑗=𝜕²𝑉𝜕𝑥_𝑗𝑥_𝑖, (4.5) or equivalently, 𝜕𝑔_𝑖𝜕𝑥_𝑗=𝜕𝑔_𝑗𝜕𝑥_𝑖. (4.6) In our case, the symmetry condition is 𝜕𝑔₁𝜕𝑥₂=𝑥₁𝜕ℎ¹₁𝜕𝑥₂+ℎ²₁+𝑥₂𝜕ℎ²₁𝜕𝑥₂,𝜕𝑔₂𝜕𝑥₁=ℎ¹₂+𝑥₁𝜕ℎ¹₂𝜕𝑥₁+𝑥₂𝜕ℎ²₂𝜕𝑥₁. (4.7) To simplify the solution, we assume that 𝑔_𝑖's are constant. Then we have 𝜕ℎ¹₁𝜕𝑥₂=𝜕ℎ²₁𝜕𝑥₂=𝜕ℎ¹₂𝜕𝑥₁=𝜕ℎ²₂𝜕𝑥₁=0,𝜕𝑔₁𝜕𝑥₂=𝜕𝑔₂𝜕𝑥₁⇔ℎ²₁=ℎ¹₂=𝑘⇒𝑔(𝑥)=[ℎ¹₁𝑥₁+𝑘𝑥₂𝑘𝑥₁+ℎ²₂𝑥₂]. (4.8) If we choose 𝑘=0, we have 𝑔(𝑥)=𝑔₁𝑔₂=ℎ¹₁𝑥₁ℎ²₂𝑥₂. (4.9)

Step 3. Find 𝑉̇: ̇=ℎ𝑉(𝑥)=∇𝑉⋅𝑓(𝑥)=𝑔(𝑥)⋅𝑓(𝑥)₁₁𝑥₁ℎ²₂𝑥₂𝑥₂𝑥̇₂=ℎ¹₁𝑥₁𝑥₂+ℎ²₂𝑥₂𝑥̇₂. (4.10)

Step 4. Find 𝑉 from ∇𝑉 by integration: 𝑉(𝑥)=^𝑥1₀𝑔₁(𝑠₁,0)𝑑𝑠₁+^𝑥2₀𝑔₂(𝑥₁,𝑠₂)𝑑𝑠₂=^𝑥1₀ℎ¹₁𝑠₁𝑑𝑠₁+^𝑥2₀ℎ²₂𝑠₂𝑑𝑠₂=12ℎ¹₁𝑥²₁+12ℎ²₂𝑥²₂. (4.11) Therefore, we have 1𝑉(𝑥)=2ℎ¹₁𝑥²₁+12ℎ²₂𝑥²₂̇,𝑉(𝑥)=ℎ¹₁𝑥₁𝑥₂+ℎ²₂𝑥₂𝑥̇₂. (4.12)(4.13)

Step 5. Find ℎ¹₁ and ℎ²₂ that make 𝑉>0. From (4.12), 𝑉(𝑥)>0 only if ℎ¹₁,ℎ²₂>0. Let ℎ¹₁=ℎ²₂=1, then we have 𝑉̇(𝑥)=𝑥₁𝑥₂+𝑥₂𝑥̇₂=(𝜃̇−𝜃̇_𝑑)(𝜃+𝜃̈−𝜃_𝑑−𝜃̈_𝑑). (4.14)

Based on the “variable gradient” approach, we can derive the following Lyapunov function:

1𝑉(𝑥)=2𝑥²₁+12𝑥²₂, (4.15) where 𝑉∶𝐵(𝜖)→ℜ for some 𝜖>0, where 𝐵(𝜖)={𝑥∈ℜ²∶|𝑥|<𝜖} is a ball centered at the origin with a radius of 𝜖 and |⋅| is a norm on ℜ². Then the gradient of the Lyapunov function is

∇𝑉(𝑥(𝑡))=[𝑥₁𝑥₂̇],𝑉=[𝑥₁𝑥₂]𝑥₂𝑥̇₂=𝑥₂(𝑥₁+𝑥̇₂)=(𝜃̇−𝜃̇_𝑑)(𝜃+𝜃̈−𝜃_𝑑−𝜃̈_𝑑)𝑢=(³𝑙tan𝜙−𝜃̇_𝑑)(𝜃−𝜃_𝑑+𝑢₃𝑙1cos²𝜙𝜙̇−𝜃̈_𝑑). (4.16)(4.17) We would like 𝑉̇≤0, to prove stability, that is, to show that the fuzzy controller can achieve and maintain the desired 𝜃_𝑑. In (4.17), 𝑢₃/𝑙>0, cos²𝜙>0. We assume that 𝜃̇_𝑑 is bounded, and 𝜃̈_𝑑=0. On the RC car, the steering angle −30^∘≤𝜙≤30^∘.

We can always design the FLC, Φ(𝑥₁,𝑥₂), to adjust 𝜙 and 𝜙̇ to achieve the following conditions:

𝑢₃𝑙1cos²𝜙𝜙̇≥𝜃_𝑑−𝜃if𝑢₃𝑙tan𝜙−𝜃̇_𝑑𝑢>0₃𝑙1cos²𝜙𝜙̇≤𝜃_𝑑−𝜃if𝑢₃𝑙tan𝜙−𝜃̇_𝑑<0. (4.18) Therefore, 𝑉̇≤0 can always be achieved and hence the stability holds.

5. Experimental Setup

In this section, the hardware setup of the prototype vehicle for autonomous road following is presented. A fully automated robotic system is designed to control a motor vehicle using an on board camera. The hardware system includes a camera associated with a frame grabber. The camera is then connected to a Microchip PIC controller which controls the vehicle motion. This system utilizes the camera with a frame-grabber and a built-in processor to grab pictures of the road and to find the center of the road. Then it calculates the deviation between the center of the car and the center of the road, and sends the result to an FLC. The FLC decides how to turn the vehicle and sends out control signals to control the steering wheel. This is implemented in a way that the vehicle can remain between two lines while simulating highway motion.

5.1. The Remote Control Car

The Audi S4 radio control (RC) car platform is chosen because it offers a fully functional steering and drive system. The control signal for the steering and drive system is generated by the microcontroller which emulates the car's control signals. The speed and steering of the car are controlled by varying the pulse-width modulation (PWM) signals. The steering mechanism of the RC car is shown in Figure 8.

Figure 8 

The steering mechanism of the RC car.

5.2. The Mocked Road

The lane used in this project is the space between two lines as shown in Figure 8. To illustrate the control process, we use a red line and a yellow line. The distance between these two lines is constant.

5.3. Computer Vision

The CMUCam3 camera is chosen for the project. The camera uses a built in microcontroller and a frame grabber to allow postprocessing on each captured image. As the two lines which mark the road are in different colors from the rest of the road, using a multiple color tracking algorithm, we are able to track each line individually. The minimum and maximum RGB values for each line color are set in the camera code. Pixels within the RGB value range are regarded as a part of the line. A filter is applied to filter out noise. The camera finds the location and the width of each colored line in terms of pixels and calculates the centroid of each colored line. The centroid of two colored lines is then used to calculate the position error which is sent to the microcontroller. Since the vehicle is supposed to remain in the center of the two colored lines, error can be calculated if the centroid is not at the center of the image. The camera is programmed using a variation of the C language. The error is calculated according to one of the states described in Table 2. Centroids in the table refer to the 𝑥 coordinate of the image only. In the table, 𝐶𝑟 represents the centroid of the red line, 𝐶𝑦 represents the centroid of the yellow line, 𝐶𝑤 represents the centroid of the image, 𝐶 represents the centroid of the two lines, respectively.

Table 2 

Error calculation table.

5.4. Microcontroller

The PIC18F4431 microcontroller is used to interface with the camera and the car's existing circuitry. It receives the incoming position error from the RS-232 port located on the CMUCam3 board. The microcontroller then uses the error to calculate the appropriate steering signal, using the proposed FLC written in C language. The steering signal is then output to the RC car's circuit.

6. Simulation Studies and Experiment Results

In order to verify the performance of the proposed FLC and the system, simulations and experiments have been conducted.

Simulations have been done to test the performance of the fuzzy controller. The results show that the intelligent vehicle is able to follow the road with a speed of 13m/s. Figure 9 shows the tracking performance of the robotic vehicle in the Cartesian space. The road has two sharp curves and the robot starts from (0, 1) which is off the center of the road. The robot is capable of following the road. It can respond to sudden changes on the road by changing its steering angle accordingly.

Figure 9 

Following a road in Cartesian space.

Figure 10 shows the steering error and the change-in-error. Figure 11 shows the output of the fuzzy controller. It can be seen that the fuzzy controller can output smooth control signals to sudden changes in 𝑒 and 𝑒̇ to make the system stable.

(a)

(b)

Figure 10 

Error $e_{\varphi }$ and change-in-error ${\dot{e}}_{\varphi }$.

Figure 11 

Output of the fuzzy controller.

Experiments have been performed to verify the fuzzy logic controller design. In our experiments, we use a mocked road with two lines to guide the navigation of the robot. The actual trajectory of the robot is marked on the road using white powder dropping from a container attached to the rear end of the robot. Figure 12 shows the robot performing the autonomous road following task on the mocked road with curves. The robot moves at a speed of 0.9 m/s. In the figure, the line in the center of the road is the trajectory marked by the powder.

Figure 12 

The road following result using the proposed method.

In our experiments, all data, such as errors and mean square errors, are calculated based on four trials. Sample points on the actual trajectories are recorded and compared with the desired trajectory which is the center line of the road. One sample point is taken for every inch along the road. Figure 13 shows the desired trajectory and the actual trajectory we have recorded during the experiment. In the figure, the dashed line represents the desired trajectory while the dotted line represents the actual trajectory. The maximum absolute error between the center line of the road and the actual trajectory of the intelligent vehicle is 1.55 inches. The root mean square error is 0.73 inches which is relatively small compared with the width of the road which is 8 inches.

Figure 13 

The actual trajectory and the desired trajectory.

As a comparison, we also obtained simulation results for the vehicle moving at 0.9 m/s. Figure 14 shows the tracking performance of the vehicle. The desired trajectory is the one used in the experiment. It can be seen that experiment results in Figure 13 match the simulation results in Figure 14 very well. The maximum error in the experiment results is a little larger. This is because in the real world, the camera takes some time to process the raw data in order to obtain the error while in simulation the error is calculated instantaneously.

Figure 14 

The simulated trajectory and the desired trajectory.

The error and change in error for both the experiment and the simulation are plotted in Figures 15 and 16. From the figures, it can be seen that the error is larger when there is a curve in the desired trajectory. This is because the desired trajectory is changing while the actual trajectory is deviated from the desired trajectory. The designed fuzzy controller adjusts to the changes and controls the vehicle to follow the trajectory successfully. It is also noted that in the experiment results, the change in error is filtered thus it is smoother than the simulated change in error.

(a)

(b)

Figure 15 

Error and change in error in the experiment.

(a)

(b)

Figure 16 

Error and change in error in the simulation.

7. Conclusion

In this paper, an intelligent robotic vehicle which is capable of performing autonomous road following is successfully developed. A vision system installed on the vehicle is used to detect a curved road. An FLC is developed to control the steering wheel of the robotic vehicle according to the deviation from the rods. The deviation of the vehicle from the road is regarded as the error. The error and the change-in-error are used as the input of the FLC. Based on human driving experience and experiments, the FLC makes decision on the steering angle accordingly. An analysis and design of fuzzy control laws for steering control of the nonholonomic robotic vehicle are presented. Lyapunov's direct method is used to guarantee the convergence of the steering error. Simulations using the four degree of freedom nonholonomic robotic vehicle model are also conducted to investigate the performance and stability of the fuzzy controller. Experiments demonstrate that the vehicle with the proposed fuzzy controller can automatically follow the curved road.

Acknowledgment

Peter Miller, Matt Webb, and Chris Desjardins helped with the hardware setup.