DSP- and FPGA-Based Stair-Climbing Robot Design

Abstract

A stair-climbing robot is equipped with a grasping arm for capturing objects to provide service for the elders. A board based on a digital signal processor (DSP) plays the role of control center to manage all actions of two brushless DC motors for locomotion, two worm gears for torque magnification, and two DC motors for arms’ pitching in the main body. The robot is steered by fuzzy logic control (FLC) which integrates the outputs of DC bus current sensor and an inclinometer. There is a CMOS camera for vision used in the robot, and the grasping arm is controlled via the video camera for capturing objects. Simple algorithms of image processing are implemented in the field programmable gate array (FPGA) board that generates the 𝑋-axis and 𝑌-axis commands of the grasping arm to capture/load objects. Three walking experiments of the stair-climbing robot to move up and down stairs are shown in the taped pictures from videos. The object tracking and capturing by the grasping arm and camera verifies the proposed design.

1. Introduction

Research on service robots has been attended in recent years. One of the most important reasons is the growing of aging population and decreasing of working population. It is not a long-term way by hiring many foreigners with cheaper payment to work at factories or to take home-caring job for the elder or disabled. As a result, for the latter, home-caring robot is an excellent candidate capable of supporting such an aging society. Specifically, the elders can control the robots directly for service.

The robot designed to directly carry elders up and down stairs needs a large vehicle and the stair rail for moving. The robot “HRP-2” from Harada successfully climbed up 280 mm stairs by grasping the stair rail [1]. The robot “WL-16RII” [2] can walk independently and allow users to build its upper body based on their requirements, such as a walking wheelchair or as a walking support machine that is able to walk up and down stairs carrying or assisting an elder. The developed biped locomotor with Stewart platform legs successfully achieved walking up and down on stairs for 250 mm continuously and carrying 60 kg man on it. Another biped-type robot, for example, “Zero Walker-1” [3], uses its two legs to assist the aged person walking and moving up and down stairs along the handrail by stepping onto the feet of the robot [4]. However, the aforementioned robots generally need tremendous effect on expense and time. Furthermore, it is very difficult to lift an aged person by human force, and it is not very easy to have a large and heavy-weight lift machine in a normal house.

Generally, the first step of image processing for segmenting the target is background subtraction scheme [5]. Although the background subtraction scheme works fairly well in segment foreground data from nonchanging background but it will still allow objects that are paced into the workspace to be detected as potential objects. As a result, a color (or skin) detector is implemented to further filter the foreground data by converting the RGB (red, green, and blue) form of resultant image into 𝑌𝐶_𝑏𝐶_𝑟 (luminance, blueness, and redness) form [5]. Additionally, object tracking has become an important topic in robotics field [6, 7]. Some basic functions such as real time, automation, and robustness to nonideal situations are required for practical object tracking systems. The servo control structure in real-time visual tracking systems usually belongs to the category of “image-based” and “dynamic look and move” [6]. Based on image moments, three different visual predictive control architectures are proposed [7]. However, the proposed system was verified only by simulation results. A novel two-level scheme for adaptive active visual servoing of a mobile robot equipped with a pan camera was presented in [8]. The designed active visual servoing system shows advantages of a satisfactory solution for the field-of-view problem, global high servoing efficiency, and free of any complex pose estimation algorithm usually required for visual servoing systems. With respect to traditional visual servoing approaches that use geometric visual features, the visual feature used in [9] was the pixel intensity. The proposed approach has been tested in term of accuracy and robustness via several experimental conditions. A visual servoing strategy based on a novel motion-estimation technique [10] was presented for the stabilization of a nonholonomic mobile robot. The practical exponential stability can be achieved by perturbed linear system theory, despite the lack of depth information.

In the paper, a stair-climbing robot equipped with an arm will be designed to provide the service of carrying objects up and down stairs or patrolling for security. A control board including a digital signal processor (DSP) TMS320F28335 will steer the robot based on the outputs of DC bus current sensor and an inclinometer by fuzzy logic control (FLC). Without background subtraction and complex object tracking schemes, simple algorithms, such as the color filtering and locating center of the detected target, are employed in the robot system. With the aid of a CMOS camera for vision, the robot arm will track, capture, and put back the target object. The field programmable gate array (FPGA) on DE2-70 board will implement the image processing.

The paper is organized as follows for further discussion. Section 2 describes the design steps of the proposed robot, robot mechanism describing each component and the ways of climbing up and going down stairs, DSP-based controller providing all control signals and realizing fuzzy control logic, and the FPGA-based visual servo for tracking and capturing objects. Section 3 presents the experimental results of stair motion and visual servoing. Finally, Section 4 claims our conclusions.

2. Robot Design

There are three steps for the proposed robot design, mechanism, DSP-based controller, and FPGA-based visual servo.

Step I: Mechanism Design. It is well known that the most effective style of movement of a robot on a plane field is the wheel type. As obstacles and stairs exist, crawler-type and leg-type robots become better candidates for application. However, the proposed robot is equipped with roller chains which are attached with polyurethane rubber blocks to generate friction with ground and stairs for climbing up and down. The stair-climbing robot consists of a main body for moving, a front arm and a rear arm for moving up and down stairs. The main body is equipped with two brushless DC motors (BLDCMs) and their drives for locomotion, worm gears for torque amplification, two dc motors to control two arms, and DSP-based board as control center, shown in Figure 1 [11]. The chassic size of the main body is 58.5 cm × 53 cm and each arm is 48 cm × 40 cm such that the maximum length of the robot will be 154.5 cm. The robot is equipped with roller chains attached with rubber blocks used to generate friction with ground and stairs for moving. There are 136 rubber blocks with size of 3 cm × 2 cm × 1 cm attached to the roller chains, 40 for each arm and 56 for main body. The distance between any two plastic blocks is properly arranged to fix the stair brink. The moving direction of the robot is steered based on the speed difference of two BLDCMs and the information from ultrasonic sensors. The robot makes use of friction force between the roller chains/rubber blocks and stairs/ground to climb up. The front arm will be pushed down to flat top so that the main body is lifted. The front arm will be pulled up for next stair-climbing. The rear arm is kept flat during the robot going up. Figure 2 displays total travel of climbing-up motion. Similarly, Figure 3 shows summary of going-down motion.

Figure 1 

Picture of the stair-climbing robot.

Figure 2 

Total travel of climbing-up motion.

Figure 3 

Summary of going-down motion.

Step II: DSP-Based Controller Design. DSP provides the pulse-width modulation (PWM) signals of BLDCMs and DC motors and realizes the fuzzy logic rules for speed control.

The 𝑖th fuzzy rule in the fuzzy rule-base system is described as [12] 𝑅_𝑖:If𝑥₁is𝐴_𝑖1and𝑥₂is𝐴_𝑖2,then𝑦is𝑤_𝑖, (1) where 𝑤_𝑖, 𝑥_𝑗, and 𝐴_𝑖𝑗, 𝑗=1,2, 𝑖=1,2,…,𝑛 are fuzzy output variables, input fuzzy variables, and linguistic variables, respectively. Referring to Figure 4 for 𝑖th membership function with isosceles triangle shape, 𝑏_𝑖𝑗 means the length of the base and 𝑎_𝑖𝑗 stands for the abscissa of the center of the base. The membership grade of input 𝑥_𝑗 is calculated by 𝐴_𝑖𝑗𝑥_𝑗𝑥=1−2⋅_𝑗−𝑎_𝑖𝑗𝑏_𝑖𝑗,𝑗=1,2. (2)

Figure 4 

𝑖th membership function with isosceles triangle shape.

The bases of triangular membership function keep same for easily programming. By product operation, the membership grade of the antecedent proposition is calculated as 𝜇_𝑖=𝐴_𝑖1𝑥₁⋅𝐴_𝑖2𝑥₂. (3) Then the output will be ∑𝑦=^𝑛𝑖=1𝜇_𝑖𝑤_𝑖∑^𝑛_𝑖=1𝜇_𝑖. (4) Summarily, Table 1 lists the linguistic control rules and Figure 5 displays the design scheme of fuzzy logic control.

Table 1 

Linguistic control rule table.

Figure 5 

Design scheme of fuzzy logic control.

Step III: FPGA-Based Visual Servo Design. The grasping multilink arm consists of three couples of gears, three DC motors, four links, and one clamper. Referring to Figure 6, the first DC motor steers the diving gear 𝑆₁ and driven gear 𝑆₂ to determine the rotating angle. The second motor controls the gears of 𝑆₃, 𝑆_4𝑑, 𝑆_4𝑢, and 𝑆₅ together with belts to stretch the length of the arm. 𝑆_4𝑑 and 𝑆_4𝑢 are mounted in the same shaft and with the same number of gears. Then, the length 𝑋 can be calculated as 𝑋=cos𝜃×𝑙×2, (5) where 𝑙 is the length of the link. The third motor decides the open angle of the clamper.

Figure 6 

Parameter definitions of the grasping arm.

The pixel array of CMOS camera THDB-D5M used in the robot consists of a matrix of 2752 × 2004 pixels addressed by column and row [13, 14]. The 2592 × 1944 array in the center called active region represents the default output image, surrounded by an active boundary region and a border of dark pixels. Pixels of active region are output in a Bayer pattern format consisting of four “colors,” green1, green2, red, and blue (G1, G2, R, and B) to represent three filter colors [12, 13]. The first row output alternates between G1 and R pixels, and the second row output alternates between B and G2 pixels. The green1 and green2 pixels have the same color filter, but they are treated as separate colors by the data path and analog signal chain.

In order to calculate the R, G, and B intensity of each pixel, we relabel the Bayer pattern by the corresponding column-row address, shown in Figure 7. After analyzing, we summarize each expression of the color intensity for each pixel as follows.

Figure 7 

Bayer pattern by the corresponding column-row address.

(i) 𝑥 is even and 𝑦 is even: 𝑅_(𝑥,𝑦)=𝑅_{(𝑥−1,𝑦)}+𝑅_{(𝑥+1,𝑦)}()()𝐵Pixel_Count,Pixel_Count=2,𝑥−1>10,𝑥+1<26811,others,_(𝑥,𝑦)=𝐵_{(𝑥,𝑦−1)}+𝐵_{(𝑥,𝑦+1)},Pixel_CountPixel_Count=2,𝑦−1>50,𝑦+1<20021,others. (6)

(ii) 𝑥 is even and 𝑦 is odd: 𝐺_(𝑥,𝑦)=𝐺_{(𝑥−1,𝑦)}+𝐺_{(𝑥+1,𝑦)}+𝐺_{(𝑥,𝑦−1)}+𝐺_{(𝑥,𝑦+1)},()()𝑅Pixel_CountPixel_Count=4,𝑥−1>10,𝑥+1<26183,others,_(𝑥,𝑦)=𝑅_{(𝑥+1,𝑦−1)}+𝑅_{(𝑥−1,𝑦−1)}+𝑅_{(𝑥+1,𝑦+1)}+𝑅_{(𝑥−1,𝑦+1)},()()Pixel_CountPixel_Count=4,𝑥−1>10,𝑥+1<26182,others. (7)

(iii) 𝑥 is odd and 𝑦 is even: 𝐺_(𝑥,𝑦)=𝐺_{(𝑥,𝑦−1)}+𝐺_{(𝑥,𝑦+1)}+𝐺_{(𝑥−1,𝑦)}+𝐺_{(𝑥+1,𝑦)},𝐵Pixel_CountPixel_Count=4,𝑦−1>50,𝑦+1<20023,others,_(𝑥,𝑦)=𝐵_{(𝑥−1,𝑦−1)}+𝐵_{(𝑥+1,𝑦−1)}+𝐵_{(𝑥+1,𝑦+1)}+𝐵_{(𝑥−1,𝑦+1)},Pixel_CountPixel_Count=4,𝑦−1>50,𝑦+1<20022,others. (8)

(iv) 𝑥 is odd and 𝑦 is odd: 𝑅_(𝑥,𝑦)=𝑅_{(𝑥,𝑦−1)}+𝑅_{(𝑥,𝑦+1)}2,𝐵_(𝑥,𝑦)=𝐵_{(𝑥−1,𝑦)}+𝐵_{(𝑥+1,𝑦)}2, (9) where 𝑥 and 𝑦 stand for the column address and row address, respectively.

The image raw data is sent from D5M to DE2-70 board where the FPGA on DE2-70 board will handle image processing and convert the data to RGB format to display on the VGA display [15–17]. As a result, we first capture the image of experiment background to find the ranges of colors of RGB and then define their location regions for color discrimination. In order to reduce the effect of light variation, the image in RGB space will be converted into 𝑌𝐶_𝑏𝐶_𝑟 space [5]. In addition, the ranges of RGB from D5M are four times of the general image. So those the transformation can be expressed as 𝑌𝐶_𝑏𝐶_𝑟=𝑅𝐺𝐵+0.2990.5870.114−0.1687−0.331260.50.5−0.41869−0.0813164512512. (10)

3. Experiment Results

Gear ratio used in the 45 kg stair-climbing robot is 1320 and the rated dc input power and speed of the 200-W BLDCM are 24 V and 9600 rpm. A preliminary experiment that the unloaded robot climbs up and goes down a gradual stair with the rise of 120 mm and depth of 400 mm by wired control is tested. The results of every motion are shown in Figure 8 [11]. It qualifies the designed robot. Then we conduct the second experiment that the robot with loading of one arm moves up and down a steeper stair with the rise of 175 mm and depth of 280 mm. The taped pictures of the experiment and every motion are shown in Figures 9 and 10, respectively [12].

(a)

(b)

Figure 8 

Realized motion of (a) climbing up and (b) going down by wired control.

(a)

(b)

(c)

Figure 9 

Realized motion of climbing up by FLC.

(a)

(b)

(c)

Figure 10 

Realized motion of going down by FLC.

The third experiment contains image processing and grasping arm motion. In order to prevent target damage while clamping, one pressure sensor is installed inside the clamper. The pressure output after calibrating is sent to DSP for reference. Figure 11 displays the sequentially taped pictures from videos of tracking, capturing the cola can, and putting it back by the robot arm. Figure 11(a) presents the initial status of the experiment. The arm tracks the corresponding direction after the can is shifted left, shown in Figures 11(b) and 11(c). Figures 11(d) and 11(e) depict the right tracking. The arm tracks the can back to central position, shown in Figures 11(f) and 11(g). Then, the robot stretches out the arm for capturing the can and then draws the arm back, presented in Figures 11(h)–11(l). Finally, the robot puts the can back and goes back to the initial status, presented in Figures 11(m)–11(p).

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(p)

Figure 11 

The taped pictures of the experimental results on grasping.

Finally, we conduct the fourth experiment that the robot with all loading moves up a stair with the rise of 150 mm and depth of 300 mm. The taped pictures of the experiment are shown in Figure 12. The climbing-up motion presents variation in proceeding direction. This is due to the plastic bell being almost torn into broken after long time of experimental test. But, it was not found.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 12 

Realized motion of climbing up at full loading.

4. Conclusion

In the paper, we have developed a stair-climbing robot and completed experiments of moving up/down stairs and object tracking, capturing, and loading. In fact, the stair-climbing robot can provide service for the elders by capturing the specific object at one floor and then climbing up or down to another floor. In addition, the robot will patrol for security by the CCD camera around the house while more image processing functions are provided.

Acknowledgments

The authors would like to express their appreciation to Ministry of Education and National Science Council, China, under Contract no. NSC 100-2632-E-218-001-MY3, for financial supporting.