Abstract
In recent years, accelerative aging society is meeting the short supply of young and middle-aged labor. Particularly, most young people are reluctant to work in the construction industry, which has caused the labor cost of floor tiling to rise year by year. In addition, floor tiling requires workers to continuously bend over or lean over to work, which greatly jeopardizes the physical health of them. Therefore, advanced technology applied in floor tiling is highly demanded to replace the traditional manual method. On the context, the automatic method of floor tiling may promote the transformation and upgrading of the industry. Although a few robots for floor tiling have been developed, the automation of existing systems is still at a low level. This paper proposes a robot floor-tiling control strategy based on visual measurement feedback and finite-state machine, in which the calculation of tile position information in limited field of vision is obtained by an improved Canny edge detection and a Hough linear transformation. Moreover, an algorithm for complementing tile position information based on visual measurement is proposed, and the quality of tile laying is evaluated online. To evaluate the effect of the proposed control strategy, the experimental verifications are given. The experimental results indicate that the proposed method can complete the automatic floor tiling with high accuracy.
1. Introduction
1.1. Background and Meaning
With the rapid development of robot technology and sensor technology [1, 2], construction robot technology has made considerable progress for decades [3–5]. Many construction robots are applied to perform specific tasks on construction sites, such as construction fabrication robot [6], drilling robot [7], and marking robot [8]. The usage of robots to replace manual building maintenance and construction in harsh environment will improve the construction industry [9, 10], such as reducing the physical intensity labor, avoiding construction hazards, improving construction efficiency, and ensuring construction quality. Hence, researchers began to pay more attention to the research of construction robot technology [11, 12].
At present, in the decoration process, the floor tiling still relies on manual operation with some simple instruments (as shown in Figure 1), which is labor intensive and time-consuming. The workers must bend over for a long time to perform the floor tiling, which greatly harms their health [13, 14]. The floor-tiling construction requires the workers to have been qualified with professional skills to ensure construction quality. Due to differences in the proficiency of workers, some prominent problems arise in the floor-tiling construction, such as low efficiency, poor tile flatness, serious empty drum, and waste of construction materials caused by rework. In addition, floor tiling is a highly repetitive construction operation, while the size of the tiles is regular. Therefore, automatic floor tiling by the robot has the potential to replace manual operation, which is in accord with modern development direction of floor tiling.
1.2. Related Work
Over the past decade, researchers have been devoting greatly into the research of floor-tiling robots. In 1996, Apostolopoulos et al. [15] proposed a concept of mobile robot to automatically lay tiles. In 2000, Navon [16] developed a floor-tiling robot that realized the location of tiles based on vision, and its efficiency is expected to increase by 2 to 5 times. In 2011, Khan et al. [17] presented a semiautomatic floor-tiling robot, which requires some manual operations to achieve floor tiling. In 2019, Li et al. [18] proposed a vision-based adaptive control method for tile paving, and Liu et al. [19, 20] developed a mobile robot for floor tiling that used laser sensors to measure size and position of tiles. However, in existing related studies, the floor-tiling methods adopt semiautomation or human-machine cooperation, and only some simple visual measurements are carried out. The robustness of the systems is poor. Currently, a simple fact is that the automation and intelligence degree of the robots are low.
1.2.1. Ceramic Tile Position Measurement
The ceramic tile position measurement is a prerequisite for accurate floor-tiling control. The existing research results mainly used laser sensors [17, 19, 20] and vision sensors [16–18] to measure the size and position of tiles. The measurement methods with laser sensors require multiple laser sensors or a single sensor for multipoint measurement. Multipoint measurement will make the measurement process more complex and time-consuming. Furthermore, the movement of the laser will induce new errors. The visual measurement method is simple and position characteristic information of ceramic tiles can be extracted from a single image. In addition, the application of robots in industrial environments provides a reference for the application on construction site. Tlach et al. [21] used a measuring assembly on the robot to achieve position measurement, which required the sensor probe to contact the measured object. As for robot online path optimization, Božek [22] proposed a robot path optimization which is applied to spot welding applications in automotive industry. This method realizes online real-time measurement to provide data support for path optimization.
Part of the research has studied ceramic tile measurement in a factory environment. Golkar and Prabuwono [23] proposed an image processing algorithm to measure the length of ceramic tiles. The systems used a canny edge detection to identify the boundary of ceramic tiles and calculate the size of ceramic tiles, in which deviation was less than 2 mm. The function of this system is to distinguish the ceramic tiles of different specifications on the production line, so the dimensional accuracy is low. Hocenski et al. [24] proposed a defect detection method based on edge detection technology and adopted histogram subtraction to solve the problem of threshold definition of the canny edge detector. This method has good adaptability to ceramic tiles with different textures. Emam and Sayyedbarzani [25] used binocular vision combined with two cameras to measure the size deviation of ceramic tiles, in which measurement accuracy could reach 0.06 mm. Golkar et al. [26] used three cameras to measure the different dimensions of ceramic tiles (edge curvature, length and width, edge crack, and thickness) from multiple perspectives and obtained the target from the background by using a histogram subtraction. An attempt is made to find a suitable threshold to improve the system accuracy that made an effect to succeed. This system was used for automatic quality inspection of ceramic tiles in a production line. The ceramic tile measurement above mainly focuses on a single ceramic tile which is only one side or top view of one tile in the field of view, in which feature extraction is relatively simple.
On the construction site, Deng et al. [27] used Canny edge detection and Hough transform to detect the boundary of the ceramic tile to realize the monitoring of the construction progress of floor tiling. With the comparison of the edge detection results based on different gradient threshold settings, the suitable edge detector threshold was determined. However, the method adopted a manual method to set the threshold, and the measurement accuracy was low and could not meet the needs of robot floor tiling. Lin and Fang [28] proposed a vision-based quality evaluation system for floor tiling, which received the images of finished tiling and extracted geometric features from the images and then evaluated and determined the construction quality of floor tiling. Navon [16] paid attention to the position of the tiles and the damage of the tiles during the floor-tiling process, but the grayscale values of pixels between 31 and 224 were determined as a mistake through the experience in advance, and the robustness was poor.
1.2.2. Robot Floor-Tiling Control Strategy
At present, there is little research on robot floor-tiling control strategies. Apostolopoulos et al. [15] proposed a concept of the floor-tiling process control method based on position and force feedback, but the controller did not carry out actually. Navon [16] used a semiautomatic floor-tiling control method, which required manual assistance for positioning. Khan et al. and Li et al. [17, 18] mainly discussed the control method of floor tiling with a vision servo system. The experiments showed that the control method had better robustness, but the floor-tiling conditions were relatively simple. Some researchers have also studied the control methods of component installation in other task scenarios. Roveda et al. proposed a control framework [29] that can implement industrial interaction tasks in uncertain work scenarios. They also describe a human-robot cooperation methodology to install heavy and bulky components based on marker-based visual servoing and force control [30]. Therefore, the stable control system for robotic floor tiling needs an in-depth study.
The control strategy based on the finite-state machine is to summarize it as a finite state when the system is running, and the state switching is realized by external signal excitation. Therefore, the whole system has good stability. The finite-state machine control method has been applied in many fields such as the centralized control for DC microgrids [31], the integrated navigation and control for robots [32], the fuel cell management strategy [33], and the road automatic driving control [34]. Floor tiling is a repetitive construction process. Each floor tiling can be divided into a finite number of motion states. Applying finite-state machine theory to control floor tiling can simplify the control complexity of the floor-tiling process and improve the robustness of the system.
1.3. Innovation of This Article
This paper aims at realizing a stable control method of floor tiling and proposes a floor-tiling control strategy based on the vision measurement in the limited FOV and finite-state machine control strategy. The main contributions of this paper are summarized as follows:(1)It proposes a fast and robust ceramic tile measurement method based on the vision with a limited FOV. Firstly, an image of the floor-tiling process is obtained in the limited FOV. Next, the edge lines of ceramic tiles are attained through an improved Canny edge detection algorithm and Hough transform algorithm. Finally, a clustering method is proposed to distinguish the edges of ceramic tiles, and the deviation between the desired position and actual position of the tile is calculated.(2)A floor-tiling control strategy based on finite-state machine theory is proposed. The floor-tiling process is split into finite states, and the important state parameters, especially the deviation from the actual position of the ceramic tile to the desired position of that, drive the finite-state machine to act floor-tiling state transition. The control strategy can systematically summarize the state of the floor-tiling process and improve the robustness of the control system.
The rest of this paper is organized as follows. In Section 2, the framework of the floor tile robot II (FTR-II) system is briefly described, including software and hardware systems. It states the ceramic tile measurement method used to obtain the position of ceramic tiles in Section 3. After that, a floor-tiling control strategy based on a finite-state machine is proposed in Section 4. In Section 5, the experimental verifications with the floor-tiling robot system based on the proposed method are given. Finally, Section 6 summarizes the conclusions and future work.
2. Floor-Tiling Robot II (FTR-II) System
Floor-tiling robot II (FTR-II) system that was used to verify the robotic tile-paving control method mentioned in this article consists of four function layers: user layer, system layer, drive layer, and execution layer, as shown in Figure 2.
The user layer is mainly for human-computer interaction. The execution layer implements the subactions of the tile-laying process, and its hardware includes UR10 robot, mobile platform, tile grabbing, and sensors. When the contact force exceeds 10 kg, the robot will immediately stop working and issue a warning, so the safety of FTR-II can be guaranteed. The driver layer is to solve the problem that the system layer cannot directly communicate with the execution layer hardware. The system layer realizes system control and real-time data analysis in the process of tile laying. The hardware and software components of the FTR-II system are shown in Table 1.
3. Vision-Based Ceramic Tile Position Measurement
3.1. Floor Tiling Process Analysis
Figure 3 shows the entire process of floor tiling on the site. Firstly, the worker manually lays the ceramic tile no.1 as a benchmark and the rest ceramic tiles are laid fully automatic by the robot in a sequence shown in Figure 3 from no. 2 to no.12. Excluding the floor-tiling process of the ceramic tile 1, the situations of the entire floor tiling are classified into 3 cases:(a)When the robot is laying the ceramic tiles no. 2 or no.3, the ceramic tile is on the right side of the placed tile No.1. Since there is no ceramic tile on the upper side, only the left ceramic tile no. 1 can be used as a benchmark, as shown in Figure 4(a).(b)When the robot is laying the ceramic tiles no. 4, no.7, or no.10, the ceramic tile to be laid is on the lower side of the laid tiles. Since there is no ceramic tile in the left of the view, only the top ceramic tile may be used as a benchmark, as shown in Figure 4(b).(c)When the robot is laying the other ceramic tiles, different from conditions (a) and (b), the ceramic tiles to be laid are all on the lower-right side of the ceramic tiles that are already laid. Thus, the ceramic tiles on the left and the top side could be used as a benchmark, as shown in Figure 4(c).
(a)
(b)
(c)
It should be noted that, as the three cases above, more tiles can be laid.
3.2. Obtain Ceramic Tile Image in a Limited FOV
The FTR-II can take an image to distinguish the position of a tile from the adjacent one. Then, the coordinate deviation between the actual position and the desired position is calculated. We hope to obtain an image, clear enough, and extract accurate edge feature information from it. We are supposed to take two main problems into consideration while determining the camera position:(1)The edge features of the ceramic tiles measured are detected accurately by the edge detection algorithm(2)It is required to reduce the viewing distance between the camera and the ceramic tile to ensure that the actual physical size of each pixel point mapping can be as small as possible
The camera is mounted on the robotic arm to ensure that the distance between the camera and the tile is small enough to take a clearer image. However, the position of the robotic arm and the tile being laid is fixed. If the robot takes multiple images to cover the full workspace and merge them to create a single image, the camera should be mounted on another manipulator to realize that the position of the tiles remains unchanged during the process of taking images, which would make the system more complicated. According to the above analysis, this paper adopts a method of obtaining ceramic tile images with a limited field of view. Figure 5 shows the measurement method diagram of floor tiling based on the vision. The end effector with four suckers is grabbing a ceramic tile to lay, and a camera is mounted on the robotic arm with a connecting device in order to obtain images in the process of ceramic tile laying. It is important to note that the spatial position relationship between the camera and the laid tile is relatively fixed. Figure 6 shows the images of the ceramic tiles in the limited FOV under the three different conditions.
(a)
(b)
(c)
(d)
3.3. Edge Detection of Ceramic Tile Image
The Canny edge detection algorithm [35] was firstly proposed by Canny in 1986 and has been widely used in various computer vision systems [36, 37]. Canny edge detection is not susceptible to noise interference and can detect real weak edges. One advantage is that two different thresholds are selected to obtain edges by excluding weak edges. However, the method of manually setting the high and low thresholds is not highly adaptable to varied environment.
This paper proposes an improved Canny edge detection algorithm to extract the edges of the ceramic tiles. Figure 7 demonstrates the implementation process of the algorithm, including taking an image, Grayscale image, Gaussian filtering, nonmaximum suppression, Gradient calculation, Otsu adaptive threshold, and edge detection image.
3.3.1. Convert Image to Grayscale
Since the image obtained by the camera is a color one, we use the formula to convert a color image to a gray-level one.
3.3.2. Gaussian Filtering
The discrete Gaussian convolution kernel matrix is dimension, which is calculated by formula (1). Therein, is the variance and k is the dimension of the kernel matrix. It is noticeable that the choice of will affect the performance of the Canny detector. With increasing size, the detector is less sensitive to noise, but the positioning error of edge detection will also increase slightly. This paper uses a 3 ∗ 3 window for detection, and the pixel to be filtered is e. After Gaussian filtering, the gray value of pixel e is shown in formula (2):where is the convolution symbol and means the sum of all elements in the matrix.
3.3.3. Gradient Calculation
and are the first-order partial derivatives in the -direction and -direction, respectively. The magnitude and direction of the gradient are shown in formulas (3) and (4):
3.3.4. Nonmaxima Suppression (NMS)
Nonmaximum suppression is used to obtain a more accurate image edge, which consists of many single pixels. As shown in Figure 8, the gradient direction of the observed pixel is perpendicular to the direction of the image edge. There are two adjacent pixel points not on the edge of the image, but the gradient direction may be calculated by formula (4). The gray value of the observed pixel is compared with the gray values of its adjacency pixels, and the gray value of the pixel nonmaximum is set to 0. Later on, the pixels on the image with nonzero gray values are traversed, and all nonmaximum pixels are eliminated.
3.3.5. Self-Adaptive Threshold Selection Based on Otsu Algorithm
The image processed by the NMS method still contains a lot of noise and false edges. Therefore, the Canny algorithm uses double thresholds to eliminate noise and false edges. For the Canny algorithm, it is very important to choose a reasonable set of thresholds. The traditional Canny algorithm uses manually selected double-threshold values and cannot adapt to various environments where the overall brightness is varied. This paper uses a self-adaptive method based on the Otsu to select the threshold value [38].
It is supposed that is the grayscale range of the image pixels. Taking as the boundary, the set of pixels is divided into and , and the probabilities are and , in the given order. The average gray values of and are and , respectively. Hence, the average gray value of is calculated as . The variance between the two classes is expressed in
Calculating the maximum value of through formula (5), we can obtain the upper threshold value that is marked as .
Figure 9 reveals the detection results of the traditional Canny edge detection algorithm and the improved Canny edge detection algorithm, where a and b are, respectively, the upper and lower limit thresholds. Comparing Figures 9(a) and 9(b), it is suggested that the detection results of the traditional Canny algorithm vary greatly by manually setting different thresholds. With self-adaptive selecting upper and lower thresholds, the edges detected based on the improved Canny edge detection algorithm illustrated in Figure 9(c) are better than that based on the traditional one.
(a)
(b)
(c)
The gray values of the image are greatly affected by the light, so the same upper and lower threshold cannot be used to detect edges of the tile image obtained in the construction environment, which has different lighting conditions. The improved Canny detection algorithm has better light intensity adaptability and is taken in this study.
3.4. Extract Straight Lines and Recognize Tile Boundaries
In a cartesian coordinate system, a straight line is expressed as , and points on the same line have the same b and k values. If the straight line is converted to the parameter coordinate system, it should be expressed as . Points on the same straight line have the same and values, so in the parameter coordinate system, they intersect at one point. When the number of parameter straight lines that intersect at the same point reaches the preset value, we think that we have found enough points to form a straight line. This is the principle of Hough transform (HT). However, it is possible that the two edges of the image to detect have the same and value.
In this paper, we use progressive probabilistic Hough transform (PPHT) to extract tile edges in images [39]. In comparison with Hough transform, the main improvements of the PPHT algorithm are described as follows:(1)Use randomly selected pixels from the input image instead of traversing all pixels(2)Set the maximum pixel interval threshold and the minimum straight line length threshold allowed to form the connecting line segment(3)Find the longest line segment that is unbroken and does not exceed a given threshold
By setting the maximum pixel interval allowed to compose the connecting line segment, the PPHT algorithm can effectively distinguish two-tile edges detected as one edge by the HT algorithm. For example, Figure 10(b) shows that line 1 detected based on the HT algorithm is composed of two-tile boundaries, which coincide exactly on a straight line. When the PPHT algorithm is used to extract a straight line from the above grayscale image (Figure 10(a)), two straight lines can be acquired accurately (Figure 10(c)).
(a)
(b)
(c)
The tile acquired through the PPHT is expressed in the Cartesian coordinate system as . The set of detected points is , where is the number of detected points on the line marked as . Furthermore, the average values of and are calculated aswhere is the number of points included in . The recognition strategy of tile boundaries is targeted to sort and compare the values of and , which are shown in Table 1 and finally to realize the recognition of 8 tile boundaries.
Table 2 describes a strategy for recognizing the tile boundaries. The summary of the strategy is as follows:(1)Sort .(2)Pick the minimum two points and , and then, compare them with the corresponding values of and .The smaller one is no. 1 boundary, and the larger one is no. 5 boundary.(3)Pick the maximum two points and , and then, compare the corresponding values of and . The smaller one is no. 4 boundary and the larger one is no. 8 boundary.(4)Sort . The same principle recognizes the tile boundaries 2, 3, 6, and 7.
The result is presented in Figure 11. We can get 8 straight lines numbered from 1 to 8.
3.5. Position Calculation and Coordinate Transformation
When we are laying ceramic tile, the deviation between the current position and the target position is calculated to drive the end effector of the robot to move. In the two-dimensional rectangular coordinate system, movement of the ceramic tile from current position to desired position can be divided into two processes: horizontal translate along vector and rotate degree about counterclockwise around . is the geometric center of the desired position and is the geometric center of the current position, as shown in Figure 12.
In the three-dimensional motion space of the robot, the motion matrix of the robot end effector is expressed as
3.6. Complement Position Information of Ceramic Tiles
To evaluate the quality of tile laying, we need to wait for all tiles to be laid, and the tiles initially solidify after 24 hours to ensure that the ground has sufficient strength. Afterwards, workers can step on the tiles to check the quality of the tiling. Therefore, this paper proposes a real-time quality evaluation method of laying tile based on the position information of tiles obtained during the tile laying process. However, the ceramic tile visual measurement servo system employs a limited FOV method to obtain images so that it cannot obtain the position information of the entire tiles through a single image. Therefore, only when all tiles are laid, we can obtain the position information of tiles.
This requires us to use the position information of partial tiles, which are classified by the three different conditions mentioned in front part, to complement the global information of the tiling. The specific method is as follows.
In Figure 13, tiles 2 and 3 are complemented by data collected under working condition 1, tiles 4, 7, and 10 are complemented by collecting data under working condition 2, and other tiles are complemented by collecting data under working condition 3.
The use of image complements’ algorithms will introduce an assumption: the base tile is in side length and in size deviation. The international standard BSENISO10545-2:1997 defines the characteristics of various shape and size errors of ceramic tiles, as well as the measurement and inspection methods of related characteristics [40]. The international standard ISO13006 : 2012 specifies the tolerance range of shape and size of various types of ceramic tiles [41].
We will analyze the impact of the reference tiles error on the measurement results. The actual size of tile 1 is , where is the x-coordinate deviation and is the y-coordinate deviation. When we are laying tile 2, the size is calculated as , where and are the vertical position deviation and horizontal position deviation of visual inspection, respectively. However, the actual size of tile 2 is . In the same way, the tiles in the first column will introduce a x-coordinate deviation , and the tiles in the first row will introduce a y-coordinate deviation . When the tiling effect is evaluated, and as intermediate variables do not affect the final evaluation of the straightness of the tiles. After the effect is eliminated, we conduct a real-time quality evaluation of tiling, and the algorithm outline is as follows:(1)Obtain the pixel coordinates of the corner point from the edge of the tile measured based on vision(2)Use the least square method to fit a straight line(3)Calculate the distance of each point from points on the fitted straight line
4. Control Strategy Based on FSM
4.1. Floor-Tiling Operation Control Variable and Status Parameter Information
The floor-tiling robot system consists of a vision sensor, a mechanical arm, an end effector, a laser sensor, an edge detector, and a position calculator. In the floor-tiling process, the robot operation control variables correspond to different system execution units, which are shown in Table 3.
The operating status of the robot mainly reflects the current working status of the floor-tiling robot and is recorded in the form of status parameters, which mainly include the position information of tiles and operating conditions of tiling. Table 4 shows the system operation state parameters.
4.2. Floor-Tiling Control Based on FSM
A finite-state machine (FSM) is a mathematical model with discrete inputs and outputs. It can record the internal state of historical inputs, determine the control execution ability of the next state and behavior according to the current input, and can effectively describe the life cycle of a system.
This paper proposes a floor-tiling control strategy based on FSM for the first time. To facilitate the analysis, the FSM describes the action of laying tile aswhere is a set of n robot states, is a set of n robot program actions, is a set of n robot state parameters, and is a set of n state transition conditions.
The relationship between the 4 sets above is expressed as follows:
To facilitate the analysis and description of the state transition process, a directed graph is defined as formula (10) based on the graph theory and formula (9):
The floor-tiling states’ set is abstracted as the vertices of the directed graph, and the state transition condition set C is abstracted as the edges of the directed graph; then, the edge set is defined as follows:where is the condition for the transition from state to state , which is abstracted as the right of . When the value of is 1, the condition is satisfied. When the value is 0, the condition is not satisfied, and the value is −1, which means that the condition does not exist. The r adjacent vertices of are expressed as follows:where . The FSM diagram of robot floor tiling is shown in Figure 14.
The state information of the robot floor tiling is shown in Table 5. The process of the proposed control strategy for robotic laying a tile is as follows:(1)In the state, the system checks the state of the robot and performs initialization operations.(2)When receiving the signal , the system turns to and takes a floor-tiling image.(3)When the image has been acquired, the system runs the edge detector for edge detection. Under working condition 1, the system turns to ; under working condition 2, the system turns to ; under working condition 3, the system turns to .(4)When the edge detection is completed, the system turns to and performs plane position calculation to generate a coordinate transformation matrix composed of the translation and rotation from the current position to the desired position.(5)When the matrix has been calculated, the system turns to . The system measures the height of the tile to lay from the ground with a laser sensor.(6)When the position of the tile, including vertical height and horizontal posture, has been acquired, the system controls the robotic arm to move the tile from the current position to the desired position, and the system sequentially turns to , , and .(7)The system repeats steps (2)–(5) to calculate the position of the tile. If the position deviation of the tile exceeds the preset value, the system runs step (6).(8)When the position deviation of the tile is within the preset value range, the system turns to , the suction cup is released, and the laying of a tile is completed.
With the aforementioned method, the robotic floor tiling is realized.
5. Experimental Verification and Discussion
5.1. Experiment Preparation
To evaluate the performance of the proposed control strategy, this paper uses the floor-tiling robot II (FTR-II) to perform experimental verification at a laboratory site. Before the experiment, the sizes of 12 tiles are measured to ensure that they are suitable and avoid undermining the experimental results. Each tile is measured 3 times, and the average value is calculated. As shown in Table 6, the average deviation is 0.05 mm, and the maximum deviation is 0.14 mm.
5.2. Experiment at a Laboratory Site
Figure 15 shows the results of the floor-tiling experiment at a laboratory site. FTR-II continuously lays 12 tiles with a size of 300 mm × 300 mm on the ground, and the expected gap between adjacent tiles is 3 mm. The tiling sequence is shown in Figure 3, where tile 1 is used as a benchmark tile. With the limit of the robotic arm working range, the system lays tiles from no. 2 to no. 6 when the robot mobile platform is standstill. Then, the robot mobile platform moves 600 mm back to meet the working distance range of the robotic arm, and the system lays tiles from no. 7 to no. 12. It should be noted that the movement of the mobile platform will cause a deviation of 2 mm. Since the system uses vision for real-time positioning, the deviations above will not affect the quality of laying tiles.
After the paving is completed, a line laser is used to detect the position of the tiles 2–12. The coordinate data of 48 points are obtained in one floor-tiling experiment, each tile in the 12 tiles with 4 fixed points. We have repeated the experiments 3 times, obtained a total of 144 points of coordinates and deviation values, and calculated the position deviation and , which is shown in Table 7.
Figure 16 illustrates the frequency distributions of and in the coordinate system. The average deviations in the x-direction and y-direction are 0.04 mm and 0.11 mm, respectively, while 98.5% of the measured values have deviations <0.5 mm. Noticeably, there is no abnormal interruption in the whole floor-tiling process. The system takes 385 s to lay all 12 tiles, and the time required for place one ceramic tile is 32.1 s. Thus, we confirm that the control strategy proposed in this study can be used to perform accurate floor-tiling operations.
(a)
(b)
5.3. Position Deviation Based on the Online Evaluation
We get a total of 48 corner coordinates in 8 rows and 6 columns. The corner coordinates of the tiles in the coordinate system are distributed as shown in Figure 17. The black point is the origin coordinate, the blue points are the reference corner, the red points are the measurement corner, and the green points are the tile corner complemented. The coordinates’ deviation of each point is calculated, and the results are shown in Table 8, where is the x-coordinate deviation and is the y-coordinate deviation.
Table 8 shows the deviation results calculated by the complement method. The average deviations in the x-direction and y-direction are 0.06 mm and 0.18 mm, respectively, while 91.5% of the measured values have deviations <0.5 mm.
By comparing Table 6 with Table 7, the deviation of online evaluation is significantly higher than the deviation of the measurement deviation, which is shown in Table 9. It is supposed that the size of the tile is 300 mm × 300 mm, which allows the deviation of the actual size of the tile to be introduced into the online evaluation system.
6. Conclusion and Future Research
This paper presents a robotic floor-tiling control method based on visual measurement feedback and finite-state machine. The self-developed FTR-II floor-tiling robot was used to conduct experiments at a laboratory site. Through the analysis of the experimental results, it can be concluded that(1)The control method of robot floor tiling proposed in this study is performed successfully(2)The deviations in the x- and y-coordinates are 0.04 mm and 0.11 mm, respectively, and 98.5% of all deviations are below 0.5 mm(3)The online evaluation system can realize the real-time online quality evaluation of floor tiling
However, the main limitation in this study is that the measurement method based on the limited FOV presupposes that the size of the tile is ideal. Dimensional deviation will be accumulated and will cause the tile position to exceed the permitted range. A possible effective solution is that the position information of multiple tiles is obtained at one time by adjusting the height of the camera. In addition, the detector in this paper can only identify the boundary of monochromatic tiles, and the edge detector will be brittle for patterned tiles.
The research of this paper may be not only applicable to control robotic floor-tiling scenes but also use for reference in the construction work under similar working conditions, such as curtain wall installation [42], brick masonry [43], and ceiling installation [44, 45]. In order to realize that the floor-tiling robot is able to perform actual tiling operations on real construction sites, more in-depth research direction may be developed on the research results of this paper, such as to solve the navigation and positioning problem of the mobile platform on construction site [46], to monitor the progress of the floor-tiling construction combined with BIM technology [47, 48], and to achieve precise positioning of tile laying with contact constraints through hybrid position/force control.
In future research, an edge detector should be developed to adapt to more general lighting conditions and tiles with different patterns. The authors will also develop a hybrid force/position control algorithm based on the visual measurement and force feedback. It may serve for the floor-tiling robot to achieve operation on real construction site and also develop the approaches to plan path and process monitoring on the construction site by the BIM and the mobile platform navigation and positioning technology.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest regarding the publication of this paper.
Acknowledgments
This study was sponsored by the BUCEA Doctor Graduate Scientific Research Ability Improvement Project and Key Science and Technology Projects of China (KZ202110016024).