Abstract
We present insight into how contextual awareness can be derived from, and improve, a fusion algorithm combing a WSN and a passive RFID for autonomous mobile robot navigation. Contextual awareness of not where the robot is, but rather the context in which it exists in relation to the environment and human user serves to improve accuracy in navigation, alters the speed of the robot, and modifies its behavior. The WSN system, using a virtual potential field, provides fast general navigation in open areas and the RFID provides precision navigation near static obstacles and in narrow areas. We verified the effectiveness of our approaches through navigational and guidance experiments.
1. Introduction
As robots become more common in our everyday lives, the need for an awareness beyond what simple sensors can detect also grows. While navigation of a mobile robot can be performed with only such data, a behavior that allows it to seamlessly integrate with humans is desirable. In this paper, we present contextual awareness concepts as derived from a wireless sensor network (WSN) and radio frequency identification (RFID) fusion approach to indoor, mobile robot navigation. A WSN-based navigation system allows a robot to move at faster speeds than an RFID-only approach, albeit with reduced accuracy. Conversely, RFID is a well known and utilized technology that can provide high levels of precision, but it requires the robot to move at a slower speed in order to ensure that all tags are read. By combining the two, we have developed a system capable of moving at relatively high speeds when precision is not a priority and slower speeds when the robot is moving in critical areas or higher accuracy is needed. Experiments were conducted with our fusion approach, as well as with an omnivision camera included, to show not only the method’s efficacy, but also how the system can be simply further augmented.
1.1. Previous and Related Works
The use of radio-based systems for navigation is a well-developed field, and a WSN can be used in a similar fashion. In [1], a virtual potential field was created by combining information from nodes deployed around the navigable area for an indoor mobile robot. The radio signal strength intensities (RSSI) were combined with that field in order to allow the robot to navigate. In that same system, a servo-mounted camera combined with a CamShift algorithm-based method, [2], allowed the robot to estimate the human’s distance and angle relative to the robot. RFID technology is well known in robotic applications involving localization and navigation. A passive RFID-only method was presented in [3] for mobile robot navigation. The locations of previously read tags were combined with those of currently read tags to estimate not only the location, but also the pose of the robot, thereby navigating towards the goal.
Service robotics in general is a growing field, and to increase navigational accuracy, techniques combining image processing systems, laser range finders, sonar sensors, and inertial sensors have typically been proposed [4–7]. These approaches are not without their drawbacks, however. In the case of image processing, computational complexity is always a concern, as is image degradation due to obstacles and ambient light. Distance measuring sensors such as sonar and lasers can also suffer from instability due to anomalies or nonreflective surfaces in the external environment. In order to increase performance, low level sensors such as encoders can also be used. However, they often suffer from some compounding errors, which can lead to poor navigation when the system runs for extended periods of time. Another problem that can arise is that of growing costs. Laser range finders and cameras in particular can be expensive, and when several robots working in the same area are considered, the expense can grow quickly. Instead, creating an infrastructure to which other systems can be added or used in has been proposed [8–10]. By incorporating sensors into the building itself, the robots can be freed to use smaller, less expensive sensors while still performing the same tasks.
Additionally, these aforementioned sensors can of course be used to provide location awareness, or in other words, they provide the robot with a position estimate from which it can make decisions on which action it should take. Contextual awareness, which can of course to a certain extent include location information, goes further by determining how the robot should behave in certain situations, often creating a more natural behavior in terms of integration into environments where humans are living and working. For example, in [11], this concept of a more seamless integration is achieved by allowing the robot to determine contexts based on objects and their affordances through a Bayesian intent recognition system. We do not require such an algorithm to determine context, as at our current stage, we only focused on impacting the navigational behavior based on the context, and so the modifications in behavior are a result of the navigational algorithms. In [12], we see a tour guide robot that develops a context-based map in order to improve the tours provided. Our system does not perform SLAM, and again, it relies on the context to be determined based on the fusion algorithms. Contextual awareness is also useful in applications when a mobile robot is not involved. The system in [13] monitors energy usage in a home to develop certain “Energy-Prone” contexts. This shows how monitoring the human can provide clues on how a system, in our case a guidance robot, can modify its behavior to improve the quality and efficiency of the tasks being performed.
2. Wireless Sensor Network
Originally used in applications involving the collection of data from sensors spread over a large area and still used for various commercial and industrial purposes, wireless sensor network (WSN) technology also provides abilities useful to a mobile robot navigation system, as well as the capabilities for acquisition and monitoring of the surrounding environment. Moreover, being ad-hoc and scalable means that the navigable area for the robot can be simply extended by adding more nodes into the field. For the WSN nodes used in these researches, we utilized Crossbow Technology MICAz MPR2400 Motes, with a Chipcon CC2420, IEEE 802.15.4 compliant transceiver, operating in the 2400 MHz to 2483.5 MHz band, and an integrated Atmega128L micro-controller.
Much as in [14, 15], we utilize a WSN field to serve as the core of the navigation system, with WSN nodes placed throughout the field in a prescribed way, as determined based on our experience using the system. Nodes are, in general, placed at critical areas such as both concave and convex corners and long straight areas. The location of these nodes is stored in a map, which represents the navigable area of the robot. The map is created before run time and is accessed statically. It is not used in order to perform localization, but rather to determine the direction each node will “suggest” to the robot. The direction the node suggests is calculated as a unit vector, . The direction of is first calculated as the direction from the sensor to the goal. If an obstacle, such a wall, is detected on the map between the said node and goal along that direction, then the direction is next calculated towards a “safety point”, a subgoal which is precoded into the map. These safety points are also determined by the user, but they are generally placed where paths would intersect, such as the center of where two hallways cross. Ideally, the safety points are decided such that there is at least one visible from any node, ensuring a path can be calculated regardless of the goal and the robot’s initial position. The nature of the map’s encoding allows these checks to be performed using ray tracing, a technique commonly used in graphics programming to determine, for example, the ambient light in a 3D rendered scene. It should be noted that this map is not used to determine the path of the robot expressly; that is, no offline path planning is performed. At run time, the robot only accesses the map in the form of a table, with the nodes listing their suggested directions.
With these directions calculated for each node, the system performs navigation by combining them with weights in a summation. The weights are calculated based on a running average of the radio signal strength intensities (RSSI) of each node. The running average is a set of ten viable RSSI values, as read from messages received by the node mounted on the robot. Here, viable means a signal strength that is determined to be reliable, based on its similarity to the existing values in the set. A running average is maintained for all nodes, and each time the system recalculates the direction the robot should move in, it sorts these values from largest to smallest. Then, the direction the robot will move in, , is calculated by performing vector addition with the nodes having the two largest RSSI averages, as described in (1). While the nature of WSN-based systems and radiowaves in general can make it difficult to accurately estimate a distance based on instantaneously perceived RSSI, there is an analogue to be made between RSSI and relative distance; that is, a shorter distance will result in a stronger RSSI. As such, and by using the running averages, we can assume that in general the two nodes with the highest RSSI averages will be the two closest nodes to the robot. Using (1), we can then create a virtual potential field that will have a trend toward the goal. In the event the goal is set to a node, which is recommended as it results in a more reliable navigation, (2) is used, as the goal’s direction cannot point to itself. As we are at this point concerned only with incoming RSSI values, the message content of the WSN is inconsequential. They are programmed to transmit a counter at a rate of twice per second:
3. RFID System
Often used in robotics applications, radio frequency identification (RFID) systems are a well-developed and widely used technology. Their properties lend themselves well to tasks such as object recognition and tracking as well as navigation. Though there are many permutations, a passive RFID system generally consists of a reader, one or more antennas, a number of tags. For our research we selected the relatively compact and inexpensive Midrange Reader Module made by Texas Instruments which operates in the 13.5 MHz frequency. It also features an anticollision function, which makes it capable of reading multiple RFID tags simultaneously. As we intended to mount our antenna to the underside of the robot, we constructed a circular antenna capable of reading tags within 5 cm, vertically, and 10 cm, horizontally, of the center of the antenna. This was an ideal range for the RFID tags we deployed on the floor of our navigable area. As the robot passes over RFID tags, they are automatically read, and their serial numbers are checked against a table which holds their suggested direction, similarly to how WSN directions are stored. Conversely, RFID tag directions are goal independent, and so their directions are calculated only based on their deployment.
These deployments, presented initially in [14], can be thought of as variations on two basic types, strips and radials. For the researches presented here, we focused on the radial layout, for its versatility. We laid out tags in three concentric arcs, spanning 90° or 270°. The tags are spaced such that they are at most 15 cm apart, in order to ensure the robot will read at least one tag when it is over a deployment. As was said, the direction of each node is determined not by the position of the goal, but rather by the shape of their deployment. This is due to the conceptual difference in the contextual awareness derived from the WSN and RFID systems. The WSN is conceptually pushing the robot toward the goal; however, the RFID system is conceptually pushing the robot away from obstacles. As such, the direction of tags is calculated as vectors emanating away from the center of the circle their arc is from. This means that the calculation is simple once they tag deployments are placed.
4. Approaches to Contextual Awareness
4.1. Contextual Awareness from Relational Position
The main facet of WSN/RFID fusion is how the two incoming signals should be merged to determine the direction the robot should ultimately move in. In general, the system always moves in the direction provided by the WSN; however, when an RFID is read, its direction should be incorporated in order to avoid some obstacles or otherwise to increase precision. When a tag becomes readable, it can be inferred that the robot is near such an obstacle. However, the context of which direction the robot is moving in relation to the said obstacles can be used to improve how the two signals are merged. By comparing the suggested directions of WSN and the RFID, we can determine if the robot is inbound or outbound, to or from the obstacle.
The robot is deemed outbound from an obstacle if the suggested direction of the RFID tag, , is within 90° of , the direction suggested by the WSN. Contextually, we can infer the robot is moving away from an obstacle. As such, the robot is most likely not in immediate danger of colliding with the obstacle, and so the slight modification of the WSN suggestion can be accomplished by performing vector addition with the two suggestions, as described in (3). However, if is not within 90° of , contextually we can infer the robot is inbound or is approaching an obstacle. In this case, the likelihood of colliding with the obstacle is larger, and so a larger course correction is preferable. To do this, we calculate the two directions perpendicular to the tag’s suggested direction and choose the direction that is the closest to the WSN’s suggestion. This is more readily understood by considering the suggested directions as numerical angles instead of vectors and , respectively. These perpendicular angles are compared as in (4). If the comparison in (4) evaluates to true, (5) is used to calculate . If instead the values are evaluated as true for (6), then (7) is used:
4.2. Contextually Aware Speed Adjustment
The next form of contextual awareness that can be derived from the WSN/RFID fusion system is an awareness of how quickly the robot should move in relation to critical areas in the field. As has been said, nodes and tags are placed in critical areas, such as corners or near large, static obstacles. As such, when even the closest WSN nodes are deemed to be fairly distant, that is, their RSSI are low, the robot can assume that it is relatively distant from walls and other obstacles. This means the robot can increase its speed, based on the context. By modifying its speed using (8), not only will the robot approach its maximum speed in open areas, but it will also slow down as the RSSI increase, as the context implies that the robot is nearing a critical area and may need to perform some higher precision navigation. The slowing also ensures RFID tags will not be missed, a common problem with RFID tags if the antenna moves over them too quickly:
In (8), represents the highest RSSI of field nodes, and represent the maximum and minimum speeds the robot should move at, is the maximum value for signal strength expected from the field nodes, and is the speed the robot will be set to move at. Although it is rare, it is possible for the RSSI of a node to temporarily spike beyond what the field has displayed previously. The values of and are therefore used to restrict the robot from performing in a manner that could potentially damage it. In the event a tag is read, the robot is set to move at , regardless of what speed the evaluation of results in.
The ideal values for and , of course, will depend on the RFID reader, and to some extent the robot platform, used in the system. In order to determine the correct values for our setup, we performed experiments running the robot in a straight line, approaching a WSN node. Along this path, it passed over an RFID tag. We ran the robot at various speeds and tested at which speeds the tag was readable. Based on these experiments, our robot was able to read messages from the WSN at speeds of approximately 42 cm/s, the maximum speed of the robot with its current payload. Predictably, RFID tags became unreadable at this speed. The maximum speed at which a tag was readable was approximately 17 cm/s. At this speed, however, the tags were only readable about of the time, and we determined this was insufficient as the RFID tags are used to prevent the robot from colliding with large obstacles. In the interest of safety, we set to 8 cm/s.
4.3. Contextual Awareness through Vision Subsystem
When robots are introduced into working and living spaces of humans, their need for contextual awareness becomes more critical. This increasing need for flexibility was one of the aspects we considered in developing this WSN/RFID fusion system, in the form of the ease with which other subsystems could be incorporated into it. For example, in order to extend our navigation robot to the task of a guide robot, we can add a vision system to provide a new level of contextual awareness to the robot. By using an Artray 200MI CMOS camera mounted vertically under a hemispherical mirror, we created an omnicamera capable of viewing the entire 360° area around the robot. Running a modified version of the well-known CamShift algorithm, [2], we could not only track a human user’s relative angular position to the robot, but also gain some estimation of the robot’s relative distance. As shown in the top portion of Figure 1, the omnivision camera provides a circular view around the robot. The human can be seen interacting with the robot via the touch screen. In the bottom portion, we see the above image after processing. The red, outer frame is the bounding box around the user, in this case, the user’s clothes. The yellow, inner frame is the sampling region for the CamShift algorithm. The most basic implementation of the CamShift algorithm works by first selecting a section of a frame, often selected by color, and then creating a bounding box around it which is used to minimize the search area of subsequent frames. This allows tracking to be processed more efficiently, as we are not expecting the human to jump from one side of the robot to the other in an instant. To further improve the performance, we use only the color within the yellow box of the previous frame to search the new area. This allows changes in ambient lighting to occur without the system losing the tracking of the human. As this experiment was conducted in a closed environment with only one user, the color of the user’s shirt was sufficient for tracking. As the guidance proceeds and the human’s relative distance changes, the size of the outer bounding box will also change; the inner box however remains constant. Continual sampling of the inner frame allows the vision system to continue tracking despite changes in ambient lighting. Based on the ratio, (9), between the current box size, size, and when the human was interfacing with the screen, , we can also make some contextual inference about how far the human is from the robot and by extension how the human is behaving in relation to, the robot within the context of a guidance task. In this case, we set the robot’s speed, , to zero when the ratio is below some limit, which we determined based on the size of our experimental space. Otherwise, the robot’s speed is calculated as described in (8):
(a)
(b)
In Figure 2, we see an overview of how the navigation takes place. In practice, the WSN, RFID, and camera subsystems run in concurrent processes on the robot, but their values are evaluated in the order as shown. First, the navigation begins with the initial ratio being set when the user interacts with the robot to set a goal. Then the navigation proceeds by first calculating the suggested direction of the WSN subsystem after which the presence of RFID tags is checked. If one or more tags are detected, the robot adds the directions of each via the fusion algorithm explained above; otherwise, it skips this step. Finally, before using the new suggested direction to adjust the robot’s speed and heading, the relative distance between the robot and human is checked by the vision subsystem. If the human is deemed to be too far as described above, then the process enters a loop until this distance is brought under the threshold. In this simplified example, the robot stops and waits for the human to return, but other behaviors could be performed at this step. After the robot’s speed and heading are adjusted, the robot checks if it has reached the goal, in which case it stops and navigation ends; otherwise, it again calculates the direction to the goal and so on.
5. Experimental Validation
In order to demonstrate the efficacy of the proposed methods, we developed two tests: one for the vision and positional context and the other for positional context and speed adjustment. For both experiments, we used a prototype robot platform called Chamuko, Figure 3, which is built upon the MobileRobots Pioneer 3 DX differential drive base. The P3-DX is equipped with sonar and bumper sensors, but neither of them was used. Instead the system relied solely on the system we have presented in this paper. The P3-DX used internal sensors to determine its pose, and a WSN node was mounted to the front bumper to interface with the WSN field. As described above, the RFID antenna was mounted to the underside of the robot chassis. For the first experiment, the omnivision camera was installed on top of the robot, behind the display.
5.1. Experimental Conditions: Speed and Positional Contexts
In this iteration of experiments, we set up our navigation infrastructure in an actual hallway of an existing building. The robot is set at a far end from an elevator, with a relatively wide open area in between, Figure 4. The open area before the elevator was chosen in order to highlight the speed changes as the robot approaches critical areas. Once the robot successfully navigates to the elevator, a tag mat placed in front of the elevator notifies it that it must wait for the doors to open. Once the elevator doors have opened, a WSN node inside the elevator, Figure 5, is sensed by the robot. The signal from the node inside the elevator is blocked completely by the doors, and so is only detectable when the doors are open. This alerts the robot to move forward into the elevator, where the RSSI of the node inside is used to stop the robot. In the second experiment, RFID tags were deployed in 90° arcs. However, as this experiment was not conducted in a controlled environment but rather in a real world setting, we mounted the RFID tags to easily installed vinyl mats, Figure 6.
(a)
(b)
5.2. Results
In Figure 7, we can see screen captures of an exemplar run of the system. The robot successfully crosses from the hallway to the elevator and then is able to enter once the doors are opened. Figure 8 shows the path taken in the same run. It can be seen that initially the robot moves toward the elevator door, nearly colliding with the wall. However, this is easily corrected by the RFID tag mat that is placed there and the robot smoothly navigates around it to line up with the elevator. After waiting for the doors to open, it then successfully enters and stops inside the elevator. Along the right side of the image, we can three “zones” marked. These relate to the zones marked in Figure 9, a graph showing the robot’s rate of speed that was commanded to the robot throughout the navigation. In reality, the robot speed fluctuates when the robot turns, resulting in temporarily slightly lower values than those depicted in the graph. Additionally, the robot comes to a full stop in front of and inside the elevator, which is detected by RFID tags. These are not included into the speed calculation and as such are not reflected in the graph.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
5.3. Experimental Conditions: Vision and Positional Contexts
In this experiment, we tested two conditions: could the robot detect when the human stopped following correctly and could the robot use its positional awareness to smoothly navigate through a tight space. In a controlled space, we arranged six tables, Figure 10, in staggered sets of two, such that a path between them requires consecutive sharp turns. The gap between a pair of tables was 90 cm, a width that would not be navigable using the WSN alone. Radial deployments of tags were arranged as described above, with an entrance and exit set for each obstacle, resulting in a total of 156 RFID tags being used. Also placed on either side of the entrance to a gap and at the start and goal were 8 WSN nodes. In this experiment, the robot’s speed was kept at 12.2 cm/s, in order to ensure all tags were read. To test the contextual awareness of the human’s behavior in relation to guidance, the human user intentionally erred in following the robot, failing to turn after passing between the first set of obstacles.
5.4. Results
In the environment described above, we tested the system with both cases; the human following correctly and making an error. In both cases, the robot was able to successfully navigate to goal. It also correctly stopped to wait when the human user did not initially make the turn into the second set of obstacles. In Figure 11, screen captures can be seen of an exemplary run of the system when the human has erred. Figure 12 shows the corresponding path taken by the robot. The curved line represents the path of the robot, and the dashed line shows the error the human makes in following the robot. When the human walked too far, the robot initially continued on, until the distance between them exceeded the threshold and the robot stopped at the point marked by the “X.” Once the human returned to a prescribed distance from the robot, it resumed navigation until arriving at the goal. In Figure 13, we can see values estimated from a video recording of the exemplar run. The estimated distance between the human and the robot, blue line, from the same run is depicted. When the human walks straight instead of turning, the distance exceeds the threshold, red line, and the robot comes to a stop, to wait. We can see that once the human’s distance comes under the threshold again, the robot proceeds. The green line represents the robot’s estimated speed.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
6. Discussion on Experimental Results
The focus of this paper was to show the contextual awareness that can be produced from the WSN/RFID fusion navigation approach. Additionally, it was desirable that the system be easily augmented with other systems, as a navigation system is often only one aspect of a robot’s task.
The contextual awareness of not precisely how far the robot is from a wall, but rather its approaching of a critical area was also used to adjust the robot’s speed. This means that it is not simple location awareness. Looking at Figure 9, we can see a clear pattern as the navigation proceeds. The robot’s speed gradually decreases over time, with it mostly moving at its minimum speed in front of the elevator. This is expected, as the area in front of the elevator is a critical area and highly populated with RFID tags. The instantaneous drops in speed visible are caused either by tags being read or the robot slowing itself in order to turn. Using the RSSI of the WSN node inside the elevator, we also showed how the presence of signal, as opposed to the signal itself, can be used to determine a context. In this case, the presence or absence of signal from the WSN node can be used to determine the state of the elevator doors.
In the second experiment, we showed how such an addition of an omnivision subsystem could be used to modify the navigational behavior of the robot. As the robot moved through a series of obstacles, it was capable of integrating the tracking information from the camera, thereby performing actual guidance. That is to say, the robot was able to ensure the human was correctly following it. In Figure 12, the overall smoothness of the robot’s path shows that the video system has no effect on the navigation other than to provide a contextual awareness of whether or not the human is following. This seamless integration shows the flexibility of the WSN/RFID system to be combined with other types of sensors or inputs, such as laser range finders or speech recognition systems. Additionally, the contextual awareness of the robots position in relation to obstacles was also shown to be useful in processing the fusion of the respective WSN and RFID signals. While some would argue that this is location awareness, we consider this not to be the case, as the actual location of the tag is inconsequential, but rather the conceptual state of whether it is moving toward or away from an obstacle is what we can derive by comparing the WSN and RFID signals.
7. Conclusion and Future Work
In this paper, we presented contextual awareness as displayed in a WSN/RFID fusion approach to indoor, mobile robot navigation. Capable of providing fast, general navigation in open areas, it can slow down for more precise navigation in narrow spaces. The inclusion of contextual awareness to the WSN/RFID fusion navigational scheme improves the reliability of the system and its ability to incorporate a behavioral pattern into its task. In the experimental evidence, we showed how contextual awareness from a new subsystem can easily be incorporated in the form of an omnivision camera subsystem. A 360° area around the robot is monitored using an omnivision camera, and the human user is detected and tracked, allowing the system to be aware of the human’s relative distance. With this contextual awareness, the system was able to perform not only navigation, but also true guidance, as the robot stopped and waited when the human strayed too far away. In both experiments, the relation between WSN and RFID inputs was used to determine the robot’s positional relation to obstacles in the environment. This contextual awareness is used to determine how the inputs should be fused. When a robot is determined to be moving away from an obstacle, it can be assumed that it is not in immediate danger of a collision, and so only a small modification of the WSN’s suggested direction is sufficient. Conversely, when the robot is approaching an obstacle, a larger modification is necessary in order to ensure no collision occurs. The final contextual awareness presented in this paper is that of adjusting the robot’s speed in relation to its distance from nodes. When nodes are relatively far away, it can be assumed that no static obstacles exist near the robot, and so the speed can be increased. Of course, as the robot approaches the nodes, and by extension obstacles, then the system reduces its speed, not only improving handling, but also ensuring that any RFID tags that may exist in the area will be reliably read. These last two capabilities are displayed in a second experiment, in which a robot autonomously approaches and then enters an elevator.
The future plans for this system involve the furthering of the system’s contextual awareness and its overall usefulness in a number of applications. By including a higher awareness of where the robot is conceptually, we hope to develop methods to further modify the navigation behavior in relation to external stimulus, from not only humans in and around the navigable area, but also from other robots and the environment itself. We also plan to incorporate a method by which WSN and RFID layouts can be automatically determined by software, further simplifying the installation process into existing environments.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.