Abstract
Grasping is a useful ability that allows manipulators to constrain objects to a desired location or trajectory. Whole-arm grasping is a specific method of grasping an object that uses the entire surface of the manipulator to apply contact forces. Elephant trunks and snakes and octopus arms are illustrative of these methods. One of the greatest challenges of whole-arm grasping in poorly defined environments is accurately identifying the perimeter of an object. Existing algorithms for this task use restrictive assumptions or place unrealistic demands on the required hardware. Here, a new algorithm (termed Octograsp) has been developed as a method of gaining information on the shape of the grasped object through tactile information alone. The contact information is processed using an inverse convex hull algorithm to build a model of the object’s shape and position. The performance of the algorithm is examined using both simulated and experimental hardware. Methods of increasing the level of contact information through repeated contact attempts are presented. It is demonstrated that experimentally obtained, coarsely spaced, contact information can result in an accurate model of an object’s shape and position.
1. Introduction
To grasp an object is to seize or hold it, thereby constraining it to a desired position or trajectory. Industrial robots implement finger-tip grasping (based around biological inspiration from humans) to perform tasks on objects that are normally well defined, often rigid, and always considerably smaller than the robotic manipulator. Robotic systems are increasingly being considered for new application areas where the operating requirements are more complex and less well defined; consider the use of robotic end-effectors for the assembly of complex machines with components of varying size and consistency. For applications such as these, an alternative approach to robotic grasping is required. Whole-arm grasping is an alternative to finger-tip grasping [1, 2] that is used by animals such as elephants, snakes and octopuses. Whole-arm grasping uses the entire surface of the manipulator to provide contacts between the object and manipulator. This allows the distribution of grasp forces and an increased surface area to aid the grasping of smooth objects. For example, consider the scenario of Figure 1 where a 10-link multiple section robot arm has encircled an object and is applying grasping forces. Each link has the capability of exerting forces at various points along the objects body. The grasping control aim is to exert the minimum forces whilst grasping the object.
(a)
(b)
At each of the contacts between an object and manipulator there is a force applied to the object . This force causes a torque around the centre of mass. The torque and force can be combined into a single vector called a wrench, as shown in where is a vector from the centre of mass of the object to the contact point . The set of all wrenches forms the wrench space.
The quality of a grasp is a measure of its ability to reject all possible external wrenches acting on the object, one measure often used being the wrench ball measure [3]. The unfortunate drawback of wrench ball method is that it does not give any indication of stability. If a method simply attempts to maximise the measure, then points on extremes of facets are chosen. This risks the possibility of unstable grasps due to positioning errors, which may place contacts past edges thereby drastically changing the grasp. This problem is mitigated using whole-arm grasping as the idea is to apply lots of contact, which provides more redundancy in the grasp.
An accurate definition of the perimeter of an object is required so that the locations and magnitude of constraining forces can be identified. Existing approaches make additional demands in terms of equipment and processing power in order to be effective. In general, they require additional equipment, such as cameras, graphics cards, or force sensitive skins, and/or are computationally expensive. Current methods assume the availability of a tactile sensor skin that stretches over the surface of the manipulator; an example method is by Asano et al. [4] who use the contact location and force information to dynamically manipulate an object. Unfortunately, the fully force sensitive skin has yet to be developed, and so these methods remain simulations. This may yet change due to new material research. One such material is ionic electroactive polymers as used by Shahinpoor [5] and, more recently, Biddiss and Chau [6] who both describe using IPMCs as touch sensors. The tactile information can be processed to obtain a model of the shape and position of the unknown object. Methods for discovering the shape and location of an object through contact information alone must be able to intentionally, systematically, and reactively contact the object. The control algorithms must react to the contacts such that the positional disturbance of the object is minimised. These reactive methods can be used to fully encircle an object and gain information about the surface of that object.
Busch [7] describes three reactive methods that could be employed to grasp an object by using whole-arm grasping. The first method is the same method of grasping as McMahan et al. [8], Walker et al. [9], and Desai et al. [10]; Busch calls this method Zorro. The method contains two competing controllers, one of which attempts to make the end-effector of the robot circle around the object, while the other attempts to keep the body of the robot away from the object. The combination of these two controllers causes the arm to tighten around the object as the end-effector circles the object. The second method is the Follow-the-Leader method. This method operates by initially making the end-effector contact and follow the surface of the object, while the remaining links follow the path the end-effector took along the surface of the object. This approach is based on the work by Reznik and Lumelsky [11]. Both approaches require the availability of a force sensitive skin to detect contacts, contours, and forces. The last method is called Lasso and requires the use of sensors covering the entirety of the arm that can detect the distance to any object normal to the sensor. The Lasso method encircles the object without making contact with it, meanwhile, the object’s structure is analysed for grasp planning purposes with the hypothetical sensors. All three of these methods analyse the structure of the object on the fly and reactively move to adapt the grasp to the object. Unfortunately the Lasso and Follow-the-Leader methods require additional complex sensors, whilst the Zorro method requires the knowledge of the object’s position and shape.
A new method that does not require additional sensors other than those needed for actuation in a closed-loop system and also does not require excessive assumptions/prior knowledge may therefore be beneficial. The contributions of this paper are in demonstrating how to interpret and combine the coarse contact information provided by such a method, through a novel and systematic object exploration algorithm. The approach taken here is as follows. A snake-like serial-link manipulator is first controlled to encircle a free-moving object. This is performed using a joint-based impedance control scheme that stops the links from moving as soon as contact is detected. Multiple attempts at encircling the object are made that include placing the manipulator in a different reference position relative to the object. The contact information obtained from these attempts is preserved and added to the gathered data. This data is then passed to an algorithm called the inverse convex hull. The algorithm filters the contact information from the innermost links and forms a continuous shape that is representative of the object being grasped from them.
The rest of this paper is organised as follows. Section 2 details a newly proposed novel method for gathering contact information. Section 3 describes a method for analysing that contact information to model the shape and position of the object to be grasped. Section 4 describes the experimental equipment used to examine the performance of algorithms, which is demonstrated and discussed in Section 5.
2. Gathering the Contact Information
A new method inspired by the motion of an octopus’s arm has been developed that can reactively and systematically interact with the object to gain the required contact information to determine its shape for grasping. An octopus has an amazing amount of flexibility, but it only uses relatively simplistic and planar movements [12]. One method the octopus has for grasping an object begins by bringing the arm into contact with the object a small distance down from the tip of the arm. Progressively more links of the arm are then brought into contact with the object until the object is entirely encircled [9]. This method can be thought of as a combination of the Zorro and Follow-the-Leader methods. The method does not use the object’s shape information but instead relies on its octopus-like movements that can result in a partial envelopment of the object. This in turn will provide the required contact information to analyse the object’s shape, although multiple attempts at contacting the object may be required to gain enough information to accurately model the object completely. The method proposed here mimics this process and has been termed an Octograsp in homage to its inspiration.
In the following work, a fixed base serial link manipulator with rigid links is used to demonstrate the Octograsp algorithm. The algorithm initializes the serial link manipulator by aligning all the joints to a straight line. During the wraparound stage, only one joint of the serial link manipulator is moved at a time, until the object is completely encircled. Moving only one joint at a time may improve efficiency in terms of the amount of contacts being made, while also preventing the arm from displacing the object during the motion. This is illustrated in Figure 2(a) where all of the links have been moved simultaneously, and a gap results between the object and arm. In Figure 2(b) only the base joint has been moved causing the rest of the arm to sweep across the environment. This method results in a flat contact between the object and arm.
(a)
(b)
A feed-forward computed torque controller using the Recursive Newton Euler dynamics with a secondary PID controller for disturbance rejection is used to decouple the joints and achieve the desired torques. The manipulator itself is controlled using a joint-based impedance controller that is described in detail in Section 5. It has been shown in previous work [13] that the same methods outlined in this work operate well without the additional torque sensors; they are used here to improve the performance of the impedance controller.
2.1. Detecting the Contacts
The joint angles can provide valuable information on the perimeter of an object. At each instant in time, the desired and actual positions of the joints are known. If the error between them breaches a normal operating error threshold, then it can be assumed that a contact has occurred and react accordingly. This type of process is called a model reference controller [14].
The initial experiments using the Octograsp algorithm showed that although the object can be encircled with a snake-like arm, very limited information regarding the contacts is available with one attempt. This is especially the case when using the highly simplified experimental robot described in Section 4. To compensate for this, and to gain more information, multiple attempts at grasping can be made. Two methods are proposed towards this. (i) The first involves moving the base of the arm to a different initial position relative to the object, resulting in new contact information. There may, however, be errors in the global movement or measurement of the base. (ii) The second alters the contact positions of the links by limiting the rotation of the baselink during encirclement; a reduction in the base link rotation restricts the reach of the arm and thereby repositions the links. The amount the baselink which is restricted is referred to as a back amount in the remainder of the paper (e.g., a back amount of 0.5 radians means the base link stopped rotating 0.5 radians from an unrestricted encirclement). The technique of restricting the baselink will be referred to from now on as the multiple orientations method. Once contact information has been obtained, a method is required to process this information to gain a model of the shape and location of the object.
3. Interpreting the Contact Information
A numerical simulation was created to develop algorithms to process contact information into object perimeter information. Thirty rigid independent links were randomly placed on the perimeter of a simulated square. It can be seen from Figure 3(a) that the absence of links passing through the central part of the figure suggests that the shape is square. A method will now be described that can model the shape of an object by inferring it from this absence of links.
(a)
(b)
3.1. Inverse Convex Hull
The innermost parts of the links represent the object circumference. An algorithm is required to indentify the contact points and convert them into an object perimeter. The commonly used convex hull algorithm produces a perimeter that encloses a set of points. The problem described here is the opposite to a convex hull, as a perimeter that contains no points (links) is required. This problem can be solved using the inverse convex hull algorithm [15], which selects a set of points that forms a shape that contains no points. The inverse convex hull algorithm first inverts the distance of discrete points from an origin, following which the convex hull of that set is found. The origin point needs to be located within the object, ideally as close to the centre as possible. In this work, the mean of the contact locations gives the origin, since the mean returns a position within the object as long as the arm contacts the object evenly across its entire surface. The corresponding points from the original set are the inverse convex hull set. Since the inverse convex hull processes discrete points, it requires discrete link positions. This involves converting the continuous links into a set of individual points. The number of points that each link is converted into, that is, the level of discretisation, is a user-controlled parameter. Figure 3(b) shows the discretisation of the links, using 10 points per line.
Each of the points is represented as an -dimensional vector that specifies its position relative to an origin; the origin needs to be located within the object. In the 3D case, would be represented as
The distance of each of the points from the origin is
To invert that distance, such that the distant points are now the closest and vice versa, the distance is raised to a negative power which is another user-controlled parameter. The inverted scalar distance is calculated as
Finally, the direction of each point needs to be preserved such that only the distance is inverted. The convex hull is then generated from those inverted data points , where
This process has been applied to all of the points in Figure 3(b) with the result shown in Figure 4. Each of the points are at the same angle as they originally were, but the points that were the closest to the origin are now the furthest and vice versa. The shape that this conversion forms depends upon the value of . Low values, such as 0.05 as used in this example, tend to squeeze the points together forming a circular shape (as seen in Figure 4).
The final step involves finding the convex hull of the inverted points and matching them to the corresponding points within the original set. Figure 5 shows the original discretised points and the polygon that is the inferred shape of the object. It can be seen that the inferred object, which is a square, is not 100% accurate, due to the lack of contacts at certain points along the length of the square. Therefore, the aim of the algorithm is to gain as much surface contact over the object as possible. However, even with full contact, the algorithm has two problems. Firstly, even with perfect information, an amount of rounding at corners will occur, the precise amount depends on the value of . Secondly, modelling of concave objects will be troublesome as the inverse convex hull algorithm tends to smooth over them. Further discussion of these problems and their consequences are presented later in this section.
3.2. Inverse Convex Hull Parameters
The two user-controlled parameters of the inverse convex hull algorithm, level of discretisation and , will each have an effect on the quality of the resulting object shape generated. Investigations were conducted to examine the effects of each parameter and to determine their optimal values. Three shapes were used in each of the investigations: an equilateral triangle, a square, and a regular pentagon. A circle of diameter 2 units encloses the shapes and therefore lengths of their sides were 1.7321, 1.4142, and 1.1756 arbitrary distance units, respectively. Equation (6) is used to measure the accuracy by returning a percentage error () with zero being the best and lowest possible outcome as follows: where is the area of the actual shape, and is the area of the generated shape, is the area of the intersection between the actual and generated shapes. The area of intersection between two shapes is difficult to calculate for the case where the two shapes can be any size and shape. Therefore, an algorithm [16] was used that breaks the objects into very small discrete elements that are analysed for whether the shapes overlap, intersect, or do not overlap in that element. The sum total of the area of those that overlap and intersect approximates the area of overlap. Decreasing the size of the discrete elements increases the accuracy of the resulting intersection calculation.
3.2.1. Investigating the Discretization Parameter
The set of previously mentioned shapes was analysed with the inverse convex hull algorithm several times with varying levels of discretisation. To isolate the discretization parameter from errors in measured input data, the shapes themselves were used as the input to the algorithm instead of the data obtained from robot contact in simulation. was held at a constant value of 0.05 for the duration of this experiment.
The percentage area error for the three shapes when the level of discretisation varies between 10 and 1000 points per segment of the shape is shown in Figure 6. The area error is very low for all shapes and discretisation levels since perfect information is being fed to the inverse convex hull algorithm. It can be seen that the error is quite variable in the region between 10 and 200 points but settles down to a consistent low value beyond that region. A value of more than 100 and less than 300 points per segment is therefore recommended since lower values are not consistent, whilst higher values increase the amount of data that needs to be processed. Figure 7 shows an approximately linear correlation between the level of discretisation and the computation time. This is confirmed by the fact that the objects with fewer sides consume lesser time during computation.
3.2.2. Investigating the Parameter
The effect of varying (see (4)) while keeping discretization level constant at 100 was investigated, with the same three objects serving as input to the inverse convex hull algorithm. Figure 8 shows this effect on the area error for values between 0.05 and 1. The graph’s trend in general shows an increase in error as is increased. This is due to an increased amount of rounding at corners.
4. Experimental Platform
The proposed algorithm was evaluated using experimental equipment. This stage is crucial as a large proportion of work in the area of multilink snake-like robots for manipulation is restricted to simulations due to ideal assumptions and technology demands. The experimental equipment is designed to demonstrate whole-arm grasping with coarse contact information. The manipulator consists of four links and contains minimal sensors. The hardware has been designed to be modular such that it can be reconfigured with a minimal effort and allows for the possibility of correction if a particular configuration is found to be lacking. The experimental manipulator (shown in Figure 9) consists of four identical modules with each module consisting of a motor, gearbox, potentiometer, and torque sensor. The control of the arm and the acquisition and processing of contact data are performed using LabView on a standard desktop PC.
The results obtained from the experiments are 2D, although the algorithm can process 3D information by exploring multiple planes using one of the two methods explained in Section 2. 2D information may be sufficient for grasping purposes; the only disadvantage is that potentially stronger grasps in alternative planes will not be discovered. However, there is generally little difference between the strength of good and optimal grasps [17].
5. Joint-Based Impedance Control
Individual joints are controlled by an impedance controller. Impedance control attempts to implement a dynamic relationship between manipulator variables such as force and end point position, rather than controlling each of these variables alone [18]. A conventional impedance controller [19] defines a relationship between the position and velocity error of the end-effector and the force the end-effector applies (and therefore the torques at each joint). The joint-based impedance controller operates instead in the joint space allowing for a dependable contact behaviour at the cost of having to manually calculate a trajectory. This is a biologically inspired scheme that is capable of mimicking muscular compliance. As the error in tracking a certain response deviates from the expected behaviour, the impedance scheme reduces the amount of torque applied, resulting in reduction of overall torque of the joint. Hence less force is exerted by the link tip, and this reduces the risk of displacing the object. Furthermore, having such a scheme in place allows for smooth transition between the contact and not contact states and enables smoother actuator loads with minimum jerk.
Under ideal conditions, the computed torque controller enables joint space trajectories to be tracked very closely. Closed-loop torque control response of one of the joints to a chirp signal of magnitude 1 Nm, varying in normalized frequency between 0 and 0.05 Hz, is shown in Figure 10. The desired torque is accurately tracked with some phase lag and friction effects.
A closed-loop torque controller for a spring constant of 20 N/m was implemented at the joint tip. End-effector force for a single link was measured using a sensor and compared to the desired force generated by the impedance scheme. The corresponding stiffness from the relationship of force and position was computed. The results show a good agreement between the theoretical and experimental responses as seen in Figure 11.
(a)
(b)
6. Evaluation of Performance
The performance of the Octograsp algorithm at analysing the shape of an object was examined by varying the algorithm parameters and comparing the generated model of the object to the actual object. It must be noted that due to the limited number and size of the links, the objects being examined could not be fully encircled and touched. As a result, the evaluation of modeling will only focus on the sides being contacted. The accuracy of the generated model when multiple attempts at contacting the object are made from the same relative position (method 2 from Section 2.1(ii)) was examined. The parameters that were controlled were (i) the number of attempts at encircling and (ii) the amount the base joint of the robot moves back after each attempt (back amount). A square object was positioned and orientated the same for each run of the experiment with camera-based tracking used to verify the exact location. The inverse convex hull algorithm was used to generate a model of the object from the obtained contact information and compared to the known location and shape of the object to find the accuracy error.
An example of the contact information provided by a typical Octograsp is shown in Figure 12(a). The example shows four attempts at grasping the object with a back amount of 0.1 radians. The four-end configurations of the arm (one sequence of lines with dots for each attempt) are shown in relation to the actual object (dashed lines). The path of the centre of the free-moving square object (obtained from camera-based tracking) shows the displacement of the object over time due to contact. The major disturbance to the object is caused when the link of the arm fully coincides with a face of the square. As a result, later attempts by link 4 pass through where the square shape should be positioned, since the square is no longer present having been pushed by previous attempts. The resulting model generated by the inverse convex hull algorithm is shown in Figure 12(b).
(a)
(b)
Figures 13(a)–13(d) show how the shape is progressively builtup and improved upon after each successive attempt. The displacement of the object causes the right hand side to be underestimated while the lack of contacts on the other sides causes those sides to be overestimated in size; the overestimation arises from a lack of information that could be readily corrected through a longer snake arm or multiple robot positions. The important result is that the side of the square encircled accurately represents the objects original shape.
(a)
(b)
(c)
(d)
The object was encircled from two different base positions, with the second position rotated by radians from the first position. There were four attempts per position with a back amount of 0.1 radians. It was found that the accuracy error reduced significantly after each position change attempt, dropping by about 30%. Analysing the object from the second position greatly improves upon the accuracy and compensates for the displacement of the object.
The object disturbance over time for the first four encircling attempts is shown in Figure 14. Each contact produces a small but noticeable increase in object disturbance. At roughly 55 seconds, there is a prominent spike that is the cause of the increase in model error. The overall disturbance is however less than 16 mm after four encircling attempts. This is a significant result for the proposed joint-based impedance control scheme given that the robot weighs 1.672 Kg and the object only weighs 200 g.
The information provided by the eight attempts at contacting the object and the resulting model produced by that contact information are shown in Figure 15. This result illustrates that the accuracy of the produced model can be increased (in comparison to results from Figure 12) by contacting the object multiple times from different positions and orientations. Object displacement can be limited by making the manipulator lightweight in comparison with the object and also by reducing the manipulator’s speed of operation. The amount of contact information gathered can be improved by increasing the number of links and reducing their size.
(a)
(b)
6.1. Improvements through Increased Number of Links
To demonstrate the effectiveness of the algorithms under representative conditions, a simulated robot that consists of greater number of smaller links was analysed. The simulation setup is shown in Figure 16. The dynamic simulation environment was created in MSC Visual Nastran 4D and controlled using Matlab-Simulink.
The Octograsp algorithm was used to encircle the cuboid object from three positions, each rotated by degrees from each other. Two attempts at encircling the object with a back amount of 0.7 radians were made in each position for a total of 6 attempts. As before, the resulting contact information was processed using the inverse convex hull algorithm to determine the object’s shape and compared with the known shape, position, and orientation of the object. The results were similar to that of the experimental situation in that there is a significant decrease in representation accuracy error when encircling from the second position (reduced from 19.9% to 9% on the third attempt). The final error for this experiment is only 4% and demonstrates that an increased number of links of reduced size provides for higher resolution data.
Figure 17 shows the contact information provided by each encircling attempt and the resulting object representation from the simulation. Having several small links in comparison to the length of the side of the object allows the arm to contact much more of the straight sides of the object in parallel, which in turn provides a large amount of useful contact information that can produce an accurate representation of the object.
(a)
(b)
7. Conclusion
In this paper, methods for building a model of an object from tactile information have been presented. The Octograsp algorithm allows snake-like robots to gain contact information regarding the shape of an object. The inverse convex hull algorithm can then process that contact information in order to determine the object’s shape and position. Both of the methods require no external sensors, such as touch sensors, but also do not preclude their use. The method is generic enough to work on any kinematic serial chain (snake-like) robot. In general, the Octograsp and inverse convex hull algorithm can generate accurate models of objects without the need for sophisticated sensors. The algorithms were implemented experimentally as well as in simulation, with the experimental kit providing coarse information and demonstrating the feasibility of the method for real-world application, while the simulation was used to demonstrate the accuracy of the method based on drawbacks identified with the experimental equipment.
It was found that the best approach was to select as many positions around the object as possible but only make two attempts in each position, one where direct contact is made and another with as large a shift in the base joint as possible. This ensured that the largest amount of different surface contact is made, with minimal object disturbance.
A drawback of the proposed approach is that concave objects cannot be accurately represented with these methods. However, these concave elements are often not required to obtain a good grasp and may in some cases be too small in comparison to the robot’s size to fully utilise. The second issue is that of object disturbance; a minor disturbance has a negative effect on the resulting accuracy of the object representation.