Feedforward Neural Network for Force Coding of an MRI-Compatible Tactile Sensor Array Based on Fiber Bragg Grating
This work shows the development and characterization of a fiber optic tactile sensor based on Fiber Bragg Grating (FBG) technology. The sensor is a 3 3 array of FBGs encapsulated in a PDMS compliant polymer. The strain experienced by each FBG is transduced into a Bragg wavelength shift and the inverse characteristics of the sensor were computed by means of a feedforward neural network. A 21 mN RMSE error was achieved in estimating the force over the 8 N experimented load range while including all probing sites in the neural network training procedure, whereas the median force RMSE was 199 mN across the 200 instances of a Monte Carlo randomized selection of experimental sessions to evaluate the calibration under generalized probing conditions. The static metrological properties and the possibility to fabricate sensors with relatively high spatial resolution make the proposed design attractive for the sensorization of robotic hands. Furthermore, the proved MRI-compatibility of the sensor opens other application scenarios, such as the possibility to employ the array for force measurement during functional MRI-measured brain activation.
Tactile sensors transduce quantities, such as force, pressure, temperature, vibration, and slip, through the physical interaction with the object.
The first interest in touch-sensing technology arose between the end of the seventies and the beginning of the eighties, when researchers started investigating its application in the field of robotics [1, 2]. Tactile sensors were considered crucial to provide information about the interaction between robotic tools and environment, leading to a growing interest in systems able to reproduce the sense of touch: for instance, if a task is performed using a robotic manipulator, sensory feedback similar to those possessed by humans is essential to explore and interact with the environment (including humans and objects). Therefore, huge efforts were made to design and develop tactile sensors that are specific for force and pressure measurements during contact events.
The area of tactile sensing interested many application fields since 1999, as reviewed by Lee and Nicholls : in particular, biomedical applications were considered as the new domain where tactile sensing was likely to play a role. Along this pathway, during the last decades touch-sensing technologies attracted interest within various biomedical scenarios [4, 5], which can be summarized in three main categories: minimal access surgery , prosthetics and artificial skin , and smart interfaces for biomechanical measurements [8, 9]. Recently, attention in tactile sensing technologies further rose due to the spread of hand-held consumer touch-based devices such as smart phones and tablets.
A number of technologies were investigated for the development of tactile sensors, including principles of sensing such as piezoresistivity, piezoelectricity, and capacitance change; anyway a set of requirements independent of the technology can be identified for tactile sensors for force and pressure measurement inspired for the human tactile system, for example, the capability of detecting both static and dynamic forces, high spatial resolution and small size, compliant sensing surface, low hysteresis, high repeatability, and low discrimination threshold. All these features should be owned by the sensing system in compliance with the specific purpose  and be reflected in the calibration method, either with model-based approaches [11, 12] or with neural networks [13, 14].
Many efforts have been lavished by researchers to merge the aforementioned principles of sensing with microfabrication technology, because of several valuable features (e.g., small size, high sensitivity, accuracy and precision, and low power consumption) of microfabricated devices for artificial tactile sensing in biomedical applications .
Nevertheless, standard technologies are not able to overcome some issues, such as hysteresis, power consumption, and immunity to electromagnetic interferences. Therefore, novel materials and sensing elements (e.g., based on optics, fluidics, and ionic polymer metal composites) are under continuous investigation.
In particular, fiber optic sensors are gathering increasing interest for tactile sensing applications, due to their valuable features, such as reduced mass (few grams), small dimensions (usually the diameter is about 300–500 µm, but fibers with diameter of 10 µm can be employed), flexibility, immunity to electromagnetic interferences, and compatibility in Magnetic Resonance (MR) environments.
MR imaging (MRI) is a popular imaging technique used for both diagnostics and therapeutics in medicine. During MRI-based procedures, the monitoring of patient physiological parameters (e.g., temperature and breathing frequency) can provide crucial information about the patient conditions; therefore MRI-compatible sensors are attractive. A particular application of MRI is the functional MRI (fMRI), which assesses the brainy activities during specific task, aiming to map the physiological behavior and to understand how normal functions are disrupted in disease. Within this scenario, MRI-compatible force sensors allow measuring force during movements in humans and quantify the kinematics of motor task performance during fMRI .
Fiber optic sensors are the MRI-compatible tool par excellence, since the material used to fabricate the optical fibers does not perturb magnetic fields inside the MR-scanner, preserving the quality of diagnostic information [17, 18].
Fiber optic tactile sensors can be based on different principles of sensing, for instance, interferometry , micro- and macrobending [20, 21], hybrid optoelectronic solutions , and Fiber Bragg Grating (FBG) technology [23, 24] that we consider highly attractive and that was evaluated in the present study.
FBGs are short segments of optical fiber, which are ad hoc fabricated to be sensitive to strain. Optical fibers provided with FBGs merge the above-described features of plain optical fibers with peculiar characteristics, such as the use of the wavelength shift as sensing signal. This feature of FBG sensors avoids the issue of light intensity fluctuations and allows producing an array within the same fiber. Also, FBGs guarantee fast response, negligible hysteresis, and the possibility to discriminate uniform and nonuniform load adopting FBGs with special design (e.g., long-period, chirped, and tilted). The main drawback of FBGs is related to the simultaneous sensitivity to strain and temperature that can, however, be compensated by using additional FBGs in particular configurations to act as reference temperature sensors [25, 26].
FBGs were deeply investigated since 2006 to be housed in microsurgical tools, catheters, and needle biopsy  and, recently, also for the design of distributed tactile sensors for mimicking skin-like surfaces [26, 27]. Gastaldo and colleagues  compared tactile sensors based on FBGs with sensors composed of strain gauges and infrared sensors, showing the ability of FBG-based sensors to distinguish the application point of the load and the shape of different contact objects. Heo and colleagues  proposed two different designs for FBG-based tactile sensors, respectively, for low spatial resolution applications (mimicking human body skin) and high spatial resolution applications (as for finger skin). The two systems, characterized in the force range up to 5 N, showed a nonlinear calibration curve and tailored designs.
In this work we describe the simple design and characterization of a FBG-based tactile sensor array. The analysis of the response of the array while varying the probing location was carried out to assess the ability of the sensors to detect and discriminate loads in generalized conditions. A neural network was used to fit an inverse model of the device via a backpropagation training function, in order to quantitatively evaluate the system potential use as a measurement instrument. This machine learning approach was introduced with the aim of obtaining the sensor inverse characteristics by means of a learn-by-example method, rather than by identifying a set of parameters that minimize the average error of a model-based function . In comparison to model-based approaches, the chosen solution has limitations in its ability to explain the physics of the system; however, it is more robust with respect to possible fabrication defects that may cause deviation from the nominal mechanotransduction characteristics of the sensing system .
Finally, we investigated the compatibility of the developed sensor array for potential employment in functional MRI-measured brain activities during tactile tasks. To this aim, the FBG-based array was scanned with 1.5 T MRI.
2. Sensor Description
2.1. Fiber Bragg Grating Transduction Principle
The transduction principle of a FBG sensor is based on the phenomenon of radiation diffraction: when light propagates along a fiber housing a Bragg grating, a narrow range of wavelengths, centered around the so-called Bragg wavelength, , is diffracted in the backward direction, whereas the other wavelengths are not disturbed. The Bragg wavelength depends on the effective refractive index of the fiber core, , and on the spatial period of the grating, , according toIf the FBG experiences strain, , or temperature variation, , the Bragg wavelength will experience a shift , as expressed bywhere is the strain-optic coefficient and is the coefficient of temperature. If is negligible, a force applied to the FBG causes a strain which, in turn, causes a shift of the Bragg wavelength. Therefore, (2) can be simplified as follows:
2.2. FBG Sensor Array Design and Fabrication
In the developed system, 9 FBGs were integrated into a 3 × 3 sensors’ array (Figure 1). Each optical fiber, which constitutes the row of the array, housed three FBGs, with of 1533 nm, 1541 nm, and 1549 nm, respectively. The different of the three FBG sensors embedded into each fiber allowed separating their outputs. The fibers had an external diameter of 250 µm, and FBGs had length of 10 mm. The fibers were bonded to a plastic support by means of epoxy adhesive (Araldite 2011). The plastic support was built using a rapid prototyping printer (Project 3000, 3D Printer Inc.). Each sensitive segment was designed like a fixed beam, supported by the plastic support.
The FBGs placed on the same fiber have a distance of 11 mm, whereas the fibers were placed at relative distance of 15 mm (Figure 1(a)). In order to prevent fibers from mechanical crack, nine safety zones were designed on the surface of the plastic support, each in correspondence with the FBGs (Figure 1(b)).
The fibers were encapsulated within a 2 mm thick layer of Polydimethylsiloxane (PDMS) . The PDMS (Sylgard 184, Dow Corning, US) was obtained mixing the curing agent with the prepolymer in a 1 : 10 ratio, was cured at room temperature for 48 hours, and had 850 kPa Young’s modulus.
3. Methods for the Evaluation of the Sensor Array
The FBG-based array was experimentally evaluated to obtain the inverse relationship between the applied force and the wavelength shift via a quasi-static protocol. The sensor response was evaluated with different indentation locations. Some metrological properties, such as repeatability and accuracy, were estimated.
3.1. Experimental Setup and Protocol
A cylindrical probe, with external diameter of 5 mm (surface of 16 mm2), was used to apply semistatic deformation to the surface of the array. The material testing machine (Instron, 5900 series), suitable to apply compression to the sensor surface, was used to move the indenter along the -axis with constant speed of 0.1 mm·s−1, and the compression force was measured by a load cell (measurement range from −10 N to 10 N and accuracy of 0.25% of reading value) with sampling frequency of 10 Hz. Two translational stages (PT1, Thorlabs, with differential micrometer drive 150-811 ST) were used to translate the sensor along two perpendicular directions, - and -axes, with the aim of accurately controlling the position of the indenter above the desired surface coordinates. The change of (e.g., ) experienced by the FBGs during the compression tests was measured by means of an optical spectrum analyzer (OSA, Optical Sensing Interrogator, sm125, Micron Optics, resolution of 1 pm) and acquired by a graphical user interface (LabVIEW, National Instruments). The indentation force was servo-controlled up to 8 N, and the force measured by the load cell was synchronized with the values of measured by the OSA. Data sampling frequency was 10 Hz.
During the indentation experiments the probe was applied in 12 selected sites of the sensor surface (Figure 2(c)) with the aim of retrieving an inverse relationship between the applied to the sensor surface and the experienced by the FBG. Each experiment was repeated 3 times per each indentation site, thus obtaining a dataset with 36 sessions.
The MRI compatibility of the sensor array was tested by placing the sensor array within MRI scanner (GE, Optima MR360 1.5 T). Images of the sensor were acquired by 1.5 T scan, using a T2-weighted sequence.
3.2. Data Analysis Methods
The inverse function of the sensor array was estimated by means of a feedforward neural network (Figure 3, implemented with the Neural Network Toolbox in Matlab) composed of 9 input neurons (each associated to the output wavelength of a FBG sensor), 49 hidden neurons, and 1 output neuron (estimating the applied force ). The internal layer of the network was dimensioned so as to have a hidden neuron per each half-pitch of the 3 × 3 lattice of FBG sensors (per each row of sensors: 3 hidden neurons for the FBG sensors, plus 2 for the in-between positions, plus 2 for the external positions, therefore obtaining 7 × 7 hidden neurons in total). The Levenberg-Marquardt backpropagation method was used to train the network, with Mean Squared Error (MSE) performance function and 1000 training epochs as a maximum.
A first assessment was carried out by training the network including all the 36 indentation experiments and randomly selecting 25% of the time-samples of each indentation position as training data, 50% as validation data, and 25% as test data. The MSE performance, the error histogram, and the regression performance were evaluated for the training, validation, and test data.
A second more challenging assessment was carried out in order to evaluate the network ability to generalize with respect to the point of application of the load. To this aim, a Monte Carlo method was used, randomly dividing (with the Matlab randperm function) the 36 indentation sessions to 18 network-calibration sessions and 18 network-evaluation sessions, with 200 repetitions of the randomized calibration/evaluation procedure. For the network-evaluation sessions, we computed the distribution and the 25th, 50th, and 75th percentile of the mean and standard deviation of the errors that were obtained by estimating the applied force via the neural network.
The indentation force was tracked by the FBGs via a coherent modulation of the wavelength registered by the OSA (Figure 4). Such modulation was monotonic and highly repeatable (Figure 5). The repeatability was quantified through the percentage value of the standard deviation, %, calculated as the ratio between the standard deviation and the mean value of , equal to 1.6% for FBG1. The modulation of Bragg wavelength was dependent on the indentation position with respect to the FBG sensor: the larger the distance between the indentation site and the FBG sensor, the lower the measured by the grating (Figure 6).
When evaluating the neural network according to the first procedure described in Section 3.2, a 21 mN root mean squared error (RMSE) was achieved in the validation set (433 mN2 MSE, Figure 7), without offset in the error distribution (Figure 8) and with high regression quality as confirmed by the coefficients being very close to 1 for the calibration, validation, and test data sets and for all datasets grouped together (Figure 9).
When evaluating the neural network according to the second, more challenging, procedure described in Section 3.2, the 25th, 50th, and 75th percentile of RMSE of the estimated force were 124 mN, 199 mN, and 354 mN, respectively, across 200 instances of the randomized selection of experimental sessions (Figure 10), whereas the 25th, 50th, and 75th percentile of the mean error of the estimated force were −38 mN, 3 mN, and 34 mN, respectively (Figure 11).
Lastly, the test of the sensor array inside of MRI scanner proves the features of MRI safety and MRI compatibility of the sensor, since the sensor did not produce any artifacts on the images.
5. Discussion and Conclusions
This study presented the design and preliminary evaluation of FBG-based tactile sensor array. Its basic working principle, already assessed in the literature of this field, has been further investigated in terms of the following: simplified fabrication process, proved MRI-compatibility, and force coding by means of feedforward neural network.
The sensor had a simple design and fabrication process, requiring the following: three fibers to house FBGs, a plastic support fast manufactured by 3D printer, and a PDMS compliant polymeric material to cover and protect sensors as well as to distribute the load above the array surface. The sensor did not require a specific fabrication of the sensitive part surface (e.g., mesa in polymeric layer, as in ), and a quasi-static indentation protocol has been implemented to characterize the array in the measurement range 0–8 N, close to the requirements of tactile sensing for prosthetics .
The feedforward neural network allowed achieving 21 mN RMSE error in the force estimation over the 8 N experimented load range while including all probing sites in the training procedure. While estimating sensor performance under more challenging generalized probing conditions, the 25th, 50th, and 75th percentile of RMSE were 124 mN, 199 mN, and 354 mN, respectively, across 200 instances of a Monte Carlo randomized selection of experimental sessions.
Lastly, the sensor array was tested to be MRI compatible, since it did not produce any adverse imaging artifacts. This feature makes the sensor array suitable to be employed during tests of functional MRI, aiming at concurrent measurement of the brain activation of human subject and the applied forces, for neuroscience investigations [16, 31].
Considering other sensing principles from the well-established ones, such as piezoresistive, piezoelectric, and capacitive sensors, to the innovative ones, such as electroactive polymers and fluidic sensors, FBGs offer many advantages. The main added value of FBG-based sensors, along with the absence of electromagnetic interferences and MR compatibility, is related to the possibility of increasing the number of FBGs within the array. Indeed, as the number of sensors increases, wiring issues become relevant, but the ease of housing a number of FBGs within the same fiber allows reducing the encumbrance of the wiring. Also, another advantage is the wide measurement range in comparison to several tactile sensors developed for robotic or prosthetic purposes [32–35], which have been characterized within a measurement range typically limited up to 2.5 N. A limitation of FBG technology could be represented by the need for temperature compensation, since FBGs are sensitive to both mechanical strain and temperature change, showing a behavior similar to strain gauges. Therefore, specific designs are invoked to arrange FBG for temperature reference measurement in positions that do not experience strain and whose change of Bragg wavelength could be only due to temperature changes . Particular configurations aim to compensate the optical output of the other FBGs in the same array, dedicated to strain measurement. On the other hand, under specific design conditions, it is possible to take advantage of this intrinsic feature of FBGs, which can be used to implement a multimodal tactile sensor, providing both strain and temperature information [37, 38].
The encouraging results and the attractive performances of the feedforward neural network lead to the investigation of more accurate fabrication processes of the proposed prototype, with the aim of minimizing chirping effect during FBG strain, controlling cross-talk among sensors, and evaluating the optimal design for the compensation of temperature influence.
Conflict of Interests
The authors declare no conflict of interests.
Paola Saccomandi conceived, designed, and performed the experiments and contributed to the writing of the paper; Calogero Maria Oddo contributed to data analysis and to the writing of the paper; Loredana Zollo contributed to the design of the experiments and contributed to the writing of the paper; Domenico Formica contributed to the design and fabrication of the sensor; Rocco Antonio Romeo contributed to the design and performance of the experiments; Carlo Massaroni contributed to data acquisition and data analysis and contributed to the writing of the paper; Michele Arturo Caponero contributed to the design and fabrication of the sensor; Nicola Vitiello contributed to data analysis and contributed to the writing of the paper; Eugenio Guglielmelli contributed to the writing of the paper; Sergio Silvestri contributed to the writing of the paper; Emiliano Schena contributed to the design and performance of the experiments as well as to the writing of the paper.
This research was supported by PRIN project “HANDBOT-Biomechatronic Hand Prostheses Endowed with Bio-Inspired Tactile Perception, Bidirectional Neural Interfaces and Distributed Sensorimotor Control” (CUP: B81J12002680008) and by the national project PPR2 (Control of Hand Prosthesis by Invasive Neural Interfaces), funded by the National Institute for Insurance against Industrial Injuries (CUP: E58C13000990001).
J. Clot and Z. Stojiljkovic, “Integrated behaviour of artificial skin,” IEEE Transactions on Biomedical Engineering, vol. 24, no. 4, pp. 396–399, 1977.View at: Google Scholar
W. D. Hillis, Active Touch Sensing, 1981.
M. Leinewebera, G. Pelza, M. Schmidtb, H. Kappertc, and G. Zimmer, “New tactile sensor chip with silicone rubber cover,” Sensors and Actuators A: Physical, vol. 84, pp. 236–245, 2000.View at: Google Scholar
J. Voldman, M. L. Gray, and M. A. Schmidt, “Microfabrication in biology and medicine,” Annual Review of Biomedical Engineering, no. 1, pp. 401–425, 1999.View at: Google Scholar
J.-S. Heo, C.-H. Han, and J.-J. Lee, “System design and evaluation of the robot tactile sensor using the microbending fiber optic sensors,” in Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN '07), pp. 14–18, IEEE, August 2007.View at: Publisher Site | Google Scholar
J. Z. Liu, T. H. Dai, T. H. Elster, V. Sahgal, R. W. Brown, and G. H. Yue, “Simultaneous measurement of human joint force, surface electromyograms, and functional MRI-measured brain activation,” Journal of Neuroscience Methods, vol. 101, no. 1, pp. 49–57, 2000.View at: Publisher Site | Google Scholar
C. M. Oddo, L. Beccai, G. G. Muscolo, and M. C. Carrozza, “A biomimetic MEMS-based tactile sensor array with fingerprints integrated in a robotic fingertip for artificial roughness encoding,” in Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO '09), pp. 894–900, IEEE, Guilin, China, December 2009.View at: Publisher Site | Google Scholar