Table of Contents Author Guidelines Submit a Manuscript
Applied Bionics and Biomechanics
Volume 2018, Article ID 5852307, 15 pages
https://doi.org/10.1155/2018/5852307
Research Article

On the Reliability and Repeatability of Surface Electromyography Factorization by Muscle Synergies in Daily Life Activities

1Department of Economics, Engineering, Society and Business Organization (DEIM), University of Tuscia, Via Del Paradiso, 47, Viterbo 01100, Italy
2Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Via Eudossiana, 18, Roma 00184, Italy

Correspondence should be addressed to Stefano Rossi; ti.sutinu@issor.onafets

Received 19 November 2017; Revised 24 May 2018; Accepted 17 August 2018; Published 22 November 2018

Academic Editor: Craig P. McGowan

Copyright © 2018 Juri Taborri et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Muscle synergy theory is a new appealing approach for different research fields. This study is aimed at evaluating the robustness of EMG reconstruction via muscle synergies and the repeatability of muscle synergy parameters as potential neurophysiological indices. Eight healthy subjects performed walking, stepping, running, and ascending and descending stairs’ trials for five repetitions in three sessions. Twelve muscles of the dominant leg were analyzed. The “nonnegative matrix factorization” and “variability account for” were used to extract muscle synergies and to assess EMG goodness reconstruction, respectively. Intraclass correlation was used to quantify methodology reliability. Cosine similarity and coefficient of determination assessed the repeatability of the muscle synergy vectors and the temporal activity patterns, respectively. A 4-synergy model was selected for EMG signal factorization. Intraclass correlation was excellent for the overall reconstruction, while it ranged from fair to excellent for single muscles. The EMG reconstruction was found repeatable across sessions and subjects. Considering the selection of neurophysiological indices, the number of synergies was not repeatable neither within nor between subjects. Conversely, the cosine similarity and coefficient of determination values allow considering the muscle synergy vectors and the temporal activity patterns as potential neurophysiological indices due to their similarity both within and between subjects. More specifically, some synergies in the 4-synergy model reveal themselves as more repeatable than others, suggesting focusing on them when seeking at the neurophysiological index identification.

1. Introduction

Several locomotive activities, such as walking, running, and ascending and descending stairs, are commonly and continuously performed in daily life [1]. Though perceived as simple, these activities involve the coordination of a high number of muscles of lower limbs [2]. It is generally understood that the central nervous system (CNS) can reduce the dimensionality of neural activation outputs to control muscles and achieve a predefined movement [3, 4]. The reduction is obtained through the simultaneous coactivation of muscle groups, addressed as muscle synergies [5, 6]. Parameter estimation of muscle synergy models is usually obtained from the factorization of the electromyographic (EMG) signals by using decomposition algorithms [7]. The nonnegative matrix factorization [8] is the most widespread, even though other methods, such as principal or independent component analysis and inverse Gaussian, can lead to similar results [7, 9].

The factorization of the EMG signals by means of the muscle synergy model has been used in several tasks, such as locomotion [9, 10], balance [11, 12], reaching [13, 14], or sport gestures, such as cycling, bench press, and rowing [1517]. Nowadays, the goal of using EMG or muscle synergy parameters as neurophysiological indices, in clinical routine [1820], sport [17, 21], and robot control [2225], is gaining more and more appeal [26]. As regards clinical routine, Rodriguez et al. [19] demonstrated that patients with Parkinson’s disease adopted a simplified muscle control, i.e., a lower number of muscle synergies, during walking with respect to healthy subjects, while Shuman et al. [18] provided a comparison between children with cerebral palsy (CP) and age-matched controls, assessing a lower number of muscle synergies in the experimental group. A lower number of muscle synergies was also observed in poststroke patients [27, 28].

In sport field, Frère and Hug [21] showed that 3 synergies are sufficient to explain the EMG signals during backward giant swing on the high bar performed by expert gymnasts. Kristiansen et al. [17] found that the CNS has the capability to activate only two muscle synergies during bench press exercises in bodybuilders. As an example of cross-contamination in the robotic field, Artemiadis and Kyriakopoulos [22] and Lunardini et al. [23] used muscle synergies to control a robot for the rehabilitation of the upper limbs, in adults and children with CP, respectively.

To verify that muscle synergies are not a mere output of a mathematical approach but a useful tool to understand the organization of CNS in the achievement of a motor task, several studies investigated the methodological issues related to EMG factorization. In particular, Steele et al. [29] evaluated the effects of both the number and the specific examined muscles, finding that the two variables affect the structure of muscle synergies. Tresch and colleagues [7], instead, demonstrated that different factorization algorithms allow reaching equal results, as also confirmed by Santuz et al. [9]. The influence of filter parameters required for the EMG preprocessing on the muscle synergies was demonstrated considering both the repeatability [9] and the accuracy [30] of the selected number of muscle synergies. Finally, Oliveira et al. [31] focused their study on the effects of averaging or concatenating repetitions of the same task, assessing that the average allows obtaining a higher reconstruction quality even though it neglects the contribution of the step-to-step variability. In addition, to evaluate the viability of using muscle synergy parameters as neurophysiological indices, the robustness in the muscle synergy extraction needs to be deeply investigated. An excellent within-subject repeatability, both within and between days, has been already demonstrated during walking trials of children with cerebral palsy [18] and during walking and running trials of healthy subjects [9]. As regards the between-subject repeatability, it has been showed to be good in bench press exercises [17] and pedaling trials [15]. All the abovementioned papers have some methodological limitations since they did not quantify (i) the within- and between-subject repeatability in tasks different to the gait involving daily life activities, such as stepping in place and ascending and descending stairs, that could be implemented as useful scenarios in the control of lower limb robotic devices and (ii) the within- and between-subject robustness of the EMG reconstruction by means of a specific muscle synergy model considering both the overall reconstruction and each muscle individually. To the best of authors’ knowledge, all these methodological implications have not been evaluated. A thorough investigation is mandatory before validating the neurophysiological parameters based on the muscle synergy extraction and to give insightful information to researchers involved in the control design of lower limb robotic devices.

The aim of the present study is to investigate the reliability of muscle synergy theory, following two crucial questions still untapped. Firstly, we investigated the robustness of the EMG reconstruction in within- and between-subject analysis, focusing on both the overall reconstruction and the reconstruction of each single muscle. Robustness of EMG reconstruction, in fact, is a mandatory requirement when using EMG factorization in clinical, robotic, and sport applications [9, 26]. Secondly, we assessed the within-subject repeatability, i.e., both within and between sessions, and the between-subject repeatability of the parameters provided by the muscle synergy model in several daily life activities, to evaluate the viability of muscle synergy parameters as robust neurophysiological indices.

2. Materials and Methods

2.1. Experimental Protocol

Eight healthy volunteers (three males and five females, age: 27.8 ± 3.8 years, height: 1.72 ± 0.12 m, mass: 61.5 ± 14.9 kg) were enrolled in the experimental protocol. Subjects have never had known neuromuscular and vestibular pathologies. A written informed consent was obtained from the participants according to the ethical standards outlined in the 1964 Declaration of Helsinki.

The protocol consisted of five tasks: walking (W), stepping in place (S), running (R), ascending stairs (A), and descending stairs (D). W and R were performed through a pathway of 15 m at the self-selected speed, and the acquisition was stopped at the end of the pathway after one passage. S task lasted for 15 s, and each subject was free to choose the preferred cadence. In A and D tasks, subjects were asked to perform only one ascent (A) or descent (D) per repetition using a staircase with 20 steps. All tasks were repeated five times in barefoot condition. The entire protocol was repeated for three different sessions, separated each other by at least 24 h. Consequently, each subject performed a total of 75 trials that are five repetitions of five tasks for three sessions.

The activity of 12 muscles, shown in Figure 1, was recorded from the dominant leg, identified as the one used to kick a ball [32]. All the subjects were right dominant. The placement of passive surface Ag/AgCl circular electrodes (BlueSensor M, Ambu, Ballerup, Denmark) in differential configuration was performed by a skilled operator, the same for all subjects, according to the SENIAM guidelines [33]. Following these guidelines, the maximum electrode size in the direction of the muscle fibers is equal to 9 mm. In addition, these guidelines ensure placing electrodes far enough from the innervation zone, which can influence the recorded signals. Such recommendations were chosen as SENIAM is a recognized association in EMG research [34]. The repetitions within the same session were performed without removing the electrodes, while electrode replacement was performed between sessions. No measures were conducted to ensure the same positioning neither a marker on the subject was drawn among different sessions. Thus, the potential effects of intraoperator electrode replacement have been taken into account as a variable for muscle synergy parameter repeatability.

Figure 1: Muscles recorded during the experimental protocol from the dominant leg of each subject: example of electrode placement. 1: tensor fasciae latae (TFL); 2: rectus femoris (REF); 3: vastus lateralis (VLAT); 4: vastus medialis (VMED); 5: peroneus longus (PERO); 6: tibialis anterior (TA); 7: soleus (SOL); 8: gluteus maximus (GLU); 9: semitendinosus (SEMT); 10: biceps femoris (BIF); 11: gastrocnemius lateralis (LGAS); 12: gastrocnemius medialis (MGAS).

The foot of the dominant side was equipped with two footswitches (Footswitch FSR sensors, Wave, Cometa, Italy) underneath the toe and the heel. EMG and footswitch signals were synchronously acquired at 2000 Hz via a wireless system for electromyography (Wave, Cometa, Milan, Italy).

2.2. Data Processing

The footswitch status (pressed/not pressed) was used to identify the strides in W and R tasks and the single event in S, A, and D tasks, both defined as the interval between two consecutive heel strikes of the same foot. In order to avoid the influence of acceleration and deceleration phases and consider the same number of cycles for all tasks, only data corresponding to eight strides in W and R or steps in S, A, and D were selected from the acquired signals. In particular, for W and R tasks, the eight strides are the ones in the center of the acquired signal. In addition, the footswitch outputs were used to compute the cadence, expressed in steps/min, related to each subject in each repetition and session, independently for each task. Then, mean and SD were computed for each task across subjects and sessions. The ratio between SD and mean, expressed as percentage, allows calculating the coefficient of variation (CoV), quantifying the variability of the cadence in each task.

Following the methodological guidelines proposed by Santuz et al. [9], the mean value was removed from EMG signals and a high-pass filter at 50 Hz was applied. EMG signals were rectified and low-pass filtered at 20 Hz, to extract the envelope. All filters were 4th-order, zero-phase, Butterworth type. Negative values in the resulting signal due to a low-pass filter overshoot were artificially set to zero. The envelope signal was then divided into the eight previously identified strides based on footswitch data, interpolated to 1000 frames, and averaged across the strides, to obtain a higher reconstruction quality [31]. The amplitude of the EMG vector was normalized with respect to the maximum activation, defined as the maximum value found in a set including all tasks and all sessions. Thus, a single value for each muscle was used for the normalization process as suggested by de Marchis et al. [35]. An EMG matrix () was obtained for each task, each repetition, and each session by grouping the -sample signals of the considered muscles by row. In our case, was equal to 12 and was equal to 1000.

The extraction of muscle synergy parameters from EMG matrix was performed via the nonnegative matrix factorization (NNMF), which allows decomposing the EMG signals into muscle synergy vectors () and temporal activity patterns (), according to a linear combination, as in the following equation [8]: where represents the number of muscle synergies in the tested model and the residual was considered as the difference between the acquired EMG matrix and the reconstructed one. Briefly, each is a time-invariant vector, composed by positive weights, indicating the relative contribution of each muscle to the -th synergy. Each is a time-variant waveform vector hypothesized as the neural command for the activation of the -th synergy.

Equation (1) can provide multiple solutions for and , once the number of synergies in the model is selected. We solved the equation 12 times, increasing from 1 to , using the following parameters: 50 replicates and 1000 maximum iterations for minimizing squared residual between acquired and reconstructed signals [18]. We performed the NNMF in MATLAB (2012b, MathWorks, Inc., Natick, Massachusetts, United States).

2.3. Data Analysis

The variability account for (VAF) was chosen to assess the similarity between the acquired and reconstructed EMG signals [36], computed for all tested numbers of synergies. VAF was computed as uncentered Pearson’s coefficient, expressed as percentage. In particular, Pearson’s correlation coefficient between the acquired and the reconstructed EMG matrices was used to compute the global VAF (), while the local VAF () was calculated by correlating the acquired and the reconstructed EMG signal of each muscle.

For each subject, task, repetition, and session, we selected the model with the minimum number of synergies (NoS), which simultaneously met two selection criteria:  ≥ 90% and  ≥ 75% [15, 36]. Range of NoS for each subject across the three sessions was calculated, and median value and statistic frequency of each NoS were computed across all the repetitions and subjects, individually for each task. Thus, the potentiality of the NoS to be considered as a neurophysiological index is assessed focusing on its repeatability both within and between subjects in all examined tasks.

Focusing only on the most common model selected by the subjects considering all the examined tasks, we evaluated the repeatability of the muscle synergies considering three analyses: (i) within-subject-within-session (WW), to quantify the robustness of the muscle synergy extraction with respect to the variability induced by different muscle activations of the same subject to achieve the same motor task; (ii) within-subject-between-sessions (WB), to quantify the robustness with respect to the variability of the muscle activation of the same subject due to electrode replacement; and (iii) between-subjects (B), for quantifying the robustness of the algorithm with respect to different muscle activations by different subjects to achieve the same motor task.

As regards the WW analysis, mean and SD of the and were computed individually for each subject and each session. Then, the minimum and maximum of the mean value and their standard deviations (SDs) across subjects and sessions were selected for each task. SDs quantify the robustness of reconstruction goodness of EMG data gathered from the same subject in the same session. Moreover, to assess the robustness of the , we computed the cosine similarity (), which is the ratio between the scalar product and the product of the Euclidian norms of two vectors, as in the following equation [9]: where and represent the pair of tested in turn. This index ranges from 0 to 1, corresponding to no and perfect similarity, respectively. The threshold value for to assume similarity was set to 0.60 [3]. While to assess the similarity of the , we computed the coefficient of determination () as follows [9]: where and represent the pair of tested in turn. ranged from to 1, where 1 indicates the perfect similarity and 0.70 is the threshold to assess the similarity [9].

Considering that the order of synergies in the model obtained via NNMF is not consistent among different runs of the algorithm [18], we performed a -means cluster analysis to select and to order similar synergies between repetitions, sessions, and subjects for each task [37]. In particular, the most correlated synergy vectors that fell within one cluster according to the -mean outputs were considered related to the same -th synergy in the selected model between repetitions, sessions, and subjects for each task.

For WW, and values were obtained, respectively, by pairwise comparisons of the and of the related to the five repetitions in the same session. Thus, we performed 10 comparisons for each subject, each session, each task, and each synergy in the selected model. Mean and SD of and of across the comparisons were computed; then, minimum and maximum and related SDs across subjects and sessions were selected. Each SD was used to quantify the robustness of both the muscle synergy vectors and the temporal activity patterns in the same session performed by the same subject. Moreover, mean and SD of the overall model were computed by independently averaging the and related to all muscle synergies.

As concerns WB analysis, mean and SD of the and were computed individually for each subject, considering the values obtained during all the sessions together. Then, the minimum and maximum of the mean value and the related SDs across subjects were evaluated for each task. SDs quantify the robustness of the reconstruction goodness of the EMG data gathered from the same subject in different sessions. As regards the repeatability of the muscle synergy vectors and the temporal activity parameters, we firstly averaged the and the related to the five repetitions of each session, obtaining three and the . Then, and values were computed by pairwise comparisons of the three and of the three . Thus, we performed three comparisons for each subject, each task, and each synergy in the selected model. Then, mean and SD of and of across the comparisons were computed, and minimum and maximum and related SDs across subjects were evaluated. These parameters quantify the robustness of the muscle synergy vectors and temporal activity patterns in different sessions performed by the same subject. Moreover, mean and SD of the overall model were computed by independently averaging the and the related to all muscle synergies.

Finally, for the B analysis, mean and SD of the and were computed considering the values obtained during all the sessions and all subjects together. SDs were used to quantify the robustness of the reconstruction goodness of the EMG data gathered from different subjects in different sessions. In addition, to compute and values, we firstly averaged the and the of the three sessions individually for each subject and then, we performed pairwise comparisons among subjects using the obtained mean and mean . Thus, we performed 28 comparisons for each task and each synergy in the selected model. Then, mean and SD of and of across the comparisons were computed to quantify the robustness of the muscle synergy vectors and the temporal activity pattern in different sessions performed by different subjects. Moreover, mean and SD of the overall model were computed by independently averaging the and the related to all muscle synergies.

Considering the results of the , , , , , and , we assessed the potentiality of both the and , related to the overall composition and to the -th synergy in the selected muscle synergy model, to be considered as neurophysiological indices.

2.4. Statistical Analysis

To assess the reliability of the method, ICC was computed both on the and for each task, considering all sessions together. ICC values in the range 0.00–0.39 were classified as poor, 0.40–0.59 as fair, 0.60–0.74 as good, and 0.75–1.00 as excellent [38].

All data, with the exception of the NoS, were tested for normality with the Shapiro-Wilk test, and they resulted normally distributed. For all the following test, a significance level equal to 0.05 was set.

A Kruskal-Wallis nonparametric test was performed to evaluate statistical differences in the NoS, considering the task as the independent variable. If statistical differences were found, Dunn’s test was conducted to analyze where the differences were present.

One-way repeated measurement ANOVA tests were conducted for mean values of the , considering muscles as the independent variable. The Greenhouse-Geisser correction was adopted if the assumption of sphericity was violated. Bonferroni’s test for multiple comparisons was performed when statistical differences were found.

The influence of the -th synergy on and in the selected synergy model was tested through one-way ANOVAs, independently for each task. When the statistical differences were found, Bonferroni’s test for multiple comparisons was performed.

Statistical analysis was conducted using SPSS software package (IBM-SPSS Inc., Armonk, NY, USA).

3. Results

The following cadence values were obtained by processing the footswitch outputs: (i) 120 ± 10 steps/min with a variability equal to 8.3% for W, (ii) 110 ± 8 steps/min with a variability equal to 7.2% for S, (iii) 155 ± 15 steps/min with a variability equal to 9.7% for R, (iv) 100 ± 6 steps/min with a variability equal to 6.0% for A, and (v) 102 ± 9 steps/min with a variability equal to 8.8% for D.

As regards NoS, the ranges for each subject related to all tasks were reported in Table 1. Taking into account the three sessions, no subject recruited the same muscle synergy model during all examined repetitions related to each task. Considering all subjects, NoS ranged from 3 to 6 and the median value is always equal to 4 for all the tested daily life activities. However, some statistical differences were found among the activities: W vs. S, W vs. R, S vs. R, S vs. A, R vs. A, and R vs. D (always ).

Table 1: Range of number of synergies for each subject across the sessions related to each task.

In Figure 2, the frequencies of the selected synergy model for all tasks are shown. The most frequent model was the 4-synergy one for all tasks, with the exception of running (3 synergies). Moreover, it was also found that the 6-synergy model was the less likely selected for all tasks since it was the one associated with the minimum value of frequency occurrence, with the exception of stepping in place (3 synergies). Means and SDs of the as function of the number of synergies for each task are reported in Figure 3. To compare the results across the daily life tasks, we focused only on the 4-synergy model, as it resulted the most selected model considering all the subjects and all the tasks together.

Figure 2: Frequency of occurrence of the synergy model for each daily life activities. stands for statistical differences.
Figure 3: Mean and standard deviation of the across the subjects as function of the number of muscle synergies for each task.

Focusing on reliability analysis, ICC values of the and among sessions are reported in Table 2. The methodology showed an excellent reliability for the global reconstruction in all tasks. Considering each muscle, reliability ranged from fair to excellent in walking and from good to excellent in the other tasks.

Table 2: Interclass correlation coefficient (ICC) evaluated among sessions of variability account for values in the 4-synergy model for all tasks and for global EMG matrix and each muscle. Cells indicate the range of ICC values.

Taking into account the within-subject-within-session (WW) analysis, and values were always above 90.4% and 71.6%, respectively (Table 3). The highest values of SD were 1.2% and 9.7% for the global and the local reconstructions, respectively. As reported in Table 4, ranged from 0.71 to 0.99 considering all synergies and all activities, while a SD up to 0.24 was found. The highest overall mean value, averaging all synergies in the model, was related to A, i.e., 0.94, while the lowest one was related to D, i.e., 0.89. Taking into account the similarity related to the temporal activity patterns, ranged from 0.80 to 0.99 considering all synergies and all tasks, while the maximum value of SD was 0.10 (Table 5). The highest overall mean value was related to the R (0.94) and the lowest one to the D (0.91).

Table 3: Maximum and minimum and related standard deviation (SD) of the and related to each task.
Table 4: Maximum and minimum and related standard deviation (SD) of related to each task. The overall means evaluated considering all the synergies together are reported in italic.
Table 5: Maximum and minimum and related standard deviation (SD) of related to each task. The overall means evaluated considering all the synergies together are reported in italic.

As concern the within-subject-between-session (WB) analysis, a paradigmatic example of the differences of and during the three sessions related to one subject in one task is shown in Figures 4 and 5. Table 6 reports that the lowest and values were 91.9% and 74.2%, respectively, while the highest SD values were 1.9% and 9.8%. By analyzing Table 7, ranged from 0.84 to 0.99 considering all synergies and all activities and a SD up to 0.08 was found. The results obtained by averaging the outcomes of all muscle synergies in the model showed the highest overall mean value of the for the R and A (0.95) and the lowest one for the W (0.92). Focusing on Table 8, ranged from 0.85 to 0.99 with a maximum SD value equal to 0.08. Considering the overall composition, R was associated to the highest value (0.91) and the other tasks to the lowest one (0.90).

Figure 4: A paradigmatic example of the mean value of muscle synergy vectors in the 4-synergy model related to the ascending task for one subject for each session. The mean has been obtained averaging the repetitions of the same session. For visualization purposes, muscle synergy vectors are normalized to unity. Muscle numbers: 1: TFL; 2: REF; 3: VLAT; 4: VMED; 5: PERO; 6: TA; 7: SOL; 8: GLU; 9: SEMT; 10: BIF; 11: LGAS; 12: MGAS.
Figure 5: A paradigmatic example of the mean value of the temporal activity pattern in the 4-synergy model related to the running task for one subject for each session. The mean has been obtained averaging the repetitions of the same session. For visualization purposes, temporal activity patterns are normalized to unity.
Table 6: Minimum and maximum and relative standard deviation (SD) of the and related to each task.
Table 7: Maximum and minimum and relative standard deviation (SD) of the related to each task. The overall means evaluated considering all the synergies together are reported in italic.
Table 8: Maximum and minimum and relative standard deviation (SD) of the related to each task. The overall means evaluated considering all the synergies together are reported in italic.

Finally, considering the between-subject analysis (B), the mean values of both the muscle synergy vectors and the temporal activity patterns considering all subjects and all sessions are shown in Figures 6 and 7 for each task. The histograms and waveforms are useful to clarify the inner composition of the synergies in the model, necessary to understand the following results in terms of repeatability related to each specific synergy. Table 9 reports the minimum value of 93.9% and 84.3% for the and and a maximum SD value of 1.7% and 7.6%, respectively. As regards the statistical outcomes, the mean value of related to the GLU was statistically different from the same parameter computed for LGAS () during W task. Statistical differences were found also in S task, between related to TA and MGAS (). As regards A task, statistical differences were found between the related to the TFL and the one computed for REF (), VLAT (), VMED (), PERO (), TA (), SOL (), and MGAS () and other differences were observed between related to the GLU and the ones computed for REF (), VLAT (), VMED (), PERO (), SOL (), and MGAS (). No statistical differences were found in R () and D (). The values were in the range of 0.79–0.90, with a SD up to 0.10 (Table 10). Differences among the specific synergies in the model were found in W (always ), R (always ), and D (always ), as reported in Table 10. By averaging the of all muscle synergies in the model, the highest overall mean value was associated to the A (0.86) and the lowest to the S (0.82). Finally, the felt in the range 0.84–0.92, with a SD up to 0.08. Differences among the specific synergies in the model were found in all tasks (always ), as reported in Table 11. By taking into account the overall composition, all tasks reached an equal to 0.87, with the exception of D (0.84).

Figure 6: Mean value of muscle synergy vectors in the 4-synergy model related to each daily life activities. The mean has been obtained by averaging across subjects the muscle synergy vector mean of the three sessions of each subject. For visualization purposes, muscle synergy vectors are normalized to unity. Muscle numbers: 1: TFL; 2: REF; 3: VLAT; 4: VMED; 5: PERO; 6: TA; 7: SOL; 8: GLU; 9: SEMT; 10: BIF; 11: LGAS; 12: MGAS.
Figure 7: Mean value of the temporal activity pattern in the 4-synergy model related to each daily life activities. The mean has been obtained by averaging across subjects the temporal activity pattern mean of the three sessions of each subject. For visualization purposes, temporal activity patterns are normalized to unity.
Table 9: Mean value and standard deviation (SD) of and related to each task. Differences among muscles are reported in superscript.
Table 10: Mean value and standard deviations (SDs) related to the evaluated synergy by synergy for each task. Differences between pair of synergies in each task are reported in superscript with the relative roman number; stands for statistical differences among all synergies. The overall means evaluated considering all the synergies together are reported in italic.
Table 11: Mean value and standard deviations (SDs) related to the evaluated synergy by synergy for each task. Differences between pair of synergies in each task are reported in superscript with the relative roman number; stands for statistical differences among all synergies. The overall means evaluated considering all the synergies together are reported in italic.

4. Discussion

The present study represents an experimental insight on the reliability of synergy-based EMG signal factorization. Specifically, we evaluated the within- and between-subject variability of the reconstruction of the EMG signals associated with the technique. Further, we aimed to assess the repeatability of the muscle synergy parameters, which are number of muscle synergies, goodness of reconstruction, muscle synergy vectors, and temporal activity patterns, both within and between sessions during a set of daily life activities.

4.1. Is the EMG Reconstruction Reliable and Repeatable in Within-Subject and Between-Subject Analysis?

Results related to the reliability of the muscle synergy extraction permit to assume that factorization of EMG data via muscle synergies expresses an excellent reliability, considering the global reconstruction of muscle activities during the examined tasks. On the contrary, fair reliabilities related to local reconstruction of some muscles confirm the findings reported by Kristiansen et al. [17], recommending a particular attention to the variability of the method, when handling data from some specific muscles such as biceps brachii, triceps brachii, and the rectus femoris, which is also found to be the one with the lowest value of ICC in this study. These results could be ascribed to the high between-subject variability of the VAF, as already shown by Kristiansen et al. [17].

The walking task showed a lower reliability than the other motor tasks, suggesting that the CNS is able to control these movements by means of different activation patterns, as also reported by De Marchis et al. [39]. Comparing walking with running, the outcomes of lower reliability can be justified considering that the gait cycle becomes less variable as speed increases [40]; instead, comparing walking with the other examined tasks, we speculate that external constraints, such as stairs, cause a decrease of gait variability as also reported by Donath et al. [41].

As regards the repeatability of EMG reconstruction goodness, the high value of the and the low SD values proved that the selection criterion on the global reconstruction was always met in both within- and between-subject analyses for all tasks. Thus, the variability induced by the different muscle activations of the same subject, that is, the recruitment of different NoS to perform the same task, does not influence the goodness of the linear combination used for the global reconstruction. Considering these outcomes, we can affirm the robustness of the EMG factorization by means of the 4-synergy model if researches have to pay attention only to the global reconstruction of the EMG activity. By looking to the high SD values related to the , we can affirm that the selection of the most common synergy model can lead to inaccurate EMG reconstructions related to some specific muscles; in particular, the reconstruction of the EMG signals related to the TFL, TA, and GLU generally appeared to be the one less accurate in all tasks when a 4-synergy model is selected. Finally, the statistical differences found in the mean values of in walking, stepping, and ascending stairs suggest to carefully selecting the muscles to include in the experimental setup, to respect the goodness of the reconstruction required by the specific application. The greater number of statistical differences was found for the gluteus, both in walking and in ascending stairs, and the tensor fasciae latae in the ascending task. These findings can be ascribed to the more relevant soft tissue and cross talk artifacts due to the position of the electrodes [42].

By summarizing the findings related to the reliability and repeatability analysis of the VAF, the extraction of muscle synergy by means of the nonnegative matrix factorization can be used in several applications, requiring robust goodness of EMG reconstruction, such as clinical analysis, sports performance evaluation, or robotics. However, the use of EMG factorization via the muscle synergies is recommended only in applications that require a reliable overall reconstruction of the acquired EMG signals. In fact, when the reconstruction of the single muscle has been required, such as for controlling a robotic device via muscle synergies [22] or monitoring the severity of a pathology [43], the actual reliability and variability of the specific muscle reconstruction have to be considered.

4.2. Might Muscle Synergy Parameters Be Considered as Neurophysiological Indices?

Both the range (3–6) and the median value (4) of NoS found in this study for all tasks provide a further confirmation of the hypothesis that the CNS reduces the complexity of muscle activation for achieving a motor task, also in activity never explored before, such as ascending and descending stairs. Concerning the frequency values of NoS, we observed that our results, related to the walking and stepping, are in accordance with the literature [5, 9, 36, 44]. Conversely, a lower dimensionality of the synergy model is observed in the running task with respect to Santuz et al. [9]. In particular, they analyzed the activation of lower and upper body muscles and found a greater number of muscle synergies in running than in walking. This difference could be ascribed to the reduced number of muscles that we analyzed, as we neglected those in the upper body. It is worth noticing that synced arm movements are necessary to keep balance in running [45]. Thus, we speculate that the absence of upper body muscles in the experimental setup could explain how the dimensionality of our model is in contrast with literature for running task, but in accordance for walking and stepping, where the recruitment of upper body muscles is less crucial for the body balance. Moreover, statistical differences on the median value found for the NoS in running compared with those in the other tested daily life activities suggest that the EMG signals gathered during running can be reconstructed with a lower number of synergies. The absence of a frequency of NoS equal to 100% in all tasks, both within and between sessions for healthy subjects with similar demographic characteristics, might be ascribed to the capability of the CNS to generate different activation profiles for reaching the same motor task, resulting in a different number of muscle synergies [39]. The ability of NoS to discriminate between healthy subjects and patients with neurological diseases was previously assessed in patients with cerebral palsy [18] and Parkinson’s disease [19]. In particular, the authors found a maximum spread of 2 NoS between the control and patient groups in walking tasks. Our findings showed a 2 NoS difference also in within-subject and between-subject analyses performed on a healthy population. Consequently, the NoS lacks in the sufficient repeatability level to be considered a robust neurophysiological index.

Focusing on the weights of , instead, we observed that was always above the threshold value of 0.60 for each synergy and for the overall model; however, the variability related to some synergies is not negligible. In fact, SDs indicated that values spanned also under the similarity threshold of 0.60, considering a confidence interval of 95.5%, i.e., mean value ± 2SD. The variability in the muscle synergy vectors could be addressed to the CNS capability to generate different within-subject activation profiles to achieve the same motor task [15]. It confirms our findings related to the different NoS to perform the same task by the same subject. Moving to the , the variability can be considered negligible since the SD values related to each synergy and the overall model were above the similarity threshold also considering the confidence interval of 95.5%. By summarizing, the variability of the in within-session analysis implies also values lower than 0.60; thus, our results suggest caution in assuming that the weights related to the -th muscle synergy are consistent across repetitions of the same task performed by the same subject if an average across the repetitions is not conducted. Similar repeatability with respect to the one observed by Santuz et al. [9] for the intra- and interday repeatability in walking and running was observed in this study. Thus, we can speculate that the use of a treadmill, as in [9], does not influence the walking and running patterns [46]. As regards the , the mean values were lower than those evaluated in the WW and WB analyses. These findings can be mainly ascribed to the well-known between-subject variability of the EMG signals [47]. As an alternative hypothesis, they could be due to the variability of the biomechanical data in the execution of the same task by different subjects. However, such effect appeared limited, as shown by the variability of the cadence always lower than 10%. However, some -th synergies and the overall composition of the muscle synergy vector reached mean values above the similarity threshold also with a confidence of 95.5%, affirming themselves repeatable across subjects and potentially useful as neurophysiological indices. The only exception of this finding was the IV synergy related to the stepping task. From the statistical results, it emerged that the third for walking and descending stairs and the fourth for running can be considered the most repeatable synergy vectors related to each motor task. On the contrary, the absence of significant differences in stepping and in ascending stairs did not allow selecting only one as the most repeatable. We can conclude that the muscle synergy vectors related to some specific synergy in the 4-synergy model can be considered acceptable as neurophysiological indices due to demonstrated similarity across a cohort of subjects having similar demographic characteristics and health condition. In addition, it is worth noticing that in W, D, and R, it is suggested to use only the most repeatable .

The same trend of the muscle synergy vectors can be observed for the repeatability analysis related to the temporal activity parameters by considering the WW and WB. In fact, and values are always above the threshold value of 0.70 for each synergy and the overall model, but the value of some SD related to indicated that some comparisons in WW analysis did not reach the set threshold if a confidence level of 95.5% is considered. The found values are in line with the one obtained by Santuz et al. [9]. As for the , the variability in the temporal activity patterns suggests that the CNS can organizes different muscle activations to achieve the same motor task [15]. Thus, our results suggest caution in assuming that the each in the 4-synergy model is consistent across repetitions of the same task performed by the same subject in the same session. By moving to the , all the values related to each synergy and to the overall model were above the similarity threshold also considering a confidence interval at 95.5%, with the exception of the third synergy for walking and the first synergy for the stepping. However, by considering the statistical outcomes, we can select the neurophysiological index in the most repeatable temporal activity pattern for each task, in particular, the one related to the second synergy for stepping and running and the one related to the first synergy for the descending stairs. A further selection to identify the most repeatable between the one related to the first and second synergies in W and among the one related to the first, third, and fourth synergies in ascending stairs cannot be performed with the found statistical outcomes. In conclusion, we can state that also the temporal activity patterns related to some specific synergy in the 4-synergy model can be assumed as neurophysiological indices due to the promising results related to their repeatability. As for , the selection of the most repeatable is preferred in S, R, and D.

Thus, we can conclude that both the and the can be considered as feasible neurophysiological indices due to the verified good repeatability in within- and between-subject analysis. It is worth noticing that the verified repeatability has two different meanings: the repeatability of the indicates that the weighting of the examined muscles in the -th synergy is similar across subjects, while the repeatability of the shows that the neuromotor signal modulating the excitation of the -th synergy is consistent across subjects [16].

5. Conclusion

In this paper, we analyzed the reliability and repeatability of the EMG factorization by means of muscle synergy theory in several daily life activities, such as walking, running, stepping, and ascending and descending stairs. These analyses have been conducted both within and between subjects. Actually, the final goal of the study is to identify the most potential neurophysiological index among the muscle synergy parameters. The outcomes of our study endorse muscle synergy factorization as a robust tool to interpret the CNS activation model, as the repeatability can be mainly affected by the intrinsic EMG signal variability and the electrode replacement rather than the factorization algorithm. A 4-synergy model can be used for all examined tasks if only the global reconstruction has to be considered, while attention has to be paid in the choice of muscles if their specific reconstruction has to be performed. Moreover, we recommend focusing on the muscle synergy vectors or the temporal activity patterns rather than the number of the synergies, when seeking at the identification of a neurophysiological index.

Conflicts of Interest

The authors have no conflicts of interest to report.

References

  1. G.-M. Jeong, P. H. Truong, and S.-I. Choi, “Classification of three types of walking activities regarding stairs using plantar pressure sensors,” IEEE Sensors Journal, vol. 17, no. 9, pp. 2638-2639, 2017. View at Publisher · View at Google Scholar · View at Scopus
  2. F. E. Zajac, R. R. Neptune, and S. A. Kautz, “Biomechanics and muscle coordination of human walking: part I: introduction to concepts, power transfer, dynamics and simulations,” Gait & Posture, vol. 16, no. 3, pp. 215–232, 2002. View at Publisher · View at Google Scholar · View at Scopus
  3. A. d’Avella, P. Saltiel, and E. Bizzi, “Combinations of muscle synergies in the construction of a natural motor behavior,” Nature Neuroscience, vol. 6, no. 3, pp. 300–308, 2003. View at Publisher · View at Google Scholar · View at Scopus
  4. R. R. Neptune, D. J. Clark, and S. A. Kautz, “Modular control of human walking: a simulation study,” Journal of Biomechanics, vol. 42, no. 9, pp. 1282–1287, 2009. View at Publisher · View at Google Scholar · View at Scopus
  5. Y. P. Ivanenko, R. E. Poppele, and F. Lacquaniti, “Five basic muscle activation patterns account for muscle activity during human locomotion,” The Journal of Physiology, vol. 556, no. 1, pp. 267–282, 2004. View at Publisher · View at Google Scholar · View at Scopus
  6. E. Bizzi, V. C. K. Cheung, A. d'Avella, P. Saltiel, and M. Tresch, “Combining modules for movement,” Brain Research Reviews, vol. 57, no. 1, pp. 125–133, 2008. View at Publisher · View at Google Scholar · View at Scopus
  7. M. C. Tresch, V. C. K. Cheung, and A. d'Avella, “Matrix factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets,” Journal of Neurophysiology, vol. 95, no. 4, pp. 2199–2212, 2006. View at Publisher · View at Google Scholar · View at Scopus
  8. D. D. Lee and H. S. Seung, “Learning the parts of objects by non-negative matrix factorization,” Nature, vol. 401, no. 6755, pp. 788–791, 1999. View at Publisher · View at Google Scholar · View at Scopus
  9. A. Santuz, A. Ekizos, L. Janshen, V. Baltzopoulos, and A. Arampatzis, “On the methodological implications of extracting muscle synergies from human locomotion,” International Journal of Neural Systems, vol. 27, no. 05, article 1750007, 2017. View at Publisher · View at Google Scholar · View at Scopus
  10. J. Taborri, E. Palermo, D. Masiello, and S. Rossi, “Factorization of EMG via muscle synergies in walking task: evaluation of intra-subject and inter-subject variability,” in 2017 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1–6, Turin, Italy, 2017. View at Publisher · View at Google Scholar · View at Scopus
  11. L. H. Ting and J. M. Macpherson, “A limited set of muscle synergies for force control during a postural task,” Journal of Neurophysiology, vol. 93, no. 1, pp. 609–613, 2005. View at Publisher · View at Google Scholar · View at Scopus
  12. G. Torres-oviedo and L. H. Ting, “Subject-specific muscle synergies in human balance control are consistent across different biomechanical contexts,” Journal of Neurophysiology, vol. 103, no. 6, pp. 3084–3098, 2010. View at Publisher · View at Google Scholar · View at Scopus
  13. S. Muceli, A. T. Boye, A. d'Avella, and D. Farina, “Identifying representative synergy matrices for describing muscular activation patterns during multidirectional reaching in the horizontal plane,” Journal of Neurophysiology, vol. 103, no. 3, pp. 1532–1542, 2010. View at Publisher · View at Google Scholar · View at Scopus
  14. A. D’Avella, A. Portone, and F. Lacquaniti, “Superposition and modulation of muscle synergies for reaching in response to a change in target location,” Journal of Neurophysiology, vol. 106, no. 6, pp. 2796–2812, 2011. View at Publisher · View at Google Scholar · View at Scopus
  15. F. Hug, N. A. Turpin, A. Guével, and S. Dorel, “Is interindividual variability of EMG patterns in trained cyclists related to different muscle synergies ?” Journal of Applied Physiology, vol. 108, no. 6, pp. 1727–1736, 2010. View at Publisher · View at Google Scholar · View at Scopus
  16. N. A. Turpin, A. Guével, S. Durand, and F. Hug, “No evidence of expertise-related changes in muscle synergies during rowing,” Journal of Electromyography and Kinesiology, vol. 21, no. 6, pp. 1030–1040, 2011. View at Publisher · View at Google Scholar · View at Scopus
  17. M. Kristiansen, A. Samani, P. Madeleine, and E. A. Hansen, “Muscle synergies during bench press are reliable across days,” Journal of Electromyography and Kinesiology, vol. 30, pp. 81–88, 2016. View at Publisher · View at Google Scholar · View at Scopus
  18. B. Shuman, M. Goudriaan, L. Bar-on, M. H. Schwartz, K. Desloovere, and K. M. Steele, “Repeatability of muscle synergies within and between days for typically developing children and children with cerebral palsy,” Gait & Posture, vol. 45, pp. 127–132, 2016. View at Publisher · View at Google Scholar · View at Scopus
  19. K. L. Rodriguez, R. T. Roemmich, B. Cam, B. J. Fregly, and C. J. Hass, “Persons with Parkinson’s disease exhibit decreased neuromuscular complexity during gait,” Clinical Neurophysiology, vol. 124, no. 7, pp. 1390–7, 2013. View at Publisher · View at Google Scholar · View at Scopus
  20. N. Hesam-Shariati, T. Trinh, A. G. Thompson-Butel, C. T. Shiner, and P. A. McNulty, “A longitudinal electromyography study of complex movements in poststroke therapy. 2: changes in coordinated muscle activation,” Frontiers in Neurology, vol. 8, p. 277, 2017. View at Publisher · View at Google Scholar · View at Scopus
  21. J. Frère and F. Hug, “Between-subject variability of muscle synergies during a complex motor skill,” Frontiers in Computational Neuroscience, vol. 6, 2012. View at Publisher · View at Google Scholar · View at Scopus
  22. P. K. Artemiadis and K. J. Kyriakopoulos, “EMG-based control of a robot arm using low-dimensional embeddings,” IEEE Transactions on Robotics, vol. 26, no. 2, pp. 393–398, 2010. View at Publisher · View at Google Scholar · View at Scopus
  23. F. Lunardini, C. Casellato, T. D. Sanger, and A. Pedrocchi, “Synergy-based myocontrol of a two degree of freedom robotic arm in children with dystonia,” in Converging Clinical and Engineering Research on Neurorehabilitation, vol. 15, pp. 595–599, Springer, Cham, Switzerland, 2017. View at Publisher · View at Google Scholar · View at Scopus
  24. D. Brunelli, E. Farella, D. Giovanelli, B. Milosevic, and I. Minakov, “Design considerations for wireless acquisition of multichannel sEMG signals in prosthetic hand control,” IEEE Sensors Journal, vol. 16, no. 23, pp. 8338–8347, 2016. View at Publisher · View at Google Scholar
  25. Y. Su, M. H. Fisher, A. Wolczowski, G. D. Bell, D. J. Burn, and R. X. Gao, “Towards an EMG-controlled prosthetic hand using a 3-D electromagnetic positioning system,” IEEE Transactions on Instrumentation and Measurement, vol. 56, no. 1, pp. 178–186, 2007. View at Publisher · View at Google Scholar · View at Scopus
  26. J. Taborri, V. Agostini, P. K. Artemiadis et al., “Feasibility of muscle synergy outcomes in clinics, robotics, and sports: a systematic review,” Applied Bionics and Biomechanics, vol. 2018, Article ID 3934698, 19 pages, 2018. View at Publisher · View at Google Scholar · View at Scopus
  27. E. Ambrosini, C. de Marchis, A. Pedrocchi et al., “Neuro-mechanics of recumbent leg cycling in post-acute stroke patients,” Annals of Biomedical Engineering, vol. 44, no. 11, pp. 3238–3251, 2016. View at Publisher · View at Google Scholar · View at Scopus
  28. D. J. Clark, L. H. Ting, F. E. Zajac, R. R. Neptune, and S. A. Kautz, “Merging of healthy motor modules predicts reduced locomotor performance and muscle coordination complexity post-stroke,” Journal of Neurophysiology, vol. 103, no. 2, pp. 844–857, 2010. View at Publisher · View at Google Scholar · View at Scopus
  29. K. M. Steele, M. C. Tresch, and E. J. Perreault, “The number and choice of muscles impact the results of muscle synergy analyses,” Frontiers in Computational Neuroscience, vol. 7, 2013. View at Publisher · View at Google Scholar · View at Scopus
  30. E. Scalona, J. Taborri, Z. Del Prete, E. Palermo, and S. Rossi, “EMG factorization during walking: does digital filtering influence the accuracy in the evaluation of the muscle synergy number?” in IEEE International Symposium on Medical Measurements and Applications, pp. 1–6, Rome, Italy, 2018. View at Publisher · View at Google Scholar
  31. A. S. Oliveira, L. Gizzi, D. Farina, and U. G. Kersting, “Motor modules of human locomotion: influence of EMG averaging, concatenation, and number of step cycles,” Frontiers in Human Neuroscience, vol. 8, 2014. View at Publisher · View at Google Scholar · View at Scopus
  32. M. Germanotta, J. Taborri, S. Rossi et al., “Spasticity measurement based on tonic stretch reflex threshold in children with cerebral palsy using the pedianklebot,” Frontiers in Human Neuroscience, vol. 11, p. 277, 2017. View at Publisher · View at Google Scholar · View at Scopus
  33. D. F. Stegeman and H. J. Hermens, “Standards for surface electromyography: the European project ‘surface EMG for non-invasive assessment of muscles (SENIAM)’,” pp. 108–112, 2007, http://www.med.uni-jena.de/motorik/pdf/stegeman.pdf. View at Google Scholar
  34. J. Kilby, K. Prasad, and G. Mawston, “Multi-channel surface electromyography electrodes: a review,” IEEE Sensors Journal, vol. 16, no. 14, pp. 5510–5519, 2016. View at Publisher · View at Google Scholar · View at Scopus
  35. C. de Marchis, M. Schmid, D. Bibbo, A. M. Castronovo, T. D'Alessio, and S. Conforto, “Feedback of mechanical effectiveness induces adaptations in motor modules during cycling,” Frontiers in Computational Neuroscience, vol. 7, 2013. View at Publisher · View at Google Scholar · View at Scopus
  36. K. E. Zelik, V. la Scaleia, Y. P. Ivanenko, and F. Lacquaniti, “Can modular strategies simplify neural control of multidirectional human locomotion?” Journal of Neurophysiology, vol. 111, no. 8, pp. 1686–1702, 2014. View at Publisher · View at Google Scholar · View at Scopus
  37. H. Spath, The Cluster Dissection and Analysis Theory FORTRAN Programs Examples, Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1985.
  38. D. V. Cicchetti, “Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology,” Psychological Assessment, vol. 6, no. 4, pp. 284–290, 1994. View at Publisher · View at Google Scholar · View at Scopus
  39. C. De Marchis, M. Schmid, D. Bibbo, I. Bernabucci, and S. Conforto, “Inter-individual variability of forces and modular muscle coordination in cycling: a study on untrained subjects,” Human Movement Science, vol. 32, no. 6, pp. 1480–1494, 2013. View at Publisher · View at Google Scholar · View at Scopus
  40. K. Jordan, J. H. Challis, and K. M. Newell, “Walking speed influences on gait cycle variability,” Gait & Posture, vol. 26, no. 1, pp. 128–134, 2007. View at Publisher · View at Google Scholar · View at Scopus
  41. L. Donath, O. Faude, R. Roth, and L. Zahner, “Effects of stair-climbing on balance, gait, strength, resting heart rate, and submaximal endurance in healthy seniors,” Scandinavian Journal of Medicine & Science in Sports, vol. 24, no. 2, pp. e93–e101, 2014. View at Publisher · View at Google Scholar · View at Scopus
  42. A. Fratini, M. Cesarelli, P. Bifulco, and M. Romano, “Relevance of motion artifact in electromyography recordings during vibration treatment,” Journal of Electromyography and Kinesiology, vol. 19, no. 4, pp. 710–718, 2009. View at Publisher · View at Google Scholar · View at Scopus
  43. L. Tang, X. Chen, S. Cao, D. Wu, G. Zhao, and X. Zhang, “Assessment of upper limb motor dysfunction for children with cerebral palsy based on muscle synergy analysis,” Frontiers in Human Neuroscience, vol. 11, 2017. View at Publisher · View at Google Scholar · View at Scopus
  44. F. Lacquaniti, Y. P. Ivanenko, and M. Zago, “Patterned control of human locomotion,” The Journal of Physiology, vol. 590, no. 10, pp. 2189–2199, 2012. View at Publisher · View at Google Scholar · View at Scopus
  45. C. J. Arellano and R. Kram, “The effects of step width and arm swing on energetic cost and lateral balance during running,” Journal of Biomechanics, vol. 44, no. 7, pp. 1291–1295, 2011. View at Publisher · View at Google Scholar · View at Scopus
  46. P. O. Riley, G. Paolini, U. Della Croce, K. W. Paylo, and D. C. Kerrigan, “A kinematic and kinetic comparison of overground and treadmill walking in healthy subjects,” Gait & Posture, vol. 26, no. 1, pp. 17–24, 2007. View at Publisher · View at Google Scholar · View at Scopus
  47. M. P. Kadaba, H. K. Ramakrishnan, M. E. Wootten, J. Gainey, G. Gorton, and G. V. B. Cochran, “Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait,” Journal of Orthopaedic Research, vol. 7, no. 6, pp. 849–860, 1989. View at Publisher · View at Google Scholar · View at Scopus