Abstract
Background: Kinematic and muscle patterns underlying hand grasps have been widely investigated in the literature. However, the identification of a reduced set of motor modules, generalizing across subjects and grasps, may be valuable for increasing the knowledge of hand motor control, and provide methods to be exploited in prosthesis control and hand rehabilitation.
Methods: Motor muscle synergies were extracted from a publicly available database including 28 subjects, executing 20 hand grasps selected for daily-life activities. The spatial synergies and temporal components were analyzed with a clustering algorithm to characterize the patterns underlying hand-grasps.
Results: Motor synergies were successfully extracted on all 28 subjects. Clustering orders ranging from 2 to 50 were tested. A subset of ten clusters, each one represented by a spatial motor module, approximates the original dataset with a mean maximum error of 5% on reconstructed modules; however, each spatial synergy might be employed with different timing and recruited at different grasp stages. Two temporal activation patterns are often recognized, corresponding to the grasp/hold phase, and to the pre-shaping and release phase.
Conclusions: This paper presents one of the biggest analysis of muscle synergies of hand grasps currently available. The results of 28 subjects performing 20 different grasps suggest that a limited number of time dependent motor modules (shared among subjects), correctly elicited by a control activation signal, may underlie the execution of a large variety of hand grasps. However, spatial synergies are not strongly related to specific motor functions but may be recruited at different stages, depending on subject and grasp. This result can lead to applications in rehabilitation and assistive robotics.
Introduction
The use of the hands is one of the most crucial capabilities for daily activities. The loss of a hand can substantially reduce the quality of life of a person, since it strongly affects physical capabilities in performing activities of daily living (ADL) and it represents a relevant social problem considering that people with a major upper limb loss were ~41,000 in USA in 2005. The number of amputees is expected to double by 2050 (Atkins et al., ; Ziegler-Graham et al., ).
Hand grasps are mainly composed of two main stages: the reach-to-object and the grasp itself. The first phase is divided into two sub-phases, consisting of the transport of the hand done by the arm, whose motion law is characterized by a bell-shaped velocity (Fan et al., ), and the hand pre-shaping, required for adapting the hand to the object to grasp, which occurs after ~60–70% of the reaching phase (Hu et al., ). The grasp phase is determined by several parameters, including the force closure (force needed to close the hand around the object and to achieve a stable grasp), grasp stability (the ability to resist external forces), and grasp security (resistance to slippery objects, which is depending on the configuration of the grasp; Cutkosky, ; Cipriani et al., ). A third phase is reported in some articles (Liarokapis et al., ) and represent the release of the object; a fourth phase can be considered too, involving the return of the arm and hand to the rest position.
Hand grasps have been investigated mainly in the domain of finger joint kinematics and past studies have developed qualitative taxonomies to describe and cluster different types of grasps (Cutkosky, ). The main distinction among grasps was between power grasps and precision grasps but many other features can be taken into account for grasp characterization, such as the limb configuration for the task execution or the geometry of the object to grasp.
Considering the complexity of hand control, involving a remarkable number of degrees of freedom and redundancy, both at the muscle and skeletal levels, many studies in the literature applied feature extraction methods to identify a subset of the original data for an accurate description of hand functioning, even if reduced in dimensionality.
A recent study (Jarrassé et al., ) investigated a set of hand grasps by considering a 15-degree-of-freedom (dof) Cyberglove. The study used a Principal Component Analysis (PCA)-based technique for the extraction of kinematic motor synergies and showed that no more than 4 PCs are needed to explain ~95% of the total variation. The first and second PCs accounted for about 90% of data variation, leading the author to suggest that these two components might be enough to control (or even mechanically design) an upper-limb prosthesis, even if pattern refinement can be achieved by adding further PCs. In Patel et al. (), kinematic synergies were extracted by using a PCA-based algorithm. While the first PC accounts for more than half of the total variation, the rest is distributed across many PCs, indicating that a quite large set of motor modules is needed to reconstruct the original kinematics. Seven synergies were extracted in Thakur et al. () for the explanation of >90% of the total variance of a set of hand-grasps and hand motions. A comprehensive study on hand grasps by Santello et al. () suggests that the modules that underlie the control of the hand are basically two. However, the study also remarks that the remaining variation, accounted for by further synergies, is not due to noise but to motor control modules needed for fine tuning.
The fact that a limited number of modules may account for a large variety of grasps is thus commonly deduced from the literature. A recent study by Prevete et al. () investigated the hypothesis of sparsity applied to kinematic synergies during hand grasps. According to this study, sparsity might be found both at the spatial synergy level (indicating that spatial modules may incorporate only some joints or muscles) and in the coordination of the synergies, in which only a reduced number of overlapping modules contribute to the execution of an action. A combination of the two conditions, called double sparsity hypothesis, can happen as well. This concept fits well with previous research on dimensionality reduction, with the addition that sparsity could partially explain the different number of synergies extracted in different studies (together with varying study designs).
Despite the kinematic patterns being exploited more often for hand analysis, some studies have investigated the dimensionality reduction problem from the point of view of muscle synergies. The muscle synergy approach is based on decomposition algorithms that identify groups of co-activating muscles (synergies) that are coordinated by time-varying activation commands. The extracted patterns may be influenced by several factors regarding sEMG, including fatigue, sweating, changes in electrode or arm positioning (Farina et al., ), clinical parameters of the subjects (e.g., level of the amputation, phantom limb sensation intensity; Atzori et al., ), the BMI (Atzori et al., ), other anatomical characteristics of the subjects (Farina et al., ) or training in using myoelectric prostheses (Cipriani et al., ). Few studies addressed these effects until now, and the effect on the resulting muscle synergies. Considering upper limb synergies, Ortega et al. observed that synergy structure was conserved with fatigue, but interestingly synergy activation coefficients decreased on average by 24.5% with fatigue development (Ortega-Auriol et al., ). In Tagliabue et al. () two-digit grasping is analyzed. A reduced number of modules (2–3) is needed to explain the largest part of the variation for each grasp and the correlation between muscle and kinematic primitives is suggested, justifying synergy-based analysis in both domains. Considering two arrays of sEMG-electrodes, positioned distally and proximally on the forearm, Castellini and van der Smagt () found that the combination of 3 muscle synergies could account for a set of 5 hand grasps, on both sets of the electrodes. The “main synergy” represents a “global, indistinct” co-activation pattern, while the other two synergies account for dorsal and ventral patterns, respectively.
Overduin et al. () used the time-varying muscle synergy model to analyze a set of 25 grasps of two monkeys and found that three synergies could explain 71% of the total sEMG variation for proximal muscles, 83% for the wrist and extrinsic hand muscles and 81% among intrinsic muscles. The first of the three synergies was linked to the muscles involved in the reach phase operated by proximal muscles and distal flexors, the second was characterized by bimodal activation of distal muscles and the third, more related to the transport of the object, featured by proximal muscles and distal extensors.
The main challenge of using muscle synergies to analyze hand grasps is represented by the impossibility to track all the muscles involved in the grasps, as hand muscles are hard to acquire due to their small size, which can easily produce cross-talk, and due to encumbrance of probes/wires on the palm of the hand that can prevent a physiological grasp execution. Nevertheless, the reduction of the dimensionality is still a crucial process for the comprehension of the patterns underlying hand use and grasps. In fact, motor modules are considered to be the basis of motor control organization at the neural level (Schmidt, ; d'Avella et al., ). Furthermore, once recognized, the basic modules might be employed as references for the study of motor control, to evaluate pathological conditions and to control prosthetic devices. Dexterous, naturally controlled surface electromyography (sSEMG) prostheses would better allow amputees to perform personal needs such as eating or using tools. Prosthetics companies and scientific research are advancing toward this, but dexterous naturally controlled prosthetic hands are not yet available, in the market as well as in scientific research mainly due to control problems (Atzori and Muller, ) related to robustness. Clinical parameters of the amputation were demonstrated to affect control capabilities (Atzori et al., ). In order to foster the improvement of control systems for sEMG hand prostheses, a publicly available dataset for robotic hand prosthesis control (the Ninapro database1) was released in 2014 (Atzori et al., ), and extended with several additional datasets afterwards (Krasoulis et al., ; Pizzolato et al., ). Currently, the database includes over 120 subjects (including 11 trans-radial amputees), repeating as naturally as possible up to 53 hand movements with several acquisition setups ranging in price from a few hundred to several thousand dollars. The aim of Ninapro is to foster the improvement of the field by allowing the development and test of advanced machine learning methods. However, the path to natural control of dexterous prosthetic hands can also be paved by the simplification of the problem, for instance via the identification of a set of motor primitives sufficient to control a comprehensive set of hand grasps.
The application of muscle and postural hand synergies to myoelectric hand prostheses development and low level control was recently suggested in literature and tested in specific settings, while high level control strategies are still not extensively explored. The application of postural hand synergies to hand prostheses development is particularly evident in the development of the PISA/IIT Softhand, a robotic hand actuated by a single motor (Catalano et al., ). The application of postural hand synergies to low level control approaches can be defined as controlling a dexterous robotic hand with few (usually 4) independent input signals that modulate some of the first synergies (usually the first one-two) in the robotic hand, leading the robotic hand to reproduce several hand grasps (Matrone et al., , ; Segil and Weir, ).
In the literature, there are several open points regarding hand grasp synergies that can be investigated in more detail. Some of the more refined studies, providing state-of-the-art methods, involve a large variety of grasps but a limited number of subjects, or map a reduced number of grasps compared to the ones that are needed for daily life activities, lacking generalization of results. Furthermore, a limited number of studies focuses on muscle patterns rather than on hand kinematics. Lastly, most studies focused especially on the spatial organization of motor modules, while the temporal components were less analyzed.
Following the previous considerations, the aim of this study is threefold. First, to provide a set of benchmark muscle hand synergies extracted from the publicly available NinaPro database, that includes a considerable number of subjects while repeating a comprehensive number of hand grasps; second, to evaluate the effects of the reduction of dimensionality of the dataset on the accurateness in reconstructing the original dataset of synergies; third, to characterize the spatial and temporal features of the subjects included in the dataset.
Materials and methods
Acquisition set-up
The flow-chart of the study is portrayaed in (Figure 1). The acquisition setup included 12 surface EMG (sEMG) electrodes and a data glove. The sSEMG electrodes were a double differential Delsys Trigno wireless system, measuring the myoelectric signals at 2 kHz with a baseline noise inferior to 750 nV RMS. The Trigno integrated a 3-axes accelerometer sampled at 148 Hz. Electrode positioning was performed with the aim of combining precise anatomical positioning (DeLuca, ) and a dense sampling approach (e.g., Fukuda et al., ). Eight electrodes were equally spaced around the forearm at the height of the radio-humeral joint. Four electrodes were placed on the main activity spots, respectively, of the flexor and the extensor digitorum superficialis, the biceps and the triceps brachii, which were identified by palpation by trained researchers by trained researchers (Figure 2). The data glove (CyberGlove II, CyberGlove Systems LLC 2) allowed to measure hand kinematics using 22-sensors. Considering that the primary objective of this study was to characterize the hand grasps rather than the dynamics of the reaching phase at proximal level, the choice of the NinaPro database is reasonable, since it includes recordings from extrinsic hand muscles.
Figure 1
Figure 2
Participants
The data used in this experiment were from the publicly available NinaPro database that currently includes 7 datasets of sEMG and kinematic data from over 120 subjects (including 11 trans-radial amputees), performing (or imagining to perform) up to 53 different hand movement (Atzori et al., ). The datasets used for this study were from the second dataset (DB2), which includes 40 intact subjects. A 28-subject subset of the original dataset was used for this study. The subjects include 19 males, 9 females; 24 right handed, 4 left handed; average age 29.64 with standard deviation 3.1 years (data summarized in Table 1). Twelve subjects were excluded from the analysis because the proper extraction of synergies was prevented by noise of the sEMG channels. The decomposition algorithm applied to extract synergies would have been influenced, even in case of removal of the affected channels from the analysis.
Table 1
| Subjects | Gender | Age | Laterality |
|---|---|---|---|
| DEMOGRAPHIC DATA | |||
| N = 28 | 19M-9F | 29.64 ± 3.1 | 24R-4L |
Summary of the demographic data of the involved subjects.
Experimental protocol for acquisition
This section briefly describes the acquisition protocol. For more details about the protocol, please refer to Atzori et al. (). During the experiment, the subjects were asked to sit at a desktop with the arms relaxed on the table and to repeat a set of movements with their right hand as naturally as possible. The entire experiment included 49 movements plus rest, divided into three exercises and extracted from the ADL literature, thus including movements from categories, such as personal needs, eating or use of tools (Smurr et al., ). In this work, we consider only the set of hand grasps, i.e., the first 20 movements of the second exercise (Figure 3). The subjects were asked to repeat the movements represented in short films that were shown on the screen of a laptop with their right hand and they were asked to concentrate on mimicking the movements rather than on exerting high forces. Each movement was repeated 6 times, with each repetition lasting 5 s and separated by the other movements by 3 s of rest. The experiment was approved by the Ethics Commission of the Canton Valais (Switzerland) and before data acquisition, the subjects were given a thorough written and oral explanation of the experiment itself and were asked to sign an informed consent.
Figure 3
Data analysis: synergies extraction
The Data Analysis was fully performed with Matlab 2014a with custom-developed software. First, kinematic recordings (“restimulus” signal of the NinaPro database) were used to separate movement phases. Data from 12 sEMG channels were bidirectionally high-pass filtered at 50 Hz (Butterworth filter, 7th order) to remove motion artifacts, rectified, Hilbert-transformed (Matlab hilbert), low-pass filtered with a cut-off frequency of 10 Hz (Butterworth filter, 7th order) to remove noise with mono-directional filtering. sEMG data from each subject and each trial were pooled in single aggregated matrices and synergies were extracted using the non-negative matrix factorization (NMF) algorithm (Cheung et al., ; Tresch et al., ). The NMF decomposes the sEMG matrix into the product of two matrices, the first one representing time-invariant, neurally coded synergies (wi), and the second one representing time-variant activation commands for each synergy (ci), as in Equation (1):
where, for each of the recorded muscles, sEMG(t) represents the sEMG data at time t and N is the total number of extracted synergies.
The order of the factorization r was chosen, increasing from 1 to 50 (to limit the dimensionality for synthesis). For each r, the NMF algorithm was applied 1,000 times in order to avoid local minima. The repetition accounting for the highest variance of the signal was chosen as the representative of order r. The number of synergies was chosen as the minimum r explaining at least 90% of the variance of the signal (Clark et al., ). Further synergies were added only if the total amount of variation was increased of at least 5% for each further synergy.
Synergy clustering
In the literature of motor synergies, standard analysis methods may include the definition of clusters to group synergies according to their spatial composition. The set of extracted synergies can be clustered to obtain a limited number of spatial patterns, each one represented by a centroid (mean spatial synergy).
In this work, the extracted synergies were included into a single cluster analysis. Grouping all the modules could lead to complex matching between each spatial component and the corresponding motor function (Scano et al., ). In fact, it was reported in Roh et al. () that synergies related to the same motor function may split into two or more clusters. As a consequence, the correspondence between the phases of the grasps and the motor synergy recruitment is not always clearly identifiable. In fact, in the majority of the cases, the synergy prevailing in terms of magnitude of the temporal components is the one characterizing the moment of the grasp hold. However, a relevant number of subjects may show patterns more complex to identify.
However, performing the clustering procedure on the whole dataset allowed to provide a comprehensive overview of all the modules involved in hand grasping tasks. Furthermore, a comprehensive mapping of hand grasps is proposed by considering the whole dataset for analysis.
The cluster analysis was conducted using the k-means clustering algorithm. The algorithm was applied to an aggregated matrix containing the whole dataset of muscle synergies extracted from all subjects. Each clustering order, ranging from 1 (minimum) to 50 (maximum), was tested by repeating the algorithm 200 times and selecting the best solution for each order according to the metrics described in the following section.
Selection of the number of clusters
The selection of the appropriate number of clusters (mean spatial synergies, each one represented by a cluster centroid) was made by pondering the following metrics (Bora et al.,
):
The Mean Euclidean Distance (MED) of the population from the reference centroids, indicating the quality of the clustering, as a synthetic index for each clustering order. The lower the Mean Euclidean Distance, the better elements fit into their cluster.
When the k-means clustering procedure is applied, the number of desired clusters N must be specified. Defining as M the number of elements to be clustered (in this case, the number of spatial synergies), N can range between 1 and the total number of the clustered elements (1<=N<=M).
When N = 1, the clustering procedure classifies a population within a single group: thus, the cluster solution 1 is (implicitly) the mean of a population, and corresponds to the lowest level of precision in approximating a population with a clustering procedure. Following the previous considerations, the Normalized Euclidean Distance (NED) was computed by considering the cluster solution 1 as the source of maximum clustering error, which was set to 1. Thus, the NED for each clustering order i was computed as:
The slope of the Normalized Euclidean Distance (NED') is NED derivative. NED indicates how the precision of the cluster analysis increases when increasing the order of the clustering.
Each of the previous three metrics can be considered for the choice of the clustering order, by imposing a threshold on the reconstruction accuracy.
Whatever metric is selected, the choice is driven by the principle of using a parsimonious number of clusters for synthesis power (the lowest possible number of clusters, given a reasonable descriptive precision). The threshold selected by the experimenters in this work was 5%. Consequently, the number of clusters was selected as the minimum number needed to have the NED < 0.05.
The hypothesis that justifies the use of cluster analysis is that the dataset can be represented with a chosen number of cluster centroids depending on the maximum error that the experimenter is willing to accept. Depending on the application, the tolerance can be increased or reduced, describing the original dataset of motor modules with a specific level of precision (and a choice of dimensionality).
Spatial and temporal components analysis
The characterization of the obtained mean spatial synergies was furtherly specified by considering all the pairwise dot products between their compositions. Each temporal component, initially associated with its respective spatial synergy, was matched to its relative centroid after cluster identification. Then, all the temporal components were averaged to extract a mean temporal component for each cluster, representing a mean activation of the spatial synergy in time. Finally, the characterization of temporal components was concluded by considering the correlations between the mean temporal components.
Summary of outcome measures and statistics
Given the aims of the study (see Introduction): “First, to provide a set of benchmark muscle hand synergies extracted from publicly available data
1including a considerable number of subjects that perform a comprehensive number of hand grasps; second, to evaluate the effects of the reduction of dimensionality of the dataset on the accurateness in reconstructing the original dataset of synergies; third, to characterize the spatial and temporal features of the sample of subjects included in the dataset,” the following outcome measures were defined:
Outcome 1: Definition of the complete dataset of extracted muscle synergies of healthy subjects in freely executed grasps; methods and statistics: NMF algorithm for factorization; 90% of the VAF + minimum slope 0.05 for each further extracted synergy.
Outcome 2: Definition of cluster centroids for muscle synergies in freely executed grasps; methods and statistics: k-means clustering; lowest normalized Euclidean distance to define the number of centroids.
Outcome 3a: Characterization of the spatial composition of the centroids; methods and statistics: dot products between pairwise centroids to assess their difference in composition.
Outcome 3b: Characterization of the temporal features of the centroids; methods and statistics: Pearson correlations between temporal components.
Results
Extracted synergies
The extracted synergy dataset is summarized in Figure 4 by portraying the mean spatial synergy compositions and cumulated temporal component profiles. Synergies were grouped within grasps, and matched according to the similarity of their temporal components, computed with the Pearson's correlation coefficient. For compactness of the representation, only the first two synergies of each extracted dataset were portrayed (while, three modules were extracted in some grasps).
Figure 4
K-means cluster order selection
The whole dataset of spatial synergies, which is composed of 966 extracted modules, was clustered according to the k-means algorithm, with a clustering order ranging from 1 to 50. Figure 5 shows the graphs with the metrics used for the choice of a reasonable number of clusters as synthetic representation of the spatial synergies of the dataset. Increasing the order of the clustering leads to a monotonic decrease of the NED. Thresholding the NED (at 0.05, as explained in the methods), only 10 clusters are needed to approximate the original dataset. It can also be observed that a further increase of the order of the clustering provides only slightly increased precision in describing the dataset.
Figure 5
Clustering on spatial synergies and temporal components analysis
The results of the clustering procedure are shown in Figure 6. The 10 identified centroids (mean spatial synergies) are portrayed (composition coefficients), along with the number of synergies of the original dataset that are addressed to each centroid, expressed as percentage of the original dataset. It can be seen that the extracted synergies are quite uniformly distributed on the centroids, each one representing between 7 and 15% of the original dataset of motor modules. Figure 7 depicts a polar and histogram-based representation of the extracted mean spatial synergies, along with the associated mean temporal components. The temporal components are shown for each of the spatial modules referring to each of the centroids, along with their mean. Analysis of temporal components shows that some centroids are found mostly in the central phase of the grasp (e.g., centroid 3 and centroid 6), while others mainly in the pre-shaping and release phases (e.g., centroid 1 and centroid 2). Following these results, in order to provide characterization of the summarized groups of motor modules, the similarity of mean spatial synergies and temporal components was assessed as well. Figure 8 shows the similarity, expressed as the dot product, among all the pairwise mean spatial synergies. It can be seen that the mean spatial centroids have a pairwise dot product ranging from 0.65 to 1, indicating that some muscle groups are shared between several patterns. Similarly, Figure 9 shows the correlation matrix between each temporal component, expressing the temporal relation that links each spatial synergy to the others. In this case, results show high variability, and indicate that some mean temporal components are very closely related to others (e.g., temporal components 5 and 6), while others are very different (e.g., temporal components 1 and 6). These results are critically analyzed in the following paragraphs.
Figure 6
Figure 7
Figure 8
Figure 9
Discussion
On the extracted synergies
An interesting result of this study is that, with the used method for synergy extraction, a number of modules ranging from 1 to 3 is sufficient for reconstructing the majority of the original sEMG in each grasp. As a consequence, a limited number of patterns is needed to achieve a grasp, which is a relevant result considering the availability of high redundancy at the muscle and kinematic level. This is seen in some patterns that are often repeated and especially in the co-activating group composed of f1-f8-finger flexors that are very often grouped together, especially in the hold phase. In most of the cases, two activation patterns are recognizable: a strong co-activation, often (but not always) corresponding to the grasp/hold phase, and two minor co-activating patterns in the pre-shaping and release phases that are often grouped in a single synergy. This result is particularly interesting considering that only two electrodes were not positioned on the forearm (respectively, biceps and triceps) and comparing the results with the results obtained by d'Avella et al. () and Liarokapis et al. (). In these studies, the biceps is activated during the reaching phase in confirming that it is indeed an active reaching component, being active in the pre-shaping and release phase. This result suggests that the pre-shaping and release synergies may represent hand opening, before (pre-shaping) and after grasping. The number of phases seems to be in accordance with those proposed by Liarokapis et al. () and seems to reproduce on the hand part of the results obtained in previous studies in terms of time varying muscular synergies for shoulder and arm. Furthermore, it should be remarked that the movements considered in this study were not performed against gravity, reducing consistently the involvement of shoulder muscles.
While not extensively discussed in this paper, the remarkable repeatability of the temporal components might be a further motor-control feature aimed at simplifying muscle coordination, as a strategy exploited by the CNS to perform hand grasps. These results are in accordance with the previous findings in the literature, that showed that, in respect to the original dimensionality of the control, the number of modules underlying grasps is probably strongly reduced (Santello et al., ; Overduin et al., ; Jarrassé et al., ).
Cluster analysis and control of precision
On a comprehensive dataset of 20 grasp types, performed by 28 healthy subjects, 10 spatial motor modules, properly elicited in time, are enough to describe the whole dataset with good accuracy, generalizing through subjects. Such results are coherent with the notion that the central nervous system may embed a modular structure that relies on a limited number of predefined co-activation patterns to produce motor outcome at the hand level. These findings are in accordance with previous results that demonstrated that a small subset of synergies can generalize across tasks and suggest that they represent basic building blocks underlying natural human hand motions (Thakur et al., ).
The main spatial synergies were not directly linkable to specific grasp types or motor functions, suggesting that the spatial modules that can be employed for the execution of different grasp types. Furthermore, each spatial module can be elicited at different stages. Together with previous findings, these results suggest that grasp types and muscle synergies may not be univocally related: some muscle patterns may be used for different grasp types or, vice versa, the same grasp might be controlled with slightly different muscle synergies depending on the subject.
These results also reflect some intrinsic features of the human grasping related to proximal forearm and hand muscles control. This study suggests that a large variety of hand grasp types can be performed with a limited number of patterns. However, it should be considered that the proposed protocol was meant for applications related to control of prosthesis for trans-radial amputees, measuring the activity of proximal muscles. Coherently, previous studies sharing proximal muscle based protocols showed that a few basic patterns are responsible for a variety of grasp types (Castellini and van der Smagt, ). On the contrary, considering a fine recording of the muscles of the hand more differentiated patterns may be observed, even if due to the difficulty of recording muscle activity directly on the hand the motor primitives related to the hand are computed and analyzed in a kinematic domain (Jarrassé et al., ; Prevete et al., ).
Temporal components
The analysis of temporal components underlines that spatial patterns may be recruited at different stages of a grasp, with variability related both to the subject who executes the grasp and the type of grasp. This result is confirmed by the high correlation of the temporal components of many clusters. However, mean temporal components suggest that some patterns are more often used during the grasp phase with a monophasic, bell-shaped activation profile, while other patterns are biphasic and usually activate when the hand opens, so in the approaching/pre-shaping phase and in the release phase rather than in the middle of the grasp. Such findings can be taken into account for several applications related to high level robotic hand and prosthesis control, as described in section Impact of the Muscle Synergy Dataset.
Impact of the muscle synergy dataset
A limited number of motor modules (e.g., 10), properly elicited in time can approximate the entire dataset for all subjects with high accuracy (5% error in respect to approximating the dataset with its mean, in the case of 10 motor modules). Ideally, each movement considered in the experiment can potentially be reproduced as a combination of spatial synergies, thus providing prostheses with higher dexterity (a higher number of movements that can be controlled) starting from a set of a few robustly controlled modules. Hand muscle synergies may be applied to high level control approaches, consisting of training subjects to reproduce and modulate the sEMG patterns that correspond to the muscular hand synergies (or combinations of them) and apply pattern recognition algorithms to recognize the results. This strategy may be an alternative to the control systems currently described in literature. As said in the introduction, robotic hands that reproduce hand movements by modulating the main postural hand synergies have already been presented in literature (Matrone et al., , ; Segil and Weir, ). However, high level control systems have not been extensively studied. Developing high level control systems based on time dependent muscle-hand synergies and training subjects to perform them may link the subjects' intentions with the movement of a robotic hand naturally, by exploiting the same synergies. Such result may lead to natural myoelectric control of robotic hands, a challenge currently not yet achieved in literature. If replicated on hand amputees, this result can potentially have applications in rehabilitation and assistive robotics in order to improve the control of dexterous prosthetic hands, by joining robotics and neuroscience findings (Santello et al., ).
Usually, in machine learning the training data (used to train a model) and the test data (used to test it) are taken from the same distribution. However, this is not always easy, in particular when using deep learning approaches that require large amounts of data for training. To overcome distribution mismatches, transfer learning and domain adaptation approaches have been used in several domains, including computer vision (Saenko et al., ; Tommasi et al., ), and natural language processing (Ben-David et al., ; Daumé et al., ). In myoelectric control, several studies explored the use of previous models from different subjects to reduce the amount of required training data (Farina et al., ; Tommasi et al., ; Patricia et al., ), but performance increase was not confirmed after proper model optimization (Gregori et al., ). The fact that the motor modules are common to the subjects can provide physiological foundations to include within the prosthesis a subject-independent motor memory. Prosthesis control could then be produced as “plug and play,” improve control robustness for a specific subject through successive calibration, and improve its adaptability to other subjects too. In this context, properly choosing the motor modules and the movements to be reproduced (in order to maximize dexterity, robustness and correspondence to ADLs) is potentially interesting to improve the rehabilitation capabilities of hand prostheses. However, it is an open question in the field of how exactly extracted synergies are mapped into motor functions: previous studies employing clustering procedures or synergy combination theories (Prevete et al., ) showed that the mapping between “physical space” of the end effector and the extracted muscle synergies may be due to different exploitation mechanisms.
In this study, it is proposed that a linear combination of centroids, properly activated by their temporal components, can be enough to reconstruct the physical space of the end effector in a large variety of grasp types with high accuracy. However, the authors are aware that the noticeable reduction of the original dataset implies that the original sEMGs are reconstructed with a pre-determined level of precision. The proper tradeoff between accuracy and synthesis needs to be tested in future work where the reduced dataset is integrated into a real control system.
Despite the potential provided by the muscle synergy analysis, several limitations and issues related to the method should be considered. Recent studies reported that pre-processing, including filtering and normalization techniques, might lead to different results and interpretation of the data (Shuman et al., ; Kieliba et al., ). While it is commonly accepted to normalize the duration of the tasks to a common phase axis, as it was done in this study, uniform guidelines for EMG pre-processing for synergies extraction are still missing in the literature. Consequently, pre-processing could be a source of data misinterpretation. Furthermore, the insurgence of fatigue was not inspected in this study, while it was demonstrated in the literature that fatigue may influence the recruitment of synergies, even if their spatial composition is preserved (Ortega-Auriol et al., ). As described in the section Introduction, several factors may have an effect on sEMG signal and make synergies tough to be generalized. Those may include fatigue, despite the acquisition protocol was carefully designed to induce low fatigue on subjects, even in the case of patients (Atzori et al., ), and future developments should also consider these variables for a complete assessment.
Lastly, the model of human grasps described in this paper can potentially provide insights for calibrated interventions of rehabilitation robotics. Several implications can be found considering neurological or orthopedic rehabilitation of the hand (Bissolotti et al., ; Vanoglio et al., ). In recent studies, the exploitation of devices for hand rehabilitation has shown to lead to promising, therapeutic results that can be further enhanced by training muscle synergy-oriented exercises, based on a detailed knowledge of motor synergies (Scano et al., ).
Conclusion
In this paper, muscle synergies were extracted from the recordings of a publicly available dataset. The extracted synergies were clustered from a cohort of 28 subjects executing a variety of hand grasps. The synergies are often characterized by two temporal activation patterns: a strong co-activation corresponding to the grasp/hold phase, and two minor co-activating patterns related to hand opening (visible in the pre-shaping and release phase). The conclusions of this article suggest that a limited number of time-dependent motor modules, correctly elicited by a control activation signal, may underlie the execution of a large variety of hand grasps. However, spatial synergies are not strongly related to a specific motor functions but have a sparse recruiting timing.
Statements
Author contributions
AS designed the experiment, wrote the software for synergy extraction and clustering, elaborated the data, and wrote the paper. AC participated to data analysis and interpretation, and wrote the paper. LM participated to data analysis and wrote the paper. HM acquired the NinaPro database and wrote the paper. MA acquired the NinaPro database, participated in the design of the study and to data interpretation, and wrote the paper.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
1.^Ninapro database: http://ninapro.hevs.ch
References
1
AtkinsD. J.HeardD. C.DonovanW. H. (1996). Epidemiologic overview of individuals with upper-limb loss and their reported research priorities. J. Prosthet. Orthot.8, 2–11.
2
AtzoriM.GijsbertsA.CastelliniC.CaputoB.HagerA. G.ElsigS.et al. (2014a). Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data1:140053. 10.5061/dryad.1k84r
3
AtzoriM.GijsbertsA.KuzborskijI.HeynenS.HagerA.-G. M.DeriazO.et al. (2014b). Characterization of a benchmark database for myoelectric movement classification. Trans. Neural. Syst. Rehabil. Eng. 23, 73–83. 10.1109/TNSRE.2014.2328495
4
AtzoriM.GijsbertsA.CastelliniC.CaputoB.HagerA. G.ElsigS.et al. (2016). Effect of clinical parameters on the control of myoelectric robotic prosthetic hands. J. Rehabil. Res. Dev.53, 345–358. 10.1682/JRRD.2014.09.0218
5
AtzoriM.MüllerH. (2015). Control capabilities of myoelectric robotic prostheses by hand amputees: a scientific research and market overview. Front. Syst. Neurosci.9:162. 10.3389/fnsys.2015.00162
6
Ben-DavidS.BlitzerJ.CrammerK.KuleszaA.PereiraF.VaughanJ. W. (2010). A theory of learning from different domains. Mach. Learn.79, 151–175. 10.1007/s10994-009-5152-4
7
BissolottiL.VillafañeJ. H.GaffuriniP.OrizioC.ValdesK.NegriniS. (2016). Changes in skeletal muscle perfusion and spasticity in patients with poststroke hemiparesis treated by robotic assistance (Gloreha) of the hand. J. Phys. Ther. Sci. 28, 769–773. 10.1589/jpts.28.769
8
BoraM.JyotiD.GuptaD.KumarA. (2014). Effect of different distance measures on the performance of K-means algorithm: an experimental study in Matlab. arXiv:1405.7471 [Preprint].
9
CastelliniC.van der SmagtP. (2013). Evidence of muscle synergies during human grasping. Biol. Cybern.107, 233–245. 10.1007/s00422-013-0548-4
10
CatalanoM.GrioliG.FarnioliE.SerioA.PiazzaC.BicchiA. (2014). Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int. J. Robot. Res.33, 768–782. 10.1177/0278364913518998
11
CheungV. C.d'AvellaA.TreschM. C.BizziE. (2005). Central and sensory contributions to the activation and organization of muscle synergies during natural motor behaviors. J. Neurosci.25, 6419–6434. 10.1523/JNEUROSCI.4904-04.2005
12
CiprianiC.AntfolkC.ControzziM.LundborgG.RosenB.CarrozzaM. C.et al. (2011). Online myoelectric control of a dexterous hand prosthesis by transradial amputees. IEEE Trans. Neural Syst. Rehabil. Eng. 19, 260–270. 10.1109/TNSRE.2011.2108667
13
CiprianiC.ZacconeF.MiceraS.CarrozzaM. C. (2008). On the shared control of an SEMG-controlled prosthetic hand: analysis of user–prosthesis interaction. IEEE Trans Robot.24, 170–184. 10.1109/TRO.2007.910708
14
ClarkD. J.TingL. H.ZajacF. E.NeptuneR. R.KautzS. A. (2010). Merging of healthy motor modules predicts reduced locomotor performance and muscle coordination complexity post-stroke. J. Neurophysiol.103, 844–857. 10.1152/jn.00825.2009.
15
CutkoskyM. R. (1989). On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Trans. Robot. Automat.5, 269–279.
16
DauméH.III.KumarA.SahaA. (2010). Frustratingly easy semi-supervised domain adaptation, in Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, Association for Computational Linguistics (Uppsala), 53–59.
17
d'AvellaA.PortoneA.FernandezL.LacquanitiF. (2006). Control of fast-reaching movements by muscle synergy combinations. J. Neurosci.26, 7791–7810. 10.1523/JNEUROSCI.0830-06.2006
18
DeLucaC. J. (1997). The use of surface electromyography in biomechanics. J. Appl. Biomech. 13, 135–163.
19
FanJ.HeJ.TilleryS. I. H. (2006). Control of hand orientation and arm movement during reach and grasp. Exp. Brain Res.171, 283–296. 10.1007/s00221-005-0277-6
20
FarinaD.CesconC.MerlettiR. (2002). Influence of anatomical, physical, and detection-system parameters on surface. EMG86, 445–456. 10.1007/s00422-002-0309-2
21
FarinaD.JiangN.RehbaumH.HolobarA.GraimannB.DietlH.et al. (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. Neural. Syst. Rehabil. Eng. IEEE Trans. 22, 797–809. 10.1109/TNSRE.2014.2305111
22
FukudaO.TsujiT.KanekoM.OtsukaA.TsujiO. F. T. (2003). A human-assisting manipulator teleoperated by SEMG signals and arm motions. IEEE Trans. Robot. Autom. 19, 210–222. 10.1109/TRA.2003.808873
23
GregoriV.GijsbertsA.CaputoB. (2017). Adaptive learning to speed-up control of prosthetic hands: a few things everybody should know, in International Conference on Rehabilitation Robotics (London).
24
HuY.OsuR.OkadaM.GoodaleM. A.KawatoM. (2005). A model of the coupling between grip aperture and hand transport during human prehension. Exp. Brain Res.167, 301–304. 10.1007/s00221-005-0111-1
25
JarrasséN.RibeiroA. T.SahbaniA.BachtaW.Roby-BramiA. (2014). Analysis of hand synergies in healthy subjects during bimanual manipulation of various objects. J. Neuroeng. Rehabil.11:113. 10.1186/1743-0003-11-113
26
KielibaP.TropeaP.PirondiniE.CosciaM.MiceraS.ArtoniF. (2018). How are muscle synergies affected by electromyography pre-processing?IEEE Trans. Neural. Syst. Rehabil. Eng.26, 882–893. 10.1109/TNSRE.2018.2810859
27
KrasoulisA.KyranouI.Suphi ErdenM.NazarpourK.VijayakumarS. (2017). Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements, J. NeuroEng. Rehabil. 14:71. 10.1186/s12984-017-0284-4
28
LiarokapisM. V.ArtemiadisP. K.KyriakopoulosK. J.ManolakosE. S. (2013). A learning scheme for reach to grasp movements: on SEMG-based interfaces using task specific motion decoding models. IEEE J. Biomed. Health Inform.17, 915–921. 10.1109/JBHI.2013.2259594
29
MatroneG. C.CiprianiC.CarrozzaM. C.MagenesG. (2012). Real-time myoelectric control of a multi-fingered hand prosthesis using principal components analysis. J. Neuroeng. Rehabil.9:40. 10.1186/1743-0003-9-40
30
MatroneG. C.CiprianiC.SeccoE. L.MagenesG.CarrozzaM. C. (2010). Principal components analysis based control of a multi-DoF underactuated prosthetic hand. J. Neuroeng. Rehabil.7:16. 10.1186/1743-0003-7-16
31
Ortega-AuriolP. A.BesierT. F.ByblowW. D.McMorlandA. J. (2018). Fatigue influences the recruitment, but not structure, of muscle synergies. Front. Hum. Neurosci.12:217. 10.3389/fnhum.2018.00217
32
OverduinS. A.d'AvellaA.RohJ.BizziE. (2008). Modulation of muscle synergy recruitment in primate grasping. J. Neurosci.28, 880–892. 10.1523/JNEUROSCI.2869-07.2008
33
PatelV.ThukralP.BurnsM. K.FlorescuI.ChandramouliR.VinjamuriR. (2017). Hand grasping synergies as biometrics. Front. Bioeng. Biotechnol.5:26. 10.3389/fbioe.2017.00026
34
PatriciaN.TommasitT.CaputoB. (2014). Multi-source adaptive learning for fast control of prosthetics hand, in Pattern Recognition (ICPR), 2014 22nd International Conference on IEEE (Stockholm), 2769–2774.
35
PizzolatoS.TagliapietraL.CognolatoM.ReggianiM.MüllerH.AtzoriM. (2017). Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS ONE10, 1–16. 10.1371/journal.pone.0186132
36
PreveteR.DonnarummaF.d'AvellaA.PezzuloG. (2018). Evidence for sparse synergies in grasping actions. Sci. Rep.8:616. 10.1038/s41598-017-18776-y
37
RohJ.RymerW. Z.PerreaultE. J.YooS. B.BeerR. F. (2013). Alterations in upper limb muscle synergy structure in chronic stroke survivors, J. Neurophysiol.109, 768–781. 10.1152/jn.00670.2012
38
SaenkoK.KulisB.FritzM.DarrellT. (2010). Adapting visual category models to new domains, in European Conference on Computer Vision (Berlin, Heidelberg: Springer), 213–226.
39
SantelloM.BianchiM.GabicciniM.RicciardiE.SalviettiG.PrattichizzoD.et al. (2016). Hand synergies: integration of robotics and neuroscience for understanding the control of biological and artificial hands. Phys. Life Rev.17, 1–23. 10.1016/j.plrev.2016.02.001
40
SantelloM.FlandersM.SoechtingJ. F. (1998). Postural hand synergies for tool use. J. Neurosci.18, 10105–10115.
41
ScanoA.ChiavennaA.MalosioM.Molinari TosattiL.MolteniF. (2017). Muscle synergies-Based characterization and clustering of Poststroke patients in reaching movements. Front. Bioeng. Biotechnol.5:62. 10.3389/fbioe.2017.00062
42
ScanoA.ChiavennaA.MalosioM.Molinari TosattiL.MolteniF. (2018). Robotic assistance for upper limbs may induce slight changes in motor modules compared with free movements in stroke survivors: a cluster-based muscle synergy analysis. Front. Hum. Neurosci.12:290. 10.3389/fnhum.2018.00290
43
SchmidtR. A. (1975). A schema theory of discrete motor skill learning. Psychol. Rev.82:225.
44
SegilJ. L.WeirR. F. (2013). Design and validation of a morphing myoelectric hand posture controller based on principal component analysis of human grasping. IEEE Trans. Neural Syst. Rehabil. Eng.22, 249–257. 10.1109/TNSRE.2013.2260172
45
ShumanB. R.SchwartzM. H.SteeleK. M. (2017). Electromyography data processing impacts muscle synergies during gait for unimpaired children and children with cerebral palsy. Front. Comput. Neurosci.11:50. 10.3389/fncom.2017.00050
46
SmurrL. M.GulickK.YancosekK.GanzO. (2008). Managing the upper extremity amputee: a protocol for success. J. Hand Ther. 21, 160–176. 10.1197/j.jht.2007.09.006
47
TagliabueM.CiancioA. L.BrochierT.EskiizmirlilerS.MaierM. A. (2015). Differences between kinematic synergies and muscle synergies during two-digit grasping. Front. Hum. Neurosci.9:165. 10.3389/fnhum.2015.00165
48
ThakurP. H.BastianA. J.HsiaoS. S. (2008). Multidigit movement synergies of the human hand in an unconstrained haptic exploration task. J. Neurosci.28, 1271–1281. 10.1523/JNEUROSCI.4512-07.2008
49
TommasiT.OrabonaF.CaputoB. (2010). Safety in numbers: Learning categories from few examples with multi model knowledge transfer, in Proceedings of IEEE Computer Vision and Pattern Recognition Conference (No. EPFL-CONF-192668) (San Francisco, CA).
50
TommasiT.OrabonaF.CastelliniC.CaputoB. (2013). Improving control of dexterous hand prostheses using adaptive learning. IEEE Trans. Robot.29, 207–219. 10.1109/TRO.2012.2226386
51
TreschM. C.CheungV. C.d'AvellaA. (2006). Matrix Factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets matrix factorization algorithms for the identification of muscle synergies : evaluation on simulated and experimental data sets. J. Neurophysiol.95, 2199–2212. 10.1152/jn.00222.2005
52
VanoglioF.BernocchiP.MulèC.GarofaliF.MoraC.TaveggiaG.et al. (2017). Feasibility and efficacy of a robotic device for hand rehabilitation in hemiplegic stroke patients: a randomized pilot controlled study. Clin. Rehabil.31, 351–360. 10.1177/0269215516642606
53
Ziegler-GrahamK.MacKenzieE. J.EphraimP. L.TravisonT. G.BrookmeyerR. (2008). Estimating the prevalence of limb loss in the United States: 2005 to 2050. Arch. Phys. Med. Rehabil.89, 422–429. 10.1016/j.apmr.2007.11.005
Summary
Keywords
muscle synergies, centroids, synergies clustering, hand grasps, spatial synergies, temporal components, NinaPro Database
Citation
Scano A, Chiavenna A, Molinari Tosatti L, Müller H and Atzori M (2018) Muscle Synergy Analysis of a Hand-Grasp Dataset: A Limited Subset of Motor Modules May Underlie a Large Variety of Grasps. Front. Neurorobot. 12:57. doi: 10.3389/fnbot.2018.00057
Received
02 May 2018
Accepted
27 August 2018
Published
25 September 2018
Volume
12 - 2018
Edited by
Dingguo Zhang, Shanghai Jiao Tong University, China
Reviewed by
Shingo Shimoda, RIKEN Center for Brain Science (CBS), Japan; Kazutaka Takahashi, University of Chicago, United States
Updates
Copyright
© 2018 Scano, Chiavenna, Molinari Tosatti, Müller and Atzori.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Alessandro Scano alessandro.scano@stiima.cnr.it
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.