Smart Textiles that Teach: Fabric-Based Haptic Device Improves the Rate of Motor Learning

by   Vivek Ramachandran, et al.

People learn motor activities best when they are conscious of their errors and make a concerted effort to correct them. While haptic interfaces can facilitate motor training, existing interfaces are often bulky and do not always ensure post-training skill retention. Here, we describe a programmable haptic sleeve composed of textile-based electroadhesive clutches for skill acquisition and retention. We show its functionality in a motor learning study where users control a drone's movement using elbow joint rotation. Haptic feedback is used to restrain elbow motion and make users aware of their errors. This helps users consciously learn to avoid errors from occurring. While all subjects exhibited similar performance during the baseline phase of motor learning, those subjects who received haptic feedback from the haptic sleeve committed 23.5 evaluation phase. The results show that the sleeve helps users retain and transfer motor skills better than visual feedback alone. This work shows the potential for fabric-based haptic interfaces as a training aid for motor tasks in the fields of rehabilitation and teleoperation.


page 5

page 8

page 14


Can rhythm be touched? An evaluation of rhythmic sketch performance with augmented multimodal feedback

Although it has been shown that augmented multimodal feedback has a faci...

A Computational Framework for Motor Skill Acquisition

There have been numerous attempts in explaining the general learning beh...

uBAM: Unsupervised Behavior Analysis and Magnification using Deep Learning

Motor behavior analysis is essential to biomedical research and clinical...

Visuohaptic augmented feedback for enhancing motor skills acquisition

Serious games are accepted as an effective approach to deliver augmented...

Haptic human-human interaction does not improve individual visuomotor adaptation

Haptic interaction between two humans, for example, parents physically s...

High Aptitude Motor Imagery BCI Users Have Better Visuospatial Memory

Brain computer interfaces (BCI) decode the electrophysiological signals ...

DroneStick: Flying Joystick as a Novel Type of Interface

DroneStick is a novel hands-free method for smooth interaction between a...

1 Introduction

Over the past decade, robotic teaching aids have been developed to train people in a variety of motor activities by providing sensory feedback [20, 50, 46, 25, 33, 45, 9]. Motor learning is an error-driven process and the rate of learning depends on how sensory feedback is provided during training. Typically, people learn to rectify their errors based on a combination of visual, auditory, and haptic feedback [50]. Motor training by haptic feedback is of particular interest to researchers because it can be applied directly to the part of the body where corrective action is needed [19, 57]. This effectiveness hinges on two critical and intimately linked factors - the haptic interface and the training method [30, 36, 12].

Haptic interfaces should provide reliable, intuitive, and clear feedback when required and be unobtrusive when they are not. Within the scope of motor training, haptic interfaces are generally employed in two fields - rehabilitation and teleoperation [33, 1]. Existing haptic interfaces are mostly grounded i.e., they are fixed to a rigid base, which limits their applicability to tasks that do not require much user displacement, such as object manipulation [16]. With the miniaturisation of electronic components, such as integrated circuits and batteries, wearable interfaces have become a viable alternative because they allow users a greater degree of mobility [39]. This increased mobility has been demonstrated through multiple exoskeletons for gait rehabilitation and robot teleoperation that improve user performance in the specific tasks for which they are designed [37, 21, 15, 42, 53, 3, 27, 2, 40, 5, 48, 47]. However, these interfaces are bulky because of the heavy actuators that they use, such as motors and pumps, which cause user fatigue over prolonged use. In fact, actuators are not always necessary to help users perform motor tasks better. A recent study showed how a simple, unpowered clutch and spring ankle exoskeleton can increase human walking efficiency by re-purposing their expended energy [10]. Indeed, new lightweight, fabric-based haptic interfaces have been developed in recent years to circumvent the problems associated with using heavy actuators and promote user comfort for continuous usage [26, 57, 4, 27, 11, 44, 41, 55, 8, 17, 24, 58] (Table 1). Amongst these fabric-based haptic interfaces, certain interfaces use electrostatic adhesive (EA) clutches to apply kinesthetic feedback through movement braking and passive springs [18, 17, 44, 14, 13]. They operate at low power ( ) and are easily integrated into other textile-based wearables, such as clothing. These new types of fabric-based, low-power-consuming haptic interfaces are less complex in mechanical design compared to existing interfaces, which are composed of rigid components. The absence of actuators compels users to rely on the feedback to both identify and correct their errors, which helps them learn the motor task faster. Furthermore, unlike existing interfaces, these interfaces are not designed for any one specific task alone. Rather, they can be re-purposed to help train users in a variety of motor tasks.

Table 1: Comparison of existing textile-based haptic devices and the one presented in this study

A successful haptic training method ensures that users are immersed in learning the motor activity and are provided timely feedback to improve their performance over the course of training. Existing haptic-based training methods can be broadly divided into two categories - haptic guidance and error amplification [35, 6]. In the former, the haptic system physically guides users to minimize errors they commit during training and accomplish a task. In the latter, the haptic system amplifies user errors to intentionally increase the difficulty of the task [34, 23]. Studies show that haptic guidance increases user performance during training compared to baseline conditions, but that performance levels precipitate when the guidance is not provided [43]. Some have posited that the guidance overtly increases user dependency [56]. This dependency curtails both skill retention and skill transfer post-training. On the other hand, error-amplifying systems deliver longer periods of skill retention at the expense of longer training periods [54]. However, some studies state that the comparative effects of haptic guidance and error amplification on motor learning cannot be generalised, because they are subject to the type of motor activity [35, 32]. Nonetheless, there is some consensus that the method of feedback provision and the resulting outcome is dependent on the user skill level [30, 31]. Accordingly, novice users benefit more from haptic guidance, whereas expert users gain more from error amplification. There is also recent evidence that suggests, allowing users to select the type and magnitude of haptic feedback can accelerate the rate of motor learning [45, 29]. These studies conclusively show that users must actively utilize haptic feedback to rectify their errors and acquire motor skills. However, so far, this active motor learning has only been demonstrated with haptic interfaces made with rigid and bulky devices [45].

Here, we present a novel haptic interface that promotes active user involvement in rectifying movement errors during motor learning and is composed of only soft, fabric-based components. The interface consists of a programmable elbow sleeve that comprises multiple fabric-based EA clutches, which can rapidly restrict the joint movements of a wearer. In this study, the haptic sleeve is programmed to afford users a margin of error, but provides a motion-blocking feedback to 1) make users aware of their error, and 2) prevent these errors from growing. The haptic feedback operates like a wall that confines the motion range. Through heightened awareness of their errors, users can consciously avoid them in future iterations of the task. In this study, we experimentally show that the proposed haptic interface increases the training success to learn, retain, and transfer motor skills in a drone teleoperation task.

2 Results

2.1 Design of the electroadhesive haptic sleeve

The electroadhesive haptic sleeve is a fabric-based exoskeleton that can be programmed to constrain elbow extension and flexion. It is composed of two electroadhesive clutches and three body attachments (Figure 1A, Figure S1, Supporting Information). Each clutch is a parallel plate capacitor composed of overlapping dielectric-coated electrodes. The EA clutches used in this study are an improved version of our previously described fabric-based clutches to generate higher holding forces[44]. The higher forces are generated by replacing the fabric-electrodes of our previous clutches with metallized biaxially-oriented polyethylene terephthalate sheets. This was done to avoid dielectric cracking on the electrode surface, which was observed with the fabric electrode over prolonged usage. Furthermore, surface irregularities and wrinkling were prevented by replacing the fabric electrodes with the metallized plastic sheets.

Figure 1: Wearable haptic system to train users for motor activities. (A) Design and composition of the fabric-based electroadhesive clutch. When a high voltage is applied across the dielectric-coated electrode plates, the clutch engages and the clutch plates adhere to each other, thus increasing the maximum holding force. (B

) The maximum holding force of the clutch is proportional to the applied voltage. Each data point is the mean value of five measurements trials and the error bars represent one standard deviation from this mean. (

C) When the clutch is disengaged by removing the voltage, the adhesion decreases and the time required for the holding force to drop (gray shading) is measured as the disengagement time. (D) The wearable system is composed of two clutches, one each on the ventral and dorsal faces of the human arm. The clutches are held in place with the help of attachment straps on the forearm, the elbow joint, and the upper arm. The ventral clutch restricts elbow extension and the dorsal clutch restricts elbow flexion.

When a voltage in the order of is applied across the overlapping electrode plates, they adhere to each other through electrostatic adhesion. As a result, the maximum holding force i.e., the tensile force needed to ply the plates apart longitudinally increases. In this engaged state, the magnitude of the holding force is dictated by the applied voltage (Figure 1B). When the clutch is disengaged by removing the high voltage, it recovers its initial mechanical properties within (Figure 1C). The elbow joint is a hinge-type joint. Hence, the motion of the elbow joint can be blocked in a targeted manner by having one clutch each to block the flexion and extension of the forearm about the upper arm, without affecting the mobility of the remaining body. To block forearm extension and flexion independently, one clutch is attached to the ventral face of the arm and another to the dorsal face, respectively. Each clutch can be extended longitudinally from its rest length by virtue of low stiffness springs, which ensure that the user does not experience any hindrance to natural mobility when haptic feedback is not applied. When the voltage is applied, the electrostatic adhesion prevents further longitudinal extension of the clutch and thus constrains joint rotation. The perception of the haptic feedback is quantified by the magnitude of the holding force of the EA clutches. This magnitude is dependent on the dimension of the clutch plates, the dielectric thickness, the number of the clutch pairs and the applied voltage. Perception of the haptic devices can be tuned by changing any of these parameters. The ventral and dorsal clutches are worn by placing them between body attachments that are anchored to the forearm, the elbow joint, and the upper arm (Figure 1D, Movie S1, Supporting Information). The forearm and the upper arm attachments are made of long strips of elastane bonded to loop fastener strips (Velcro) on the exterior and a layer of silicone (Ecoflex 00-20, Smooth-On) on the interior (Figure S2, Supporting Information). Once the clutches are mounted, a custom dial strap with a BOA tightening system (3M ACE Brand, MN, USA) is worn over the upper arm attachment to secure the anchoring. The manufacturing process of the forearm and upper arm attachments involves bonding the elastane to a membrane of cured Ecoflex 00-20 by first, casting an uncured layer of Ecoflex 00-20 on the elastane, then, placing the cured membrane on top of the uncured layer and finally, oven-curing the entire composite. This part of the process was adapted from earlier work carried out by colleagues [51]

. The Ecoflex is comfortable and adheres to human skin. The loop fastener on the body attachment mates with the hook fastener of each clutch. The elbow joint attachment is necessary for the functioning of the dorsal clutch. A semi-cylindrical 3D printed guide is sewn onto the pad at the elbow joint to allow for longitudinal extension of the dorsal clutch (Figure S3, Supporting Information). Two inertial measurement units (IMU, by Xsens Technologies) are attached to the forearm and upper arm body attachments to measure the elbow angle.

2.2 Motor learning to teleoperate a drone

We assessed the applicability of the EA haptic sleeve as a teaching aid for drone teleoperation tasks and used user errors as the performance metric to determine skill retention and transfer. The experiments consisted of two drone teleoperation tasks - path following to examine the effect of haptic training on the retention of motor skills, and waypoint navigation to determine the transfer of those skills (Figure 2A). Both tasks are frequently used methods in the context of drone teleoperation. Path following is often employed to navigate drones through long, narrow pipelines for maintenance and inspection or navigate through cluttered environments that are inaccessible to humans. Waypoint navigation is commonly used to help drones map areas of interest or carry out aerial monitoring [38]. For the path following motor task, subjects were asked to control the altitude of a drone flying through a cylindrical tube with multiple vertical bends while avoiding collisions with the walls. For the waypoint navigation motor task, subjects were asked to control the altitude of a drone through a series of rings positioned at different heights. The performance of the path following task was computed by measuring the altitude error with respect to the desired centreline of the tube throughout its length. The performance of the waypoint navigation task was computed by measuring the difference in height between the drone and the centre of each traversed ring. Waypoint navigation is chosen for the skill transfer test because it is similar to path following. Greater task similarity between the skill retention and transfer tests improves the chances of skill transfer. Despite their similarities though, waypoint navigation provides users more freedom in terms of drone movement than path following as the performance is only evaluated at discrete intervals. However, this additional freedom comes at the risk of increased movement errors if the user does not retain and transfer the necessary skills to control the drone.

Figure 2: Experimental pipeline for motor learning (A) Subjects performed two motor tasks - waypoint navigation (rings) and path following (tubes). Cubic splines produced reference trajectories for baseline and evaluation phases of both tasks while sine curves defined by amplitude () and wavelength () were used for the training phase of path following. (B) Subjects in each group received differentiated feedback when they traversed an error threshold within the training phase tubes: first-person view (FPV), FPV + arrows, or FPV + EA clutches. The training sessions are listed chronologically with the and values. (C) A linear mapping was made between the subject’s elbow angle and the drone’s altitude. (D) The experimental pipeline is shown in chronological order.

There is a linear mapping between the elbow angle measured by the IMUs and the altitude of the simulated drone (Figure 2C, Movie S2, Supporting Information). The IMU readings of the elbow angle are continuous i.e., they are non-discrete. The amplitude limits are determined by initially measuring the elbow angles for each subject at maximum forearm flexion and extension and then, scaling the altitude limits of the drone in the simulated environment to match the elbow angle limits. The path following task consisted of three phases - baseline, training, and evaluation, while the waypoint navigation tasks consisted of two phases - baseline and evaluation. In chronological order, subjects performed the baseline phase of waypoint navigation, the baseline, training, and evaluation phases of path following, and finally, the evaluation phase of waypoint navigation (Figure 2D). The baseline and evaluation phases of both the waypoint navigation and path following tasks consisted of three sessions each, during which subjects received only visual feedback from the drone’s camera in first-person view (FPV) via a desktop display (Movie S3, Movie S4, Supporting Information). Cubic splines were used to produce the reference trajectories for the baseline and evaluation phases of both tasks. The tubes for the training phase of the path following task were produced with sine curve reference trajectories. This is due to the periodic nature of sine curves, which allowed participants to become acclimatized to the nature of the experiment. Since each sine curve is defined by a specific combination of an amplitude and a wavelength, three amplitudes (, , and ) and three wavelengths (, , and ) were used to produce nine distinct tubes. Prior to the commencement of the human subject studies, the chronology of the training sessions was set by randomizing the order of the amplitude and wavelength combinations (Figure 2B). This was done to prevent biasing effects due to amplitude or wavelength. This randomization was only performed once, and the chronological order was maintained for all subjects. During the training phase, subjects received additional sensory feedback and were grouped accordingly: group 1 - FPV; group 2 - FPV with yellow arrows indicating corrective action; and group 3 - FPV with EA clutches providing haptic feedback to block elbow joint rotation (Figure 2B, Movie S5, Movie S6, Movie S7, Supporting Information). An error threshold of was set above and below the centreline of the training tubes, which was not visible to the subject. Whenever subjects in groups 2 and 3 flew the drone past this threshold, they received the additional sensory feedback (arrows/EA clutches) to prevent them from colliding with the tube wall. For group 2, the arrows pointed downward when the drone was close to the ceiling and they pointed upward when the drone was close to the floor. Similarly, for group 3, the dorsal clutch blocked elbow flexion to prevent ceiling collisions and the ventral clutch blocked elbow extension to prevent floor collisions.

2.3 Acquisition and retention of motor skills

Subject performance errors were measured for both teleoperation tasks and the temporal changes in their performance described the motor learning characteristics of each group (Figure 3A). Differences in performance errors between groups for the same task phases revealed the effects of differentiated feedback provision. The performance errors of the path following task were compared to understand how each of the groups attempted to acquire and retain motor skills specific to this task (Figure 3C). Subjects in groups 2 and 3 committed fewer errors (32.89% and 37.56% respectively) in the evaluation phase compared to the baseline phase for the path following task. On the other hand, subjects in group 1 committed 12.65% more errors during the evaluation phase of path following compared to the baseline phase. A paired t-test showed that the performance of subjects in group 1 (FPV) deteriorated from baseline to evaluation for the path following task indicating that the training phase between the baseline and evaluation phases did not benefit the subjects and may have even been deleterious to them (-2.535, 0.05). On the other hand, subjects in group 2 (FPV + arrows) and group 3 (FPV + EA clutches) performed significantly better in the evaluation phase compared to the baseline, which implies that they were able to learn the task over the course of training (group 2: =23.76, 0.01; group 3: =10.041,

0.01). Indeed, a one-way analysis of variance (ANOVA) followed by a

post-hoc Holm-Sidak corrected t-test showed that there were no significant differences between the performances of the three groups at baseline (=0.34, =0.71). This outcome ruled out any biases that might have been introduced by relative differences in subject task expertise prior to training. On the other hand, the same statistical treatment revealed that the performance errors of subjects in group 2 and group 3 were lower than those of subjects in group 1 after evaluation, thus showing the retention of motor skills after training (=84.93, 0.01). During the training phase of path following, the subjects of each group received differentiated feedback as explained earlier. In addition to the received sensory feedback, the performance of each group were subjected to the effects of different amplitude and wavelength combinations that defined each tube’s reference trajectory (Figure 3D). For each group, these effects were resolved by averaging their performance errors of a set of three training sessions. Each set had the same amplitude ( - sessions 1, 3, 9; - sessions 2, 4, 7; - sessions 5, 6, 8; Figure 3E) or the same wavelength ( - sessions 2, 5, 9; - sessions 3, 4, 6; - sessions 1, 7, 8; Figure 3F). For a given amplitude or wavelength, each comparison was made between groups using a one-way ANOVA followed by a post-hoc Holm-Sidak corrected t-test. Groups 2 and 3, which received augmented visual and haptic feedback, respectively, performed significantly better than group 1 for each training set grouped by amplitude (=0.5, =20.64, 0.01; =1, =16.6, 0.01; =1.5, =12.33, 0.01). Similar results were obtained when comparing the performance errors of groups 2 and 3 with respect to group 1 for training sets grouped by wavelength ( = 8, =14.75, 0.01; = 10, =23.5, 0.01; = 12, =22.18, 0.01). These two results pertaining to the amplitude and wavelength effects on group performance errors showed that the additional sensory feedback alerted subjects in groups 2 and 3 in time to switch between extension and flexion before colliding with the tube walls. A repeated measured ANOVA followed by a post-hoc Holm-Sidak corrected t-test showed that subjects in each group performed progressively worse as the amplitude of the tube reference trajectory increased with observable significant differences (group 1: =40.71, 0.01; group 2: =144.14, 0.01; group 3: =50.75, 0.01). This is expected given the increasing difficulty to alternate between forearm extension and flexion at larger elbow angles. The post-hoc Holm-Sidak corrected t-test values for pairwise comparisons within-group for different amplitudes are reported (Data file S1, Supporting Information). The repeated measures ANOVA did not reveal significant differences in performance for group 1 for varying wavelengths (=2.74; =0.09). However, a repeated measures ANOVA followed by post-hoc Holm-Sidak corrected t-test revealed that groups 2 and 3 performed significantly better for a wavelength of compared to (group 2: =4.93, =0.01; group 3: =4.71, =0.02). The less marked differences in wavelength effects within each group was perhaps due to the chosen drone forward speed. Different drone speeds might have produced more noticeable differences.

Figure 3: Results from the motor learning human subject study. (A) Motor learning curves of each subject group are plotted as a function of mean performance error against experimental phases of both motor tasks in chronological order. Subject groups: group 1 (FPV, red), group 2 (FPV + arrows, black), group 3 (FPV + EA clutches, green); waypoint navigation task - baseline (light blue, striped) and evaluation (light blue, solid) phases; path following - baseline (pink, striped), training (light purple, solid), and evaluation (pink, solid) phases. (B)-(C) Between-group and within-group comparisons of performance errors during baseline and evaluation phases of waypoint navigation and path following tasks, *P0.05, **P0.01. (D) Heat map of group performance errors during the training phase of path following shows the effects of amplitude () and wavelength () combinations. (E)-(F) Comparisons between-group and within-group of subject performance during the training phase of path following. Each plotted point is the average performance error of a group for three training sessions with the same amplitude (E) or wavelength (F). Comparison between group 1 and group 3, P0.01. # Comparison between group 1 and group 2, P0.01. Within-group comparison between different amplitudes or wavelengths colour-coded to match groups, P0.01. All error bars in this figure show one standard deviation above and below the reported mean performance error.

2.4 Transfer of motor skills

Subject had to utilize the motor skills garnered during the path following task to improve upon their baseline performance in the waypoint navigation transfer task (Figure 3B). A one-way ANOVA with post-hoc Holm-Sidak t-test corrections established no significant differences between the baseline performances of each group, thus reinforcing the absence of expertise bias as shown for the baseline phase results of path following (=0.12, =0.88). While all subjects in each group performed better during the evaluation phase of the waypoint navigation task compared to their baseline, the relative improvement in performance was more pronounced for subjects in group 2 and 3, where the error percentage drop was 42.23% and 45.64% respectively, compared to the 9.97% drop for subjects in group 1. A paired t-test showed that the performance errors did not change significantly for subjects in group 1 between the baseline and evaluation phases (=1.27, =0.23). This may be attributed to inadequacies in motor skills acquired during the path following task. In the same vein, subjects in groups 2 and 3, who exhibited performance improvement for the path following task, produced similar results for waypoint navigation. There were significant differences between their baseline and evaluation phase performances showing that the skills acquired from path following transferred (group 2: =3.82, 0.01; group 3: =7.31, 0.01). A one-way ANOVA with post-hoc Holm-Sidak t-test corrections showed that subjects in groups 2 and 3 performed significantly better during the evaluation phase compared to group 1, thereby showing the relative benefits of having been trained with additional sensory feedback (=0.12, 0.01).

Figure 4: Subject responses to questionnaires filled after the baseline, training, and evaluation phases for path following. Questions posed determined various qualitative aspects of their motor training, including their comfort levels with both the wearable interface and the control of the drone control, as well as their own assessment of performance improvement at the end of each of these phases. At the end of the training phase, subjects in the test groups that received additional feedback in the form of corrective arrows or EA clutches rated the degree to which this feedback helped them over the course of the phase. Finally, subjects provided an overall rating of their experience in participating in the motor training exercise. With the exception of the yes/no response to the question about their performance self-assessment, responses to all other questions were provided on a 7-point Likert scale between 0 and 6.

2.5 Subjective assessment of drone teleoperation and sleeve comfort

We assessed each subject’s comfort in learning to control the drone while wearing the haptic sleeve using questionnaires. Since differentiated feedback was only provided during the path following phase, subjects filled three questionnaires, each corresponding to the three phases of path following - baseline, training, and evaluation at the conclusion of each respective phase. Subjects were asked to assess the degree to which they were in control of the drone’s movement, their level of comfort with the haptic interface, and a self-evaluation of their performance improvement over the recently concluded phase by indicating a grade on a 7-point Likert scale (0-6). For all groups, at least of the subjects rated both their degree of control over the drone’s movement and their level of comfort with the wearable system between 4 and 6 (both included) for all three phases (Figure 4). Subjects of groups 2 and 3 were asked to rate the helpfulness of the additional sensory feedback (arrows and EA clutches) in performing the tasks as part of the training questionnaire. Despite the quantitative results which indicated that subjects in these two groups improved upon their baseline performance, only of them found the additional sensory feedback (arrows or EA clutches) to be qualitatively helpful. It is interesting that participants do not perceive the feedback as qualitatively helpful and yet, they tend to perform better. The rather counter intuitive response from the subjects may be due to the additional feedback creating sensory overload, but this claim cannot be verified as part of our study. For the self-evaluation of performance improvement subjects could answer “Yes” or “No”. For group 1, the percentage of subjects who responded to their self-evaluation of performance improvement as “Yes” increased (70% - “Baseline”; 80% - “Training”; 90% - “Evaluation”). On the other hand, the percentage reduced for group 2 (90% - “Baseline”; 70% - “Training” and “Evaluation”). The percentage of affirmative responses increased for group 3 over the phases (80% - “Baseline” and “Training”; 90% - “Evaluation”). In the evaluation questionnaire for all groups, subjects were also asked to rate their overall experience in participating in the experimental study. All subjects rated their overall experience between 4 and 6 in participating in the experiments.

3 Conclusion

We described a wearable haptic sleeve that uses fabric-based EA clutches to impart kinesthetic feedback by blocking body joint movement and experimentally showed its functionality as a teaching aid for motor activities in drone teleoperation tasks. This study examined and compared the effects of providing haptic feedback for motor training with different forms of visual feedback.

The results show that subjects in the control group, who received FPV visual feedback did not acquire and retain sufficient motor skills to improve path following performance and subsequently, were unable to perform better in the transfer waypoint navigation task. Instead, subjects who received either augmented visual feedback in the form of arrows to correct elbow movement or haptic feedback from the electroadhesive haptic sleeve to physically block elbow rotation, displayed performance improvement from baseline to evaluation for both path following and waypoint navigation. This performance improvement could be attributed to users relying on both types of feedback to determine their errors and consciously learning to to avoid them. This conclusion is reinforced by the observable improvement in performance level with respect to the control group immediately after receiving additional haptic/visual feedback. While this additional feedback assists subjects in identifying and avoiding errors during the training phase, their newly acquired motor skills are retained even after the training phase. This show that the subjects did not become overtly dependent on the additional feedback.

The increase in performance errors with the amplitude of tube reference trajectories for all subject groups can be ascribed to the difficulty in rapidly alternating between forearm extension and flexion, especially when approaching the maximum and minimum elbow angle limits. Indeed, the performance errors of specific training sessions on average were higher for all groups than individual sessions within the baseline and evaluation phase due to the training tubes having larger amplitudes. The relatively small differences in performance for all groups with changing tube wavelength may be due to the chosen drone speed. For higher drone speeds, we might expect to see greater performance differences with fewer errors being committed for larger tube wavelengths.

The comparable beneficial effects of haptic feedback and of corrective arrow displays are noteworthy because augmented visual feedback is generally accepted as a benchmark in feedback-based training. Indeed, there are no observable statistically significant differences between the two forms of additional feedback based on the data collected during the subject studies. While the haptic feedback is just as effective as a teaching aid as augmented visual feedback, haptic feedback could be qualitatively more helpful than augmented visual feedback when provided directly to the part of the body responsible for erroneous motion. This is because concentrating the relay of augmented sensory feedback through a single feedback channel (vision) can severely increase the risk of sensory overload over prolonged periods of robot teleoperation [52]. Furthermore, haptic feedback could be used instead of vision when visual feedback from the robot is occluded [52, 28]. Certainly, visual occlusions are not uncommon when operators need to inspect infrastructure, for example maintenance after the occurrence of a natural disaster. In these instances, first-person visual perspective may not reveal concealed structures in the robot’s periphery due to a variety of reasons, including insufficient lighting. Another possibility is that visual feedback may be cut off from the operator intermittently due to poor transmission. In addition, haptic feedback can be provided to the visually impaired. In this work, the haptic feedback was binary i.e., either it behaved as a compliant cloth or it blocked the range of motion. In the future, lightweight variable stiffness technologies which use low melting point materials and shape memory materials that can provide a range of blocking forces could also be employed for different training tasks. Furthermore, extension of this technology to other joints, such as the wrist and fingers, could be used for more complex teleoperation and rehabilitation tasks.

4 Experimental Section

Manufacturing of electroadhesive haptic clutches

Each clutch consisted of three pairs of dielectric-coated electrodes that were interleaved in an interdigitated architecture. The electrodes ( ) were biaxially-oriented polyethylene terephthalate that were metallised on one surface. The metallised surface was coated with a layer of high- dielectric ink, a ferroelectric composition of Barium Titanate and Titanium Dioxide (Luxprint 8153, DuPont). The dielectric was oven-cured at for . Post-curing, the solid dielectric that remained was thick. The non-metallised surface of each electrode was bonded to a sheet of polymethyl methacrylate (PMMA). Each of the three pairs of interleaved electrodes overlapped with their dielectric surfaces in contact. By virtue of the interdigitated architecture, one electrode of each pair was maintained at high voltage with respect to its paired electrode that was grounded when the clutch was engaged. Therefore, the two sets (high voltage and ground) of three electrodes that were maintained at the same voltage were bonded together using electrically conductive copper‐plated polyester fabric tape (CN‐3190, 3M). Each set was adhered to a strip of woven fabric. long silicon springs (Zim Fluo diameter elastic fishing line, Autain Peche) were attached between one electrode set and the woven fabric strip adhered to the other electrode set via bolted snap fasteners. A sleeve of rip-stop fabric was used to ensure that the electrodes plates do not move laterally. Hook fastener (Velcro) strips are stitched onto the woven fabric to mount the clutches to the forearm and upper arm body attachments. The clutches are operated by a customized printed circuit board that can apply high voltages when serial commands are sent via Bluetooth from the computer running the motor learning simulations. The net weight of each clutch was .

Mechanical and electrical characterisation of clutches

The maximum holding force of each clutch was measured by performing force-displacement tests with a materials testing machine Instron 5965 (Instron, Norwood, MA, USA). The ends of the clutch were fixed to the vices of the tensile tester. Each of the three clutch pairs have an initial dielectric overlap area of 120 . The clutch was engaged at different voltages (, , , and ) and loaded in tension at a rate of . The maximum holding force was reported for each voltage as an average of five measurement trials.

To determine the clutch disengagement time, the clutch was initially fixed between the vices of the tensile tester and then engaged at . The clutch was loaded in tension until it reached the maximum holding force at which point the clutch is disengaged by setting the applied voltage from the board to zero. At the same time, a serial command was sent to the tensile tester to register the time of disengagement. The disengagement characteristics were reported by averaging the precipitating holding force measurements for three trials. The disengagement time was measured by computing the drop from the initially measured maximum holding force corresponding to the applied voltage.

Simulation framework for drone teleoperation

The drone flight tasks were simulated using Gazebo, an open source robotic simulator


. Gazebo provided 3D visual scene rendering and simulated on-board sensors such as RGB cameras. The simulation environment and drone model were based on our previous work on machine-learning-based multi-drone control

[49]. For the flight tasks, single-integrator dynamics were used to move the drone through the environment. The drone flight was restricted within the x-z plane and moved forward at a constant speed of . For each of the three baseline and evaluation sessions of waypoint navigation, a set of 20 rings of 1m diameter each were generated in the environment. The rings were spaced at equal intervals of along the x-axis with the openings facing the drone’s direction of travel. For each session, the reference trajectory passing through the ring centres was created using cubic spline, such that spline slope was zero at the ring centres. Furthermore, the z-position of the ring centres were constrained to remain between and above the ground. For the baseline and evaluation sessions of path following, diameter tubes measuring in horizontal length were rendered in the same x-z plane as the drone, appearing in front of it. The centreline of each tube, which was the drone’s reference trajectory was also produced using cubic splines. The trough and the peak of the trajectory were constrained to and respectively. For the training sessions, 1m diameter tubes measuring in horizontal length were produced with tube centreline reference trajectories as sine curves. To generate nine distinct tubes, nine distinct combinations of amplitude and wavelength were used. The three amplitudes values were , , and , and the three wavelength values were , , and .

Human subject study

A total of 30 adult subjects (ages between 23 and 42, mean = 29.86, standard deviation = 4.32) were recruited, primarily from the university. Each of the subjects were healthy, had normal hearing and normal or corrected-to-normal vision. None of the subjects had prior knowledge of the tasks they were expected to perform. They were randomly assigned to the three groups with 10 subjects each (6 men, 4 women in each group). The subjects provided written informed consent, and the study was approved by the EPFL Human Research Ethics Commission. Before the motor learning studies were performed, preliminary psychophysical measurements were made to ascertain the Just Noticeable Difference in terms of the holding force needed to constrain their elbow flexion and extension. The applied DC voltage was directly proportional to the maximum holding force. A voltage value of corresponding to the results of the Just Noticeable Difference was used as the operating voltage for the engagement of the clutches during the experiments. The dimensions of the haptic devices were determined a priori by using the anthropometric data collected from 10 individuals, specifically, their forearm length, their upper arm length, and the changes in length on the ventral and dorsal arm faces associated with forearm extension and flexion about the upper arm respectively. While the haptic devices are robust to repeated usage, the elastic fishing line used as the low-stiffness spring can become brittle at the knots, as is often the case with some types of vulcanized rubber. To prevent any deleterious effect as a result of this changing mechanical property, the springs were replaced after every 5 subjects. The clutch plates were operated at a voltage that would not cause electrical shorting. To prevent the accumulation of surface charges on each clutch plate due to the DC voltage operation, the polarity of the clutch plates was reversed for each subject after every session of each experimental phase.


All analysis on the data gathered from the simulated flight tasks was carried out in MATLAB 2019b (Mathworks, MA, US). The paired t

-test is used when comparing a pair of normally distributed sets of data subjected a single condition, here the effect of the training condition for each group. To compare three or more groups of normally distributed data obtained from different sample populations, an Analysis of Variance (ANOVA) needs to be performed first to ascertain whether there are groups of data that are significantly different from other groups of data. After ascertaining these differences, a post hoc

t-test is performed, whereby the pair-wise t

-test statistics are corrected to avoid minimise the occurrence of false positive results. A repeated measure ANOVA is required when working with multiple sets of data collected for the same population, but subjected to different treatments. A paired-sample

t-test was performed to compare the performance errors of baseline and evaluation phases for each group for both path following and waypoint navigation tasks. To compare the differences between groups for the same phase (baseline and evaluation of each task), a one-way ANOVA followed by post-hoc Holm-Sidak correction t-test was employed. The same method was used for the analysis of the path following training phase, where statistical comparisons were made between groups for specific amplitude or wavelength values. For the same group, comparisons were performed for changing amplitude or wavelength values by carrying out a repeated measures ANOVA followed by a post-hoc Holm-Sidak corrected t-test. [7]. Each of the significance tests were performed by assuming a significance level of =0.05. All statistical results are available in Data file S1, Supporting Information.

Supporting Information

Supporting Information is available from the Wiley Online Library or from the author.


The authors would like to thank Olexandr Gudozhnik for fabricating the electronic board used to power the electroadhesive clutches and his continued technical support over the course of the human subject experiments. We would also to thank Ronan Hinchet for his insightful comments to improve the performance of electroadhesive clutches. We greatly appreciate the support of Enrico Ajanic whose endeavours have immensely benefited us in the writing of this article. This work was partially funded by the European Union’s Horizon 2020 research and innovation program under grant agreement ID: 871479 AERIAL-CORE as well as the Swiss National Science Foundation through the Swiss National Centre of Competence in Research (NCCR) Robotics.



  • [1] D. A. Abbink, M. Mulder, and E. R. Boer (2012) Haptic shared control: smoothly shifting control authority?. Cognition, Technology & Work 14 (1), pp. 19–28. Cited by: §1.
  • [2] A. T. Asbeck, S. M. De Rossi, K. G. Holt, and C. J. Walsh (2015) A biologically inspired soft exosuit for walking assistance. The International Journal of Robotics Research 34 (6), pp. 744–762. Cited by: §1.
  • [3] A. T. Asbeck, K. Schmidt, I. Galiana, D. Wagner, and C. J. Walsh (2015) Multi-joint soft exosuit for gait assistance. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 6197–6204. Cited by: §1.
  • [4] M. Bianchi, G. Valenza, A. Serio, A. Lanata, A. Greco, M. Nardelli, E. P. Scilingo, and A. Bicchi (2014) Design and preliminary affective characterization of a novel fabric-based tactile display. In 2014 IEEE Haptics Symposium (HAPTICS), pp. 591–596. Cited by: §1.
  • [5] J. Bimbo, C. Pacchierotti, M. Aggravi, N. Tsagarakis, and D. Prattichizzo (2017) Teleoperation in cluttered environments using wearable haptic feedback. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3401–3408. Cited by: §1.
  • [6] A. E. Bouchard, H. Corriveau, and M. Milot (2015) Comparison of haptic guidance and error amplification robotic trainings for the learning of a timing-based motor task by healthy seniors. Frontiers in systems neuroscience 9, pp. 52. Cited by: §1.
  • [7] G. Cardillo (2008) Anovarep: compute the anova for repeated measures and holm-sidak test for multiple comparisons if anova is positive. MATLAB Central File Exchange. Cited by: §4.
  • [8] C. W. Carpenter, S. T. M. Tan, C. Keef, K. Skelil, M. Malinao, D. Rodriquez, M. A. Alkhadra, J. Ramírez, and D. J. Lipomi (2019) Healable thermoplastic for kinesthetic feedback in wearable haptic devices. Sensors and Actuators A: Physical 288, pp. 79–85. Cited by: §1.
  • [9] J. C. Chan, H. Leung, J. K. Tang, and T. Komura (2010) A virtual reality dance training system using motion capture technology. IEEE transactions on learning technologies 4 (2), pp. 187–195. Cited by: §1.
  • [10] S. H. Collins, M. B. Wiggin, and G. S. Sawicki (2015) Reducing the energy cost of human walking using an unpowered exoskeleton. Nature 522 (7555), pp. 212–215. Cited by: §1.
  • [11] H. Culbertson, C. M. Nunez, A. Israr, F. Lau, F. Abnousi, and A. M. Okamura (2018) A social haptic device to create continuous lateral motion using sequential normal indentation. In 2018 IEEE Haptics Symposium (HAPTICS), pp. 32–39. Cited by: §1.
  • [12] H. Culbertson, S. B. Schorr, and A. M. Okamura (2018) Haptics: the present and future of artificial touch sensation. Annual Review of Control, Robotics, and Autonomous Systems 1, pp. 385–409. Cited by: §1.
  • [13] S. B. Diller, S. H. Collins, and C. Majidi (2018) The effects of electroadhesive clutch design parameters on performance characteristics. Journal of Intelligent Material Systems and Structures 29 (19), pp. 3804–3828. Cited by: §1.
  • [14] S. Diller, C. Majidi, and S. H. Collins (2016) A lightweight, low-power electroadhesive clutch and spring for exoskeleton actuation. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 682–689. Cited by: §1.
  • [15] A. Duschau-Wicke, M. Guidali, A. Schück, H. Vallery, and R. Riener (2010) Haptic constraints for rehabilitation robots: an overview. Biomed Tech 55, pp. 1. Cited by: §1.
  • [16] A. Gupta and M. K. O’Malley (2006) Design of a haptic arm exoskeleton for training and rehabilitation. IEEE/ASME Transactions on mechatronics 11 (3), pp. 280–289. Cited by: §1.
  • [17] R. Hinchet and H. Shea (2020) High force density textile electrostatic clutch. Advanced Materials Technologies 5 (4), pp. 1900895. Cited by: §1.
  • [18] R. Hinchet, V. Vechev, H. Shea, and O. Hilliges (2018) Dextres: wearable haptic feedback for grasping in vr via a thin form-factor electrostatic brake. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 901–912. Cited by: §1.
  • [19] M. Hirano, M. Sakurada, and S. Furuya (2020) Overcoming the ceiling effects of experts’ motor expertise through active haptic training. Science Advances 6 (47), pp. eabd2558. Cited by: §1.
  • [20] M. J. Johnson (2006) Recent trends in robot-assisted therapy environments to improve real-life functional performance after stroke. Journal of NeuroEngineering and Rehabilitation 3 (1), pp. 29. Cited by: §1.
  • [21] V. Klamroth-Marganska, J. Blanco, K. Campen, A. Curt, V. Dietz, T. Ettlin, M. Felder, B. Fellinghauer, M. Guidali, A. Kollmar, et al. (2014) Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial. The Lancet Neurology 13 (2), pp. 159–166. Cited by: §1.
  • [22] N. Koenig and A. Howard (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), Vol. 3, pp. 2149–2154. Cited by: §4.
  • [23] J. Lee and S. Choi (2010) Effects of haptic guidance and disturbance on motor learning: potential advantage of haptic disturbance. In 2010 IEEE Haptics Symposium, pp. 335–342. Cited by: §1.
  • [24] J. Lee, D. Kim, H. Sul, and S. H. Ko (2020) Thermo-haptic materials and devices for wearable virtual and augmented reality. Advanced Functional Materials, pp. 2007376. Cited by: §1.
  • [25] J. Lieberman and C. Breazeal (2007) TIKL: development of a wearable vibrotactile feedback suit for improved human motor learning. IEEE Transactions on Robotics 23 (5), pp. 919–926. Cited by: §1.
  • [26] S. Liu, K. Ma, B. Yang, H. Li, and X. Tao (2020) Textile electronics for vr/ar applications. Advanced Functional Materials, pp. 2007254. Cited by: §1.
  • [27] J. H. Low, W. W. Lee, P. M. Khin, N. V. Thakor, S. L. Kukreja, H. L. Ren, and C. H. Yeow (2017) Hybrid tele-manipulation system using a sensorized 3-d-printed soft robotic gripper and a soft fabric-based haptic glove. IEEE Robotics and Automation Letters 2 (2), pp. 880–887. Cited by: §1.
  • [28] M. Macchini, T. Havy, A. Weber, F. Schiano, and D. Floreano (2020) Hand-worn haptic interface for drone teleoperation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 10212–10218. Cited by: §3.
  • [29] L. Marchal-Crespo, J. Lopez-Oloriz, L. Jaeger, and R. Riener (2014) Optimizing learning of a locomotor task: amplifying errors as needed. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5304–5307. Cited by: §1.
  • [30] L. Marchal-Crespo, S. McHughen, S. C. Cramer, and D. J. Reinkensmeyer (2010) The effect of haptic guidance, aging, and initial skill level on motor learning of a steering task. Experimental brain research 201 (2), pp. 209–220. Cited by: §1, §1.
  • [31] L. Marchal-Crespo, L. Michels, L. Jaeger, J. López-Olóriz, and R. Riener (2017) Effect of error augmentation on brain activation and motor learning of a complex locomotor task. Frontiers in neuroscience 11, pp. 526. Cited by: §1.
  • [32] L. Marchal-Crespo, N. Rappo, and R. Riener (2017) The effectiveness of robotic training depends on motor task characteristics. Experimental brain research 235 (12), pp. 3799–3816. Cited by: §1.
  • [33] L. Marchal-Crespo and D. J. Reinkensmeyer (2009) Review of control strategies for robotic movement training after neurologic injury. Journal of neuroengineering and rehabilitation 6 (1), pp. 1–15. Cited by: §1, §1.
  • [34] L. Marchal-Crespo, M. van Raai, G. Rauter, P. Wolf, and R. Riener (2013) The effect of haptic guidance and visual feedback on learning a complex tennis task. Experimental brain research 231 (3), pp. 277–291. Cited by: §1.
  • [35] M. Milot, L. Marchal-Crespo, C. S. Green, S. C. Cramer, and D. J. Reinkensmeyer (2010) Comparison of error-amplification and haptic-guidance training techniques for learning of a timing-based motor task by healthy individuals. Experimental brain research 201 (2), pp. 119–131. Cited by: §1.
  • [36] M. Mulder, D. A. Abbink, and E. R. Boer (2012) Sharing control with haptics: seamless driver support from manual to automatic control. Human factors 54 (5), pp. 786–798. Cited by: §1.
  • [37] T. Nef, M. Mihelj, and R. Riener (2007) ARMin: a robot for patient-cooperative arm therapy. Medical & biological engineering & computing 45 (9), pp. 887–900. Cited by: §1.
  • [38] A. Otto, N. Agatz, J. Campbell, B. Golden, and E. Pesch (2018) Optimization approaches for civil applications of unmanned aerial vehicles (uavs) or aerial drones: a survey. Networks 72 (4), pp. 411–458. Cited by: §2.2.
  • [39] C. Pacchierotti, S. Sinclair, M. Solazzi, A. Frisoli, V. Hayward, and D. Prattichizzo (2017) Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE transactions on haptics 10 (4), pp. 580–600. Cited by: §1.
  • [40] F. A. Panizzolo, I. Galiana, A. T. Asbeck, C. Siviy, K. Schmidt, K. G. Holt, and C. J. Walsh (2016) A biologically-inspired multi-joint soft exosuit that can reduce the energy cost of loaded walking. Journal of neuroengineering and rehabilitation 13 (1), pp. 1–14. Cited by: §1.
  • [41] W. Park, E. Shin, Y. Yoo, S. Choi, and S. Kim (2019) Soft haptic actuator based on knitted pvc gel fabric. IEEE Transactions on Industrial Electronics 67 (1), pp. 677–685. Cited by: §1.
  • [42] P. Polygerinos, Z. Wang, K. C. Galloway, R. J. Wood, and C. J. Walsh (2015) Soft robotic glove for combined assistance and at-home rehabilitation. Robotics and Autonomous Systems 73, pp. 135–143. Cited by: §1.
  • [43] D. Powell and M. K. O’Malley (2012) The task-dependent efficacy of shared-control haptic guidance paradigms. IEEE transactions on haptics 5 (3), pp. 208–219. Cited by: §1.
  • [44] V. Ramachandran, J. Shintake, and D. Floreano (2019) All-fabric wearable electroadhesive clutch. Advanced Materials Technologies 4 (2), pp. 1800313. Cited by: §1, §2.1.
  • [45] G. Rauter, N. Gerig, R. Sigrist, R. Riener, and P. Wolf (2019) When a robot teaches humans: automated feedback selection accelerates motor learning. Science Robotics 4 (27). Cited by: §1, §1.
  • [46] D. J. Reinkensmeyer, J. L. Emken, and S. C. Cramer (2004) Robotics, motor learning, and neurologic recovery. Annual review of biomedical engineering 6. Cited by: §1.
  • [47] C. Rognon, V. Ramachandran, A. R. Wu, A. J. Ijspeert, and D. Floreano (2019) Haptic feedback perception and learning with cable-driven guidance in exosuit teleoperation of a simulated drone. IEEE Transactions on Haptics 12 (3), pp. 375–385. Cited by: §1.
  • [48] C. Rognon, A. R. Wu, S. Mintchev, A. Ijspeert, and D. Floreano (2018) Haptic guidance with a soft exoskeleton reduces error in drone teleoperation. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pp. 404–415. Cited by: §1.
  • [49] F. Schilling, J. Lecoeur, F. Schiano, and D. Floreano (2019) Learning vision-based flight in drone swarms by imitation. IEEE Robotics and Automation Letters 4 (4), pp. 4523–4530. Cited by: §4.
  • [50] R. Sigrist, G. Rauter, R. Riener, and P. Wolf (2013) Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review 20 (1), pp. 21–53. Cited by: §1.
  • [51] A. Tonazzini, J. Shintake, C. Rognon, V. Ramachandran, S. Mintchev, and D. Floreano (2018) Variable stiffness strip with strain sensing for wearable robotics. In 2018 IEEE International Conference on Soft Robotics (RoboSoft), pp. 485–490. Cited by: §2.1.
  • [52] J. M. Walker and A. M. Okamura (2020)

    Continuous closed-loop 4-degree-of-freedom holdable haptic guidance

    IEEE Robotics and Automation Letters 5 (4), pp. 6853–6860. Cited by: §3.
  • [53] M. Wehner, B. Quinlivan, P. M. Aubin, E. Martinez-Villalpando, M. Baumann, L. Stirling, K. Holt, R. Wood, and C. Walsh (2013) A lightweight soft exosuit for gait assistance. In 2013 IEEE international conference on robotics and automation, pp. 3362–3369. Cited by: §1.
  • [54] C. K. Williams, L. Tremblay, and H. Carnahan (2016) It pays to go off-track: practicing with error-augmenting haptic feedback facilitates learning of a curve-tracing task. Frontiers in psychology 7, pp. 2010. Cited by: §1.
  • [55] W. Wu and H. Culbertson (2019) Wearable haptic pneumatic device for creating the illusion of lateral motion on the arm. In 2019 IEEE World Haptics Conference (WHC), pp. 193–198. Cited by: §1.
  • [56] G. Wulf, S. Chiviacowsky, E. Schiller, and L. T. G. Ávila (2010) Frequent external focus feedback enhances motor learning. Frontiers in psychology 1, pp. 190. Cited by: §1.
  • [57] J. Yin, R. Hinchet, H. Shea, and C. Majidi (2020) Wearable soft technologies for haptic sensing and feedback. Advanced Functional Materials, pp. 2007428. Cited by: §1, §1.
  • [58] M. Zhu, A. H. Memar, A. Gupta, M. Samad, P. Agarwal, Y. Visell, S. J. Keller, and N. Colonnese (2020) Pneusleeve: in-fabric multimodal actuation and sensing in a soft, compact, and expressive haptic sleeve. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12. Cited by: §1.