Kinesthetic feedback devices are well studied in human computer interaction. The ability to exert forces against the user has important applications in areas such as tele-operation, rehabilitation and surgical simulation. With the advent of consumer Virtual Reality (VR), there is a resurgence of interest in replacing the ‘missing’ sensations of haptic feedback from virtual worlds. The generation of force feedback is challenging due the conflicting requirements of the robots. Devices must be large and stiff to render forces with high fidelity across large working volumes. However, large devices made of stiff materials are typically expensive, difficult to back-drive, and slower to respond. In VR it is desirable to generate force feedback wherever the user is currently operating, but large and stiff devices typically have to be grounded due to their mass.
We propose the concept of docking haptics - a class of hybrid system that enables the dynamic re-configuration of different types of haptic devices to extend their workspace and capabilities. We suggest that by combining wearables or hand-helds with grounded devices we can extend the system capabilities beyond those of the individual devices and provide plausible feedback in a broader range of immersive simulations.
For example, a problem with worn devices such as hand exoskeletons is that while they can provide force feedback, it is only referenced to a nearby point on the body. Users can feel forces between their fingers & wrist or palm and so sense the tension of an object’s surface, but not the strain in their muscles or tendons in the arm, torso or legs, leaving them unable to get a sensation of weight.
A grounded device can simulate weight easily, but its range is limited by its mechanical configuration. We suggest that in this situation, the user could be wearing a hand exoskeleton continuously, and this exoskeleton would dock with a grounded device only when necessary to generate ground-referenced force. Figure 1 shows an example of such a situation. Even with a single grounded device, this would alleviate the need for the user to remain with the working volume of the robot. With multiple robots - either two hand exoskeletons or multiple active or passive grounded devices - the potential design space for novel hybrid devices is very large.
This paper presents the following contributions:
Introduction and initial exploration of the docking haptics concept.
Presentation of a prototype combining a hand exoskeleton and six Degree of Freedom (DOF) grounded force-feedback device.
A user study demonstrating the success of the prototype in conveying weight sensations to the hand.
2 Related Work
Haptic devices can provide both cutaneous and kinesthetic sensations [burdea_force_1996, culbertson_haptics:_2018]. In this paper we will focus on the latter, specifically force-feedback devices, but will discuss the generalization to broader haptic feedback in Section 6. There are many ways of categorizing force feedback devices. A common first distinction is between grounded devices and worn devices. Grounded devices can apply significant forces to the user because they transmit the force through an active or passive device to the ground and so are limited only by the device compliance. In contrast a worn device can transmit forces from one part of the body to another. An example is a hand exoskeleton that can transmit force between fingers and wrist, and thus generate the sensation of grasp restriction.
Another categorization is active versus passive. Active devices change their shape, either to produce force directly (e.g. through a stylus) or to provide different surfaces to encounter. Passive devices do not have to be rigid, but are purely reactive, whether held in the hand or encountered.
A final categorization that we will use is encountered versus encumbering haptics. With the former, the user does not hold or wear a device or object; they reach out to encounter it. In an encumbered system, the user wears or holds devices or objects. The docking haptic concept can utilize devices across all three of these categorizations.
2.1 Worn and Held Devices
The use of hand-mounted devices to provide force feedback in VR has a long history. The Rutgers Master II was used in VR in the mid 1990s [langrana_integration_1995]. The commercial CyberGrasp system has been available since the early 2000s [cyberglove_systems_llc_cybergrasp_nodate]. The complexity of articulation of the hand has led to a wide variety of devices and actuation methods. The Rutgers Master II is typical of palm-mounted devices that restrict finger closure with pressure applied on the front of the finger; the CyberGrasp is typical of exoskeletons with actuators and levers mounted dorsally. The application of similar devices in rehabilitation has engendered extensive research on building such devices (see recent reviews [heo_current_2012, bos_structured_2016]). There has been a resurgence of interest in such devices within Human Computer Interaction (HCI) using new actuation techniques. Recent examples include the RML glove [ma_rml_2015], Wolverine [choi_wolverine:_2016] and Dexmo [Gu2016]. The last of these became a commercial product, and is the glove used in our prototype described in Section 4. Other types of exoskeleton can be used in VR. A full arm exoskeleton was developed by Bergamasco et al. [bergamasco_arm_1994]. A recent review of these devices can be found in Gopura et al. [gopura_developments_2016]. Our concept could extend to haptics that dock at other points on exoskeletons as discussed in Section 6.4.
An alternative to hand mounting is to have the user carry a device that changes shape or applies other effects. There is a strong relation to the general area of shape-changing devices (see later in Section 2.3), however we focus on devices that are not meant to be seen by the user and so are suitable for use in VR. The Shifty device uses a moving weight to change the perception of the shape being held [zenner_shifty:_2017]. Grabity, generates kinaesthetic cues through grasp restriction and asymmetric skin deformation [choi_grabity:_2017]. NormalTouch and TextureTouch are both hand-held devices that simulate contact normal and coarse texture, respectively [benko_normaltouch_2016]. The Haptic Revolver [whitmire_haptic_2018] generates shear, texture and shape cues through a wheel that can revolve under the user’s finger. Each of these could in principle be docked with other devices as we describe later. An alternative to our approach is presented by Heo et al. [heo_thors_2018]. They generate apparently ground-referenced forces in a non ground-fixed manner using large fans. The device has an effectively unrestricted working range but the forces generated are limited.
2.2 Grounded Devices
The archetypal grounded device is the Phantom [massie_phantom_1994] and its variants: the user holds a stylus and forces can be transmitted from the base. However the active range of this, and similar devices, is limited by the mechanical linkage. The Phantom is small and portable, but with limited range and stiffness. Similar systems vary in their trade-offs. The Hapticmaster [lammertse_hapticmaster_2002] is stiffer and exerts very high forces (), while the Haption Virtuose 6D [haption_virtuose_nodate] is equally stiff, not as strong, but has more DOFs.
Pairs of grounded devices can be extended to support two hands [talvas_survey_2014]. They can also be extended to support force feedback on the fingers [galiana_multi-finger_2015]. For example, the Hiro III robot comprises a robot arm with a five-fingered hand attached [endo_five-fingered_2009]. Each finger is actuated. The user puts the tips of their fingers in small thimbles close to the tips of the robot hand’s fingers. An alternative approach can be found with the previously mentioned CyberGrasp hand exoskeleton. Attached to the CyberForce controller [cyberglove_systems_llc_cyberforce_nodate], it becomes a grounded device. Two such devices were used together with a HMD to form a seated haptic workstation [ott_two-handed_2010]. This configuration of grounded device combined with a hand-exoskeleton is similar to one configuration of our hybrid system: when the parts are docked together, our prototype described in Section 4 has similar user affordances.
The main limitation of grounded devices for general use in VR is their working volume. Several systems have attempted to build larger workspaces by increasing the reach or number of articulations of the robot [ueberle_vishard10_2004, gosselin_large_2008, gosselin_widening_2007]. The Haption Inca system [haption_inca_nodate] is a novel type of cable-actuated large-scale workspace, based on the Spidar concept [sato_spidar_2002]. This mechanism could be implemented at a variety of scales. Alternatively a high precision controller can be connected to a wider area moving platform as in the Haption Scale 1 system [haption_scale1_nodate]. Finally, the high precision controller could be connected to a mobile platform [peer_new_2008, pavlik_interacting_2013]. Our concept of docking haptics extends such systems to be more general: if a mobile platform is available, it might be recruited into an ensemble system.
2.3 Encountered Devices
The devices covered so far require the user to hold or wear a device. Encountered devices allow the user to reach out and touch surfaces with their bare hands. Encountered type devices may be passive or active. In a passive system, the real world is represented within VR so that virtual objects may have some solidity [insko_passive_2001]. At the very simplest, an object that the user can pick up and hold can be considered a type of passive or encountered device [hinckley_passive_1994]. Otherwise, the passive object might be furniture or walls that the user might engage with in various ways. There has also been a lot of interest in active encountered displays that can change shape or be reconfigured [mcneely_robotic_1993]. Within the HCI community there has been enormous interest in the past few years in shape-changing interfaces (see reviews [coelho_shape-changing_2011, sturdee_analysis_2018]). We propose that both active and passive encountered devices might be recruited into docking systems.
3 Docking Haptics Concept
The docking haptics concept proposes that by dynamically docking haptic devices we can explore a much larger design space for haptic feedback. By docking, we mean one device attaching to another with a temporary joint. The joint may be rigid, or have some DOFs.
While some types of multi-part shape-changing robot fall under this definition (e.g. [zhao_robotic_2017]) we are interested in exploring the space by combining the best capabilities of different classes of device.
A first example, which we have prototyped as described in Section 4, is the combination of a hand exoskeleton and grounded mechanical arm (see Figure 1 for an image of the prototype). As illustrated in Figure 2(a)-(b), as the user moves their hand into the target area of the grounded device, the grounded device can intercept and attach to the hand. The two robots then act as one device that could be modelled by a single kinematic chain. Note that in Figure 2, we use the equipment in our implementation to illustrate the concept, but other configurations and equipment are possible.
When docked, the combined robot has some combination of the degrees of freedom of the two underlying robots. Table 1 shows some examples. Connecting a device to a grounded robot explicitly constrains it to that robot, and the type of constraint will depend on the joint (see Section 3.2). Please note that the description of the Dexmo glove is somewhat simplified as it has different articulations for each powered joint, see [Gu2016].
|Device||Type||Translation Volume||Rotational Volume||Translation Force||Rotational Force|
|Device B||Hand-worn||6 of||-||6 of|
|Figure 2a/2b||A with B||or||+ 6 of|
|Figure 2e||2 of A with B||or||+ 6 of||+ 6 of|
|Figure 2f||2 of A with B||or||+ 6 of||+ 6 of|
|Dexmo||Hand-worn||5 of||-||5 of|
|Our Prototype||Virtuose 6D with Dexmo||or||+ 6 of XX||+ 5 of|
|1 Acts as B outside range of A|
|2 Acts as B outside range of both A|
|3 Acts as Dexmo outside of range of Virtuose 6D|
|4 In the prototype we lose one degree of rotational control and constraint around the joint|
|5 Coincident with the metacarpal of each finger|
By using multiple grounded devices we expand the range of possibilities. We could provide grounded feedback at multiple locations: this could fit with the simulation of a workspace with, for example, tool selection on one bench or rack and a workpiece on another (Figure 2(c)). Thus, some haptic feedback can be provided over the entire space, with enhanced feedback in important task-related areas. Alternatively, two grounded devices could be adjacent: this could support two-handed interaction; double the force potential on a single hand; or a larger working space for one-hand by handing over between the two grounded devices (Figure 2(d)-(f)).
The robots could be of different types: the grounded device might be mobile or an encounter-style robot; the user might hold a simple hand-held device with very simple haptic feedback that could dock with passive haptics or a shape-changing device (Figure 2(g)-(i)).
3.2 Types of Docking
Docking creates a connection between two bodies. If we consider this as a mechanical problem, we would be forming kinematic pairs [hartenberg1964kinematic]. Examples of such pairs include revolute or hinged joints, prismatic joints and planar joints. However, we want to effect the joint temporarily, so we could consider mechanisms such as electromagnets, hydraulics (e.g. suction) or mechanical linkages (e.g. a grabber).
Electromagnetic coupling in an attractive option. We can create surfaces that can attach at many points by using ferromagnetic materials on one device and magnetic coils on the other. We can also design around the breaking force of the joint: we can set the magnetic strength so that the devices decouple without software intervention before enough force is exerted to endanger the user or machine. A simple attachment of a magnet to a plate, as discussed in Section 4, depends on the friction between the plate and the magnet once the electromagnet is activated. This could act as a plate joint (two translation and one rotation around plane normal) or hinged joint (one rotation around plane normal), see Figure 3(a)-(b). If we can target a small plate in a holder, we can ensure the axis of rotation (Figure 3(c)) or we can couple the devices using a profile around the joint so that they are effectively rigid under lower magnetic force (Figure 3(d)). That small plate could itself be on a joint (e.g a slider or hinge, Figure 3(e)-(f)), or other combinations.
Joint type is important to the range of forces that can be applied. We may lose DOFs, such as only being able to apply force tangential to the attachment plane, or in the case of our prototype, lose the ability to apply torque to the hand. This suggests a trade-off between joint simplicity and force transfer capability. We may need to model the joint’s behaviour so it can be properly considered as part of the kinematic chain. This suggests some implementation options such as targeting very precise docking, or allowing opportunistic docking where we measure the relative transforms of the two bodies afterwards to allow proper kinematic treatment. Further, each body could have one or more attachment surfaces, so the docking could be effected with whichever pairs of surfaces provided the best range for the resulting kinematic chain and the task required.
So far, we have considered joints that have well-defined constraints and free directions. In future work, more complex interactions such as electromagnetic force at a distance, elastic or flexible connections, etc., could be considered where the mechanism itself contributes unique forces to the haptic rendering.
4 Prototype Implementation
We constructed a proof-of-concept dockable haptic workspace. The wearable was a Dexmo glove and the arm was a Haption Virtuose 6D with an electromagnet-based docking mechanism. Both units were controlled by the same Unity application. The complete apparatus is shown in Figure 1:Right.
4.1 Dexmo Glove
The Dexmo glove by Dexta Robotics (Figure 1:Left, with our modifications) is a new iteration of a passive-admittance based device presented by Gu et al. [Gu2016]. The current version has 5 force-feedback DOFs, 11 sensed DOFs, and 10 uninstrumented DOFs. Force feedback is provided by motors that pull normal to the fingertips via a link-bar arrangement. The Dexmo glove has an admittance based API. Local feedback loops with contact-drum [Springer2002] like behaviour are parameterised by normalised target rotations and unit-less spring constants for each actuator. The domain of the normalised parameters are the rotations at the extremes of finger flex and extension, measured during a calibration for each user.
As the Dexmo is under-instrumented, the forward model that drives the virtual hand must make some assumptions. Namely, that the user adopts a power-grasp pose [Cobos2010], and the Distal Interphalangeal (DIP) & Proximal Interphalangeal (PIP) are linearly proportional to the Metacarpal (MCP) [Li2007a, Hrabia2013, Cobos2008]. Finger abduction (spread) is measured directly. The virtual hand consisted of a graphical model from the Dexmo SDK and a skeleton that included joint limits. Bio-mechanical models of the hand are available [VanDerHulst2012, Buchholz1992, Cobos2008]
, but for this project we calibrated ours by eye as the skeleton itself was not physically correct. The normalised parameters from the Dexmo sensors directly interpolated joint rotations between their limits. Their limits represent the extreme hand poses that users adopt during calibration. The Dexmo is tracked using an HTC Vive tracking puck.
4.2 Haption Arm
The Virtuose 6D by Haption is a robotic arm designed for ground-fixed tool based haptics. It has 6 DOF
, able to set both tool position and orientation within its workspace. The Virtuose supports both impedance and admittance based control. We use it in admittance mode. The Virtuose SDK works with end-effector or virtual tool transforms in a Cartesian coordinate system.
The programming model for admittance mode is to set target transforms and speeds in a high-frequency () callback. These are used to compute the parameters for the robot’s control loops. The user-API communicates with a process that runs the robot over UDP, so the SDK can be used on any computer. Indeed, the arm was connected to a different machine than the HMD.
4.3 Magnetic Dock
The robot attached to the hand using an electromagnet. The Dexmo cover was replaced with a mild-steel plate. The Virtuose tool was replaced with an attachment hosting the magnet and a tracker for open-loop docking (Section 4.4). The magnet power supply was controlled with simple serial commands to a microcontroller over USB.
The magnet exerts very high forces axially, but it is easy to pivot off. For the prototype, we fixed the tool orientation and instructed users to keep their hands flat to avoid applying any torque to the attachment. Relaxing these constraints is discussed in Section 6.1.
The working volumes and force characteristics of the Dexmo and Virtuose devices and the hybrid device resulting from docking with the specific magnetic joint, are shown in the lower part of Table 1.
Tracking and interception is a well studied problem in robotics, in the forms of navigation guidance and visual servoing [Lin1989, Agah2004, Mehrandezh2000, Borg2002, Chwa2005, Kunwar2006]
. For this prototype we simply attempted to match the transform of the magnet to a target point on the hand (Pure Pursuit). Control of the Virtuose for docking and interception was performed in the arm’s base frame - the native frame of the admittance API - with Unity’s coordinate system. The Virtuose API takes a target transform (displacement and orientation) of the end-effector pivot. To compute this, we transform the target by the inverse of the magnet’s local transform. This gives the true effector target. The Virtuose maintains an estimate of its current position, however the accuracy we found was too low for reliable interception. Instead we use another Vive puck to track the effector and compute corrections, as per Equation1, where and are respectively the target and current effector transforms to and from the API. World space transforms are determined by the external trackers, which operate in a coordinate system referenced to the real world by the SteamVR calibration performed outside the application. Refer to Figure 4.
4.5 Virtual Environment
The Virtual Environment (VE) was constructed in Unity 2018 and used the inbuilt PhysX engine to perform the simulation. The Dexmo glove provides a managed library that was integrated directly with Unity. The Virtuose C library was thinly wrapped with P/Invoke. The Dexmo should not be updated at more than while the Virtuose must be updated at no less than . The Dexmo and magnet controller ran in the main Unity thread, while the Virtuose SDK ran its own thread to issue callbacks, and in these callbacks we implemented force control and tracking & interception, based on parameters set from the main thread.
4.6 Physics Simulation
PhysX is an impulse based rigid-body simulation. The graphical hand model was provided with kinematic colliders. That is, PhysX would apply impulses to dynamic objects to resolve collisions with the visual geometry of the hand. We drove the simulation using rigid body colliders as opposed to force-reflection in order to allow users to grasp items. The power-grasp assumption and limited instrumentation of the glove make it very difficult for users to correctly balance point-forces on rigid bodies for stable grasping, or facilitate force-closure. With colliders, the simulation incorporates additional forces that cannot be actuated by the glove, reducing transparency but increasing intuitiveness and controlability. Dynamic objects are left to the physics engine to simulate. We observe the impulses that drive their behaviour and use them to control the haptic devices.
4.7 Robot Control
The Dexmo glove has an admittance-based API. The force feedback parameter is an angular position in the same space as the sensor parameters - effectively an interpolant between the extremes of flex and extension. Once set the glove will prevent the user exceeding the specified rotation/finger flex. To compute this, we detect collisions between the distal phalange colliders and world geometry. We let the physics engine resolve the collision by moving the world, but before this happens we compute a hypothetical de-penetration transform that moves the phalange colliders instead. From these hypothetical positions, we determine numerically their extent between the extreme poses, and thus the ‘contact-drum’ parameter. The stiffness of the surface was fixed.
We use the Virtuose in admittance mode, though for force-feedback we implement an impedance based controller on top of this, as it was most natural to work with forces at this stage. The controller computes an appropriate displacement each frame based on Hooke’s Law, the spring-constant being defined by the stiffness of the robot’s control loop.
The forces on the hand are rendered by the local loop on the glove itself. The forces for the arm are computed from the impulses applied by the hand-colliders. These are transformed into forces and summed. The result is transformed into the arm’s local space and applied through the effector. This is based on the observation that when a user is squeezing an object, equal and opposite forces will be applied to opposing hand colliders. When the object is supported externally, world referenced forces will be transferred through world geometry, and hand-referenced forces will cancel. If the hand is supporting an object against gravity, inertia or other geometry, world referenced forces will be transferred through a subset of hand colliders. The net forces in this case are non-zero. The palm-fixed actuators cannot drive these, so they must be applied through the arm.
5 Proof of Concept Experiment
To prove the docking haptics concept we performed a within-subjects study. 18 people from within the organisation performed sorting tasks in VR under the three conditions in Table 2.
|Dexmo Haptic Feedback||✓||✓||✓|
|Robot Force Rendering||✓|
5.1 Apparatus & World
The robotic apparatus is the system described previously. The VR headset was an HTC Vive. An example of participants’ view of the world is shown in Figure 5. Participants could pass their virtual hand through the translucent worktop, but the other items were physically simulated and provided haptic feedback. The three cans had their rotations constrained. Other items did not have a role in the task, but could be manipulated by the participant during the induction stage as part of their explorations.
5.2 Task & Procedure
Utility was evaluated via an objective sorting task in which participants were presented with three visually identical cans and asked to arrange them in order of increasing weight using “whatever cues they could perceive” to distinguish between them.
Participants were inducted with verbal and written instructions. After giving verbal consent, they were assisted in putting on the Dexmo Glove and a calibration procedure was performed outside of VR. The participants then put on the headset and positioned themselves in front of the virtual desk by moving in the real world. They were then given an unlimited period in which to interact with world (conditions identical to Free) in order to become comfortable in the virtual space and with the hand’s interaction with it.
Once the participants were ready, the trials were completed in immediate succession. The experimenter informed them which condition was next, reset the can positions, and where necessary docked the arm. Participants had an unlimited amount of time to arrange the cans, after which the procedure repeated for the remaining conditions. After all conditions, participants exited VR and completed a questionnaire.
The condition order was randomized per-participant. Can weights were always , and . The order was randomized per trial. These masses were tuned to be perceptibly different while not generating destabilisingly large impulses. The masses are nominally in , but the robots’ actuations were not physically accurate.
To evaluate the quality of experience, participants were given a questionnaire. Participants were given the same five questions about each condition and told to judge the experience against the real world. Responses were on a five-point (negative (1) to positive (5)) linear scale.
How easy was it to pick up an object?
How well were you able to explore using touch?
How easy was it to tell the difference between the weights?
How well could you disambiguate surface texture?
How well were you able to move as you desired?
Question 4 was a control question as the system did not render or simulate the dynamics of surface texture.
A one-way ANOVA on the number of correctly ordered pairs per trial () showed that user performance was significantly better () in the Forcefeeback condition than in the other two, statistically indistinguishable, conditions (). The condition had no significant effect on completion time.
As can be seen, users find the docking mechanism inhibits their range of motion but does not significantly affect their ability to pick up objects, though in general they did not find it easy to grasp objects. As expected users gained no benefit from docking itself, and only in the Forcefeedback condition were they able to distinguish between the weights, as indicated by both questionnaire results and task performance. The responses in the Forcefeedback were more varied than in the other conditions. This is possibly due to varying individual tolerances to the force jitter caused by the suboptimal physics simulation.
6 Discussion and Future Work
6.1 Mount Design
Our mount was successful in that it supported interception, docking and the conveyance of weight. Users felt that it limited their ability to explore however. We suspect this is due to the artificial rotation constraints, imposed because the attachment cannot transfer torque. The magnet exerts axially, but requires little effort to pivot or peel from the surface. One solution would be for the robot to match the orientation of the hand, actively compensating for user applied torque. This requires more precise control of the robot than we demonstrate, and more importantly, such an approach would not be able to apply torque.
Another solution is a mechanical linkage such as a universal joint. The magnet could remain attached with optimal alignment without active compensation. The control system would have to model the behaviour of the joint to ensure forces were always applied normal to the attachment however, and the arm would still not be able to apply torque.
A mount design could control different DOFs with different techniques. For example, the magnet could enter a tube rather than dock anywhere on a plate. The tube would mechanically limit off-normal rotations, but axial forces remain entirely magnetic. This design is mechanically simple and highly rigid, but would require very accurate interception by the grounded robot.
6.2 Robot Capabilities
Where rigid docks are used, robot range-of-motion will become perhaps the most important characteristic. Active compensation is necessary for any practical system, as the Dexmo glove and robot together are heavy. Active compensation may require more precise control, but it could also be performed with force sensors only, which the Virtuose supports.
The Virtuose had errors in its forward kinematics and as such required an external tracker to intercept the hand. These error sources present challenges but not necessarily significant ones. Works on robot guidance often express rendezvous algorithms in terms of relative velocities (e.g. [Mehrandezh2000, Borg2002, Chwa2005, Kunwar2006]). With external tracking, all that is required for these is to transform velocity commands so that resulting accelerations are correct in the global frame - for example, with a change-of-basis recomputed each tracking frame. Such an approach is far simpler than attempting to model the error sources across the entire workspace. Since external tracking is required for the base location and wearable location, there is little disadvantage to tracking the tool.
Future systems will have to explore more complex algorithms than we demonstrate, to support faster hand motion, robot grasping and avoidance of obstacles, most notably the hand itself.
6.3 Physics Simulation
Future implementations will need a better integration in order to render physically correct forces. Driving the simulation with colliders instead of force-reflection resulted in an intuitive and stable simulation even at low (for haptics) frame rates. Game-optimised engines may have difficulty with complex arrangements of colliders like those of the hand, resulting in distracting jitter and extreme responses. More stable simulation approaches such as Position Based methods [Bender2012], or advanced simulation engines such as MuJoCo [todorov_mujoco:_2012], could provide a better experience.
6.4 Other Configurations
Our prototype demonstrates just one possible docking configuration. Section 3 suggested other approaches. One aspect to explore is the type and direction of docking. Even with magnetic docking, we had the option of mounting the magnet on the hand exoskeleton or the grounded robot; swapping this around may provide more opportunities. For example, one obvious configuration is an encounter-style robot that instead of only rigid objects (e.g. [araujo_snake_2016]), holds plates or surfaces that can be docked with. This suggests that the hand-device, be it a held controller, or a hand exoskeleton, could have multiple magnets to connect.
Another area to explore is the combination of docking with haptic retargeting [azmandian_haptic_2016]. Given that joints may be stronger in some directions than others, and that users seem relatively insensitive to small relative position offsets, we can perhaps construct docking configurations that appear to generate forces from a wider range of directions and strengths than is physically realizable.
In docking haptics, a system dynamically couples different components at run-time to build the most appropriate combination for a particular task and environment. As the user transitions throughout the virtual world the capabilities of the hybrid system change.
We introduced docking haptics and a number of proposals on how it may work with current device capabilities. We prototyped an example system from a Dexmo exoskeleton glove and Haption Virtuose arm, to explore a hybrid grounded/un-grounded system. Using pure-pursuit, the arm was able to intercept the glove and exert forces in cooperation with the glove to render weight. Using a straightforward net force calculation on different collider sets we could control the system with minimal knowledge of the individual device dynamics. In a user study we validated the concept by demonstrating that even with its limitations, our docking haptics prototype conveys weight with sufficient fidelity to significantly improve performance in an object sorting task.
The design space of docking haptics systems is very large and requires considerable work in various areas, but by designing with docking haptics in mind, alternative trade offs could be made leading to devices that better inter-operate for more flexible and general purpose feedback in VR.