The "Kinesthetic HMD": Enhancing Self-Motion Sensations in VR with Head-Based Force Feedback

08/23/2021 ∙ by Antoine Costes, et al. ∙ 0

The sensation of self-motion is essential in many virtual reality applications, from entertainment to training, such as flying and driving simulators. If the common approach used in amusement parks is to actuate the seats with cumbersome systems, multisensory integration can also be leveraged to get rich effects from lightweight solutions. In this short paper, we introduce a novel approach called the "Kinesthetic HMD": actuating a head-mounted display with force feedback in order to provide sensations of self-motion. We discuss its design considerations and demonstrate an augmented flight simulator use case with a proof-of-concept prototype. We conducted a user study assessing our approach's ability to enhance self-motion sensations. Taken together, our results show that our Kinesthetic HMD provides significantly stronger and more egocentric sensations than a visual-only self-motion experience. Thus, by providing congruent vestibular and proprioceptive cues related to balance and self-motion, the Kinesthetic HMD represents a promising approach for a variety of virtual reality applications in which motion sensations are prominent.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 4

page 5

page 6

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

With the blossoming of virtual reality (VR) consumer-grade devices in the last five years, numerous applications for VR bloomed in the entertainment, healthcare and manufacturing industries [14]. Impressive advances were achieved in visual and audio quality, with audio spatialization being now common place and human-eye resolution about the be reached [1]. However, although VR experiences are often praised as ”fully immersive”, bodily sensations are still largely missing, while they are crucial for several aspects of user experience [22].

The perception of self-motion, in particular, can be induced visually but relies on multisensory cues, and notably on vestibular cues, i.e. acceleration and motion applied to the head [5]. Although visual cues are usually more precise for perceiving displacement, in some cases they can be dominated by vestibular cues [13] [10].

This perceptual discrepancy leads to cybersickness effects as well as user disinvestment (in particular for whole-body displacement such as vehicle driving, flying, falling, etc), both of which are major issues in many VR applications. As a result, VR content producers tend to adapt to those technological limitations instead of fitting the best possible user experience. The most stunning example is probably the use of teleportation as a relatively standard locomotion system, despite huge design efforts for other metaphors with better realism and ecological agency

[3]. In order to address this lack of physical sensations, over the last years both researchers and industrialists proposed a large number of haptic VR peripherals (see [30] for a review).

Yet only a subset of them addressed motion sensations. If the common approach used in amusement parks is to actuate the seats with cumbersome systems, multisensory integration can also be leveraged to get rich effects from lightweight solutions [8][18].

Fig. 1: The Kinesthetic HMD: head-based force feedback enhancing self-motion sensations.

In this short paper, we propose a novel approach called the ”Kinesthetic HMD” to enhance self-motion sensations in immersive VR applications. It consists in actuating a head-mounted display (HMD) with force feedback in order to provide rich and versatile self-motion effects. Similarly to a motion platform, our approach provides motion cues to the user, but at the level of his/her head, so to simulate whole-body accelerations and enhance illusory self-motion. In contrast with actuated seats, the Kinesthetic HMD stimulates the vestibular system rather than the whole body.

In the remainder of this paper, we first present related work on VR peripherals providing motion sensations. Then, the Kinesthetic HMD approach is introduced and its hardware, software and safety design is discussed. We demonstrate a flight simulator use case with a proof-of-concept prototype featuring a high-end grounded 6-DoF (degrees of freedom) haptic arm. Then we present a user study focusing on the haptic enhancement of visual self-motion sensations, before finally discussing our experimental results and possible future works.

2 Related work

Although “vection” (illusory self-motion) was historically considered as a visual illusion, it can be defined more broadly as a subjective conscious experience of self-motion” [20]. Its correlations with presence and immersion are studied, yet their relationship remain unclear [21] [23][28]. It was found that exposure to vection in VR modulates vestibular processing [11].

The vestibular system provides information about the angular rotation speed and linear acceleration of the head in space, which are crucial for self-motion estimation

[7]. The relationship between visual and vestibular contributions to motion sensations is still not fully understood [5], and they seem to be dynamically reweighted [10]. Yet it can be said that the visual system is specialized for position and velocity estimation, while the vestibular system is optimized for acceleration processing.

This explains why the visual motion provided by a HMD, despite its ability to evoke illusory displacement, remains ”incomplete” and not sufficient to provide compelling inertial sensations. In order to get physical sensations, there is a need for physical stimulation. Haptic devices provide kinesthetic and/or tactile stimulation. The vast majority of haptic solutions for VR are either wearable, holdable or grounded, and stimulate the finger or the hand in order to reproduce contact mechanics, material properties and physics of digital interaction (see [30] for a review). Much less addressed self-motion sensations.

One approach to induce vection is to stimulate a large part of the skin, usually by integrating vibrators in a chair [26]. Yet the most common technological solution is to provide actual motion to the seat: moving seats are usually found in amusement arcades for racing games and in theme parks for so-called 4D cinemas [2]. Those systems being usually cumbersome and expensive, and yet limited by a restricted amplitude, several researchers proposed simplified versions relying on illusions or sensory substitution [25][27]. In particular, Danieau et al. proposed to affix multiple force-feedback devices to a chair, stimulating specific parts of the body (hands and head), in order to provide inexpensive 6-DoF motion effects [8].

Applying haptic stimulation on specific regions of the body (feet, hand or head) is indeed a promising approach to achieve rich effects with a minimalist setup. It was shown that a haptic stimulation of the feet could induce self-motion sensations while standing still in various virtual reality scenes [18], and modulates vection even in a seated position [9][16]. Lécuyer et al. investigated the perception improvement of visual turns by reproducing the turn angle (or its opposite) through a haptic handle [17]. Ouarti et al. used a similar setup to show that a haptic feedback in the hands could improve duration and occurrence of visually-induced illusory self-motion for linear and curved trajectories [19]. Bouyer et al. extended this approach to an interactive video game context [4].

HMD-embedded haptics is a quite recent research topic. Peng et al. showed that step-synchronized vibrotactile stimuli on the head could significantly reduce cybersickness [36], while Wolf et al. proposed to combine vibrotactile and thermal feedback inside the HMD for an increased presence [37]. Gugenheimer et al. attached flywheels to an Oculus Rift DK2 to generate torque feedback on the head [12]. A major drawback of this solution is the lack of transparency, as when the flywheel turns faster it builds up inertia against user’s movements. Kon et al. suggested to leverage the intriguing so-called ”hanger reflex” effect, which is an involuntary head rotation arising from a specific pressure distribution, in order to provide various illusory forces [15]. However this technique does not allow precise force rendering as it relies on muscular reflex that is likely to vary a lot among individuals. Chang et al. proposed a pulley-based mechanism to produce normal force on the HMD [6]. This system could improve immersion in boxing or swimming simulations, but did not aim at generating self-motion sensations. Finally, Wang et al. integrated skin-stretch modules inside a HTC Vive Pro to provide haptic feedback on the user’s face [29]. In a motorcycle racing simulation context, their system would simulate weight, bumping transients, inertial turns, and wind pressure. However, to our knowledge, grounded force feedback was never applied to a VR headset.

3 The Kinesthetic HMD approach

3.1 Concept

The Kinesthetic HMD is a novel approach to enhance self-motion sensations in virtual reality with head-based force feedback. It can be viewed as an augmentation of the HMD, adding precise forces and displacement to images and sound provided to the head. The generated vestibular and proprioceptive cues emphase the visual motion to produce stronger and more compelling sensations of begin accelerated, although the user, standing or seated, remains in the same place.

The force feedback should be congruent with visual motion, for instance being proportional to ego-acceleration. Yet the rendering algorithm can be adapted to the locomotion context. When the user walks virtually, the haptic device can simulate head movements in compliance with the walking pace. When the user gets in virtual car and drives, the force feedback can push or pull to simulate acceleration and braking.

The Kinesthetic HMD approach requires at least four components: a force-feedback device, a VR headset, a head clamping part transmitting the forces, and a software platform for VR content and haptic rendering. Such a system implies hardware, software as well as safety considerations, detailed in the reminder of this section.

3.2 Hardware considerations

Applying force feedback to the head raises a number of technical challenges, especially as producing vestibular cues requires steady directional forces, able to move the head relatively to the thorax in one to three directions. First, the actuator should benefit of adequate backing. Second, the system should be comfortable enough not to break immersion. Lastly, the forces should be correctly transmitted to the head.

Backing Most force feedback devices requires backing, either on the body (wearable) or on a fixed frame (grounded). Backing on the shoulders (or any body part other than the head) is complicated because of their mobility relatively to the head. Holdable devices, which could be totally embedded on a helmet, come with significant limitations like motion impediment and saturation (reaction wheels and weight shifting devices), or buzzing (asymmetric vibrations). Moreover their force capability is directly limited by their moving mass, and therefore their carrying weight.

Comfort The head is more sensitive than other body parts to noise, motion restriction or extra weight. Therefore mechanical actuators should ideally be properly isolated, or departed from the point of application. Also, the mechanical transmission should not present any risk of wrong movements for the neck.

Point of application The force feedback should not only be produced, but also properly transmitted, and choosing the adequate clamping system can be an issue.. The clamping part should not slip on the head, yet it should remain comfortable. Pressure should be limited and therefore area of contact maximized, but avoiding ears or any sensitive part. Even with a capable device, producing a constant directional force on the head is challenging because when the head rotates the alignment with its mass center is easily lost, resulting in unwanted counter torques.

3.3 Software considerations

Designing the proper self-motion rendering algorithms for head-based force feedbacl is not straight-forward for several reasons, detailed in this subsection. First, the haptic stimulus should act as a metaphor, as we want the user to feel an illusory whole-body motion. Second, the provided forces and displacement should ensure safety and all values are not acceptable. The consequent filtering can impede realism, especially for real-time interactive content that is not known in advance. Third, the effect of the force feedback might depend on the user’s position, which should be taken in account in the rendering loop.

Haptic rendering Several authors used velocity-based vestibular feedback to study vection [34] [35], but a comparative study suggested that acceleration is a better choice, producing stronger and more consistent illusory self-motion sensations [19]. Another advantage of acceleration over velocity is that the summed amplitude over time is lower, and thus is less prone to workspace limitation issues. Among possible haptic metaphors, two were pointed out by previous works: the direct mode and the indirect mode [17] [4]. In the direct mode, the haptic feedback is proportional to visual acceleration: this means if the virtual vehicle speeds up, the haptic device will push forwards, simulating physical acceleration. In the indirect mode, the haptic feedback is inverted: when the virtual vehicle speeds up the haptic device will pull backwards, simulating the physical body displacement.

Interactivity Just like the visual content, the haptic effects can be fully interactive (i.e. generated from user input), pre-recorded (i.e. experienced passively), or a mix of both (i.e. actively triggered or modulated). Pre-recorded content can be analyzed and scaled to mitigate extreme values while preserving realism. Fully interactive experiences might require thorough real-time filtering, depending on the content. Trade-offs can be found with hybrid strategies, for instance triggering a pre-recorded sequence only when the user reaches the adequate part of the workspace.

Motion filtering As our force feedback is expected to move the head, the user will lean (assuming they stay in the same position). The leaning amplitude will be limited either by the user’s anatomy or the haptic device’s capabilities. Just like with actuated seats, the rendering algorithms should avoid reaching the edges of the workspace, and eventually come back towards the center below perceptual thresholds, which can be achieved with so-called ”washout filters” [38]. One potential issue in the context of head-based haptic rendering is to determine which perceptual thresholds (vestibular or proprioceptive) to consider.

Fig. 2: Flight simulation use case: the force feedback is inversely proportional to visual acceleration.

3.4 Safety considerations

Respecting anatomical limits and avoiding any risk of injury is of course a major requirement for any haptic setup, and even more when applied on the head. Although defining general-case safe limits is tricky, an order of magnitude of seem to be a low value both for neck muscular force in any direction for healthy adults [32] and for neck loadings involved by common non-injurious physical activities [33]. Thus, values of one order of magnitude lower (dozens of ) might be considered for the maximal applied forces in healthy and warmed-up adults. In addition to limiting the total applied force, we suggest to limit the applied jerk (to avoid false moves), to have a hardware kill switch, and to design pre-session tests to adapt the gain to each individual. If the user is standing, the risk of unbalance and fall has also to be considered.

4 Proof of concept

In order to showcase our concept, we developed a prototype (see Figure 1 for the detailed components) based on a high-end grounded haptic arm and tuned it for a flight simulation scenario. That is, we wanted to provide the user strong and compelling sensations of acceleration, deceleration, gravitational roll and turbulence. Also, we wanted our demo to be accessible without prior skills, so we chose a glider flight scenario, which requires minimal pilot skills. The full scenario is detailed in the accompanying video.

Fig. 3: The rigid headband and the custom 3D-printed connector.

4.1 Hardware components

We used a Virtuose 6D 35-45 haptic arm, which provides strong forces and torques in a workspace of about 1.3mx5mx1m. We placed its base about 1m above the floor in order to have the user’s head in the middle of the workspace.

By using a grounded device, we don’t load the head with extra weight, and don’t expose it to motor sound and vibrations.

For the HMD we used a HTC Vive headset with an unofficial ”Rift S style” rigid headband found online. We designed and 3D-printed a mechanical connector replacing the Virtuose handle, plugged at the back of the headband and secured with a screw (see Figure 3). The use of a 6-DoF haptic arm allows for cancelling counter torques when applying steady 3-DoF forces.

4.2 Software components

We built upon a glider flight scenario of a standard software flight simulator (X-Plane 11). Flight simulation softwares output a series a real-time motion data, and notably acceleration values to be used for motion platforms. We used a python script to receive the flight simulation acceleration data through UDP and to compute the haptic rendering. The applied forces were negatively proportional to visual acceleration. In order to cancel counter torques, when force modulus exceeded the applied torques were simulating a virtual cylinder joint along the direction of the applied force. When the forces were lower than there were no applied torques in order to let the head free to move.

Another advantage of flight simulation softwares is that they allow for recording and replaying scenarios. Therefore the user can choose between the intense experience of a fully interactive flight, or the comfort of a pre-recorded flight without the stress of manoeuvring the aircraft. In both cases, the force feedback is rendered in real-time from the acceleration values output.

No ”washout filter” was implemented as applied forces usually lasted only a few seconds and were limited to , so the user would not reach the edges of the workspace during our scenario.

4.3 Safety considerations

The Virtuose arm has a hardware security switch usually controlled by a proximity sensor integrated to the handle, so that force feedback is disabled whenever the handle is not held. We replaced it with a mechanical button acting as a kill switch that could be held either by the user or the experimenter. We limited the total applied force to . The gain factor between visual acceleration and applied forces could also be refined upon user request to adjust experience intensity.

5 User study

In order to evaluate our system’s ability to enhance visually-induced self-motion sensations, we conducted a user study comparing the motion sensations evoked by visual and visuo-haptic displacement stimuli. On the basis of Ouarti et al.’s results, we chose to focus on acceleration-based haptic rendering, and also to evaluate both direct mode and indirect mode, as Ouarti et al. suggested that half the people would prefer one and half the people the other [19].

5.1 Hypotheses and objectives

In the context of virtual reality applications, we are not only interested in strengthening motion sensations, but also making them more compelling. That is, providing an illusory self-motion experience rather than a ”scrolling landscape” feeling. In other words, we want the motion to be felt as more egocentric (self-motion rather than landscape motion) as well as more intense (with vivid bodily sensations). Comfort should also be considered, as visuo-haptic discrepancy can produce immersion-breaking awkwardness if they are not congruent.

Therefore we designed several displacement stimuli and evaluated the sensations they produced as follow:

  • Relative motion: am I moving in a fixed an environment or am I at rest watching a moving environment?

  • Acceleration: do I have bodily sensations similar to being in a moving vehicle with eyes closed?

  • Comfort: how pleasant or unpleasant is the motion experience?

We designed our study to test the following hypotheses:

  • H1: visuo-haptic stimuli induce more egocentric motion sensations than a similar visual stimulus

  • H2: visuo-haptic stimuli induce stronger bodily sensations than a similar visual stimulus

  • H3: some participants would have higher ratings (in relative motion, acceleration and/or comfort) for the direct mode, and some other for the indirect mode

Fig. 4: The temporal acceleration pattern used for the stimuli. The curve shows acceleration () over time ().

5.2 Stimuli

The visual context was designed so as to be as neutral as possible. The virtual environment was an empty space filled with a random spatial distribution of 20000 white cubes, vanishing in a black fog at about 70 meters distance. In order to mitigate the lack of embodiment and eventual vertigo symptoms, a white 3mx4m rectangle was used as a symbolic ground.

The displacement stimulus was based on a double step acceleration pattern (see Figure 4): a forward acceleration for 5 seconds followed by a deceleration of 5 seconds. In order to avoid transient effects, those two steps were eased with 0.5sec long sinusoidal curves, so the acceleration magnitude was of a maximal of 5 for 4 seconds.

The visual stimulus was identical for all conditions: the cubes were accelerated according to the previously depicted pattern, creating an ambiguous relative displacement between them and the participant and ground. The three experimental conditions were as follow:

  • H_NONE: no force feedback

  • H_DIRECT: force feedback is proportional to the acceleration pattern: pushing during acceleration, pulling during deceleration

  • H_INDIRECT: the force feedback is proportional to the opposite of the acceleration pattern

In both H_DIRECT and H_INDIRECT conditions, the haptic arm maintained head orientation constant during the stimulus to avoid any lever arm effect.

5.3 Participants

17 participants (2 females, age 23-54, mean=35.3, SD=10.7) volunteered for the experiment. They were all recruited in the research center, had corrected-to-normal vision, and no balance disorder history. They all signed an informed consent form prior to participating to the experiment. In order to prevent cybersickness symptoms, participants were asked to hydrate well before participating, and were free to have a break anytime during the experiment. One extra participant could not finish the experiment because of cybersickness symptoms, thus their incomplete data was removed from the results.

5.4 Experimental design

Each participant would attend a single session of about 50mn consisting in three phases: an introduction phase, an exploration phase, and an experimental phase.

In the introduction phase, the participant read and signed an informed consent form conforming to the declaration of Helsinki. In order to clarify the vocabulary used during the experiment, the participant was orally given examples of different motion sensations related from a displacement stimulus: estimated traveled distance, relative motion (which of me or the landscape is actually moving ?), and ”acceleration”, that is the non-visual sensation of being moved that can be felt when closing your eyes in a moving vehicle.

Fig. 5: The visual stimulus used in the experiment.

Then, the participant was introduced to the protocol through the exploration phase. The participant sat on a stool and adjusted the HMD for a correct vision. The experimenter progressively explained the experimental conduct by running 8 trials (4 H_NONE, 2 H_DIRECT, 2 H_INDIRECT) and answering any question (except about the nature of the stimuli). In order to avoid surprise effects, the haptic arm was not connected to the HMD before the third trial (the first visuo-haptic stimulus). On one of the visuo-haptic trial, the participant was asked to perform a ”security break exercise”: they would say a safeword any time during the stimulus, after which the experimenter immediately released the kill switch to disable haptic feedback and cancel the trial. The participant’s answers were not recorded during the introduction phase. After the eighth trial, the participant was offered to have a break, and the experimenter made sure they had no remaining question about the protocol or the task, before moving to the experimental phase.

The experimental phase consisted in a randomized block of 30 trials (10 repetitions of each three conditions). All the trials followed the same structure. First, the participant validated the launch of the trial. Then, a visual target was displayed forward for 1.5 seconds (see Figure 5). Then the stimulus was played for 10 seconds. Finally, the participant rated the stimulus on Relative Motion, Acceleration and Comfort scales (see Figure 6).

Fig. 6: User rating during the experiment.

Participants were asked to keep their gaze towards the direction indicated by the target at the beginning of the trial. They were asked to maintain their gaze direction and simply remain comfortable if the haptic stimulus would modify their posture.

5.5 Results

Participants’ ratings for are shown on Figure 7. Relative Motion was mostly rated positive (egocentric motion) in the H_DIRECT and H_INDIRECT conditions, while answers were mixed for the H_NONE condition. Comfort ratings were mostly weakly positive for the H_DIRECT and H_NONE condition, and mixed for the H_INDIRECT condition. Acceleration was mostly rated above 3/5 in the H_DIRECT and H_INDIRECT conditions, while mostly under 3/5 in the H_NONE condition.

For each of the three variables, as the normality assumption was violated (Shapiro–Wilk’s normality test, ), a Friedman test was conducted independently followed by a post-hoc Wilcoxon-signed ranks test with a Bonferroni correction. For Relative Motion and Acceleration, significant differences () were found between H_NONE and H_DIRECT and between H_NONE and H_INDIRECT. For Comfort, a significant difference () was found between H_NONE and H_INDIRECT only. We didn’t find any effect of age, gender or previous experience of virtual reality on any of the three variables.

Fig. 7:

Ratings distributions across all participants. Red dots are outliers.

Taken together our result validate both H1 and H2, with significantly higher ratings of Relative Motion and Acceleration for visuo-haptic stimuli compared the to the visual-only stimulus. Surprisingly, the rating distributions for both H_DIRECT and H_INDIRECT were very similar, and contrary to H3 we could not split the results in two populations preferring one or the other type of haptic feedback, as suggested by previous work [19]. The only noticeable difference was the Comfort ratings, slightly more negative for the H_INDIRECT condition, leading to significant differences with the H_NONE condition, which ratings were mostly positive (almost never judged as uncomfortable).

6 General Discussion

6.1 Approach

The informal reactions to our flight simulator prototype confirmed that the visuo-haptic experience was much more compelling than the visual-only one. After having tried the haptic-enhanced scenario, the visual-only scenario felt ”empty” or ”flat”, and also more prone to dizziness. It seemed that both low-frequency and high frequency haptic feedback contributed strongly to the quality of experience: the roaring take-off would lack of power without the slow and strong pull, but would also lack of realism without the swift turbulences.

The rigid headband can be used separately from the haptic arm, but also from the HMD, which means we could use it with other visual conditions, from a simple screen to video-projected space or augmented reality glasses. A quick fastener on various haptic devices could allow for a versatile usage.

Depending on the application the displacement might be active or passive, and the user can be seated or standing. The design of head-based haptic feedback for active displacement might lead to novel locomotion techniques, improving user performance and/or quality of experience. When standing, the workspace can be limited depending on the chosen haptic device, and hardware should also be adapted to imbalance and falling issues.

The lack of bodily sensations is a major issue for virtual reality, limiting user quality of experience and agency, and increasing cybersickness occurrence. Our approach opens thrilling possibilities to apply haptic cinematography principles, as proposed by Danieau et al. [39]. Beyond the realism of physical simulations, creators can make use of non-diegetetic effects to increase dramatic intensity, and direct attention with staging or anticipation techniques. More generally, haptics could be seen as a media by itself, able to improve storytelling and make better narratives, just like cinema sound design.

6.2 User Study

Taken together the results of our study show that, as expected, the visuo-haptic stimuli induced more vivid and more compelling self-motion sensations than a similar visual-only stimulus. Those enhanced motion sensations might arise from vestibular and/or proprioceptive cues. As the participants could resist more or less to the haptic feedback to maintain their posture, their leaning amplitude could vary a lot between the stimuli. However we could not find, formally or informally, a clear difference in the accompanying sensations. On top of that there was a significant difference in amplitude between individuals depending on their weight (which varied with a factor of two between participants). Yet even weighted individuals with no visible leaning might answer with high self-motion ratings. Further research should clarify the respective contributions of vestibular and proprioceptive cues.

The vestibular haptic cues might also alter the estimations travel distance, as suggested by previous work [13]. We initially planned to include this task in our study, but the pilot tests revealed an surprisingly high cognitive load for making both distance and velocity/acceleration estimations. After providing a traveled distance estimate, the participants could hardly make assessments about relative motion or non-visual sensations and vice-versa, as if those two kinds of task had conflicting short memory processes. The perception of displacement given by visual and visuo-haptic stimuli remains to be studied.

6.3 Future Work

Head-based force feedback opens exciting possibilities in terms of haptic rendering, training simulation and virtual reality quality of experience. It also comes with various application-related challenges to address in future work.

Our prototype based on a 6-DoF haptic arm was able to provide stable forces in any of the three directions. Yet both visual and vestibular perceptual thresholds are anisotropic [31], thus directionality could be taken in account when elaborating head-based visuo-haptic stimuli, and the psychophysical thresholds could be investigated.

High individual variance, habituation and attentional phenomenons as well as influence from various top-down factors

[24] need to be taken in account to achieve the rendering of bodily sensations. Control laws could be adjusted depending on the user as well as the virtual environment context, like getting in a virtual vehicle or flying. Those adjustment might be achieved with offline tests to estimate user sensibility, as well as tuned in real-time depending on user’s state, similarly to motion platform washout algorithms. The role of habituation and attention could also be further studied.

For specific scenarios, smaller haptic devices could be used. For instance, for a flight simulation altitude control exercise, the force feedback might only simulate gravitational tilt: when the plane pitches up, the user is pulled backwards, and when the plane is pointing down the user is pushed forwards.

7 Conclusion

In this short paper, we proposed a novel approach to make self-motion sensations stronger and more compelling in virtual reality applications, by means of head-based force feedback. The provided vestibular and proprioceptive cues enhance the perception of self-motion, making it more egocentric and more bodily, while preserving user’s comfort.

We discussed the technical and scientific challenges raised by head-based force feedback and demonstrated a flight simulator use case with a proof-of-concept prototype based on a high-end grounded haptic arm. We evaluated our system with a user study focused on the motion sensations provided by visuo-haptic displacement stimuli. Our results showed that the visuo-haptic rendering induced more vivid and more egocentric sensations of self-motion than a similar visual-only rendering.

References

  • [1] Varjo website. https://varjo.com/products/vr-3. Accessed: 2021-03-04.
  • [2] Wikepedia notice on 4d film. https://en.wikipedia.org/wiki/4D_film. Accessed: 2021-03-04.
  • [3] C. Boletsis. The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology. Multimodal Technologies and Interaction, 1(4):24, 2017.
  • [4] G. Bouyer, A. Chellali, and A. Lécuyer. Inducing self-motion sensations in driving simulators using force-feedback and haptic motion. In 2017 IEEE Virtual Reality (VR), pp. 84–90. IEEE, 2017.
  • [5] Z. Britton and Q. Arshad. Vestibular and multi-sensory influences upon self-motion perception and the consequences for human behavior. Frontiers in neurology, 10:63, 2019.
  • [6] H.-Y. Chang, W.-J. Tseng, C.-E. Tsai, H.-Y. Chen, R. L. Peiris, and L. Chan. Facepush: Introducing normal force on face with head-mounted displays. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 927–935, 2018.
  • [7] K. E. Cullen. The vestibular system: multimodal integration and encoding of self-motion for motor control. Trends in neurosciences, 35(3):185–196, 2012.
  • [8] F. Danieau, J. Fleureau, P. Guillotel, N. Mollet, A. Lécuyer, and M. Christie. Hapseat: producing motion sensation with multiple force-feedback devices embedded in a seat. In Proceedings of the 18th ACM symposium on Virtual reality software and technology, pp. 69–76, 2012.
  • [9] I. Farkhatdinov, N. Ouarti, and V. Hayward. Vibrotactile inputs to the feet can modulate vection. In 2013 World Haptics Conference (WHC), pp. 677–681. IEEE, 2013.
  • [10] C. R. Fetsch, A. H. Turner, G. C. DeAngelis, and D. E. Angelaki. Dynamic reweighting of visual and vestibular cues during self-motion perception. Journal of Neuroscience, 29(49):15601–15612, 2009.
  • [11] M. Gallagher, R. Dowsett, and E. R. Ferrè. Vection in virtual reality modulates vestibular-evoked myogenic potentials. European Journal of Neuroscience, 50(10):3557–3565, 2019.
  • [12] J. Gugenheimer, D. Wolf, E. R. Eiriksson, P. Maes, and E. Rukzio. Gyrovr: Simulating inertia in virtual reality using head worn flywheels. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pp. 227–232, 2016.
  • [13] L. R. Harris, M. Jenkin, and D. C. Zikovitz. Visual and non-visual cues in the perception of linear self motion. Experimental brain research, 135(1):12–21, 2000.
  • [14] K. Kim. Is virtual reality (vr) becoming an effective application for the market opportunity in health care, manufacturing, and entertainment industry? European Scientific Journal, ESJ, 12:14–14, 2016.
  • [15] Y. Kon, T. Nakamura, and H. Kajimoto. Hangerover: Hmd-embedded haptics display with hanger reflex. In ACM SIGGRAPH 2017 Emerging Technologies, pp. 1–2. 2017.
  • [16] E. Kruijff, A. Marquardt, C. Trepkowski, R. W. Lindeman, A. Hinkenjann, J. Maiero, and B. E. Riecke. On your feet! enhancing vection in leaning-based interfaces through multisensory stimuli. In Proceedings of the 2016 Symposium on Spatial User Interaction, pp. 149–158, 2016.
  • [17] A. Lécuyer, M. Vidal, O. Joly, C. Mégard, and A. Berthoz. Can haptic feedback improve the perception of self-motion in virtual reality? In 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS’04. Proceedings., pp. 208–215. IEEE, 2004.
  • [18] N. C. Nilsson, R. Nordahl, E. Sikström, L. Turchet, and S. Serafin. Haptically induced illusory self-motion and the influence of context of motion. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pp. 349–360. Springer, 2012.
  • [19] N. Ouarti, A. Lécuyer, and A. Berthoz. Haptic motion: Improving sensation of self-motion in virtual worlds with force feedback. In 2014 IEEE Haptics Symposium (HAPTICS), pp. 167–174. IEEE, 2014.
  • [20] S. Palmisano, R. S. Allison, M. M. Schira, and R. J. Barry. Future challenges for vection research: definitions, functional significance, measures, and neural bases. Frontiers in psychology, 6:193, 2015.
  • [21] J. D. Prothero, H. G. Hoffman, D. E. Parker, T. A. Furness III, and M. J. Wells. Foreground/background manipulations affect presence. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 39, pp. 1410–1414. SAGE Publications Sage CA: Los Angeles, CA, 1995.
  • [22] M. Reiner. The role of haptics in immersive telecommunication environments. IEEE Transactions on Circuits and Systems for Video Technology, 14(3):392–401, 2004.
  • [23] B. E. Riecke, J. Schulte-Pelkum, M. N. Avraamides, M. Von Der Heyde, and H. H. Bülthoff. Scene consistency and spatial presence increase the sensation of self-motion in virtual reality. In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, pp. 111–118, 2005.
  • [24] B. E. Riecke, D. Västfjäll, P. Larsson, and J. Schulte-Pelkum. Top-down and multi-modal influences on self-motion perception in virtual reality. In Proceedings of HCI international 2005, pp. 1–10, 2005.
  • [25] M. Rietzler, T. Hirzle, J. Gugenheimer, J. Frommel, T. Dreja, and E. Rukzio. Vrspinning: Exploring the design space of a 1d rotation platform to increase the perception of self-motion in vr. In Proceedings of the 2018 Designing Interactive Systems Conference, pp. 99–108, 2018.
  • [26] F. Soave, N. Bryan-Kinns, and I. Farkhatdinov. A preliminary study on full-body haptic stimulation on modulating self-motion perception in virtual reality. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics, pp. 461–469. Springer, 2020.
  • [27] S.-Y. Teng, D.-Y. Huang, C. Wang, J. Gong, T. Seyed, X.-D. Yang, and B.-Y. Chen. Aarnio: passive kinesthetic force output for foreground interactions on an interactive chair. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13, 2019.
  • [28] A. Väljamäe, P. Larsson, D. Västfjäll, and M. Kleiner. Vibrotactile enhancement of auditory-induced self-motion and spatial presence. Journal of the Audio Engineering Society, 54(10):954–963, 2006.
  • [29] C. Wang, D.-Y. Huang, S.-w. Hsu, C.-E. Hou, Y.-L. Chiu, R.-C. Chang, J.-Y. Lo, and B.-Y. Chen. Masque: Exploring lateral skin stretch feedback on the face with head-mounted displays. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 439–451, 2019.
  • [30] D. Wang, K. Ohnishi, and W. Xu. Multimodal haptic display for virtual reality: A survey. IEEE Transactions on Industrial Electronics, 67(1):610–623, 2019.
  • [31] B. T. Crane, K. Ohnishi, and W. Xu. Human visual and vestibular heading perception in the vertical planes. Journal of the Association for Research in Otolaryngology, 15(1), 87-102., 2014.
  • [32] J. Lecompte, PhD. Thesis. Biomecanique du segment tete-cou in vivo & aeronautique militaire. Approches neuromusculaire et morphologique. Arts et Métiers ParisTech, 2007.
  • [33] J.R. Funk., J.M. Cormier, C.E. Bain, H. Guzman, E. Bonugli, and S.J. Manoogian. Head and neck loading in everyday and vigorous activities. Annals of biomedical engineering, 39.2, 766-776, 2011.
  • [34] B.E. Riecke. Simple user-generated motion cueing can enhance self-motion perception (Vection) in virtual reality. In Proceedings of the ACM symposium on Virtual reality software and technology, 2006.
  • [35] G. Vailland, Y. Gaffary, L. Devigne, V. Gouranton, B. Arnaldi, and M. Babel. Vestibular Feedback on a Virtual Reality Wheelchair Driving Simulator: A Pilot Study. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20), 2020.
  • [36] Y. H. Peng, C. Yu, S. H. Liu, C. W. Wang, P. Taele, N. H. Yu and M. Y. Chen. WalkingVibe: Reducing Virtual Reality Sickness and Improving Realism while Walking in VR using Unobtrusive Head-mounted Vibrotactile Feedback. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–12, 2020.
  • [37] D. Wolf, M. Rietzler, L. Hnatek and E. Rukzio. Face/on: multi-modal haptic feedback for head-mounted displays in virtual reality. In IEEE transactions on visualization and computer graphics, 25(11), 3169-3177. 2019.
  • [38] F. Danieau, A. Lécuyer, P. Guillotel, J. Fleureau, N. Mollet and M. Christie. A Kinesthetic Washout Filter for Force-Feedback Rendering. In IEEE transactions on haptics, 8(1), 114-118, 2014.
  • [39] P. Guillotel, F. Danieau, J. Fleureau, and I. Rouxel. Introducing Basic Principles of Haptic Cinematography and Editing. In WICED , 15-21, 2016.