Virtual Co-Embodiment: Evaluation of the Sense of Agency while Sharing the Control of a Virtual Body among Two Individuals

11/08/2019 ∙ by Rebecca Fribourg, et al. ∙ 0

In this paper, we introduce a concept called "virtual co-embodiment", which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared avatar. In addition, we conducted an experiment to investigate the influence of users' level of control over the shared avatar and prior knowledge of their actions on the users' sense of agency and motor actions. The results showed that participants are good at estimating their real level of control but significantly overestimate their sense of agency when they can anticipate the motion of the avatar. Moreover, participants performed similar body motions regardless of their real control over the avatar. The results also revealed that the internal dimension of the locus of control, which is a personality trait, is negatively correlated with the user's perceived level of control. The combined results unfold a new range of applications in the fields of virtual-reality-based training and collaborative teleoperation, where users would be able to share their virtual body.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 7

page 8

page 10

page 11

page 12

page 13

page 14

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The emerging use of self-avatars in virtual reality (VR) has uncovered numerous novel possibilities to explore the relation between body and mind [1, 2]. Indeed, avatars in VR enable original experiences as they can be altered and controlled in numerous ways. For example, it is possible to be embodied in avatars with a different gender [3] or with morphological changes such as possessing a hand with six fingers [2]. Such experiences have helped to better understand how users perceive their virtual representation in VR and to explore how users are willing to accept a virtual body that differs from their own in terms of visual aspect and control schemes. Furthermore, the need to collaborate in VR reinvigorates research interests in developing new ways for users to collaboratively interact in a virtual environment (VE), especially through their avatars [4, 5]. More specifically, several VR shared experiences, in which two individuals can share a first-person view, have been explored, especially in terms of gesture training or collaborative teleoperation [6, 7]. However, previous studies on this topic did not involve scenarios in which several users could be embodied in the same avatar.

In this paper, we introduce a new concept, termed “virtual co-embodiment.” While the concept of “co-embodiment” has been recently defined outside of the scope of VR [8], this is the first study to the best of the authors’ knowledge to define “virtual co-embodiment” as a situation that enables a user and another entity (e.g., another user, robot, or autonomous agent) to be embodied in the same virtual avatar. Such a situation raises the question about how sharing a virtual body influences ones’ perception and actions in the VE.  Potential applications of this new concept range from VR-based motion training to collaborative teleoperation. As a first step, this study focused on two users sharing the same virtual body. In VR, the sense of embodiment (SoE) is a theoretical framework widely used to evaluate how users perceive and accept their avatar to be their own representation in the virtual world [9, 10]. This framework is often divided into three dimensions [1]: the sense of agency (SoA), sense of self-location (SoSL), and sense of body ownership (SoBO). However, owing to the particularity of the virtual co-embodiment experience, in which users share control over their virtual body, and the potential implications that sharing this control would increase the interaction capabilities of users in a VE, we decided to focus our efforts on the assessment of the SoA. The ability to modulate the sharing of avatar control enables the possibility to assess the SoA when two users collaborate to achieve a task while embodied in the same virtual avatar. Previous research explored the influence of perceptual and motor mismatches over the SoA. Such studies showed that it is possible for users to feel agency toward actions they did not perform [11], and highlighted interesting insights regarding the SoA with its possible modulations, inspiring the following question: To which level can users experience a SoA over a shared virtual avatar?

To answer this question, we conducted a VR experiment in which 12 pairs of individuals participated. Each pair was embodied in the same shared avatar from a first-person perspective (1PP) and was asked to perform different tasks in the VE while sharing the avatar control. The control was shared by averaging the position and orientation of the hands of both users according to a predefined level of control for each user (from no-control to full-control) and by animating the avatar accordingly.  Our two main hypotheses were as follows: (1) the SoA would be positively correlated with the degree of control over the shared avatar; and (2) the higher the prior knowledge over the task to perform, the higher would be the SoA.Overall, our results support our main hypotheses, showing that the SoA over a shared avatar is significantly dependent on both the users’ level of control and freedom of movements inferred by the type of task. Whether users possessed the same prior knowledge of the action to perform also influences the SoA. Interestingly, we observed that users tend to feel some control over the avatar even when they actually have little or no control over the virtual body, in cases where their movements are more constrained. This suggests that even with little or no control over a shared avatar, users are capable of feeling some agency toward the virtual body. Our results also reveal that the internal dimension of the locus of control (LoC), which is a user personality trait, is negatively correlated with the manner in which users feel in control. Our combined results are promising for possible applications in the fields of VR-based motion training or collaborative teleoperation, where sharing the avatar control with another user could emphasize the efficiency of previously developed systems [6, 12].

The main contributions of this paper are twofold: the concept of “virtual co-embodiment” is introduced, and this is the first study to provide a baseline for a more in-depth analysis of virtual co-embodiment, and avatar control more generally, on human behavior and self-perception.

2 Related Work

According to Kilteni et al. [1], the SoE consists of SoA, SoBO, and SoSL. As stated earlier, SoA refers to the feeling of control (FoC) over actions and their consequences, whereas SoBO refers to one’s self-attribution of body and SoSL refers to one’s spatial experience of being inside a body. In VR, the SoA can easily be elicited when the user motion is mapped onto the virtual body in real-time or near real-time [1]. Such visuomotor congruence can also induce SoBO [13], as long as the virtual body is structurally and morphologically similar with one’s biological body [1]. In contrast, SoSL can be achieved by an immersion from 1PP as it is highly determined from the egocentric visuospatial perspective.

While observing a virtual-realistic body from the 1PP with congruent visuomotor cues is considered to be sufficient for inducing the SoE in VR, several studies have also demonstrated that incongruent visuomotor feedback can affect the SoE. In particular, both SoA and SoBO have been found to be reduced when a discrepancy exists between vision and motor information [13, 14, 15]; however, they can still be induced to some extent. For instance, Maselli and Slater [16] showed that visual realism of the avatar favors the SoE, despite the presence of incongruent visuomotor and visuotactile cues. More recently, Kokkinara et al. [17] showed that both SoBO and SoA can be induced over a virtual-body walking from a 1PP, even though participants are actually seated and only allowed head movements. Such findings suggest that participants can feel SoA and SoBO in some situations despite visuomotor discrepancies. However, how the sharing of the control of a virtual body with another user influences the SoE is unknown. Therefore, we focus on sharing the control of a virtual body in this study, and the following sections will cover related work on the SoA and shared body experiences.

2.1 SoA

2.1.1 Theory

As stated earlier, the SoA is considered as one of the components of the SoE in the VR field. However, in the fields of philosophy and psychology, the SoA is considered to form a fundamental aspect of self-awareness together with SoBO [18]. Therefore, numerous studies on SoA have been conducted in the fields of philosophy and psychology to examine human consciousness. Although the mechanisms of human consciousness are still not fully understood, two influential theoretical views have been put forward: a comparator model [19] and retrospective inference view [20]. The comparator model suggests that the comparison between predicted and actual consequences of an action through sensorimotor processes determines the SoA [19, 21]. Thus, the mismatch caused by spatial and temporal distortion of movements or outcomes can attenuate the SoA [22]. Indeed, numerous studies have shown evidence that discrepancies between the actual movement and the corresponding visual feedback [15, 14] or sensory outcome [21, 23] of the action negatively affect the SoA. In comparison, retrospective inference view emphasizes external situational cues [20]. According to Wegner’s theory of apparent mental causation [20], the SoA arises if (1) an intention precedes an observed action (priority), (2) the intention is compatible with this action (consistency), and (3) the intention is the most likely the cause of this action (exclusivity). Therefore, priming is often used to modulate the SoA by manipulating prior conscious thought about an outcome [24, 25, 26]. However, the SoA is increasingly recognized as being based on a combination of internal motor signals and external evidence about the source of actions and effects [24, 11, 27]. Thus, although spatial and temporal contiguity between one’s own and observed movements are the main cues for SoA [22, 14, 15], higher-level cognitive processes, such as background beliefs and contextual knowledge relating to the action, also influence the induction of SoA [28, 29].

2.1.2 Measures

The measurements of SoA are generally categorized as implicit and explicit measures [28]. Implicit measures, such as sensory attenuation [21], intentional binding [30, 31], and neurophysiological markers [32] assess a correlation of voluntary actions about the agentic experience [28]. Alternatively, explicit measures are based on the subjective judgments of the FoC, the authorship or attribution of the actions or their corresponding outcomes [23, 26, 25, 11, 33]. Most studies have used explicit measures, especially in VR [34, 35, 36]. These measures are typically assessed in paradigms using button presses which produce sensory feedback [23] or simple movements [14, 15, 20, 37, 38, 33, 34]. For simple movements, a moving cursor associated with a joystick [14, 15] or mouse [20, 37] and a visual feedback of hand movements through a mirror [38], a TV-screen [33], or VR [34] are often used.

Besides these experimental measurements, some personality traits have been shown to be related to SoA. Neurological studies have shown that patients with schizophrenia tend to feel abnormal SoA, i.e., they have less ability to distinguish sensations due to self-caused actions from those due to external sources [18, 15, 39]. Indeed, schizotypy personality traits, an indicator of a predisposition to schizophrenia, have been shown to be correlated with an abnormal SoA [40]. In addition, some studies have revealed that SoA is correlated with one’s personality of LoC, which has been often used in the fields of education, health, and clinical psychology. LoC refers to the degree to which people believe that they have control over the outcome of events in their lives, as opposed to external forces beyond their control [41]. The Internal-Personal-Chance (IPC) test [42] is one of the measurements for LoC, indicating a person’s relative standing on each of the three dimensions of internal, powerful others, and chance. Among them, the individuals with a strong internal LoC believe events in their life are derived primarily from their own actions. In a study including manipulations of the SoA in VR, based on the principles of priority, exclusivity, and consistency, Jeunet et al. [32] suggested that the internal dimension of LoC is positively correlated with participants’ level of agency.

2.1.3 Illusory SoA

As described earlier, spatial displacement or temporal delay between action and outcome attenuates the SoA [22, 14, 15]. However, we feel illusory SoA over distorted movements as long as the displacement or delay is under the threshold. For example, a recent study using VR showed that spatial manipulations of of angular offset from 1PP did not attenuate SoA [34]; this showed much lower detection thresholds than previous studies without VR [14, 15]. In addition, illusory SoA can occur over the actions or outcomes made by someone else when there is a close match between prior intentions and subsequent actions. In a classic study by Nielsen [38], participants were instructed to draw a straight line to the goal point. After some repetitions, the experimenter secretly inserted a mirror so that the participants were looking at another person’s hand in a mirror. They experienced the illusory SoA and attributed the hand to their own. In Wegner and Wheatley’s “I-spy” experiment [20], participants and an experimenter jointly controlled a cursor. Auditory priming of action-relevant thoughts induced illusory SoA even through the cursor was being controlled by someone else. This suggests that post-hoc judgments of SoA can easily be distorted in a joint action when the action source is ambiguous. Yokosaka et al. [43] reported that when participants watched their own and another person’s hand motion alternately from 1PP, they felt illusory SoA over the movement, although they were aware that they were not performing a united motion.

Moreover, illusory SoA is possible over body movements even when no actual corresponding action is being performed. In the “helping hands“ experiment by Wegner et al. [11], participants watched themselves in a mirror while an experimenter standing directly behind them extended and moved his or her arms as if the participants themselves moved their arms. They reported that participants felt an illusory SoA for another person’s hands when they were primed about instructions for that person’s movements in advance, although they factually did not move. VR is also used to induce illusory SoA when passively observing movements of a walking avatar from 1PP [17]. To summarize, in situations where individuals do not move, the action priming and movement observation from 1PP are considered to be important for illusory SoA. Therefore, we believe that users might experience all the three aforementioned types of illusory SoA in a virtual co-embodiment experience, as the feedback component originates partially from one’s own movements and partially from someone else’s movements.

2.2 Shared Body Experiences

Some previous studies have developed shared body experiences, e.g., two individuals sharing 1PP [44, 45], kinesthetic experiences [46], or body representations [44, 47, 48, 49, 47, 50]. In particular, Petkova et al. [44] introduced the perceptual illusion of body swapping and showed that 1PP of another person’s body, in combination with the receipt of correlated multisensory information from the body, was sufficient for inducing body ownership. Mutual paralleled first-person-view-sharing systems, in which a person can observe others’ first-person video perspectives as well as their own perspective in realtime, are also used in entertainment, remote collaboration, and skill transmission systems [45, 7, 51]. Other approaches have also explored the sharing of other senses, e.g., BioSync [46] which is an interpersonal kinesthetic communication system allowing users to sense and combine muscle contraction and joint rigidity bidirectionally through electromyogram measurement and electrical muscle stimulation. Lastly, the enfacement phenomenon is a self–other merging experience, in which participants reported that morphed images of themselves and their partner contained more self than other only after synchronous multisensory stimulation on their faces [47, 48, 49].

3 Co-embodiment platform

Fig. 2: Physical setup: two users are physically sitting in front of each other and are immersed in the same avatar from a 1PP.

In this section, we discuss about the proposed virtual co-embodiment platform, which was used to conduct an experiment, as described in Section 4.

The platform was developed in Unity and allows two users to share the same virtual environment and interact in real time, while being embodied in the same avatar. Our setup is based on two HTC Vive head-mounted displays (HMDs) with two HTC Vive controllers to immerse participants in the VE. The application was run on Unity 2018.1.0f2 at a constant frame rate of 90 Hz, and both computers were physically connected on the same network to minimize latency. Users were embodied in an anthropomorphic virtual avatar from a 1PP (see Figure 2). In the center of the tracking zone, two chairs and a table were placed, enabling users to sit in front of each other. The physical furniture had its virtual counterpart in the VE providing both a reference frame and passive haptic feedback. The VE in which users were immersed comprised an empty room.

In terms of avatar appearance, we chose to use a realistic model in our experiment as well as immerse users in a 1PP, as these criteria were reported by recent studies to be important for enhancing the overall SoE  [16]. As animation and control quality are known to be strongly linked to the SoA, we primarily focused on avatar animation. This was especially challenging in our case owing to the shared control of the avatar. Note that, the differentiation of avatar animation and control inputs is necessary for its computation.

In the case of a single-user situation, the animation of the avatar depends solely on the control inputs of this user. However, in this study, the control inputs result from the combination of the inputs of two users. We therefore implemented a method that allowed the sharing of the avatar control with another user. As a virtual view that does not correspond to the user’s own head movement could cause motion sickness, each user observed his/her own perspective in accordance with his/her head movement; the head position and orientation of the HMDs were not shared. Regarding the controller, we computed the weighted average of the real-time position and orientation of each user’s controller, and applied it to the shared avatar’s controller. The weight defining the level of control could be continuously changed from 0% to 100%. The weighted average position and orientation were then computed by interpolating between user controller positions and orientations.

Further, we chose to focus on the animation of the arms and torso because, as stated in [32], the arms and hands are the main body medium for interactions in VEs. In addition, in our setup, as users were seated on a chair, only animation for the upper body was required, which was animated through inverse kinematics using the Final IK Unity package. The Final IK computed inverse kinematics using position and rotation inputs of the head and controllers of the shared avatar, obtained through the previous shared control computation. Users could thus observe the same shared avatar, the movements of which, computed by inverse kinematics, would follow more or less their own hand according to their level of control at a certain time.

4 Experiment

4.1 Experiment Summary

We conducted an experiment, in which we explored the influence of the degree of control of an avatar shared with another person on one’s own SoA. More precisely, we address the two following questions. Does the degree of shared control have an impact on one’s FoC toward the avatar? Does the predictability of the avatar movement have an impact on one’s FoC toward the avatar?

In literature, the SoA was shown to largely depend on the degree of discrepancy between the predicted sensory feedback of an action and the actual outcome [23]. In addition, participants were observed to feel illusory SoA over distorted movements when the discrepancy is under a certain threshold [34]. Moreover, other studies focused on situations in which participants did not move and experienced illusory SoA toward movements they did not perform, when they had prior knowledge of the action and were immersed in a 1PP [11]. Based on these findings, we hypothesized that the level of control over the avatar shared between the two participants would influence the SoA. We also hypothesized that the freedom of movement in the task and whether both participants had the same prior knowledge of the action would also influence the SoA.

To test these hypotheses, we designed an experiment in which two participants were immersed simultaneously in a VE and were embodied in the same avatar. More precisely, the experiment was divided into three successive phases: the first exposure phase, followed by the main experiment phase, and finally the last exposure phase.

  1. The first exposure phase was conducted to allow the users to be accustomed to the shared body control and experimental environment (see Figure 3). Moreover, we took advantage of this phase to evaluate users’ SoA and SoBO to assess their level of embodiment when possessing full (independent body) or half control (shared body) over the virtual avatar.

  2. To explore the influence of the level of shared control toward the avatar on the SoA, five controlling weights were considered between 0% and 100%. In addition, to evaluate the influence of the freedom of movement and the intention toward an action on the SoA, three tasks were considered (Figure 4).

  3. This phase was conducted to evaluate  potential training effects of the main experiment over agency and ownership ratings.

Fig. 3: Exposure phase in which participants were asked to light candles (left); main experiment in which participants were asked to touch some spheres with the tip of their controller (right). Images are shown from third-person perspective for illustrative reasons.
Fig. 4: The three tasks that the users were asked to perform. Free task: participants had to choose which sphere to touch (left). Target task: the sphere that the participants were to touch was highlighted (center). Trajectory task: the sphere to touch was highlighted and participants had to follow a path from the table to the sphere with the tip of their controller (right).

4.2 Participants

Twenty-four male participants from the university campus participated in the experiment [age: 5 (SD)]; they were recruited from among both students and staff. They were all unaware with respect to the purpose of the experiment, had normal or corrected-to-normal vision, and gave written and informed consent. The study conformed to the declaration of Helsinki. The participants were paired with those they had never interacted with earlier. Among the participants, seven had no previous experience with VR, fourteen had limited previous experience, and three were familiar with VR. All participants were right-handed male Caucasians, to match the visual appearance of the virtual avatar as much as possible.

4.3 Experimental Protocol

The overall organization of the experiment is summarized in Figure 5 and is further described as follows.

Upon their arrival, participants read and signed the experiment consent form and filled in a demographic questionnaire. They also completed the IPC cognitive test [42]. The internal score computed from this test was used later to measure LoC and explore its influence on the SoA. Then, they were briefed about the experiment through an explanatory video. They were explained that they would share a body and control over it with the other participant. After the explanation, they were instructed to sit on a chair in front of a table facing each other and wear an HMD to get immersed in the VE (Figure 2).

As previously explained, the experiment was divided into three phases, which the participants experienced in order: the first exposure phase, main experiment, and last exposure phase. In addition, while participants were immersed in the VE, they were instructed not to talk or interact with each other. As the tasks to perform only required motions of the right arm, we decided to focus on the right arm and did not animate the left arm. Participants were therefore asked to keep their left arm along their torso and not move it. After the experiment, they were instructed to remove their HMDs and provide general comments and feedback through a web form. The overall process took approximately 1 h.

4.3.1 First and last exposure phases

Participants started with the first exposure phase and finished with the last exposure phase, in which they were asked to light candles using a virtual lighter (Figure 3, left). Once participants had lit all their candles, the candles would extinguish, and the participants were asked to light them again. This task lasted for 2 min, and each phase was repeated twice (2 blocks): once with half of the avatar control for each participant, and once with full control over their own avatar. Each block would finish with an ownership and an agency questionnaire, which consisted of 11 items (Table I); the participants answered based on a scale ranging from 1 to 7 by pressing buttons on the controller in their hand. Each participant thus answered the questionnaires four times.

4.3.2 Main experiment

In the main experiment, the avatar was always shared and the weight of avatar control for a participant varied between 0% and 100% (respectively 100% minus this weight for the other participant). We considered five weights between 0% and 100% to evaluate how differences in the degree of control would impact participants’ SoA. Thus, we hypothesized that the SoA would be positively correlated with the degree of control.

Participants were asked to perform three tasks involving touching one virtual sphere among four spheres, with the extremity of virtual controller hold in the right hand. Four spheres were presented in front of the participants, all at equivalent distances from their right hand. More precisely, by using the original 3D model of the HTC Vive controller, we attached a short rod with a small sphere on top; this tip collided with the sphere (Figure 3, right).

There were three types of tasks: free, target, and trajectory. The different tasks contrasted from each other with respect to the freedom of movement they allowed and whether both participants possessed the same prior knowledge of the same action to perform. More precisely, in the free task, each participant was free to choose which sphere to touch (Figure 4, left). In the target task, the sphere to touch was highlighted with a colored halo (see Figure 4, center). Similarly, in the trajectory task, the sphere to touch was highlighted and the participants were asked to follow a path from the table to the highlighted sphere by using the tip of the controller; this task required more precision (see Figure 4, right).

These three tasks were selected in line with the hypothesis that constraints in the movements and prior knowledge of the action to perform (i.e., the intention toward the action) both impact the SoA. In the free task, each participant was free to choose which sphere to touch (Figure 4, left), under a condition where the movement of participants was not restricted and where the movement intention was not assuredly shared as participants might not decide to touch the same sphere. In the target task, the sphere to touch was highlighted with a colored halo (Figure 4, center), under the condition that the movement was not restricted and the movement intention was shared as both participants focused on touching the same sphere. In the trajectory task, the sphere to touch was highlighted and participants were to follow a path from the table to the sphere by using the tip of the controller. This task required more precision (Figure 4, right), and included both movement restriction and the shared intention, as participants had to follow a specific path to touch the same sphere.

These choices were driven by the demonstration of previous studies that SoA increases when participants have more action choices [52]. However, in our case, owing to changes in the level of control over the avatar, the more the participants had the choice of the action (in the free task compared to the target and trajectory tasks), the more the visuomotor discrepancies were expected between participants and avatar movements. We thus supposed that the SoA would be higher for the target and trajectory tasks with smaller visuomotor discrepancies. Considering the results of Wegner et al. [11], we also expected that SoA would be higher in tasks where the intention of movement was shared (in target and trajectory tasks compared to free task).

In each task, participants performed 45 trials. For each trial, the participants started observing their own avatar over which they had full control. To ensure that both participants had the same initial position, they were asked to place their right hand holding the controller on the table on a specific virtual reference and remain on the initial reference. After 2 s, the four spheres were displayed in red with a message “don’t move yet”. The message disappeared after 2 s, the spheres turned blue, and then the participants could perform the task. When a sphere (any of the four spheres for the free task, specified sphere in other tasks) was touched for 1 s by the tip of the controller of the shared avatar, the task was over and the following question was asked to both participants: “On a scale ranging from 1 to 7, how much did you feel in control during this trial?”. As such, we followed the same protocol as that used by Jeunet et al. [32] to assess the SoA through a question that is easily understandable by participants and proved to relate to the judgment of agency. Participants provided a rating between 1 and 7 to validate their choice using the controller. When both participants had answered the question, they were asked to place their hand on the highlighted spot to start the next trial.

4.4 Experimental Design

Fig. 5: Diagram of experimental flow.

4.4.1 First and Last Exposure Phases

Variable Question
Agency 1) The virtual arm moved just like I wanted to, as if it was obeying my will.
2) I felt as if I was controlling the movement of the virtual arm.
3) I felt as if the virtual arm was controlling my will.
4) I felt as if the virtual arm was controlling my movements.
5) I felt as if the virtual arm had a will of its own.
6) I felt as if I was causing the movement I saw.
Ownership 1) I felt as if I was looking at my own arm.
2) I felt as if the virtual arm was part of my body.
3) I felt as if the virtual arm was my arm.
4) I felt as if I had no longer a right arm, as if my right arm had disappeared.
5) I felt as if the virtual arm was from someone else’s body.
TABLE I: Questionnaire used in the first and last exposure phases. Questions in italics are control questions regarding agency and ownership.

A within-subject design was set up for these experimental phases, where we considered two independent variables: control and stage. The main variable (control) considered whether the participants were sharing the avatar, and possessed two levels: 1) participants sharing the avatar with 50% control each (Half) or 2) participants having full control over their own avatars (Full). The stage variable determined whether the task was completed in the first or last part of the experiment, and thus had two levels: First and Last. This part of the experiment was divided into two blocks corresponding to the two levels of the control condition. In both first and last exposure phases, whether participants would start with one block or the other was fully counterbalanced in the experiment.

The measured data (dependent variables) in a questionnaire were inspired from previous work [53, 10, 54], where questions were divided in two groups: agency and ownership (Table I). For each question, participants were asked to provide rating based on a 7-point Likert scale.

4.4.2 Main Experiment

We also adopted a within-subject design for the main part of the experiment, considering two independent variables: weight and task. The weight variable determined the degree of control the participants had over the avatar and had five levels (W0, W25, W50, W75, and W100). For each pair, weight was inverted between participants, i.e., the sum of the controlling weights of the two participants was always 100%. Task corresponded to the three tasks included in the experiment (Free, Target, and Trajectory; see Section 4.3.2 for details). The main experiment was divided into three blocks. To minimize the ordering effect, the orders of the blocks and tasks were counterbalanced following a Latin square design. Each iteration of Task in one block comprised one training trial (with half control of the avatar) and three repetitions of the five trials (for the five levels of Weight). The order of Weight levels within the three repetitions was fully counterbalanced. Without considering the training trials, each participant performed 135 trials. Each trial lasted around 3 s.

The measured data (dependent variables) considered the performance and behavioral measurements. Regarding performance, we measured task-completion time, i.e., the time required to select the sphere after it turned blue (in seconds). Regarding behavioral measures, the motions (position and orientation per frame) of the participants’ and shared avatar’s controllers were recorded during the trials. Finally, the subjective FoC ratings for the question, “How much did you feel in control?” asked after each trial were rated on a 7-point Likert scale. Participants also reported general comments and feedback at the end of the experiment.

In summary, considering our experimental design, our main hypotheses are as follows.

  • When the degree of control (Weight) decreases, the FoC ratings decrease.

  • The FoC ratings will be higher for the tasks in which movements are more constrained (Trajectory   Target  Free).

  • Participants with a higher Internal score of LoC experience higher FoC.

5 Results

Fig. 6:

Left: Point plot of the mean subjective ratings of Feeling of Control (FoC) considering Weight of control and Task during main experiment. Right: Scatter plots with linear regression lines of FoC ratings on Weight for each task (

Free: =0.83, Target: =0.66, Trajectory: =0.74). Error bars (left) and translucent bands (right) indicate 95% CIs. A total of 10,000 bootstrap samples were used to estimate each 95% CI.

5.1 Main Experiment

Eleven trials out of all 3240 trials were excluded from the analysis after a visual inspection of the raw data revealed that either the task completion time, participant motion, or motion of the avatar exhibited abnormal values (values outside the range of three standard deviations from the mean). ANOVA analyses were conducted when the normality assumption (Shapiro–Wilk’s normality test) was not violated (

). In particular, two-way ANOVA analyses with repeated measures were conducted, considering the within-group factors of Weight (5 levels: W0, W25, W50, W75, and W100) and Task (3 levels: Free, Target, and Trajectory

). When the sphericity assumption was violated (Mauchly’s sphericity test), the degrees of freedom were corrected using the Greenhouse–Geisser correction. In addition,

was provided for the quantitative comparison of effect sizes. Finally, Tukey’s post-hoc tests () were conducted to check the significance for pairwise comparisons of the parametric data.

When the normality assumption was violated (Shapiro–Wilk’s normality test, ), Friedman test was conducted for each task independently followed by a post-hoc Wilcoxon-signed ranks test. For multiple post-hoc comparisons, Holm correction was applied for the non-parametric data. As for the correlation analyses, Pearson’s r () was used for parametric data and Spearman’s r () was used for non-parametric data.

5.1.1 Feeling of Control (FoC)

The two-way ANOVA analysis revealed a significant main effect of Task [, , ] and of Weight [, , ]. However, the two-way ANOVA also exhibited a significant interaction effect between Task and Weight [, , ]. First, Tukey’s post-hoc tests indicated that, for all tasks, the FoC significantly decreased as the degree of control (Weight) decreased ( for all), which is further supported by the primary effects on Weight. Thus, this result supports [H1]. Next, when comparing the FoCs for each Weight level (see Figure 6 left), Tukey’s post-hoc tests demonstrated that, for the W0 Weight, the FoC was significantly higher for the Target task than for the other tasks (both ). In contrast, for the W25, W50, W75 levels of Weight, the FoC was significantly lower for the Free task than for the other tasks (all ). Finally, for the W100 Weight, the post-hoc tests did not exhibit any significant difference. Thus, these results only support [H2] partially, as although the Free task (the less constrained task) consistently obtained the lowest FoC ratings (except for the W100), this effect was not visible between Target and Trajectory tasks.

As the ANOVA analysis indicated that the strongest effects originated from the Weight factor, to further characterize the relationship between FoC and the Weight factor, a linear regression analysis was conducted across participants for each task (Figure

6 right). The regression equations were

The regression equations exhibited linear positive correlations between the FoC and the Weight. To determine whether the computed slopes differed significantly from 0, we computed the slope of each participant’s linear regression and conducted a t-test (

: Slope is equal to 0): (Free: t(23)=35.665, p .001, Target: t(23)=13.219, p .001, Trajectory: t(23)=16.622, p .001). The results of the t-test indicated that the mean slopes all significantly differed from 0. These results further support [H1]. Section 5.2 further analyzes the FoC ratings in correlation with the IPC scores.

5.1.2 Task Completion Time

Fig. 7: Box plots of the task completion time considering the different Weight groups for each task.

Because the task completion time was dependent on the weights of the two participants (their sum adding to 100%), for the task completion time analysis, the Weight group factor had only three levels: W0-W100, W25-W75, and W50-W50 (see Figure 7). In addition, owing to the different natures of each task (aimed movement, path following task), we did not assess the differences among Tasks for the task completion time. Therefore, we conducted three Friedman tests considering Weight group as a factor, one for each task. The Friedman tests exhibited significant differences among the task completion times of the Weight groups only for the Free task (=14, p .001), and no significant differences were found for the Target task (=0.17, n.s.) or the Trajectory task: (=3.5, n.s.). Post-hoc pairwise comparisons indicated that for the Free task, the task completion time was significantly smaller in the W0-W100 condition (W0-W100 W25-W75: Z=-2.81, p.01, W0-W100 W50-W50: Z=-3.30, p.01). No significant differences were found between W25-W75 and W50-W50 (Z=-1.68, p .092).

5.1.3 Motion Data

Fig. 8: Box plots of the mean offsets between the positions of the avatar’s hand and the participant’s actual hand considering the Weight for each Task.
Fig. 9: Scatter plots with linear regression lines of FoC ratings on mean offsets between the positions of the avatar’s hand and the participant’s actual hand for each Task. Translucent bands indicate 95% CIs.

The offsets (Euclidean distance) between the positions of the participant’s and avatar’s hands were calculated for each frame and then averaged for each trial (see Figure 8). This value provided a rough estimate of the overall visuo-motor discrepancies for each trial. We excluded the W100

condition from the analysis as the discrepancy was 0 regardless of the Task (condition with full control). The residuals did not follow a normal distribution; thus, Friedman tests were considered. In addition, the analysis considered each Task independently. Friedman tests exhibited significant differences of the mean offsets among Weights for all Tasks: (

Free: =56.75, p .001, Target: =67.25, p .001, Trajectory: =61.85, p .001). Post-hoc pairwise comparisons indicated that the mean offsets were significantly smaller when the Weight was larger for all comparisons in all Tasks (p.001 all) except for the comparison between offsets in the W0 and W25 conditions for the Free task.

An additional correlation analysis was performed to assess the link between the mean offset across all weights and the perceived FoC. The correlation analysis revealed that the offsets were negatively correlated with FoC for all Tasks: Free: =-0.84, p.001, Target: =-0.84, p.001, Trajectory: =-0.83, p.001) (See Figure 9).

Moreover, to check if the mean offsets would vary between tasks, another analysis was performed for each weight. Friedman tests revealed significant differences among the mean offsets of Tasks for W0 (=28.58, p .001), W25 (=32.33, p .001), W50 (=37.33, p .001), and W75 (=32.33, p .001). Post-hoc pairwise comparisons showed that for W0, W25, W50, and W75 the mean offsets were significantly higher for Free compared to Target and Trajectory (both p.001).

Finally, to gain some insight regarding the global behavior of users during each trial, speed profiles were computed for each participant per Weight and Task for each trial. Speed profiles were normalized in time by resampling the values at 100 intervals between the start (time 0%) and end of the trial (time 100%). We then computed the mean and standard deviation of the speed profiles between all participants as reported in Figure 10. To compare the speed profiles for each Task and for each interval, we conducted a Friedman test considering Weight as a factor. Tasks were not compared among each other as the nature of each Task was different. Among those intervals, post-hoc pairwise comparisons (Wilcoxon signed-rank tests) were performed to find pairwise differences among different Weights. The results of pairwise comparisons are also summarized in Figure 10, in which each Weight is denoted by a color; lighter colors are associated with lower Weights and vice-versa, and colored segments are placed at the intervals in which significant differences were found. Thus, the presence of a colored segment indicates that a significant difference (p .05) was found between the current interval and the corresponding interval of the color-coded condition.This result allows us to highlight the tasks in which changes in the control induced differences in participant behavior. For example, for Target and Trajectory tasks, the Weight seems to only have a visible impact at the end of the motion, in particular for W0 and W25, whereas more discrepancies were found for the Free task.

Fig. 10: Averaged Speed profiles between all participants for each Weight and Task, normalized in time and re-sampled at 100 intervals. Colored segments were placed at intervals in which significant differences (Friedman and Wilcoxon pairwise comparison tests) were found. Colors are associated to a specific Weight, from the lightest (lower Weight) to the darkest (higher Weight).

5.2 Personality Traits

Fig. 11: Scatter plots with linear regression lines between the internal score of the IPC test and the regression coefficient terms obtained between the FoC ratings and Weight for each participant (slope (top) and intercept (down)). Translucent bands indicate 95% CIs. 10,000 bootstrap samples were used to estimate each 95% CI.

According to the responses of the IPC test, each participant obtained three scores (from 0 to 48), one for each dimension of the IPC test (i.e. Internal, Powerful Others, and Chance). Each score was calculated by adding the responses of the eight items for each dimension and a constant of 24. Similar to previous studies, only the Internal dimension was assessed [32], as it was found to be the dimension that was more related to the FoC. A high rating on the Internal score indicates that the subject has a strong internal Locus of Control (i.e., they believe that events in their life derive primarily from their own actions).

First, to verify whether participants with higher internal score of IPC tended to experience higher FoC when they had full control (W100), we conducted a correlation analysis between the internal scores and the mean FoC scores in the W100 condition for each task. As a result, no significant correlation was found between the internal score and the FoC: Free (=0.23, p .29), Target (=0.33, p .11), Trajectory (=0.25, p .23). This result might be explained by a ceiling effect of very high values of FoC in the W100 condition. This result does not support [H3].

In contrast to previous studies, the modulation of the participant’s control was quantified by the Weight parameter. This enables us to analyze the correlation of the internal component of the IPC with the FoC in a wider range of FoC values. First, as already detailed in Subsection 5.1.1, we computed the correlation between the Weight and the FoC for each participant. The intercept coefficient could be considered as the FoC “baseline,” while the slope could be related to the “sensitivity” to changes in the participant’s control. In other words, the slope provides information on how much the change in the participant’s control influences the FoC, and the intercept provides a lower bound for the FoC. In practice, in our scenario, both parameters are strongly correlated because there is a strong ceiling effect for the FoC at W100. Thus, we computed the regression equations of FoC on Weight for each participant and performed correlation analyses of both the slopes and intercepts with the participants’ score of the Internal dimension of the Locus of Control (from the IPC test). The results show a positive correlation between the slope and the Internal dimension for each Task (Free: r=0.54, p.01, Target: r=0.47, p.05, Trajectory: r=0.49, p.05, see Figure 11 up), as well as the negative correlation between the intercept and the Internal dimension for Target and Trajectory tasks (Target: r=-0.44, p.05, Trajectory: r=-0.47 p.05), and a marginally significant effect for the Free task (r=-0.40, p=.05) (See Figure 11 down).

These results seem to suggest that participants with a higher Internal score were more sensitive to changes in the avatar control as they had lower intercept values and higher slope values.

5.3 First and Last Exposure Phases

Fig. 12: Box plots of mean ratings of agency (Left) and ownership (Center) obtained in the questionnaires in First and Last exposure phases. Right: Scatter plots with linear regression lines of agency ratings on ownership ratings. Translucent bands indicate 95% CIs. 10,000 bootstrap samples were used to estimate each 95% CI.

The agency and ownership ratings for the First and Last exposure phases were aggregated and averaged (control item answers were inverted) to compute one agency and one body ownership score per participant. Owing to the non-parametric nature of the data and the need of testing interaction effects, we applied an aligned rank transform (ART) on the data. This procedure enables the use of ANOVA to analyze the interaction effects with non-parametric data [55]. Two-way repeated measures ANOVAs with the within-subjects factors Control (2 levels: Half and Full) and Stage (2 levels: First and Last) were performed for both agency and ownership scores. Regarding the agency scores, the ANOVA revealed a significant main effect of Weight [, , ] and Stage [, , ] (Figure 12 Left). In addition, the Weight Stage interaction effect was significant [, , ]. Thus, we only report the post-hoc tests for the interaction effect. First, post-hoc pairwise comparisons using the Wilcoxon signed-rank test (Holm corrected) showed that in both First and Last phases the agency scores were significantly higher in the Full condition than in the Half condition (First: Z=-5.29, p.001, Last: Z=-5.29, p.001). Second, in both Full and Half conditions, the agency scores were higher in the Last than First phases (Full: Z=-2.58, n.s., Half: Z=-2.09, n.s.).

Regarding the ownership scores, the ANOVA showed a main effect of Weight [, , ] and a marginally significant main effect of Stage [, , ] (See Figure 12 Center). The Weight Stage interaction effect was not significant [, , ]. Similar to the agency ratings, the ownership scores were significantly higher for the Full condition. In addition, we conducted a correlation analysis between the agency and the ownership scores for each participant, showing that ownership was positively correlated with agency in the Half condition (=0.54, p.01), but not in the Full condition (=0.31, p 0.14) (See Figure 12 Right).

6 Discussion

In this section, we discuss how the results can be interpreted in terms of SoA, which is measured by subjective judgments of FoC over the participants’ actions. We also provide additional insights regarding the Locus of Control and the relation between SoA and SoBO.

6.1 Main Results

The SoA results show that changes in the degree of control clearly influenced the SoA, which validated [H1]. More precisely, the FoC ratings, which were treated as an explicit measure of the SoA according to previous studies [11, 26], increased linearly with the increase in the degree of control for all three tasks. This result can be explained by the fact that the higher the degree of control is, the closer the visual feedback of the avatar hand is to the actual hand position of the participant, thereby reducing visual mismatch between the movements of the avatar and the participants’ actual movements. As stated by Farrer et al. [56], our ability to recognize SoA from the visual cues of movement tend to decrease in case of mismatch between visual feedback and actual movement, i.e., when there are visuo-proprioceptive discrepancies, which could justify the correlation observed between the SoA and the degree of control. The participants’ feedback is also in line with this interpretation, as they expressed their disturbance when their arm was controlled out of their will: “It was confusing when the hand was going in the direction I intended it to go but the speed did not totally match my movements”.

Another interesting result reveals that when participants had no control over the avatar (W0), the SoA was higher for the Target task than for the Free and Trajectory tasks. While we hypothesized that the nature of the task could influence the perceived SoA, the tasks differed in two main aspects. The first difference relates to whether participants shared an intention toward the action to perform. In the Target and Trajectory tasks, the sphere to be touched was indicated, meaning that both participants shared the same intention of action: touching the same sphere. On the contrary, in the Free task, participants could have different spheres to touch in mind; this sometimes resulted in a difference between the intention, the sphere a participant wanted to touch, and the resulting action, the sphere finally touched by the shared avatar. According to Wegner et al. [20], SoA arises if (1) an intention precedes an observed action (priority), (2) the intention is compatible with this action (consistency), and (3) the intention is the most likely cause of this action (exclusivity). In the Target and Trajectory tasks, the three principles of priority, consistency, and exclusivity are more likely to be respected as participants share the same intention. Independently of their degree of control, the controller of the shared avatar will therefore reach the targeted sphere. This would support why SoA ratings where higher in the Target and Trajectory tasks when participants had no control over the shared avatar. The second difference was in the visual difference between participants and avatar hand positions (See Figure 8) depending on the task. Indeed, results showed for example that visuo-motor and visuo-proprioceptive discrepancies were lower in the Target task compared to Free when participants had no control. This can be because in the Target task, participants have the indication of which sphere to touch, resulting globally in the same movement toward the target sphere. Following the statements of Farrer et al. [14] that visuo-motor discrepancies tend to decrease the SoA, this could explain why the SoA was higher in Target than in the Free task where participants had no control at all. However, these results only partially support [H2].

Furthermore, a surprising result is that in the Target and Trajectory tasks, participants tended to overestimate their SoA, feeling some SoA despite the absence of control. From the analysis of speed profiles, we observed that major differences between control weights were found in the Free task, whereas only some differences were observed in the Target and Trajectory tasks, mostly at the end of trials. This seems to show that participants tended to have similar reaching behaviors regardless of their degree of control in the tasks where the goal was shared. Other authors also observed that the SoA was affected when the avatar’s and the participant’s speed of movement differed [34], but not with spatial shift of movement without speed alteration. These results could explain why participants tended to overestimate their SoA in the Target and Trajectory tasks, as we can see that even with no or very low control, participants still performed the task in a similar manner, therefore minimizing spatio-temporal discrepancies.

We also observed during the experiment that some participants reported a pure illusion of the control: “Sometimes, when the task was accomplished in an excellent manner, I wondered if it was actually me who had moved the arm ”. It is known how high-level contextual information (whether participants believe that the outcome is either triggered by themselves or by somebody else) can influence intentional binding [29], referring to the implicit measure of the conscious experience of SoA [31]. Depending on whether participants were more or less aware of their degree of control over the avatar may have affected their SoA. Furthermore, another feedback particularly illustrates potential future studies: “I had the impression that sometimes no one controlled my movement and that I was actually watching a video”. Indeed, sharing the control of the avatar with an autonomous virtual agent instead of another person would be an interesting topic to explore, in line with other studies which explored the influence of human and computer co-actors over the SoA in joint actions [57]. In particular, they showed that SoA for self-generated actions was inhibited when the participants knew that a computer was the co-actor of the action, which would be interesting to explore in the context of our co-embodiment setup.

6.2 SoA and Personality Traits

According to the results of the correlation analyses between the slope or intercept of FoC and the internal dimension of the locus of control (Figure 11), the intercept of the regression of FoC scores on the weight factor was negatively correlated with the Internal scores, especially when participants had little or no control over the virtual body, which does not validate H3. More precisely, participants with a high Internal score tend to have their feeling of agency be more impacted by changes in the level of control.

In previous studies, the Internal score was observed to be positively correlated with participants’ SoA when participants were immersed in a VE and embodied in their own virtual avatar over which they had full control [32]

. Our results do not support those findings, probably due to the ceiling effect we observed on SoA when participants had full control. However, we herein investigated the influence of the Locus of Control one-step further, exploring the influence of the Internal score on the SoA when participants did not have full control over their avatar. We found that participants with a high Internal score tend to have their SoA more impacted by changes in their degree of control of the avatar. People with a high Internal score are known to attribute the consequences to themselves rather than to chance or other more powerful entities and tend to believe that they have personal control over performance and rewards. However, such a definition does not commonly consider body movements. Given the little amount of previous work linking LoC and SoA, the results from such analyses should thus be treated with considerable caution. On the one hand, our results seem to suggest that people who tend to attribute consequences to themselves are possibly more aware of their own actions and thus notice more when they do not have control. On the contrary, people with a high Internal score might attribute events, movements included, to themselves and then attribute the movements of the avatar they did not cause to themselves. We would thus expect from participants to experience a high SoA even with no control over the shared avatar. While our results are in contradiction with this hypothesis, it would be in agreement with Desantis et al.’s study 

[29] wherein they showed that when participants believe that they have control over the environment, intentional binding, an implicit measure of the SoA, is stronger. However, in our analysis, we only tried to correlate the Internal score with FoC, an explicit measure of the SoA. As previous findings do not always agree on whether implicit and explicit measures of agency relate to the same thing [58], it would be interesting to also consider correlating implicit measures of the SoA with the Internal score. Therefore, our results on the influence of the Internal score of the Locus of Control over the SoA demonstrate the need for further investigation on the topic.

6.3 Sense of Embodiment

Results from the agency and ownership questionnaires in the first and last exposure phases showed that having only half the control of an avatar significantly decreased both SoBO and SoA compared to when they fully controlled an avatar (Figure 12 Left and Center). Such results are in line with numerous previous studies showing that asynchronous visual information with reference to participants’ own movements eliminates both SoBO [59, 60, 35] and SoA [15, 14]. In addition, our results showed that agency and ownership scores were positively correlated when each participant had half of the control of the avatar, whereas no correlation was found when they had full control over their own avatar (see Figure 12, right). As for the relationship between SoBO and SoA, some studies indicate that both experiences can partially double dissociate [60, 23, 61, 62] while some others suggest that they may strengthen each other if they co-occur [10, 63, 59, 64] (For review, see [65]). While we observed a ceiling effect of the agency scores when participants had full control, the positive correlation found in the half condition indicates a close relationship between SoA and SoBO.

Considering such positive correlations, the induction of the stronger SoBO over the virtual body can be considered to make SoA stronger and vice versa. Indeed, Kokkinara et al. [34] observed that illusory SoA occurred despite the distortion of movements being larger than the detection thresholds of discrepancies found in previous studies. They also remarked that their results might be due to the full-body ownership illusion. In our study, Figure 9 indicates that in the Free task, participants felt more than half control when the distance between participant’s and avatar’s controller positions were below 0.1 m on average. As SoBO is known to be affected by top–down factors such as the congruence of the structural and morphological features between one’s own and virtual bodies [9], making the features more congruent might therefore induce a stronger SoA. It is also considered to increase the detection threshold of visuo-motor discrepancies. In VR, some studies have exploited such visuo-motor discrepancies to enhance passive haptics or improve manipulations by changing the mapping of movements from the physical to the virtual space [66, 67, 68]. The interplay between SoBO and SoA is a subject of psychological interests, but seeking to reduce the detection threshold of visuo-motor discrepancies by strengthening SoBO might also be useful to VR applications.

In addition, there has been some evidence showing the dynamic relationship between self-attribution and sensorimotor systems. In Nielsen’s study [38], participants experienced the illusory SoA and attributed the experimenter’s hand in a mirror to their own while drawing a straight line. In particular, when the experimenter distorted their movement so that he/she drew a curved line, participants still attributed the movement to themselves and moved in the opposition to the experimenter’s movement to compensate for the error between the predicted and actual movements. This means that as long as they attributed a movement to themselves, they tried to control it. Asai [69] also reported that illusory self-attribution of fake movements might coordinate sensory input and motor output. Conversely, when participants became aware of the uncontrollability of the cursor, the illusory self-attribution was also dismissed. In our experiment, the degree of control was different for each trial. Therefore, participants could not fully adapt to it. However, in case of a constant degree of control, participants might feel a stronger SoA since visuo-motor adaptation might enable participants to predict the avatar’s movements.

6.4 SoA in Joint Action

We perform joint actions together with others in our daily lives, e.g., carrying heavy things, and admirably coordinate our plans and actions to achieve our joint goal. Indeed, in such cases, individuals build up a shared motor plan, incorporating others’ actions into their own motor system during a joint action [57]. In joint actions, there is therefore an automatic formation of a new agentic identity (a ’we’ identity) [57], and we feel the sense of us.

In the virtual co-embodiment situation where two individuals jointly control one avatar, as mutually coordinated actions of self and other produce the united movements, individuals might therefore also feel a sense of us. In our experiment, we found it particularly surprising that participants were able to immediately coordinate their actions to the joint goal even with the completely novel way of interacting and the lack of verbal communication.

Nevertheless, according to participants’ feedback, this collaborative behavior was not shared between all participants and some of them even tended to get the feeling of competing while performing the task: “I felt in competition especially for the free task”, “I sometimes felt in competition when we both had control and wanted to go on different spheres”. We also observed that the time to complete the task was higher when the control was equally shared between participants compared to when one participant had more control than the other in the Free task. Such differences could be caused by the adoption of “leader/follower” behaviours when one participant has more control that the other; however, further investigation would be necessary to explore such a hypothesis. Overall, research on virtual co-embodiment could therefore contribute to studies of joint action that investigate the mechanisms of how individuals coordinate their actions online, which is the essential capacity of humans as social beings.

6.5 Future Work

Despite the interesting insights gained from our experiment, we believe that there are still other aspects that would require further research.

First, our study focused on a particular virtual co-embodiment experience, namely two users sharing an avatar to accomplish simple tasks with different degrees of control. The results showed that users were able to perform the tasks and their SoA positively correlated with their degree of control. Additionally, previous knowledge of actions to be performed significantly increased their SoA. However, owing to the inter-relation of sharing the avatar and the actual degree of control, clearly quantifying the effects of each is difficult at this stage. Thus, the actual effect of being embodied in the same virtual avatar with someone else remains unclear. Does the mere fact of knowing that you share your avatar with someone else have an impact on the perception over the avatar? This is still an unanswered question that would require additional experiments, e.g., a virtual co-embodiment scenario in which a user shares the avatar with an autonomous agent.

Second, the proposed control scheme demonstrated that a partial degree of control can still elicit an SoA over a shared virtual body and that the motor actions performed in such a context resemble the ones performed with full control of the virtual body. Our implementation was meant to evaluate a novel concept, for which we tested one of the potential shared-control schemes. For example, as the shared control of the avatar head was particularly problematic, we decided that each user would keep full control of the avatar head as sharing its control might require unwanted changes at the user’s viewpoint. Such situations could lead the user to be prone to motion sickness. However, in situations where users are allowed to move freely around, a more complex scheme would therefore be required as the overall shared posture might be different than the users’ own posture. This would therefore require exploring more complex control schemes, techniques for switching control schemes depending on the situations and objectives, or even supporting more people embodied in the same avatar. Moreover, even at the level of controlling individual body parts, different control schemes can be considered. In our implementation, we averaged the positions and orientations of the controllers, but other methods could, for instance, explore splitting the control of different body parts or taking control depending on a certain movement threshold.

Third, virtual co-embodiment has a variety of potential applications such as remote training or entertainment. In a manner similar to our method, Yang and Kim’s “Just Follow Me” [6] method visualizes the motion of the trainer as a ghost, superimposed on the avatar of the trainee in the virtual environment. A similar method was also proposed in augmented and mixed realities for remote guidance and collaboration [70, 71]. In contrast, a system based on the principle of virtual co-embodiment could allow trainers to control a trainee’s movements to different degrees depending on the training needs and allow them to interact with each other through body movements while sharing the same experience. The results of our study showed that even when participants had no control over the avatar, they overestimated their FoC when the situation constrained the movements and indicated a shared goal. It suggests that in the training situation, the trainee could feel SoA over their body even when the body is fully controlled by the trainer. In addition, training could be made more effective by changing the degree of control depending on the learning phase, which in turn would require designing efficient and intuitive ways to adapt the degree of control to the situation. Moreover, it would be interesting to compare the cognitive load inferred by our system with the one felt in an approach similar to the “Just Follow Me” method [6], searching if one method is more susceptible to increase the cognitive load of the trainee while learning through an application. This will also open new opportunities to explore how mismatching the actual and announced degrees of control influences the user’s SoA, e.g., by telling both users that they have a 75% control even though they actually have 50% control each. Overall, considering new means of making users efficiently collaborate in future applications, e.g., through the use of verbal interactions, visual cues, and interaction design, will be important.

Lastly, as virtual co-embodiment is a merging experience with someone else, it has the possibility to produce cognitive effects on users. Indeed, shared bodily experiences such as the enfacement illusion (i.e., self-other face-perception modification by synchronous multisensory stimulation) [47, 48, 49] are known to produce both perceptual and social binding. A stranger stimulated in synchrony was judged as more similar, physically and in terms of personality, and as closer to the self [47, 50]. In addition, enfacement was positively correlated with the participant’s empathic traits and with the physical attractiveness that the participants attributed to their partners [49]. In this sense, co-embodiment could be used as a tool for psychological investigations of the “self”.

7 Conclusion

In this paper, we introduced the concept of “virtual co-embodiment”, a situation that enables a user and another entity (e.g., another user, robot, autonomous agent) to be embodied in the same virtual avatar. In addition, we described an experiment that examined the influence of the degree of control of an avatar shared with another person on one’s own SoA, as well as the influence of the predictability of avatar movements. Our results indicated that participants succeeded frequently in estimating their actual level of control over the shared avatar. Interestingly, they tended to overestimate their feeling of control when the visual feedback of the avatar’s movements was closer to their actual movements, as well as when they had prior knowledge of the action to be performed. In addition, our results showed that participants performed similar motions regardless of their level of control. Finally, our results reveal that the internal dimension of the locus of control is negatively correlated with the participants’ perceived FoC.

Taken together, these findings not only corroborate and extend previous studies, but they also pave the way for further applications in the field of VR-based training and collaborative tele-operation applications in which users would be able to share their virtual body.

Acknowledgments

We wish to thank the participants who took part in our experiment. This work was sponsored by the Region Bretagne, the Inria IPL Avatar project, and the JSPS Overseas Challenge Program for Young Researchers.

References

  • [1] K. Kilteni, R. Groten, and M. Slater, “The Sense of Embodiment in Virtual Reality,” Presence Teleoperators Virtual Environ., vol. 21, no. 4, pp. 373–387, 2012.
  • [2] L. Hoyet, F. Argelaguet, C. Nicole, and A. Lécuyer, ““Wow! I Have Six Fingers!”: Would You Accept Structural Changes of Your Hand in VR?” Front. Robot. AI, vol. 3, no. May, pp. 1–12, 2016.
  • [3] T. C. Peck, M. Doan, K. A. Bourne, and J. J. Good, “The effect of gender body-swap illusions on working memory and stereotype threat,” IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 4, pp. 1604–1612, 2018.
  • [4] M. E. Latoschik, D. Roth, D. Gall, J. Achenbach, T. Waltemate, and M. Botsch, “The effect of avatar realism in immersive social virtual realities,” in Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, 2017, pp. 39:1–39:10.
  • [5] Y. Pan and A. Steed, “The impact of self-avatars on trust and collaboration in shared virtual environments,” PLOS ONE, vol. 12, no. 12, p. e0189078, 2017.
  • [6] U. Yang and G. J. Kim, “Implementation and evaluation of ”just follow me”: An immersive, VR-based, motion-training system,” Presence Teleoperators Virtual Environ., vol. 11, no. 3, pp. 304–323, 2002.
  • [7] H. Kawasaki, H. Iizuka, S. Okamoto, H. Ando, and T. Maeda, “Collaboration and skill transmission by first-person perspective view sharing system,” in Proc. - IEEE Int. Work. Robot Hum. Interact. Commun., 2010, pp. 125–131.
  • [8]

    M. Luria, S. Reig, X. Z. Tan, A. Steinfeld, J. Forlizzi, and J. Zimmerman, “Re-embodiment and co-embodiment: Exploration of social presence for robots and conversational agents,” in

    Proceedings of the 2019 on Designing Interactive Systems Conference.   ACM, 2019, pp. 633–644.
  • [9] K. Kilteni, A. Maselli, K. Kording, and M. Slater, “Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception,” Front. Hum. Neur., vol. 9, 2015.
  • [10] M. R. Longo, F. Schüür, M. P. Kammers, M. Tsakiris, and P. Haggard, “What is embodiment? A psychometric approach,” Cognition, vol. 107, no. 3, pp. 978–998, 2008.
  • [11] D. M. Wegner, B. Sparrow, and L. Winerman, “Vicarious agency: Experiencing control over the movements of others,” J. Pers. Soc. Psychol., vol. 86, no. 6, pp. 838–848, 2004.
  • [12] G. Gomez, C. Plasson, F. Elisei, F. Noël, and G. Bailly, “Qualitative assessment of an immersive teleoperation environment for collaborative professional activities in a ”beaming” experiment,” in European conference for Virtual Reality and Augmented Reality, 2015.
  • [13] M. V. Sanchez-Vives, B. Spanlang, A. Frisoli, M. Bergamasco, and M. Slater, “Virtual hand illusion induced by visuomotor correlations,” PLoS One, vol. 5, no. 4, 2010.
  • [14] C. Farrer, M. Bouchereau, M. Jeannerod, and N. Franck, “Effect of distorted visual feedback on the sense of agency,” Behav Neurol, vol. 19, no. 1-2, pp. 53–57, 2008, 18413918[pmid]. [Online]. Available: https://www.ncbi.nlm.nih.gov/pubmed/18413918
  • [15] N. Franck, C. Farrer, N. Georgieff, M. Marie-Cardine, J. Daléry, T. D’Amato, and M. Jeannerod, “Defective recognition of one’s own actions in patients with schizophrenia,” Am. J. Psychiatry, vol. 158, pp. 454–459, 2001.
  • [16] A. Maselli and M. Slater, “The building blocks of the full body ownership illusion,” Front. Hum. Neurosci., vol. 7, 2013.
  • [17] E. Kokkinara, K. Kilteni, K. J. Blom, and M. Slater, “First person perspective of seated participants over a walking virtual body leads to illusory agency over the walking,” Scientific Reports, vol. 6, 2016.
  • [18] S. Ghallager, “Philosophical conceptions of the self: implications for cognitive science,” Trends Cogn. Sci., vol. 4, no. 1, 2000.
  • [19] C. D. Frith, S.-J. Blakemore, and D. M. Wolpert, “Abnormalities in the awareness and control of action,” Philosophical Transactions of the Royal Society of London B: Biological Sciences, vol. 355, no. 1404, pp. 1771–1788, 2000.
  • [20] D. M. Wegner and T. Wheatley, “Apparent mental causation: Sources of the experience of will,” Am. Psychol., vol. 54, no. 7, pp. 480–492, 1999.
  • [21] S. J. Blakemore, C. D. Frith, and D. M. Wolpert, “Spatio-temporal prediction modulates the perception of self-produced stimuli.” J. Cogn. Neurosci., vol. 11, no. 5, pp. 551–559, 1999.
  • [22] P. Haggard and V. Chambon, “Sense of agency,” Current Biology, vol. 22, no. 10, pp. R390 – R392, 2012.
  • [23] A. Sato and A. Yasuda, “Illusion of sense of self-agency: Discrepancy between the predicted and actual sensory consequences of actions modulates the sense of self-agency, but not the sense of self-ownership,” Cognition, vol. 94, no. 3, pp. 241–255, 2005.
  • [24] J. W. Moore, D. M. Wegner, and P. Haggard, “Modulating the sense of agency with external cues,” Conscious. Cogn., vol. 18, no. 4, pp. 1056–1064, 2009.
  • [25] D. Wenke, S. M. Fleming, and P. Haggard, “Subliminal priming of actions influences sense of control over effects of action,” Cognition, vol. 115, no. 1, pp. 26–38, 2010.
  • [26] K. Linser and T. Goschke, “Unconscious modulation of the conscious experience of voluntary control,” Cognition, vol. 104, no. 3, pp. 459–475, 2007.
  • [27] D. M. Wegner, “Precis of ”The Illusion of Conscious Will”,” Behav. Brain Sci., vol. 27, pp. 649–692, 2004.
  • [28] J. W. Moore, “What is the sense of agency and why does it matter?” p. 1272, 2016.
  • [29] A. Desantis, C. Roussel, and F. Waszak, “On the influence of causal beliefs on the feeling of agency,” Conscious. Cogn., vol. 20, no. 4, pp. 1211–1220, 2011.
  • [30] P. Haggard, S. Clark, and J. Kalogeras, “Voluntary action and conscious awareness,” Nat. Neurosci., vol. 5(4), pp. 382–385, 2002.
  • [31] J. W. Moore and S. S. Obhi, “Intentional binding and the sense of agency: A review,” Conscious. Cogn., vol. 21(1), pp. 546–561, 2012.
  • [32] C. Jeunet, L. Albert, F. Argelaguet, and A. Lécuyer, “” Do you feel in control?”: Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments,” IEEE Trans. Vis. Comput. Graph., pp. 1–10, 2018.
  • [33] E. Daprati, N. Franck, N. Georgieff, J. Proust, E. Pacherie, J. Dalery, and M. Jeannerod, “Looking for the agent: An investigation into consciousness of action and self-consciousness in schizophrenic patients,” Cognition, vol. 65, no. 1, pp. 71–86, 1997.
  • [34] E. Kokkinara, M. Slater, and J. López-Moliner, “The Effects of Visuomotor Calibration to the Perceived Space and Body, through Embodiment in Immersive Virtual Reality,” ACM Trans. Appl. Percept., vol. 13, no. 1, pp. 1–22, 2015.
  • [35] K. Ma and B. Hommel, “The role of agency for perceived ownership in the virtual hand illusion,” Conscious. Cogn., vol. 36, pp. 277–288, 2015.
  • [36] G. Tieri, E. Tidoni, E. F. Pavone, and S. M. Aglioti, “Mere observation of body discontinuity affects perceived ownership and vicarious agency over a virtual hand,” Exp. Brain Res., vol. 233, no. 4, pp. 1247–1259, 2015.
  • [37] J. Metcalfe and M. J. Greene, “Metacognition of agency,” J. Exp. Psychol. Gen., vol. 136, no. 2, pp. 184–199, 2007.
  • [38] T. I. Nielsen, “Volition: a new experimental approach,” Scand. J. Psychol., vol. 4, no. 1, pp. 225–230, 1963.
  • [39] F. De Vignemont and P. Fourneret, “The sense of agency: A philosophical and empirical review of the ”Who” system,” Conscious. Cogn., vol. 13, no. 1, pp. 1–19, 2004.
  • [40] T. Asai and Y. Tanno, “Highly schizotypal students have a weaker sense of self-agency,” Psychiatry Clin. Neurosci., vol. 62, no. 1, pp. 115–119, 2008.
  • [41] H. Levenson, “Differentiating among internality, powerful others, and chance,” in Res. with Locus Control Constr., 1981, pp. 15–63.
  • [42] L. Hanna, “Activism and powerful others: Distinctions within the concept of internal-external control,” Journal of Personality Assessment, vol. 38, no. 4, pp. 377–383, 1974.
  • [43] T. Yokosaka, H. Iizuka, T. Yonemura, D. Kondo, H. Ando, and T. Maeda, “Alternating images of congruent and incongruent movement creates the illusion of agency,” Scientific Reports, vol. 4, pp. 6201 EP –, 2014.
  • [44] V. I. Petkova and H. H. Ehrsson, “If I were you: Perceptual illusion of body swapping,” PLoS One, vol. 3, no. 12, 2008.
  • [45] S. Kasahara, M. Ando, K. Suganuma, and J. Rekimoto, “Parallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing,” Proc. 2016 CHI Conf. Hum. Factors Comput. Syst. - CHI ’16, pp. 1561–1572, 2016.
  • [46] J. Nishida and K. Suzuki, “bioSync: A Paired Wearable Device for Blending Kinesthetic Experience,” in Proc. 2017 CHI Conf. Hum. Factors Comput. Syst. - CHI ’17, 2017, pp. 3316–3327.
  • [47] M. Mazzurega, F. Pavani, M. P. Paladino, and T. W. Schubert, “Self-other bodily merging in the context of synchronous but arbitrary-related multisensory inputs,” Exp. Brain Res., vol. 213, no. 2-3, pp. 213–221, 2011.
  • [48]

    M. Tsakiris, “Looking for Myself: Current Multisensory Input Alters Self-Face Recognition,”

    PLoS One, vol. 3(12), p. e4040, 2008.
  • [49] A. Sforza, I. Bufalari, P. Haggard, and S. M. Aglioti, “My face in yours: Visuo-tactile facial stimulation influences sense of identity,” Soc. Neurosci., vol. 5, no. 2, pp. 148–162, 2010.
  • [50] A. Tajadura-Jiménez, S. Grehl, and M. Tsakiris, “The other in me: Interpersonal multisensory stimulation changes the mental representation of the self,” PLoS One, vol. 7, no. 7, 2012.
  • [51] R. Poelman, O. Akman, S. Lukosch, and P. Jonker, “As if being there: mediated reality for crime scene investigation,” in Proc. ACM Conf. Comput. Support. Coop. Work - CSCW ’12, 2012, p. 1267.
  • [52] Z. Barlas, W. E. Hockley, and S. S. Obhi, “The effects of freedom of choice in action selection on perceived mental effort and the sense of agency,” Acta Psychol. (Amst)., vol. 180, pp. 122–129, 2017.
  • [53] M. Botvinick and J. Cohen, “Rubber hands ’feel’ touch that eyes see,” Nature, vol. 391, no. 6669, pp. 756–756, 1998.
  • [54] A. Kalckert and H. H. Ehrsson, “The spatial distance rule in the moving and classical rubber hand illusions,” Consciousness and Cognition, vol. 30, pp. 118 – 132, 2014.
  • [55] J. O. Wobbrock, L. Findlater, D. Gergle, and J. J. Higgins, “The aligned rank transform for nonparametric factorial analyses using only anova procedures,” in Proc. 2011 Annu. Conf. Hum. factors Comput. Syst. - CHI ’11, 2011, p. 143.
  • [56] C. Farrer, N. Franck, J. Paillard, and M. Jeannerod, “The role of proprioception in action recognition,” Consciousness and Cognition, vol. 12, no. 4, pp. 609 – 619, 2003.
  • [57] S. S. Obhi and P. Hall, “Sense of agency and intentional binding in joint action,” Exp. Brain Res., vol. 211, no. 3-4, pp. 655–662, 2011.
  • [58] J. A. Dewey and G. Knoblich, “Do implicit and explicit measures of the sense of agency measure the same thing?” PLOS ONE, vol. 9, no. 10, p. e110118, 2014.
  • [59] D. Banakou and M. Slater, “Body ownership causes illusory self-attribution of speaking and influences subsequent real speaking,” Proc. Natl. Acad. Sci., vol. 111, no. 49, pp. 17 678–17 683, 2014.
  • [60] A. Kalckert and H. H. Ehrsson, “Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency.” Front. Hum. Neurosci., vol. 6, p. 40, 2012.
  • [61] N. Braun, J. D. Thorne, H. Hildebrandt, and S. Debener, “Interplay of Agency and Ownership: The Intentional Binding and Rubber Hand Illusion Paradigm Combined,” PLoS One, vol. 9, no. 11, p. e111967, 2014.
  • [62] M. Tsakiris, M. R. Longo, and P. Haggard, “Having a body versus moving your body: Neural signatures of agency and body-ownership,” Neuropsychologia, vol. 48, no. 9, pp. 2740–2749, 2010.
  • [63] M. Tsakiris, G. Prabhu, and P. Haggard, “Having a body versus moving your body: How agency structures body-ownership,” Conscious. Cogn., vol. 15, no. 2, pp. 423–432, 2006.
  • [64] T. Dummer, A. Picot-Annand, T. Neal, and C. Moore, “Movement and the rubber hand illusion,” Perception, vol. 38(2), pp. 271–280, 2009.
  • [65] N. Braun, S. Debener, N. Spychala, E. Bongartz, P. Sörös, H. H. O. Müller, and A. Philipsen, “The senses of agency and ownership: A review,” Frontiers in Psychology, vol. 9, p. 535, 2018.
  • [66] A. Lecuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet, “Pseudo-haptic feedback: can isometric input devices simulate force feedback?” in Proc. IEEE Virtual Real., 2000, pp. 83–90.
  • [67] M. Azmandian, M. Hancock, H. Benko, E. Ofek, and A. D. Wilson, “Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences,” in Proc. 2016 CHI Conf. Hum. Factors Comput. Syst. - CHI ’16, 2016, pp. 1968–1979.
  • [68] L. Kohli, M. C. Whitton, and F. P. Brooks, “Redirected touching: The effect of warping space on task performance,” IEEE Symp. 3D User Interfaces 2012, 3DUI 2012 - Proc., pp. 105–112, 2012.
  • [69] T. Asai, “Feedback control of one’s own action: Self-other sensory attribution in motor control,” Conscious. Cogn., vol. 38, 2015.
  • [70] M. L. Chenechal, T. Duval, V. Gouranton, J. Royan, and B. Arnaldi, “Vishnu: virtual immersive support for helping users an interaction paradigm for collaborative remote guiding in mixed reality,” in 2016 IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE), 2016, pp. 9–12.
  • [71] W. Huang, L. Alem, and F. Tecchia, “HandsIn3D: Supporting Remote Guidance with Immersive Virtual Environments,” in Human-Computer Interaction – INTERACT 2013, 2013, pp. 70–77.