betaFeedback

WorksComputer Science

Remote Expert Evaluation of Tabletop Avatar Gaze Control in Asymmetric Multi-Scale MR–VR Collaboration

Abstract

This paper proposes an integrated setup for a multi-scale asymmetric collaboration scenario in the context of a remote-expert use case, focusing on the expert side. In this system, the remote expert (with an exocentric view in Mixed Reality (MR)) uses a Tangible User Interface (TUI) to precisely position and rotate their avatar within a scaled tabletop World-in-Miniature (WIM). Inside the WIM, the expert's avatar and the operator share the same scale and egocentric view, while the expert retains a global tabletop overview of the plant to support collaboration. Interviews with industry experts during the participatory design phase revealed the need for an egocentric view inside the plant on the remote-expert side to enable full collaboration. We designed and studied three interaction-visualisation combinations to integrate and control the expert avatar's gaze: VR (fully immersive with direct head control), Head (floating screen with direct head control), and Touch (on-avatar screen with touchpad control). We evaluated these methods with 29 participants across two fault-detection tasks: Same Machine, where both collaborators inspect the same machine, and Twin Machine, where they inspect identical machines in separate locations. The VR condition, which combines immersion, direct head control, and a full-scale view, yielded the strongest overall performance and user experience, while the tabletop avatar setup was well received for collaboration. Our findings inform the design of gaze-control techniques for asymmetric MR-VR collaboration and reveal challenges and opportunities for future work.

CCS Concepts

• Human-centered computing → Empirical studies in interaction design; Mixed / augmented reality.

This work is licensed under a Creative Commons Attribution 4.0 International License.

VRST ’26, Sendai, Japan

© 2026 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-2811-2/2026/11

https://doi.org/10.1145/3822517.3848661

ACM Reference Format: Carlos Mosquera, Neven Elsayed, Ernst Kruijff, Joseph Newman, Valeriya Lehrbaum, and Eduardo Veas. 2026. Remote Expert Evaluation of Tabletop Avatar Gaze Control in Asymmetric Multi-Scale MR–VR Collaboration. In 32nd ACM Symposium on Virtual Reality Software and Technology (VRST ’26), November 16–18, 2026, Sendai, Japan. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3822517.3848661

Introduction

Collaboration in Extended Reality (XR) is moving towards scenarios where individuals with heterogeneous hardware and disparate roles work together across the reality-virtuality continuum, giving rise to asymmetric setups [2, 12]. This is made increasingly convenient by consumer XR video see-through Head Mounted Displays (HMDs) [1, 20, 31], which let each collaborator move between fully immersive Virtual Reality (VR) and Mixed Reality (MR) or Augmented Reality (AR) and adopt the mode best suited to their role. This is particularly relevant in Remote Expert scenarios, the most common form of asymmetric collaboration, which typically pair a local operator with a remote knowledgeable person [45]. Such support can also span differences in scale, as in multiscale setups like the Giant-Ant [5, 17], where a Giant (Director) with an AR exocentric view guides an Ant (Actor) with a VR egocentric view through complex environments such as a navigational maze [17] or a co-manipulation task [5]. A common enabling metaphor is the World-in-Miniature (WIM), a 3D replica that functions as both map and input space [8]. A tabletop WIM lets the expert retain an exocentric plant overview while the immersed operator works at 1:1 scale [2, 5]. These asymmetric viewpoints, however, create the viewpoint gap [39]: workspace awareness degrades as perspectives diverge [2], and conveying the expert point of attention within the operator’s immersive view remains a key design challenge [45].

Through interviews with industry experts (see § 3.1), we found that remote experts need an in-plant egocentric view in parallel with the exocentric tabletop overview to collaborate effectively. Yet, there is no established way to integrate and control this expert-centric egocentric view. In this work, we systematically explore this open design space, focusing on the expert side. We present an asymmetric MR–VR setup for remote industrial fault detection, integrating existing techniques—tabletop WIM, TUI control, and workspace awareness (WA) cues—into a coherent system for empirical evaluation. We adopt TUIs because they ground avatar manipulation on the physical tabletop, providing the stability and spatial precision that mid-air gestures lack [32], while co-locating control with the miniature plant the expert is inspecting. Our system empowers a Giant expert, using an MR-HMD, to manipulate their mini avatar within a tabletop WIM via a tangible prop. By physically controlling the avatar’s position and rotation, the expert establishes spatial presence in the operator’s (VR user) workspace. To enhance situational awareness, the expert can observe their avatar’s egocentric view through a floating virtual screen or by transitioning into a full egocentric perspective. How best to visualize and control this view remains an open question.

To answer that, we evaluated three interaction–visualization combinations for controlling the avatar’s gaze in an industrial fault detection context. In a user study with 29 participants acting as remote experts, we measured performance and experience across

two collaborative tasks: (1) Same Machine, where both collaborators work on the same machine, and (2) Twin Machine, where they inspect identical machines in different locations. Our goal was to determine how MR point-of-view and gaze-control techniques affect performance, user experience, and collaboration in asymmetric MR–VR remote-expert work, and to derive design implications.

Our contributions are: (1) An asymmetric MR–VR remote-expert setup using a tabletop WIM and tangible avatar control in a multiscale configuration. (2) Three interaction-visualization techniques (VR, Head, Touch) for controlling and presenting the expert avatar’s gaze. (3) A user study with 29 participants across two fault-detection tasks, yielding design implications for asymmetric MR–VR remote-expert collaboration.

Related Work

This research builds on prior work in asymmetric MR collaboration, WA, tabletop TUIs, and WIM. We review these areas to ground our mixed-space scenario, where a tabletop MR expert uses a tangible prop to control their avatar position, rotation and gaze.

Asymmetric Collaboration in Mixed Reality

In recent years, research in Collaborative Virtual Environments (CVEs) has shifted from symmetric interaction toward asymmetric viewpoints, scales, and interfaces to support distinct user roles. A major milestone in this direction was the transitional interface enabling seamless switching between Shared AR and Shared VR Environments for co-located collaboration [26]. In parallel, the MagicBook allowed users to move along the reality–virtuality continuum, inspecting an exocentric AR scene on a physical book or “flying inside” for an egocentric VR view [4]. Together, these systems demonstrated how mixed AR/VR collaborators can maintain mutual awareness through visual metaphors — AR users seeing VR partners as miniature avatars, and VR users perceiving AR partners as large virtual heads above the scene.

One of the earliest rigorous evaluations of mixed-space collaboration showed that pairing an AR exocentric Director with a VR egocentric Actor improved performance over individual work and was comparable to VR-VR collaboration [17]. The study also highlighted that misalignment between control and visual spaces harms performance by increasing the mental effort of coordinate transformations [17]. This exocentric–egocentric paradigm was later extended into the Giant–Ant configuration for concurrent object manipulation, where the Giant provides coarse global translations and the Ant performs fine in-situ adjustments [5].

Another exocentric-egocentric scenario is the god-like interaction metaphor, which links indoor tabletop interfaces with outdoor AR users [47]. In this setup, an expert maintains a global overview and guides the immersed user via a giant hand that appears to descend from the sky [47]. This concept was later adapted for architectural design in Dollhouse VR, which introduced a "see-through ceiling" that projects the tabletop user’s face into the virtual space to support social awareness and eye contact [22, 49].

Recent systems leverage AR-VR role asymmetries and tangible interaction to support guidance across different scales [41, 42, 43]. In GMC, an AR Giant can physically steer a VR Miniature’s viewpoint via a TUI [42]. Snow Dome adds recursive scaling while preserving a shared reference frame [40]. These asymmetric setups make it difficult to understand a partner’s focus and perspective across radically different viewpoints (viewpoint gap)[2, 12]. To bridge this gap, researchers have focused on workspace awareness cues.

Workspace Awareness (WA)

WA is a real-time understanding of how others are interacting with a shared workspace—who is present, where they are working, and what they are doing [18, 19]. While it often emerges naturally in co-located work through visible actions and their effects, digital and distributed settings reduce perceptual access and weaken nonverbal cues, which can undermine WA [18, 19]. In 3D collaborative virtual environments, systems therefore add explicit awareness cues (e.g., gaze rays and view cones) to convey attention and field of view (FOV) and to mitigate the XR viewpoint gap [2, 10, 19].

In MR collaboration, HMDs reduce eye contact and facial-expression cues, weakening social and WA [3, 18, 19]. To compensate, designers add explicit awareness cues and metaphors—from global orientation aids to local, over-the-shoulder views—to communicate where a partner is looking and what they can see [44]. In asymmetric MR–VR setups, 3D pointing metaphors (e.g., 2D pointer/3D ray) and viewpoint cues such as FoV frustums and head-gaze rays help bridge the XR viewpoint gap and improve coordination [2, 9] [39].

Tabletop Tangible User Interfaces

TUIs couple digital information to physical objects and environments, leveraging physical affordances to make interaction more understandable [14, 24]. A central benefit is graspable object control, where users manipulate virtual content through physical bricks that provide direct, embodied control [15, 24]. In tabletop settings, graspable tangibles support spatial tasks for maps/landmarks [24] and for physically grounded object manipulation [25]. Recent work further indicates that tabletop-grounded tangibles (e.g., knob-shaped proxies) can outperform mid-air gestures by exploiting the table for stability and ergonomic support [32].

In collaborative settings, tabletop interfaces support shared discussion and workspace awareness [27, 37]. In asymmetric collaboration, actuated tokens and independent viewpoints can improve co-presence and task performance by making remote intent and activity visible on the shared table [28, 37]. HoloBots [23] extends this approach by coupling holographic telepresence with gaze-driven tabletop robots as physical surrogates, enabling miniature-body interaction and supporting intent communication.

World in Miniature (WIM)

Although our system uses a real tabletop MR setup, the WIM metaphor remains central [38, 48]. WIMs present an environment as a manipulable miniature that supports overview, navigation, and at-a-distance manipulation [7, 8]. Early work showed that tangible props (e.g., clipboards) can anchor perception and ease transitions between miniature and life-size views [8, 48]. Later systems improve WIM travel with pre-oriented teleportation and viewpoint previews, reducing mental rotation and disorientation [11, 56].

Prior work shows the value of complementing exocentric and egocentric viewpoints in asymmetric mixed-space collaboration, yet the viewpoint gap still limits WA. Recent work highlights visual WA cues (e.g., FoV frustums and head-gaze rays) as key for coordinating attention in MR. However, TUIs for direct avatar control in tabletop WIMs remain underexplored, and their combination with WA cues to jointly control avatar position and gaze remains unevaluated. We address this gap by evaluating such a scenario.

System Design

We use the industrial remote-expert scenario as our guiding use case; findings may generalize to other collaborative contexts.

Industrial Use Case

We focus on the remote expert use case, which is among the most popular scenarios for XR collaboration [12]. In this scenario, a knowledgeable person guides a local operator through a physical task, such as industrial fault detection. This setup supports asymmetric roles: the expert provides guidance and oversight, while the operator performs hands-on work in the real environment.

To gain deeper insights into the remote expert fault detection context, through a participatory design process we conducted unstructured interviews with three XR industry experts. From these discussions, we derived several requirements for the expert role. First, the expert needs to maintain a global overview of the entire plant to effectively coordinate with an operator focused on a specific area or machine. Second, for close inspection, the expert needs to see from the same perspective as the operator in order to understand their viewpoint and local context. Third, real-world scenarios may involve multiple experts and operators working together, requiring support for multi-party collaboration.

Based on the analysis of the fault detection workflow, we modeled the task as comprising two main phases: an initial navigation and wayfinding phase, in which the expert guides the operator from their current location to the target machine, followed by a joint inspection and repair phase, in which both collaborators closely inspect the machine, identify the fault, and coordinate the repair action. Other domains may benefit similarly from this asymmetric MR–VR collaboration setup, such as plant design, where on-site operators can be directly involved in layout and process discussions, or emergency response scenarios, where a remote expert can guide and teleoperate a robot deployed in the affected area.

Tabletop Avatar Design

A key component is a tabletop with an augmented, scaled WIM plant in MR. This gives the expert a persistent global overview, addressing the first requirement. We chose a tabletop specifically because it provides a stable physical surface for TUI-based avatar control [32], spatially anchors the WIM so that positions on the miniature map directly to plant locations, and lets multiple co-located experts gather around a shared reference for face-to-face coordination [6, 33]. Prior work has used tabletop WIMs for asymmetric collaboration [22, 47].

Another requirement is that the expert can see from the operator’s perspective to correctly interpret the local situation and provide grounded instructions. Prior work on collaborative XR has extensively analyzed such egocentric, view-aligned configurations—such as shared first-person views and overlaid annotations—where both operator and expert inhabit the same scale and perspective in a virtual or augmented scene [27, 36, 54]. In our case, however, this egocentric requirement is in tension with an exocentric tabletop overview of the plant, which is valuable for planning, navigation, and maintaining workspace awareness [2, 45].

Several systems have explored transitions between exocentric and egocentric perspectives in XR, allowing users to move between an overview miniature and an immersive first-person view [4, 26, 30]. While these transitions can support spatial understanding, they expose a drawback for our scenario: when the expert is fully immersed in an egocentric view, they temporarily lose direct access to the global tabletop representation needed for coordination and navigation across distant parts of the plant. At the same time, the task’s navigation phase requires the expert to direct the operator to new locations, where prior work has relied on exocentric god hand metaphors or tangible props to indicate paths and targets from above the scene [23, 28, 50].

We replace the disembodied god hand with an embodied expert avatar in the tabletop WIM. The avatar is a controllable proxy: the expert positions it anywhere in the WIM to indicate locations and paths while retaining the exocentric overview. When needed, the expert triggers a transition to the avatar’s viewpoint, entering the scene through their miniature self. This has two advantages. First, the expert controls where they transition. Second, in egocentric view both users are co-located at equal scale, supporting mutual understanding and first-person guidance [16, 35]. The avatar thus mediates between overview and immersion.

Tangible Prop Design

We control the expert avatar with a tangible prop, as TUIs provide rich physical affordances and precise control, especially on a tabletop surface, where they improve ergonomics and precision for virtual object manipulation [32]. Following design guidelines for compact, hand-scale tabletop tangibles, we gave the prop a rounded, compact form factor with a cylindrical shape that enhances controllability and fine-grained adjustments through its graspable form and continuous rotational affordances [52], letting the expert simultaneously control their avatar’s position and orientation.

For tracking, we embedded a Sony Toio [46] swarm robot, providing compact 2D tabletop tracking [34]. To extend interaction beyond pure manipulation, we integrated a rounded touchscreen driven by an ESP32-based board [55] and a portable battery, enabling self-contained visual feedback and touch input. Side handles improve grip while keeping the touchscreen unobstructed; on-device buttons offer additional input channels. The resulting prop weighs 56g, with a diameter of 45mm and height of 39mm, as shown in Figure 1(d).

Asymmetric MR–VR Collaboration

Figure 1. Asymmetric MR–VR Collaboration (remote-expert use case): (a) Operator using Desktop Virtual Reality (VR). (b) Operator’s egocentric VR view. (c) Remote expert using Mixed Reality (MR) interacting with their avatar in the World-in-Miniature (WIM) plant. (d) Tangible prop with integrated touchpad for avatar control. (e) Remote expert’s exocentric view.

Workspace Awareness Components

Following recent WA surveys [2], we designed the system to support the main WA questions (who, where, and where-looking) through visual cues. WA informs the system design; the study evaluates how the expert visualizes and controls these cues, not WA itself.

For Who (presence/identity), each user is represented by a full-body avatar with a distinctive color and a name label rendered above the head. This makes it easy to identify collaborators at a glance and attribute actions to specific users.

For Where (location), both expert and operator can directly control their avatars position in the shared MR workspace, keeping movement and spatial arrangements visible to all collaborators.

For Where (gaze/view), rather than a full frustum, we used a head-gaze indicator, in line with findings that subtle gaze cues are less intrusive yet effective for WA [44, 54]. We cast a ray from the avatar’s head along its viewing direction, rendering a thin line that ends in a small hit sphere where it meets the environment, continuously indicating each collaborator’s focus of attention (Figure 1b).

Apparatus And Collaboration Server

The expert used the Quest 3 MR headset, a standalone device with full-color passthrough, operated via two 6-DoF controllers or hand tracking [31]. The operator used a desktop PC. For networking, we used a cloud-hosted WebSocket server to synchronize avatars and head poses in real time, inspired by existing frameworks [51].

The system combines a tabletop MR setup, a tangible prop for avatar control, and explicit WA cues including a head-gaze ray, making it crucial that the expert can perceive their avatar’s view and control its gaze. Finding no prior work on combined view-and-control techniques for a miniature avatar, we conducted a pilot to inform the technique design.

Interaction Design Pilot Phase

Effectively using the head-gaze cue depends on two aspects: how the avatar’s view is visualized and how the expert controls it. Three industry experts (§ 3.1) participated in this pilot to evaluate alternatives and finalize the methods for evaluation.

Viewpoint Representation Alternatives

Prior work has provided egocentric views of tabletop scenes by transitioning from an exocentric tabletop view to full VR [4, 30].

While intuitive, letting users experience the scene from within-it has drawbacks in our case: switching to VR removes the exocentric plant overview, can reduce spatial orientation [13], and raises cognitive load from the mode switch and context change [17]. To address this, we draw inspiration from WIM research, where preview windows show the target viewpoint before teleportation to reduce disorientation [11, 21]. We adapt this to our tabletop setting: instead of transitioning the expert fully into VR, a dedicated display shows what the miniature avatar sees, letting the expert inspect and adjust its viewpoint while preserving the exocentric overview.

The size, shape, and placement of such preview windows form a coupled design space rather than independent factors. In our pilot, we therefore explored two holistic alternatives. First, a small screen on the tangible prop, compact to limit occlusion [11]. Second, a larger preview around the tabletop that dynamically follows the HMD [7, 21]. We iteratively refined both with domain experts.

Based on their feedback, we converged on three final visualizations. The fully VR view needed little adjustment, presenting a 1:1 egocentric mapping of the operator’s perspective. The medium preview is a floating rectangular screen (1.0×0.5 m1.0 \times0.5\ \mathrm{m}), positioned on the opposite side of the tabletop to stay visible as the expert moves. The compact preview is a small circular screen (30 cm30\ \mathrm{cm} diameter) mounted on the tangible prop that tilts to face the expert, improving visibility while minimizing occlusion.

Control Alternatives

This dimension concerns how the expert controls the avatar’s gaze. Given the visualization alternatives above, full VR lends itself naturally to direct head mapping, where the expert’s head movements directly define the avatar’s viewing direction.

For preview windows, beyond direct head mapping, we explored indirect head control, where the avatar’s gaze depends on the head’s pose relative to the screen—like a portal—so moving around it shifts the viewpoint, letting the expert “peek” into the WIM without fully entering VR [7, 11, 21]. In addition, we investigated external control via input on the tangible prop itself, such as a touchpad on top, keeping control co-located with the avatar proxy and consistent with our tangible design. We excluded mid-air gestures and detached devices, as they cannot be cleanly integrated into the prop and conflict with our goal of a compact, self-contained TUI.

In expert evaluation, indirect head control, though initially promising, caused confusion as its mapping felt unnatural, while the prop-mounted touchpad was valued for keeping the device compact. We thus narrowed the design to two control methods: direct head and touchpad control. These two control methods were then paired with the three visualizations to form the final conditions (§ 4.3).

Final Interaction Design

Based on expert feedback, we paired each visualization with a gaze control method. The fully VR visualization used its natural direct head control (VR condition). The floating rectangular preview used direct head control, preferred over touch here (Head condition). The small circular on-top screen used touchpad control, as experts found a preview spatially and functionally coupled with the touch surface easier to focus on (Touch condition).

All conditions shared the same initial sequence: the expert positioned and oriented the avatar on the WIM plant, then pressed and held the prop against the tabletop via the handles to enable gaze control. This maintains continuous contact with the prop: this is vital in VR, where the user cannot see the tabletop or prop and would otherwise struggle to relocate it. Once gaze control is enabled, each condition proceeds differently:

4.3.1 VR Condition. After pressing the button, the expert transitions into fully immersive VR at the avatar’s current position, controlling its gaze naturally through head movements for a first-person view into the plant. A hitsphere at the head-gaze ray’s endpoint marks the expert’s focus, enabling precise control and clear communication to the operator. Figure 2a illustrates this condition.

Interaction methods and tasks

Figure 2. Interaction Methods: (a) VR: Fully immersive, direct head control, (b) Head: Floating screen, direct head control, (c) Touch: Small on-top avatar screen, touch control. Tasks: (d) Task 1 (T1): Same Machine, (e) Task 2 (T2): Twin Machine.

4.3.2 Head Condition. After pressing the button, the expert remains in the MR tabletop environment, seeing the avatar’s egocentric view on the large rectangular preview screen, with its gaze controlled directly by head movements. A hitsphere at the head-gaze ray’s endpoint indicates where he is looking, communicating his focus to the operator while still retaining the exocentric tabletop overview. Figure 2b illustrates this condition.

4.3.3 Touch Condition. As in Head, the expert stays in the MR tabletop environment, but the avatar’s egocentric view now appears on the small circular screen mounted on the tangible, with its head-gaze controlled via the integrated touchpad. This lets the expert keep the tabletop overview while simultaneously monitoring the on-prop egocentric view. Unlike Head, where attention often shifts toward the distant large screen, this better supports continuous awareness of both the miniature plant and the avatar’s egocentric view. Figure 2c illustrates this condition.

The three conditions trade off egocentric view size against parallel tabletop visibility. VR offers a full-scale, immersive first-person view but temporarily loses direct tabletop access. Head reduces it to a large but bounded preview while restoring continuous access to the miniature plant and all avatars’ head-gaze rays. Touch shrinks it to a small on-prop screen but maximizes parallel monitoring of the tabletop and avatar viewpoint, giving strong awareness of both global context and local detail.

Study Design

Collaboration Task Design

In designing the collaboration tasks, we grounded our choices in the industrial remote expert requirements outlined in § 3.1 and in prior work on asymmetric MR collaboration and navigation support [17, 42, 47]. The aim was to create a compact, repeatable scenario that captures the main phases of a typical remote fault detection session while stressing the need for effective gaze communication between expert and operator.

The first phase addresses navigation and wayfinding. Using the exocentric view of the tabletop plant WIM, the remote expert identifies the target machine by spotting status indicators that may be difficult to notice from the operator perspective. The expert then guides the immersed operator to the machine. Rather than relying only on spoken turn-by-turn directions as in audio-only support [17], the expert uses their "mini" avatar as a landmark, as in the "god-hand" [47]. By moving the avatar along the route and placing it near the destination, the expert highlights the path and key reference points, which the operator sees at full (1:1) scale.

The second phase centers on joint machine close inspection. Once the operator reaches the desired machine, both collaborators inspect it iteratively to identify the faulty component. This tightly coupled work increases the need for precise reference to specific parts and clear gaze communication. The task therefore directly exercises the expert’s gaze-control methods, requiring the expert to position the avatar at the machine and steer its gaze with one of the three interaction techniques to coordinate attention with the operator.

Inspection Puzzle

To evaluate non-expert participants in a fault-detection-inspired context, we designed a simplified machine inspection puzzle rather than a real industrial one. Our goal was to preserve the core collaboration characteristics of remote fault detection—joint inspection, mutual reference to parts, and agreement on the faulty component—while keeping the task easy to learn. We drew inspiration from prior collaborative research tasks [39], where two users inspect objects and must align their gazes on the same target for a passcode to be revealed, then verbally confirm whether their values match.

Adapting this idea, we implemented a sticker-solver puzzle. Once both users were co-located, several unrevealed stickers appeared on the machine’s sides, each (10×1010 \times10 cm) a potential fault indicator. Placing the gaze hit-sphere on a sticker revealed a passcode: the expert saw a number, the operator a letter. The pair verbally checked these against a printed index-to-letter sheet held by the expert (e.g., 1→a1 \to a, 2→b2 \to b, etc). If no match, they tried other stickers until finding the correct one. The operator then selected it to trigger the repair and “fix” the machine. This mechanic promotes active gaze use and verbal negotiation while keeping the task simple.

Fault Detection Tasks

We defined two tasks sharing the same goal—finding the machine fault—but differing in the collaborators’ spatial arrangement. Both use the inspection puzzle described above as their close-inspection mechanic. The first is a well-studied configuration where both users jointly analyze the same machine; the second places them at different plant locations. This second design emerged from expert input, where certain diagnostics compare a functional reference machine with a faulty one to localize the problem, demanding greater parallel attention to the exocentric overview than the first.

Task 1 (T1): Same Machine. Both expert and operator collaborate on the same machine. From the tabletop overview, the expert identifies the faulty machine by its red status indicator and guides the operator there. Once the operator enables it, the inspection puzzle activates: stickers appear around the machine, and the collaborators coordinate their head-gaze on the same stickers to reveal the candidate number-letter combination. After jointly identifying the correct sticker, the operator selects it to complete the repair. Figure 2d illustrates this task.

Task 2 (T2): Twin Machine. The expert and operator work on two spatially separated but identical machines: a faulty one and a reference. From the tabletop overview, the expert identifies both via status lights, guides the operator to the faulty (orange) machine, and moves their avatar to the reference (green) one. After the operator enables the faulty machine, the inspection puzzle activates on both: matching marks appear. To reveal a candidate number-letter combination, each collaborator looks at the same corresponding mark on their own machine; a single gaze reveals the code (the expert sees a number, the operator a letter). They then verbally determine which mark yields a matching pair per the predefined mapping, and, once identified, the operator selects it to repair the faulty machine. Figure 2e illustrates this task.

Study Procedure

Sessions took place in a quiet room. After welcoming the participant, the supervisor explained the study’s purpose, obtained informed consent, and gave a short overview of the setup, tasks, inspection puzzle, and gaze interaction methods. Participants then completed a pre-questionnaire on demographics and prior MR/VR experience. The supervisor adjusted the table height, configured the MR environment by positioning a virtual screen and aligning the virtual plant via QR-code tracking, and fitted the HMD for comfort.

A short two-part training followed: participants first practiced all interaction methods in a dedicated scene, then rehearsed both fault detection tasks with a virtual collaborator, including navigation and the sticker-based inspection puzzle. In the actual study, participants took the role of the remote expert, communicating via a Teams call with the operator—a study supervisor who followed a standardized script, acting as a naive operator led by the expert and providing only neutral acknowledgments to ensure consistent behavior across conditions—and completed 36 trials, with short breaks available on request. Afterward, the system automatically stored the data, and the participant completed a post-study questionnaire on their experience and subjective impressions. Including briefing, training, and questionnaires, each session lasted approximately 1 hour and 20 minutes.

All procedures for this study were approved by the TU Graz Ethics Committee (GZ: EK-102/2025).

Experiment

For the experiment, we evaluated the three avatar head-gaze interaction methods introduced in Section 4.3 (VR, Head, Touch) across the two fault detection tasks described above (T1: Same Machine and T2: Twin Machine). To control puzzle behavior, we defined two difficulty levels based on the number of incorrect stickers that had to be inspected before the solution became available: at level 1, the first inspected sticker was always incorrect and the next one was the solution; at level 2, the first two inspected stickers were incorrect and the solution appeared only on a later sticker. The system tracked inspected stickers and dynamically set the solution position accordingly.

We used a within-subjects design in which all participants, acting as remote experts, experienced all interaction methods, both tasks, and both difficulty levels. Conditions were counterbalanced using a Latin square distribution to reduce order effects. In total, each participant completed 2 tasks × 3 interaction methods × 2 difficulty levels × 3 repetitions, resulting in 36 trials per participant. Because each condition combines a specific visualization, display, and control method, the study compares bundled interface configurations rather than isolated factors; causal attribution to individual dimensions is not possible.

Hypotheses

We formulated our hypotheses to compare the 3 avatar’s gaze control methods in terms of task performance, perceived workload, and user preference, as well as the overall effectiveness of the tabletop avatar control for remote collaboration.

H1 (T1 – Same Machine). For T1, which focuses on detailed inspection around a single machine, we expect the VR condition to outperform Head and Touch. Alignment between interaction and visualization spaces has been shown to reduce mental effort and improve performance [17], and our pilot (§ 4.3) indicated that direct head control in an immersive view felt most natural for precise targeting. We therefore hypothesize that VR will yield faster task completion, lower perceived workload, and higher user preference.

H2 (T2 – Twin Machine). For T2, which requires parallel attention to two spatially separated machines, we expect Head and Touch to outperform VR. As discussed in § 3.2, full immersion temporarily removes direct access to the exocentric tabletop overview needed for cross-machine coordination. Because Head and Touch preserve continuous tabletop visibility while T2 demands monitoring both machines, we hypothesize they will yield faster completion, lower workload, and higher preference than VR.

H3 (Overall tabletop avatar control). Across all conditions, we expect the tabletop avatar control to support effective remote collaboration. Tabletop tangibles provide stability and precision for spatial control [32], and physical tokens linked to virtual agents have been shown to increase co-presence [28]. Because our avatar combines tangible manipulation with workspace awareness cues on a shared tabletop, we hypothesize that participants will rate it as effective for collaboration and coordination, and report high levels of presence and preference for the remote-expert role.

Measures

For the quantitative data, the system logged per-trial timestamps to compute three metrics:

  • Time Completion (solution_time−enabling_machine_time): Since all conditions share the same navigation and wayfinding phase, timing begins when the operator enables the machine, isolating the joint inspection phase where the gaze-control methods differ.

  • Overshoot: Count of tangible (re)selections, reflecting how often the expert needed to interrupt gaze control to reposition the avatar before re-engaging.

  • Gaze Overshoot: Count of times the gaze hit-sphere entered a sticker area, reflecting a combination of deliberate inspections and unintentional re-entries during gaze steering.

For qualitative data, participants completed a post-study questionnaire in which they rated each interaction method separately: a fast motion sickness scale, the NASA-TLX (workload), the System Usability Scale (SUS, usability and preference), and four items adapted from the IPQ (physical presence, involvement, attention to virtual plant, attention to tabletop). Further items addressed embodiment, interaction naturalness, and collaboration aspects (coordination and communication quality). Open-ended questions invited comments on each condition and on the overall the setup.

Results

29 participants (18 men and 11 women) were voluntarily recruited from the company, age M=28.9M = 28.9 (SD=5.2SD = 5.2). 17 participants did not wear corrective glasses, and 12 did. They rated their AR/VR experience on a 1–10 scale with M=4.4M = 4.4 (SD=2.5SD = 2.5), and their HMD experience with M=2.4M = 2.4 (SD=2.3SD = 2.3). Participants reported no physical or mental condition. In total, 998 of 1,044 expected trials (29 × 36) were analysed; the remainder were excluded due to one participant’s recording malfunction (subjective data retained) and a few time-constrained sessions that ended early.

Data Analysis Approach

Shapiro–Wilk tests indicated non-normal distributions for all measures; we therefore used Friedman tests with Holm-corrected post-hoc Wilcoxon signed-rank comparisons throughout. To account for difficulty levels, completion time, overshoot, and gaze overshoot were normalized by the minimum required steps (difficulty + 1). Outliers were removed before aggregation using Z-score filtering (∣z∣>3|z| > 3). Per-participant scores were then computed as the median across repetitions and difficulty levels for each condition, yielding one value per participant per method. An ART ANOVA revealed no significant interaction between interaction method and task type for completion time (F(2,135)=0.64F(2,135) = 0.64, p=0.530p = 0.530), confirming that method effects were consistent across both tasks; T1 and T2 were therefore analyzed separately. All analyses were conducted in R. Detailed statistical results, including pairwise comparisons, effect sizes, outlier exclusion counts, questionnaire items with response scales, and per-condition descriptive statistics for all subjective measures, are provided in the supplementary material.

T1 Same Machine

Quantitative Measures. A Friedman test revealed significant differences in completion time across interaction methods (χ22=16.93\chi^2_2 = 16.93, p<0.001p < 0.001, W=0.302W = 0.302). Post-hoc Wilcoxon tests revealed that Touch took significantly longer than VR and Head, while no significant differences were found between VR and Head (Figure 3). A Friedman test revealed significant differences in overshoot across interaction methods (χ2=9.16\chi^{2} = 9.16, p=0.01p = 0.01, W=0.164W = 0.164). Post-hoc Wilcoxon tests revealed that VR took significantly less overshoot than Touch and Head, while no significant differences were found between Touch and Head (Figure 3). No significant differences were found for gaze overshoot.

Pairwise results across interaction methods per task: completion time and overshoot

Figure 3. Pairwise results across interaction methods per task. Left: Completion Time. Right: Overshoot.

Task load, User Choice

For task load, Friedman tests revealed significant effects of interaction method on all NASA–TLX subscales: Mental Demand (χ2=14.25\chi^{2} = 14.25, p<.01p < .01, W=0.25W = 0.25), Physical Demand (χ2=34.07\chi^{2} = 34.07, p<.001p < .001, W=0.59W = 0.59), Temporal Demand (χ2=7.05\chi^{2} = 7.05, p<.05p < .05, W=0.12W = 0.12), Performance (χ2=30.13\chi^{2} = 30.13, p<.001p < .001, W=0.52W = 0.52), Effort (χ2=21.27\chi^{2} = 21.27, p<.001p < .001, W=0.37W = 0.37), and Frustration (χ2=17.15\chi^{2} = 17.15, p<.001p < .001, W=0.30W = 0.30). In all subscales, VR performed better than Touch. Figure 4 illustrates all results in detail.

NASA–TLX factors for all tasks and user preference by task

Figure 4. Subjective results. Left: NASA–TLX subscales by method and task. Right: Method preference by task.

When asked their preferred method, 18 of 29 participants chose VR, 2 Head, and 9 indicated both VR and Head as equally preferred.

T2 Twin Machine

Quantitative Measures

A Friedman test revealed significant differences in completion time across interaction methods (χ2=13.5\chi^{2} = 13.5, p=0.0012p = 0.0012, W=0.241W = 0.241). Post-hoc Wilcoxon tests revealed that Touch took significantly longer than VR; no significant differences were found between VR and Head or between Head and Touch (Figure 3). No significant differences were found for overshoot or gaze overshoot.

Task load, User Choice

For task load in TwinMachine, Friedman tests again revealed significant effects of interaction method on all NASA–TLX subscales: Mental Demand (χ2=13.25\chi^{2} = 13.25, p<.01p < .01, W=0.24W = 0.24), Physical Demand (χ2=29.96\chi^{2} = 29.96, p<.001p < .001, W=0.52W = 0.52), Temporal Demand (χ2=6.25\chi^{2} = 6.25, p<.05p < .05, W=0.11W = 0.11), Performance (χ2=26.94\chi^{2} = 26.94, p<.001p < .001, W=0.46W = 0.46), Effort (χ2=19.16\chi^{2} = 19.16, p<.001p < .001, W=0.34W = 0.34), and Frustration (χ2=18.97\chi^{2} = 18.97, p<.001p < .001, W=0.33W = 0.33). In all subscales except in Temporal Demand, VR performed better than Touch; further results for the other tasks are shown in Figure 4.

When asked to select their preferred method, 11 of 29 participants chose VR, 7 chose Head, 2 chose Touch, 8 indicated both VR and Head as equally preferred, and 1 selected all methods.

General interaction

System Usability Scale (SUS)

For SUS scores, a Friedman test revealed a significant effect of interaction method on usability, χ22=24.50\chi^{2}_{2} = 24.50, p<.001p < .001, W=0.42W = 0.42. VR showed the highest SUS scores, followed by Head and then Touch; post-hoc pairwise comparisons are shown in Figure 5.

General subjective results: system usability scale and collaboration

Figure 5. General subjective results. Left: System Usability Scale. Right: Overall collaboration feedback.

Presence

For the presence questionnaire, Friedman tests showed significant effects of interaction method on sense of physical presence (χ22=33.59\chi^{2}_{2} = 33.59, p<.001p < .001, W=0.58W = 0.58), involvement (χ22=34.03\chi^{2}_{2} = 34.03, p<.001p < .001, W=0.59W = 0.59), and attention to the virtual plant (χ22=17.63\chi^{2}_{2} = 17.63, p<.001p < .001, W=0.30W = 0.30), but not on attention to the tabletop (χ22=2.00\chi^{2}_{2} = 2.00, p=.37p = .37, W=0.03W = 0.03). In all subscales except tabletop attention, VR and Head outperformed Touch.

Embodiment

For embodiment items, Friedman tests showed significant effects of interaction method on perceived naturalness of controlling the avatar (χ22=24.15\chi^{2}_{2} = 24.15, p<.001p < .001), accuracy of avatar gaze (χ22=15.59\chi^{2}_{2} = 15.59, p<.001p < .001), feeling that the avatar was an extension of the self (χ22=18.89\chi^{2}_{2} = 18.89, p<.001p < .001), and naturalness and effortlessness of interacting with the virtual plant (χ22=33.32\chi^{2}_{2} = 33.32, p<.001p < .001). Across all items, VR showed the highest ratings, followed by Head, and then Touch, indicating stronger embodiment in VR, intermediate embodiment with Head, and lower embodiment with Touch.

General collaboration and coordination

As for the whole system, participants reported a generally positive collaborative experience. Ratings were high for partner awareness and communication (both M=4.59M = 4.59, SD=0.57SD = 0.57), shared understanding (M=4.48M = 4.48, SD=0.51SD = 0.51), task effectiveness and effective teamwork (both M=4.41M = 4.41, SD=0.63SD = 0.63), and perceived benefit of the different perspectives for collaboration (M=4.38M = 4.38, SD=0.68SD = 0.68). The transition into first-person (avatar) view was rated somewhat lower, though still positive (M=3.86M = 3.86, SD=0.95SD = 0.95).

Qualitative Feedback

Head: Participants found Head control natural, intuitive, and low effort, valuing the simultaneous view of the virtual plant and their real surroundings. Criticism focused on the avatar view sometimes falling out of sight, requiring awkward head movement or repositioning, along with the limited preview size and difficulty aligning their perspective with the avatar’s.

Touch: Participants found Touch easy to learn. They valued seeing both the plant and the avatar PoV on a stable screen, avoiding head/neck movement, and found it familiar, like a tablet. Criticism focused on the small view window and control area, which made it hard to see, orient, and place the gaze target, and on the non-ergonomic, fine-motor-demanding button hold during touchpad use. Some also found it slower and less intuitive, with small touches causing large, hard-to-predict view changes.

VR: Participants described VR as the most natural and effortless method, with fluid control, fast targeting, strong immersion, and a vivid sense of being “inside” the environment. They noted it made stickers easy to see and the gaze easy to steer, improving awareness of the collaborator’s focus and shared perspective. Criticism focused on full immersion hindering access to tabletop context, forcing users to exit VR to check the reference sheet or regain an overview, alongside occasional positioning issues needing large head movements or feeling “stuck” in awkward poses. Some also noted discomfort or motion sickness and found repeated tabletop–first-person transitions abrupt and cognitively demanding.

Overall Experience. Participants found the tabletop and tangible setup natural, intuitive, and smooth, with low mental and physical effort once understood. They valued the physically grounded control as comfortable and precise, and saw the system as effectively combining a global overview with fine-grained, avatar-level control to navigate efficiently while supporting partner awareness and shared understanding. Most described the experience as enjoyable and playful, noting it “felt like playing a game” and that they “had a lot of fun,” with some imagining themselves using the system in a real supervisor role.

Discussion

Touch Underperformed

Overall, Touch showed lower performance, which can be explained by participants’ feedback. Although the touchpad on the tangible prop initially seemed attractive for its compactness and co-location, it caused notable usability problems: the small control area and limited screen hindered orientation in the avatar’s egocentric view, leading to imprecise gaze targeting and higher cognitive load. Most importantly, having to press a button while simultaneously operating the touchpad introduces a dual-task demand that can negatively affect fine motor performance. Dual-task interference is known to reduce movement efficiency and increase completion time in upper-limb motor tasks due to competition for limited attentional resources 1. Additionally, concurrent cognitive and physical demands can increase indicators of wrist and forearm muscle fatigue during precision tasks2. Differences in pointing-device design can further influence hand posture and forearm muscle activity, affecting comfort and perceived control.

Moreover, feedback on Touch was mixed: some participants valued its precise, "calculated" fine control, while others felt it reduced control because small movements sometimes caused large or unpredictable view changes. This split likely reflects individual differences and prior experience with input modalities, which have been shown to influence both task performance and subjective usability ratings of interaction devices 3.

T1: Same Machine, VR strongest configuration

In T1, VR and Head outperformed Touch in completion time, while VR showed significantly fewer re-engagements (overshoot) than both Head and Touch. VR also scored best across all NASA–TLX subscales and was preferred by the majority of participants. These results indicate that for detailed inspection around a single machine, the VR configuration provided the strongest overall performance and user experience. Because the conditions bundle visualization, display, and control method, isolating the contribution of individual factors is not possible. However, the subjective data offers converging evidence for several candidate mechanisms. First, direct sensorimotor head-gaze mapping: both VR and Head, which share direct head control, received high general ratings for naturalness of avatar control and gaze accuracy, while Touch scored lowest on both; this aligns with prior work showing that misalignment between interaction and visualization spaces increases mental effort 4. Second, a full-scale view with larger targets: in T1, VR yielded the lowest physical demand and highest self-rated performance across NASA–TLX subscales, suggesting that interacting with targets at full scale reduced motor and perceptual effort. Third, reduced split attention: general presence ratings (collected across tasks) showed that VR received significantly higher physical presence and involvement scores, both with large effect sizes, while reported attention to the tabletop did not differ across conditions, suggesting that VR’s immersion strengthened engagement without participants perceiving a loss of tabletop awareness. Fourth, stronger spatial presence: general embodiment ratings showed VR scored highest on feeling that the avatar was an extension of the self, consistent with the sense of “being there” reported in the qualitative feedback. These factors likely act in combination rather than independently. Since VR did not significantly outperform Head in completion time, H1 is partially supported: confirmed for overshoot, workload, and preference, but not for task completion time.

In addition, some participants reported brief discomfort and moments of disorientation when transitioning from the MR tabletop view into the immersive VR viewpoint and back. Prior work similarly notes that switching viewpoints and scales can increase disorientation and cognitive effort due to the abrupt context change [^13, ^33]. In our study, this effect appeared limited: post-study Fast Motion Sickness (FMS) ratings did not indicate substantial motion sickness. Nevertheless, repeated or prolonged VR exposure could still amplify discomfort in longer sessions. To mitigate this, future designs could smooth the transition using gradual visual blending (e.g., progressive transparency or fade filters) rather than an instantaneous switch 5.

T2: Twin Machine, VR and Head top performance

In T2, contrary to our hypothesis (H2), Head did not outperform VR. Quantitative measures showed no significant differences, although subjective measures favored VR. This suggests that Head’s parallel tabletop awareness may have provided some benefit, but did not translate into measurably better performance. As with VR, Head bundles multiple factors (a floating preview, direct head control, and continuous tabletop access), so its underperformance relative to expectations cannot be attributed to a single cause. However, participant feedback points to a specific design limitation. The system was designed for the full task workflow, including navigation, which required the user to move freely around the table. This meant that when gaze control was activated, the floating screen’s initial orientation depended on where the expert happened to be standing. When participants faced the table center, the screen placement worked well; otherwise, starting head control produced awkward initial orientations, forcing uncomfortable head postures or avatar repositioning. The candidate mechanisms discussed for T1 (direct sensorimotor mapping, full-scale view, reduced split attention, and spatial presence) may have outweighed Head’s parallel awareness advantage, compounded by this screen-placement issue. However, the bundled design prevents attributing Head’s underperformance to any single factor.

This screen-positioning problem represents a design flaw that future work should address, either by isolating the screen-position factor to evaluate its impact independently, or by improving the dynamic positioning algorithm to adapt to the user’s current orientation. Resolving this limitation would enable a fairer assessment of whether parallel tabletop awareness genuinely benefits split-attention tasks like T2.

Comparing the two tasks descriptively, the performance gap between Head and VR narrowed in T2, though the ART ANOVA found no significant method-by-task interaction. Subjective preferences showed a clearer shift: more participants chose Head or rated Head and VR equally in T2 than in T1, suggesting that Head’s parallel tabletop awareness was more appealing when the task demanded attention to separated machines. Despite this subjective shift, H2 is not supported by the quantitative results.

Positive collaboration with tabletop-tui-avatar

Despite design flaws in Head and Touch, the tabletop avatar setup was well received: participants rated partner awareness, communication, shared understanding, and task effectiveness highly. These ratings reflect subjective experience rather than a direct measure of workspace awareness. The setup supports remote MR collaboration regardless of which gaze method is used.

The TUI prop for avatar placement and rotation was well accepted. This positive reception concerns the prop for positioning and orienting the avatar on the tabletop—a function distinct from the Touch gaze-control method, which received mixed feedback (see above). Participants reported natural, precise control, supporting our choice of a graspable, table-supported controller [24, 32].

Additionally, our positive findings provide a basis to discuss how we can contextualize and compare our system, even without a single, clear baseline to assess performance. If we focus on the wayfinding phase, we can argue that using the tangible avatar as a persistent landmark may provide stronger spatial grounding than pure voice commands [17], because it augments speech with an explicit, shared spatial reference in the miniature workspace. Likewise, compared to egocentric “god-hand” pointing metaphors [47], embodying the expert as an avatar with a head-gaze ray adds an additional layer of reference: the avatar’s placement communicates where to go, while the head-gaze ray communicates what to attend to once there. Together, these cues can support a smoother transition from global navigation to local inspection by making both destinations and attentional targets visible in the shared scene.

Consequently, H3 is supported by the results.

Limitations and Future Work

The complexity of an asymmetric collaboration scenario forces design and research decisions. We focused on the expert MR tabletop user. Future work should also evaluate the operator’s perspective, including how the social presence is impacted by a collaborator with 2 representation entities (god-view and tabletop avatar).

Although our system is designed so multiple tabletop or experts can be added, we only tested a single expert and operator. Future work should explore how the system scales to multiple experts and operators, including how to manage multiple avatars and perspectives in the shared space.

Conditions were counterbalanced to reduce order effects; however, we did not analyze learning or fatigue across the 36 trials, as this was not the focus of the study. Cumulative fatigue or improved familiarity with the system may have influenced later trials. Future work could examine trial-by-trial performance trends to disentangle these effects.

Head’s screen-positioning and Touch’s touchpad usability emerged as design flaws that our small expert-only pilot did not catch. Future iterations should address these issues before reassessing parallel awareness. Overall, all three techniques support simultaneous exocentric and egocentric awareness, but over different spans: Touch favors exocentric context with limited egocentric detail, Head is more balanced, and VR favors egocentric detail at the cost of direct tabletop access. Users can often bridge the two views mentally and prefer whichever offers better control. Future work should validate improved designs with a larger, more diverse sample and focus either on tasks that truly demand concurrent awareness or on interfaces enabling fast, low-friction switching between views.

Finally, this setup opens new opportunities for interaction and collaboration in a space with no prior baseline. We propose our work as such a baseline, on top of which future research can explore novel interaction techniques or challenge our design decisions.

Conclusion

We presented and evaluated an integrated tabletop avatar setup for remote MR collaboration, enabling expert gaze control via three interaction-visualization conditions (VR, Head, Touch) alongside a tabletop overview. A 29-participant study showed that the VR condition, which combines immersion, direct head control, and a full-scale view, yielded the strongest performance and user experience; the tabletop avatar setup was well received for collaboration. Our findings suggest that conditions offering direct sensorimotor mapping and reduced attentional switching best support precise gaze control in asymmetric MR–VR collaboration, and reveal design challenges for this emerging space.

References

References

  1. [1]Apple Inc. 2026. Apple Vision Pro. https://www.apple.com/apple-vision-pro/. Accessed: [26 February 2026].
  2. [2]R. Assaf, D. Mendez, and R. Rodrigues. 2024. Cues to fast-forward collaboration: A Survey of Workspace Awareness and Visual Cues in XR Collaborative Systems. Computer Graphics Forum 43, 2 (2024), e15066. doi:10.1111/cgf.15066 eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.15066.DOI
  3. [3]Agnese Augello, Giuseppe Cagnese, Luigi Gallo, and Pietro Neroni. 2023. Design Strategies to Enhance Awareness in MR Collaborative Systems. In Extended Reality, Luca Tomassone De Paolis, Pasquale Arpaia, and Marco Sacco (Eds.). Springer Nature Switzerland, Cham, 424–436. doi:10.1007/978-3-031-43401-3_28DOI
  4. [4]Mark Billinghurst, Hirokazu Kato, and Ivan Poupyrev. 2001. The MagicBook: a transitional AR interface. Computers And Graphics 25, 5 (2001), 745–753. doi:10.1016/S0097-8493(01)00117-0 Mixed realities - beyond conventions.DOI
  5. [5]Morgan Le Cheneg, Thierry Duval, Valerie Gouranton, and Bruno Arnaldi. 2016. When the giant meets the ant: an asymmetric approach for collaborative and concurrent object manipulation in a multi-scale environment. In 2016 IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE). IEEE, Greenville, SC, USA, 18–22. doi:10.1109/3DCVE.2016.7563562DOI
  6. [6]Yi Fei Cheng, Tiffany Luong, Andreas Rene Fender, Paul Streli, and Christian Holz. 2022. ComfortTable User Interfaces: Surfaces Reduce Input Error, Time, and Exertion for Tabletop and Mid-air User Interfaces. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 150–159. doi:10.1109/ISMAR55287.2022.00029
  7. [7]Dane Coffey, Nicholas Malbraaten, Trung Bao Le, Iman Borzanjani, Fotis Sotiropoulos, Arthur G. Erdman, and Daniel F. Keefe. 2012. Interactive Slice WIM: Navigating and Interrogating Volume Data Sets Using a Multisurface, Multitouch VR Interface. IEEE Transactions on Visualization and Computer Graphics 18, 10 (Oct. 2012), 1614–1626. doi:10.1109/TVCG.2011.283DOI
  8. [8]Kurtis Danyluk, Barrett Ens, Bernhard Jenny, and Wesley Willett. 2021. A Design Space Exploration of Worlds in Miniature. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–15. doi:10.1145/3411764.3445098DOI
  9. [9]Thierry Duval, Thi Thuong Huyen Nguyen, Cédric Fleury, Alain Chaufaut, Georges Dumont, and Valérie Gouranton. 2014. Improving awareness for 3D virtual collaboration by embedding the features of users’ physical environments and by augmenting interaction tools with cognitive feedback cues. Journal on Multimodal User Interfaces 8, 2 (June 2014), 187–197. doi:10.1007/s12193-013-0134-zDOI
  10. [10]Jeff Dyck. [n. d.]. Awareness in 3D Shared Workspaces. [(n. d.)]. https://www.academia.edu/129050263/Awareness_in_3D_Shared_Workspaces
  11. [11]Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2017. Travel in large-scale head-worn VR: pre-oriented teleoperation with WIMs and previews. In 2017 IEEE Virtual Reality (VR). 475–476. doi:10.1109/VR.2017.7892386 ISSN: 2575-5334.DOI
  12. [12]Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies 131 (2019), 81–98. doi:10.1016/j.ijhcs.2019.05.011 50 years of the International Journal of Human-Computer Studies. Reflections on the past, present and future of human-centred technologies.DOI
  13. [13]Nico Feld and Benjamin Weyers. 2021. Mixed Reality in Asymmetric Collaborative Environments: A Research Prototype for Virtual City Tours. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 250–256. doi:10.1109/VRW52623.2021.00053DOI
  14. [14]Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing 8, 5 (Sept. 2004), 347–358. doi:10.1007/s00779-004-0297-4
  15. [15]George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95). ACM Press/Addison-Wesley Publishing Co., USA, 442–449. doi:10.1145/223904.223964DOI
  16. [16]Adélaïde Genay, Anatole Lécuyer, and Martin Hachet. 2022. Being an Avatar “for Real”: A Survey on Virtual Embodiment in Augmented Reality. IEEE Transactions on Visualization and Computer Graphics 28, 12 (Dec. 2022), 5071–5090. doi:10.1109/TVCG.2021.3099290DOI
  17. [17]R. Grasset, P. Lamb, and M. Billinghurst. 2005. Evaluation of mixed-space collaboration. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05). 90–99. doi:10.1109/ISMAR.2005.30DOI
  18. [18]Carl Gutwin and Saul Greenberg. 1996. Workspace Awareness for Groupware. doi:10.1145/257089.257284 Pages: 209.
  19. [19]Carl Gutwin and Saul Greenberg. 2002. A Descriptive Framework of Workspace Awareness for Real-Time Groupware. Computer Supported Cooperative Work (CSCW) 11, 3-4 (Sept. 2002), 411–446. doi:10.1023/A:1021271517844DOI
  20. [20]HTC Corporation. 2026. Vive Focus Vision. https://www.vive.com/eu/product/vive-focus-vision/overview/. Accessed: [26 February 2026].
  21. [21]Yen-Ming Huang, Tzu-Wei Mi, and Liwei Chan. 2025. Preview Teleport: An Occlusion-Free Point-and-Teleport Technique Enhanced with an Augmented Preview. IEEE Transactions on Visualization and Computer Graphics 31, 5 (May 2025), 2546–2556. doi:10.1109/TVCG.2025.3549138DOI
  22. [22]Hikaru Ibayashi, Yuta Sugiura, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Masaaki Mochimaru, and Takeo Igarashi. 2015. Dollhouse VR: a multi-view, multi-user collaborative design workspace with VR technology. In SIGGRAPH Asia 2015 Emerging Technologies (Kobe, Japan (SA ’15). Association for Computing Machinery, New York, NY, USA, Article 8, 2 pages. doi:10.1145/2818466.2818480DOI
  23. [23]Keiichi Ihara, Mehrad Faridan, Ayumi Ichikawa, Ikkaku Kawaguchi, and Ryo Suzuki. 2023. HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed Reality. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–12. doi:10.1145/3586183.3606727 arXiv:2307.16114 [cs].DOI
  24. [24]Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. ACM, Atlanta Georgia USA, 234–241. doi:10.1145/258492.258715
  25. [25]H. Kato, K. Tachibana, M. Tanabe, T. Nakajima, and Y. Fukuda. 2003. MagicCap: a tangible interface for virtual objects manipulation in table-top augmented reality. In 2003 IEEE International Augmented Reality Toolkit Workshop. 75–76. doi:10.1109/ART.2003.1320434DOI
  26. [26]K. Kiyokawa, H. Takemura, and N. Yokoya. 1999. A collaboration support technique by integrating a shared virtual reality and a shared augmented reality. In IEEE SMC’99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028), Vol. 6. 48–53 vol.6. doi:10.1109/ICSMC.1999.816444
  27. [27]Jan Kalkmeier, Emiel Harmsen, Sander Giesselink, Dennis Reidsma, Mariët Theune, and Dirk Heylen. 2018. With a little help from a holographic friend: the OpenIMPRESS mixed reality telepresence toolkit for remote collaboration systems. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology. ACM, Tokyo Japan, 1–11. doi:10.1145/3281505.3281542
  28. [28]Myungho Lee, Nahal Norouzi, Gerd Bruder, Pamela J. Wisniewski, and Gregory F. Welch. 2018. The physical-virtual table: exploring the effects of a virtual human’s physical influence on social interaction. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology. ACM, Tokyo Japan, 1–11. doi:10.1145/3281505.3281533DOI
  29. [30]Geert Lugtenberg, Kazuma Mori, Yuki Matoba, Theophilus Teo, and Mark Billingshurst. 2023. The MagicBook Revisited. In 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). 801–806. doi:10.1109/ISMAR-Adjunct6011.2023.00177DOI
  30. [31]Meta Platforms Inc. 2026. Meta Quest 3. https://www.meta.com/de/en/quest/quest-3/. Accessed: [26 February 2026].
  31. [32]Carlos Mosquera, Neven ElSayed, Ernst Kruijff, Joseph Newman, and Eduardo Veas. 2025. A Study of Performance and Interaction Patterns in Hand and Tangible Interaction in Tabletop Mixed Reality. In Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology (VRST ’25). Association for Computing Machinery, New York, NY, USA, 1–11. doi:10.1145/3756884.3766001DOI
  32. [33]Jens Müller, Johannes Nagermann, Jonathan Wieland, Ulrike Pfeil, and Harald Reiterer. 2019. A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices. In Proceedings of Mensch und Computer 2019 (Hamburg, Germany) (MuC ’19). Association for Computing Machinery, New York, NY, USA, 399–410. doi:10.1145/3340764.3340773DOI
  33. [34]Ken Nakagaki, Joanne Leong, Jordan L. Tappa, João Wilbert, and Hiroshi Ishii. 2020. HERMITS: Dynamically Reconfiguring the Interactivity of Self-propelled TUIs with Mechanical Shell Add-ons. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. ACM, Virtual Event USA, 882–896. doi:10.1145/3379337.3415831DOI
  34. [35]Huyen Nguyen and Tomasz Bednarz. 2020. User Experience in Collaborative Extended Reality: Overview Study. In The 17th EuroVR International Conference. Valencia, Spain, 41–70. doi:10.1007/978-3-030-62655-6_3DOI
  35. [36]Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, Charlotte NC USA, 405–415. doi:10.1145/2807442.2807497DOI
  36. [37]Youngmin Park and Woontack Woo. 2006. The ARTable: An AR-Based Tangible User Interface System. In Technologies for E-Learning and Digital Entertainment, Zhigeng Pan, Ruth Aylett, Holger Dener, Xiaogang Jin, Stefan Göbel, and Li Li (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1198–1207.
  37. [38]Randy Pausch, Tommy Burnette, Dan Brockway, and Michael E. Weiblen. 1995. Navigation and locomotion in virtual worlds via flight into hand-held miniatures. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques (SIGGRAPH ’95). Association for Computing Machinery, New York, New York, NY, USA, 399–400. doi:10.1145/218380.218495 [39] Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019. The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Frontiers in Robotics and AI Volume 6 - 2019 (2019). doi:10.3389/frobt.2019.00005
  38. [40]Thammathip Piumsomboon, Gun A. Lee, and Mark Billinghurst. 2018. Snow Dome: A Multi-Scale Interaction in Mixed Reality Remote Collaboration. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–4. doi:10.1145/3170427.3176495
  39. [41]Thammathip Piumsomboon, Gun A. Lee, Jonathon D. Hart, Barrett Ens, Robert W. Lindeman, Bruce H. Thomas, and Mark Billinghurst. 2018. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. doi:10.1145/3173574.3173620DOI
  40. [42]Thammathip Piumsomboon, Gun A. Lee, Andrew Irlitti, Barrett Ens, Bruce H. Thomas, and Mark Billinghurst. 2019. One Shoulder of the Giant: A Multi-Scale Mixed Reality Collaboration with 360 Video Sharing and Tangible Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland, UK) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–17. doi:10.1145/3290605.3300458
  41. [43]Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: a collaborative virtual and augmented reality system for remote collaboration. In SIGGRAPH Asia 2017 Emerging Technologies (Bangkok, Thailand) (SA ’17). Association for Computing Machinery, New York, NY, USA, Article 3, 2 pages. doi:10.1145/3132818.3132822DOI
  42. [44]Luciana Provenzano, Julie Delzons, Patricia Plénacoste, and Johann Vandromme. 2007. Designing Viewpoint Awareness for 3D Collaborative Virtual Environment Focused on Real-Time Manipulation of Multiple Shared Objects. In Virtual Reality, Randall Shumaker (Ed.). Springer, Berlin, Heidelberg, 147–156. doi:10.1007/978-3-540-73335-5_17DOI
  43. [45]Mickael Sereno, Xiaoyu Wang, Lonni Besançon, Michael J. McGuffin, and Tobias Isenberg. 2022. Collaborative Work in Augmented Reality: A Survey. IEEE Transactions on Visualization and Computer Graphics 28, 6 (June 2022), 2530–2549. doi:10.1109/TVCG.2020.3032761 Conference Name: IEEE Transactions on Visualization and Computer Graphics.DOI
  44. [46]Sony Corporation. 2026. Toio Core Cube Specification. https://toio.github.io/toio-spec/en. Accessed: 22 June 2026.
  45. [47]Aaron Stafford, Wayne Piekarski, and Bruce H. Thomas. 2006. Implementation of god-like interaction techniques for supporting collaboration between outdoor AR and indoor tabletop users. In 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. 165–172. doi:10.1109/ISMAR.2006.297809DOI
  46. [48]Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual reality on a WIM: interactive worlds in miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95). ACM Press/Addison-Wesley Publishing Co., USA, 265–272. doi:10.1145/223904.223938DOI
  47. [49]Yuta Sugiura, Hikaru Ibayashi, Toby Chong, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Takashi Shinmura, Masaaki Mochimaru, and Takeo Igarashi. 2018. An asymmetric collaborative system for architectural-scale space design. In Proceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (Tokyo, Japan) (VRCAI ’18). Association for Computing Machinery, New York, NY, USA, Article 21, 6 pages. doi:10.1145/3284398.3284416DOI
  48. [50]John Underkoffler and Hiroshi Ishii. 1999. Urp: a luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI ’99. ACM Press, Pittsburgh, Pennsylvania, United States, 386–393. doi:10.1145/302979.303114DOI
  49. [51]Unity Technologies. 2026. Netcode for GameObjects. https://docs.unity3d.com/Packages/[email protected]/manual/index.html. Accessed: 22 June 2026.
  50. [52]Simon Voelker, Kjell Ivar Øvergård, Chat Wacharamanotham, and Jan Borchers. 2015. Nobody Revisited: A Comparison of User Performance between Tangible and Virtual Rotary Knobs. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (Madeira, Portugal) (ITS ’15). Association for Computing Machinery, New York, NY, USA, 35–38. doi:10.1145/2817725.2817725
  51. [54]Peng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Weiping He, Li Zhang, Jiaxiang Du, and Shuxia Wang. 2018. Do You Know What I Mean? An MR-Based Collaborative Platform. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). 77–78. doi:10.1109/ISMAR-Adjunct.2018.00038DOI
  52. [55]Waveshare. 2026. ESP32-S3-Touch-LCD-1.46B Documentation. https://www.waveshare.com/wiki/ESP32-S3-Touch-LCD-1.46B. Accessed: 22 June 2026.
  53. [56]Tim Weissker, Matthis Franzgrote, and Torsten Kuhlen. 2024. Thy This for Size: Multi-Scale Teleportation in Immersive Virtual Reality. IEEE Transactions on Visualization and Computer Graphics 30, 5 (May 2024), 2298–2308. doi:10.1109/TVCG.2024.3372043DOI

Paper details

vlnc:W_w2pvcyc6
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents