“Augmented” Virtual Reality for User Tests

• Research
• User experience
• VR
• XR

Share on Twitter : “Augmented” Virtual Reality for User Tests (New window) Share on Linkedin : “Augmented” Virtual Reality for User Tests (New window)

Assessing the User Experience, in Context, Using Virtual Reality

Orange, like various stakeholders in the field of consumer testing [1], has an interest in virtual reality as a context simulation tool to assess the user experience [2], combining both internal validity (reliability and reproducibility of results) and ecological validity (ability to extend results beyond the laboratory). Through its ability to simulate environments close to reality, the great freedom it offers in terms of creation and customization, and its compact and mobile nature, this technology could be an attractive alternative for best assessing the user experience, by immersing testers in real-life user conditions. Indeed, the context in which a service is used impacts the resulting user experience [3], but its various components (physical, temporal, social, etc. [4]) are difficult to incorporate into a laboratory protocol. In situ tests, that is, tests carried out in the actual context in which the product or service is used, make it possible to obtain results that are more representative of the actual uses and associated feelings. However, this comes at the detriment of internal validity, and such tests are also costly and more difficult to implement. It is claimed that the limitations of the laboratory and in situ approaches explain the difficulty in predicting whether or not a product will be a success [5]. An initial study [6], in which assessments carried out in the laboratory, in virtual reality and in the participants’ homes were compared, showed that virtual reality did not introduce major bias. However, no physical interaction was required in this virtual environment: Participants tested the Djingo assistant’s functions by means of voice interaction. A large part of the services subject to consumer testing involve interaction via a manually activated graphical interface, either directly (touch screen) or indirectly using a device (mouse or remote control), or the handling of objects (Livebox, cables or instructions, for example). Performing these tests in a virtual environment raises the following question: Should we simulate the handling in a fully virtual manner (representation of objects in the environment and hand tracking, for example) or should we propose technical solutions that maintain the tactile aspect of the user experience for the most natural interaction possible? It was with the aim of answering these research questions that an initial study was carried out, focusing on the example of the remote control, an item that is needed to allow interaction with Orange’s TV service.

Virtual Reality User Tests to Assess Orange TV Service Interfaces

The Rennes Customer Test Center regularly conducts user test campaigns to assess Orange TV service interfaces. In order to test the service in conditions representative of our customers’ daily use, these tests could be carried out in a 3D environment to make it possible to vary the viewing context (TV size, distance or day/night) or to simulate events such as network problems. This option is being explored at Orange Innovation for various services and Virtual Reality interaction solutions. TV interface user tests have recently been carried out with ten Orange TV service customers immersed in a virtual lounge, with the aim of comparing different technical solutions for interacting with the TV service.  For each solution, the participant was asked to perform simple tasks such as changing channel, accessing the live program, viewing detailed information regarding the live program, viewing content in Replay mode, etc. At the end of each session, the participant assessed their experience via a presence questionnaire [7] composed of about 20 questions to probe feelings in relation to the ability to perform actions, realism, the ability to examine things, interface quality, performance and haptics.

What Technical Solutions Are Available for a Remote Control in Virtual Reality?

Interacting with Orange TV in virtual reality requires access to the TV stream within the virtual environment and the ability to interact, using a remote control to control the TV stream. Access to the TV stream was achieved by integrating a web browser into the virtual environment. As for the interaction with a remote control, three technical solutions were tested, which are described below with the user feedback obtained during the testing. These solutions are integrated into a virtual reality user testing tool called ImmersiveTestU, which works on a network and allows the experimenter to see various 3D views of the test underway on a PC.

• Virtual Remote Control with Hand Tracking

The first solution (Capture 1) involves displaying a virtual remote control in the environment and using the hand tracking technology of Meta headsets [8]: The user interacts as they would with a keypad, using their index finger to press the buttons, with visual and audio feedback associated with the selection and/or pressing of the button. This solution offers no haptic feedback.

User feedback shows there is a clear disadvantage to this solution. Several participants reported difficulties, with this solution being perceived to be too sensitive to the slightest movement.  As a result, users reported that their attention was drawn to the interaction with the remote control itself, rather than the tasks to navigate the TV service. Participants also said they were inconvenienced by not being able to use their virtual thumb to press the keys, as they do on their remote control at home. Thus, the interaction was perceived as difficult and unnatural.

Capture 1: Virtual Remote Control with Hand Tracking

• Virtual Remote Control with Controller

The second solution (Capture 2) is similar to the first but users interact with the virtual remote control by holding a controller in their hand. The user interacts with a virtual “pointer/laser” that originates from the controller and enables the user to select the buttons with which they wish to interact. The visual and audio feedback of the previous solution is complemented by haptic feedback (vibration).

This controller-based solution which, in principle, is not realistic either, was nevertheless viewed as natural and was preferred to the hand-tracking solution. This is probably because it was clearly perceived to be a device specific to the virtual reality, but one which allowed effective interaction. Furthermore, the act of holding an object that provided haptic feedback was seemingly appreciated and allowed users to better project themselves in a remote control handling situation. Nevertheless, the results show that the handling of the controller still required attentional resources, to the detriment of the main task.

Capture 2: Virtual Remote Control with Controller

• Real Remote Control with Overlaid Video

The third solution allows the user to hold a real remote control in their hand (Capture 3). A webcam attached to the virtual reality headset allows the webcam video stream to be overlaid in the VR view. Use of a green background makes it possible to isolate the video stream, so as to retain only the hand and the remote control. This solution offers the advantage of having the same haptic feel as in a normal interaction with Orange TV.

This solution was appreciated because it was perceived to be a natural solution that did not interfere with the interaction task, despite the technical faults experienced: highly sensitive autofocus function of the webcam, which created blur; a lack of responsiveness related to communication problems between the remote control and the PC. This is therefore a promising solution that would make it possible to imagine various interactions with real objects within a virtual environment.

Capture 3: Real Remote Control with Overlaid Video

Continuation of the Work

This study confirms the value of virtual reality as an alternative to better assess user experience, by immersing testers in more realistic conditions of use. However, these initial results underline the need to ensure that interaction in the virtual environment is not only effective but also natural.  A fourth solution is being explored with a virtual representation of the real object with which the user interacts. This approach would combine the visual quality of the 3D environment (slightly degraded due to the overlaid video stream) and the natural aspect of the interaction, through its haptic component (holding the physical object in hand, consistent with its representation in the virtual world). This solution involves the development of a specific remote control, one that has the same buttons as the Orange remote control but also offers “touch” information thanks to capacitive sensors.

The improvement of the ImmersiveTestU user testing tool entails the integration of functions available on certain headsets, such as eye tracking to determine which objects are focused on by the user (and for how long). Furthermore, ImmersiveTestU is also being used as part of a thesis on contextual effects, the aim of which is to further knowledge on the subject: Of all the contextual components—physical, temporal, social, etc.—which ones have a significant influence on the user experience? What tools and methods can be proposed to take them into account? Virtual reality is an interesting area for further research, but for which components?