Context

A remote troubleshooting assistant for the physical products is in need for many scenarios.  Most of their solutions are targeting trained users and technicians, which require skills and devices on both sides. In scenarios like the printing machine user experience, where end users are neither familiar with special devices like AR glasses, nor having access to special tools, the needs for remote troubleshooting assistant are under-addressed. For example, if the paper is jammed in a public printer, all users might need to wait for a day or more to get it fixed by an off-site technician.

Design concept

Projection mapping is used to enhance existing manuals  with indexical reference (pointing out machine parts, for instance). Support technician is available for remote consultation using voice communication, remote pointing and optional video feed from site.

Experiential qualities and related concepts

Collaborative, control, embodiment, fluency, hybrid, trustworthy, visibility.

Augmentation modality

Visual, (auditory).

Technical options

To develop an intuitive remote troubleshooting assistant, three physical elements and one psychological element are essential for the user experience:

1. Shared view – users and technicians should be able to see where and what is the problem.
2. Shared annotation – they should be able to indicate where and what they are talking about.
3. Fluent instruction and reaction– the communication should be instant and instructive.
4. Comfortable interaction – both users and technicians should feel confident and have control on what they are doing and sharing.

Currently, there are three types of AR solutions on the market, none of them fully fulfils all the above requirements.

Type one: AR manual

Exemplified by “Cisco AR solution for Cisco Technical Contents“, some AR solutions are served as an AR manual. Users can use mobile devices to take photos or record the physical products/parts, then the AR application will add an information/manual overlay on the products’/parts’ images. This type of solution  has no flexibility to diagnose problems outside of the predicted scenarios, and it is lack of interaction with technicians. This means an AR manual is not a suitable solution for this project.

Type two: Interaction through double screens

Exemplified by “IBM AR solution for technology support“, users and technicians can have shared views and annotations through two interactive screens, but users’ hands are occupied by screen devices. Their following instruction activities are fragmented. But, as visualized in the animation “Indifix AR customer support“, this double screens solution might be necessary at a mobile/outdoor situation, due to its advantage in terms of mobility, less requirement on devices, and potentially lower-cost to set up.

Type three: Instruction through AR glasses

Exemplified by “Remote AR support for HoloLens“, although users’ hands are freed by using AR glasses, current development of AR glasses didn’t provide users a comfortable and intuitive experience. Most of the solutions today don’t work smooth in real situations. However, it can still be the best available solution for some professional settings where mobility and users’ fluence of activities to follow the instructions are important.

Therefore, in this project, we limit our design scope to a fixed-location setup and look at a different solution for the instructions which allows users’ hands stay free from holding the instruction device while following the instructions. The latest 3D projection technology is the key enabler of the following design concept.

At this stage, we will focus on an optimal user experience for the remote troubleshooting assistance. The technical details and cost feasibility remain uninvestigated since it is largely depending on the context.

Prototyping techniques

For this design idea, it is important to find out whether the projection works on a 3D-object, and what are the limitations. The video below recorded the experiment of the sign projection on a physical printer. It helped to identify which color, shape, animation and movement of the signs work best for the design story.

An important finding which has shaped the design story is that the sign projection on a 3D object gets deformed. It means any form or shape of the sign cannot communicate its meaning accurately all the time, thus we choose to use only the red dot as the sign for annotation, to avoid miscommunication. The sign will only serve as an indicator for position, while the instruction for actions could be communicate through screen interface and technician’s verbal communication.

Experiment to determine projected pointer (video).

Source

Yin He, 2018.
Link to Video