Why AR is Worth a Thousand Words to Frontline Workers

When phone calls, video conferences, and paper manuals aren’t sufficient for distant coaching or remote repair instruction, augmented reality may be the answer

Giving instruction from a distance has its challenges. For example, have you ever tried, over the phone, to talk a new teenage driver into changing a flat tire? Or to help a friend program their garage door opener using their car’s built-in emulator? Or to coach someone about installing a new smart thermostat?

You can use all the body language you want, and enunciate as clearly as you know how, but it can still be hard because what is obviously a toggle switch to you, may be in someone else’s eyes just a strange-looking widget. And beyond the issues of nomenclature, minor differences in a part’s positioning, color, or size can make even the most articulate set of directions virtually useless. Some things just need to be seen to be understood.

Now consider this: instead of visualizing how you might try to talk your elderly aunt through connecting her new wi-fi router, consider what might happen on the factory floor or in a field installation of a major industrial organization like an auto assembly line, a maritime facility, an electrical substation, an offshore drilling rig, or an air traffic control tower. There, as with all types of machinery, specialized pieces of capital equipment are going to require maintenance and occasional repairs. Staff technicians may know how to do most of it, but everyone needs a little help now and then.

Until Covid-19 disabled the economy, the solution frequently involved dispatching an expert from the equipment’s original maker to install, adjust, fix, or otherwise service the device in question. But the pandemic changed that strategy. Travel has been sharply restricted, and companies have been hesitant about welcoming visitors to their facilities – even certified experts. Instead, what many industrial operations are now doing involves virtual coaching using AR – augmented reality-based remote assistance.

AR is not just another flavor of the videoconferencing technologies that have by now become so routine for office employees working from home. It’s not Zoom. It’s not Facetime. It’s not Skype. It’s different. And for frontline workers, it’s considerably more useful.

Augmented reality-based remote assistance is software technology that enables users to superimpose text, tables, drawings or other graphic material onto a video image over the cell phone network. So, for example, a panel of switches could become annotated on the screen in ways that allow someone on site to readily identify which switches controls what functions. An expert connected to an on-site technician could see the device in question on the screen of their own phone, tablet, or PC. They could then highlight the part that needs attention by, for instance, dropping a virtual arrow or circling it with their finger on the screen, and those annotations will stick to the object, even when the technician’s camera is moving around.

The key advantage of AR based remote assistance is that it provides an instantaneous feedback system, where users interact with the actual elements of their work, as well as with one another. And it’s not just two people talking and gesturing while seated at their desks; it’s a highly mobile form of communication that includes on-site images, annotations, and graphic augmentation as well as a two-way voice connection, focused on solving practical problems.

In some cases, using this technology can even keep people safe. For example, it can prevent exposure to hazardous situations by limiting the number of healthcare workers who need to be in the hospital room of a Covid-19 patient, or of people who have to be in an industrial environment where radioactivity, dangerous gases, or other risks are present. But its greatest advantages may actually be in less perilous situations, where IT personnel can, for instance, help to maintain continuity in their company’s workflow by using AR to show their colleagues working from home how to change a printer cartridge, connect a scanner to their laptop, and otherwise keep their work equipment running.

Depending on the specific software, other features are available as well. Among them: recording a session for future training, retaining images, looping in multiple users, issuing push notifications, transferring files, object character recognition, and more. Many are available as smartphone apps from the same Apple and Google stores that consumers use. And, depending on how the software will be used, some of them are even free.

Video conferencing can be good. But the shortcomings of using it as an organization’s primary form of contact with its customers, suppliers and colleagues in the field are now well-recognized. However AR based remote assistance is, by design, a more intermittent, special-purpose vehicle of communication. So even after the pandemic has passed and more employees return to their familiar office surroundings, this is likely to become part of an enduring arsenal of tools that frontline technicians and operating personnel can rely on to fix real-world problems.