5 approaches to combine Augmented Reality with Artificial Intelligence for better servicing machinery and equipment

Technology | min. reading time

19th of November 2020

Martin Plutz Blog oculavis
Gunter Dueck oculavis guest article
Artificial intelligence (AI) is one of the hype topics that has been driving machinery and equipment manufacturers not just since yesterday. Expectations were lowered by the growing awareness that artificial intelligence in products and processes is not easily available and that the effort required to develop appropriate AI based systems should not be underestimated. In addition to the actual algorithms that make machine learning possible, deep domain-specific knowledge is needed above all to enable these algorithms to produce meaningful results.

Predictive analytics as main field of application of artificial intelligence in the service operations

From the point of view of most machine builders, it is meaningful, for example, to make an estimate of the time of failure of a machine or specific components. This data-based prediction of machine states is usually referred to as predictive analytics, and the knowledge advantage that can be achieved can be used to optimize maintenance times, the replacement of spare parts or the early incorporation of planned downtimes into the end customer's production planning and control. But what happens if problems occur during planned maintenance or replacement of spare parts? The employee from the end customer's maintenance department may not know exactly how to replace a specific spare part or how to carry out maintenance correctly. Or the travel time of a technician of the machine manufacturer to the end customer takes so long or may not be possible at all due to travel restrictions that the time advantage of a prediction based on predictive analytics is simply eaten up.

From data level to the human level of action

These examples make it clear that artificial intelligence can quickly result in demand for human action. This is exactly where Augmented Reality based solutions come in to optimize these human interactions, such as the manual maintenance or repair mentioned above. Visual remote support with Augmented Reality annotations can be used, for example, to ensure that the expert of a machine manufacturer accompanies a complicated maintenance operation from a distance to ensure that the work is carried out correctly. If the machine manufacturer has already prepared the knowledge required for the maintenance in the form of instructions, the correct work processes can be displayed step-by-step on a tablet, smartphone or smart glasses to an onsite operator at the machine. This means that remote guidance by an expert can be replaced by self-guidance based on digital instructions and workflows for standardizable procedures. The challenge in this overall process is to make the right information available at the right time and in the right place. This can be achieved, for example, by integrating predictive maintenance with augmented reality systems. When the need for action is triggered by a data-based prediction, maintenance staff on site are informed by email or push notification, work through an Augmented Reality-based workflow for guided maintenance and, if necessary, consult a specialist from the machine manufacturer.

One glance says more than a thousand data points?

Thus, the described processes can be integrated into a hybrid Smart Service Chain, which combines the strengths of artificial intelligence in the analysis of data with the human strengths in the execution of complex decision and work situations. Augmented Reality based solutions such as visual assistance or digital workflows offer a perfect complement to ensure a fast ability to act when malfunctions of machines and systems are detected. They can also be used to evaluate the prediction of algorithms through a human view of the situation.

Computer Vision for the automated extraction of information from images

This is exactly where artificial intelligence approaches can be used to supercharge this human view of a remote expert with special abilities. In contrast to the predictive applications described so far, here the manifestations of AI are located in the field of computer vision and in a certain way expand the senses of the user, as the following examples show:

  • The automatic extraction of text on photos or in a video call using Optical Character Recognition (OCR) makes it possible, for example, to recognize the serial number of a machine so that it does not have to be entered manually.
  • If the text is written in a language that the user himself does not understand (e.g. a safety instruction on a machine or similar), automatic translation services based on artificial intelligence can be used not only to automatically recognize a text, but also to display it directly in another language.
  • Sometimes, however, an expert who is remotely connected does not want or is not allowed to see everything. For example, to ensure the privacy of people who happen to walk through a scene, AI-based face recognition approaches can be used to make people unrecognizable in images and videos.
  • Furthermore, AI-based approaches can be used not only for the recognition of faces, but also for the recognition of patterns e.g. for quality assurance of processes or for the recognition of components e.g. for the automatic detection of spare parts that have to be identified during maintenance.

A brief look into the future

These specific examples not only show applications of artificial intelligence in the service of mechanical and plant engineering but are also intended to provide a realistic orientation about what is possible today. Of course, there are already more far-reaching ideas than those presented in this article, such as completely replacing a remote expert with an algorithm that visually analyzes a live video stream from a technician on site in real time, uses Natural Language Processing (NLP) to understand the spoken word, derives the right recommendations from it, and uses text-to-speech technologies to communicate to the employee what needs to be done. This can serve as a vision for the future, but with the technological possibilities of today, it is unlikely to become a realistic option for the future soon. And the question of whether we want such a vision is still to be answered separately as an individual, company and society. How do you see the future of AI and AR?

Would you like to know which of these ideas are already included in oculavis SHARE and what our roadmap regarding the use of artificial intelligence looks like? You are invited to contact us and start a conversation on these topics!

My bookmarks

Further articles from the category Technology

The Industrial Metaverse. Definition, applications and potentials in mechanical engineering & the industry
Next Level Video Support: Multi User Call in oculavis SHARE 9.0
Resilient and Tactile 5G End-to-End Infrastructure for Privacy- and Safety-Proofed Remote Assistance and Worker Safety
Digital workflows enable self guidance in machine related work
Microsoft Edge Browser switches to Chromium and is now supported by oculavis SHARE

Recommended articles

Exchange, networking & insights into the latest technologies. Industry experts & thought leaders are getting together at All About Remote
Bosch CDO distinguishes oculavis for outstanding startup collaboration as part of the Open Bosch Awards 2023
The Future of Field Service – More than 100 field service leaders join the kick-off event in the United States
We enable the knowledge to travel and not the technicians. Remote Expert Services in the Mining Industry.
Back to blog homepage
About oculavis SHARE › Features › Integration › White-Label Creator › Cloud and On Premise Setup › Licensing and Pricing › Book a live demo Start a Free Trial
oculavis SHARE for the industry › Service Solution for Machine Builders › Maintenance, Repair & Overhaul › Partner Solutions › Customer Success Solutions
Built on innovation › Augmented Reality › Smart Glasses › Industrial Internet of Things › Artificial Intelligence
Find out more › App Downloads › Events › oculavis tube › Blog cast › Jobs › About oculavis
🇩🇪 🇺🇸 🇫🇷