Remote assistance, remote support, and visual assistance
09/09/2024
Explore the differences between remote assistance, remote support, and visual assistance to enhance the effectiveness
5 approaches to combine Augmented Reality with Artificial Intelligence for better servicing machinery and equipment
Written by
Martin Plutz
Martin Plutz
- TECHNOLOGY
- 7 min. reading time
Artificial intelligence (AI) is one of the hype topics that has been driving machinery and equipment manufacturers not just since yesterday. Expectations were lowered by the growing awareness that artificial intelligence in products and processes is not easily available and that the effort required to develop appropriate AI based systems should not be underestimated. In addition to the actual algorithms that make machine learning possible, deep domain-specific knowledge is needed above all to enable these algorithms to produce meaningful results.
Predictive analytics as main field of application of artificial intelligence in the service operations
From the point of view of most machine builders, it is meaningful, for example, to make an estimate of the time of failure of a machine or specific components. This data-based prediction of machine states is usually referred to as predictive analytics, and the knowledge advantage that can be achieved can be used to optimize maintenance times, the replacement of spare parts or the early incorporation of planned downtimes into the end customer’s production planning and control. But what happens if problems occur during planned maintenance or replacement of spare parts? The employee from the end customer’s maintenance department may not know exactly how to replace a specific spare part or how to carry out maintenance correctly. Or the travel time of a technician of the machine manufacturer to the end customer takes so long or may not be possible at all due to travel restrictions that the time advantage of a prediction based on predictive analytics is simply eaten up.
From data level to the human level of action
These examples make it clear that artificial intelligence can quickly result in demand for human action. This is exactly where Augmented Reality based solutions come in to optimize these human interactions, such as the manual maintenance or repair mentioned above. Visual remote support with Augmented Reality annotations can be used, for example, to ensure that the expert of a machine manufacturer accompanies a complicated maintenance operation from a distance to ensure that the work is carried out correctly. If the machine manufacturer has already prepared the knowledge required for the maintenance in the form of instructions, the correct work processes can be displayed step-by-step on a tablet, smartphone or smart glasses to an onsite operator at the machine. This means that remote guidance by an expert can be replaced by self-guidance based on digital instructions and workflows for standardizable procedures. The challenge in this overall process is to make the right information available at the right time and in the right place. This can be achieved, for example, by integrating predictive maintenance with augmented reality systems. When the need for action is triggered by a data-based prediction, maintenance staff on site are informed by email or push notification, work through an Augmented Reality-based workflow for guided maintenance and, if necessary, consult a specialist from the machine manufacturer.
One glance says more than a thousand data points?
Thus, the described processes can be integrated into a hybrid Smart Service Chain, which combines the strengths of artificial intelligence in the analysis of data with the human strengths in the execution of complex decision and work situations. Augmented Reality based solutions such as visual assistance or digital workflows offer a perfect complement to ensure a fast ability to act when malfunctions of machines and systems are detected. They can also be used to evaluate the prediction of algorithms through a human view of the situation.
Computer Vision for the automated extraction of information from images
This is exactly where artificial intelligence approaches can be used to supercharge this human view of a remote expert with special abilities. In contrast to the predictive applications described so far, here the manifestations of AI are located in the field of computer vision and in a certain way expand the senses of the user, as the following examples show:
- The automatic extraction of text on photos or in a video call using Optical Character Recognition (OCR) makes it possible, for example, to recognize the serial number of a machine so that it does not have to be entered manually.
- If the text is written in a language that the user himself does not understand (e.g. a safety instruction on a machine or similar), automatic translation services based on artificial intelligence can be used not only to automatically recognize a text, but also to display it directly in another language.
- Sometimes, however, an expert who is remotely connected does not want or is not allowed to see everything. For example, to ensure the privacy of people who happen to walk through a scene, AI-based face recognition approaches can be used to make people unrecognizable in images and videos.
- Furthermore, AI-based approaches can be used not only for the recognition of faces, but also for the recognition of patterns e.g. for quality assurance of processes or for the recognition of components e.g. for the automatic detection of spare parts that have to be identified during maintenance.
A brief look into the future
These specific examples not only show applications of artificial intelligence in the service of mechanical and plant engineering but are also intended to provide a realistic orientation about what is possible today. Of course, there are already more far-reaching ideas than those presented in this article, such as completely replacing a remote expert with an algorithm that visually analyzes a live video stream from a technician on site in real time, uses Natural Language Processing (NLP) to understand the spoken word, derives the right recommendations from it, and uses text-to-speech technologies to communicate to the employee what needs to be done. This can serve as a vision for the future, but with the technological possibilities of today, it is unlikely to become a realistic option for the future soon. And the question of whether we want such a vision is still to be answered separately as an individual, company and society. How do you see the future of AI and AR?
Would you like to know which of these ideas are already included in oculavis SHARE and what our roadmap regarding the use of artificial intelligence looks like? You are invited to contact us and start a conversation on these topics!
FURTHER ARTICLES FROM THIS CATEGORY
The Industrial Metaverse. Definition, applications and potentials in mechanical engineering & the industry
11/09/2023
Discover how the Industrial Metaverse is already being applied in manufacturing today and the concrete
Next Level Video Support: Remote Factory Acceptance Testing (FAT) with the Multi User Call in oculavis SHARE 9.0
02/19/2022
The main innovation of the latest version 9.0 of the remote process platform oculavis SHARE
Privacy and safety proofed visual assistance with 5G and oculavis SHARE
01/28/2022
Together with the Chair of Digital Communication Systems at Ruhr University Bochum, SMS Digital GmbH,
Empowering shop floor workers, service technicians or engineers with digital workflows in oculavis SHARE
10/24/2020
The multiple award-winning Augmented Reality based remote support platform oculavis SHARE is extended by a
Microsoft Edge Browser switches to Chromium and is now supported by oculavis SHARE
02/06/2020
On January 15th 2020, Microsoft Edge launched a new, Chromium-based browser, with the purpose to
TECHNOLOGY
Remote assistance, remote support, and visual assistance
Explore the differences between remote assistance, remote support, and visual assistance to enhance the effectiveness
Anna Layko
09/09/2024
TECHNOLOGY
The Industrial Metaverse. Definition, applications and potentials in mechanical engineering & the industry
Discover how the Industrial Metaverse is already being applied in manufacturing today and the concrete
Anna Layko
11/09/2023
TECHNOLOGY
Next Level Video Support: Remote Factory Acceptance Testing (FAT) with the Multi User Call in oculavis SHARE 9.0
The main innovation of the latest version 9.0 of the remote process platform oculavis SHARE
Martin Plutz
02/19/2022
TECHNOLOGY
Privacy and safety proofed visual assistance with 5G and oculavis SHARE
Together with the Chair of Digital Communication Systems at Ruhr University Bochum, SMS Digital GmbH,
Daniel Mirbach
01/28/2022
TECHNOLOGY
Empowering shop floor workers, service technicians or engineers with digital workflows in oculavis SHARE
The multiple award-winning Augmented Reality based remote support platform oculavis SHARE is extended by a
Nico Swienty
10/24/2020
TECHNOLOGY
Microsoft Edge Browser switches to Chromium and is now supported by oculavis SHARE
On January 15th 2020, Microsoft Edge launched a new, Chromium-based browser, with the purpose to
Daniel Mirbach
02/06/2020
TECHNOLOGY
Is the hype over? The consolidation of the smart glasses market
The market for augmented reality headsets, also known as smart glasses has gone through a
Martin Plutz
01/07/2020
TECHNOLOGY
5G Retrofit of Machines for Realtime Applications of Augmented Reality
Together with seven partners from research and industry, oculavis GmbH is developing Augmented Reality (AR)
Nico Swienty
11/06/2019
