APM Terminals begins exploring Virtual and Augmented Reality technologies

APM Terminals begins exploring Virtual and Augmented Reality technologies
Microsoft Hololens, Credit: Guido van Nispen

APM Terminals has started to examine the use cases of Virtual Reality (VR), Augmented Reality (AR) and Extended Reality (XR) technologies in the ports sector, with a pilot project involving Microsoft Hololens headsets in the pipeline.

The terminal operator believes by utilising these technologies, it can achieve considerable benefits in engineer support, health and safety and its “go and see” (Gemba) philosophy with regards to leaner forms of working.

With the COVID-19 pandemic accelerating the use of remote working practices, APM Terminals found that using VR, AR and XR technologies could provide a “richer” experience for different employees across the company.

Gavin Laybourne, chief information officer at APM Terminals, told CM: “It’s about offering more data and the richness of data that our team can have instantaneously. An engineer on a crane in Los Angeles can be in contact with an engineer in Finland.

“Not only can both look at the same thing, but the engineer in Finland can point towards using his/her knowledge of documentation or experiences and send it in real time to that other person in Los Angeles.”

Whereas previously the company’s engineers may have used a GoPro camera or an iPad to liaise with colleagues when working remotely, these newer technologies give them more tools to be effective, noted Laybourne.

There are also potential cost benefits while reducing the number of people on the terminal, which is deemed better from a health and safety point of view.

The technologies further allow the port operator to evolve its health and safety training by providing a virtual terminal playground for trainees so they can experience different scenarios from home.

In the hectic environment of a container port, artificial intelligence (AI) can be used in conjunction with VR, AR and XR to spot Health, Safety, Security and Environment (HSSE) issues, enabling intervention in real time.

Laybourne added: “It’s not just using this to look, it’s also connecting other technologies such as AI. In one of our terminals, we’ve got more than 50 cameras running 24 hours a day using AI with Microsoft to spot HSSE issues such as people coming out of their trucks or being in walking in areas they shouldn’t be.

“This is flagged straight away when there’s an issue and an intervention is needed. If we have this technology bolted onto AR, VR and XR, then you get an even more powerful tool from an HSSE point of view.”

With regards to the Gemba way of working, APM Terminals is keen to “go and see” how operators are doing their daily work to gain a greater understanding of where improvements can be made.

In the past, company personnel have been flown to specific terminals for physical go and sees, but the pandemic spurred a need for better remote tools.

Laybourne explained: “On a mobile phone, you can use the camera to go see. But with these technologies, somebody not on the terminal can wear a headset and effectively be in the terminal during a live go see. They can be projected what should happen versus what they see in reality.”

Overall, he is convinced that VR, AR and XR will be part of the daily business in the coming years, as the efficiency benefits become evident.

Concerns over the high level of connectivity required to make the technology viable have dissipated, in his opinion, due to its constant evolution, making stable connectivity the essential factor in deploying the solution.

With regards to bringing APM Terminals’ workforce on board, Laybourne noted: “Part of the opportunity is how we can engage with the front line so they don’t see this as a threat. Everyone wants to be safe and we need to understand that it helps them, not hinders them.”