Summary
Large scale manufactures such as The Boeing Company spend large sums of money each year to train new personnel in complex assembly tasks. This often requires time spent in a classroom environment, reading manuals, and studying under an expert assembler. All of this costs both time and money, and reduces the productivity of the assembly line. The Factory of the Future research and design project funded by The Boeing Company sought to reimagine this training process and reduce training costs by employing augmented reality hardware.
Previous work
Starting in 2012, Iowa State University began investigating the application of AR technology to assembly training. Since then, rigorous research has been conducted through five research studies to understand the trade-offs of different AR hardware, interface elements, and navigation techniques. Ultimately, these studies confirmed the benefits of using an AR Tablet based system over electronic manuals to deliver assembly instructions. These studies also helped to identify more effective interface elements for navigation and occlusion in AR assembly tasks. Based on the findings of these studies, AR guided assembly instructions were developed and tested using the Microsoft HoloLens Head-Mounted Display.
Microsoft HoloLens Guided-Assembly Instructions
A study was conducted for which AR instructions were developed for the Microsoft HoloLens AR Head-Mounted Display. This version of the AR work instructions allowed the user to operate in a more hands-free manner during the assembly task while still seeing the spatially registered instructions. The team hypothesized that the hands-free, contextual work instructions provided by the Microsoft HoloLens would reduce time and errors in the assembly task compared to previous Tablet based AR instructions.
Development
The interface and interaction development for the HoloLens assembly application was created using Unity 3D and the HoloToolkit, which has since been replaced by the Microsoft MixedRealityToolkit. The HoloLens has its own native tracking system, which was paired with Vuforia to enable 2D image tracking. Two image targets were placed on the walls and used to initialize the position of the HoloLens in the environment. When the image targets were not in the HoloLens field of view, the device’s inertial measurement unit was used to determine its position relative to the rendered AR content. Users interacted with the AR interface by gazing at the target and then activating a Bluetooth clicker attached to their non-dominant wrist. This kept the user’s hands free for assembly tasks.
Method
Participants completed a mock wing assembly task using the Microsoft HoloLens. The assembly required the participants to assemble wooden parts using large metal fasteners. Completion time, error count, and Net Promoter Score (NPS) were measured and recorded. This data was compared to data from previous studies conducted at the Virtual Reality Applications Center, which employed Desktop Model Based Instructions (MBI), Tablet MBI, and Tablet AR instructions for the same task.
Results
The use of HoloLens AR instructions led to time savings of 16% over the Tablet AR instructions on first time assemblies. HoloLens users also had lower error rates than non-AR users. Despite the performance benefits of the HoloLens AR instructions, this condition had a lower NPS than the Tablet AR group. The qualitative data showed that some users thought the HoloLens device was uncomfortable and that the tracking was not always exact.
Conclusion
Although the users favored the Tablet AR condition, the HoloLens condition had significantly faster assembly times. The authors recommend using the HoloLens for complex guided assembly instructions with minor changes, such as allowing the user to toggle the AR instructions on and off at will.