Aircraft Inspection with Virtual Reality
Posted: 2018-8-27
By:
Holly Jordan
Source:
AFRL Materials and Manufacturing Directorate
AFRL researchers are hard at work finding unique ways to make aircraft maintainers’ lives a little easier through a technology many associate most closely with video gaming.
Using augmented reality, the Materials and Manufacturing Directorate is exploring ways to simplify and expedite Nondestructive Inspection, or NDI. NDI is the inspection of materials or structures to detect flaws or other undesired conditions without causing unwanted damage.
NDI is a complex process, and inspectors require significant training to gain certification. The inspection task can become even more difficult when inspectors are required to access internal hard-to-reach regions of an aircraft to inspect for cracks or other defects. They accomplish this task using a variety of techniques, one of which is based on eddy current sensors. These sensors induce electrical fields on the surface of materials to detect surface-breaking flaws. Inspectors must guide the eddy current probe precisely and accurately to ensure the required regions of interest are properly covered. This task is not easy, especially in confined spaces, as they must simultaneously review data displayed on the inspection equipment and follow written instructions at the same time. It can be a tiring job at best, and overwhelming at worst.
"Even though the inspectors have everything they need, because they are looking at the monitor and technical documents while simultaneously scanning the surface, they are doing the inspection under very challenging conditions," said Nondestructive Evaluation Technology Lead Eric Lindgren. He added that this level of multitasking can be very fatiguing.
To help alleviate the burden, AFRL researchers took an innovative approach. Using commercial-off-the-shelf augmented reality systems, they are developing a capability to display all the information inspectors need within their direct field of view. This approach allows inspectors to view the area to be scanned, the inspection equipment display, and the technical documentation and other needed materials, all at the same time, thereby eliminating the need to scan several displays and/or hard copies in parallel. In this way, inspectors can increase their focus on the process at hand.
Users of this inspection capability can navigate and customize the display with a set of simple verbal commands and hand gestures while following menu prompts visible in the augmented reality field of view. According to the AFRL research team, the capability potentially can be developed further to highlight the critical areas to be inspected and, ultimately, provide a guide for how to accomplish the inspection. The team’s future development plan also includes guidance on the position of the inspection sensor to ensure the intended region of interest is properly assessed.
"We’ll be able to record the entire position history of the probe," said research engineer Alex Brown.
Although this is an early proof-of-concept effort, the potential benefits for maintainers are very promising.
"By using augmented reality, everything they’re doing is projected into their field of view, so they can see it floating in space right in front of them," said Lindgren. "Now they have something that makes it easier to guide and enhance the inspection process."
This task is being addressed by AFRL researchers in coordination with the AFRL Virtual, Augmented, and Mixed Reality for Aircraft Maintenance integrated product team. The collaborative group is working toward exploiting the full potential of the technology, improving accuracy, and investigating the best ways to integrate it into maintenance processes. The plan for the next few years is to develop additional capability, verify its performance in a laboratory environment, and demonstrate it in operational use.
As augmented reality technology becomes more advanced, the researchers will seek to make the wearable device smaller and, in its most ideal form, integrate it into safety glasses, which maintainers are already required to wear. As the technology is being matured, the AFRL team plans to partner with the maintenance community to enable further capability and usability improvements through field testing.
"We want to make this technology as easy to wear as glasses or a hard hat," said Brown. "If the maintainers are comfortable using the product, then we know it will be successful."
Ultimately, the team hopes to incorporate spatial registration technology, something that is currently being developed through another AFRL program. This capability would further automate, track, and record the position of the inspection sensors, thereby providing improved guidance and greatly enhancing data collection, analysis, and documentation.
"We want to simplify and facilitate," said Lindgren. "Our main goal is to make it easier and faster for the maintainers by enabling them to avoid the multi-input, multi-parameter, multi-tracking scenario that makes their job so challenging."
Image
AFRL Materials and Manufacturing Directorate researchers Capt. Sarah Wallentine and Alex Brown demonstrate the use of augmented reality for nondestructive inspections. AFRL researchers are developing a capability that uses the technology to display all necessary information for performing an aircraft inspection within the user's field of view, eliminating the need to view information from multiple sources simultaneously. (U.S. Air Force photo/David Dixon)
Using augmented reality, the Materials and Manufacturing Directorate is exploring ways to simplify and expedite Nondestructive Inspection, or NDI. NDI is the inspection of materials or structures to detect flaws or other undesired conditions without causing unwanted damage.
NDI is a complex process, and inspectors require significant training to gain certification. The inspection task can become even more difficult when inspectors are required to access internal hard-to-reach regions of an aircraft to inspect for cracks or other defects. They accomplish this task using a variety of techniques, one of which is based on eddy current sensors. These sensors induce electrical fields on the surface of materials to detect surface-breaking flaws. Inspectors must guide the eddy current probe precisely and accurately to ensure the required regions of interest are properly covered. This task is not easy, especially in confined spaces, as they must simultaneously review data displayed on the inspection equipment and follow written instructions at the same time. It can be a tiring job at best, and overwhelming at worst.
"Even though the inspectors have everything they need, because they are looking at the monitor and technical documents while simultaneously scanning the surface, they are doing the inspection under very challenging conditions," said Nondestructive Evaluation Technology Lead Eric Lindgren. He added that this level of multitasking can be very fatiguing.
To help alleviate the burden, AFRL researchers took an innovative approach. Using commercial-off-the-shelf augmented reality systems, they are developing a capability to display all the information inspectors need within their direct field of view. This approach allows inspectors to view the area to be scanned, the inspection equipment display, and the technical documentation and other needed materials, all at the same time, thereby eliminating the need to scan several displays and/or hard copies in parallel. In this way, inspectors can increase their focus on the process at hand.
Users of this inspection capability can navigate and customize the display with a set of simple verbal commands and hand gestures while following menu prompts visible in the augmented reality field of view. According to the AFRL research team, the capability potentially can be developed further to highlight the critical areas to be inspected and, ultimately, provide a guide for how to accomplish the inspection. The team’s future development plan also includes guidance on the position of the inspection sensor to ensure the intended region of interest is properly assessed.
"We’ll be able to record the entire position history of the probe," said research engineer Alex Brown.
Although this is an early proof-of-concept effort, the potential benefits for maintainers are very promising.
"By using augmented reality, everything they’re doing is projected into their field of view, so they can see it floating in space right in front of them," said Lindgren. "Now they have something that makes it easier to guide and enhance the inspection process."
This task is being addressed by AFRL researchers in coordination with the AFRL Virtual, Augmented, and Mixed Reality for Aircraft Maintenance integrated product team. The collaborative group is working toward exploiting the full potential of the technology, improving accuracy, and investigating the best ways to integrate it into maintenance processes. The plan for the next few years is to develop additional capability, verify its performance in a laboratory environment, and demonstrate it in operational use.
As augmented reality technology becomes more advanced, the researchers will seek to make the wearable device smaller and, in its most ideal form, integrate it into safety glasses, which maintainers are already required to wear. As the technology is being matured, the AFRL team plans to partner with the maintenance community to enable further capability and usability improvements through field testing.
"We want to make this technology as easy to wear as glasses or a hard hat," said Brown. "If the maintainers are comfortable using the product, then we know it will be successful."
Ultimately, the team hopes to incorporate spatial registration technology, something that is currently being developed through another AFRL program. This capability would further automate, track, and record the position of the inspection sensors, thereby providing improved guidance and greatly enhancing data collection, analysis, and documentation.
"We want to simplify and facilitate," said Lindgren. "Our main goal is to make it easier and faster for the maintainers by enabling them to avoid the multi-input, multi-parameter, multi-tracking scenario that makes their job so challenging."
Image
AFRL Materials and Manufacturing Directorate researchers Capt. Sarah Wallentine and Alex Brown demonstrate the use of augmented reality for nondestructive inspections. AFRL researchers are developing a capability that uses the technology to display all necessary information for performing an aircraft inspection within the user's field of view, eliminating the need to view information from multiple sources simultaneously. (U.S. Air Force photo/David Dixon)
