Virtual reality flight simulations in Carleton University’s Advanced Cognitive Engineering (ACE) Laboratory have pilots airborne without their feet ever leaving campus.
These simulations aim to help researchers design a cognitive health screening tool to reimagine aviation safety. Carleton’s ACE Lab developed the CANFLY test to measure pilots’ cognitive abilities while flying to ensure they can safely support themselves and avoid accidents.
CANFLY’s lead researcher, Kathleen Van Benthem, said up to 80 per cent of accidents in general aviation — a stream of aviation focused on non-commercialized flight, like search and rescue or recreational flying — are caused by cognitive error.
“The decisions pilots make, knowing what’s around them, remembering to do things — all that is [the] reason why other accidents happen, and that’s the majority. That’s something we can work on,” Van Benthem said.
The general aviation accident rate is much higher than in commercialized aviation, Van Benthem said. “In Canada, on average, there’s an accident every 11 days and every couple weeks that involves a fatality.”
CANFLY uses virtual reality to mimic how it looks and feels to fly a plane. Pilots put on a headset and see graphics of a virtual cockpit window to see a video game-like simulation of Ottawa as they fly over.
“It actually feels like they’re sitting there at the front of the seat, and they can look around and when they take off, everything moves around,” said Emily Larkin, a psychology master’s student and CANFLY researcher.
Pilots use their flight skills to navigate the gamified world, listening to radio chatter and managing controls scaled to be accurate to planes they’re accustomed to flying. They simultaneously complete more complicated tasks such as recollecting specific details of radio calls and maintaining the correct altitude and airspeeds, which vary throughout the flight.
“They take off at one airport, they go to another airport, fly around and then they keep going,” Larkin said. “We pause it at certain times, the same time every flight, so we can gather the same information from people and compare it.”
Test results help answer researchers’ questions intended to reveal aspects of pilots’ cognitive abilities. More than 250 pilots have been tested and 21 students contributed to the research throughout the study’s 15 years and counting lifetime.
“We’ll ask them, ‘how fast were you going?’ or, ‘where were you when we just paused the scenario? Can you point it out on a map?’” Larkin said.
Measuring how many “critical incidents” occur, which include crashes, getting lost or significant equipment damage, is another critical part of the ACE Lab’s study, Van Benthem said.
“This way we can make things really tricky, see what we can bring out in terms of people’s skills, and then we see how they did on here: does that predict the real-life risk?”
The data collected measures the pilot’s likelihood that a cognitive incident will happen and compares pilots’ virtual flight results to their responses to the cognitive questions.
The researchers then look at these two pieces of information, compare them and decipher meaning from the comparisons, Larkin said. The results are used to create personalized predictions for pilot behaviours and overall cognitive patterns.
“Do these two things relate to each other in a certain direction? How strong is that relationship? … Does that relate to their history that they have?”
To create prediction models, the ACE Lab uses modern machine-learning AI tools, which are similar to how streaming platforms use prediction techniques to recommend shows, Van Benthem said.
“[Streaming platforms] are collecting data on you, and they take that and they crunch all those numbers, and then they’ll predict by saying you might like to watch this or this,” she said.
The same data collection, crunch and output is used in CANFLY, Van Benthem said. Instead of a recommendation for the latest reality TV show, the information output is pilots’ likelihood for cognitive incident occurrences. It can also indicate poor results in one or more of the abilities measured during flight, such as situation awareness or task management.
“You can’t have all those [full-scale] simulators across the country and get pilots in there and have somebody assess them. It’s just a lot of work,” Van Benthem said.
By using virtual reality simulation, the space and resources needed are smaller.
Simulation specialist James Howell said although it doesn’t look very impressive, the smaller design created greater immersiveness because of their ability to scale the control placement to where they usually are in a real aircraft.
“When you look at it, it’s just kind of holes cut in the desk,” Howell said. “But the important thing is that we get the exact placements that our pilots are used to in the actual cockpit itself.
“We need to have something where you get an actual touch feedback with the real world, to get that immersiveness into it,” he said.
In April 2024, CANFLY was scaled down further to become an online test that can be taken remotely or in addition to the virtual reality simulation. Larkin said the data collected through the remote test aims to validate the results from the simulation component of the study.
Van Benthem said a remote test might even be better for pilots. “We could really, what we call it, ‘put it in the hands of pilots’ so they could then assess themselves whenever they wanted to.”
The pilots don’t currently receive their test results. But according to Larkin, pilots said it’s given them the opportunity to reflect on valuable skills when flying.
“It gives them an opportunity to really think and go, ‘oh, OK, how important is that?’ Or, ‘how am I doing?’” Larkin said. “It’s getting them to reflect on things that they’re used to doing or used to hearing, but not necessarily remembering.”
Featured image by Kyla Silva.
