VTT has developed a tool to support astronauts in complex repair tasks with the assistance of augmented reality, AR

The European Space Agency (ESA) has been testing a tool developed by VTT as part of its training and operations system. The tool, using augmented reality, has been developed to support astronauts in training and operations during a space flight.

In space, all astronauts perform maintenance and repair tasks. To support them, they have completed thorough training, and have access to detailed instructions on computer screen and, if necessary, to experts on ground. A permanent contact with ground would undoubtedly improve the crew performance, but communications is limited on exploration missions to Moon and Mars.  However, delving any deeper into operations in training could make the training period unreasonably long.

VTT has developed a tool that contributes to ESA’s goal to give unambiguous guidance to astronauts in complex maintenance and repair tasks both during the space mission and when training for it before the flight. The use of augmented reality (AR) makes the tool unique. The work instructions are provided as text, graphics or speech to the astronaut’s Microsoft HoloLens-AR head mounted display. For example, an arrow may directly indicate a lever in the device being serviced and show in which direction it is to be turned. The astronaut may also access the device’s service history on his or her AR display as well as its status report that is being observed both in space and on the ground.

“The tool we have developed reduces the risk of human error and significantly speeds up the work performance. It is also extremely well suited for both supervised and individual training of astronauts. The tool uses augmented reality in a new way that is of great help in demanding maintenance and installation tasks of critical importance with a view to security or financial costs in other sectors as well, such as mines, paper mills and nuclear power plants,” says Principal Scientist Kaj Helin from VTT.

AR tool could train astronauts on the way to Mars

The cooperation between ESA and VTT has been close for well over a decade. In the latest project, which began in September, the AR tool will be integrated into the training and operations system currently under development.

“There is a strong interest in the AR field from ESA and the way VTT works with challenges of interest to ESA. So VTT is on a path that we ESA see as very fruitful – both for spacecraft testing and human space flight applications,” says technical officer Mikael Wolff from ESA’s European Space Research and Technology Centre ESTEC.

The importance of the AR tool will increase as the distances become longer on space missions. At the moment, a space mission lasting slightly over one year is preceded by training that takes about two years. When flying to Mars, the outward journey alone will take more than six months, and when the mission arrives on the planet, the crew has to manage novel environments and new technologies. It is clear that, using traditional training methods, the duration of training would need to be multiplied.

With the AR tool, the astronauts could practice the maintenance and repair of equipment to be used on Mars during the long outward journey. VTT has already created a future scenario where astronauts can practice the maintenance of, for example, the Mars rover in a setting resembling the real environment.

The initial work was carried out in a joint VTT and ESA project with the contract number 4000125238/18/NL/AF/as, and the work will continue for the next 18 months in the project 4000127710/19/NL/GLC that was launched in September. Another partner in the project is the Irish SKYTEK, which is developing mobiPV, the training and operations system for ESA astronauts.