Mainstream virtual reality is still in its infancy but it's been tackling the final frontier – space – for years. Now the latest hardware, such as the Oculus Rift, HTC Vive and Samsung Gear VR headsets, combined with sophisticated 360-degree filming techniques, is finally bringing space exploration to the masses.
VR's ability to produce large-scale 3D environments not only offers armchair astronauts a glimpse of what life is like in space, it is now also more helpful than ever at aiding real-life astronauts in their jobs both on Earth and out there.
What's more, augmented reality can offer spacemen and women a new perspective by merging the worlds of Earth and space. Compared to VR, though, augmented reality still has some way to go before we start to see smartglasses and AR helmets both on our faces and in our homes.
While virtual reality has been relatively slow to catch on in the mainstream, NASA has been using it for more than 25 years because it's simply one of the best ways to replicate space while remaining safely on home soil.
"Simulated environments have always been important in astronaut training," explains Jason Crusan, director of NASA's Advanced Exploration Systems Division.
The Mercury, Gemini and Apollo astronaut crews all spent at least one third of their training time in simulators and contemporary crews use VR simulations to train for tasks on the International Space Station (ISS).
Early NASA headsets were improvised affairs – the first prototype of the Virtual Environment Workstation headset was built from a motorcycle helmet – and the American space agency has continued to update the tech involved. Astronauts now use NASA's Virtual Reality Lab (VRL), located at the Johnson Space Center, to train for missions aboard the ISS. Using a headset, real-time graphics and motion simulators, astronauts train to carry out tasks during microgravity spacewalks.
A vital part of the training involves using their powered jetpack – the Simplified Aid For EVA Rescue (SAFER) – which carries very limited fuel, to navigate their way back to the ISS should they get stranded in space.
The ability to recreate a life-size 3D environment makes VR ideal for astronaut training and now NASA is looking at using augmented reality to keep reality in the frame.
Augmented reality on the ISS
NASA's Scott Kelly wearing HoloLens
AR differs from VR in that it overlays graphics into your view of the real world, making it ideal for enhancing training simulations. NASA has teamed up with Microsoft for Project Sidekick, which sees astronauts testing out the HoloLens headset on the space station. The idea is to enable experts on Earth to see what the astronaut sees during complex tasks so that they can give them guidance over Skype.
Our Mars explorers have been stuck on one side of a computer screen
What's more, holographic instructions can be used to assist astronauts in completing more complicated manoeuvres. The tech was recently tested out by crew on the ISS, including British astronaut Tim Peake, and it's hoped that in future it will boost the astronauts' efficiency and cut down on the amount of pre-mission training needed.
Also with Microsoft, NASA has developed software called OnSight that enables Earth-bound astronauts to virtually work on Mars using a HoloLens and data from the Mars Curiosity rover.
"Previously, our Mars explorers have been stuck on one side of a computer screen. This tool gives them the ability to explore the rover's surroundings much as an Earth geologist would do field work here on our planet," said Jeff Norris, JPL's OnSight project manager.
It's not all about helping astronauts to prepare for missions, VR and AR can also give us regular Earth dwellers an idea of what life is like in orbit.
OnSight for Earth-bound astronauts
NASA is already producing 360-degree videos for Facebook, such as its popular Curiosity Rover video, which users can navigate around by dragging the cursor or tilting their phone. And it's only a matter of time before the space agency is producing more content that can be viewed on a VR headset. The European Space Agency has also been gradually releasing some stunning 360-degree videos of each module of the ISS, which look great when played on YouTube and viewed using Google Cardboard.
In collaboration with MIT's Space Systems Laboratory and Fusion Media, NASA is developing the interactive Mars 2030 experience for Oculus Rift, Google Cardboard, and Samsung Gear VR. The experience, which debuted at SXSW 2016, will be free and use real operational and hardware concepts that NASA and MIT are currently studying and building.
"Beyond practical uses for training, virtual reality offers us a compelling method to share the work we've been doing to design sustainable human missions and to inspire the next generation of pioneers in space," said Crusan.
Mars 2030, shown above in the lead image, is first person and controlled by an Xbox controller when played on the Rift. The idea is that it's part game, part education app with landscapes and gravity mechanics created from real NASA research and data, including that collected by a Mars rover.
Travelling around, the player can experience huge Martian tunnels that run for hundreds of kilometers as well as explore a simulation of what Mawrth Vallis Crater, which used to be a lake, might look like now. Next to the likes of The Martian experience for Gear VR, this will be the real deal when it's released later this year.
The space app race
Liftoff VR for Google Cardboard features a SpaceX Falcon rocket
While hardcore gamers and those with money to burn might invest in a pricier headset like the HTC Vive or Sony PlayStation VR, those who just want a nose around the galaxy might be limited to the much more affordable Google Cardboard.
There's already a rapidly increasing pool of VR apps to choose from, including the exhilarating Liftoff VR app for Cardboard that lets you take part in the launch of a SpaceX Falcon rocket, which you then have to land on a floating jetty in the sea using an interactive target.
VR Planetarium lets you float around the planets in our solar system, and Gravity Space Walk on Cardboard is also simple but effective. Titans of Space on the Samsung Gear VR is also worth a go, as it lets you autopilot around the solar system on a tiny one-person spaceship, stopping to look at planets, moons and stars along the way. The graphics aren't amazing, but the sense of scale is genuinely giddying.
There's even a Moon Landing app, which shows a tiny Apollo 11 lunar module landing on your Apple Watch when viewed through your iPhone screen. It's pretty basic, and a little buggy but it's one of the few examples of augmented reality space apps so far. There's definitely scope for more AR-based space history from the upcoming HoloLens and Magic Leap.
While some of the apps are fun, but not necessarily scientifically accurate, a team from the Royal Observatory of Edinburgh is developing StarSightVR to help people learn more about the night sky. This will take the form of an online presenter-led planetarium show that can be viewed from home, on Oculus. The makers are planning to start trials this year.
Meanwhile, the Kickstarter-funded Space VR project is sending a 3D 360-degree camera to the cupola of the ISS so that Earth-bound space nuts can experience a virtual space station from their own homes. The makers hope to make the content available on iOS and Android, plus all existing VR headsets, including FOVE, the first to include eye tracking.
Educating astronauts of the future
Immersive VR's Apollo 11 experience
As well as showing us what life looks like on the space station, VR is a great way of communicating the history of space travel. Also crowdfunded on Kickstarter, the Oculus-based Apollo 11 VR experience, created by the Immersive VR Education studio, uses archive footage from NASA combined with 3D computer modelling to recreate the 1969 moon landing. While the virtual lunar mission will definitely appeal to those who watched the moon landing live on TV, it's primarily aimed at students.
"With the cost of the hardware now falling into line with what is affordable, we see virtual reality as being one of the biggest technological revolutions since the introduction of the mobile phone," says Immersive VR Education's Sandra Whelan, adding, "It will give students a deep appreciation for the vastness of our universe and the wonders that lie within it, much more than reading a book or watching a video on a television screen."
Tested out by Apollo 16 astronaut and the 10th (and youngest) person to walk on the moon, Charlie Duke, who described it as "wonderful", the experience has certainly earned its astronaut wings. The makers are now working on a Mars experience.
And VR isn't just for older students, thanks to the revamped View-Master. The classic stereoscopic children's toy recently got a virtual reality update and offers 'experience packs' for Wildlife, Destinations and Space, with the latter offering kids the chance to interact with planets, spacecraft and star constellations.
Zooming around the Apollo Lunar Landing Module and landing it on the moon is far more immersive and, hopefully, memorable than any textbook could ever be. And the View-Master's space pack is one of the best uses of educational VR we've seen so far.
To 360 and beyond
VR and AR are slowly opening up the mysteries of space to those who can't witness them first-hand, but what does the future hold? As the technology improves, and the gap between what space agencies are using and what space nerds can afford looks set to close, there's no doubt that immersive experiences could bring a unique energy and enthusiasm to learning about space.
The likes of SpaceX and Virgin Galactic could be a few failed launches away from greatness. NASA plans to send humans to an asteroid by 2025 and to Mars in the 2030s. Even missions like the Curiosity Rover need humans behind the tech, (almost) every step of the way. Space apps for virtual and augmented reality aren't just built for excitement and education, they're built with preparation in mind too.