The following describes what mixed reality (XR) and virtual reality (VR) mean and looks at the differences between the various technologies.
It starts with a brief look at the origins of virtual reality. The main part describes how the technologies work and how they differ. It closes with an overview of which of these technologies are used at Reiser Simulation and Training GmbH (RST) today and in future.
The origins of virtual reality
The beginnings of virtual reality date back to 1962. At that time the first immersive system was presented by Morton Heilig (a cameraman by profession). “Immersive” comes from the word “immersion”, meaning that an individual dives into something or becomes absorbed in it. A distinction is made between mental and physical immersion.
Mental: a person becomes deeply absorbed in an action mentally. That happens, for example, when reading a gripping book or watching an adventurous film.
Physical: certain input and output devices address several of a person’s senses. They therefore immerse themselves in another world. A head-mounted display, also known as VR glasses, for example, blocks out reality, and the user experiences a high degree of physical immersion because wherever they look, they are always in virtual reality. This can be reinforced by audio output devices.
Sensorama by Morton Heilig
(Sources: left – U.S. Patent #3050870, right – telepresence.com)
The Sensorama was specified by Morton Heilig in 1955 and the first prototype was published in 1962. It is regarded as the starting signal for the development of virtual reality. It was about the size of an arcade machine. A wide variety of sensory impressions were simulated with it and it supported, among other things, the display of three-dimensional content that is standard today, in colour and with stereo sound. The Sensorama was also able to simulate vibrations, smells or wind, for example.
The Telesphere Mask was a first step towards the VR headsets we know today and was likewise developed by Morton Heilig. It was used in combination with the Sensorama and gave the user an entirely new freedom of movement and a wider field of view.
Telesphere Mask by Morton Heilig
(Source: U.S. Patent #2955156A)
Sword of Damocles by Ivan Sutherland
(Source: etsanggarp.blogspot.com)
“The Sword of Damocles” resembled the Telesphere Mask, but in 1966 Ivan Sutherland (a computer scientist) was the first to fit the device with head-tracking technology. That laid the foundation for the levels of interaction we know from VR headsets today.
How they work and how they differ
After that brief look at the historical origins, the details of the various terms are set out below.
The reality-virtuality continuum is important for placing the different technical terms. It describes the gradual transition from pure reality all the way to a computer-generated, fully virtual world. Reality and virtual reality form the left and right boundaries. All states in between are assigned to mixed reality. Augmented reality (AR) and augmented virtuality (AV) fall into this range.
Reality-virtuality continuum
VR can be described as computer-generated reality. Using a graphics engine (part of a computer program for rendering computer graphics), a VR headset delivers high-resolution images directly in front of the user’s eyes. The process is called rendering, and it converts 3D scenes into sequences of 2D images. The headset shows each eye a slightly offset image, which creates the three-dimensional impression in the mind (stereoscopic vision). Powerful graphics cards are needed to achieve the frame rate required for this. To heighten VR, hearing is addressed with acoustic signals alongside vision.
With AR, a real environment is extended or supplemented with inputs such as sound, graphics or video. Digital content is added to existing reality (our surroundings) with the help of devices such as a smartphone or tablet.
Using camera images and the relevant apps, digital content is overlaid on the surroundings. AR has three characteristics:
- merging of reality with virtual elements
- interaction in real time
- three-dimensional
AV is the opposite of AR. With AV, real objects are inserted into a virtual world. These objects can also be controlled by the user. A very simple application can be seen in many film productions. Actors perform in front of a blue or green screen. Software solutions mean that the actor is ultimately shown in a computer-generated, virtual world.
AV application with green screen
All of these techniques have advanced considerably in recent years, which has also led to greater use in private households (mostly in the case of VR headsets). The everyday use of AR is likewise increasing, for example when translating texts in real time with a translation app. AR is also used in industry. A technician can have installation and maintenance information shown in a headset, for example. That allows them to operate or service unfamiliar devices or machines intuitively and to complete their tasks efficiently.
Technologies at RST today and in future
As part of the continuous improvement of the full-flight simulator product group, AR technology was successfully implemented a few years ago.
In the full-flight simulators, the third crew member (hoist operator) wears AR glasses. The training value is increased in that the hoist operator sees the hoist cable and the person to be rescued, an avatar, when looking out of the helicopter door. Depending on the inputs at the instructor operator station, the avatar can give hand signals to which the third crew member has to respond.
RST currently has various development projects under way to extend the existing product portfolio with the established mixed reality technologies. New products are also being developed to cover the wide-ranging training needs of our customers. We want to play a major part in shaping the future of training systems and devices, to increase realism further and also to offer significantly smaller systems, and thereby to make a clear contribution to flight safety.
RST metaverse 2024+
We are happy to answer any questions on this subject. We will also report on new aspects of the topic in future.
Michael Holz
Head of Operations
