Immersive TV, defined as TV that offers an enhanced sense of immersion in the video scene being watched, is a word used today to talk about 3D or the Ultra High Definition (UHD) format. And in fact, the more the video format improves, the more optimal is the image resolution and the more palpable the illusion of depth, enabling viewers to experience image and sound as closely as possible to what they are in the “real” world.
Being at the centre of things and everywhere!
Right now, two immersive video technologies are being developed simultaneously, one being at a more advanced stage than the other. 360° video has effectively reached a fairly mature stage and the amount of content based on the technology is now growing fast. “Technically, the idea is to capture a scene using a camera filming from different angles – a spherical device holding several cameras that film a scene from different viewpoints,” explains Henri Sanson, head of Data & Knowledge research at Orange Labs. “As a result, viewers find themselves in the centre of the scene, although they are limited in their movements: this is what is called the three degrees of freedom, where users can look up, down, or to the side, but can’t move their heads in those directions. 360° video is particularly well-suited to use with a headset – a virtual reality headset – but even then, this does not overcome the limitations of the three degrees of freedom.”
The second emerging application is super multi-view video. Unlike 360° video, characterised by a single framing system, super multi-view uses a network of cameras surrounding a scene or part of a scene. “In this case, we talk about convergent filming, which is a bit different from immersion: in convergent, you are no longer in the centre but everywhere. Users can navigate freely and choose their point of view completely independently. Super multi-view, which gives rise to a raft of technical challenges in terms of camera calibration and timing, data transmission, recreation of intermediate views, etc., is still at the research stage.” But at Orange, our Research people are deeply involved in this area, especially in the standardisation aspects and are working alongside research institute b-com.
No image without sound and vice versa
Images are nothing without sound, especially as, while our binocular field of vision covers 120°, our ears hear in full-on, all-round, 360° sound! Sound recording systems equivalent to the ones we mentioned above for filming now exist. As a result, 360° sound recording helps to capture representations of the three-dimension sound field. “The issue is then to code, synchronise and align the sound recording with the filmed images so that users can perceive the sound emanating from an object in a scene in line with the image, and independently of their viewing angle.”
Immersive Video/TV hand-in-hand with virtual reality
At the same time as this new video content is becoming slicker and gaining traction, the emergence and development of virtual reality are providing a significant opportunity for adopting and expanding immersive video/TV. “At the outset, virtual reality was largely limited to computer-generated imagery, but CGI is very expensive. The advent of 360° video means we can now create virtual reality content for much less. However, 360° video does not necessarily need VR to exist and be watched – a TV screen is all you need.”
And what does the future hold? Henri Sanson notes that virtual reality, augmented reality, and immersive TV are all converging towards a single point and will flesh out a mixed reality, where computer-generated components can be recomposed in a real scene, while it will be possible to place objects from a real scene in a computer-generated environment. “A look at Microsoft’s demonstration of holoportation will convince you: the computer-generated and the real have never been so close…”
