Three-dimensional video services are predicted to be the next big thing in entertainment and communication industries. Concepts using 3D video in science fiction have fueled the excitement around this topic. With the aid of recent advances in capture, display and compression technologies, 3D video is taking the next step towards becoming a reality in the consumer domain with realistic opportunities.
Applications of 3D Video
One of the most exciting applications that could be enabled by 3D video is telepresence (or virtual reality) - delivering to people the feeling of being immersed in a place other than their actual location. This could enable new ways of collaboration, and increase the efficiency of online meetings.
A similar application is the extension of traditional TV entertainment using displays capable of 3D rendering. The simplest form of 3DTV is a stereoscopic TV, where the display is capable of rendering two views, one for each eye, so the scene is perceived as 3D.
More sophisticated displays are able to render and display multiple views in such a way that the perceived 3D video depends on the user’s location with respect to the TV For example, the user can move his/her head to see what is behind a certain object in a scene while watching TV. This feature brings a new form of interactivity and an immersive feeling to TV viewers that have never been experienced before.
The Joint Video Team of the Video Coding Experts Group of the ITU-T and the Moving Picture Experts Group (MPEG) of ISO/IEC has recently devoted part of its effort to extend the widely deployed H.264/AVC standard to enable these new services and developed the Multiview Video Coding (MVC) extension. The MVC standard has the following features that are useful in terms of enabling many of the 3D video services:
1. Compression of 3D Video
Multiview video sequences are captured by different cameras in different positions and through different angles, they are just different representations of the same scene. Therefore, there exists an additional redundancy in the view direction for 3D video, called view redundancy. The MVC standard utilizes prediction between different views to exploit this redundancy and improve the compression ratio.
2. Scalability and Adaptation
To meet the requirements of the applications exemplified above, scalability and adaptability also need to be supported. For example, in 3DTV scenario advanced displays capable of displaying multiple views would be decoding more views than stereoscopic displays that display only two views. The MVC standard defines efficient ways so that any subset of the views is easily separable from the entire bitstream. As the regular two dimensional TV applications are still widely used, backwards compatibility has been an important target for MVC. The MVC standard achieved backwards compatibility by defining the bitstream so that a compliant H.264/AVC decoder can decode a single 2D view and discard the rest of the data, whereas a compliant MVC decoder can decode all the views and generate the 3D video. Backwards compatibility is also supported by the related communication protocols for transport over the MPEG-2 Transport Stream and the Internet Protocol (IP) – a device capable of receiving an H.264/AVC stream over the MPEG-2 Transport Stream or the Real-Time Transport Protocol over IP is also capable of receiving an MVC stream over these protocols.
3. Computational Complexity
The amount of information that needs to be processed in 3D video is significantly higher than that of regular 2D video. Moreover, the additional dependency between views could potentially become problematic in terms of implementation complexity. In order to enable practical implementations and reduce the complexity of 3D video codecs, the MVC standard utilizes efficient ways to buffer the pictures used for prediction, and enable parallel processing of separate views. In addition, the MVC standard does not change the underlying coding tools used in the H.264/AVC standard, so that the existing hardware accelerators and optimized software implementations that are widely deployed could be reused.