Next-generation Virtual and Augmented Reality exploration and standardization

Abstract

The keynote will present developments over the past ten years in immersive media standards, as well as recent technological breakthroughs in next-generation virtual and augmented reality.

For instance, ISO/IEC MPEG and ITU-T VCEG have recently issued the 3D-HEVC multiview video compression standard, which reaches unpreceded compression performances for coding the video streams captured with dense, linear camera arrangements. With the aim of supporting future, high-quality 3D light field displays and Free Navigation virtual/augmented reality applications with sparse, curvelinear camera setups, new coding and view synthesis rendering techniques should be developed.

In view of the high challenges, MPEG and JPEG engage in defining a collaboration for developing these next-generation Virtual Reality standards, supporting 360 degree 3D video and virtual walkthrough applications with correct motion parallax and visual accomodation cues. Point Cloud and Light Field technologies are the select candidates to serve this purpose. Commonalities between these technologies will be presented, aiming at identifying the most promising exploration and standardization approaches.

About the presenter

Prof. Gauthier Lafruit got his Master in Electromechanical Engineering, and his PhD in applied sciences in hardware implementations for wavelet image coding, both from the Vrije Universiteit Brussel/Brussels University – Belgium, in 1989 and 1995 respectively. He started his multimedia research career in 1996 with Imec – Belgium, enjoyed a two-years stay with a Special Research Fund at Hasselt University – Belgium, and finally joined the LISA department of L’Université Libre de Bruxelles/Brussels University – Belgium, in october 2014.

Prof. Lafruit actively contributes to innovative developments in the fields of immersive media and virtual/augmented reality that are explored worldwide in entertainment (movies, games), cultural and information media (virtual museums, immersive journalism), steadily evolving towards photo-realistic free-viewpoint navigation modalities and glasses-free 3D viewing, studying advanced multi-camera view interpolation techniques.

Since 2014, Prof. Lafruit is co-chair of the FTV (Free viewpoint TV) working group in the international MPEG standardization committee, and since February 2016 he is also co-chair of the Light Field ad-hoc group. Prof. Lafruit actively contributes to MPEG-FTV, MPEG-LightFields, MPEG-ARAF (3D Augmented Reality in MPEG) and JPEG-PLENO (plenoptic camera study in JPEG) standardization activities.

Prof. Lafruit teaches 3D graphics and virtual/augmented reality in OpenGL, and imaging courses with GPGPU programming in CUDA and OpenCL. He has published around eighty MPEG/JPEG standardization working documents over the past ten years, as well as a couple of hundreds of papers in the domain of video and 3D graphics processing.