What’s up with stereographic 3D panoramic video ?

Wednesday, February 12, 2014

We have recently seen some interesting developments for the creation of « Stereoscopic 3D » – S3D – 360 videos : Panocam3D has been working on a solution for some time now which will be available later this year. Jim Watters has been sharing some experiments recently in producing stereoscopic 3D videos with inexpensive Mobius cameras as well. And recently, 360heros also announced the development of a GoPro based spherical rig for producing stereoscopic panoramic videos.


Panocam3D, Jim Watters DIY camera, 360heros new stereo rig (left to right ) :

the Panocam3D camera Jim Watters' rig 360 heros spherical S3D rig


360 stereoscopic 3D video ? How could that be ?

Isn’t the production of stereoscopic 3D imagery tied to the recording of two precisely offset viewpoints ? And isn’t 360 panoramic imagery essentially the recording of the surrounding environment from a single viewpoint ?

While the recording of two panoramas from two different viewpoints can produce the intended depth effect in a very narrow range, it is obviously inefficient at recording consistent depth information even over a 180 degrees field of view. However, by ‘rotating’ these two viewpoints around the center of view of the panorama, so that each rotation holds accurate depth information for a portion of the entire scene, it is possible to produce stereo 3D panoramas. How accurate the sens of depth greatly depends on taking a rational approach to the recording and processing of the images.

This idea was first introduced explored in the late 90’s and matured in the early 2000’s. It involves rationally extracting slices from images at each angle with a single camera offset from the viewpoint, to recompose a single panoramic image for each eye. Alternatively, 2 cameras can be used and various setups have been experimented. These same approaches have been used by skilled panoramic photographers to produce 360 degrees ‘3D panoramas’ using PTGui or Hugin with outstanding results.


shooting S3D panoramas


Nowadays, there are actually many different approaches to stereo vision. But as cinematography is essentially a sequence of photographs, the methodology described above has naturally been adapted to produce stereo 3D panoramic videos. By recording in multiple directions with an accurate offset between cameras set up in pairs – one for each eye – two panoramic videos can be produced.

The workflow for creating such stereo 3D videos involves creating a stitching template with all cameras together, and optimizing each ‘eye set’ of cameras precisely. The resulting « global » template is then split into two different panorama stitching templates in order to process a single panoramic video for each eye in VideoStitch. When viewed together in an appropriate way, these provide an accurate sens of depth.


Now how do we actually watch and enjoy such 360 degree S3D panoramic video scenes ?

As always since the origin of stereoscopic photography, the solution involves some kind of vision apparatus. The left and right eye panoramic videos are combined into a single video depending on the target viewing technology. This is commonly achieved with red/cyan overlay called “anaglyph”, or else by assembling these Side-By-Side (or top-bottom).


3D head mounted devices
Holmes stereoscope (1861), Anaglyph 3D glasses & the Oculus HMD.


The Oculus rift obviously comes to mind. In a previous blog post, we covered viewing 360 videos for the Oculus using VRPlayer, Side-By-Side video playback in VRPlayer is straightforward, proceed as for a regular 360 video and chose the appropriate Side-by-Side layout. The latest stereo 3D 360 video samples we could get our eyes ‘into’ provided us with an amazing feeling of depth emphasizing the immersive experience of 360 video with this device.

As the vr.js library and plugin for Chrome / Firefox makes it possible to develop browser based Oculus Rift applications, we can even foresee webGL 360 video players supporting S3D video content in the future. But should one necessarily dive into the captivating isolation of head mounted displays in order to sense the depth of 3D panoramic videos ?

Primarily used today for projection of computer generated content, stereo 3D cylindrical screens and stereo 3D full-domes provide exciting environments for collective immersive contemplation. And now with more affordable S3D 360 video rigs, and with the processing power of VideoStitch to handle this massive amount of pixels, we sure hope to see more experiments for « live action » content in this field too.


the ICCI 360 cylindrical screen


Panocam3D : http://www.panocam3d.com/video3d360.html
Jim Watters : http://photocreations.ca/3D/index.html
360heros : http://www.360heros.com/2014/01/worlds-first-fully-spherical-3d-360-video-and-photo-gear/



Other News