EU-funded SceneNet aims at 3D recreation of live events from mobile videos

April 23, 2014
A European Union (EU)-funded research project called SceneNet aims to develop technology to combine video feeds from mobile phones held up by different spectators around an arena to reconstruct the event in 3D.

A European Union (EU)-funded research project called SceneNet aims to develop technology to combine video feeds from mobile phones held up by different spectators around an arena to reconstruct the event in 3D. SceneNet is getting research funding under the EU's Seventh Framework Programme (FP7).

SceneNet involves several technological challenges: on-device preprocessing that requires immense computer power, efficient transmission of the video streams, development of accurate and fast methods for registration between the video streams, and 3D reconstruction. All of these tasks have to run at near real-time rates.

Israelis Chen and Nizan Sagiv got the idea when they were at a Depeche Mode concert in Tel Aviv five years ago. "While I was busy looking at the show, Nizan was watching the crowds," says Chen, SceneNet project coordinator. "He could not help noticing the huge number of faint lights from mobile-phone screens. People were taking videos of the show. Nizan thought that combining all the videos taken by individuals into a synergetic, enhanced, and possibly 3D video could be an interesting idea. We discussed the concept for many months, but it looked too futuristic, risky, and complicated."

Israel-Europe collaboration
They went to the Israel-Europe R&D directorate (ISERD; Tel Aviv) for advice and contacted Peter Maass of the University of Bremen (Bremen, Germany) and Pierre Vandergheynst of Ecole Polytechnique Fédérale (EPFL; Lausanne, Switzerland), with whom Chen had worked on an earlier FP7 project.

The result is SceneNet, awarded €1.33 million by the European Commission and coordinated by Chen's and Nizan's company SagivTech (Ra'anana, Israel), specialists in computer vision and parallel computing. SceneNet runs until January 2016 and consists of four European partners: the University of Bremen, EPFL, Steinbeis Innovation (Stuttgart, Germany), and European Research Services (Münster, Germany).

The first year of the project has seen the team develop the mobile infrastructure for the video feeds, a mechanism for tagging them, and their transmission to a cloud server. They’ve also developed basic tools for a human-computer interface that will allow users to view the 3D video from any vantage point "in the arena" and edit the film themselves. This, they believe, will help create online communities to share the content. With this in mind, the partners are to study privacy and intellectual property rights issues during the next two years of the project.

Rights and privacy concerns permitting, the technology might also be used to recreate other events in 3D, such as breaking news or sports, or in the tourism or surveillance sectors. The partners are also looking at shooting static, as well as active, objects from various angles, to create instructions that can be sent on to 3D printers.

For more info, see http://scenenet.uni-bremen.de/

Sponsored Recommendations

Melles Griot® XPLAN™ CCG Lens Series

March 19, 2024
IDEX Health & Science sets a new standard with our Melles Griot® XPLAN™ CCG Lens Series fluorescence microscope imaging systems. Access superior-quality optics with off-the-shelf...

Spatial Biology

March 19, 2024
Spatial Biology refers to the field that integrates spatial information into biological research, allowing for the study of biological systems in their native spatial context....

Custom-Engineered Optical Solutions for Your Application

March 19, 2024
We combine advanced optical design and manufacturing technology, with decades of experience in critical applications, to take you from first designs to ongoing marketplace success...

Semrock Optical Filters Resources

March 19, 2024
Looking for more information about Semrock optical filters? Explore sets by fluorophore, download the 2023 Semrock catalog and more.

Voice your opinion!

To join the conversation, and become an exclusive member of Laser Focus World, create an account today!