Body-mounted cameras view surroundings, capture subject's motions

Aug. 10, 2011
Pittsburgh, PA--A new approach to capturing live human motion has been developed by researchers at Disney Research, Pittsburgh (DRP) and Carnegie Mellon University (CMU).

Pittsburgh, PA--A new approach to capturing live human motion has been developed by researchers at Disney Research, Pittsburgh (DRP) and Carnegie Mellon University (CMU). Intended to provide the information needed to animate digital models of actors for movies, the technique takes an approach exactly the reverse of traditional motion-capture approaches. Rather than using a set of external fixed cameras to track the subject's movements, a number of portable wireless cameras are strapped to various locations on the subject: as he or she moves about, the cameras look out at the fixed world around them.

The wearable camera system makes it possible to reconstruct the relative and global motions of the subject, thanks to a process called structure from motion (SfM). Takeo Kanade of CMU, a pioneer in computer vision, developed SfM 20 years ago to determine the 3D structure of an object by analyzing the images from a camera moving around the object, or as the object moves past the camera.

Sparse 3D info on surroundings
In the new motion-capture setup, SfM is not used primarily to analyze objects in a person's surroundings, but to estimate the pose of the cameras on the person. Velcro was used to mount 20 lightweight cell-phone-type cameras on the limbs and trunk of each subject, with the cameras each calibrated with respect to a reference structure. Each person then performed a range-of-motion exercise that allowed the system to automatically build a digital skeleton and estimate positions of cameras with respect to that skeleton.

The SfM software was then used to estimate rough position and orientation of limbs as the subject moved through an environment and to collect sparse 3D information about the environment to provide context for the captured motion. The rough position and orientation of limbs serves as an initial guess for a refinement step that optimizes the configuration of the body and its location in the environment.

The great freedom of the DRP/CMU technique allows motion capture to occur almost anywhere--for example, during a stroll or run through a neighborhood, or even while swinging on monkey bars. And the 3D data acquired by the SfM relies on the natural surroundings, eliminating the need for setting up any fixed external markers.

Motion capture for everyone
As video cameras become ever smaller and cheaper, "I think anyone will be able to do motion capture in the not-so-distant future," said Takaaki Shiratori, a postdoctoral associate at DRP, who presented the new technique this past Monday (August 8) at SIGGRAPH 2011, the International Conference on Computer Graphics and Interactive Techniques (Vancouver, BC, Canada).

The quality of motion capture from body-mounted cameras does not yet match the fidelity of traditional motion capture, Shiratori said, but will improve as the resolution of small video cameras continues to improve. In addition, the technique requires a significant amount of computational power; a minute of motion capture now can require an entire day to process. Future work will include efforts to find computational shortcuts, for example through parallel processing.

For more info and a video, see the DRP/CMU project website.

About the Author

John Wallace | Senior Technical Editor (1998-2022)

John Wallace was with Laser Focus World for nearly 25 years, retiring in late June 2022. He obtained a bachelor's degree in mechanical engineering and physics at Rutgers University and a master's in optical engineering at the University of Rochester. Before becoming an editor, John worked as an engineer at RCA, Exxon, Eastman Kodak, and GCA Corporation.

Sponsored Recommendations

Melles Griot® XPLAN™ CCG Lens Series

March 19, 2024
IDEX Health & Science sets a new standard with our Melles Griot® XPLAN™ CCG Lens Series fluorescence microscope imaging systems. Access superior-quality optics with off-the-shelf...

Spatial Biology

March 19, 2024
Spatial Biology refers to the field that integrates spatial information into biological research, allowing for the study of biological systems in their native spatial context....

Custom-Engineered Optical Solutions for Your Application

March 19, 2024
We combine advanced optical design and manufacturing technology, with decades of experience in critical applications, to take you from first designs to ongoing marketplace success...

Semrock Optical Filters Resources

March 19, 2024
Looking for more information about Semrock optical filters? Explore sets by fluorophore, download the 2023 Semrock catalog and more.

Voice your opinion!

To join the conversation, and become an exclusive member of Laser Focus World, create an account today!