Non-Rigid Structure-From-Motion: Estimating
Shape and Motion with Hierarchical Priors

Lorenzo Torresani, Aaron Hertzmann, Chris Bregler

 

We present methods for recovering time-varying shape and motion of non-rigid 3D objects from uncalibrated 2D point tracks. For example, given a video recording of a talking person, we would like to estimate the 3D shape of the face at each instant, and learn a model of facial deformation. Time-varying shape is modeled as a rigid transformation combined with a non-rigid deformation. Reconstruction is ill-posed if arbitrary deformations are allowed, and thus additional assumptions about deformations are required. We propose using a Probabilistic Principal Components Analysis (PPCA) shape model, and describe a reconstruction algorithm that simultaneously estimates 3D shape and motion for each instant, learns the PPCA model parameters, and robustly fills-in missing data points. We then extend the model to model temporal dynamics in object shape, allowing the algorithm to robustly handle severe cases of missing data.

paper:

Revised from earlier conference papers:


software:

A Matlab implementation of our algorithm can be downloaded here.


datasets:

The datasets used in the experiments of the journal article are available here as Matlab files: