April 2002

BACK   

In this talk, I shall present recently developed methods for model-based tracking of deformable and highly articulated objects like humans in monocular video sequences. Potential applications include model-based human motion capture, animation and virtual reality, video indexing, human-computer interaction and intelligent surveillance.

Monocular 3D body tracking is a difficult problem: for reliable tracking at least 30 joint parameters need to be estimated, subject to highly nonlinear physical constraints; the problem is chronically ill-conditioned as about a third of the d.o.f (the depth-related ones) are almost unobservable in any given monocular image; and matching an imperfect, highly flexible, self-occluding model to cluttered image features is intrinsically hard. The ambiguity, nonlinearity and non-observability make the parameter-space cost surface multi-modal, unpredictable and ill-conditioned, so minimizing it is difficult. Local minima are a perennial problem and it is therefore extremely important to ensure that a set of globally representative solutions is found and subsequently tracked at all times.

Balancing determinism and stochasticity, we have developed and investigated three families of algorithms for efficiently attacking this and similar problems: (i) the 'Eigenvector Tracking' and 'Hypersurface Sweeping' are deterministic algorithms for building local `maps' of the nearby minima and the `transition pathways' linking them, (ii) 'Covariance-Scaled Sampling' is a Bayesian estimation framework combining deterministic mode finding using continuous optimization subject to physical constraints and inflated-covariance-scaled sampling to focus the samples in probable low cost regions, and (iii) 'Hyperdynamics' is an importance sampling method that accelerates the multiple-mode exploration behavior of sequential Markov Chain Monte Carlo samplers by using the local gradient and curvature of the input distribution to construct an adaptive importance sampler focused on 'transition states' -- codimension-1 saddle points representing 'mountain passes' linking adjacent cost basins.

In a related research effort, I've investigated the incremental structure and motion estimation for adaptive and deformable models. We have used motion consistency constraints in order to incrementally reveal new model structure and to reconstruct and integrate it in a flexible way, in the model-based tracking process. As a result, the model shape coverage is automatically improved and its motion estimation process is robustified.

During the talk, I shall outline the modeling of these problems, present the formulation of the developed algorithms, discuss detailed results, potential applications, and highlight directions for future research.

BACK

Last Updated: April 17, 2002