Appearance vs. Motion.pdf

Recognizing Human Figures and Actions Greg Mori Simon Fraser University Goal ? Action recognition – Where are the people? – What are they doing? ? Applications – Image understanding, image retrieval and search – HCI – Surveillance – Computer Graphics ? 3-pixel man ? Blob tracking ? 300-pixel man ? Find and track limbs Far field Near field Medium field ? 30-pixel man ? Coarse-level actions Outline ? Human figures in motion – Action Recognition ? Localizing joint positions – Exemplar-based approach – Parts-based approach ? Motion Synthesis – Novel graphics application Appearance vs. Motion Jackson Pollock Number 21 (detail) Action Recognition ? Recognize human actions at a distance – Low resolution, noisy data – Moving camera, occlusions – Wide range of actions (including non-periodic) Our Approach ? Motion-based approach – Classify a novel motion by finding the most similar motion from the training set – Use large amounts of data (“non-parametric”) ? Related Work – Periodicity analysis ? Polana Nelson; Seitz Dyer; Bobick et al; Cutler Davis; Collins et al. – Model-free ? Temporal Templates [Bobick Davis] ? Orientation histograms [Freeman et al; Zelnik Irani] ? Using MoCap data [Zhao Nevatia, Ramanan Forsyth] Gathering action data ? Tracking – Simple correlation-based tracker Figure-centric Representation ? Stabilized spatio-temporal volume – No translation information – All motion caused by person’s limbs ? Good news: indifferent to camera motion ? Bad news: hard! ? Good test to see if actions, not just translation, are being captured input sequence Remembrance of Things Past ? “Explain” novel motion sequence by matching to previously seen video clips – For each frame, match based on some temporal extent Challenge: how to compare motions? run walk left swing walk right jog database How to describe motion? ? Appearance – Not preserved across different clothing ? Gradients (spatial, temporal) – same (e.g. contrast reversal) ? Edges – Unreliable at this scale ? O