Motion sculptures : automating artistic visualization of shape and time

Abstract

We present a method for automatically visualizing complex human and object motion via 3D motion sculptures -- a representation that conveys the 3D structure swept by the human's body as it moves through space. Given only a monocular RGB video as input, our algorithm computes the motion sculpture by estimating the object's 3D geometry over time, and then renders the motion sculpture under different rendering styles such as sculpture material, scene lighting, and floor reflections. We develop a 3D-aware image-based rendering approach to insert motion sculptures into a synthetic scene or back into the original video. This results in high-quality artistic visualizations of motion. Because motion sculptures are 3D, they can be viewed from arbitrary viewpoints and even physically printed. As such, they may reveal space-time information that is undetected by the naked eyes and allow the viewer to interpret how different parts of the object interact over time. Our method automates the process of motion sculpture creation, making this manual process typically done only by professional artists accessible to novice users and applicable to standard videos. We show results on various scenes involving complex motions such as sports actions and dancing.