A Pipeline for Synthesizing Action-conditioned Human Motion from Raw Motion Capture Data

Abstract

In many sports, less-experienced trainees will often draw inspiration from videos of experts. While this can be an effective tool for improvement, this process lacks the ability for the trainee to specifically focus on improving their skills based on the limitations of their current abilities, body type, and weaknesses.

Since sports are very competitive, there exists a need to convert expert movements to a series of standardizable forms and movements that can then be pedagogically applied to the differing needs of various trainees: specifically, their different abilities, body types, and weaknesses.

Effectively, this conversion requires a pipeline that can take an input of motion capture data, automatically label the markers used, create a skeletal representation, and then train a machine learning model to accurately synthesize human motion, conditioned on the action type.

The outputted motions can be rendered for any body type, and could be customized to the trainee. The designed pipeline is not fencing specific – it is highly adaptable to the nature of the data or sport, robust to errors and noise, as well as tightly integrated in an easy-to-use library.