Capturing Distributions over Worlds for Robotics with Spatial Scene Grammars

Abstract

Having a precise understanding of the distribution over worlds a robot will face is critical to most problems in robotics. This distribution informs mechanical and software design specifications, provides strong priors to perception, and quantifies the real-world relevance of simulation and lab testing. However, representing and quantifying this distribution is an open and difficult problem, as these worlds can vary in myriad continuous and discrete ways. This thesis is concerned with a particular class of probabilistic procedural models – spatial scene grammars – that are tailored to describe hybrid discrete-and-continuous distributions over environments with varying numbers, types, and spatial poses of objects. We develop a spatial scene grammar formulation that is sufficiently expressive to capture the structure of practically relevant environments, but is carefully restricted to remain amenable to various forms of probabilistic inference. We show that we can sample diverse scenes from these grammars, even under the presence of constraints on scene contents and object poses; that we can parse scenes with this grammar model via a novel set of mixed-integer parsing techniques to achieve detailed scene understanding and part-level outlier detection; and that we can fit unknown parameters in the model to data via an approximate expectation-maximization algorithm.