Neurocomputational Modeling of Human Physical Scene Understanding

Abstract

Human scene understanding involves not just localizing objects,but also inferring latent attributes that affect how the scene mightunfold, such as the masses of objects within the scene. Theseattributes can sometimes only be inferred from the dynamicsof a scene, but people can flexibly integrate this information toupdate their inferences. Here we propose a neurally plausibleEfficient Physical Inferencemodel that can generate and updateinferences from videos. This model makes inferences over theinputs to a generative model of physics and graphics, usingan LSTM based recognition network to efficiently approximaterational probabilistic conditioning. We find that this model notonly rapidly and accurately recovers latent object information,but also that its inferences evolve with more information in away similar to human judgments. The model provides a testablehypothesis about the population-level activity in brain regionsunderlying physical reasoning.