Cooperate to compete : composable planning and inference in multi-agent reinforcement learning

Abstract

Cooperation within a competitive social situation is a essential part of human social life. This requires knowledge of teams and goals as well as an ability to infer the intentions of both teammates and opponents from sparse and noisy observations of their behavior. We describe a formal generative model that composes individual planning programs into rich and variable teams. This model constructs optimal coordinated team plans and uses these plans as part of a Bayesian inference of collaborators and adversaries of varying intelligence. We study these models in two environments: a complex continuous Atari game Warlords and a grid-world stochastic game, and compare our model with human behavior.