Neural MMO: Massively Multiagent Simulation and Learning
Name
suarez-jsuarez-phd-eecs-2024-thesis.pdf
Description
Thesis PDF
Size
48.45 MB
Format
Adobe PDF
Checksum (MD5)
a2abd59a8293a9a174e79aee14cd3559
Author(s)
Suarez, Joseph
Advisor(s)
Isola, Phillip
Date Issued
May 2024
Publisher
Massachusetts Institute of Technology
Abstract
Neural MMO is a massively multi-agent environment for reinforcement learning research. It is designed to push the boundaries of environment complexity while maintaining computationally efficiency for academic research. Agents in Neural MMO can forage for a variety of resources, engage in strategic combat with each other, defeat scripted enemies for loot, level-up various interdependent professions, acquire tools, weapons, equipment, etc., and exchange items on a global market. Neural MMO was among the first many-agent simulators for reinforcement learning research, and it is still unique among environments today. To my knowledge, no other project provides large agent populations, high per-agent complexity, and efficient simulation at once. These properties make Neural MMO a suitable environment for a variety of research topics in multi-agent learning that would be difficult to explore without such a simulator. The environment can process 128 agents at up to 25x real-time on a single CPU core, totaling 3,000 agent-steps per second. This speed is owed to simulation techniques borrowed from the games industry. In the course of developing Neural MMO, I made several adaptations for the specifics of reinforcement learning, such as the two-layer structure of Neural MMO's observations and actions and the efficient internal data representation. The contributions of this work include these adapted methods as general-purpose tools for designing RL environments. Through my own experiments and from the results of a series of competitions that I hosted on Neural MMO, we have seen agents capable of long-term coherent strategies, multi-tasking across various objectives, and conditioning for specific goals. The largest discovery of this project has been the extent to which standard reinforcement learning methods with limited compute are able to solve complex tasks. Neural MMO is free and open-source software under the MIT license with comprehensive documentation at neuralmmo.github.io and a 1000+ member community Discord.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution 4.0 International (CC BY 4.0)
Copyright retained by author(s)
Persistent DSpace Link
