Motor learning on a heaving plate via improved-SNR algorithms

Abstract

Creatures in nature have subtle and complicated interactions with their surrounding fluids, achieving levels of performance as yet unmatched by engineered solutions. Model-free reinforcement learning (MFRL) holds the promise of allowing man-made controllers to take advantage of the subtlety of fluid-body interactions solely using data gathered on the actual system to be controlled. In this thesis, improved MFRL algorithms, motivated by a novel Signal-to-Noise Ratio for policy gradient algorithms, are developed, and shown to provide more efficient learning in noisy environments. These algorithms are then demonstrated on a heaving foil, where it is shown to learn a flapping gait on an experimental system orders of magnitude faster than the dynamics can be simulated, suggesting broad applications both in controlling robots with complex dynamics and in the study of controlled fluid systems.