Large-Scale Optimization using Reinforcement Learning, Dynamic Programming, and Column Generation

Abstract

One of the most enduring challenges in large-scale optimization is determining how to push the boundaries of scalability without compromising on performance or rigor. For decades, the exponential advances in computational power offered a straightforward solution: bigger problems could simply be tackled by bigger machines. However, in recent years, it has become increasingly apparent that pure computational force alone can no longer keep pace with the ever-growing complexity and scale of real-world applications. Additionally, despite the remarkable success of general-purpose methods for linear and integer optimization, these methods often struggle when confronted with domains that involve intricate dynamics, massive dimensionality, or a need for fine-grained sequential decisions. The simple question thus arises: can we design new optimization methods that scale more appropriately? In this thesis, we propose using dynamic programming, reinforcement learning, and column generation as a practical way to address this need across a variety of settings.

We begin by developing and refining our methodology within the context of reinforcement learning and dynamic programming. We then move on to the application of column generation, and finally show how these techniques can be combined to supercharge fundamental machine learning methods with large-scale optimality.