Developing approximation architectures for decision-making in real-time systems

Abstract

This thesis studies the design of basis functions in approximate linear programming (ALP) as a decision-making tool. A case study on a robotic control problem shows that feature-based basis functions are very effective because they are able to capture the characteristics and cost structure of the problem. State-space partitioning, polynomials and other non-linear combinations of state parameters are also used in the ALP. However, design of these basis functions requires more trial-and-error. Simulation results show that control policy generated by the approximate linear programming algorithm matches and sometimes surpasses that of heuristics. Moreover, optimal policies are found well before value function estimates reach optimality. The ALP scales well with problem size and the number of basis functions required to find the optimal policy does not increase significantly in larger scale systems. The promising results shed light on the possibility of applying approximate linear programming to other large-scale problems that are computationally intractable using traditional dynamic programming methods.