Improved Complexity Analysis for the Proximal Bundle Algorithm Under a Novel Perspective

Abstract

The proximal bundle algorithm (PBA) is a fundamental and computationally effective algorithm for solving optimization problems with non-smooth components. We investigate its convergence rate in two settings. We first focus on a composite setting where one function is smooth and the other is piecewise linear. We interpret a sequence of null steps of the PBA as a Frank-Wolfe algorithm on the Moreau envelope of the dual problem. In light of this correspondence, we first extend the linear convergence of Kelley's method on convex piecewise linear functions from the positive homogeneous to the general case. Building on this result, we propose a novel complexity analysis of PBA and derive a O (epsilon^-4/5) iteration complexity, improving upon the best known O (epsilon^-2) guarantee. This approach also unveils new insights on bundle management. We then present the first variant of the PBA for smooth objectives, achieving an accelerated convergence rate of O(epsilon^-1/2 log(epsilon^-1)), where epsilon is the desired accuracy. Our approach addresses an open question regarding the convergence guarantee of the PBA, which was previously posed in two recent papers. We interpret the PBA as a proximal point algorithm and base our proposed algorithm on an accelerated inexact proximal point scheme. Our variant introduces a novel null step test and oracle while maintaining the core structure of the original algorithm. The newly proposed oracle substitutes the traditional cutting planes with a smooth lower approximation of the true function. We show that this smooth interpolating lower model can be computed as a convex quadratic program. We finally show that Nesterov acceleration can be effectively applied when the objective is the sum of a smooth function and a piecewise linear one.