《2016 Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization》.pdf

Math. Program., Ser. A DOI 10.1007/s10107-013-0737-x FULL LENGTH PAPER Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization Guanghui Lan Received: 22 January 2013 / Accepted: 13 December 2013 © Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2013 Abstract The main goal of this paper is to develop uniformly optimal first-order methods for convex programming (CP). By uniform optimality we mean that the first- order methods themselves do not require the input of any problem parameters, but can still achieve the best possible iteration complexity bounds. By incorporating a multi- step acceleration scheme into the well-known bundle-level method, we develop an accelerated bundle-level method, and show that it can achieve the optimal complexity for solving a general class of black-box CP problems without requiring the input of any smoothness information, such as, whether the problem is smooth, nonsmooth or weakly smooth, as well as the specific values of Lipschitz constant and smoothness level. We then develop a more practical, restricted memory version of this method, namely the accelerated prox-level (APL) method. We investigate the generalization of the APL method for solving certain composite CP problems and an important class of saddle-point problems recently studied by Nesterov (Math Program 103:127–152, 2005). We present promising numerical results for these new bundle-level methods applied to solve certain classes of semidefinite programming and stochastic program- ming problems. Keywords Convex programming · Complexity · Bundle-level · Optimal methods The paper is a combined version of the two manuscripts previously submitted to Mathematical Programming, namely: “Bundle-type methods uniformly optimal for smooth and nonsmooth convex optimization” and “Level methods uniformly optimal for composite and structured nonsmooth convex optimization”. The au