  The development of the mlpack C++ machine learning library
(http://www.mlpack.org/) has required the design and implementation of a
flexible, robust optimization system that is able to solve the types of
arbitrary optimization problems that may arise all throughout machine learning
problems. In this paper, we present the generic optimization framework that we
have designed for mlpack. A key priority in the design was ease of
implementation of both new optimizers and new objective functions to be
optimized; therefore, implementation of a new optimizer requires only one
method and implementation of a new objective function requires at most four
functions. This leads to simple and intuitive code, which, for fast prototyping
and experimentation, is of paramount importance. When compared to optimization
frameworks of other libraries, we find that mlpack's supports more types of
objective functions, is able to make optimizations that other frameworks do
not, and seamlessly supports user-defined objective functions and optimizers.
