  This thesis studies derivative-free optimization (DFO), particularly
model-based methods and software. These methods are motivated by optimization
problems for which it is impossible or prohibitively expensive to access the
first-order information of the objective function and possibly the constraint
functions.
  In particular, this thesis presents PDFO, a package we develop to provide
both MATLAB and Python interfaces to Powell's model-based DFO solvers, namely
COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. Moreover, a significant part of
this thesis is devoted to developing a new DFO method based on the sequential
quadratic programming (SQP) method. Therefore, we present an overview of the
SQP method and provide some perspectives on its theory and practice. In
particular, we show that the objective function of the SQP subproblem is a
natural quadratic approximation of the original objective function in the
tangent space of a surface. Finally, we elaborate on developing our new DFO
method, named COBYQA after Constrained Optimization BY Quadratic
Approximations. This derivative-free trust-region SQP method is designed to
tackle nonlinearly constrained optimization problems that admit equality and
inequality constraints. An important feature of COBYQA is that it always
respects bound constraints, if any, which is motivated by applications where
the objective function is undefined when bounds are violated. We expose
extensive numerical experiments of COBYQA, showing evident advantages of COBYQA
compared with Powell's DFO solvers. These experiments demonstrate that COBYQA
is an excellent successor to COBYLA as a general-purpose DFO solver.
  This is the Ph.D. thesis finished under the supervision of Dr. Zaikun Zhang
and Prof. Xiaojun Chen at The Hong Kong Polytechnic University. Financial
support was provided by the UGC of Hong Kong under the Hong Kong Ph.D.
Fellowship Scheme.
