  In statistics, it is common to encounter multi-modal and non-smooth
likelihood (or objective function) maximization problems, where the parameters
have known upper and lower bounds. This paper proposes a novel derivative-free
global optimization technique that can be used to solve those problems even
when the objective function is not known explicitly or its derivatives are
difficult or expensive to obtain. The technique is based on the pattern search
algorithm, which has been shown to be effective for black-box optimization
problems. The proposed algorithm works by iteratively generating new solutions
from the current solution. The new solutions are generated by making movements
along the coordinate axes of the constrained sample space. Before making a jump
from the current solution to a new solution, the objective function is
evaluated at several neighborhood points around the current solution. The best
solution point is then chosen based on the objective function values at those
points. Parallel threading can be used to make the algorithm more scalable. The
performance of the proposed method is evaluated by optimizing up to
5000-dimensional multi-modal benchmark functions. The proposed algorithm is
shown to be up to 40 and 368 times faster than genetic algorithm (GA) and
simulated annealing (SA), respectively. The proposed method is also used to
estimate the optimal biomarker combination from Alzheimer's disease data by
maximizing the empirical estimates of the area under the receiver operating
characteristic curve (AUC), outperforming the contextual popular alternative,
known as step-down algorithm.
