  This paper presents a novel and lightweight hyperparameter optimization (HPO)
method, MOdular FActorial Design (MOFA). MOFA pursues several rounds of HPO,
where each round alternates between exploration of hyperparameter space by
factorial design and exploitation of evaluation results by factorial analysis.
Each round first explores the configuration space by constructing a
low-discrepancy set of hyperparameters that cover this space well while
de-correlating hyperparameters, and then exploits evaluation results through
factorial analysis that determines which hyperparameters should be further
explored and which should become fixed in the next round. We prove that the
inference of MOFA achieves higher confidence than other sampling schemes. Each
individual round is highly parallelizable and hence offers major improvements
of efficiency compared to model-based methods. Empirical results show that MOFA
achieves better effectiveness and efficiency compared with state-of-the-art
methods.
