  We consider the problem of optimizing expensive black-box functions over
high-dimensional combinatorial spaces which arises in many science,
engineering, and ML applications. We use Bayesian Optimization (BO) and propose
a novel surrogate modeling approach for efficiently handling a large number of
binary and categorical parameters. The key idea is to select a number of
discrete structures from the input space (the dictionary) and use them to
define an ordinal embedding for high-dimensional combinatorial structures. This
allows us to use existing Gaussian process models for continuous spaces. We
develop a principled approach based on binary wavelets to construct
dictionaries for binary spaces, and propose a randomized construction method
that generalizes to categorical spaces. We provide theoretical justification to
support the effectiveness of the dictionary-based embeddings. Our experiments
on diverse real-world benchmarks demonstrate the effectiveness of our proposed
surrogate modeling approach over state-of-the-art BO methods.
