  We study deep neural networks with binary activation functions (BDNN), i.e.
the activation function only has two states. We show that the BDNN can be
reformulated as a mixed-integer linear program which can be solved to global
optimality by classical integer programming solvers. Additionally, a heuristic
solution algorithm is presented and we study the model under data uncertainty,
applying a two-stage robust optimization approach. We implemented our methods
on random and real datasets and show that the heuristic version of the BDNN
outperforms classical deep neural networks on the Breast Cancer Wisconsin
dataset while performing worse on random data.
