  Training certifiably robust neural networks is an important but challenging
task. While many algorithms for (deterministic) certified training have been
proposed, they are often evaluated on different training schedules,
certification methods, and systematically under-tuned hyperparameters, making
it difficult to compare their performance. To address this challenge, we
introduce CTBENCH, a unified library and a high-quality benchmark for certified
training that evaluates all algorithms under fair settings and systematically
tuned hyperparameters. We show that (1) almost all algorithms in CTBENCH
surpass the corresponding reported performance in literature in the magnitude
of algorithmic improvements, thus establishing new state-of-the-art, and (2)
the claimed advantage of recent algorithms drops significantly when we enhance
the outdated baselines with a fair training schedule, a fair certification
method and well-tuned hyperparameters. Based on CTBENCH, we provide new
insights into the current state of certified training and suggest future
research directions. We are confident that CTBENCH will serve as a benchmark
and testbed for future research in certified training.
