  The increasing demand for tabular data analysis calls for transitioning from
manual architecture design to Neural Architecture Search (NAS). This transition
demands an efficient and responsive anytime NAS approach that is capable of
returning current optimal architectures within any given time budget while
progressively enhancing architecture quality with increased budget allocation.
However, the area of research on Anytime NAS for tabular data remains
unexplored. To this end, we introduce ATLAS, the first anytime NAS approach
tailored for tabular data. ATLAS introduces a novel two-phase
filtering-and-refinement optimization scheme with joint optimization, combining
the strengths of both paradigms of training-free and training-based
architecture evaluation. Specifically, in the filtering phase, ATLAS employs a
new zero-cost proxy specifically designed for tabular data to efficiently
estimate the performance of candidate architectures, thereby obtaining a set of
promising architectures. Subsequently, in the refinement phase, ATLAS leverages
a fixed-budget search algorithm to schedule the training of the promising
candidates, so as to accurately identify the optimal architecture. To jointly
optimize the two phases for anytime NAS, we also devise a budget-aware
coordinator that delivers high NAS performance within constraints. Experimental
evaluations demonstrate that our ATLAS can obtain a good-performing
architecture within any predefined time budget and return better architectures
as and when a new time budget is made available. Overall, it reduces the search
time on tabular data by up to 82.75x compared to existing NAS approaches.
