  For deployment, neural architecture search should be hardware-aware, in order
to satisfy the device-specific constraints (e.g., memory usage, latency and
energy consumption) and enhance the model efficiency. Existing methods on
hardware-aware NAS collect a large number of samples (e.g., accuracy and
latency) from a target device, either builds a lookup table or a latency
estimator. However, such approach is impractical in real-world scenarios as
there exist numerous devices with different hardware specifications, and
collecting samples from such a large number of devices will require prohibitive
computational and monetary cost. To overcome such limitations, we propose
Hardware-adaptive Efficient Latency Predictor (HELP), which formulates the
device-specific latency estimation problem as a meta-learning problem, such
that we can estimate the latency of a model's performance for a given task on
an unseen device with a few samples. To this end, we introduce novel hardware
embeddings to embed any devices considering them as black-box functions that
output latencies, and meta-learn the hardware-adaptive latency predictor in a
device-dependent manner, using the hardware embeddings. We validate the
proposed HELP for its latency estimation performance on unseen platforms, on
which it achieves high estimation performance with as few as 10 measurement
samples, outperforming all relevant baselines. We also validate end-to-end NAS
frameworks using HELP against ones without it, and show that it largely reduces
the total time cost of the base NAS method, in latency-constrained settings.
Code is available at https://github.com/HayeonLee/HELP.
