  Neural architecture search, which aims to automatically search for
architectures (e.g., convolution, max pooling) of neural networks that maximize
validation performance, has achieved remarkable progress recently. In many
application scenarios, several parties would like to collaboratively search for
a shared neural architecture by leveraging data from all parties. However, due
to privacy concerns, no party wants its data to be seen by other parties. To
address this problem, we propose federated neural architecture search (FNAS),
where different parties collectively search for a differentiable architecture
by exchanging gradients of architecture variables without exposing their data
to other parties. To further preserve privacy, we study differentially-private
FNAS (DP-FNAS), which adds random noise to the gradients of architecture
variables. We provide theoretical guarantees of DP-FNAS in achieving
differential privacy. Experiments show that DP-FNAS can search
highly-performant neural architectures while protecting the privacy of
individual parties. The code is available at
https://github.com/UCSD-AI4H/DP-FNAS
