  Embedded systems demand on-device processing of data using Neural Networks
(NNs) while conforming to the memory, power and computation constraints,
leading to an efficiency and accuracy tradeoff. To bring NNs to edge devices,
several optimizations such as model compression through pruning, quantization,
and off-the-shelf architectures with efficient design have been extensively
adopted. These algorithms when deployed to real world sensitive applications,
requires to resist inference attacks to protect privacy of users training data.
However, resistance against inference attacks is not accounted for designing NN
models for IoT. In this work, we analyse the three-dimensional
privacy-accuracy-efficiency tradeoff in NNs for IoT devices and propose Gecko
training methodology where we explicitly add resistance to private inferences
as a design objective. We optimize the inference-time memory, computation, and
power constraints of embedded devices as a criterion for designing NN
architecture while also preserving privacy. We choose quantization as design
choice for highly efficient and private models. This choice is driven by the
observation that compressed models leak more information compared to baseline
models while off-the-shelf efficient architectures indicate poor efficiency and
privacy tradeoff. We show that models trained using Gecko methodology are
comparable to prior defences against black-box membership attacks in terms of
accuracy and privacy while providing efficiency.
