  Neural networks are seeing increased use in diverse Internet of Things (IoT)
applications such as healthcare, smart homes and industrial monitoring. Their
widespread use makes neural networks a lucrative target for theft. An attacker
can obtain a model without having access to the training data or incurring the
cost of training. Also, networks trained using private data (e.g., medical
records) can reveal information about this data. Networks can be stolen by
leveraging side channels such as power traces of the IoT device when it is
running the network. Existing attacks require operations to occur in the same
order each time; an attacker must collect and analyze several traces of the
device to steal the network. Therefore, to prevent this type of attack, we
randomly shuffle the order of operations each time. With shuffling, each
operation can now happen at many different points in each execution, making the
attack intractable. However, we show that shuffling in software can leak
information which can be used to subvert this solution. Therefore, to perform
secure shuffling and reduce latency, we present BlackJack, hardware added as a
functional unit within the CPU. BlackJack secures neural networks on IoT
devices by increasing the time needed for an attack to centuries, while adding
just 2.46% area, 3.28% power and 0.56% latency overhead on an ARM M0+ SoC.
