  Tiny machine learning (TinyML) is a rapidly growing field aiming to
democratize machine learning (ML) for resource-constrained microcontrollers
(MCUs). Given the pervasiveness of these tiny devices, it is inherent to ask
whether TinyML applications can benefit from aggregating their knowledge.
Federated learning (FL) enables decentralized agents to jointly learn a global
model without sharing sensitive local data. However, a common global model may
not work for all devices due to the complexity of the actual deployment
environment and the heterogeneity of the data available on each device. In
addition, the deployment of TinyML hardware has significant computational and
communication constraints, which traditional ML fails to address. Considering
these challenges, we propose TinyReptile, a simple but efficient algorithm
inspired by meta-learning and online learning, to collaboratively learn a solid
initialization for a neural network (NN) across tiny devices that can be
quickly adapted to a new device with respect to its data. We demonstrate
TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM. The
evaluations on various TinyML use cases confirm a resource reduction and
training time saving by at least two factors compared with baseline algorithms
with comparable performance.
