  Given video data from multiple personal devices or street cameras, can we
exploit the structural and dynamic information to learn dynamic representation
of objects for applications such as distributed surveillance, without storing
data at a central server that leads to a violation of user privacy? In this
work, we introduce Federated Dynamic Graph Neural Network (Feddy), a
distributed and secured framework to learn the object representations from
multi-user graph sequences: i) It aggregates structural information from nearby
objects in the current graph as well as dynamic information from those in the
previous graph. It uses a self-supervised loss of predicting the trajectories
of objects. ii) It is trained in a federated learning manner. The centrally
located server sends the model to user devices. Local models on the respective
user devices learn and periodically send their learning to the central server
without ever exposing the user's data to server. iii) Studies showed that the
aggregated parameters could be inspected though decrypted when broadcast to
clients for model synchronizing, after the server performed a weighted average.
We design an appropriate aggregation mechanism of secure aggregation primitives
that can protect the security and privacy in federated learning with
scalability. Experiments on four video camera datasets (in four different
scenes) as well as simulation demonstrate that Feddy achieves great
effectiveness and security.
