  We present a novel adaptive online learning (AOL) framework to predict human
movement trajectories in dynamic video scenes. Our framework learns and adapts
to changes in the scene environment and generates best network weights for
different scenarios. The framework can be applied to prediction models and
improve their performance as it dynamically adjusts when it encounters changes
in the scene and can apply the best training weights for predicting the next
locations. We demonstrate this by integrating our framework with two existing
prediction models: LSTM [3] and Future Person Location (FPL) [1]. Furthermore,
we analyze the number of network weights for optimal performance and show that
we can achieve real-time with a fixed number of networks using the least
recently used (LRU) strategy for maintaining the most recently trained network
weights. With extensive experiments, we show that our framework increases
prediction accuracies of LSTM and FPL by ~17% and 28% on average, and up to
~50% for FPL on the worst case while achieving real-time (20fps).
