  Trajectory prediction aims to predict the movement trend of the agents like
pedestrians, bikers, vehicles. It is helpful to analyze and understand human
activities in crowded spaces and widely applied in many areas such as
surveillance video analysis and autonomous driving systems. Thanks to the
success of deep learning, trajectory prediction has made significant progress.
The current methods are dedicated to studying the agents' future trajectories
under the social interaction and the sceneries' physical constraints. Moreover,
how to deal with these factors still catches researchers' attention. However,
they ignore the \textbf{Semantic Shift Phenomenon} when modeling these
interactions in various prediction sceneries. There exist several kinds of
semantic deviations inner or between social and physical interactions, which we
call the "\textbf{Gap}". In this paper, we propose a \textbf{C}ontextual
\textbf{S}emantic \textbf{C}onsistency \textbf{Net}work (\textbf{CSCNet}) to
predict agents' future activities with powerful and efficient context
constraints. We utilize a well-designed context-aware transfer to obtain the
intermediate representations from the scene images and trajectories. Then we
eliminate the differences between social and physical interactions by aligning
activity semantics and scene semantics to cross the Gap. Experiments
demonstrate that CSCNet performs better than most of the current methods
quantitatively and qualitatively.
