  Humans experience the world through constantly changing visual stimuli, where
scenes can shift and move, change in appearance, and vary in distance. The
dynamic nature of visual perception is a fundamental aspect of our daily lives,
yet the large majority of research on object and scene processing, particularly
using fMRI, has focused on static stimuli. While studies of static image
perception are attractive due to their computational simplicity, they impose a
strong non-naturalistic constraint on our investigation of human vision. In
contrast, dynamic visual stimuli offer a more ecologically-valid approach but
present new challenges due to the interplay between spatial and temporal
information, making it difficult to disentangle the representations of stable
image features and motion. To overcome this limitation -- given dynamic inputs,
we explicitly decouple the modeling of static image representations and motion
representations in the human brain. Three results demonstrate the feasibility
of this approach. First, we show that visual motion information as optical flow
can be predicted (or decoded) from brain activity as measured by fMRI. Second,
we show that this predicted motion can be used to realistically animate static
images using a motion-conditioned video diffusion model (where the motion is
driven by fMRI brain activity). Third, we show prediction in the reverse
direction: existing video encoders can be fine-tuned to predict fMRI brain
activity from video imagery, and can do so more effectively than image
encoders. This foundational work offers a novel, extensible framework for
interpreting how the human brain processes dynamic visual information.
