  We introduce a novel text-to-pose video editing method, ReimaginedAct. While
existing video editing tasks are limited to changes in attributes, backgrounds,
and styles, our method aims to predict open-ended human action changes in
video. Moreover, our method can accept not only direct instructional text
prompts but also `what if' questions to predict possible action changes.
ReimaginedAct comprises video understanding, reasoning, and editing modules.
First, an LLM is utilized initially to obtain a plausible answer for the
instruction or question, which is then used for (1) prompting Grounded-SAM to
produce bounding boxes of relevant individuals and (2) retrieving a set of pose
videos that we have collected for editing human actions. The retrieved pose
videos and the detected individuals are then utilized to alter the poses
extracted from the original video. We also employ a timestep blending module to
ensure the edited video retains its original content except where necessary
modifications are needed. To facilitate research in text-to-pose video editing,
we introduce a new evaluation dataset, WhatifVideo-1.0. This dataset includes
videos of different scenarios spanning a range of difficulty levels, along with
questions and text prompts. Experimental results demonstrate that existing
video editing methods struggle with human action editing, while our approach
can achieve effective action editing and even imaginary editing from
counterfactual questions.
