  Informational videos serve as a crucial source for explaining conceptual and
procedural knowledge to novices and experts alike. When producing informational
videos, editors edit videos by overlaying text/images or trimming footage to
enhance the video quality and make it more engaging. However, video editing can
be difficult and time-consuming, especially for novice video editors who often
struggle with expressing and implementing their editing ideas. To address this
challenge, we first explored how multimodality$-$natural language (NL) and
sketching, which are natural modalities humans use for expression$-$can be
utilized to support video editors in expressing video editing ideas. We
gathered 176 multimodal expressions of editing commands from 10 video editors,
which revealed the patterns of use of NL and sketching in describing edit
intents. Based on the findings, we present ExpressEdit, a system that enables
editing videos via NL text and sketching on the video frame. Powered by LLM and
vision models, the system interprets (1) temporal, (2) spatial, and (3)
operational references in an NL command and spatial references from sketching.
The system implements the interpreted edits, which then the user can iterate
on. An observational study (N=10) showed that ExpressEdit enhanced the ability
of novice video editors to express and implement their edit ideas. The system
allowed participants to perform edits more efficiently and generate more ideas
by generating edits based on user's multimodal edit commands and supporting
iterations on the editing commands. This work offers insights into the design
of future multimodal interfaces and AI-based pipelines for video editing.
