  Video-Based Design (VBD) is a design methodology that utilizes video as a
primary tool for understanding user interactions, prototyping, and conducting
research to enhance the design process. Artificial Intelligence (AI) can be
instrumental in video-based design by analyzing and interpreting visual data
from videos to enhance user interaction, automate design processes, and improve
product functionality. In this study, we explore how AI can enhance
professional video-based design with a State-of-the-Art (SOTA) deep learning
model. We developed a prototype annotation platform (MarkupLens) and conducted
a between-subjects eye-tracking study with 36 designers, annotating videos with
three levels of AI assistance. Our findings indicate that MarkupLens improved
design annotation quality and productivity. Additionally, it reduced the
cognitive load that designers exhibited and enhanced their User Experience
(UX). We believe that designer-AI collaboration can greatly enhance the process
of eliciting insights in video-based design.
