  The fast pace of advances in AI promises to revolutionize various aspects of
knowledge work, extending its influence to daily life and professional fields
alike. We advocate for a paradigm where AI is seen as a collaborative co-pilot,
working under human guidance rather than as a mere tool. Drawing from relevant
research and literature in the disciplines of Human-Computer Interaction and
Human Factors Engineering, we highlight the criticality of maintaining human
oversight in AI interactions. Reflecting on lessons from aviation, we address
the dangers of over-relying on automation, such as diminished human vigilance
and skill erosion. Our paper proposes a design approach that emphasizes active
human engagement, control, and skill enhancement in the AI partnership, aiming
to foster a harmonious, effective, and empowering human-AI relationship. We
particularly call out the critical need to design AI interaction capabilities
and software applications to enable and celebrate the primacy of human agency.
This calls for designs for human-AI partnership that cede ultimate control and
responsibility to the human user as pilot, with the AI co-pilot acting in a
well-defined supporting role.
