  Instrumental genesis through interdisciplinary collaboration-reflections on
the emergence of a visualisation framework for video annotation data XML This
paper presents, discusses and reflects on the development of a visualization
framework for the analysis of the temporal dynamics of audiovisual
expressivity. The main focus lies on the instrumental genesis process (Rabardel
1995; Longchamp 2012)-a concept trying to express and analyze the co-evolution
of instruments and the practices they make possible-underlying this
development. It is described through the collaboration and communication
processes between computer science scholars and humanities scholars in finding
new ways of visualizing complex datasets for exploration and presentation in
the realm of film-studies research. It draws on the outcome and concrete usage
of the visualizations in publications and presentations of a research group,
the AdAproject, that investigates the audiovisual rhetorics of affect in
audiovisual media on the financial crisis (2007-). These film analyses are
based on theoretical assumptions on the process of film-viewing, the relation
of the viewer's perception and the temporally unfolding audiovisual images, and
a methodical approach that draws on 'steps' in the research process such as
segmentation, description and qualification, called eMAEX (Kappelhoff et al.
2011-2016) to reconstruct these experiential figurations (Bakels et al. 2020a,
2020b). The main focus of this paper is the process of iterative development of
visualizations as interactive interfaces generated with the open-source
software Advene, that were an integral part of the research process. In this
regard, the timeline visualization is not only of interest for visual
argumentation in (digital) humanities publications, but also for the creation
of annotations as well as the exploration of this data. In the first part of
the paper we describe this interdisciplinary collaboration as instrumental
genesis on a general level-as an evolving and iterative process. In the second
part we focus on the specific challenge of designing a visualization framework
for the temporal dynamics of audiovisual aesthetics. Lastly we zoom out by
reflecting on experiences and insights that might be of interest for the wider
digital humanities community.
