  This chapter describes a semantic dialogue system for radiologists in a
comprehensive case study within the large-scale MEDICO project. MEDICO
addresses the need for advanced semantic technologies in the search for medical
image and patient data. The objectives are, first, to enable a seamless
integration of medical images and different user applications by providing
direct access to image semantics, and second, to design and implement a
multimodal dialogue shell for the radiologist. Speech-based semantic image
retrieval and annotation of medical images should provide the basis for help in
clinical decision support and computer aided diagnosis. We will describe the
clinical workflow and interaction requirements and focus on the design and
implementation of a multimodal user interface for patient/image search or
annotation and its implementation while using a speech-based dialogue shell.
Ontology modeling provides the backbone for knowledge representation in the
dialogue shell and the specific medical application domain; ontology structures
are the communication basis of our combined semantic search and retrieval
architecture which includes the MEDICO server, the triple store, the semantic
search API, the medical visualization toolkit MITK, and the speech-based
dialogue shell, amongst others. We will focus on usability aspects of
multimodal applications, our storyboard and the implemented speech and
touchscreen interaction design.
