  Software tools for generating digital sound often present users with
high-dimensional, parametric interfaces, that may not facilitate exploration of
diverse sound designs. In this paper, we propose to investigate artificial
agents using deep reinforcement learning to explore parameter spaces in
partnership with users for sound design. We describe a series of user-centred
studies to probe the creative benefits of these agents and adapting their
design to exploration. Preliminary studies observing users' exploration
strategies with parametric interfaces and testing different agent exploration
behaviours led to the design of a fully-functioning prototype, called
Co-Explorer, that we evaluated in a workshop with professional sound designers.
We found that the Co-Explorer enables a novel creative workflow centred on
human-machine partnership, which has been positively received by practitioners.
We also highlight varied user exploration behaviors throughout partnering with
our system. Finally, we frame design guidelines for enabling such
co-exploration workflow in creative digital applications.
