  The acoustic space in which a sound is created and heard plays an essential
role in how that sound is perceived by affording a unique sense of
\textit{presence}. Every sound we hear results from successive convolution
operations intrinsic to the sound source and external factors such as
microphone characteristics and room impulse responses. Typically, researchers
use an excitation such as a pistol shot or balloon pop as an impulse signal
with which an auralization can be created. The room "impulse" responses
convolved with the signal of interest can transform the input sound into the
sound played in the acoustic space of interest. Here we propose a novel
architecture that can transform a sound of interest into any other acoustic
space(room or hall) of interest by using arbitrary audio recorded as a proxy
for a balloon pop. The architecture is grounded in simple signal processing
ideas to learn residual signals from a learned acoustic signature and the input
signal. Our framework allows a neural network to adjust gains of every point in
the time-frequency representation, giving sound qualitative and quantitative
results.
