  Music-to-visual style transfer is a challenging yet important cross-modal
learning problem in the practice of creativity. Its major difference from the
traditional image style transfer problem is that the style information is
provided by music rather than images. Assuming that musical features can be
properly mapped to visual contents through semantic links between the two
domains, we solve the music-to-visual style transfer problem in two steps:
music visualization and style transfer. The music visualization network
utilizes an encoder-generator architecture with a conditional generative
adversarial network to generate image-based music representations from music
data. This network is integrated with an image style transfer method to
accomplish the style transfer process. Experiments are conducted on
WikiArt-IMSLP, a newly compiled dataset including Western music recordings and
paintings listed by decades. By utilizing such a label to learn the semantic
connection between paintings and music, we demonstrate that the proposed
framework can generate diverse image style representations from a music piece,
and these representations can unveil certain art forms of the same era.
Subjective testing results also emphasize the role of the era label in
improving the perceptual quality on the compatibility between music and visual
content.
