  In today's increasingly international economy, return and volatility
spillover effects across international equity markets are major macroeconomic
drivers of stock dynamics. Thus, information regarding foreign markets is one
of the most important factors in forecasting domestic stock prices. However,
the cross-correlation between domestic and foreign markets is highly complex.
Hence, it is extremely difficult to explicitly express this cross-correlation
with a dynamical equation. In this study, we develop stock return prediction
models that can jointly consider international markets, using multimodal deep
learning. Our contributions are three-fold: (1) we visualize the transfer
information between South Korea and US stock markets by using scatter plots;
(2) we incorporate the information into the stock prediction models with the
help of multimodal deep learning; (3) we conclusively demonstrate that the
early and intermediate fusion models achieve a significant performance boost in
comparison with the late fusion and single modality models. Our study indicates
that jointly considering international stock markets can improve the prediction
accuracy and deep neural networks are highly effective for such tasks.
