  Emojis are small images that are commonly included in social media text
messages. The combination of visual and textual content in the same message
builds up a modern way of communication, that automatic systems are not used to
deal with. In this paper we extend recent advances in emoji prediction by
putting forward a multimodal approach that is able to predict emojis in
Instagram posts. Instagram posts are composed of pictures together with texts
which sometimes include emojis. We show that these emojis can be predicted by
using the text, but also using the picture. Our main finding is that
incorporating the two synergistic modalities, in a combined model, improves
accuracy in an emoji prediction task. This result demonstrates that these two
modalities (text and images) encode different information on the use of emojis
and therefore can complement each other.
