  The use of emojis affords a visual modality to, often private, textual
communication. The task of predicting emojis however provides a challenge for
machine learning as emoji use tends to cluster into the frequently used and the
rarely used emojis. Much of the machine learning research on emoji use has
focused on high resource languages and has conceptualised the task of
predicting emojis around traditional server-side machine learning approaches.
However, traditional machine learning approaches for private communication can
introduce privacy concerns, as these approaches require all data to be
transmitted to a central storage. In this paper, we seek to address the dual
concerns of emphasising high resource languages for emoji prediction and
risking the privacy of people's data. We introduce a new dataset of $118$k
tweets (augmented from $25$k unique tweets) for emoji prediction in Hindi, and
propose a modification to the federated learning algorithm, CausalFedGSD, which
aims to strike a balance between model performance and user privacy. We show
that our approach obtains comparative scores with more complex centralised
models while reducing the amount of data required to optimise the models and
minimising risks to user privacy.
