  In this article, we describe the system that we used for the memotion
analysis challenge, which is Task 8 of SemEval-2020. This challenge had three
subtasks where affect based sentiment classification of the memes was required
along with intensities. The system we proposed combines the three tasks into a
single one by representing it as multi-label hierarchical classification
problem.Here,Multi-Task learning or Joint learning Procedure is used to train
our model.We have used dual channels to extract text and image based features
from separate Deep Neural Network Backbone and aggregate them to create task
specific features. These task specific aggregated feature vectors ware then
passed on to smaller networks with dense layers, each one assigned for
predicting one type of fine grain sentiment label. Our Proposed method show the
superiority of this system in few tasks to other best models from the
challenge.
