  In this work we present a Mixture of Task-Aware Experts Network for Machine
Reading Comprehension on a relatively small dataset. We particularly focus on
the issue of common-sense learning, enforcing the common ground knowledge by
specifically training different expert networks to capture different kinds of
relationships between each passage, question and choice triplet. Moreover, we
take inspi ration on the recent advancements of multitask and transfer learning
by training each network a relevant focused task. By making the
mixture-of-networks aware of a specific goal by enforcing a task and a
relationship, we achieve state-of-the-art results and reduce over-fitting.
