  Brain tumors are an abnormal growth of cells in the brain. They can be
classified into distinct grades based on their growth. Often grading is
performed based on a histological image and is one of the most significant
predictors of a patients prognosis, the higher the grade, the more aggressive
the tumor. Correct diagnosis of a tumor grade remains challenging. Though
histopathological grading has been shown to be prognostic, results are subject
to interobserver variability, even among experienced pathologists. Recently,
the World Health Organization reported that advances in molecular genetics have
led to improvements in tumor classification. This paper seeks to integrate
histological images and genetic data for improved computer-aided diagnosis. We
propose a novel Multi-modal Outer Arithmetic Block (MOAB) based on arithmetic
operations to combine latent representations of the different modalities for
predicting the tumor grade (Grade \rom{2}, \rom{3} and \rom{4}). Extensive
experiments evaluate the effectiveness of our approach. By applying MOAB to The
Cancer Genome Atlas (TCGA) glioma dataset, we show that it can improve
separation between similar classes (Grade \rom{2} and \rom{3}) and outperform
prior state-of-the-art grade classification techniques.
