  Many real-world problems exhibit the coexistence of multiple types of
heterogeneity, such as view heterogeneity (i.e., multi-view property) and task
heterogeneity (i.e., multi-task property). For example, in an image
classification problem containing multiple poses of the same object, each pose
can be considered as one view, and the detection of each type of object can be
treated as one task. Furthermore, in some problems, the data type of multiple
views might be different. In a web classification problem, for instance, we
might be provided an image and text mixed data set, where the web pages are
characterized by both images and texts. A common strategy to solve this kind of
problem is to leverage the consistency of views and the relatedness of tasks to
build the prediction model. In the context of deep neural network, multi-task
relatedness is usually realized by grouping tasks at each layer, while
multi-view consistency is usually enforced by finding the maximal correlation
coefficient between views. However, there is no existing deep learning
algorithm that jointly models task and view dual heterogeneity, particularly
for a data set with multiple modalities (text and image mixed data set or text
and video mixed data set, etc.). In this paper, we bridge this gap by proposing
a deep multi-task multi-view learning framework that learns a deep
representation for such dual-heterogeneity problems. Empirical studies on
multiple real-world data sets demonstrate the effectiveness of our proposed
Deep-MTMV algorithm.
