  The recently proposed Multilinear Compressive Learning (MCL) framework
combines Multilinear Compressive Sensing and Machine Learning into an
end-to-end system that takes into account the multidimensional structure of the
signals when designing the sensing and feature synthesis components. The key
idea behind MCL is the assumption of the existence of a tensor subspace which
can capture the essential features from the signal for the downstream learning
task. Thus, the ability to find such a discriminative tensor subspace and
optimize the system to project the signals onto that data manifold plays an
important role in Multilinear Compressive Learning. In this paper, we propose a
novel solution to address both of the aforementioned requirements, i.e., How to
find those tensor subspaces in which the signals of interest are highly
separable? and How to optimize the sensing and feature synthesis components to
transform the original signals to the data manifold found in the first
question? In our proposal, the discovery of a high-quality data manifold is
conducted by training a nonlinear compressive learning system on the inference
task. Its knowledge of the data manifold of interest is then progressively
transferred to the MCL components via multi-stage supervised training with the
supervisory information encoding how the compressed measurements, the
synthesized features, and the predictions should be like. The proposed
knowledge transfer algorithm also comes with a semi-supervised adaption that
enables compressive learning models to utilize unlabeled data effectively.
Extensive experiments demonstrate that the proposed knowledge transfer method
can effectively train MCL models to compressively sense and synthesize better
features for the learning tasks with improved performances, especially when the
complexity of the learning task increases.
