  Feature noise and label noise are ubiquitous in practical scenarios, which
pose great challenges for training a robust machine learning model. Most
previous approaches usually deal with only a single problem of either feature
noise or label noise. However, in real-world applications, hybrid noise, which
contains both feature noise and label noise, is very common due to the
unreliable data collection and annotation processes. Although some results have
been achieved by a few representation learning based attempts, this issue is
still far from being addressed with promising performance and guaranteed
theoretical analyses. To address the challenge, we propose a novel unified
learning framework called "Feature and Label Recovery" (FLR) to combat the
hybrid noise from the perspective of data recovery, where we concurrently
reconstruct both the feature matrix and the label matrix of input data.
Specifically, the clean feature matrix is discovered by the low-rank
approximation, and the ground-truth label matrix is embedded based on the
recovered features with a nuclear norm regularization. Meanwhile, the feature
noise and label noise are characterized by their respective adaptive matrix
norms to satisfy the corresponding maximum likelihood. As this framework leads
to a non-convex optimization problem, we develop the non-convex Alternating
Direction Method of Multipliers (ADMM) with the convergence guarantee to solve
our learning objective. We also provide the theoretical analysis to show that
the generalization error of FLR can be upper-bounded in the presence of hybrid
noise. Experimental results on several typical benchmark datasets clearly
demonstrate the superiority of our proposed method over the state-of-the-art
robust learning approaches for various noises.
