  Compared with standard supervised learning, the key difficulty in
semi-supervised learning is how to make full use of the unlabeled data. A
recently proposed method, virtual adversarial training (VAT), smartly performs
adversarial training without label information to impose a local smoothness on
the classifier, which is especially beneficial to semi-supervised learning. In
this work, we propose tangent-normal adversarial regularization (TNAR) as an
extension of VAT by taking the data manifold into consideration. The proposed
TNAR is composed by two complementary parts, the tangent adversarial
regularization (TAR) and the normal adversarial regularization (NAR). In TAR,
VAT is applied along the tangent space of the data manifold, aiming to enforce
local invariance of the classifier on the manifold, while in NAR, VAT is
performed on the normal space orthogonal to the tangent space, intending to
impose robustness on the classifier against the noise causing the observed data
deviating from the underlying data manifold. Demonstrated by experiments on
both artificial and practical datasets, our proposed TAR and NAR complement
with each other, and jointly outperforms other state-of-the-art methods for
semi-supervised learning.
