  This paper is about regularizing deep convolutional networks (CNNs) based on
an adaptive framework for transfer learning with limited training data in the
target domain. Recent advances of CNN regularization in this context are
commonly due to the use of additional regularization objectives. They guide the
training away from the target task using some forms of concrete tasks. Unlike
those related approaches, we suggest that an objective without a concrete goal
can still serve well as a regularized. In particular, we demonstrate
Pseudo-task Regularization (PtR) which dynamically regularizes a network by
simply attempting to regress image representations to pseudo-regression targets
during fine-tuning. That is, a CNN is efficiently regularized without
additional resources of data or prior domain expertise. In sum, the proposed
PtR provides: a) an alternative for network regularization without dependence
on the design of concrete regularization objectives or extra annotations; b) a
dynamically adjusted and maintained strength of regularization effect by
balancing the gradient norms between objectives on-line. Through numerous
experiments, surprisingly, the improvements on classification accuracy by PtR
are shown greater or on a par to the recent state-of-the-art methods.
