  Unsupervised learning methods have recently shown their competitiveness
against supervised training. Typically, these methods use a single objective to
train the entire network. But one distinct advantage of unsupervised over
supervised learning is that the former possesses more variety and freedom in
designing the objective. In this work, we explore new dimensions of
unsupervised learning by proposing the Progressive Stage-wise Learning (PSL)
framework. For a given unsupervised task, we design multilevel tasks and define
different learning stages for the deep network. Early learning stages are
forced to focus on lowlevel tasks while late stages are guided to extract
deeper information through harder tasks. We discover that by progressive
stage-wise learning, unsupervised feature representation can be effectively
enhanced. Our extensive experiments show that PSL consistently improves results
for the leading unsupervised learning methods.
