  Recently, the method that learns networks layer by layer has attracted
increasing interest for its ease of analysis. For the method, the main
challenge lies in deriving an optimization target for each layer by inversely
propagating the global target of the network. The propagation problem is ill
posed, due to involving the inversion of nonlinear activations from
lowdimensional to high-dimensional spaces. To address the problem, the existing
solution is to learn an auxiliary network to specially propagate the target.
However, the network lacks stability, and moreover, it results in higher
complexity for network learning. In the letter, we show that target propagation
could be achieved by modeling the network s each layer with compressed sensing,
without the need of auxiliary networks. Experiments show that the proposed
method could achieve better performance than the auxiliary network-based
method.
