  Joint sparsity has attracted considerable attention in recent years in many
fields including sparse signal recovery in compressed sensing (CS), statistics,
and machine learning. Traditional convex models suffer from the suboptimal
performance though enjoying tractable computation. In this paper, we propose a
new non-convex joint sparsity model, and develop a corresponding multi-stage
adaptive convex relaxation algorithm. This method extends the idea of iterative
support detection (ISD) from the single vector estimation to the multi-vector
estimation by considering the joint sparsity prior. We provide some preliminary
theoretical analysis including convergence analysis and a sufficient recovery
condition. Numerical experiments from both compressive sensing and feature
learning show the better performance of the proposed method in comparison with
several state-of-the-art alternatives. Moreover, we demonstrate that the
extension of ISD from the single vector to multi-vector estimation is not
trivial. In particular, while ISD does not work well for reconstructing the
signal channel sparse Bernoulli signal, it does achieve significantly improved
performance when recovering the multi-channel sparse Bernoulli signal thanks to
its ability of natural incorporation of the joint sparsity structure.
