  In many problem settings, parameter vectors are not merely sparse but
dependent in such a way that non-zero coefficients tend to cluster together. We
refer to this form of dependency as "region sparsity." Classical sparse
regression methods, such as the lasso and automatic relevance determination
(ARD), which model parameters as independent a priori, and therefore do not
exploit such dependencies. Here we introduce a hierarchical model for smooth,
region-sparse weight vectors and tensors in a linear regression setting. Our
approach represents a hierarchical extension of the relevance determination
framework, where we add a transformed Gaussian process to model the
dependencies between the prior variances of regression weights. We combine this
with a structured model of the prior variances of Fourier coefficients, which
eliminates unnecessary high frequencies. The resulting prior encourages weights
to be region-sparse in two different bases simultaneously. We develop Laplace
approximation and Monte Carlo Markov Chain (MCMC) sampling to provide efficient
inference for the posterior. Furthermore, a two-stage convex relaxation of the
Laplace approximation approach is also provided to relax the inevitable
non-convexity during the optimization. We finally show substantial improvements
over comparable methods for both simulated and real datasets from brain
imaging.
