  We consider channel estimation within pulse-shaping multicarrier
multiple-input multiple-output (MIMO) systems transmitting over doubly
selective MIMO channels. This setup includes MIMO orthogonal frequency-division
multiplexing (MIMO-OFDM) systems as a special case. We show that the component
channels tend to exhibit an approximate joint group sparsity structure in the
delay-Doppler domain. We then develop a compressive channel estimator that
exploits this structure for improved performance. The proposed channel
estimator uses the methodology of multichannel group sparse compressed sensing,
which combines the methodologies of group sparse compressed sensing and
multichannel compressed sensing. We derive an upper bound on the channel
estimation error and analyze the estimator's computational complexity. The
performance of the estimator is further improved by introducing a basis
expansion yielding enhanced joint group sparsity, along with a basis
optimization algorithm that is able to utilize prior statistical information if
available. Simulations using a geometry-based channel simulator demonstrate the
performance gains due to leveraging the joint group sparsity and optimizing the
basis.
