  We present a new method for performing global redistributions of
multidimensional arrays essential to parallel fast Fourier (or similar)
transforms. Traditional methods use standard all-to-all collective
communication of contiguous memory buffers, thus necessary requiring local data
realignment steps intermixed in-between redistribution and transform steps.
Instead, our method takes advantage of subarray datatypes and generalized
all-to-all scatter/gather from the MPI-2 standard to communicate discontiguous
memory buffers, effectively eliminating the need for local data realignments.
Despite generalized all-to-all communication of discontiguous data being
generally slower, our proposal economizes in local work. For a range of strong
and weak scaling tests, we found the overall performance of our method to be on
par and often better than well-established libraries like MPI-FFTW, P3DFFT, and
2DECOMP&FFT. We provide compact routines implemented at the highest possible
level using the MPI bindings for the C programming language. These routines
apply to any global redistribution, over any two directions of a
multidimensional array, decomposed on arbitrary Cartesian processor grids (1D
slabs, 2D pencils, or even higher-dimensional decompositions). The high level
implementation makes the code easy to read, maintain, and eventually extend.
Our approach enables for future speedups from optimizations in the internal
datatype handling engines within MPI implementations.
