  This paper describes a massively parallel code for a state-of-the art thermal
lattice- Boltzmann method. Our code has been carefully optimized for
performance on one GPU and to have a good scaling behavior extending to a large
number of GPUs. Versions of this code have been already used for large-scale
studies of convective turbulence. GPUs are becoming increasingly popular in HPC
applications, as they are able to deliver higher performance than traditional
processors. Writing efficient programs for large clusters is not an easy task
as codes must adapt to increasingly parallel architectures, and the overheads
of node-to-node communications must be properly handled. We describe the
structure of our code, discussing several key design choices that were guided
by theoretical models of performance and experimental benchmarks. We present an
extensive set of performance measurements and identify the corresponding main
bot- tlenecks; finally we compare the results of our GPU code with those
measured on other currently available high performance processors. Our results
are a production-grade code able to deliver a sustained performance of several
tens of Tflops as well as a design and op- timization methodology that can be
used for the development of other high performance applications for
computational physics.
