  The simulation of large ensembles of particles is usually parallelized by
partitioning the domain spatially and using message passing to communicate
between the processes handling neighboring subdomains. The particles are
represented as individual geometric objects and are associated to the
subdomains. Handling collisions and migrating particles between subdomains, as
required for proper parallel execution, requires a complex communication
protocol. Typically, the parallelization is restricted to handling only
particles that are smaller than a subdomain. In many applications, however,
particle sizes may vary drastically with some of them being larger than a
subdomain. In this article we propose a new communication and synchronization
algorithm that can handle the parallelization without size restrictions on the
particles. Despite the additional complexity and extended functionality, the
new algorithm introduces only minimal overhead. We demonstrate the scalability
of the previous and the new communication algorithms up to almost two million
parallel processes and for handling ten billion (1e10) geometrically resolved
particles on a state-of-the-art petascale supercomputer. Different scenarios
are presented to analyze the performance of the new algorithm and to
demonstrate its capability to simulate polydisperse scenarios, where large
individual particles can extend across several subdomains.
