  Applications of combinatorial auctions (CA) as market mechanisms are
prevalent in practice, yet their Bayesian Nash equilibria (BNE) remain poorly
understood. Analytical solutions are known only for a few cases where the
problem can be reformulated as a tractable partial differential equation (PDE).
In the general case, finding BNE is known to be computationally hard. Previous
work on numerical computation of BNE in auctions has relied either on solving
such PDEs explicitly, calculating pointwise best-responses in strategy space,
or iteratively solving restricted subgames. In this study, we present a generic
yet scalable alternative multi-agent equilibrium learning method that
represents strategies as neural networks and applies policy iteration based on
gradient dynamics in self-play. Most auctions are ex-post nondifferentiable, so
gradients may be unavailable or misleading, and we rely on suitable
pseudogradient estimates instead. Although it is well-known that gradient
dynamics cannot guarantee convergence to NE in general, we observe fast and
robust convergence to approximate BNE in a wide variety of auctions and present
a sufficient condition for convergence
