  For integers $0 \leq \ell \leq k_{r} \leq k_{c} \leq n$, we give a
description for the Smith group of the incidence matrix with rows (columns)
indexed by the size $k_r$ ($k_c$, respectively) subsets of an $n$-element set,
where incidence means intersection in a set of size $\ell$. This generalizes
work of Wilson and Bier from the 1990s which dealt only with the case where
incidence meant inclusion. Our approach also describes the Smith group of any
matrix in the $\mathbb{Z}$-linear span of these matrices so includes all
integer matrices in the Bose-Mesner algebra of the Johnson association scheme:
for example, the association matrices themselves as well as the Laplacian,
signless Laplacian, Seidel adjacency matrix, etc. of the associated graphs. In
particular, we describe the critical (also known as sandpile) groups of these
graphs. The complexity of our formula grows with the $k$ parameters, but is
independent of $n$ and $\ell$, which often leads to an efficient algorithm for
computing these groups. We illustrate our techniques to give diagonal forms of
matrices attached to the Kneser and Johnson graphs for subsets of size $3$,
whose invariants have never before been described, and recover results from a
variety of papers in the literature in a unified way.
