  If the boundary of a domain in three dimensions is smooth enough, then the
decay rate of the eigenvalues of the Neumann-Poincar\'e operator is known and
it is optimal. In this paper, we deal with domains with less regular boundaries
and derive quantitative estimates for the decay rates of the Neumann-Poincar\'e
eigenvalues in terms of the H\"older exponent of the boundary. Estimates in
particular show that the less the regularity of the boundary is, the slower is
the decay of the eigenvalues. We also prove that the similar estimates in two
dimensions. The estimates are not only for less regular boundaries for which
the decay rate was unknown, but also for regular ones for which the result of
this paper makes a significant improvement over known results.
