  Invariance to nuisance transformations is one of the desirable properties of
effective representations. We consider transformations that form a \emph{group}
and propose an approach based on kernel methods to derive local group invariant
representations. Locality is achieved by defining a suitable probability
distribution over the group which in turn induces distributions in the input
feature space. We learn a decision function over these distributions by
appealing to the powerful framework of kernel methods and generate local
invariant random feature maps via kernel approximations. We show uniform
convergence bounds for kernel approximation and provide excess risk bounds for
learning with these features. We evaluate our method on three real datasets,
including Rotated MNIST and CIFAR-10, and observe that it outperforms competing
kernel based approaches. The proposed method also outperforms deep CNN on
Rotated-MNIST and performs comparably to the recently proposed
group-equivariant CNN.
