  In this paper, we consider a smooth connected finite-dimensional manifold
$M$, an affine connection $\nabla$ with holonomy group $H^{\nabla}$ and
$\Delta$ a smooth completely non integrable distribution. We define the
$\Delta$-horizontal holonomy group $H^{\;\nabla}_\Delta$ as the subgroup of
$H^{\nabla}$ obtained by $\nabla$-parallel transporting frames only along loops
tangent to $\Delta$. We first set elementary properties of
$H^{\;\nabla}_\Delta$ and show how to study it using the rolling formalism
(\cite{ChitourKokkonen}). In particular, it is shown that $H^{\;\nabla}_\Delta$
is a Lie group. Moreover, we study an explicit example where $M$ is a free
step-two homogeneous Carnot group and $\nabla$ is the Levi-Civita connection
associated to a Riemannian metric on $M$, and show that in this particular case
the connected component of the identity of $H^{\;\nabla}_\Delta$ is compact and
strictly included in $H^{\nabla}$.
