  In this article we give an explicit description of the representation matrix
of a Heisenberg type action constructed by Blanchet, Habegger, Masbaum and
Vogel. We give the matrix in terms of a ribbon graph and its admissible
colorings. We show that components of the representation matrix satisfies the
{\it external edge condition}, which is a natural combinatorial/geometric
condition for maps from the first homology of the graph. We give the explicit
formula of the trace of the action in the case of surfaces with colored
structure using the external edge condition, the Verlinde formula and
elementary counting arguments. Our formula is a generalization of the results
for a surface without colored structure, which are already known.
