  A system of Brownian motions in one-dimension all started from the origin and
conditioned never to collide with each other in a given finite time-interval
$(0, T]$ is studied. The spatial distribution of such vicious walkers can be
described by using the repulsive eigenvalue-statistics of random Hermitian
matrices and it was shown that the present vicious walker model exhibits a
transition from the Gaussian unitary ensemble (GUE) statistics to the Gaussian
orthogonal ensemble (GOE) statistics as the time $t$ is going on from 0 to $T$.
In the present paper, we characterize this GUE-to-GOE transition by presenting
the graphical expansion formula for the moments of positions of vicious
walkers. In the GUE limit $t \to 0$, only the ribbon graphs contribute and the
problem is reduced to the classification of orientable surfaces by genus.
Following the time evolution of the vicious walkers, however, the graphs with
twisted ribbons, called M\"obius graphs, increase their contribution to our
expansion formula, and we have to deal with the topology of non-orientable
surfaces. Application of the recent exact result of dynamical correlation
functions yields closed expressions for the coefficients in the M\"obius
expansion using the Stirling numbers of the first kind.
