  The fractional Brownian motion with index $\alpha$ is introduced to construct
the fractional excursion set model. A new mass function with single parameter
$\alpha$ is derived within the formalism, of which the Press-Schechter mass
function (PS) is a special case when $\alpha=1/2$. Although the new mass
function is computed assuming spherical collapse, comparison with the
Sheth-Tormen fitting function (ST) shows that the new mass function of
$\alpha\approx 0.435$ agrees with ST remarkably well in high mass regime, while
predicts more small mass halos than the ST but less than the PS. The index
$\alpha$ is the Hurst exponent, which exact value in context of structure
formation is modulated by properties of the smoothing window function and the
shape of power spectrum. It is conjectured that halo merging rate and merging
history in the fractional set theory might be imprinted with the interplay
between halos at small scales and their large scale environment. And the mass
function in high mass regime can be a good tool to detect the non-Gaussianity
of the initial density fluctuation.
