  Using tools from stochastic geometry, we develop a framework to model the
downlink rate coverage probability of a user in a given small cell network
(SCN) with massive MIMO-enabled wireless backhauls. The considered SCN is
composed of a mixture of small cells that are configured in either in-band or
out-of-band backhaul modes with a certain probability. The performance of the
user in the considered hierarchical network is limited by several sources of
interference such as the backhaul interference, small cell base station
(SBS)-to-SBS interference and the SI. Moreover, due to the channel hardening
effect in massive MIMO, the backhaul links experience long term channel effects
only, whereas the access links experience both the long term and short term
channel effects. Consequently, the developed framework is flexible to
characterize different sources of interference while capturing the
heterogeneity of the access and backhaul channels. In specific scenarios, the
framework enables deriving closed-form coverage probability expressions. Under
perfect backhaul coverage, the simplified expressions are utilized to optimize
the proportion of in-band and out-of-band small cells in the SCN in
closed-form. Finally, few remedial solutions are proposed that can potentially
mitigate the backhaul interference and in turn improve the performance of
in-band FD wireless backhauling. Numerical results investigate the scenarios in
which in-band wireless backhauling is useful and demonstrate that maintaining a
correct proportion of in-band and out-of-band FD small cells is crucial in
wireless backhauled SCNs.
