  We measure the rest-frame UV luminosity function (LF) at $z \sim 4$
self-consistently over a wide range in absolute magnitude ($-27 \lesssim M_{\rm
UV} \lesssim -20$). The LF is measured with 46,904 sources selected using a
photometric redshift approach over $ \sim 6$ deg$^2$ of the combined COSMOS and
XMM-LSS fields. We simultaneously fit for both AGN and galaxy LFs using a
combination of Schechter or Double Power Law (DPL) functions alongside a single
power law for the faint-end slope of the AGN LF. We find a lack of evolution in
the shape of the bright-end of the LBG component when compared to other studies
at $z \simeq 5$ and evolutionary recipes for the UV LF. Regardless of whether
the LBG LF is fit with a Schechter function or DPL, AGN are found to dominate
at $M_{\rm UV} < -23.5$. We measure a steep faint-end slope of the AGN LF with
$\alpha_{AGN} = -2.09^{+0.35}_{-0.38}$ ($-1.66^{+0.29}_{-0.58}$) when fit
alongside a Schechter function (DPL) for the galaxies. Our results suggest that
if AGN are morphologically selected it results in a bias to lower number
densities. Only by considering the full galaxy population over the transition
region from AGN to LBG domination can an accurate measurement of the total LF
be attained.
