  We investigate the low-lying baryon spectrum using three $N_f$=2+1+1
ensembles simulated with physical values of the quark masses and lattice
spacings of 0.080, 0.069, and 0.057 fm. The ensembles are generated using
twisted mass clover-improved fermions and the Iwasaki gauge action. The spatial
length is kept approximately the same at about 5.1 fm to 5.5 fm fulfilling the
condition $m_\pi L$> 3.6. We investigate isospin splitting within isospin
multiples and verify that for most cases the isospin splitting for these
lattice spacing is consistent with zero. In the couple of cases, for which
there is a non-zero value, in the continuum limit, the mass splitting goes to
zero. The baryon masses are extrapolated to the continuum limit using the three
$N_f$=2+1+1 ensembles and are compared to other recent lattice QCD results. For
the strange and charm quark masses, we find, respectively, $m_s$(2
GeV)=99.2(2.7) MeV and $m_c$(3 GeV)=1.015(39) GeV. The values predicted for the
masses of the doubly charmed $\Xi_{cc}^\star$, $\Omega_{cc}$ and
$\Omega_{cc}^\star$ baryons are 3.676(55) GeV, 3.703(51) GeV and 3.803(50) GeV,
respectively, and for the triply charmed $\Omega_{ccc}$ baryon is 4.785(71)
GeV.
