  We use direct $N$-body calculations to study the evolution of the unusually
extended outer halo globular cluster Palomar 4 (Pal~4) over its entire lifetime
in order to reproduce its observed mass, half-light radius, velocity dispersion
and mass function slope at different radii.
  We find that models evolving on circular orbits, and starting from a non-mass
segregated, canonical initial mass function (IMF) can reproduce neither Pal 4's
overall mass function slope nor the observed amount of mass segregation.
Including either primordial mass segregation or initially flattened IMFs does
not reproduce the observed amount of mass segregation and mass function
flattening simultaneously. Unresolved binaries cannot reconcile this
discrepancy either. We find that only models with both a flattened IMF and
primordial segregation are able to fit the observations. The initial (i.e.
after gas expulsion) mass and half-mass radius of Pal~4 in this case are about
57000 M${\odot}$ and 10 pc, respectively. This configuration is more extended
than most globular clusters we observe, showing that the conditions under which
Pal~4 formed must have been significantly different from that of the majority
of globular clusters. We discuss possible scenarios for such an unusual
configuration of Pal~4 in its early years.
