  We present a study of high-quality BaCo$_2$(PO$_4$)$_2$ single crystals via
magnetization, heat-capacity, thermal-expansion and magnetostriction
measurements. Sharp anomalies in the thermodynamic properties at $T_N=3.4\,$K
reveal a long-range antiferromagnetic order in these single-crystalline
samples, which is absent in polycrystalline BaCo$_2$(PO$_4$)$_2$. The
temperature dependent magnetic susceptibilities for in-plane and out-of-plane
magnetic fields are strongly anisotropic and reveal a pronounced easy-plane
anisotropy. A Curie-Weiss analysis implies strong orbital magnetism, as it is
known from the sister compound BaCo$_2$(AsO$_4$)$_2$ that is discussed as a
potential Kitaev spin-liquid material. When applying in-plane magnetic fields
at low temperature, BaCo$_2$(PO$_4$)$_2$ is driven to another ordered phase at
a critical field $H_{C1}\approx 0.11\,$T and then undergoes a further
field-induced transition to a highly polarized paramagnetic phase at
$H_{C2}\approx 0.3\,$T, which is again similar to the case of
BaCo$_2$(AsO$_4$)$_2$. In addition, our lowest-temperature data reveal that the
field-induced transitions in BaCo$_2$(PO$_4$)$_2$ become dominated by thermally
assisted domain-wall motion.
