  We present a comprehensive experimental investigation of Yb magnetic dimers
in Yb$_{0.04}$Y$_{0.96}$AlO$_3$, an Yb-doped Yttrium Aluminum Perovskite (YAP)
YAlO$_3$ by means of specific heat, magnetization and high-resolution inelastic
neutron scattering (INS) measurements. In our sample, the Yb ions are randomly
distributed over the lattice and $\sim 7$\% of Yb ions form quantum dimers due
to nearest-neighbor antiferromagnetic coupling along the $c$-axis. At zero
field, the dimer formation manifests itself in an appearance of an inelastic
peak at $\Delta \approx 0.2$~meV in the INS spectrum and a Schottky-like
anomaly in the specific heat. The structure factor of the INS peak exhibits a
cosine modulation along the $L$ direction, in agreement with the $c$-axis
nearest-neighbor intra-dimer coupling. A careful fitting of the low-temperature
specific heat shows that the excited state is a degenerate triplet, which
indicates a surprisingly small anisotropy of the effective Yb-Yb exchange
interaction despite the low crystal symmetry and anisotropic magnetic dipole
contribution, in agreement with previous reports for the Yb parent compound,
YbAlO$_3$ [arXiv:1904.11513, arXiv:1902.04112], and in contrast to
Yb$_2$Pt$_2$Pb [arXiv:1606.01309, arXiv:1907.01067]. The obtained results are
precisely reproduced by analytical calculations for the Yb dimers.
