  We report anisotropic magnetodielectric (MD) coupling in layered van der
Waals (vdW) antiferromagnetic (AFM) FePS$_3$ (N\'eel temperature
$T_{\mathrm{N}}$ $\sim$ 120K) with perpendicular anisotropy. Above $T_N$, while
dielectric response function along \textit{c}-axis shows frequency dependent
relaxations, in-plane data is frequency independent and reveals a deviation
from phonon-anharmonicity in the ordered state, thereby implying a connection
to spin-phonon coupling known to be indicative of onset of magnetic ordering.
At low temperature (below 40 K), atypical anomaly in the dielectric constant is
corroborated with temperature dependent DC and AC susceptibility. The
magnetodielectric response across this anomaly differs significantly for both,
in-plane and out-of-plane cases. We have explained this in terms of
preferential orientation of magnetic AFM-z alignment, implied by the in-plane
structural anisotropy as confirmed by \textit{ab-initio} calculations.
Controlling relative strength of magnetodielectric coupling with magnetic
anisotropy opens up a strategy for tracking subtle modifications of structure,
such as in-plane anisotropy, with potential application to spintronic
technologies.
