  We investigate the effect of structural deformation on the magnetic
properties of orthorhombic CePdAl$_3$ in relation to its tetragonal polymorph.
Utilizing x-ray and neutron diffraction we establish that the crystal structure
has the $Cmcm$ space group symmetry and exhibits pseudo-tetragonal twinning.
According to density-functional calculations the tetragonal-orthorhombic
deformation mechanism has its grounds in relatively small free enthalpy
difference between the polymorphs, allowing either phase to be quenched and
fully accounts for the twinned microstructure of the orthorhombic phase.
Neutron diffraction measurements show that orthorhombic CePdAl$_3$ establishes
long-range magnetic order below $T_\mathrm{N}$=5.29 (5) K characterized by a
collinear, antiferromagnetic arrangement of magnetic moments. Magnetic
anisotropies of orthorhombic CePdAl$_3$ arise from strong spin-orbit coupling
as evidenced by the crystal-field splitting of the $4f$ multiplet, fully
characterised with neutron spectroscopy. We discuss the potential mechanism of
frustration posed by antiferromagnetic interactions between nearest neighbours
in the tetragonal phase, which hinders the formation of long-range magnetic
order in tetragonal CePdAl$_3$. We propose that orthorhombic deformation
releases the frustration and allows for long-range magnetic order.
