  The single layered manganite Pr$_{0.22}$Sr$_{1.78}$MnO$_4$ undergoes
structural transition from high temperature tetragonal phase to low temperature
orthorhombic phase below room temperature. The orthorhombic phase was reported
to have two structural variants with slightly different lattice parameters and
Mn-3$d$ levels show orbital ordering within both the variants, albeit having
mutually perpendicular ordering axis. In addition to orbital ordering, the
orthorhombic variants also order antiferromagnetically with different N\'eel
temperatures. Our magnetic investigation on the polycrystalline sample of
Pr$_{0.22}$Sr$_{1.78}$MnO$_4$ shows large thermal hysteresis indicating the
first order nature of the tetragonal to orthorhombic transition. We observe
magnetic memory, large relaxation, frequency dependent ac susceptbility and
aging effects at low temperature, which indicate spin glass like magnetic
ground state in the sample. The glassy magnetic state presumably arises from
the interfacial frustration of orthorhombic domains with orbital and spin
orderings playing crucial role toward the competing magnetic interactions.
