  We investigate the interplay between Coulomb driven orbital order and
octahedral distortions in strongly correlated Mott insulators due to orbital
dilution, i.e., doping by metal ions without an orbital degree of freedom. In
particular, we focus on layered transition metal oxides and study the effective
spin-orbital exchange due to $d^3$ substitution at $d^4$ sites. The structure
of the $d^3-d^4$ spin-orbital coupling between the impurity and the host in the
presence of octahedral rotations favors a distinct type of orbital polarization
pointing towards the impurity and outside the impurity--host plane. This yields
an effective lattice potential that generally competes with that associated
with flat octahedra and, in turn, can drive an inversion of the crystal field
interaction.
