  We discuss the exclusive lepton flavor violating (LFV) decays modes based on
$b\to d\ell_i\ell_j$ and $b\to s\ell_i\ell_j$ by considering the ground state
mesons and baryons. After spelling out the expressions for such decay rates in
a low energy effective theory which includes generic contributions arising from
physics beyond the Standard Model (BSM), we show that the experimental bounds
on meson decays can be used to bound the corresponding modes involving baryons.
We find, for example, $\mathcal{B}(\Lambda_b\to \Lambda\mu\tau)\lesssim 4\times
10^{-5}$. We also consider two specific models and constrain the relevant LFV
couplings by using the low energy observables. In the first model we assume the
Higgs mediated LFV and find the resulting decay rates to be too small to be
experimentally detectable. We also emphasize that the regions favored by the
bounds $\mathcal{B}(h\to\mu\tau)^\mathrm{Atlas}$ and $\mathcal{B}(h\to
e\tau)^\mathrm{Atlas}$ are not compatible with $\mathcal{B}(\mu\to
e\gamma)^\mathrm{MEG}$ to $1\sigma$. In the second model we assume LFV mediated
by a heavy $Z'$ boson and find that the corresponding $b$-hadron branching
fractions can be $\mathcal{O}(10^{-6})$, thus possibly within experimental
reach at LHCb and Belle~II.
