  We consider a macroscopic quantum system such as a qubit, interacting with a
bath of fermions as in the Fr\"ohlich polaron model. The interaction
Hamiltonian is thus linear in the macroscopic system variable, and bilinear in
the fermions. Using the recently developed extension of Feynman-Vernon theory
to non-harmonic baths we evaluate quadratic and the quartic terms in the
influence action. We find that for this model the quartic term vanish by
symmetry arguments. Although the influence of the bath on the system is of the
same form as from bosonic harmonic oscillators up to effects to sixth order in
the system-bath interaction, the temperature dependence is nevertheless rather
different, unless rather contrived models are considered.
