  We show that there is a fundamental flaw in the application of modified
gravity theories in cosmology, taking $f(R)$ gravity as a paradigmatic example.
This theory contains a scalar degree of freedom that couples to the matter
stress-energy tensor but not to gravitational energy. However, when applied to
cosmology this theory is unable to distinguish between gravitational and
non-gravitational energy. Hence the cosmological version of the theory does not
coincide with its own macroscopic average, and we show that this leads to
order-one discrepancies. We argue that the same inconsistency is common to many
other modified gravity theories with extra degrees of freedom. Our results put
into question whether these theories can make sense as the cosmological average
of a fundamental theory, hence challenging their physical significance.
