  Observations of X-ray flares from Gamma Ray Bursts (GRBs) imply strong
constraints on possible physical models. We provide a general discussion of
these. In particular, we show that in order to account for the relatively flat
and weak optical flux during the X-ray flares, the size of the emitting region
should be $\lesssim 3\times 10^{14}$cm. The bolometric luminosity of flares
also strongly constrain the energy budget, and are inconsistent with late time
activity of a central engine powered by the spin-down of a magnetar. We provide
a simple toy model according to which flares are produced by an outflow of
modest Lorentz factor (a few tens instead of hundreds) that is launched more or
less simultaneously with the highly relativistic jet which produced the prompt
gamma-ray emission. The "slower" moving outflow produces the flare as it
reaches its photosphere. The existence of such a component would naturally
resolve the observational challenges imposed by flares, outlined in this work.
