  This paper proposes an interpretation of RLAIF as Bayesian inference by
introducing distilled Self-Critique (dSC), which refines the outputs of a LLM
through a Gibbs sampler that is later distilled into a fine-tuned model. Only
requiring synthetic data, dSC is exercised in experiments regarding safety,
sentiment, and privacy control, showing it can be a viable and cheap
alternative to align LLMs. Code released at
\url{https://github.com/vicgalle/distilled-self-critique}.
