  NeurAlly-Decomposed Oracle (NADO) is a powerful approach for controllable
generation with large language models. It is designed to avoid catastrophic
forgetting while achieving guaranteed convergence to an entropy-maximized
closed-form optimal solution with reasonable modeling capacity. Despite the
success, several challenges arise when apply NADO to a wide range of scenarios.
Vanilla NADO suffers from gradient vanishing for low-probability control
signals and is highly reliant on a regularization to satisfy the stochastic
version of Bellman equation. In addition, the vanilla implementation of NADO
introduces a few additional transformer layers, suffering from a limited
capacity especially compared to other finetune-based model adaptation methods
like LoRA. In this paper, we propose a improved version of the NADO algorithm,
namely DiNADO (norm-Disentangled NeurAlly-Decomposed Oracles), which improves
the performance of the NADO algorithm through disentangling the step-wise
global norm over the approximated oracle $R$-value for all potential
next-tokens, allowing DiNADO to be combined with finetuning methods like LoRA.
We discuss in depth how DiNADO achieves better capacity, stability and
flexibility with both empirical and theoretical results. Experiments on
formality control in machine translation and the lexically constrained
generation task CommonGen demonstrates the significance of the improvements.
