  Task-Oriented Dialogue (TOD) systems have become crucial components in
interactive artificial intelligence applications. While recent advances have
capitalized on pre-trained language models (PLMs), they exhibit limitations
regarding transparency and controllability. To address these challenges, we
propose a novel approach focusing on inferring the TOD-Flow graph from dialogue
data annotated with dialog acts, uncovering the underlying task structure in
the form of a graph. The inferred TOD-Flow graph can be easily integrated with
any dialogue model to improve its prediction performance, transparency, and
controllability. Our TOD-Flow graph learns what a model can, should, and should
not predict, effectively reducing the search space and providing a rationale
for the model's prediction. We show that the proposed TOD-Flow graph better
resembles human-annotated graphs compared to prior approaches. Furthermore,
when combined with several dialogue policies and end-to-end dialogue models, we
demonstrate that our approach significantly improves dialog act classification
and end-to-end response generation performance in the MultiWOZ and SGD
benchmarks. Code available at: https://github.com/srsohn/TOD-Flow
