  We present a simple and generic framework for auditing a given textual
conversational system, given some samples of its conversation sessions as its
input. The framework computes a SWAN (Schematised Weighted Average Nugget)
score based on nugget sequences extracted from the conversation sessions.
Following the approaches of S-measure and U-measure, SWAN utilises nugget
positions within the conversations to weight the nuggets based on a user model.
We also present a schema of twenty (+1) criteria that may be worth
incorporating in the SWAN framework. In our future work, we plan to devise
conversation sampling methods that are suitable for the various criteria,
construct seed user turns for comparing multiple systems, and validate specific
instances of SWAN for the purpose of preventing negative impacts of
conversational systems on users and society. This paper was written while
preparing for the ICTIR 2023 keynote (to be given on July 23, 2023).
