  Verification-aware programming languages such as Dafny and F* provide means
to formally specify and prove properties of programs. Although the problem of
checking an implementation against a specification can be defined mechanically,
there is no algorithmic way of ensuring the correctness of the user-intent
formalization for programs -- that a specification adheres to the user's intent
behind the program. The intent or requirement is expressed informally in
natural language and the specification is a formal artefact. The advent of
large language models (LLMs) has made strides bridging the gap between informal
intent and formal program implementations recently, driven in large parts due
to benchmarks and automated metrics for evaluation.
  Recent work has proposed evaluating {\it user-intent formalization} problem
for mainstream programming languages~\cite{endres-fse24}. However, such an
approach does not readily extend to verification-aware languages that support
rich specifications (containing quantifiers and ghost variables) that cannot be
evaluated through dynamic execution. Previous work also required generating
program mutants using LLMs to create the benchmark. We advocate an alternate
approach of {\it symbolically testing specifications} to provide an intuitive
metric for evaluating the quality of specifications for verification-aware
languages. We demonstrate that our automated metric agrees closely with mostly
GPT-4 generated and human-labeled dataset of roughly 150 Dafny specifications
for the popular MBPP code-generation benchmark, yet demonstrates cases where
the human labeling is not perfect. We believe our work provides a stepping
stone to enable the establishment of a benchmark and research agenda for the
problem of user-intent formalization for programs.
