  Assessing covariate balance (CB) is a common practice in various types of
evaluation studies. Two-sample descriptive statistics, such as the standardized
mean difference, have been widely applied in the scientific literature to
assess the goodness of CB. Studies in health policy, health services research,
built and social environment research, and many other fields often involve a
finite number of units that may be subject to different treatment levels. Our
case study, the California Sugar Sweetened Beverage (SSB) Tax Study, include
332 study cities in the state of California, among which individual cities may
elect to levy a city-wide excise tax on SSB sales. Evaluating the balance of
covariates between study cities with and without the tax policy is essential
for assessing the effects of the policy on health outcomes of interest. In this
paper, we introduce the novel concepts of the pseudo p-value and the
standardized pseudo p-value, which are descriptive statistics to assess the
overall goodness of CB between study arms in a finite study population. While
not meant as a hypothesis test, the pseudo p-values bear superficial similarity
to the classic p-value, which makes them easy to apply and interpret in
applications. We discuss some theoretical properties of the pseudo p-values and
present an algorithm to calculate them. We report a numerical simulation study
to demonstrate their performance. We apply the pseudo p-values to the
California SSB Tax study to assess the balance of city-level characteristics
between the two study arms.
