  In observational studies, propensity scores are commonly estimated by maxi-
mum likelihood but may fail to balance high-dimensional pre-treatment
covariates even after specification search. We introduce a general framework
that unifies and generalizes several recent proposals to improve covariate
balance when designing an observational study. In- stead of the likelihood
function, we propose to optimize special loss functions---covariate balancing
scoring rules (CBSR)---to estimate the propensity score. A CBSR is uniquely
determined by the link function in the GLM and the estimand (a weighted average
treatment effect). We show CBSR does not lose asymptotic efficiency to the
Bernoulli likelihood in estimating the weighted average treatment effect
compared, but CBSR is much more robust in finite sample. Borrowing tools
developed in statistical learning, we propose practical strategies to balance
covariate functions in rich function classes. This is useful to estimate the
maximum bias of the inverse probability weighting (IPW) estimators and
construct honest confidence interval in finite sample. Lastly, we provide
several numerical examples to demonstrate the trade-off of bias and variance in
the IPW-type estimators and the trade-off in balancing different function
classes of the covariates.
