  We study information matrices for statistical models by the $L^2$-Wasserstein
metric. We call them Wasserstein information matrices (WIMs), which are analogs
of classical Fisher information matrices. We introduce Wasserstein score
functions and study covariance operators in statistical models. Using them, we
establish Wasserstein-Cramer-Rao bounds for estimations and explore their
comparisons with classical results. We next consider the asymptotic behaviors
and efficiency of estimators. We derive the on-line asymptotic efficiency for
Wasserstein natural gradient. Besides, we study a Poincar\'e efficiency for
Wasserstein natural gradient of maximal likelihood estimation. Several
analytical examples of WIMs are presented, including location-scale families,
independent families, and rectified linear unit (ReLU) generative models.
