  As language technologies gain prominence in real-world settings, it is
important to understand how changes to language affect reader perceptions. This
can be formalized as the causal effect of varying a linguistic attribute (e.g.,
sentiment) on a reader's response to the text. In this paper, we introduce
Text-Transport, a method for estimation of causal effects from natural language
under any text distribution. Current approaches for valid causal effect
estimation require strong assumptions about the data, meaning the data from
which one can estimate valid causal effects often is not representative of the
actual target domain of interest. To address this issue, we leverage the notion
of distribution shift to describe an estimator that transports causal effects
between domains, bypassing the need for strong assumptions in the target
domain. We derive statistical guarantees on the uncertainty of this estimator,
and we report empirical results and analyses that support the validity of
Text-Transport across data settings. Finally, we use Text-Transport to study a
realistic setting--hate speech on social media--in which causal effects do
shift significantly between text domains, demonstrating the necessity of
transport when conducting causal inference on natural language.
