coordination-plos1-revised-v2.tex

% Template for PLoS
% Version 1.0 January 2009
%
% To compile to pdf, run:
% latex plos.template
% bibtex plos.template
% latex plos.template
% latex plos.template
% dvipdf plos.template

\documentclass[10pt]{article}
\usepackage{chngcntr}
% amsmath package, useful for mathematical formulas
\usepackage{amsmath}
% amssymb package, useful for mathematical symbols
\usepackage{amssymb}

% graphicx package, useful for including eps and pdf graphics
% include graphics with the command \includegraphics
\usepackage{graphicx}

% cite package, to clean up citations in the main text. Do not remove.
\usepackage{cite}

\usepackage{color} 

% Use doublespacing - comment out for single spacing
%\usepackage{setspace} 
%\doublespacing


% Text layout
\topmargin 0.0cm
\oddsidemargin 0.5cm
\evensidemargin 0.5cm
\textwidth 16cm 
\textheight 21cm

% Bold the 'Figure #' in the caption and separate it with a period
% Captions will be left justified
\usepackage[labelfont=bf,labelsep=period,justification=raggedright]{caption}

% Use the PLoS provided bibtex style
\bibliographystyle{plos2009}

% Remove brackets from numbering in List of References
\makeatletter
\renewcommand{\@biblabel}[1]{\quad#1.}
\makeatother


% Leave date blank
\date{}

\pagestyle{myheadings}
%% ** EDIT HERE **
\usepackage{epstopdf}
\usepackage{wrapfig}
\usepackage{epsfig, subfigure}
\usepackage{color}


%% ** EDIT HERE **
%% PLEASE INCLUDE ALL MACROS BELOW
\newcommand{\BEQ}{\begin{equation}}
\newcommand{\EEQ}{\end{equation}}
\newcommand{\BEA}{\begin{eqnarray}}
\newcommand{\EEA}{\end{eqnarray}}
\newcommand{\BGA}{\begin{gather}}
\newcommand{\EGA}{\end{gather}}
\newcommand{\comment}[1]{}
\newcommand{\aram}[1]{{\color{red} #1}}
\newcommand{\shuyang}[1]{{\color{blue} #1}}
\newcommand{\revise}[1]{{\color{black} #1}}
\newcommand{\revisetwo}[1]{{\color{black} #1}}
\newcommand{\revisethree}[1]{{\color{black} #1}}

%% END MACROS SECTION

\begin{document}

% Title must be 150 characters or less
\begin{flushleft}
{\Large
\textbf{ \revise{Understanding Confounding Effects in Linguistic Coordination: an Information-Theoretic Approach} }
}
% Insert Author names, affiliations and corresponding author email.
\\
Shuyang Gao$^{1,\ast}$, 
Greg Ver Steeg$^{1}$, 
Aram Galstyan$^{1}$
\\
\bf{1} Information Sciences Institute, University of Southern California, Marina del Rey, CA
%\\
%\bf{2} Author2 Dept/Program/Center, Institution Name, City, State, Country
%\\
%\bf{3} Author3 Dept/Program/Center, Institution Name, City, State, Country
%\\
\\
$\ast$ E-mail: Corresponding sgao@isi.edu
\end{flushleft}

% Please keep the abstract between 250 and 300 words
\section*{Abstract}
We suggest an information-theoretic approach for measuring stylistic coordination in dialogues. The proposed measure has a simple predictive interpretation and can account for various confounding factors through proper conditioning. We revisit some of the previous studies that reported strong signatures of stylistic accommodation, and find that a significant part of the observed coordination can be attributed to a simple confounding effect -  length coordination. Specifically, longer utterances tend to be followed by longer responses, which gives rise to spurious correlations in the other stylistic features. We propose a test to distinguish correlations in length due to contextual factors (topic of conversation, user verbosity, etc.) and turn-by-turn coordination. We also suggest a test to identify whether stylistic coordination persists even after accounting for length coordination and contextual factors. 

% Please keep the Author Summary between 150 and 200 words
% Use first person. PLoS ONE authors please skip this step. 
% Author Summary not valid for PLoS ONE submissions.   

\section*{Introduction}
Communication Accommodation Theory~\cite{Giles1991} states that people tend to adapt their communication style (voice, gestures, word choice, etc.) in response to the person with whom they interact.  Originally, experiments on linguistic accommodation were confined to small scale laboratory settings with a handful of participants. 
The recent proliferation of digital (or digitized) communication data offers an opportunity to study nuances of human communication behavior on much larger scales. A number of recent studies have indicated presence of stylistic coordination in communication~\cite{Ireland2010,Gonzales2010,Danescu-Niculescu-Mizil2011WWW,Danescu-Niculescu-Mizil2012WWW}, where one person's use of a linguistic feature (e.g. prepositions) increases the probability that a response will include the same feature. Linguistic style  coordination (or {\em matching})  has been used to predict relationship  stability~\cite{Ireland2010} and negotiation outcomes~\cite{Taylor2008}, understand group cohesiveness~\cite{Gonzales2010}, and infer relative social status and  power relationships among individuals~\cite{Danescu-Niculescu-Mizil2012WWW}. 
 
%%%% old version
\comment{
%Here we suggest an information-theoretic approach for measuring stylistic coordination in dialogs. Namely, given a temporally ordered sequence of utterances (verbal or electronic statements depending on the context) by two individuals, we characterize their stylistic coordination with {\em time-shifted} (or time-delayed) mutual information. The proposed coordination measure characterizes the dependence between the stylistic features of the original post and the response. In addition, we provide a computational framework to account for confounders when measuring  stylistic coordination. 
%
%We revisit some of the case studies where linguistic coordination was reported and demonstrate that the observed correlations in linguistic features can be explained by confounding factors rather than linguistic accommodation theory. 
%In particular, we show that  most of the correlation between stylistic features is explained by the  length of utterance. This is because longer utterance from user $Y$ tends to solicit a longer response from user $X$~\cite{Niederhoffer2002}. Thus, if the probability of an utterance containing a feature, e.g. \textit{prepositions} or \textit{words whose second letter is ``r''}, depends only on length, this will create the illusion of stylistic coordination on the given feature.
%
%Next, we focus on length correlations, and  examine whether it is due to turn-by-turn coordination between the participants, or can be attributed to other contextual factors.  We construct a statistical permutation test and demonstrate unequivocally that the turn-by-turn length coordination in dialogues indeed takes place. We develop a similar test for stylistic features, and demonstrate that for some features, the remnant coordination (after conditioning on length) cannot be explained by contextual factors alone and has to be due to turn-by-turn level coordination between the speakers. 
}

Most reports of linguistic style coordination have been based on correlational analysis. Thus, such claims are susceptible to various confounding effects. For instance, it is known that there is significant length coordination in dialogues,  in the sense that a longer utterance from user $Y$ tends to solicit a longer response from user $X$~\cite{Niederhoffer2002}. Thus, if the probability of an utterance containing a feature, e.g. \textit{prepositions} or \textit{words whose second letter is ``r''}, depends only on length, this will create the illusion of stylistic coordination on the given feature.

Here we attempt to remedy the problem and propose an information-theoretic framework for characterizing stylistic coordination in dialogues. Namely, given a temporally ordered sequence of utterances (verbal or electronic statements depending on the context) by two individuals, we characterize their stylistic coordination with   {\em time-shifted}  mutual information. The proposed coordination measure characterizes the dependence between the stylistic features of the original post and the response. In addition, we provide a computational framework to account for confounders when measuring  stylistic coordination. 

We revisit some of the case studies where linguistic coordination was reported and demonstrate that a significant part of the observed correlations in linguistic features can indeed be explained by length coordination rather than stylistic accommodation. In particular, most stylistic features that exhibit statistically significant correlation exhibit little to no correlation after length coordination has been taken into account. 

We also focus on the observed length correlations, and  examine whether it is due to turn-by-turn coordination between the participants, or can be attributed to other contextual factors.  We construct a statistical permutation test and demonstrate unequivocally that turn-by-turn length coordination in dialogues indeed takes place. Finally, we develop a similar test for stylistic features, and demonstrate that \revise{at least for one of the datasets}, the remnant coordination (after conditioning on length) cannot be explained by contextual factors alone and has to be due to turn-by-turn level coordination between the speakers. 




\section*{Measuring Stylistic Coordination}
\subsection*{Representing Stylistic Features}
\label{RSC}
To represent stylistic features in utterances, we use Linguistic Inquiry Word  Count (LIWC)~\cite{Pennebaker2007}, which is a dictionary-based encoding scheme that has been used extensively  for evaluating emotional and psychological dimensions in various text corpora. The latest version of the LIWC dictionary contains around 4500 words and word stems. Each word or word stem belongs to one or more word categories or subcategories. Various LIWC categories include positive and negative emotion, function words, pronouns, articles, and so on.  Here we focus on eight LIWC categories that have been used in previous studies~\cite{Danescu-Niculescu-Mizil2012WWW}: articles, auxiliary verbs, conjunctions, high-frequency adverbs, impersonal pronouns, personal pronouns, prepositions, and quantifiers. Utterances are represented as eight-component binary vectors indicating the presence or absence of each linguistic marker~\cite{Danescu-Niculescu-Mizil2012WWW}. %Generally speaking, stylistic coordination might manifest itself as correlation across different features. However, following previous work, we focus on stylistic coordination for a particular marker. 


%\paragraph{Model}
%{\bf do we want to include the Bernoulli mode here?} We now describe a simple generative model where there is no direct influence between the users, but  that still produces coordination. 
\subsection*{Information-theoretic measure  of coordination} 
\label{IT}
Each dialogue is a sequence of utterance exchanges between two participants. Following ~\cite{Danescu-Niculescu-Mizil2011WWW,Danescu-Niculescu-Mizil2012WWW} we binarize the stylistic features of utterances, so that a dialogue is represented as $\{o_k^m, r_k^m\}_{k=1}^K$, where $o_k^m,r_k^m=\{0,1\}$ indicate the absence or presence of the stylistic marker $m$, and $K$ is the total number of exchanges in a dialogue.  Since we focus on coordination between the same stylistic markers, we will drop the superscript $m$ from now on. We use the convention $O$ to represent the originator -- the person who is producing the original utterance in a single exchange, $R$ to represent the respondent -- the person who is replying to the originator. %\aram{Do we want to say something about cross-feature linguistic coordination?}. 

%Point-wise Mutual Information (PMI) between two random variables $O$ and $R$ is defined as
%\BEQ
%\text{pmi}(o,r) = \log p(o|r) - \log p(o)
%\EEQ
%\BEQ
%\text{pmi}(x,y) =  \log \frac{p(x,y)}{p(x)p(y)}= \log \frac{p(x|y)}{p(x)} = \log p(x|y) - \log p(x)
%\EEQ
%Note that $\text{pmi}(o=1,r=1)$ is similar to the coordination measure used in Refs.~\cite{Danescu-Niculescu-Mizil2012WWW}.

%The expectation of $\text{pmi}(o,r)$ over the joint distribution $p(o,r)$ is the 

Let $p(o,r)$ be the joint distribution of the random variables $O$ and $R$. We characterize the amount of stylistic coordination using Mutual Information (MI)~\cite{CoverThomas2006}; see \revisethree{S1 File} for a brief overview of basic information theoretic concepts: 
%\BEA
%I(O:R) &=& \sum_{o,r}p(o,r) \log \frac{p(o,r)}{p(o)p(r)} \nonumber \\
%           &=&H(O) - H(O|R)
%\label{eq_MI}
%\EEA
\BEA
I(O:R) = H(O) - H(O|R)
\label{eq_MI}
\EEA
%\BEQ
%I(X:Y) = \sum_{x,y}p(x,y) \log \frac{p(x,y)}{p(x)p(y)}=H(X) - H(X|Y)
%\label{eq1}
%\EEQ
where $H(O)=-\sum p(o)\log p(o)$ is the Shannon entropy of $O$, and $H(O|R)$ is the entropy of $O$ conditioned on $R$. Note that in our case the arguments are temporarily ordered: $O$ is always the initial utterance, and $R$ is the response, so that Eq.~\ref{eq_MI} in fact defines time-shifted mutual information. Thus, even though MI is symmetric with respect to  its argument, the coordination between two users may be asymmetric. 

\revise{ Recall that mutual information between two variables measures the average reduction in the uncertainty of one variable, if we know the other variable. Thus, in essence, the proposed measure of stylistic coordination quantifies how the use of a marker $m$ in an utterance of $O$'s  can help to predict $R$'s usage of $m$ in the immediate response. In contrast to linear correlation measures, mutual information is well suited for handling strongly non-linear dependencies.}

 %Dialogue coordination is a behavior that people unconsciously tend to accommodate each other's choices of function-word classes, i.e., eight LIWC categories described above. By introducing \textit{mutual information} $I(O:R)$, we are actually quantifying how the use of marker class $m$ in an utterance of $O$'s  can predict $R$'s usage of $m$ in the immediate reply. We want to see how much uncertainty of $m$ in $R$ is reduced by knowing the usage of $m$ in its previous utterance of $O$.}

We measure the correlation between two variables \textit{after} conditioning on a third variable, $Z$, via Conditional Mutual Information, defined as 
\BEQ
I(O:R|Z) = H(O|Z) - H(O|R,Z).
\label{eq:cmi}
\EEQ
Below we will use CMI to account for the confounding effect of the utterance length by conditioning on it. Namely, the actual stylistic accommodation, after accounting for the length coordination, is given by $I(O:R|L_R)$, where $L_R$ is the length of the utterance by user $R$. 

  
\subsection*{Estimating mutual information from data}  
\label{Estimation}
Given a set of samples $\{o_k, r_k\}_{k=1}^K$, our goal is to estimate mutual information between $O$ and $R$. We could do this by first calculating the empirical distribution $p(o,r)$ and then using Eq.~\ref{eq_MI}. However, it is known that this naive {\em plug-in} estimator tends to underestimate the entropy of a system. Instead, here we use the statistical bootstrap method introduced by DeDeo et al.~\cite{dedeo}, which attempts to reduce the bias of the naive estimator by estimating a bootstrap correction term. The estimate of bias comes from comparing the entropy of the empirical distribution to estimates of entropy from several bootstrap datasets drawn randomly according to the empirical distribution. See \cite{dedeo} for more details.

%While the above discrete estimator works well for evaluating mutual information between two binary variables, it is not very useful for evaluating conditional mutual information when conditioning on the length, due to limited number of sample we have. Instead, we will use continuos estimator introduced by  Kraskov \aram{write here about CMI}

While the above discrete estimator works well for evaluating mutual information between discrete stylistic variables, it is not very useful for evaluating mutual information between two length variables, due to limited number of samples we have. Instead, we will use a continuous estimator introduced by Kraskov et al.~\cite{kraskov2004estimating}.  This non-parametric estimator searches the $k$-nearest neighbors to each point, and then average the mutual information estimated from the neighborhood of each point. It has been shown that this estimator is asymptotically unbiased and consistent. 
Discussion of different entropy estimators can be found in~\cite{Wang2009} and references therein.
  
%\vspace{1mm}


%\newpage
\section*{Length  as a confounding factor}  
\label{result_1}%maybe change a title here 
We  applied our coordination measures to two datasets previously studied in~\cite{Danescu-Niculescu-Mizil2012WWW}: oral transcripts from the Supreme Court hearings, and discussion among Wikipedia editors. In the Supreme Court Data, there are 11 Judges and 311 Lawyers conversing with each other. We obtain 51,498 utterances from all the dialogues among 204 cases. In the Wikipedia dataset, users are classified into two categories, Administrators, or \textit{\em Admins}, and {\em non-Admins}. All of the users interact with each other on Wikipedia \textit{talk} pages, where they discuss issues about specific Wikipedia pages. We focus on dialogues where each participant make at least two exchanges within a dialogue, which results in over 30,000 utterances. %For both datasets, ~\cite{Danescu-Niculescu-Mizil2012WWW} reported significant stylistic coordination 

Ideally, we would like to calculate linguistic accommodation between any pair of individuals $O$ and $R$ who have participated in a dialogue. Unfortunately,  most pair-wise exchanges are rather short and do not produce sufficient samples for evaluating mutual information or conditional mutual information. Instead, we group the individuals according to their roles, and then use aggregated samples to calculate stylistic coordination between the groups.  The groups correspond to Judges and Lawyers for the Supreme Court data, and Admins and non-Admins for the Wikipedia data. 
%\aram{do we want to say something about Simpson's paradox here?} 
%\shuyang{I think Simpson's paradox can be mentioned in the next section when talking about contextual factors}

\begin{figure}[htbp]
\centering
\subfigure[][]{
    \includegraphics[width=.45\columnwidth]{figs/Lawyers_to_Judges_new.eps}
\label{fig_supreme_court_style_a}
   } 
  \subfigure[][]{%
   \includegraphics[width=.45\columnwidth]{figs/Judges_to_Lawyers_new.eps}
 \label{fig_supreme_court_style_b}
  }
     \caption{ 
     \revise{\textbf{Coordination measures for the Supreme Court data.} The red (blue) dots give the true CMI (MI). The green dots represent CMI under the null hypothesis that there is no coordination after conditioning. (a) Lawyers coordinating to Judges. (b) Judges coordinating to Lawyers. In both figures, the conditional mutual information is significantly smaller than the mutual information for all eight stylistic features, indicating length is a confounding factor.}}
     \label{fig_supreme_court_style}
    \end{figure}
    
      
    \begin{figure}[htbp]
    \centering
\subfigure[]{
    \includegraphics[width=.45\columnwidth]{figs/Non-admins_to_Admins_new.eps} \label{fig_wiki_style_a}
    } 
    \subfigure[]{
    \includegraphics[width=.45\columnwidth]{figs/Admins_to_Non-admins_new.eps} \label{fig_wiki_style_b}
    }
    
    \caption{\textbf{Coordination measures for the Wikipedia data.} (a) Non-admins coordinating to Admins. (b) Admins coordinating to Non-admins. Symbols have the same interpretation as in the previous plot.}
    \label{fig_wiki_style}
    
    \end{figure}


%\begin{figure}[htbp]
%\centering
%
%    \includegraphics[width=1.0\columnwidth]{figs/untitled.pdf} \label{fig_supreme_court_style_a}
%   
%      \caption{ 
%     We non-parametrically estimate confidence intervals for CMI under the assumption that length explains none of the correlation (blue bars) or all of it (red bars). The red (green) dots give the true CMI (MI). (a) Lawyers coordinating to Judges. (b) Judges coordinating to Lawyers.}
%    \end{figure}
  
    
Fig.~\ref{fig_supreme_court_style}  describes stylistic coordination for the Supreme Court data as measured by $I(O:R)$ and $I(O:R|L_R)$. The bias in estimators for conditional mutual information and mutual information are generally different. Therefore, rather than estimating mutual information directly, we use a conditional mutual information estimator where we condition on randomly permuted values for $L_R$. We repeat this procedure for \revisetwo{four hundred times} to produce $99\%$ confidence intervals for $I(O:R)$ (blue bars). The green bars give the 99\% confidence intervals in case there is no stylistic coordination by estimating CMI with \textit{R\rq{}s} utterances permuted (erasing any stylistic coordination). 

The blue dots show the mutual information between the corresponding stylistic features, and suggest strong linguistic correlations between the groups. This effect, however, is strongly diminished after conditioning on the length of utterances (red dots). For instance, the coordination scores on features {\em Impersonal Pronoun}, {\em Article}, and {\em Auxiliary Verb} are reduced by factors of  $\sim 6.7$, $\sim 4.8$, and $5.3$, respectively, after conditioning on length. \revise{For the feature {\em Conjunction}, the 99\% confidence interval of coordination score is above the confidence interval of \textit{zero information} before conditioning, and falls into the confidence interval of \textit{zero information} after conditioning.} Similarly, in Fig.~\ref{fig_supreme_court_style}(b), the coordination scores for five out of eight markers ({\em Impersonal Pronoun}, {\em Article}, {\em Adverb}, {\em Preposition}, {\em Quantifier}) become practically zero after conditioning, suggesting that  the observed coordination in those stylistic features are due to length correlations. 

%Specifically, in Figure~\ref{fig1a} where lawyers coordinate to judges, the score of feature \textit{preposition} decreases to zero after conditioning on length, which means this coordination can be fully explained by length other than any other linguistic accommodation. In Figure~\ref{fig1b}, scores of feature \textit{impersonal pronoun} and \textit{quantifier} also decreased to the zero confidence interval after conditioning on length. 

%\aram{check that the colors in the text and figure match } \shuyang{colors in the text and figure DO match}. 





A similar picture holds for the Wikipedia dataset shown in Fig.~\ref{fig_wiki_style}. Again, we observe non-zero mutual information in all the features. However, this correlation is significantly diminished after conditioning on length.  In fact, both non-admins coordinating to admins (Fig.~\ref{fig_wiki_style}(a)) and admins coordinating to non-admins (Fig.~\ref{fig_wiki_style}(b)) have an extremely weak  signal after conditioning on length (all below 0.005). In particular, for non-admins coordinating to admins(Fig.~\ref{fig_wiki_style}(a)), the red dots of five out of eight features lie in the zero conditional mutual information confidence interval. For these five features in Fig.~\ref{fig_wiki_style}(b), we cannot rule out the null hypothesis that all stylistic coordination is due to phenomenon of length coordination.  

Another interesting observation is that there is significant asymmetry, or {\em directionality}, in stylistic coordination. For instance, by comparing Figs.~\ref{fig_supreme_court_style}(a) and~\ref{fig_supreme_court_style}(b) we see that the mutual information is significantly higher from lawyers to judges than vice versa. A similar (albeit less pronounced) asymmetry is present for the Wikipedia data as well. This type of asymmetry has been used to suggest that the relative strength of stylistic accommodation reflects social status~\cite{Danescu-Niculescu-Mizil2012WWW}. However, Figs.~\ref{fig_supreme_court_style} and~\ref{fig_wiki_style} illustrate that the asymmetry is drastically  weakened after conditioning on length (red dots), suggesting that the phenomenon of higher stylistic coordination  from lawyers to judges (and from non-admins to admins for the Wikipedia dataset) is due to the confounding effect of length. Unfortunately, a direct assessment of this effect in a single conversation is not feasible due to the insufficient number of utterances for calculating conditional mutual information. Nevertheless, in \revisethree{S2 File} we suggest a different approach for addressing the above problem,  and find that asymmetry in stylistic coordination can be explained by asymmetry in length coordination. 


To conclude this section, we note  that some of the correlations in stylistic features persist even after conditioning on length. One can ask whether this remnant correlation is due to turn-by-turn level linguistic coordination, or can be  attributed to other confounding factors. We address this question in detail later in the text.



\section*{Understanding Length Coordination}\label{ULC}

As discussed in the previous section, the observed correlations in linguistic features can be attributed to coordination in the length of utterances. Here we analyze this phenomenon in more detail. In particular, we are interested whether the observed length correlations are due to turn-by-turn coordination, or can be attributed to other contextual factors. For instance, consider a scenario that in one conversation, {\em Alice} and {\em Bob} are always conversing using short statements, while in another conversation they exclusively use long statements, perhaps due to different topics of conversation. Length coordination is found if data from these two conversation is aggregated, however, this coordination only reflects {\em Alice}'s and {\em Bob}'s response to the topic of conversation. 
More generally, aggregating data might lead to effects similar to Simpson's paradox~\cite{Simpson1951}. 
 

%Ideally, length coordination needs to be estimated separately in each dialogue. However, in most cases, the number of samples in a dialogue is not sufficient to evaluate the length coordination of each dialogue separately. Thus, we have to estimate coordination using aggregated samples. 

%To overcome the limitation, we aggregate all the exchanges from different dialogues of the originator $X$ and the replier $Y$, as long as $X \in \mbox{Group 1}, Y \in \mbox{Group 2}$. We denote this aggregated coordination as \textit{Group to Group Coordination}.

%\subsection{Contextual Factor as a Confounder}
%\shuyang{this paragraph is similar to the last paragraph, do we want to combine them?}Notice that the length of a particular utterance may not only depend on the previous utterance he/she is replying to, but also rely on lots of other factors. For example, if Alice is an introvert, then she tends to speak less words regardless of how long the previous utterance is. Also, an utterance is also influenced by the context, i.e., theme of the dialogue. Certain kinds of dialogues might require longer utterances  compared to others. By aggregating the conversations, all of the factors like \textit{personality}, \textit{theme} would be confounded with each other and cause \textit{fake} coordination. An apt example is that in one conversation, $Alice$ and $Bob$ are always conversing in short word counts, while in the other conversation, $Alice$ and $Bob$ are talking in longer word counts. Length coordination can be found if aggregating these two conversations, however, this coordination is only due to $Alice$ and $Bob$ are accommodating to the themes of two conversations. 
\begin{figure}[htbp]
\centering
\includegraphics[width=0.3\columnwidth]{figs/Fig3.pdf} 
\caption{ \textbf{A Bayesian network model for length coordination.} The network containing contextual factors, $C$, the length of an utterance, $L_O^{(t)}$, and the length of the response, $L_R^{(t)}$. \revisetwo{(a) The lengths are correlated only due to contextual factors. (b) The lengths are correlated due to both contextual factors and potential effect of turn-by-turn level coordination (represented with the dotted line).}
 }
 \label{fig_length_model}
\end{figure}


To understand the possible extent of various confounding factors (we call them \textit{contextual factors}),  consider the Bayesian network model  that incorporates both \textit{contextual factors} and \textit{length coordination}, as shown in Fig.~\ref{fig_length_model}.  Here  $L_O$ and $L_R$ are  random variables representing the length of an utterance by the originator $O$ and the respondent $R$, respectively. \revisetwo{In the model with both solid and dashed lines in Fig.~\ref{fig_length_model}(b), $L_O$ explicitly influences $L_R$. While if we only have the soild lines in Fig.~\ref{fig_length_model}(a), $L_R$ is independent of $L_O$ after conditioning on the context $C$. Thus, the model in Fig.~\ref{fig_length_model}(a) assumes that there is only contextual coordination while Fig.~\ref{fig_length_model}(b) implies turn-by-turn coordination.} \revisethree{Note that in principle, the contextual factor $C$ can vary within a single conversation, for example, the theme of a conversation may change as time goes by. But for simplicity, we will assume that the contextual factor $C$ does not change within the dialogue or conversation.}

%A natural question can be asked about this model is whether the dotted line in Fig.~\ref{fig_length_model} exists. We can raise the following null hypothesis:
%
%${{\rm H}_{length}}$: There is no \textit{turn-by-turn} level coordination in length.
%
%If the hypothesis is accepted, the dotted line in Fig~\ref{fig_length_model} disappears, otherwise the dotted line becomes soild, meaning the existence of \textit{turn-by-turn} level length coordination.
%
\subsection*{Information-theoretic characterization of length coordination} 

A direct measure of Turn-by-turn Length Coordination ($TLC$) is given by the following conditional mutual information:
\BEQ
TLC = I(L_O:L_R|C)
\label{eq_TLC}
\EEQ
%where $L_O$, $L_R$ and $C$ are defined in section \ref{LM}.
Additionally,  we define the Overall Length Coordination ($OLC$) as
 \BEQ
 OLC = I(L_O:L_R) 
 \EEQ
Thus, $OLC$  captures not only the length coordination in a turn-by-turn level, but also the confounding behaviors between $L_O$, $L_R$ and $C$. In fact, $OLC$ can be decomposed into two items:
 \BEQ
 OLC = TLC + I(L_O:L_R:C)
 \label{eq_OLC} 
 \EEQ
 The second item of right hand side in Eq.~\ref{eq_OLC} indicates the multivariate mutual information(MMI) (also known as \textit{interaction information}~\cite{McGill54} or \textit{co-information}~\cite{bell03}), and characterizes the amount of shared information between $L_O$, $L_R$ and $C$. 

A straightforward method to test for turn-by-turn coordination is to evaluate  $TLC$ described in Eq.~\ref{eq_TLC}. Indeed, $L_O$ and $L_R$ are conditionally independent of $C$ if and only if $TLC=0$. However, direct evaluation of $TLC$ is not possible due to the lack of sufficient number of samples, e.g., the number of exchanges within a specific dialogue. Nevertheless, it is possible to test the turn-by-turn length coordination by a non-parametric statistical test as shown below. 

\subsection*{Turn-by-Turn Length Coordination Test}  
\label{TTLCT}
%\shuyang{need to define the null hypothesis $H_{length}$.}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\revisetwo{
Our null hypothesis is that there is no turn-by-turn coordination, so that all observed correlations are due to contextual factors. We now describe a procedure for testing this hypothesis. 

We denote the pairwise set of exchanges in a specific dialogue $c$ from originator $o$ and respondent $r$ as:
\BEQ
{S_{o \leftarrow r}^c} = \left\{ {o_c^k,r_c^k} \right\}_{k = 1}^{{K_c}}
\label{pairwise_ex}
\EEQ
where ${o_c^k}, {r_c^k}$ indicate the $k$th exchange (two utterances) by the originator $o$ and respondent $r$ in dialogue $c$, and $K_c$ represents the total number of exchanges in $c$. We also define the aggregated set of exchanges of user $o \in O$ and user $r \in R$ as:
\BEQ
{S_{O \leftarrow R}} = \bigcup\limits_{o \in O,r \in R} {\bigcup\limits_{c \in {C_{o,r}}} {{S_{o \leftarrow r}^c}} } 
\EEQ
where $C_{o,r}$ represents all the dialogues that involved user $o$ and $r$. %We call  $S_{A \leftarrow B}$ as exchanges $B$ to $A$. 
We can rewrite $S_{O \leftarrow R}$ element-wise as 
\BEQ
{S_{O \leftarrow R}} = \left\{ {{O_k},{R_k},{C_k}} \right\}_{k = 1}^N
\label{eq_S}
\EEQ
where $N=\left| {{S_{O \leftarrow R}}} \right|$ representing number of samples. For each triplet of right hand side in Eq.~\ref{eq_S}, $R_k$ is the reply utterance to $O_k$ in the dialogue $C_k$. Finally, from $S_{O \leftarrow R}$ we obtain the set %define the set %Thus, by using Eq.~\ref{eq_S}, the length set of $S_{A \leftarrow B}$ is defined as 
\BEQ
%L\left( {{S_{A \leftarrow B}}} \right) = \left\{ {len\left( {{A_k}} \right),len\left( {{B_k}} \right),{C_k}} \right\}_{k = 1}^N
L( {{S_{O \leftarrow R}}} ) = \left\{ {len\left( {{O_k}} \right),len\left( {{R_k}} \right)} \right\}_{k = 1}^N
\label{length_sample}
\EEQ 
where $len\left(  \cdot   \right)$ is a function representing the length of an utterance.


Consider now another sample, which is obtained by randomly permuting the respondent $r$'s utterances in the set $S_{o \leftarrow r,c}$: 
\BEQ
\widehat S_{o \leftarrow r}^c = \left\{ {o_c^k,\widehat r_{c}^{k}} \right\}_{k = 1}^{{K_c}}
\EEQ 
where $\{ {\widehat r}_c^k  \}_{k = 1}^{{K_c}}$ is a random permutation of $\left\{ {r_c^k} \right\}_{k = 1}^{{K_c}}$. By aggregation, we have,
\BEQ
\widehat S_{{{O \leftarrow R}}} = \bigcup\limits_{o \in A,r \in B} {\bigcup\limits_{c \in {C_{o,r}}} {\widehat S^c_{{{o \leftarrow r}}}} }  = \left\{ {{O_k},\widehat R_k,{C_k}} \right\}_{k = 1}^N \nonumber
\label{permute_set}
\EEQ
and
\BEQ
%L\left( {\widehat S_{{{A \leftarrow B}}}} \right) = \left\{ {len\left( {{A_k}} \right),len\left( {\widehat B_{{k}}} \right),{C_k}} \right\}_{k = 1}^N
L( {\widehat S_{O \leftarrow R}} ) = \{ {len ( {{O_k}} ),len( {\widehat R_{{k}}} )} \}_{k = 1}^N
\label{length_permute}
\EEQ
Let us assume there is no turn-by-turn coordination, so that $L_O$ and $L_R$ are conditionally independent from each other given $C$. Then, it is easy to see that under this null model, the samples $L( {{S_{O \leftarrow R}}} )$ and $L( {\widehat S_{O \leftarrow R}} )$ have the same likelihood, e.g., they are statistically equivalent. In other words, $L( {\widehat S_{O \leftarrow R}} )$  can be viewed as a new sample from the same distribution $p( l_o,l_r)$. This observation suggests the following test: We first estimate $OLC$ from the sample $L( { S_{O \leftarrow R}} )$ (denoted as $OLC_0$) and then using the within-dialogue shuffled samples $L( {\widehat S_{O \leftarrow R}} )$ (denoted as $OLC_1$). Under the null hypothesis, these two estimates should coincide. Conversely, if $OLC_0\neq OLC_1$, then the null hypothesis is rejected, suggesting that there is turn-by-turn length coordination. 
}

 The above procedure, which we call Turn-by-Turn Length Coordination Test,  is a conditional Monte Carlo test~\cite{Pesarin_Book}. The main advantage of this non-parametric test is that it requires a smaller sample size and does not need to make particular distribution assumptions. The test is non-parametric in two ways: the permutation procedure is non-parametric as well as the estimation of mutual information. We also note that in the context of stylistic coordination, a similar test was used in Ref.~\cite{Danescu-Niculescu-Mizil2011CIC}. 


%\subsection{Results}
%
%The same two datasets in Section~\ref{result_1} are tested using the Turn-by-Turn Length Coordination Test described above.
%\subsubsection{Supreme Court}

\begin{figure}[htbp]
\centering
\subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/length_coordination_supreme_court_new.eps} \label{fig_length_supreme_court}
    } 
    \subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/length_coordination_wikipedia_new.eps} \label{fig_length_wiki}
    }

\caption{\revise{\textbf{Turn-by-turn length coordination test.} (a) Supreme Court dataset. (b) Wikipedia dataset. In both two subfigures, $OLC_1$ is significantly smaller than $OLC_0$.}  
\label{fig:Lcoord}
}
\end{figure}
The results of this test are shown in Fig.~\ref{fig:Lcoord}. For the Supreme Court data, Fig.~\ref{fig:Lcoord}(a) shows that both Lawyers to Judges and Judges to Lawyers have non-zero mutual information ($OLC_0$) before permutation. The  Turn-by-Turn Length Coordination test shows that the mutual information decreases significantly after permutation(green confidence intervals, $OLC_1$), rejecting the null hypothesis that $L_O$ and $L_R$ are independent after conditioning on the contextual factor $C$. In other words, the contagion of length exists from the original utterance to the reply on a turn-by-turn level.

%\subsubsection{Wikipedia Discussion Board}
%
%\begin{figure}[htbp]
%\centering
% \includegraphics[width=1\columnwidth]{figs/length_coordination_wikipedia_new.eps}
%\caption{Length Coordination in Wikipedia across two categories of users -- \textbf{A} represents \textbf{Admins}, \textbf{N} represents \textbf{Non-Admins}
%}
% \label{fig_length_wiki}
%\end{figure}

 For the results on the Wikipedia discussion board in Fig.~\ref{fig:Lcoord}(b),  we are also able to reject the null hypothesis. Notice that the degree of mutual information $OLC$ is higher for Wikipedia than for the Supreme Court. However, one cannot make a general conclusion about the exact magnitude of turn-by-turn length coordination ($TLC$) simply by calculating the loss, i.e., $OLC_0-OLC_1$. 



\section*{Revisiting Stylistic Coordination}
\label{sec:revisit}
%We saw in Section~\ref{result_1} that, while length explained most of the observed coordination in stylistic features, for some of those features had reduced but yet statistically significant correlations even after conditioning on length. In this section we examine whether this remnant coordination is due to turn-by-turn coordination, or can be explained using other contextual factors. 
 
 We demonstrated in the previous section that strong correlations in utterance length explain most of the observed stylistic coordination. However, in some situations, there are statistically significant non-zero signals even after conditioning on length, e.g., the first feature dimension {\em Personal Pronoun} in Figs.~\ref{fig_supreme_court_style}(a) and~\ref{fig_supreme_court_style}(b). 
 We now proceed to  examine this remnant coordination. Specifically, we are interested in the following question: Does the non-zero conditional mutual information (after conditioning on length) represent  turn-by-turn level stylistic coordination, or is it due to other contextual factors? 


Toward this goal, consider the  Bayesian network in Fig.~\ref{fig_length_style_model}, which depicts conditional independence relations between the length variables $L_O$ and $L_R$; stylistic variables $F^m_O$ and $F^m_R$ with respect to a style feature $m$, and the contextual (dialogue) variable $C$. The solid arrow from $L_O$ to $L_R$ reflects our findings from the last section about the existence of turn-by-turn length coordination. The dashed arc between the features $F_O^m$ and $F_R^m$ characterizes turn-by-turn stylistic coordination. Finally, the grey  arcs between $C$, $F_O^m$ and $C$, $F_R^m$ indicate possible contextual coordination. 
\begin{figure}[htbp]

\centering
\includegraphics[width=0.3\columnwidth]{figs/fig_length_style_model_before_shuffling_new.pdf}
\caption{ \textbf{A Bayesian network for linguistic style coordination.} $L_O$ and $L_R$ represent length of the respondent and length of the originator respectively. $F^m_O$ and $F^m_R$ represent a specific style feature variable for the respondent and originator. %\aram{make the hypothesized arcs (C to features, Fo to Fr) different }
     }
\label{fig_length_style_model}
\end{figure}

We use conditional mutual information to measure the Turn-by-turn Stylistic Coordination (TSC) with respect to a specific style feature $m$:
\BEQ
TSC = I(F^m_O:F^m_R|C, L_R)
\label{eq_TSC}
\EEQ
where $F^m_O$, $F^m_R$ are binary variables indicating the feature $m$ appears or not in an utterance. Also, the Overall Stylistic Coordination(OSC) is defined as
 \BEQ
 OSC = I(F^m_O;F^m_R|L_R) 
 \label{eq_OSC}
 \EEQ
 Thus, $OSC$ is exactly the conditional mutual information introduced in Eq.~\ref{eq:cmi}. Note that, even after conditioning on length, $F^m_O$ and $F^m_R$ are still dependent of each other because they are sharing the contextual factor $C$. ($F^m_O \leftarrow C \rightarrow F^m_R$ is called a \textit{d-connected} path in~\cite{Pearl2000})%\aram{do we need this sentence here? perhaps better to restate it in plain English}. 

 Again, a direct measure of turn-by-turn stylistic coordination corresponds to non-zero $TSC$ in Eq.~\ref{eq_TSC}. However, $TSC$ is hard to evaluate due to lack of sufficient samples. Furthermore, the  shuffling test from the previous sections is not directly applicable here either, because it needs to be done in way that keeps the correlations between $L_O$ and $L_R$ intact: In other words, one can exchange utterances that have the same lengths. Since most dialogues are rather short, this type of shuffling test is not feasible, and one needs an alternative approach. 
 
 %Instead, we we have to design appropriate statistical tests that can confirm the presence of turn-by-turn stylistic coordination.


  
%Ideally, one would assume the contextual factors always play a role due to data aggregation, but if we can rule out the influence of the contextual factors, then any non-zero conditional mutual information can be only explained by turn-by-turn stylistic coordination, i.e., $OSC=TSC$. In such case, the degree of turn-by-turn stylistic coordination can be scored by $OSC$, as well as $TSC$.
 
 \subsection*{Turn-by-Turn Stylistic Coordination Test} \label{TTSCT}

 Our proposed test is based on the following idea: if we can rule out the influence of the contextual factors on stylistic correlations, then any non-zero conditional mutual information can be only explained by turn-by-turn stylistic coordination, i.e., $OSC=TSC$. Thus, the null hypothesis is that there is contextual level coordination in stylistic features. We emphasize that by contextual coordination, we are actually referring to the links from $C$ to $F_O$ and $C$ to $F_R$ in Fig.~\ref{fig_length_style_model}. %\shuyang{maybe add the color of the links here}

We follow the same notation and methodology used in previous sections. By Eq.~\ref{eq_S}, let us denote the mixed length and stylistic feature set of ${S_{O \leftarrow R}}$ as:
\revisetwo{
\BEQ
L{F_m}\left( {{S_{O \leftarrow R}}} \right) = \left\{len\left( {{O_k}} \right),len\left( {{R_k}} \right), {{f_m}\left( {{O_k}} \right),{f_m}\left( {{R_k}} \right),{C_k}} \right\}  \nonumber 
\label{eq_LF} 
\EEQ
}
where ${f_m}\left(  \cdot  \right)$ is a binary function represents whether the style feature $m$ in an utterance appears or not.

%Similar to Eq.~\ref{eq_S}, Eq.~\ref{eq_LF} can also be viewed as samples from the joint probability distribution ${p_{{L_O},{L_R},F_O^m,F_R^m,C}}\left( {l_o,{l_r},f_o^m,f_r^m,c} \right)$, where $L_O$ and $L_R$ are length random variables, $F_O^m$ and $F_R^m$ are stylistic random variables in feature $m$, C is the dialogue random variable. 

%According to the null hypothesis, the joint distribution of features and length when conditioned on the dialogues allows the following factorization (see Fig~\ref{fig_length_style_model}): w\BEA
%p( l_o,l_r,f_o^m,f_r^m|c ) &=&p ( l_o|c )  {p( l_r|l_o,c} )  \nonumber \\
%&\times&p( f_o^m|l_o,c )  p( f_r^m|f_o^m,l_r,c ) \nonumber \\
%\EEA

Consider now the shuffling procedure: we randomly permute respondent's utterances within a dialogue and obtain the set $\widehat S_ {{{O \leftarrow R}}}$ in Eq.~\ref{permute_set}.  We also define the length and feature set of $\widehat S_{O \leftarrow R}$: 
\revisetwo{
\BEQ
L{F_m}( {\widehat S_{{{O \leftarrow R}}}} ) = \{ {len ( {{O_k}} ),len ( \widehat R_k  ),{f_m} ( O_k),  f_m ( \widehat R_k ),{C_k}} \} \nonumber 
\label{eq_LF_shuffled}
\EEQ
}
Clearly,  the permutation destroys the turn-by-turn level coordination in both length and style. Thus, any remnant correlation must be due to contextual coordination, e.g., the fork $F^m_O \leftarrow C \rightarrow F^m_R$. This provides a straightforward test for the existence of contextual coordination. Indeed, we simply need to estimate the overall stylistic coordination $OSC_1$ using the shuffled sample $L{F_m}( {\widehat S_{{{O \leftarrow R}}}} )$. If $OSC_1$ is larger than zero, then there is necessarily contextual coordination. On the other hand, if $OSC_1=0$, then all the observed stylistic correlations (calculated using the original non-shuffled sample) must be due to turn-by-turn stylistic coordination. 
 



% while the conditional probability distribution ${p_{{L_O},f_O^m|C}}\left( {{l_o},f_o^m|c} \right)$ and ${p_{{L_R},f_R^m|C}}\left( {{l_r},f_r^m|c} \right)$ remain the same. Therefore, we write down the after-permutation joint probability distribution of $L{F_m}\left( {\widehat {{S_{A \leftarrow B}}}} \right)$ as:
%\BEA
%&&{p_{{L_O},\widehat {{L_R}},F_O^m,\widehat {F_R^m},C}}\left( {{l_o},\widehat {{l_r}},f_o^m,\widehat {f_r^m},c} \right) = \nonumber \\
%&&{p_C}\left( c \right){p_{{L_O},F_O^m|C}}\left( {{l_o},f_o^m|c} \right){p_{{L_R},F_R^m|C}}\left( {\widehat {{l_r}},\widehat {f_r^m}|c} \right) \nonumber \\
%\EEA
%\begin{figure}[htbp]
%\centering
% \includegraphics[width=0.3\textwidth]{figs/fig_length_style_model_after_shuffling.pdf}
%     \caption{Bayesian network after shuffling test for ${{\rm H}}_{style}^m$. Two dotted line represent the vanishing of turn-by-turn level coordination.}
%     \label{fig_length_style_model_shuffled}
%\end{figure}
%Figure~\ref{fig_length_style_model_shuffled} shows the Bayesian network after permutation under the null hypothesis ${{\rm H}}_{style}^m$.  By \textit{d-seperation}, $F_O^m$ and $\widehat{F_R^m}$ are conditional independent of $C$ and $\widehat {L_R}$, but not conditional independent of $\widehat {L_R}$ alone. Following this intuition, quantity $OSC$ is used in testing the null hypothesis:

%$OSC_1$: Estimation of $OSC$ from examples $\left\{ {len\left( {\widehat {{B_k}}} \right),{f_m}\left( {{A_k}} \right),{f_m}\left( {\widehat {{B_k}}} \right)} \right\}_{k = 1}^N$, i.e., the conditional mutual informaion $I\left( {F_O^m:\widehat {F_R^m}|\widehat {{L_R}}} \right)$.
%
%If $OSC_1$ is not statistically significant larger than zero, the null hypothesis ${{\rm H}}_{style}^m$ is ruled out and we then can follow $OSC$ in Eq.~\ref{eq_OSC} to capture the turn-by-turn stylistic coordination.

%A simple method to test  ${{\rm H}_{length}}$ is to calculate $TLC$ described in Eq.~\ref{eq_TLC}. $L_O$ and $L_R$ are conditional independent of $C$ if and only if $TLC=0$. However, there not exists sufficient statistics for the direct calculation, i.e., the number of exchanges within a dialogue is too small to estimate the conditional mutual information. Therefore, the quantity $OLC$ is used in the following test.
\begin{figure}[htbp]
\centering
\subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/Lawyers_to_Judges_turn_by_turn_new.eps} \label{fig_supreme_court_turn_by_turn_a}
 
    } 
    \subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/Judges_to_Lawyers_turn_by_turn_new.eps} \label{fig_supreme_court_turn_by_turn_b}
    }
    
    \caption{\revise{\textbf{Turn-by-turn stylistic coordination test for Supreme Court data.} (a) Lawyers coordinating to Judges. (b) Judges coordinating to Lawyers.  (Blue bars indicate the overall stylistic coordination($OSC$) before the test). One can see that after shuffling, values of $OSC_1$ are within the zero-information confidence intervals.}}
    \label{fig_supreme_court_turn_by_turn}
\end{figure}
Let us first consider the results of the above test for the Supreme Court data. From Figs.~\ref{fig_supreme_court_turn_by_turn}(a) and~\ref{fig_supreme_court_turn_by_turn}(b), one can see that for all the features, the corresponding CMI $OSC_1$ are within the zero-information confidence intervals, indicating that non-zero conditional mutual information  ($OSC$ before shuffling) cannot be attributed to contextual factors. In other words, the remnant correlations that are not explained by length coordination must be due to turn-by-turn level coordination. 

\begin{figure}[htbp]
\centering
\subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/Non_admins_to_Admins_turn_by_turn_new.eps} \label{fig_wiki_turn_by_turn_a}
 
    } 
    \subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/Admins_to_Non_admins_turn_by_turn_new.eps} \label{fig_wiki_turn_by_turn_b}
   	 }
	\caption{\revise{\textbf{Turn-by-turn stylistic coordination test for Wikipedia data.} (a) Non-admins coordinating to Admins. (b) Admins coordinating to Non-admins. (Blue bars indicate the overall stylistic coordination($OSC$) before the test). One cannot rule out the null hypothesis that the remnant stylistic coordination is due to the contextual factors.}}
        \label{fig_wiki_turn_by_turn}
\end{figure}
\revise{The situation is different for Wikipedia data. Indeed, Figs.~\ref{fig_wiki_turn_by_turn}(a) and~\ref{fig_wiki_turn_by_turn}(b) show that for the stylistic features with statistically significant remnant correlations even after conditioning on length ($OSC$), the results of the above permutation tests are rather inconclusive. Namely,  although the confidence intervals of $OSC_1$ do overlap with the zero information confidence intervals, one cannot state unequivocally that they are zero. In other words, one cannot rule out the null hypothesis that the remnant stylistic coordination is due to the contextual factors rather than turn-by-turn coordination.} 
%shuyang{how to get a conclusion from the figures is still questionable}.



%\subsection{An alternative test}
\comment{
It is possible to design alternative test based on a more fine-grained, token-level generative model. Specifically, let us represent an utterance as  $b_k=\{w_{k,j}^m\}_{j=1}^{l_k}$, where $l_k$ is the length (number of words) in utterance $k$, and $w_{k,j}^m=\{0,1\}$ is a binary variable, indicating whether the $j$th word in the $k$th utterance contains the stylistic feature $m$ or not.  Generally speaking, $w_{k,j}^m$-s might have complex dependencies within an utterance (or a dialogue). However, let us neglect those dependencies and assume that $w_{k,j}^m$-s are iid variable distributed according to some distribution.

Consider now the model in Fig.~\ref{fig_length_style_model}, without turn-by-turn stylistic coordination. In this case, $w_{k,j}^m$ are conditionally independent from each other given the context (or dialogue) $c$. This suggests the following statistical test, in which one permutes $w_{k,j}$ within a dialogue and across different utterances while leaving utterance lengths intact. If the null model (of no turn-by-turn coordination) is correct, then such a permutation should not affect the observed correlations between the features.  Conversely, if the correlation is destroyed due to permutation, then there is necessarily a turn-by-turn coordination. 

The preliminary results of this test suggest that most of the remnant correlations are indeed due to turn-by-turn coordination. However, we emphasize that this test is based on the additional assumption that each word by a given speaker within a dialogue and/or sentence is conditionally independent of other words. The validity of this assumption needs to be tested further. 
}



\section*{Discussion}


In conclusion, we have suggested an information theoretic framework for measuring and analyzing stylistic coordination in dialogues. \revisetwo{We first extracted the stylistic features from the dialogue of two participants and then used \textit{Mutual Information(MI)} as a theoretically motivated measure of dependence to characterize the amount of stylistic coordination between the originator and the respondent in the dialogue. Moreover, by introducing \textit{Conditional Mutual Information(CMI)}, which allows us to measure the correlation between two variables \textit{after} conditioning on a third variable, we are able to more accurately gauge stylistic accommodation by controlling for confounding effects like length coordination.}


\revisetwo{We then used the proposed method to revisit some of the previous studies that had reported strong stylistic coordination}. While the suggestion that one person's use of, e.g., prepositions will (perhaps unconsciously) lead the other to use more prepositions is fascinating, our results indicate that previous studies have vastly overstated the extent of stylistic coordination. In particular, we showed that a significant part of the observed stylistic coordination can be attributed to the confounding effect of length coordination. \revise{We find that for both Supreme Court and Wikipedia data, the coordination score is greatly diminished after conditioning on length. We also find that the significant asymmetry in stylistic coordination shown in the previous study~\cite{Danescu-Niculescu-Mizil2012WWW} is drastically weakened after conditioning on length. In fact, our results indicate that the asymmetry in length coordination can explain almost all the observed asymmetry in stylistic coordination.} 

% Shuyangs original version of simpson's paradox paragraph:
%\revisetwo{It is well-known that data aggregation may lead to effects similar to Simpson's paradox. In an information-theoretic framework setting, the ``paradox'' is interpreted as follows by taking section \textit{understanding length coordination} as an apt example: for $L_O$, $L_R$ and $C$, the mutual information $I(L_O:L_R)>0$, while $I(L_O:L_R|C)=0$. If we only calculate the aggregated data, i.e., $I(L_O:L_R)$, there will be artificial mutual information between $L_O$ and $L_R$. The ideal way is to calculate $I(L_O:L_R|C)$ directly, however, there are not enough samples for us to calculate the conditional mutual information. How do we still determine whether $I(L_O:L_R|C)$ is zero or not while aggregating all the data? We thus designed non-parametric statistical tests to well solve this problem caused by data aggregation. More importantly, in information theory, the Simpon-like paradox will not evoke any surprise, all we need is a causal story to evoke the surprise, e.g., the Bayesian network models shown in Figure~\ref{fig_length_model} and Figure~\ref{fig_length_style_model}.}

\revisetwo{Simpson's paradox provides a famous example of how correlations observed in a population can disappear or even be reversed after conditioning on sub-populations. 
In an information-theoretic framework setting, a similar ``paradox'' can be seen in the example illustrated by Fig.~\ref{fig_length_model}: for $L_O$, $L_R$ and $C$, the mutual information $I(L_O:L_R)>0$, while $I(L_O:L_R|C)=0$. If we only look at the aggregated data, averaging over all contexts, $C$, i.e., $I(L_O:L_R)$, there will be artificial mutual information between $L_O$ and $L_R$. Ideally, we could calculate $I(L_O:L_R|C)$ directly, however, there may not be enough samples for us to calculate the conditional mutual information for all values of $C$. How can we still determine whether $I(L_O:L_R|C)$ is zero or not while using all the data? We thus designed non-parametric statistical tests to solve this problem in general while making full use of the available data. 
More importantly, because these information-theoretic quantities directly reflect constraints on graphical models, the mystery of Simpson-like paradoxes is replaced with concrete alternatives for generative stories as depicted in Figs.~\ref{fig_length_model} and~\ref{fig_length_style_model}.}


\revise{We also observed that for some of the stylistic markers, there was diminished but still statistically significant correlations even after conditioning on length. We again designed a non-parametric statistical test for analyzing this remnant coordination more thoroughly. Our findings suggest that for the Supreme Court data, the remnant coordination cannot be fully explained by other contextual factors. Instead, we postulate that the remnant correlations in the Supreme Court data is due to turn-by-turn level coordination. For the Wikipedia data, however, our results are less conclusive, and we cannot draw any conclusion about turn-by-turn stylistic coordination. Thus, caution must be taken when making general claims about the possible origin of stylistic coordination in different settings.} 



It is possible to develop alternative tests based on a more fine-grained, token-level generative models. The main idea behind such a test is to shuffle the word tokens uttered by an individual within each dialogue, which should destroy turn-by-turn coordination. Our preliminary results based on this test suggest that most of the remnant correlations are indeed due to turn-by-turn coordination. However, we emphasize that this test requires an additional assumption whose validity needs to be verified, namely that the words used by a given speaker within a conversation are independent and identically distributed (i.i.d.). Furthermore, the test assumes stationarity, i.e.,  that the contextual factors do not vary within the course of the dialogue.  \revise{While this assumption seems reasonable in the dialogue settings considered here, it is important to note that deviations from stationarity might be yet another serious obstacle for identifying stylistic influences~\cite{win2013,steeg2012statistical}. Indeed, if we relax the stationarity condition, then any observed correlation in stylistic features might be due to temporal evolution rather than direct influence. And since any permutation-based test destroys temporal ordering, it cannot differentiate between those two possibilities.}

%{\bf Shuyang, I think everything else is done, except this paragraph. I'm still not happy with it and will look into it more carefully today.} \revise{
%In future work, it will be interesting to understand how the results presented here are  related to the broader notions of coordination and alignment in the dialogues. For instance, Brennan \& Clark~\cite{brennan1996conceptual} accounted for lexical entrainment in conversations by concept pacts, which is a shared conceptualization established by the interlocutors, {\bf Shuyang}. On the other hand, Pickering \& Garrod~\cite{pickering2004toward} proposed the \textit{interactive alignment model} for dialogue alignment. Instead of  requiring a very complex mechanism, this model offers a rather simple strategy for dialogue alignment via priming, e.g., structural priming, which helps us to understand many important types of repetitive phenomenon~\cite{pickering2008structural}, {\bf Shuyang}. Our empirical investigation first verifies the existence of `stylistic' dialogue alignment by results of the non-zero overall stylistic coordination(OSC) in two different settings: Supreme Court and Wikipedia. Secondly, our non-parametric turn-by-turn stylistic coordination test provides empirical evidence suggesting the validity of turn-by-turn level coordination in Supreme Court data, in agreement with P\&G's interactive alignment model. We hope our information-theoretic framework and the non-parametric test can further shed light on these issues.}

\revise{While our work focuses on linguistic style matching, we believe that the information-theoretic method proposed can be useful for studying more general types of linguistic coordination in dialogues, such as structural priming~\cite{bock1986syntactic,pickering2008structural}, or lexical entrainment~\cite{brennan96lexical,brennan1996conceptual}. Recall that according to the structural priming hypothesis, the presence of a certain linguistic structure in an utterance affects the probability of seeing the same structure later in the dialogue. This type of turn-by-turn coordination can naturally be captured by (time-shifted) mutual information between properly defined linguistic variables. Furthermore, using the permutation tests described here, it should be possible to differentiate between {\em historical} and {\em ahistorical} mechanisms of lexical entrainment~\cite{brennan1996conceptual}. Indeed, the former mechanism assumes some type of influence/coordination between the speakers that helps them to arrive at a common conceptualization.  The ahistorical mechanism, on the other hand, assumes that the speaker's choice of each term is an independent event affected by the informativeness and availability of the term, and some other factors, which, in our terminology, is analogous to  contextual coordination. }



In a broader context, we note that sociolinguistic analysis  has been used for assessing and predicting societally important outcomes such as health behaviors, suicidal intent, and emotional well-being, to name a few examples~\cite{Pennebaker1999,Stirman01072001,Campbell2003,Rude2004,Boals01092005}. Thus, it is imperative that such predictions are based on sound theoretical and methodological principles. Here we suggest that information theory provides a powerful computational framework for testing various hypotheses, and furthermore, is flexible enough  to account for various confounding variables. Recent advances in information-theoretic estimation are shifting these approaches from the theoretical realm into practical and useful techniques for data analysis. We hope that this work will contribute to the development of mathematically principled tools that enable computational social scientists to draw meaningful conclusions from socio-linguistic phenomena.

% You may title this section "Methods" or "Models". 
% "Models" is not a valid title for PLoS ONE authors. However, PLoS ONE
% authors may use "Analysis" 

% Do NOT remove this, even if you are not including acknowledgments
\section*{Acknowledgments}
This research was supported in part by Defense Advanced Research Projects Agency(DARPA) grant No. W911NF--12--1--0034.

%\section*{References}
% The bibtex filename
%\bibliography{template}
\begin{thebibliography}{10}
\providecommand{\url}[1]{\texttt{#1}}
\providecommand{\urlprefix}{URL }
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi:\discretionary{}{}{}#1}\else
  \providecommand{\doi}{doi:\discretionary{}{}{}\begingroup
  \urlstyle{rm}\Url}\fi
\providecommand{\bibAnnoteFile}[1]{%
  \IfFileExists{#1}{\begin{quotation}\noindent\textsc{Key:} #1\\
  \textsc{Annotation:}\ \input{#1}\end{quotation}}{}}
\providecommand{\bibAnnote}[2]{%
  \begin{quotation}\noindent\textsc{Key:} #1\\
  \textsc{Annotation:}\ #2\end{quotation}}
\providecommand{\eprint}[2][]{\url{#2}}

\bibitem{Giles1991}
Giles H, Coupland N, Coupland J (1991) Accommodation theory: Communication,
  context, and consequence, Cambridge: Cambridge University Press.
\newblock pp. 1--68.
\bibAnnoteFile{Giles1991}

\bibitem{Ireland2010}
Ireland ME, Slatcher RB, Eastwick PW, Scissors LE, Finkel EJ, et~al. (2011)
  Language style matching predicts relationship initiation and stability.
\newblock Psychological Science 22: 39--44.
\bibAnnoteFile{Ireland2010}

\bibitem{Gonzales2010}
Gonzales AL, Hancock JT, Pennebaker JW (2010) Language style matching as a
  predictor of social dynamics in small groups.
\newblock Communication Research 37: 3-19.
\bibAnnoteFile{Gonzales2010}

\bibitem{Danescu-Niculescu-Mizil2011WWW}
Danescu-Niculescu-Mizil C, Gamon M, Dumais S (2011) Mark my words!: linguistic
  style accommodation in social media.
\newblock In: Proceedings of the 20th international conference on World wide
  web. WWW '11, pp. 745--754.
\newblock \doi{10.1145/1963405.1963509}.
\newblock \urlprefix\url{http://doi.acm.org/10.1145/1963405.1963509}.
\bibAnnoteFile{Danescu-Niculescu-Mizil2011WWW}

\bibitem{Danescu-Niculescu-Mizil2012WWW}
Danescu-Niculescu-Mizil C, Lee L, Pang B, Kleinberg J (2012) Echoes of power:
  language effects and power differences in social interaction.
\newblock In: Proceedings of the 21th international conference on World wide
  web. WWW '12, pp. 699--708.
\bibAnnoteFile{Danescu-Niculescu-Mizil2012WWW}

\bibitem{Taylor2008}
Taylor PJ, Thomas S (2008) Linguistic style matching and negotiation outcome.
\newblock Negotiation and Conflict Management Research 1: 263--281.
\bibAnnoteFile{Taylor2008}

\bibitem{Niederhoffer2002}
Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social
  interaction.
\newblock Journal of Language and Social Psychology 21: 337-360.
\bibAnnoteFile{Niederhoffer2002}

\bibitem{Pennebaker2007}
Pennebaker JW, Francis ME (2007).
\newblock Linguistic inquiry and word count: A computerized text analysis
  program.
\newblock \urlprefix\url{http://www.liwc.net/}.
\bibAnnoteFile{Pennebaker2007}

\bibitem{CoverThomas2006}
Cover TM, Thomas JA (2006) Elements of Information Theory (Wiley Series in
  Telecommunications and Signal Processing).
\newblock Wiley-Interscience.
\bibAnnoteFile{CoverThomas2006}

\bibitem{dedeo}
DeDeo S, Hawkins RXD, Klingenstein S, Hitchcock T (2013) Bootstrap methods for
  the empirical study of decision-making and information flows in social
  systems.
\newblock Entropy 15: 2246--2276.
\bibAnnoteFile{dedeo}

\bibitem{kraskov2004estimating}
Kraskov A, St{\"o}gbauer H, Grassberger P (2004) Estimating mutual information.
\newblock Physical Review E 69: 066138.
\bibAnnoteFile{kraskov2004estimating}

\bibitem{Wang2009}
Wang Q, Kulkarni SR, Verd\'{u} S (2009) Universal estimation of information
  measures for analog sources.
\newblock Found Trends Commun Inf Theory 5: 265--353.
\bibAnnoteFile{Wang2009}

\bibitem{Simpson1951}
Simpson EH (1951) The interpretation of interaction in contingency tables.
\newblock Journal of the Royal Statistical Society, Series B (Statistical
  Methodology) 13: 238--241.
\bibAnnoteFile{Simpson1951}

\bibitem{McGill54}
McGill W (1954) Multivariate information transmission.
\newblock Information Theory, Transactions of the IRE Professional Group on 4:
  93-111.
\bibAnnoteFile{McGill54}

\bibitem{bell03}
Bell AJ (2003) The co-information lattice.
\newblock In: Proceedings of the 4th international symposium on Independent
  Component Analysis and Blind Source Separation. pp. 921--926.
\bibAnnoteFile{bell03}

\bibitem{Pesarin_Book}
Pesarin F, Salmaso L (2010) Permutation Tests for Complex Data: Theory,
  Applications and Software.
\newblock John Wiley \& Sons.
\bibAnnoteFile{Pesarin_Book}

\bibitem{Danescu-Niculescu-Mizil2011CIC}
Danescu-Niculescu-Mizil C, Lee L (2011) Chameleons in imagined conversations: A
  new approach to understanding coordination of linguistic style in dialogs.
\newblock In: Proceedings of the 2Nd Workshop on Cognitive Modeling and
  Computational Linguistics. Stroudsburg, PA, USA: Association for
  Computational Linguistics, CMCL '11, pp. 76--87.
\newblock \urlprefix\url{http://dl.acm.org/citation.cfm?id=2021096.2021105}.
\bibAnnoteFile{Danescu-Niculescu-Mizil2011CIC}

\bibitem{Pearl2000}
Pearl J (2000) Causality: Models, Reasoning, and Inference.
\newblock New York, NY, USA: Cambridge University Press.
\bibAnnoteFile{Pearl2000}

\bibitem{win2013}
{Ver Steeg} G, Galstyan A (2013) Minimal assumption tests for contagion in
  observational social network studies.
\newblock In: Workshop on Information in Networks(WIN).
\bibAnnoteFile{win2013}

\bibitem{steeg2012statistical}
{Ver Steeg} G, Galstyan A (2013) Statistical tests for contagion in
  observational social network studies.
\newblock In: AISTATS'13.
\bibAnnoteFile{steeg2012statistical}

\bibitem{bock1986syntactic}
Bock JK (1986) Syntactic persistence in language production.
\newblock Cognitive Psychology 18: 355-387.
\bibAnnoteFile{bock1986syntactic}

\bibitem{pickering2008structural}
Pickering MJ, Ferreira VS (2008) Structural priming: a critical review.
\newblock Psychological bulletin 134: 427.
\bibAnnoteFile{pickering2008structural}

\bibitem{brennan96lexical}
Brennan SE (1996) Lexical entrainment in spontaneous dialog.
\newblock In: International Symposium on Spoken Dialog. pp. 41--44.
\newblock \urlprefix\url{citeseer.ist.psu.edu/brennan96lexical.html}.
\bibAnnoteFile{brennan96lexical}

\bibitem{brennan1996conceptual}
Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in
  conversation.
\newblock Journal of Experimental Psychology: Learning, Memory, and Cognition
  22: 1482.
\bibAnnoteFile{brennan1996conceptual}

\bibitem{Pennebaker1999}
Pennebaker JW, King LA (1999) Linguistic styles: Language use as an individual
  difference.
\newblock Journal of Personality and Social Psychology 77: 1296--1312.
\bibAnnoteFile{Pennebaker1999}

\bibitem{Stirman01072001}
Stirman SW, Pennebaker JW (2001) Word use in the poetry of suicidal and
  nonsuicidal poets.
\newblock Psychosomatic Medicine 63: 517-522.
\bibAnnoteFile{Stirman01072001}

\bibitem{Campbell2003}
Campbell RS, Pennebaker JW (2003) The secret life of pronouns.
\newblock Psychological Science 14: 60-65.
\bibAnnoteFile{Campbell2003}

\bibitem{Rude2004}
Rude S, Gortner EM, Pennebaker J (2004) Language use of depressed and
  depression-vulnerable college students.
\newblock Cognition and Emotion 18: 1121+.
\bibAnnoteFile{Rude2004}

\bibitem{Boals01092005}
Boals A, Klein K (2005) Word use in emotional narratives about failed romantic
  relationships and subsequent mental health.
\newblock Journal of Language and Social Psychology 24: 252-268.
\bibAnnoteFile{Boals01092005}

\end{thebibliography}

\section*{Supporting Information}

\hspace{0.2in}\textbf{S1 File. Basic Concepts in Information Theory.}


\textbf{S2 File. Stylistic Coordination and Power Relationship.} 
\appendix
\newpage
\revise{

\renewcommand\thefigure{\Alph{figure}} 

\section*{S1 Basic Concepts in Information Theory}
\label{app:info}
Consider a random variable $X$ with probability distribution $p(x)\equiv p(X=x)$. In the discrete case, the \textit{Shannon} entropy is defined as: 
\BEQ \label{eq:ent_def}
H\left( X \right) =  - \sum_x {p\left( x \right)\log p\left( x \right)} 
\EEQ 
\revisethree{Note that If $X$ is a continuous variable, the sum in Eq.~\ref{eq:ent_def} is replaced by an integral.} We often talk about the entropy $H(X)$ s quantifying our uncertainty about the random variable, so that higher entropy means more uncertain (and less predictable) $X$. In particular, entropy is zero, $H(X)=0$, if and only if $X$ is perfectly predictable.   

Let us now consider two random variables, $X$ and $Y$, and let $p(x,y)$ denote their joint distribution. The (joint) entropy for $X$ and $Y$ is defined similarly as 
\BEQ
H\left( {X,Y} \right) =  - \sum_x {\sum_y {p\left( {x,y} \right)\log p\left( {x,y} \right)} } 
\label{eq:joint_entropy}
\EEQ
We can also define conditional entropy of $Y$ given $X$ (or vice versa), as follows:
\BEQ
H\left( {Y|X} \right) =  - \sum_x p(x) {\sum_y {p\left( {y|x} \right)\log p\left( {y|x} \right)} } 
\label{eq:cond_entropy}
\EEQ

If $X$ and $Y$ are independent, then their joint entropy is the sum of individual entropies, $H(X,Y)=H(X) + H(Y)$. Indeed, note that for independent $X$ and $Y$, their joint distribution factorizes as $p(x,y)=p(x)p(y)$. Plugging this factorized distribution into Eq.~\ref{eq:joint_entropy} then yields the additivity of the joint entropy.

In case $X$ and $Y$ are not independent, the degree of their dependence can be measured by {\em mutual information} (MI), defined as follows:
\BEQ
I\left( {X:Y} \right) = H\left( X \right) + H\left( Y \right) - H\left( {X,Y} \right)
\EEQ
$I\left({X:Y} \right)$ measures the correlation between $X$ and $Y$, and equals to zero if and only if $X$ and $Y$ are independent. In essence, $I(X:Y)$ measures the amount of uncertainty reduction in $X$, if we know $Y$, and vice visa. This intuition becomes more clear if we rewrite the mutual information the in the following form:
\BEA
I\left( {X:Y} \right) &=& H\left( Y \right) - H\left( Y |X \right) \\
&=& H\left( X \right) - H\left( X|Y \right) 
\EEA
The relationships between the entropies, conditional entropies, and mutual information are captured in the Venn diagram shown in Fig.~\ref{mi_explain}.
\setcounter{figure}{0} 
\begin{figure}[]
\centering
 \includegraphics[width=0.5\textwidth]{figs/mi.jpg}
\caption{The Venn diagram depicting the relationship between individual ($H(X)$, $H(Y)$), joint ($H(X,Y)$), and conditional ($H(X|Y)$, $H(Y|X)$) entropies. The intersection of the circles  is the mutual information $I(X;Y)$.
 }
 \label{mi_explain}
\end{figure}


For a more detailed account of information-theoretic concepts, we refer the reader to a classical textbook by Cover and Thomas~\cite{CoverThomas2006}.  
}
\newpage
  $ $
\newpage
\renewcommand\thefigure{\Alph{figure}} 
\section*{S2 Stylistic Coordination and Power Relationship}
It has been hypothesized that directionality of the stylistic coordination in dialogues can be predictive of power relationship between the conversations, as discussed in~\cite{Danescu-Niculescu-Mizil2012WWW}. We do indeed observe directional differences in stylistic coordination when comparing Figs.~\ref{fig_supreme_court_style}(a) and~\ref{fig_supreme_court_style}(b) with Figs.~\ref{fig_wiki_style}(a) and~\ref{fig_wiki_style}(b). However, as we elaborated above, the observed directionality can  result from the confounding effect of length coordination. 

Here we analyze this issue in more details by setting up the following prediction task (see~\cite{Danescu-Niculescu-Mizil2012WWW}). We consider all the pairs of users $(X,Y)$ who have different social status (e.g., admin vs. non-admin) and have engaged in dialogues. We then calculate stylistic coordination scores from $X$ to $Y$ and $Y$ to $X$, and examine whether those scores can be used to classify the social status of each speaker. For classification, we assume we know the status relationship for a fraction of pairs in our dataset, and then use a supervised learning method called Support Vector Machine (SVM) to predict the status of the unknown users. We perform this prediction tasks using the following three different set of features: 
\begin{itemize}
\item {\em Coordination Features}: For each pair, and for each of the eight stylistic markers, we produce two-dimensional feature vector, where the two components correspond to the mutual information score in either direction. 

\item {\em Aggregated Coordination Features}: For each pair, we aggregate the {\em Coordination Features} for all eight stylistic markers in both  directions, which results  in a sixteen-dimensional feature vector. 

\item {\em Length Coordination Features}: For each pair, we calculate the Pearson correlation coefficients between length of utterances in either direction, and use those coefficients as an input to SVM.   
\end{itemize}

For the Wikipedia data, we consider (admin, non-admin) pairs and for the Supreme Court data, we consider (justice, lawyer) pairs.

\setcounter{figure}{0} 
\begin{figure}[htbp]
\centering
\subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/PowerPrediction_SC.eps} \label{fig_sc_svm}
 
    } 
    \subfigure[]{
    \includegraphics[width=0.45\columnwidth]{figs/PowerPrediction_WIKI.eps} \label{fig_wiki_svm}
    }
 
\caption{SVM Prediction Accuracy for both stylistic coordination features and length coordination features }       
\label{fig_svm}
\end{figure}
In our experiment, we only select pairs which have at least 20 exchanges between them, so that we can calculate mutual information with reasonable accuracy. This results in 135 pairs in Supreme Court dataset and 34 pairs in Wikipedia dataset. Also, we labeled half fraction of the pairs, shuffled the training data and repeated the procedure $N=100$ times to calculate the average prediction accuracy. 

Fig.~\ref{fig_svm} depicts the prediction accuracy for each of the above scenario, together with error bars which give the 95\% confidence intervals. Since the two datasets are small, the error bars are relatively large in these situations. The results can be summarized as follows. For the Supreme Court dataset, the best prediction accuracy is achieved when using {\em Aggregated Coordination Features}, whereas for the Wikipedia dataset the best accuracy corresponds to using coordination on the feature  {\em Adverb}. More importantly, we find that using {\em Length Coordination Features} alone can predict user status with better-than-random ($50\%$) accuracy. In fact, investigating from the error bars, we cannot rule out that the hypothesized correlation between social status and (the direction of) stylistic coordination is due to the confounding effect of length coordination. %\aram{needs some polishing}

%for the Supreme Court data set, only {\em Impersonal Pronoun} and {\em Article} have the prediction accuracy over the 50\% baseline, while length correlation feature also achieve 55.8\% accuracy. For the Wikipedia data set, only the {\em Adverb} feature performs significantly higher than the baseline among all of the eight markers, while length correlation also gives us a high accuracy. Note that length correlation is the only feature that is better than 50\% in both of the domains. {}


%also allows better than random prediction accuracy. In fact, performs almost as good as stylistic coordination features in the status prediction task. power predictions. For Supreme Court data set, only "Impersonal Pronoun" and "Article" have the prediction accuracy over the 50\% baseline, while length correlation feature also achieve 55.8\% accuracy. For the Wikipedia data set, only the "Adverb" feature performs significantly higher than the baseline among all of the eight markers, while length correlation also gives us a high accuracy. Note that length correlation is the only feature that is better than 50\% in both of the domains, suggesting that the stylistic coordination may come from length correlation. 



\end{document}

coordination-plos1-revised-v2.bbl
\begin{thebibliography}{10}
\providecommand{\url}[1]{\texttt{#1}}
\providecommand{\urlprefix}{URL }
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi:\discretionary{}{}{}#1}\else
  \providecommand{\doi}{doi:\discretionary{}{}{}\begingroup
  \urlstyle{rm}\Url}\fi
\providecommand{\bibAnnoteFile}[1]{%
  \IfFileExists{#1}{\begin{quotation}\noindent\textsc{Key:} #1\\
  \textsc{Annotation:}\ \input{#1}\end{quotation}}{}}
\providecommand{\bibAnnote}[2]{%
  \begin{quotation}\noindent\textsc{Key:} #1\\
  \textsc{Annotation:}\ #2\end{quotation}}
\providecommand{\eprint}[2][]{\url{#2}}

\bibitem{Giles1991}
Giles H, Coupland N, Coupland J (1991) Accommodation theory: Communication,
  context, and consequence, Cambridge: Cambridge University Press.
\newblock pp. 1--68.
\bibAnnoteFile{Giles1991}

\bibitem{Ireland2010}
Ireland ME, Slatcher RB, Eastwick PW, Scissors LE, Finkel EJ, et~al. (2011)
  Language style matching predicts relationship initiation and stability.
\newblock Psychological Science 22: 39--44.
\bibAnnoteFile{Ireland2010}

\bibitem{Gonzales2010}
Gonzales AL, Hancock JT, Pennebaker JW (2010) Language style matching as a
  predictor of social dynamics in small groups.
\newblock Communication Research 37: 3-19.
\bibAnnoteFile{Gonzales2010}

\bibitem{Danescu-Niculescu-Mizil2011WWW}
Danescu-Niculescu-Mizil C, Gamon M, Dumais S (2011) Mark my words!: linguistic
  style accommodation in social media.
\newblock In: Proceedings of the 20th international conference on World wide
  web. WWW '11, pp. 745--754.
\newblock \doi{10.1145/1963405.1963509}.
\newblock \urlprefix\url{http://doi.acm.org/10.1145/1963405.1963509}.
\bibAnnoteFile{Danescu-Niculescu-Mizil2011WWW}

\bibitem{Danescu-Niculescu-Mizil2012WWW}
Danescu-Niculescu-Mizil C, Lee L, Pang B, Kleinberg J (2012) Echoes of power:
  language effects and power differences in social interaction.
\newblock In: Proceedings of the 21th international conference on World wide
  web. WWW '12, pp. 699--708.
\bibAnnoteFile{Danescu-Niculescu-Mizil2012WWW}

\bibitem{Taylor2008}
Taylor PJ, Thomas S (2008) Linguistic style matching and negotiation outcome.
\newblock Negotiation and Conflict Management Research 1: 263--281.
\bibAnnoteFile{Taylor2008}

\bibitem{Niederhoffer2002}
Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social
  interaction.
\newblock Journal of Language and Social Psychology 21: 337-360.
\bibAnnoteFile{Niederhoffer2002}

\bibitem{Pennebaker2007}
Pennebaker JW, Francis ME (2007).
\newblock Linguistic inquiry and word count: A computerized text analysis
  program.
\newblock \urlprefix\url{http://www.liwc.net/}.
\bibAnnoteFile{Pennebaker2007}

\bibitem{CoverThomas2006}
Cover TM, Thomas JA (2006) Elements of Information Theory (Wiley Series in
  Telecommunications and Signal Processing).
\newblock Wiley-Interscience.
\bibAnnoteFile{CoverThomas2006}

\bibitem{dedeo}
DeDeo S, Hawkins RXD, Klingenstein S, Hitchcock T (2013) Bootstrap methods for
  the empirical study of decision-making and information flows in social
  systems.
\newblock Entropy 15: 2246--2276.
\bibAnnoteFile{dedeo}

\bibitem{kraskov2004estimating}
Kraskov A, St{\"o}gbauer H, Grassberger P (2004) Estimating mutual information.
\newblock Physical Review E 69: 066138.
\bibAnnoteFile{kraskov2004estimating}

\bibitem{Wang2009}
Wang Q, Kulkarni SR, Verd\'{u} S (2009) Universal estimation of information
  measures for analog sources.
\newblock Found Trends Commun Inf Theory 5: 265--353.
\bibAnnoteFile{Wang2009}

\bibitem{Simpson1951}
Simpson EH (1951) The interpretation of interaction in contingency tables.
\newblock Journal of the Royal Statistical Society, Series B (Statistical
  Methodology) 13: 238--241.
\bibAnnoteFile{Simpson1951}

\bibitem{McGill54}
McGill W (1954) Multivariate information transmission.
\newblock Information Theory, Transactions of the IRE Professional Group on 4:
  93-111.
\bibAnnoteFile{McGill54}

\bibitem{bell03}
Bell AJ (2003) The co-information lattice.
\newblock In: Proceedings of the 4th international symposium on Independent
  Component Analysis and Blind Source Separation. pp. 921--926.
\bibAnnoteFile{bell03}

\bibitem{Pesarin_Book}
Pesarin F, Salmaso L (2010) Permutation Tests for Complex Data: Theory,
  Applications and Software.
\newblock John Wiley \& Sons.
\bibAnnoteFile{Pesarin_Book}

\bibitem{Danescu-Niculescu-Mizil2011CIC}
Danescu-Niculescu-Mizil C, Lee L (2011) Chameleons in imagined conversations: A
  new approach to understanding coordination of linguistic style in dialogs.
\newblock In: Proceedings of the 2Nd Workshop on Cognitive Modeling and
  Computational Linguistics. Stroudsburg, PA, USA: Association for
  Computational Linguistics, CMCL '11, pp. 76--87.
\newblock \urlprefix\url{http://dl.acm.org/citation.cfm?id=2021096.2021105}.
\bibAnnoteFile{Danescu-Niculescu-Mizil2011CIC}

\bibitem{Pearl2000}
Pearl J (2000) Causality: Models, Reasoning, and Inference.
\newblock New York, NY, USA: Cambridge University Press.
\bibAnnoteFile{Pearl2000}

\bibitem{win2013}
{Ver Steeg} G, Galstyan A (2013) Minimal assumption tests for contagion in
  observational social network studies.
\newblock In: Workshop on Information in Networks(WIN).
\bibAnnoteFile{win2013}

\bibitem{steeg2012statistical}
{Ver Steeg} G, Galstyan A (2013) Statistical tests for contagion in
  observational social network studies.
\newblock In: AISTATS'13.
\bibAnnoteFile{steeg2012statistical}

\bibitem{bock1986syntactic}
Bock JK (1986) Syntactic persistence in language production.
\newblock Cognitive Psychology 18: 355-387.
\bibAnnoteFile{bock1986syntactic}

\bibitem{pickering2008structural}
Pickering MJ, Ferreira VS (2008) Structural priming: a critical review.
\newblock Psychological bulletin 134: 427.
\bibAnnoteFile{pickering2008structural}

\bibitem{brennan96lexical}
Brennan SE (1996) Lexical entrainment in spontaneous dialog.
\newblock In: International Symposium on Spoken Dialog. pp. 41--44.
\newblock \urlprefix\url{citeseer.ist.psu.edu/brennan96lexical.html}.
\bibAnnoteFile{brennan96lexical}

\bibitem{brennan1996conceptual}
Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in
  conversation.
\newblock Journal of Experimental Psychology: Learning, Memory, and Cognition
  22: 1482.
\bibAnnoteFile{brennan1996conceptual}

\bibitem{Pennebaker1999}
Pennebaker JW, King LA (1999) Linguistic styles: Language use as an individual
  difference.
\newblock Journal of Personality and Social Psychology 77: 1296--1312.
\bibAnnoteFile{Pennebaker1999}

\bibitem{Stirman01072001}
Stirman SW, Pennebaker JW (2001) Word use in the poetry of suicidal and
  nonsuicidal poets.
\newblock Psychosomatic Medicine 63: 517-522.
\bibAnnoteFile{Stirman01072001}

\bibitem{Campbell2003}
Campbell RS, Pennebaker JW (2003) The secret life of pronouns.
\newblock Psychological Science 14: 60-65.
\bibAnnoteFile{Campbell2003}

\bibitem{Rude2004}
Rude S, Gortner EM, Pennebaker J (2004) Language use of depressed and
  depression-vulnerable college students.
\newblock Cognition and Emotion 18: 1121+.
\bibAnnoteFile{Rude2004}

\bibitem{Boals01092005}
Boals A, Klein K (2005) Word use in emotional narratives about failed romantic
  relationships and subsequent mental health.
\newblock Journal of Language and Social Psychology 24: 252-268.
\bibAnnoteFile{Boals01092005}

\end{thebibliography}
