  Consider a pair of random vectors $(\mathbf{X},\mathbf{Y}) $ and the
conditional expectation operator
$\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$. This work studies analytic
properties of the conditional expectation by characterizing various derivative
identities. The paper consists of two parts. In the first part of the paper, a
general derivative identity for the conditional expectation is derived.
Specifically, for the Markov chain $\mathbf{U} \leftrightarrow \mathbf{X}
\leftrightarrow \mathbf{Y}$, a compact expression for the Jacobian matrix of
$\mathbb{E}[\mathbf{U}|\mathbf{Y}=\mathbf{y}]$ is derived. In the second part
of the paper, the main identity is specialized to the exponential family.
Moreover, via various choices of the random vector $\mathbf{U}$, the new
identity is used to recover and generalize several known identities and derive
some new ones. As a first example, a connection between the Jacobian of $
\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$ and the conditional variance is
established. As a second example, a recursive expression between higher order
conditional expectations is found, which is shown to lead to a generalization
of the Tweedy's identity. Finally, as a third example, it is shown that the
$k$-th order derivative of the conditional expectation is proportional to the
$(k+1)$-th order conditional cumulant.
