A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family
Abstract
Consider a pair of random vectors $(\mathbf{X},\mathbf{Y}) $ and the conditional expectation operator $\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$. This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain $\mathbf{U} \leftrightarrow \mathbf{X} \leftrightarrow \mathbf{Y}$, a compact expression for the Jacobian matrix of $\mathbb{E}[\mathbf{U}|\mathbf{Y}=\mathbf{y}]$ is derived. In the second part of the paper, the main identity is specialized to the exponential family. Moreover, via various choices of the random vector $\mathbf{U}$, the new identity is used to recover and generalize several known identities and derive some new ones. As a first example, a connection between the Jacobian of $ \mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$ and the conditional variance is established. As a second example, a recursive expression between higher order conditional expectations is found, which is shown to lead to a generalization of the Tweedy's identity. Finally, as a third example, it is shown that the $k$-th order derivative of the conditional expectation is proportional to the $(k+1)$-th order conditional cumulant.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2021
- DOI:
- 10.48550/arXiv.2105.05106
- arXiv:
- arXiv:2105.05106
- Bibcode:
- 2021arXiv210505106D
- Keywords:
-
- Mathematics - Probability;
- Computer Science - Information Theory;
- Mathematics - Statistics Theory