【白板推导系列笔记】数学基础-概率-高斯分布-求边缘概率以及条件概率

烧灯续昼2002
26 阅读1分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路。

X∼N(μ,Σ)=1(2π)p2∣Σ∣12exp(−12(x−μ)TΣ−1(x−μ))x∈Rp,r.v. \begin{gathered} X \sim N(\mu,\Sigma)=\frac{1}{(2\pi)^{\frac{p}{2}}|\Sigma|^{\frac{1}{2}}}\text{exp}\left(- \frac{1}{2}(x-\mu)^{T}\Sigma^{-1}(x-\mu)\right)\\ x \in \mathbb{R}^{p},r.v.\\ \end{gathered}

已知

x=(xaxb),μ=(μaμb),Σ=(ΣaaΣabΣbaΣbb)xa为m×1,xb为n×1,m+n=p \begin{gathered} x=\begin{pmatrix} x_{a} \\ x_{b} \end{pmatrix},\mu=\begin{pmatrix} \mu_{a} \\ \mu_{b} \end{pmatrix},\Sigma=\begin{pmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{pmatrix}\\ x_{a}为m \times 1,x_{b}为 n \times 1,m+n=p \end{gathered}

求P(xa),P(xb∣xa)P(x_{a}),P(x_{b}|x_{a})，求得后可以由对称性得到P(xb),P(xa∣xb)P(x_{b}),P(x_{a}|x_{b})

 

定理：

已知

X∼N(μ,Σ),x∈Rp,y=Ax+B,y∈RpX \sim N(\mu,\Sigma),x \in \mathbb{R}^{p},y=Ax+B,y \in \mathbb{R}^{p}

则有

y∼N(Aμ+B,AΣAT)y \sim N(A \mu+B,A \Sigma A^{T})

 

先求xax_{a}的分布

xa=(ImOn)⏟A(xaxb)⏟xE(xa)=(ImO)(μaμb)=μaVar(xa)=(ImO)(ΣaaΣabΣbaΣbb)(ImO)=Σaa \begin{aligned} x_{a}&=\underbrace{\begin{pmatrix}I_{m} & O_{n}\end{pmatrix}}_{A}\underbrace{\begin{pmatrix} x_{a} \\ x_{b} \end{pmatrix}}_{x}\\ E(x_{a})&=\begin{pmatrix}I_{m} & O\end{pmatrix}\begin{pmatrix} \mu_{a} \\ \mu_{b} \end{pmatrix}=\mu_{a}\\ \text{Var}(x_{a})&=\begin{pmatrix} I_{m} & O \end{pmatrix}\begin{pmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{pmatrix}\begin{pmatrix} I_{m} \\ O \end{pmatrix}=\Sigma_{aa} \end{aligned}

因此xa∼N(μa,Σaa)x_{a}\sim N(\mu_{a},\Sigma_{aa})

再求xb∣xax_{b}|x_{a}的分布，令

{xb⋅a=xb−ΣbaΣaa−1xaμb⋅a=μb−ΣbaΣaa−1μaΣbb⋅a=Σbb−ΣbaΣaa−1Σab \left\{\begin{aligned}&x_{b \cdot a}=x_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}\\&\mu_{b \cdot a}=\mu_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}\mu_{a}\\&\Sigma_{bb \cdot a}=\Sigma_{bb}-\Sigma_{ba}\Sigma_{aa}^{-1}\Sigma_{ab}\end{aligned}\right.

有

xb⋅a=(−ΣbaΣaa−1In)⏟A(xaxb)⏟xE(xb⋅a)=(−ΣbaΣaa−1In)(μaμb)=μb−ΣbaΣaa−1μa=μb⋅aVar(xb⋅a)=(−ΣbaΣaa−1In)(ΣaaΣabΣbaΣbb)(−Σaa−1ΣbaTIn)=Σbb−ΣbaΣaa−1Σab=Σbb⋅a \begin{aligned} x_{b \cdot a}&=\underbrace{\begin{pmatrix}- \Sigma_{ba}\Sigma_{aa}^{-1} & I_{n} \end{pmatrix}}_{A}\underbrace{\begin{pmatrix} x_{a} \\ x_{b} \end{pmatrix}}_{x}\\ E(x_{b \cdot a})&=\begin{pmatrix}- \Sigma_{ba}\Sigma_{aa}^{-1} & I_{n} \end{pmatrix}\begin{pmatrix} \mu_{a} \\ \mu_{b} \end{pmatrix}=\mu_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}\mu_{a}=\mu_{b \cdot a}\\ \text{Var}(x_{b \cdot a})&=\begin{pmatrix}- \Sigma_{ba}\Sigma_{aa}^{-1} & I_{n} \end{pmatrix}\begin{pmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{pmatrix}\begin{pmatrix} -\Sigma_{aa}^{-1}\Sigma_{ba}^{T} \\ I_{n} \end{pmatrix}\\ &=\Sigma_{bb}-\Sigma_{ba}\Sigma_{aa}^{-1}\Sigma_{ab}=\Sigma_{bb \cdot a} \end{aligned}

因此xb⋅a∼N(μb⋅a,Σbb⋅a)x_{b \cdot a}\sim N(\mu_{b \cdot a},\Sigma_{bb \cdot a})

这里要求xb∣xax_{b}|x_{a}，即

xb⋅a=xb−ΣbaΣaa−1xaxb=xb⋅a+ΣbaΣaa−1xaxb∣xa=(xb⋅a+ΣbaΣaa−1xa)∣xaxb∣xa=xb⋅a∣xa+ΣbaΣaa−1xa∣xaxb∣xa=xb⋅a∣xa+ΣbaΣaa−1xa \begin{aligned} x_{b \cdot a}&=x_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}\\ x_{b}&=x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}\\ x_{b}|x_{a}&=(x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a})|x_{a}\\ x_{b}|x_{a}&=x_{b \cdot a}|x_{a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}|x_{a}\\ x_{b}|x_{a}&=x_{b \cdot a}|x_{a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a} \end{aligned}

这里如果有xb⋅a∣xa=xb⋅ax_{b \cdot a}|x_{a}=x_{b \cdot a}，就可以有

xb∣xa=xb⋅a+ΣbaΣaa−1xa x_{b}|x_{a}=x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}

 

若X∼N(μ,Σ)X \sim N(\mu,\Sigma)，则Mx⊥Nx⇔MΣNT=0Mx \bot Nx \Leftrightarrow M \Sigma N^{T}=0

证明：

因为x∼N(μ,Σ)x \sim N(\mu,\Sigma)，有Mx∼N(Mμ,MΣMT),Nx∼N(Nμ,NΣNT)Mx \sim N(M \mu,M \Sigma M^{T}),Nx \sim N(N \mu,N \Sigma N^{T})

cov(Mx,Nx)=E[(Mx−Mμ)(Nx−Nμ)T]=M⋅E[(x−μ)(x−μ)T]⋅NT=MΣNT\begin{aligned} \text{cov}(Mx,Nx)&=E[(Mx-M \mu)(Nx-N \mu)^{T}]\\&=M \cdot E[(x-\mu)(x-\mu)^{T}]\cdot N^{T}\\&=M \Sigma N^{T}\end{aligned}

又因为Mx⊥NxMx \bot Nx且均为高斯分布，则有Cov(Mx,Nx)=MΣNT=0\text{Cov}(Mx ,Nx)=M \Sigma N^{T}=0

 

Σ=(ΣaaΣabΣbaΣbb)xb⋅a=xb−ΣbaΣaa−1xa=(−ΣbaΣaa−1I)⏟M(xaxb)xa=(IO)⏟N(xaxb)MΣNT=(−ΣbaΣaa−1I)(ΣaaΣabΣbaΣbb)(IO)=0 \begin{aligned} \Sigma&=\begin{pmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{pmatrix}\\ x_{b \cdot a}&=x_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}=\underbrace{\begin{pmatrix} -\Sigma_{ba}\Sigma_{aa}^{-1} & I \end{pmatrix}}_{M}\begin{pmatrix} x_{a} \\ x_{b} \end{pmatrix}\\ x_{a}&=\underbrace{\begin{pmatrix} I & O \end{pmatrix}}_{N}\begin{pmatrix} x_{a} \\ x_{b} \end{pmatrix}\\ M \Sigma N^{T}&=\begin{pmatrix} -\Sigma_{ba}\Sigma_{aa}^{-1} & I \end{pmatrix}\begin{pmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{pmatrix}\begin{pmatrix} I & O \end{pmatrix}=0 \end{aligned}

因此xb⋅a⊥xa⇒xb⋅a∣xa=xb⋅ax_{b \cdot a}\bot x_{a}\Rightarrow x_{b \cdot a}|x_{a}=x_{b \cdot a }，就有

xb∣xa=xb⋅a∣xa+ΣbaΣaa−1∣xa=xb⋅a+ΣbaΣaa−1xa x_{b}|x_{a}=x_{b \cdot a}|x_{a}+\Sigma_{ba}\Sigma_{aa}^{-1}|x_{a}=x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}

因此

E(xb∣xa)=E(xb⋅a+ΣbaΣaa−1xa)=μb⋅a+ΣbaΣaa−1xa=μb−ΣbaΣaa−1μa+ΣbaΣaa−1xaVar(xb∣xa)=Var(xb⋅a+ΣbaΣaa−1xa)=Var(xb⋅a)=Σbb⋅a \begin{aligned} E(x_{b}|x_{a})&=E(x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a})\\ &=\mu_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}\\ &=\mu_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}\mu_{a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a}\\ \text{Var}(x_{b}|x_{a})&=\text{Var}(x_{b \cdot a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a})\\ &=\text{Var}(x_{b \cdot a})\\ &=\Sigma_{bb \cdot a} \end{aligned}

因此xb∣xa∼N(μb−ΣbaΣaa−1μa+ΣbaΣaa−1xa,Σbb−ΣbaΣaa−1Σab)x_{b}|x_{a} \sim N(\mu_{b}-\Sigma_{ba}\Sigma_{aa}^{-1}\mu_{a}+\Sigma_{ba}\Sigma_{aa}^{-1}x_{a},\Sigma_{bb}-\Sigma_{ba}\Sigma_{aa}^{-1}\Sigma_{ab})

avatar
文章
阅读
粉丝