详解降维-样本均值&样本方差矩阵【白板推导系列笔记】

烧灯续昼2002
87 阅读1分钟

持续创作，加速成长！这是我参与「掘金日新计划 · 10 月更文挑战」的第6天，点击查看活动详情

X=(x1x2⋯ xN)N×pT=(x1Tx2T⋮ xNT)=(x11x12⋯ x1px21x22⋯ x2p⋮ ⋮  ⋮ xN1xN2⋯ xNP)N×pxi∈Rp,i=1,2,⋯ ,N记1N=(11⋮ 1)N×1 \begin{gathered} X=\begin{pmatrix} x_{1} & x_{2} & \cdots  & x_{N} \end{pmatrix}^{T}_{N \times p}=\begin{pmatrix} x_{1}^{T} \\ x_{2}^{T} \\ \vdots  \\ x_{N}^{T} \end{pmatrix}=\begin{pmatrix} x_{11} & x_{12} & \cdots &  x_{1p} \\ x_{21} & x_{22} & \cdots  & x_{2p} \\ \vdots  & \vdots  &  & \vdots  \\ x_{N1} & x_{N2} & \cdots  & x_{NP} \end{pmatrix}_{N \times p}\\ x_{i}\in \mathbb{R}^{p},i=1,2,\cdots ,N\\ 记1_{N}=\begin{pmatrix}1 \\ 1 \\ \vdots  \\ 1\end{pmatrix}_{N \times 1} \end{gathered}

对于样本均值

xˉ=1N∑i=1Nxi=1N(x1x2⋯ xN)(11⋮ 1)N×1=1NXT1N \begin{aligned} \bar{x}&=\frac{1}{N}\sum\limits_{i=1}^{N}x_{i}\\ &=\frac{1}{N}\begin{pmatrix} x_{1} & x_{2} & \cdots  & x_{N} \end{pmatrix}\begin{pmatrix}1 \\ 1 \\ \vdots  \\ 1\end{pmatrix}_{N \times 1}\\ &=\frac{1}{N}X^{T}1_{N} \end{aligned}

对于样本方差

S=1N∑i=1N(xi−xˉ)(xi−xˉ)T \begin{aligned} S&=\frac{1}{N}\sum\limits_{i=1}^{N}(x_{i}-\bar{x})(x_{i}-\bar{x})^{T} \end{aligned}

对于∑i=1N(xi−xˉ)\sum\limits_{i=1}^{N}(x_{i}-\bar{x})有

∑i=1N(xi−xˉ)=(x1−xˉx2−xˉ⋯ xN−xˉ)=(x1x2⋯ xN)−(xˉxˉ⋯ xˉ)=XT−xˉ(11⋯ 1)=XT−xˉ1NT=XT−1NXT1N1NT=XT(IN−1N1N1NT) \begin{aligned} \sum\limits_{i=1}^{N}(x_{i}-\bar{x})&=\begin{pmatrix} x_{1}-\bar{x} & x_{2}-\bar{x} & \cdots  & x_{N}-\bar{x} \end{pmatrix}\\ &=\begin{pmatrix} x_{1} & x_{2} & \cdots  & x_{N} \end{pmatrix}-\begin{pmatrix} \bar{x} & \bar{x} & \cdots  & \bar{x} \end{pmatrix}\\ &=X^{T}-\bar{x}\begin{pmatrix}1 & 1 & \cdots  & 1\end{pmatrix}\\ &=X^{T}-\bar{x}1_{N}^{T}\\ &=X^{T}- \frac{1}{N}X^{T}1_{N}1_{N}^{T}\\ &=X^{T}\left(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\right)\\ \end{aligned}

带回原式

S=1N(x1−xˉx2−xˉ⋯ xN−xˉ)((x1−xˉ)T(x2−xˉ)T⋮ (xN−xˉ)T)=1NXT(IN−1N1N1NT)⋅(IN−1N1N1NT)TX \begin{aligned} S&=\frac{1}{N}\begin{pmatrix} x_{1}-\bar{x} & x_{2}-\bar{x} & \cdots  & x_{N}-\bar{x} \end{pmatrix}\begin{pmatrix} (x_{1}-\bar{x})^{T} \\ (x_{2}-\bar{x})^{T} \\ \vdots  \\ (x_{N}-\bar{x})^{T} \end{pmatrix}\\ &=\frac{1}{N}X^{T}\left(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\right)\cdot (\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T})^{T}X\\ \end{aligned}

记H=IN−1N1N1NT\begin{aligned} \mathbb{H}=\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\end{aligned}（H\mathbb{H}也被称为中心矩阵），上式为

S=1NXT(IN−1N1N1NT)⋅(IN−1N1N1NT)TX=1NXTH⋅HX \begin{aligned} S&=\frac{1}{N}X^{T}\left(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\right)\cdot (\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T})^{T}X\\ &=\frac{1}{N}X^{T}\mathbb{H}\cdot \mathbb{H}X \end{aligned}

对于HT\mathbb{H}^{T}有

HT=(IN−1N1N1NT)T=IN−1N1N1NT=H \begin{aligned} \mathbb{H}^{T}&=(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T})^{T}\\ &=\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\\ &=\mathbb{H} \end{aligned}

对于H2\mathbb{H}^{2}有

H2=H⋅H=(IN−1N1N1NT)(IN−1N1N1NT)=IN−2N1N1NT+1N21N1NT1N1NT \begin{aligned} \mathbb{H}^{2}&=\mathbb{H} \cdot \mathbb{H}\\ &=\left(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\right)\left(\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\right)\\ &=\mathbb{I}_{N}- \frac{2}{N}1_{N}1_{N}^{T}+ \frac{1}{N^{2}}1_{N}1_{N}^{T}1_{N}1_{N}^{T} \end{aligned}

对于1N1NT1_{N}1_{N}^{T}

1N1NT=(1⋮ 1)(1⋯ 1)=(1⋯ 1⋮  ⋮ 1⋯ 1)1N1NT1N1NT=(1⋯ 1⋮  ⋮ 1⋯ 1)(1⋯ 1⋮  ⋮ 1⋯ 1)=(N⋯ N⋮  ⋮ N⋯ N) \begin{aligned} 1_{N}1_{N}^{T}&=\begin{pmatrix} 1 \\ \vdots  \\ 1 \end{pmatrix}\begin{pmatrix} 1 & \cdots  & 1 \end{pmatrix}=\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}\\ 1_{N}1_{N}^{T}1_{N}1_{N}^{T}&=\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}\\ &=\begin{pmatrix} N & \cdots  & N \\ \vdots  &  & \vdots  \\ N & \cdots  & N \end{pmatrix} \end{aligned}

带回H2\mathbb{H}^{2}有

H2=IN−2N1N1NT+1N21N1NT1N1NT=IN−2N(1⋯ 1⋮  ⋮ 1⋯ 1)+1N2(N⋯ N⋮  ⋮ N⋯ N)=IN−2N(1⋯ 1⋮  ⋮ 1⋯ 1)+1N(1⋯ 1⋮  ⋮ 1⋯ 1)=IN−1N(1⋯ 1⋮  ⋮ 1⋯ 1)=IN−1N1N1NT=H \begin{aligned} \mathbb{H}^{2}&=\mathbb{I}_{N}- \frac{2}{N}1_{N}1_{N}^{T}+ \frac{1}{N^{2}}1_{N}1_{N}^{T}1_{N}1_{N}^{T}\\ &=\mathbb{I}_{N}- \frac{2}{N}\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}+ \frac{1}{N^{2}}\begin{pmatrix} N & \cdots  & N \\ \vdots  &  & \vdots  \\ N & \cdots  & N \end{pmatrix}\\ &=\mathbb{I}_{N}- \frac{2}{N}\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}+ \frac{1}{N}\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}\\ &=\mathbb{I}_{N}- \frac{1}{N}\begin{pmatrix} 1 & \cdots  & 1 \\ \vdots  &  & \vdots  \\ 1 & \cdots  & 1 \end{pmatrix}\\ &=\mathbb{I}_{N}- \frac{1}{N}1_{N}1_{N}^{T}\\ &=\mathbb{H} \end{aligned}

因此有Hn=H\mathbb{H}^{n}=\mathbb{H}，带回SS

S=1NXTH⋅HX=1NXTHX \begin{aligned} S&=\frac{1}{N}X^{T}\mathbb{H}\cdot \mathbb{H}X\\ &=\frac{1}{N}X^{T}\mathbb{H}X \end{aligned}

这里中心矩阵H\mathbb{H}的几何意义是，对于一个数据集XX，XHX \mathbb{H}可以认为是将数据集平移到坐标轴原点，H\mathbb{H}就是这个起到平移作用的矩阵

avatar
文章
阅读
粉丝