【概率论基础进阶】数理统计的基本概念-总体、样本、统计量和样本数字特征

烧灯续昼2002
35 阅读2分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路。

没目录，个人感觉都是概率论到目前为止最重要的一章

 

定义：如果随机变量X1,X2,⋯ ,XnX_{1},X_{2},\cdots ,X_{n}相互独立且都与总体XX同分布，则称X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}为来自总体的简单随机样本，简称样本。nn为样本容量，样本的具体观测值x1,x2,⋯ ,xnx_{1},x_{2},\cdots,x_{n}称为样本值，或称总体XX的nn个独立观测值

如果总体XX的分布为F(x)F(x)，则样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的分布为

F(x1,x2,⋯ ,xn)=P{X1≤x1,X2≤x2,⋯ ,Xn≤xn}=P{X1≤x1}P{X2≤x2}⋯P{Xn≤xn}=F(x1)F(x2)⋯F(xn)=∏i=1nF(xi) \begin{aligned} F(x_{1},x_{2},\cdots,x_{n})&=P \left\{X_{1} \leq x_{1},X_{2}\leq x_{2},\cdots ,X_{n}\leq x_{n}\right\}\\ &=P \left\{X_{1}\leq x_{1}\right\}P \left\{X_{2}\leq x_{2}\right\}\cdots P \left\{X_{n}\leq x_{n}\right\}\\ &=F(x_{1})F(x_{2})\cdots F(x_{n})\\ &=\prod\limits_{i=1}^{n}F(x_{i}) \end{aligned}

如果总体XX有概率密度f(x)f(x)，则样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots ,X_{n}的概率密度为

fn(x1,x2,⋯ ,xn)=∏i=1nf(xi) f_{n}(x_{1},x_{2},\cdots,x_{n})=\prod\limits_{i=1}^{n}f(x_{i})

如果总体XX有概率分布P{X=aj}=pj,j=1,2,⋯P \left\{X=a_{j}\right\}=p_{j},j=1,2,\cdots，则样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的概率分布为

P{X1=x1,X2=x2,⋯ ,Xn=xn}=∏i=1nP{Xi=xi} P \left\{X_{1}=x_{1},X_{2}=x_{2},\cdots ,X_{n}=x_{n}\right\}=\prod\limits_{i=1}^{n}P \left\{X_{i}=x_{i}\right\}

 

定义：样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的不含未知参数的函数T=T(X1,X2,⋯ ,Xn)T=T(X_{1},X_{2},\cdots,X_{n})称为统计量

 

作为随机变量的函数，统计量本身也是一个随机变量。如果x1,x2,⋯ ,xnx_{1},x_{2},\cdots,x_{n}是样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的样本值，T(x1,x2,⋯ ,xn)T(x_{1},x_{2},\cdots,x_{n})为统计量T(X1,X2,⋯ ,Xn)T(X_{1},X_{2},\cdots,X_{n})的观测值

 

下面所列的样本数字特征、顺序统计量都是最常用的统计量

设X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}是来自总体XX的样本，则称

样本均值

Xˉ=1n∑i=1nXi \bar{X}=\frac{1}{n}\sum\limits_{i=1}^{n}X_{i}

样本方差

S2=1n−1∑i=1n(X1−Xˉ)2 S^{2}=\frac{1}{n-1}\sum\limits_{i=1}^{n}(X_{1}-\bar{X})^{2}

样本标准差

S=1n−1∑i=1n(Xi−Xˉ)2 S=\sqrt{ \frac{1}{n-1}\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}}

样本kk阶原点矩

Ak=1n∑i=1nXik,k=1,2,,A1Xˉ A_{k}=\frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{k},k=1,2,,A_{1}\bar{X}

样本kk阶中心矩

Bk=1n∑i=1n(Xi−Xˉ)k,k=1,2,B2=n−1nS2≠S2 B_{k}=\frac{1}{n}\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{k},k=1,2,B_{2}= \frac{n-1}{n}S^{2}\ne S^{2}

 

如果已知随机变量XX的期望为μ\mu,那么可以计算方差σ2:\sigma^2:

σ2=E[(X−μ)2]\sigma^2 = E[(X-\mu)^2]

但是对于XX的具体分布是无法预知的，计算起来也比较复杂，在实际中采用采样后的样本方差进行近似，定义s2s^2来近似σ2\sigma^2:

S2=1n∑in(Xi−μ)2S^2 = \frac{1}{n} \sum_{i}^{n} (X_i- \mu)^2

同样地，对于XX的期望μ\mu的值也不清楚，只能用样本均值近似：

Xˉ=1n∑inXi\bar{X} = \frac{1}{n}\sum_{i}^{n}X_i

最后，可以得到，s2s^2的公式为：

S2=1n−1∑i=1n(Xi−Xˉ)2S^2 = \frac{1}{n-1}\sum_{i=1}^{n}(X_i - \bar{X})^2

下面是证明：

E[S2]=E[1n∑i=1n(Xi−Xˉ)2]=E[1n∑i=1n((Xi−μ)−(Xˉ−μ))2]=E[1n∑i=1n((Xi−μ)2−2(Xˉ−μ)(Xi−μ)+(Xˉ−μ)2)]=E[1n∑i=1n(Xi−μ)2−2n(Xˉ−μ)∑i=1n(Xi−μ)+1n(Xˉ−μ)2∑i=1n1]=E[1n∑i=1n(Xi−μ)2−2n(Xˉ−μ)∑i=1n(Xi−μ)+1n(Xˉ−μ)2n]=E[1n∑i=1n(Xi−μ)2−2n(Xˉ−μ)∑i=1n(Xi−μ)+(Xˉ−μ)2]\begin{aligned} E[S^2] & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}((X_i-\mu)-(\bar{X}-\mu))^2] \\ & = E[\frac{1}{n}\sum_{i=1}^{n}((X_i-\mu)^2-2(\bar{X}-\mu)(X_i-\mu)+(\bar{X}-\mu)^2)]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-\frac{2}{n}(\bar{X}-\mu)\sum_{i=1}^{n}(X_i-\mu)+\frac{1}{n}(\bar{X}-\mu)^2\sum_{i=1}^{n}1]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-\frac{2}{n}(\bar{X}-\mu)\sum_{i=1}^{n}(X_i-\mu)+\frac{1}{n}(\bar{X}-\mu)^2n]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-\frac{2}{n}(\bar{X}-\mu)\sum_{i=1}^{n}(X_i-\mu)+(\bar{X}-\mu)^2]\\ \end{aligned}

其中，

Xˉ−μ=1n∑i=1nXi−μ=1n∑i=1nXi−1n∑i=1nμ=1n∑i=1n(Xi−μ)\begin{aligned} \bar{X} - \mu & = \frac{1}{n}\sum_{i=1}^{n}X_i - \mu \\ & = \frac{1}{n} \sum_{i=1}^{n} X_i - \frac{1}{n} \sum_{i=1}^{n}\mu\\ & = \frac{1}{n}\sum_{i=1}^{n}(X_i-\mu) \end{aligned}

然后将其带入上面的式子，可以得到

E[S2]=E[1n∑i=1n(Xi−μ)2−2n(Xˉ−μ)∑i=1n(Xi−μ)+(Xˉ−μ)2]=E[1n∑i=1n(Xi−μ)2−2n(Xˉ−μ)n(Xˉ−μ)+(Xˉ−μ)2]=E[1n∑i=1n(Xi−μ)2−2(Xˉ−μ)2+(Xˉ−μ)2]=E[1n∑i=1n(Xi−μ)2−(Xˉ−μ)2]=E[1n∑i=1n(Xi−μ)2]−E[(Xˉ−μ)2]=σ2−E[(Xˉ−μ)2]\begin{aligned} E[S^2] & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-\frac{2}{n}(\bar{X}-\mu)\sum_{i=1}^{n}(X_i-\mu)+(\bar{X}-\mu)^2]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-\frac{2}{n}(\bar{X}-\mu)n(\bar{X}-\mu)+(\bar{X}-\mu)^2]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-2(\bar{X}-\mu)^2+(\bar{X}-\mu)^2]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2-(\bar{X}-\mu)^2]\\ & = E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\mu)^2]-E[(\bar{X}-\mu)^2]\\ & = \sigma^2 - E[(\bar{X}-\mu)^2] \end{aligned}

其中，

E[(Xˉ−μ)2]=E[(Xˉ−E[Xˉ])2]=var(Xˉ)=var(∑i=1nXin)=1n2var(∑inXi)(因为var(aX)=a2var(X))=1n2∑i=1nvar(Xi)(因为var(∑i=1nXi)=∑i=1nvar(Xi))=nσ2n2=1nσ2\begin{aligned} E[(\bar{X}-\mu)^2] & = E[(\bar{X}-E[\bar{X}])^2]\\ & = var(\bar{X}) \\ & = var(\frac{\sum_{i=1}^{n}X_i}{n})\\ & = \frac{1}{n^2}var(\sum_{i}^{n}X_i) (因为var(aX)=a^2var(X))\\ & = \frac{1}{n^2}\sum_{i=1}^{n}var(X_i)(因为var(\sum_{i=1}^{n}X_i)=\sum_{i=1}^{n}var(X_i))\\ & = \frac{n\sigma^2}{n^2}\\ & =\frac{1}{n}\sigma^2 \end{aligned}

所以，可得E[1n∑i=1n(Xi−Xˉ)2]=σ2−1nσ2=n−1nσ2E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2]=\sigma^2 - \frac{1}{n}\sigma^2 = \frac{n-1}{n}\sigma^2

说明利用采样得到的样本计算的方差与真实值有偏差，低估了1nσ2\frac{1}{n}\sigma^2,那么进行调整可以得到：

nn−1E[1n∑i=1n(Xi−Xˉ)2]=E[1n−1∑i=1n(Xi−Xˉ)2]=σ2\frac{n}{n-1}E[\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2] = E[\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2] = \sigma^2

可以得到

S2=1n−1∑i=1n(Xi−Xˉ)2S^2 =\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2

证明完成

作者：YaoYong

链接：样本方差为什么是n-1-推导 - 知乎 (zhihu.com)

 

性质：

如果总体XX具有数学期望E(X)=μE(X)=\mu，则

E(Xˉ)=E(1n∑i=1nXi)=1n∑i=1nE(Xi)=E(Xi)=μ \begin{aligned} E(\bar{X})&=E\left(\frac{1}{n}\sum\limits_{i=1}^{n}X_{i}\right)\\ &=\frac{1}{n}\sum\limits_{i=1}^{n}E(X_{i})\\ &=E(X_{i})=\mu \end{aligned}

如果总体XX具有方差D(X)=σ2D(X)=\sigma^{2}，则

D(Xˉ)=D(1n∑i=1nXi)=1n2∑i=1nD(Xi)=1nD(Xi)=σ2nE(S2)=E[1n−1∑i=1n(Xi−Xˉ)2]=E[1n−1∑i=1n(Xi2+Xˉ2−2XiXˉ)]=1n−1[E(∑i=1nXi2)+E(∑i=1nXˉ2)−2E(∑i=1nXiXˉ)]=1n−1[∑i=1nE(Xi2)+nE(Xˉ2)−2nE(Xˉ2)]=1n−1[∑i=1nE(Xi2)−nE(Xˉ2)]=1n−1{∑i=1n[D(Xi)+E(Xi)2]−n[D(Xˉ)−E(Xˉ)2]}=1n−1[n(σ2+μ2)−n(σ2n+μ2)]=σ2 \begin{aligned} D(\bar{X})&=D\left(\frac{1}{n}\sum\limits_{i=1}^{n}X_{i}\right)\\ &=\frac{1}{n^{2}}\sum\limits_{i=1}^{n}D(X_{i})\\ &=\frac{1}{n}D(X_{i})=\frac{\sigma^{2}}{n}\\ E(S^2)&=E\left [ \frac{1}{n-1}\sum_{i=1}^{n}\left ( X_i-\bar{X} \right )^2 \right ] \\ &=E\left [ \frac{1}{n-1} \sum_{i=1}^{n} \left ( X_i^2+\bar{X}^2-2X_i\bar{X} \right ) \right ]\\ &=\frac{1}{n-1} \left [ E\left ( \sum_{i=1}^{n}X_i^2 \right ) +E\left (\sum_{i=1}^{n} \bar{X}^2 \right ) -2E\left ( \sum_{i=1}^{n}X_i\bar{X} \right ) \right ]\\ &=\frac{1}{n-1}\left [ \sum_{i=1}^{n} E\left ( X_i^2 \right ) + nE\left ( \bar{X}^2 \right ) -2nE\left ( \bar{X}^2 \right )\right ] \\ &=\frac{1}{n-1}\left [ \sum_{i=1}^{n} E\left ( X_i^2 \right ) - nE\left ( \bar{X}^2 \right ) \right ]\\ &=\frac{1}{n-1}\left\{\sum\limits_{i=1}^{n}[D(X_{i})+E(X_{i})^{2}]-n[D(\bar{X})-E(\bar{X})^{2}]\right\}\\ &=\frac{1}{n-1}\left[n\left(\sigma^{2}+\mu^{2}\right)-n\left(\frac{\sigma^{2}}{n}+\mu^{2}\right)\right]\\ &=\sigma^{2} \end{aligned}

如果总体XX的kk解原点矩E(Xk)=μk,k=,1,2⋯E(X^{k})=\mu_{k},k=,1,2\cdots存在，则当n→∞n \to \infty时

1n∑i=1nXik→Pμk,k=1,2,⋯ \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{k}\overset{P}{\rightarrow }\mu_{k},k=1,2,\cdots

 

例1：设来自总体的样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的样本均值Xˉ\bar{X}，试证∑i=1n(Xi−Xˉ)2=∑i=1nXi2−nXˉ2\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}=\sum\limits_{i=1}^{n}X_{i}^{2}-n \bar{X}^{2}

 

∑i=1n(Xi−Xˉ)2=∑i=1n(Xi2−2XiXˉ+Xˉ2)=∑i=1nXi2−2n∑i=1nXinXˉ+∑i=1nXˉ2=∑i=1nXi2−2nXˉ2+nXˉ2=∑i=1nXi2−nXˉ2 \begin{aligned} \sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}&=\sum\limits_{i=1}^{n}(X_{i}^{2}-2X_{i}\bar{X}+\bar{X}^{2})\\ &=\sum\limits_{i=1}^{n}X_{i}^{2}-2n \frac{\sum\limits_{i=1}^{n}X_{i}}{n}\bar{X}+\sum\limits_{i=1}^{n}\bar{X}^{2}\\ &=\sum\limits_{i=1}^{n}X_{i}^{2}-2n \bar{X}^{2}+n \bar{X}^{2}\\ &=\sum\limits_{i=1}^{n}X_{i}^{2}-n \bar{X}^{2} \end{aligned}

 

很常用，建议记住

∑i=1n(Xi−Xˉ)2=∑i=1nXi2−nXˉ2\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}=\sum\limits_{i=1}^{n}X_{i}^{2}-n \bar{X}^{2}

 

如果上面都看懂的，例2可以不看，基本就是E(S2)E(S^{2})的重复

 

例2：总体XX的数学期望和方差都存在，且EX=μ,DX=σ2EX=\mu,DX=\sigma^{2}。来自总体XX的样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}，试求

  • E[∑i=1n(Xi−μ)2]E\left[\sum\limits_{i=1}^{n}(X_{i}-\mu)^{2}\right]

 

E[1n∑i=1n(Xi−μ)2]=1n∑i=1nE(Xi−μ)2=1n∑i=1nE(Xi−EXi)2=1n∑i=1nDXi=σ2 \begin{aligned} E\left[ \frac{1}{n}\sum\limits_{i=1}^{n}(X_{i}-\mu)^{2}\right]&=\frac{1}{n}\sum\limits_{i=1}^{n}E(X_{i}-\mu)^{2}\\ &=\frac{1}{n}\sum\limits_{i=1}^{n}E(X_{i}-EX_{i})^{2}\\ &=\frac{1}{n}\sum\limits_{i=1}^{n}DX_{i}\\ &=\sigma^{2} \end{aligned}

 

  • E[1n∑i=1n(Xi−Xˉ)2]E\left[ \frac{1}{n}\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}\right]

 

E[1n∑i=1n(Xi−Xˉ)2]=1nE[∑i=1nXi2−nXˉ2]=1n(∑i=1nEXi2−nEXˉ2)=1n{∑i=1n[DXi+(EXi)2]−n[DXˉ+(EXˉ)2]}=1n[n(σ2+μ2)−n(σ2n+μ2)]=n−1nσ2 \begin{aligned} E\left[ \frac{1}{n}\sum\limits_{i=1}^{n}(X_{i}-\bar{X})^{2}\right]&=\frac{1}{n}E\left[\sum\limits_{i=1}^{n}X_{i}^{2}-n \bar{X}^{2}\right]\\ &=\frac{1}{n}\left(\sum\limits_{i=1}^{n}EX_{i}^{2}-nE \bar{X}^{2}\right)\\ &=\frac{1}{n}\left\{\sum\limits_{i=1}^{n}[DX_{i}+(EX_{i})^{2}]-n[D \bar{X}+(E \bar{X})^{2}] \right\}\\ &=\frac{1}{n}\left[n(\sigma^{2}+\mu^{2})-n\left(\frac{\sigma^{2}}{n}+\mu^{2}\right)\right]\\ &=\frac{n-1}{n}\sigma^{2} \end{aligned}

 

例3：设总体X∼B(1,p)X \sim B(1,p)，则来自总体XX的样本X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}的样本均值Xˉ\bar{X}的分布律为（）

 

X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}相互独立，XiX_{i}可看成一次伯努利试验，所以∑i=1nXi\sum\limits_{i=1}^{n}X_{i}可以看成nn次独立重复伯努利试验，即

∑i=1nXi∼B(n,p) \sum\limits_{i=1}^{n}X_{i}\sim B(n,p)

有

P{∑i=1nXi=k}=Cnkpk(1−p)n−k,k=0,1,2,⋯ ,nP{∑i=1nXin=kn}=Cnkpk(1−p)n−kP{Xˉ=kn}=Cnkpk(1−p)n−k \begin{aligned} P \left\{\sum\limits_{i=1}^{n}X_{i}=k\right\}&=C_{n}^{k}p^{k}(1-p)^{n-k},k=0,1,2,\cdots ,n\\ P \left\{\frac{\sum\limits_{i=1}^{n}X_{i}}{n}=\frac{k}{n}\right\}&=C_{n}^{k}p^{k}(1-p)^{n-k}\\ P \left\{\bar{X}=\frac{k}{n}\right\}&=C_{n}^{k}p^{k}(1-p)^{n-k} \end{aligned}
avatar
文章
阅读
粉丝