What to hand in: answers to Q1–Q4.
Please write the day and time of your tutorial class on your homework.
This question is about ML estimates of transformations of the parameter.
(a) Suppose that X ∼ iid Exponential ( λ ) X \stackrel{\text{iid}}{\sim} \text{Exponential}(\lambda) X ∼ iid Exponential ( λ ) where Θ = R + + \Theta = \mathbb{R}_{++} Θ = R ++ (i.e. the strictly positive real numbers). Consider a transformed parameter τ = 1 1 + λ \tau = \frac{1}{1 + \lambda} τ = 1 + λ 1 . Write the likelihood function in terms of τ \tau τ , then maximize it to find the ML estimate τ ^ \hat{\tau} τ ^ . Confirm that τ ^ = 1 1 + λ ^ \hat{\tau} = \frac{1}{1 + \hat{\lambda}} τ ^ = 1 + λ ^ 1 , where λ ^ \hat{\lambda} λ ^ is the ML estimate of λ \lambda λ .
(b) If X ∼ Poisson ( λ ) X \sim \text{Poisson}(\lambda) X ∼ Poisson ( λ ) where Θ = R + + \Theta = \mathbb{R}_{++} Θ = R ++ , then P { X = 0 ; λ } = exp ( − λ ) P\{X = 0; \lambda\} = \exp(-\lambda) P { X = 0 ; λ } = exp ( − λ ) . Explain why the ML estimate of this probability is exp ( − λ ^ ) \exp(-\hat{\lambda}) exp ( − λ ^ ) , where λ ^ \hat{\lambda} λ ^ is the ML estimate of λ \lambda λ .
Suppose that X 1 , X 2 , . . . X_1, X_2, ... X 1 , X 2 , ... are i.i.d. Uniform[0,1] random variables, each with density f ( x ) = I ( 0 ≤ x ≤ 1 ) f(x) = \mathbb{I}(0 \leq x \leq 1) f ( x ) = I ( 0 ≤ x ≤ 1 ) .
(a) Calculate the mean and variance of log X 1 \log X_1 log X 1 .
(b) By taking logs, find a random variable X X X such that as n → ∞ n \rightarrow \infty n → ∞
( X 1 ⋅ ⋅ ⋅ X n ) 1 n e n → D X . (X_1 \cdot\cdot\cdot X_n)^{\frac{1}{\sqrt{n}}} e^{\sqrt{n}} \xrightarrow{}_D X. ( X 1 ⋅ ⋅ ⋅ X n ) n 1 e n D X .
1.(a)
首先,给定 X X X 服从指数分布,其概率密度函数 (pdf) 为:
f ( x ∣ λ ) = λ e − λ x , x > 0 f(x|\lambda) = \lambda e^{-\lambda x}, \, x > 0 f ( x ∣ λ ) = λ e − λ x , x > 0
对于一个样本点 x x x ,似然函数为上述公式。对于 iid 的样本 x 1 , x 2 , … , x n x_1, x_2, \ldots, x_n x 1 , x 2 , … , x n ,似然函数为:
L ( λ ∣ x 1 , … , x n ) = ∏ i = 1 n λ e − λ x i = λ n e − λ ∑ i = 1 n x i L(\lambda|x_1, \ldots, x_n) = \prod_{i=1}^{n} \lambda e^{-\lambda x_i} = \lambda^n e^{-\lambda \sum_{i=1}^{n} x_i} L ( λ ∣ x 1 , … , x n ) = i = 1 ∏ n λ e − λ x i = λ n e − λ ∑ i = 1 n x i
现在考虑参数转换,τ = 1 1 + λ \tau = \frac{1}{1+\lambda} τ = 1 + λ 1 。为了表示 λ \lambda λ 为 τ \tau τ 的函数,我们可以重新写:
λ = 1 − τ τ \lambda = \frac{1-\tau}{\tau} λ = τ 1 − τ
将这个转换后的 λ \lambda λ 代入上述似然函数,我们得到:
L ( τ ∣ x 1 , … , x n ) = ( 1 − τ τ ) n e − 1 − τ τ ∑ i = 1 n x i L(\tau|x_1, \ldots, x_n) = \left( \frac{1-\tau}{\tau} \right)^n e^{-\frac{1-\tau}{\tau} \sum_{i=1}^{n} x_i} L ( τ ∣ x 1 , … , x n ) = ( τ 1 − τ ) n e − τ 1 − τ ∑ i = 1 n x i
为了找到最大似然估计值 τ ^ \hat{\tau} τ ^ ,需要对上述似然函数求导并令其为 0。先求 λ ^ \hat{\lambda} λ ^ ,然后使用关系 τ ^ = 1 1 + λ ^ \hat{\tau} = \frac{1}{1+\hat{\lambda}} τ ^ = 1 + λ ^ 1 得到 τ ^ \hat{\tau} τ ^ 。
对于 λ \lambda λ ,最大似然估计 λ ^ \hat{\lambda} λ ^ ,对似然函数的对数求导并令其为0:
d d λ log L ( λ ) = n − λ ∑ i = 1 n x i = 0 \frac{d}{d\lambda} \log L(\lambda) = n - \lambda \sum_{i=1}^{n} x_i = 0 d λ d log L ( λ ) = n − λ i = 1 ∑ n x i = 0
得到
n − ∑ i = 1 n x i = 0 n = λ ∑ i = 1 n x i ⟹ λ ^ = n ∑ i = 1 n x i τ ^ = 1 1 + λ ^ = 1 1 + n ∑ i = 1 n x i n - \sum_{i=1}^{n} x_i = 0 \\ n = \lambda \sum_{i=1}^{n} x_i \\ \implies \hat{\lambda} = \frac{n}{\sum_{i=1}^{n} x_i} \\ \hat{\tau} = \frac{1}{1+\hat{\lambda}} = \frac{1}{1+\frac{n}{\sum_{i=1}^{n} x_i}} n − i = 1 ∑ n x i = 0 n = λ i = 1 ∑ n x i ⟹ λ ^ = ∑ i = 1 n x i n τ ^ = 1 + λ ^ 1 = 1 + ∑ i = 1 n x i n 1
1.(b)
对于 Poisson 分布,我们有:
P { X = 0 ∣ λ } = e − λ λ 0 0 ! = e − λ P\{X=0|\lambda\} = \frac{e^{-\lambda} \lambda^0}{0!} = e^{-\lambda} P { X = 0∣ λ } = 0 ! e − λ λ 0 = e − λ
最大似然估计 λ ^ \hat{\lambda} λ ^ 是使得数据的似然函数最大化的 λ \lambda λ 值。对于这个概率的最大似然估计,我们只需使用 λ ^ \hat{\lambda} λ ^ 代入上述公式,得到:
P { X = 0 ∣ λ ^ } = e − λ ^ P\{X=0|\hat{\lambda}\} = e^{-\hat{\lambda}} P { X = 0∣ λ ^ } = e − λ ^
2. (a)
给定 X 1 X_1 X 1 服从均匀分布 [ 0 , 1 ] [0, 1] [ 0 , 1 ] ,其概率密度函数 (pdf) 为:
f ( x ) = I ( 0 ≤ x ≤ 1 ) f(x) = \mathbb{I}(0 \leq x \leq 1) f ( x ) = I ( 0 ≤ x ≤ 1 )
随机变量 Y = log X 1 Y = \log X_1 Y = log X 1 的期望值是:
E ( Y ) = E ( log X 1 ) = ∫ 0 1 log ( x ) d x = x ( log ( x ) − 1 ) ∣ 0 1 = − 1 E(Y) = E(\log X_1) = \int_{0}^{1} \log(x) dx = x(\log(x) - 1) \Big|_0^1 = -1 E ( Y ) = E ( log X 1 ) = ∫ 0 1 log ( x ) d x = x ( log ( x ) − 1 ) 0 1 = − 1
方差是:
Var ( Y ) = E ( Y 2 ) − [ E ( Y ) ] 2 \text{Var}(Y) = E(Y^2) - [E(Y)]^2 Var ( Y ) = E ( Y 2 ) − [ E ( Y ) ] 2
其中:
E ( Y 2 ) = E ( log 2 X 1 ) = ∫ 0 1 log 2 ( x ) d x = 2 E(Y^2) = E(\log^2 X_1) \\= \int_{0}^{1} \log^2(x) dx = 2 E ( Y 2 ) = E ( log 2 X 1 ) = ∫ 0 1 log 2 ( x ) d x = 2
方差为:
Var ( Y ) = 2 − ( − 1 ) 2 = 1 \text{Var}(Y) = 2 - (-1)^2 = 1 Var ( Y ) = 2 − ( − 1 ) 2 = 1
则随机变量 Y = log X 1 Y = \log X_1 Y = log X 1 的均值和方差为:
E ( Y ) = E ( log X 1 ) = ∫ 0 1 log x d x Var ( Y ) = E ( Y 2 ) − [ E ( Y ) ] 2 E ( Y 2 ) = E ( log 2 X 1 ) = ∫ 0 1 log 2 x d x E(Y) = E(\log X_1) = \int_{0}^{1} \log x \, dx \\ \text{Var}(Y) = E(Y^2) - [E(Y)]^2 \\ E(Y^2) = E(\log^2 X_1) = \int_{0}^{1} \log^2 x \, dx E ( Y ) = E ( log X 1 ) = ∫ 0 1 log x d x Var ( Y ) = E ( Y 2 ) − [ E ( Y ) ] 2 E ( Y 2 ) = E ( log 2 X 1 ) = ∫ 0 1 log 2 x d x
2.(b)
考虑随机变量 Z n = log ( X 1 ⋅ X 2 ⋅ . . . ⋅ X n ) = log X 1 + log X 2 + . . . + log X n Z_n = \log(X_1 \cdot X_2 \cdot ... \cdot X_n) = \log X_1 + \log X_2 + ... + \log X_n Z n = log ( X 1 ⋅ X 2 ⋅ ... ⋅ X n ) = log X 1 + log X 2 + ... + log X n 。由中心极限定理,当 n → ∞ n \rightarrow \infty n → ∞ 时,样本均值收敛于真实均值,样本方差收敛于真实方差。
所以,我们考虑变换:
Z n − n E ( log X 1 ) n σ ( log X 1 ) → D N ( 0 , 1 ) \frac{Z_n - nE(\log X_1)}{\sqrt{n} \sigma(\log X_1)} \xrightarrow{}_D N(0, 1) n σ ( log X 1 ) Z n − n E ( log X 1 ) D N ( 0 , 1 )
其中,σ ( log X 1 ) \sigma(\log X_1) σ ( log X 1 ) 是 log X 1 \log X_1 log X 1 的标准差,而 N ( 0 , 1 ) N(0, 1) N ( 0 , 1 ) 的均值为 0、方差为 1 的标准正态分布。
回到题目要求,需要找到一个随机变量 X X X ,使得:
( X 1 ⋅ X 2 ⋅ . . . ⋅ X n ) 1 n e n → D X (X_1 \cdot X_2 \cdot ... \cdot X_n)^{\frac{1}{\sqrt{n}}} e^{\sqrt{n}} \xrightarrow{}_D X ( X 1 ⋅ X 2 ⋅ ... ⋅ X n ) n 1 e n D X
考虑其对数形式:
1 n log ( X 1 ⋅ X 2 ⋅ . . . ⋅ X n ) + n \frac{1}{\sqrt{n}} \log(X_1 \cdot X_2 \cdot ... \cdot X_n) + \sqrt{n} n 1 log ( X 1 ⋅ X 2 ⋅ ... ⋅ X n ) + n
由于 Z n n → D 0 \frac{Z_n}{\sqrt{n}} \xrightarrow{}_D 0 n Z n D 0 ,上式收敛于 n \sqrt{n} n 。因此,(X) 是一个以 e n e^{\sqrt{n}} e n 为参数的指数随机变量。
This question requires you to use R to verify the Weak Law of Large Numbers (WLLN). We need to generate a large vector of IID quantities, for example X ∼ iid Exponential ( λ ) X \stackrel{\text{iid}}{\sim} \text{Exponential}(\lambda) X ∼ iid Exponential ( λ ) with λ = 2 \lambda = 2 λ = 2 :
公众号:AI悦创【二维码】 AI悦创·编程一对一
AI悦创·推出辅导班啦,包括「Python 语言辅导班、C++ 辅导班、java 辅导班、算法/数据结构辅导班、少儿编程、pygame 游戏开发、Web、Linux」,全部都是一对一教学:一对一辅导 + 一对一答疑 + 布置作业 + 项目实践等。当然,还有线下线上摄影课程、Photoshop、Premiere 一对一教学、QQ、微信在线,随时响应!微信:Jiabcdefh
C++ 信息奥赛题解,长期更新!长期招收一对一中小学信息奥赛集训,莆田、厦门地区有机会线下上门,其他地区线上。微信:Jiabcdefh
方法一:QQ
方法二:微信:Jiabcdefh