跳至主要內容

HW2_24_Fall

AI悦创原创2024年9月15日大约 7 分钟...约 2054 字

Question 1

Let yiθiidUniform(θ,θ)y_i \mid \theta \sim^{iid} \text{Uniform} (-\theta, \theta), for i=1,,ni = 1, \dots, n. Assume the prior distribution for θ\theta to be Pareto(a,b)\text{Pareto}(a, b), where

p(θ)=babθb+1for θa and 0 otherwise. p(\theta) = \frac{ba^b}{\theta^{b+1}} \quad \text{for } \theta \geq a \text{ and } 0 \text{ otherwise}.

Find the posterior distribution of θ\theta.

详情

要找到后验分布 p(θy)p(\theta \mid y),我们需要将似然函数和先验分布相乘,然后进行归一化。

1. 写出似然函数:

由于 yiy_i 在给定 θ\theta 下独立同分布为 Uniform(θ,θ)\text{Uniform}(-\theta, \theta),所以每个观测值的似然为:

p(yiθ)=12θ,当 θyiθ p(y_i \mid \theta) = \frac{1}{2\theta}, \quad \text{当 } -\theta \leq y_i \leq \theta

因此,总的似然函数为:

L(θ)=i=1np(yiθ)=(12θ)n,当 θmax{y1,y2,,yn} L(\theta) = \prod_{i=1}^n p(y_i \mid \theta) = \left( \frac{1}{2\theta} \right)^n, \quad \text{当 } \theta \geq \max\{|y_1|, |y_2|, \dots, |y_n|\}

2. 写出先验分布:

先验分布为 Pareto 分布:

p(θ)=babθb+1,当 θa p(\theta) = \frac{ba^b}{\theta^{b+1}}, \quad \text{当 } \theta \geq a

3. 计算后验分布:

后验分布正比于先验分布和似然函数的乘积:

p(θy)p(θ)L(θ)=babθb+1(12θ)n=bab2nθb+1+n p(\theta \mid y) \propto p(\theta) \cdot L(\theta) = \frac{ba^b}{\theta^{b+1}} \cdot \left( \frac{1}{2\theta} \right)^n = \frac{ba^b}{2^{n}\theta^{b+1+n}}

注意到 θ\theta 的取值范围受限于先验和似然,因此:

θmax{a,maxi{yi}} \theta \geq \max\left\{ a, \max_{i}\{|y_i|\} \right\}

4. 归一化后,后验分布仍为 Pareto 分布:

因此,后验分布为参数更新后的 Pareto 分布:

θyPareto(max{a,maxi{yi}}, b+n) \theta \mid y \sim \text{Pareto}\left( \max\left\{ a, \max_{i}\{|y_i|\} \right\},\ b + n \right)

答案:

因此,θ 的后验分布是参数为 max { a, max|yᵢ| } 和 b+n 的帕累托分布,即

θyPareto(max{a,maxi{yi}}, b+n) \theta \mid y \sim \text{Pareto}\left( \max\left\{ a, \max_{i}\{|y_i|\} \right\},\ b + n \right)

Question 2

The data set neurondiffs.dat|xlsx comes from the lab of Dr. Steve Potter at the Department of Biomedical Engineering, Georgia Tech. It consists of 988 time intervals between successive firings in a cell culture of neurons. The firing times are defined as time instances when a neuron sends a signal to another linked neuron (a spike). The cells, from the cortex of an embryonic rat brain, were cultured for 18 days on multielectrode arrays. The measurements were taken while the culture was stimulated at the rate of 1 Hz. It was postulated that firing times form a Poisson process; thus the interspike intervals provided in the data set should have an exponential distribution.

(a) Check the histogram of interspike intervals and discuss its resemblance to the exponential density. What is the MLE for exponential rate parameter λ\lambda.

(b) Given the exponential model for interspike intervals TiT_i, find the posterior distribution of rate parameter λ\lambda when the prior for λ\lambda is gamma Ga(18,20)Ga(18, 20). What is the Bayes estimator for λ\lambda. Find also the posterior variance of λ\lambda).

(c) If the model for TiT_i is parameterized by a scale parameter μ\mu ( = 1/λ1/\lambda ), find the posterior mean of μ\mu if the prior on μ\mu is inverse-gamma IG(18,20)IG(18, 20).

详情

(a)

首先,我们需要绘制神经元发放间隔时间的直方图,以观察其与指数分布的相似程度。指数分布的概率密度函数为:

f(t;λ)=λeλt,t0 f(t; \lambda) = \lambda e^{-\lambda t}, \quad t \geq 0

根据提供的数据集,共有 988 个间隔时间。我们可以计算样本均值 tˉ\bar{t}

tˉ=1ni=1nti \bar{t} = \frac{1}{n} \sum_{i=1}^{n} t_i

将所有间隔时间相加,得到总和:

i=1nti=999.0156 \sum_{i=1}^{n} t_i = 999.0156

因此,样本均值为:

tˉ=999.01569881.0101 \bar{t} = \frac{999.0156}{988} \approx 1.0101

对于指数分布,参数 (\lambda) 的最大似然估计(MLE)为:

λ^=1tˉ11.01010.9900 \hat{\lambda} = \frac{1}{\bar{t}} \approx \frac{1}{1.0101} \approx 0.9900

结论: 直方图显示,间隔时间的大致分布与指数分布相符。最大似然估计得到的指数率参数 (\lambda) 为 0.9900


(b)

已知间隔时间 TiT_i 服从指数分布 Exp(λ)\text{Exp}(\lambda),先验分布为 λGamma(18,20)\lambda \sim \text{Gamma}(18, 20)

Gamma 分布的概率密度函数为:

π(λ)=β0α0Γ(α0)λα01eβ0λ \pi(\lambda) = \frac{\beta_0^{\alpha_0}}{\Gamma(\alpha_0)} \lambda^{\alpha_0 - 1} e^{-\beta_0 \lambda}

其中,α0=18\alpha_0 = 18β0=20\beta_0 = 20

似然函数为:

L(λ)=λneλi=1nti L(\lambda) = \lambda^n e^{-\lambda \sum_{i=1}^{n} t_i}

后验分布:

p(λdata)L(λ)π(λ)λn+α01eλ(β0+i=1nti) p(\lambda | \text{data}) \propto L(\lambda) \pi(\lambda) \propto \lambda^{n + \alpha_0 - 1} e^{-\lambda (\beta_0 + \sum_{i=1}^{n} t_i)}

因此,后验分布为 λdataGamma(α1,β1)\lambda | \text{data} \sim \text{Gamma}(\alpha_1, \beta_1),其中:

α1=α0+n=18+988=1006β1=β0+i=1nti=20+999.0156=1019.0156 \alpha_1 = \alpha_0 + n = 18 + 988 = 1006 \\ \beta_1 = \beta_0 + \sum_{i=1}^{n} t_i = 20 + 999.0156 = 1019.0156

贝叶斯估计(后验均值)为:

E[λdata]=α1β1=10061019.01560.9870 E[\lambda | \text{data}] = \frac{\alpha_1}{\beta_1} = \frac{1006}{1019.0156} \approx 0.9870

后验方差为:

Var[λdata]=α1β12=1006(1019.0156)29.669×104 \text{Var}[\lambda | \text{data}] = \frac{\alpha_1}{\beta_1^2} = \frac{1006}{(1019.0156)^2} \approx 9.669 \times 10^{-4}

结论: 后验分布为 Gamma(1006,1019.0156)\text{Gamma}(1006, 1019.0156),贝叶斯估计为 0.9870,后验方差约为 0.0009669


(c)

若模型以尺度参数 μ=1/λ\mu = 1/\lambda 表示,且先验分布为 μInverse-Gamma(18,20)\mu \sim \text{Inverse-Gamma}(18, 20)

逆 Gamma 分布的概率密度函数为:

π(μ)=β0α0Γ(α0)μα01eβ0/μ \pi(\mu) = \frac{\beta_0^{\alpha_0}}{\Gamma(\alpha_0)} \mu^{-\alpha_0 - 1} e^{-\beta_0 / \mu}

似然函数为:

L(μ)=(1μ)nei=1ntiμ L(\mu) = \left( \frac{1}{\mu} \right)^n e^{-\frac{\sum_{i=1}^{n} t_i}{\mu}}

后验分布:

p(μdata)L(μ)π(μ)μnα01e(β0+i=1nti)/μ p(\mu | \text{data}) \propto L(\mu) \pi(\mu) \propto \mu^{-n - \alpha_0 - 1} e^{-\left( \beta_0 + \sum_{i=1}^{n} t_i \right) / \mu}

因此,后验分布为 μdataInverse-Gamma(α1,β1)\mu | \text{data} \sim \text{Inverse-Gamma}(\alpha_1, \beta_1),其中:

α1=α0+n=18+988=1006β1=β0+i=1nti=20+999.0156=1019.0156 \alpha_1 = \alpha_0 + n = 18 + 988 = 1006 \\ \beta_1 = \beta_0 + \sum_{i=1}^{n} t_i = 20 + 999.0156 = 1019.0156

后验均值为:

E[μdata]=β1α11=1019.015610051.0141 E[\mu | \text{data}] = \frac{\beta_1}{\alpha_1 - 1} = \frac{1019.0156}{1005} \approx 1.0141

结论: 后验分布为 ( \text{Inverse-Gamma}(1006, 1019.0156) ),尺度参数 ( \mu ) 的后验均值为 1.0141

Question 3

A lifetime XX (in years) of a particular device is modeled by a Weibull distribution

f(xν,θ)=νθxν1exp{θxν},x0, f(x \mid \nu, \theta) = \nu \theta x^{\nu - 1} \exp \{ -\theta x^{\nu} \}, \quad x \geq 0,

with shape parameter ν=3\nu = 3 and unknown rate parameter θ\theta. The lifetimes of X1=3X_1 = 3, X2=4X_2 = 4, and X3=2X_3 = 2 are observed. Assume that an expert familiar with this type of devices suggested an exponential prior on θ\theta with rate parameter λ=52\lambda = \frac{5}{2}.

(a) For the prior suggested by the expert, find the posterior distribution of θ\theta.

(b) What are the posterior mean and variance? No need to integrate if you recognize to which family of distributions the posterior belongs.

详情

(a) 求后验分布:

首先,我们有先验分布为指数分布,参数为λ=52\lambda = \frac{5}{2},即:

p(θ)=λeλθ=52e52θ,θ0. p(\theta) = \lambda e^{-\lambda \theta} = \frac{5}{2} e^{-\frac{5}{2} \theta}, \quad \theta \geq 0.

观测数据为X1=3X_1 = 3, X2=4X_2 = 4, X3=2X_3 = 2,且寿命服从Weibull分布,形状参数ν=3\nu = 3。因此,似然函数为:

L(θ)=i=13f(xiν,θ)=i=13νθxiν1eθxiν=[νθ]3i=13xiν1eθi=13xiν. L(\theta) = \prod_{i=1}^{3} f(x_i \mid \nu, \theta) = \prod_{i=1}^{3} \nu \theta x_i^{\nu - 1} e^{-\theta x_i^{\nu}} = [\nu \theta]^3 \prod_{i=1}^{3} x_i^{\nu - 1} e^{-\theta \sum_{i=1}^{3} x_i^{\nu}}.

将先验分布和似然函数相乘,得到未归一化的后验分布:

p(θx)L(θ)p(θ)=[νθ]3(i=13xiν1)eθ(i=13xiν+λ). p(\theta \mid x) \propto L(\theta) \cdot p(\theta) = [\nu \theta]^3 \left( \prod_{i=1}^{3} x_i^{\nu - 1} \right) e^{-\theta \left( \sum_{i=1}^{3} x_i^{\nu} + \lambda \right)}.

由于 ν\nuxix_i 都是已知常数,因此可以将它们视为比例常数,后验分布的关键部分为:

p(θx)θ3eθ(λ+i=13xiν). p(\theta \mid x) \propto \theta^3 e^{-\theta \left( \lambda + \sum_{i=1}^{3} x_i^{\nu} \right)}.

计算i=13xiν\sum_{i=1}^{3} x_i^{\nu}

x1ν=33=27,x2ν=43=64,x3ν=23=8,i=13xiν=27+64+8=99. x_1^{\nu} = 3^3 = 27, \quad x_2^{\nu} = 4^3 = 64, \quad x_3^{\nu} = 2^3 = 8, \\ \sum_{i=1}^{3} x_i^{\nu} = 27 + 64 + 8 = 99.

因此,后验分布为:

p(θx)θ3eθ(52+99)=θ3eθ2032. p(\theta \mid x) \propto \theta^3 e^{-\theta \left( \frac{5}{2} + 99 \right)} = \theta^3 e^{-\theta \frac{203}{2}}.

这与Gamma分布的形式一致,Gamma分布的概率密度函数为:

Gamma(α,β):f(θ;α,β)=βαΓ(α)θα1eβθ,θ0. \text{Gamma}(\alpha, \beta): \quad f(\theta; \alpha, \beta) = \frac{\beta^\alpha}{\Gamma(\alpha)} \theta^{\alpha - 1} e^{-\beta \theta}, \quad \theta \geq 0.

因此,后验分布为参数为 α=4\alpha = 4β=2032\beta = \frac{203}{2} 的 Gamma 分布:

θxGamma(4,2032). \theta \mid x \sim \text{Gamma}\left(4, \frac{203}{2}\right).

(b) 求后验均值和方差:

对于 Gamma 分布 Gamma(α,β)\text{Gamma}(\alpha, \beta),均值和方差分别为:

E[θ]=αβ,Var[θ]=αβ2. \text{E}[\theta] = \frac{\alpha}{\beta}, \quad \text{Var}[\theta] = \frac{\alpha}{\beta^2}.

因此,后验均值为:

E[θx]=42032=8203. \text{E}[\theta \mid x] = \frac{4}{\frac{203}{2}} = \frac{8}{203}.

后验方差为:

Var[θx]=4(2032)2=420324=162032. \text{Var}[\theta \mid x] = \frac{4}{\left( \frac{203}{2} \right)^2} = \frac{4}{\frac{203^2}{4}} = \frac{16}{203^2}.

答案总结:

(a) 后验分布为θxGamma(4,2032)\theta \mid x \sim \text{Gamma}\left(4, \frac{203}{2}\right)

(b) 后验均值为8203\frac{8}{203},后验方差为16(203)2\frac{16}{(203)^2}

公众号:AI悦创【二维码】

AI悦创·编程一对一

AI悦创·推出辅导班啦,包括「Python 语言辅导班、C++ 辅导班、java 辅导班、算法/数据结构辅导班、少儿编程、pygame 游戏开发、Web、Linux」,全部都是一对一教学:一对一辅导 + 一对一答疑 + 布置作业 + 项目实践等。当然,还有线下线上摄影课程、Photoshop、Premiere 一对一教学、QQ、微信在线,随时响应!微信:Jiabcdefh

C++ 信息奥赛题解,长期更新!长期招收一对一中小学信息奥赛集训,莆田、厦门地区有机会线下上门,其他地区线上。微信:Jiabcdefh

方法一:QQ

方法二:微信:Jiabcdefh

你认为这篇文章怎么样?
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
评论
  • 按正序
  • 按倒序
  • 按热度
通知
关于编程私教&加密文章