Statistics 2 Problem Sheet 2

What to hand in: answers to Q1–Q2.

Please write the day and time of your tutorial class on your homework.

You should also work through the R exercise in Q3. It is very important that you make progress with R now: do not leave it until just before the first Computer Practical. If you are struggling with Q3 you should attend the Computing Office Hours.

1. Suppose that $X \stackrel{\text{iid}}{\sim} \text{Exponential}(\lambda)$ , so that the log-likelihood is

$$\ell(\lambda; x) = \log f_n(x; \lambda) = c + n \log \lambda - \lambda \bar{x}$$

where $\bar{x} := n^{-1} \sum_{i=1}^{n} x_i$ , and c is some constant. This is a differentiable function of λ,and so to find the Maximum Likelihood estimate (ML estimate), we can differentiate and then solve the first-order conditions. Differentiating gives

$$\ell'(X; \lambda) = \frac{n}{\lambda} - n\bar{x}$$

Setting this equal to zero and solving for λ gives

$$\frac{n}{\lambda} - n\bar{x} = 0 \implies \lambda = \frac{1}{\bar{x}}.$$

Note that $\ell''(\lambda; X) = -\frac{n}{\lambda^2} < 0 \text{ for all } \lambda$ , so this value maximizes $\ell$ . Hence the ML estimate is

$$\hat{\lambda}(X) = \frac{1}{\bar{x}}$$

Now derive the ML estimates for the following distributions in a similar manner. Note that the final distribution must be handled differently, as you will see that $f(x; a, b)$ is not a smooth function of a or b.

(a)$X \stackrel{\text{iid}}{\sim} \ \text{Poisson}(\lambda)$

(b) $X \sim \text{Binomial}(n, p)$ , where n is specified (i.e. only p is unknown).

(c) $X \stackrel{\text{iid}}{\sim} \ \text{Normal}(\mu, \sigma^2)$

(d) $X \stackrel{\text{iid}}{\sim} \ \text{Uniform}(a, b)$

2. In this question and the next we will use the Pareto distribution, whose PDF may be written

$$f_X(x; x_0, \theta) = \frac{\theta x_0^\theta}{x^{\theta + 1}} \mathbb{I}(x \ge x_0),$$

$\text{where } x_0 > 0 \text{ and } \theta > 0.$

(a) Show that the CDF, or distribution function, of $X$ is

$$F_X(x; x_0, \theta) = \begin{cases} 1 - \left(\frac{x_0}{x}\right)^\theta \mathbb{I} (x \geq x_0) \end{cases}$$

(b) Show that the quantile function of $X$ is

$$F^{-1}_X(u; x_0, \theta) = x_0(1 - u)^{-1/\theta}$$

$\text{for} \space 0 < u < 1.$

[Hint: the quantile function is defined as follows: for any ( $u \in (0,1)$ ) one wants to find the smallest ( $x \in \mathbb{R}$ ) such that

$$F_X(x) \geq u,$$

that is more formally,

$$x = F_X^{-}(u) := \inf\{y \in \mathbb{R} : F_X(y) \geq u\}.$$

$F_X^{-}$ is a generalized inverse, as in general $F_x$ need not be continuous and strictly increasing. In the present case you can find $F_X^{-1}$ , the actual inverse of $F_x$ , e.g. by finding $x \in \mathbb{R}$ such that $Fx(x) = u$ for an arbitrary $u \in (0,1)$].

(c) How can we generate random quantities from this distribution?

[ $\text{Hint:} \text{ show that the CDF of the r.v. } Y = F_X^{-1}(U) \text{ for } U \sim \text{Uniform}(0, 1) \text{ is the same as } X \sim \text{Pareto}(x_0, \theta)$ ]

3. In R we can write functions ppareto, dpareto, qpareto, and rpareto, to compute the probability function of x, the density function of x, the quantile function of u, and n random realizations of X.

---

1. 导出以下分布的最大似然估计（ML estimates）

(a) $X \stackrel{\text{iid}}{\sim} \text{Poisson}(\lambda)$

给定 Poisson 分布的概率密度函数：

$$f(x;\lambda) = \frac{e^{-\lambda} \lambda^x}{x!}$$

似然函数为：

$$L(\lambda;x) = \prod_{i=1}^{n} f(x_i;\lambda) = e^{-n\lambda} \lambda^{\sum x_i} / \prod_{i=1}^{n} x_i!$$

对数似然函数为：

$$\ell(\lambda;x) = -n\lambda + \log(\lambda) \sum_{i=1}^{n} x_i - \log \prod_{i=1}^{n} x_i!$$

求导并令其为0，得到：

$$\frac{d\ell}{d\lambda} = -n + \frac{\sum x_i}{\lambda} = 0$$

从中我们可以解得：

$$\lambda = \frac{\sum_{i=1}^{n} x_i}{n} = \bar{x}$$

(b) $X \sim \text{Binomial}(n, p)$

给定二项分布的概率密度函数：

$$f(x;n,p) = \binom{n}{x} p^x (1-p)^{n-x}$$

对数似然函数为：

$$\ell(p;x) = \log \binom{n}{x_i} + x_i \log p + (n-x_i) \log(1-p)$$

求导并令其为 0，得到：

$$\frac{d\ell}{dp} = \frac{x_i}{p} - \frac{n-x_i}{1-p} = 0$$

解得：

$$p = \frac{x_i}{n}$$

(c) $X \stackrel{\text{iid}}{\sim} \text{Normal}(\mu, \sigma^2)$

对于正态分布，其概率密度函数为：

$$f(x;\mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$$

对数似然函数为：

$$\ell(\mu, \sigma^2;x) = -\frac{n}{2}\log(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2$$

对于正态分布，最大似然估计为：

$$\mu = \bar{x}$$

和

$$\sigma^2 = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2$$

(d) $X \stackrel{\text{iid}}{\sim} \text{Uniform}(a, b)$

对于均匀分布，最大似然估计为：

$$a = \min(x_1, x_2, \dots, x_n)$$

和

$$b = \max(x_1, x_2, \dots, x_n)$$

2. Pareto 分布

(a) 显示 CDF

从PDF开始，我们可以得到CDF：

$$F_X(x; x_0, \theta) = \int_{x_0}^{x} f_X(t; x_0, \theta) dt = \int_{x_0}^{x} \frac{\theta x_0^\theta}{t^{\theta + 1}} dt = 1 - \left(\frac{x_0}{x}\right)^\theta \mathbb{I} (x \geq x_0)$$

(b) 显示分位数函数

根据提示，我们有：

$$F_X(x) = u$$

插入 CDF，并解出 x，得到：

$$x = F_X^{-1}(u; x_0, \theta) = x_0(1 - u)^{-1/\theta}$$

(c) 生成随机数

根据提示，我们有：

$$Y = F_X^{-1}(U)$$

其中( $U \sim \text{Uniform}(0, 1)$ )。

这意味着可以从均匀分布中取样，并通过 Pareto 的逆 CDF 将其转换为 Pareto 分布的随机样本。

公众号：AI悦创【二维码】

AI悦创·编程一对一

AI悦创·推出辅导班啦，包括「Python 语言辅导班、C++ 辅导班、java 辅导班、算法/数据结构辅导班、少儿编程、pygame 游戏开发、Web、Linux」，全部都是一对一教学：一对一辅导 + 一对一答疑 + 布置作业 + 项目实践等。当然，还有线下线上摄影课程、Photoshop、Premiere 一对一教学、QQ、微信在线，随时响应！微信：Jiabcdefh

C++ 信息奥赛题解，长期更新！长期招收一对一中小学信息奥赛集训，莆田、厦门地区有机会线下上门，其他地区线上。微信：Jiabcdefh

方法一：QQ

方法二：微信：Jiabcdefh