Statistics 2 Problem Sheet 4

Question

What to hand in: answers to Q1–Q4.

Please write the day and time of your tutorial class on your homework.

1. For each of the following regular statistical models (so, e.g., the Fisher Infor-mation Regularity Conditions hold), derive the score function $\ell'(\theta; x)$ and the Fisher Information, $I (\theta)$ , where in both cases you may need to replace $\theta$ with the parameter of interest (e.g. p in the case where $X$ is Geometric or Binomial).

(a) $X \sim Geometric(p)$;

(b) $X \sim Binomial(n, p)$ where n is specified;

(c) $X \sim \mathcal{N}(\mu, \sigma^2)$ where $\sigma^2$ is specified;

(d) $X \sim \text{Pareto}(x_0, \theta)$ where $x_0$​ is specified (see Worksheet 2 for the Pareto probability density function).

2. In each of the cases in the previous question, verify that $\mathbb{E}\left[ \ell'(\theta; X) ; \theta \right] = 0$.

3. In the case where $X \sim \text{Pareto}(x_0, \theta)$ , with both $x_0$ and θ unknown, explain why condition (A1) from the lecture notes is not satisfied.

4. Suppose that $X \stackrel{\text{iid}}{\sim} \ \text{Poisson}(\lambda)$ and consider the ML estimator $\hat{\lambda}(X) = n^{-1} (X_1 + \dots + X_n)$.We showed in the lectures that this estimator is unbiased and has a mean squared error of $n^{-1}\lambda$ . Now we derive its sampling distribution. Its CDF is

$$\mathbb{P}\left(\hat{\lambda}(X) \leq v; \lambda\right) = \mathbb{P}\left\{n^{-1} (X_1 + \dots + X_n) \leq v; \lambda\right\} \\ = \mathbb{P}\{X_1 + \dots + X_n \leq nv; \lambda\} \\ = \mathbb{P}\{Y \leq nv; \lambda\}$$

where $Y \sim Poisson(n\lambda)$, because the sum of n independent and identical Poisson variates with rate $\lambda$ is Poisson with rate $n\lambda$. Therefore the sampling distribution has distribution function

$$\mathbb{P}\left(\hat{\lambda}(X) \leq v; \lambda\right) = \sum_{i=0}^{\lfloor nv \rfloor} e^{-n\lambda} \frac{(n\lambda)^i}{i!}$$

where ${\lfloor nv \rfloor}$ is the largest integer not greater than nv. We can program this as follows:

cdf.poison.mle <- function(v, lambda, n) {
    ppois(n * v, lambda = n * lambda)
}

Now we can plot this for a particular choice of n and $\lambda$, and add on the true value of $\lambda$:

n <- 12
lambda <- 3.2
v <- seq(from = 1, to = 5, length = 1001)
plot(v, cdf.poison.mle(v, lambda = lambda, n = n), type = "l",
     main = "CDF of the ML estimator for lambda",
     xlab = "lambda", ylab = "CDF")
abline(v = lambda, col = "darkgreen")
text(lambda, 0, labels = "True value of lambda", pos = 4, col = "darkgreen")

1. 得分函数和Fisher信息

(a) $X \sim \text{Geometric}(p)$

$$\ell'(p; x) = \frac{(1 - p)^{1 - x}}{p} \left( \frac{p(1 - p)^{x - 1}(1 - x)}{1 - p} + (1 - p)^{x - 1} \right) \\ I(p) = -\frac{-p^2x + 2p - 1}{p^2(p^2 - 2p + 1)}$$

(b) $X \sim \text{Binomial}(n, p)$

$$\ell'(p; x) = \frac{p^{-x}(1 - p)^{-n + x}}{\binom{n}{x}} \left( \frac{p^x(1 - p)^{n - x}(-n + x)\binom{n}{x}}{1 - p} + \frac{p^x x (1 - p)^{n - x}\binom{n}{x}}{p} \right) \\ I(p) = -\frac{-n p^2 + 2 p x - x}{p^2(p^2 - 2p + 1)}$$

(c) $X \sim \mathcal{N}(\mu, \sigma^2)$

$$\ell'(\mu; x) = -0.5 \sigma^{-2} (2\mu - 2x) \\ I(\mu) = \sigma^{-2}$$

(d) $X \sim \text{Pareto}(x_0, \theta)$

$$\ell'(\theta; x) = \frac{x^{\theta + 1} x_0^{-\theta}}{\theta} \left( -\theta x^{-\theta - 1} x_0^{\theta} \log(x) + \theta x^{-\theta - 1} x_0^{\theta} \log(x_0) + x^{-\theta - 1} x_0^{\theta} \right) \\ I(\theta) = \theta^{-2}$$

2. 验证 $\mathbb{E}\left[ \ell'(\theta; X) ; \theta \right] = 0$

基于常规性条件，对于上述所有分布，我们都有:

$$\mathbb{E}\left[ \ell'(\theta; X) ; \theta \right] = 0$$

3. $X \sim \text{Pareto}(x_0, \theta)$ 与条件 (A1)

当 $x_0$ 和 $\theta$ 都是未知的时，条件 (A1) 不被满足。这是因为可能存在多组 $x_0$ 和 $\theta$ 的值，它们产生相同的累积分布函数 (CDF)，从而使参数不可识别。

公众号：AI悦创【二维码】