積率母関数

$$ \begin{aligned} &M_X(t)^{\prime}|{t=0} = \mathbb{E}[\exp{(tX)}^{\prime}]|{t=0} =\mathbb{E}[X\exp{(tX)}]|{t=0}= \mathbb{E}[X]\\ \\ &\frac{d}{dt}\log M_X(t) = \frac{M_X^{\prime}(t)}{M_X(t)} = \Box \Leftrightarrow M_X^{\prime}(t) = \Box\cdot M_X(t)\\ \\ &M_X(t)^{\prime\prime}|{t=0} = \mathbb{E}[\exp{(tX)}^{\prime\prime}]|{t=0} =\mathbb{E}[X\exp{(tX)}^\prime]|{t=0}= \mathbb{E}[X^2\exp(tX)]|_{t=0}= \mathbb{E}[X^2] \end{aligned} $$

尤度関数

$$ \begin{aligned} L(\theta) &= \prod_{i = 1}^{n}f(x_i\mid\theta)\\ \\ LL(\theta) &= \log L(\theta) = \sum_{i = 1}^{n}\log f(x_i\mid\theta)\\ \\ S(\theta) &= \frac{\partial}{\partial\theta}LL(\theta)\\ \\ S(\hat\theta) &= \frac{\partial}{\partial\theta}LL(\hat\theta) = 0\\ \\ \mathbb{E}[S(\theta)] &= 0\\ \\ I(\theta)&= \mathbb{V}[S(\theta)]\\ &=\mathbb{E}[(S(\theta) - \mathbb{E}[S(\theta)])^2]\\ &=\mathbb{E}[S(\theta)^2]\\ \\ &=\mathbb{E}\left[-\frac{\partial^2}{\partial\theta^2}\log L(\theta)\right] \end{aligned} $$

離散確率分布

ベルヌーイ分布

$$ X \sim \mathbf{Ber}(p)\\ p(x) = p^x(1-p)^{1-x} $$

$$ \begin{aligned} M_X(t) &= \mathbb{E}[\exp{(tX)}]\\ &= \sum_{x = 0}^{1}\exp{(tx)}p^x(1-p)^{1-x}\\ &=(1-p)+\exp{(t)}p \\ \end{aligned} $$

$$ \begin{aligned} \mathbb{E}[X] &= M_X(t)^{\prime}|{t=0} \\ &= \frac{d}{dt}(1-p)+\exp{(t)}p|{t=0} \\ &= \exp(t)p|_{t=0} \\ &= p \end{aligned} $$

$$ \begin{aligned} \mathbb{E}[X^2] &= M_X^{\prime\prime}(t)|{t=0} \\ &= \frac{d}{dt}\exp(t)p|{t=0}\\ &= \exp(t)p|_{t=0}\\ &= p \end{aligned} $$

$$ \begin{aligned} \mathbb{V}[X] &= \mathbb{E}[X^2] - \mathbb{E}[X]^2\\ &= p - p^2 \\ &= p(1-p) \end{aligned} $$

$$ \begin{aligned} \log L(p) &= \log \prod_{i = 1}^n p(x_i)\\ &= \log \prod_{i = 1}^n p^x(1-p)^{1-x}\\ &= \sum_{i = 1}^n(x\ln p + (1 - x)\ln(1-p)) \end{aligned} $$

$$ \begin{aligned} &S(p) = \frac{d}{dp}\log L(p) = 0\\ &\Leftrightarrow\\ &\frac{d}{dp}\sum_{i = 1}^n(x\ln p + (1 - x)\ln(1-p))\\ &=\sum_{i = 1}^n\left(\frac{x}{p} - \frac{(1 - x)}{(1-p)}\right)\\ &=\sum_{i = 1}^n\left(\frac{x(1-p) - p(1-x)}{p(1-p)}\right)\\ &=\sum_{i = 1}^n\left(\frac{x- p}{p(1-p)}\right) \\ &=\frac{\sum x- np}{p(1-p)} = 0\\ &\Leftrightarrow\\ &\sum x- np = 0\\ &\Leftrightarrow\\ &\sum x = np\\ &\Leftrightarrow\\ &p = \frac{1}{n}\sum x \end{aligned} $$