AR(1)参数估计

Problem

设 \(AR(1)\) 模型为 \(y_t=\phi_0+\phi_1y_{t-1}+\epsilon_t\)，\(\epsilon_t\mathop{\sim}\limits^{i.i.d.} N(0,\sigma_\epsilon^2)\) ，求解参数 \(\phi_0,\phi_1,\sigma_\epsilon^2\)。

最小二乘法

假设观测样本为 \(\{y_1,y_2,\ldots,y_T\}\)，则可通过最小二乘求解参数 \(\phi_0\) 和 \(\phi_1\)。

\(\mathop{min}\limits_{\phi_0,\;\phi_1}\;L(\phi_0,\phi_1)=\sum\limits_{t=2}^T\left(y_t-\phi_0-\phi_1y_{t-1}\right)^2\)

分别对 \(\phi_0\) 和 \(\phi_1\) 求偏导，可得：

\(\begin{cases} \dfrac{\partial L}{\partial\phi_0}=-2\sum\limits_{t=2}^T\left(y_t-\phi_0-\phi_1 y_{t-1}\right)=0 \\ \dfrac{\partial L}{\partial\phi_1}=-2\sum\limits_{t=2}^Ty_{t-1}\left(y_t-\phi_0-\phi_1y_{t-1} \right)=0 \end{cases}\)

求解可得。

若令 \(\phi=(\phi_0,\phi_1)'\)，\(X_t=(1,Y_{t-1})^\prime\)，则有

\(Y_t=X_t^\prime\phi+\epsilon_t,\;t=2,\ldots,T\) 且 \(\hat{\phi}=\mathop{argmin}\limits_{\phi}\;E(Y_t-X_t^\prime\phi)^2\)

求偏导可得：

\(\dfrac{\partial}{\partial\phi}E(Y_t-X_t^\prime\phi)^2=-2E(Y_t-X_t^\prime\phi)\dfrac{\partial}{\partial\phi}X_t^\prime\phi=-2E(Y_t-X_t^\prime\phi)X_t=-2EX_t(Y_t-X_t^\prime\phi)=-2EX_tY_t+2EX_tX_t^\prime\phi=0\)

进而可得：

\(\hat{\phi}=\left(\begin{matrix}\hat{\phi_0}\\\hat{\phi_1}\end{matrix}\right)=(EX_tX_t^\prime)^{-1}EX_tY_t=\left(\begin{matrix}1&EY_{t-1}\\EY_{t-1}&EY_{t-1}^2 \end{matrix}\right)^{-1}\left(\begin{matrix}EY_t\\EY_{t-1}Y_t\end{matrix}\right)=\left(\begin{matrix}1&\dfrac{1}{T-1}\sum\limits_{t=2}^Ty_{t-1}\\\dfrac{1}{T-1}\sum\limits_{t=2}^Ty_{t-1}&\dfrac{1}{T-1}\sum\limits_{t=2}^Ty_{t-1}^2 \end{matrix}\right)^{-1}\left(\begin{matrix}\dfrac{1}{T-1}\sum\limits_{t=2}^Ty_t\\\dfrac{1}{T-1}\sum\limits_{t=2}^Ty_{t-1}y_t \end{matrix}\right)\)

极大似然估计

假设 \(AR(1)\) 平稳，即 \(|\phi_1|<1\).

设 \(\mu=Ey_t\)，进而有：\(\mu=\phi_0+\phi_1\mu\)，故\(\mu=\dfrac{\phi_0}{1-\phi_1}\)；设 \(\sigma^2=Var(y_t)\)，进而有：\(\sigma^2=\phi_1^2\sigma^2+\sigma_\epsilon^2\)，故 \(\sigma^2=\dfrac{\sigma_\epsilon^2}{1-\phi_1^2}\).

因此，我们可以得到 \(y_1\) 的分布：\(y_1\sim N(\dfrac{\phi_0}{1-\phi_1},\dfrac{\sigma_\epsilon^2}{1-\phi_1^2})\).

为什么 \(y_1\) 服从正态分布?

\(\begin{align*} y_t&=\phi_0+\phi_1y_{t-1}+\epsilon_t \\ &=\phi_0+\phi_1\phi_0+\phi_1^2y_{t-2}+\phi_1\epsilon_{t-1}+\epsilon_t \\ &=\phi_0+\phi_1\phi_0+\phi_1^2\phi_0+\phi_1^3y_{t-3}+\phi_1^2\epsilon_{t-2}+\phi_1\epsilon_{t-1}+\epsilon_t \\ &=\cdots \\ &=\sum\limits_{i=0}^{n-1}\phi_1^i\phi_0+\phi_1^ny_{t-n}+\sum\limits_{i=0}^{n-1}\phi_1^i\epsilon_{t-i} \end{align*}\)

\(\begin{align*} &\because\; |\phi_1|<1, \\ &\therefore\; \phi_1^n\rightarrow0\;(n\rightarrow\infty) \\ &又\because\; \epsilon_t\sim N(0,\sigma_\epsilon^2) \\ &\therefore\; y_t服从正态分布 \end{align*}\)

另一方面，由于 \(E(y_t|y_{t-1})=\phi_0+\phi_1y_{t-1}\)，\(Var(y_t|y_{t-1})=\sigma_\epsilon^2\)，我们可以得到：\(y_t|y_{t-1}\sim N(\phi_0+\phi_1y_{t-1},\sigma_\epsilon^2)\).

为什么 \(y_t|y_{t-1}\) 服从正态分布？
在 \(y_t=\phi_0+\phi_1y_{t-1}+\epsilon_t\) 中，\(y_{t-1}\) 给定，\(\epsilon_t\) 服从正态分布，故 \(y_t|y_{t-1}\) 服从正态分布。

于是，设 \(\theta=(\phi_0,\phi_1,\sigma_\epsilon^2)^\prime\)，可以得到似然函数为：

\(\begin{align*} L(\theta)&=f_\theta(y_1,\dots,y_T) \\ &=\prod\limits_{t=2}^T f_\theta(y_t|y_1,\dots,y_{t-1})f_\theta(y_1) \\ &=\prod\limits_{t=2}^Tf_\theta(y_t|y_{t-1})f_\theta(y_1) \\ &=\prod\limits_{t=2}^T\dfrac{1}{\sqrt{2\pi\sigma_\epsilon^2}}exp\left(-\dfrac{(y_t-\phi_0-\phi_1y_{t-1})^2}{2\sigma_\epsilon^2}\right)\dfrac{1}{\sqrt{2\pi\sigma_\epsilon^2/(1-\phi_1^2)}}exp\left(-\dfrac{(y_t-\phi_0/(1-\phi_1))^2}{\sigma_\epsilon^2/(1-\phi_1^2)} \right) \end{align*}\)

\(\begin{align*} l(\theta)&=lnL(\theta) \\ &=-\dfrac{T-1}{2}ln2\pi\sigma_\epsilon^2-\sum\limits_{t=2}^T\dfrac{(y_t-\phi_0-\phi_1y_{t-1})^2}{2\sigma_\epsilon^2}-\dfrac{T-1}{2}ln\dfrac{2\pi\sigma_\epsilon^2}{1-\phi_1^2}-\sum\limits_{t=2}^T \dfrac{(y_t-\phi_0/(1-\phi_1))^2}{\sigma_\epsilon^2/(1-\phi_1^2)} \end{align*}\)

对 \(-l(\theta)\) 用梯度下降可求解参数。

条件极大似然估计

给定 \(y_1\)，那么条件似然函数为：

\(\begin{align*} L(\theta)&=f_\theta(y_2,\dots,y_T|y_1) \\ &=\prod\limits_{t=2}^Tf_\theta(y_t|y_1,\dots,y_{t-1}) \\ &=\prod\limits_{t=2}^Tf_\theta(y_t|y_{t-1}) \\ &=\prod\limits_{t=2}^T\dfrac{1}{\sqrt{2\pi\sigma_\epsilon^2}}exp\left(-\dfrac{(y_t-\phi_0-\phi_1y_{t-1})^2}{2\sigma_\epsilon^2}\right) \end{align*}\)

\(\begin{align*} l(\theta)&=lnL(\theta) \\ &=-\dfrac{1}{2}\sum\limits_{t=2}^Tln2\pi\sigma_\epsilon^2-\sum\limits_{t=2}^T\dfrac{(y_t-\phi_0-\phi_1y_{t-1})^2}{2\sigma_\epsilon^2} \end{align*}\)

同样对 \(-l(\theta)\) 用梯度下降可求解参数。