MHVAE

1. MHVAE模型

1.1 MHVAE简介

VAE (Variational Autoencoder) 假设隐变量只有一层，HVAE (Hierarchical VAE)假设隐变量有 \(T\) 层，MHVAE (Markovian HVAE) 在 HVAE 的基础上假设层与层之间满足马尔可夫性 (Markov Property) ，即当前的状态只与前一个状态有关。根据 Markov Property 容易得到等式 (1) (2)：

\[\begin{align} \underbrace{p\left (\boldsymbol{x},\boldsymbol{z}_{1:T}\right )}_{\text{Joint Distribution}} =& \underbrace{p\left (\boldsymbol{z}_T\right )}_{\text{Prior}}p_{\theta}\left (\boldsymbol{x}\mid \boldsymbol{z}_1\right ) \prod_{t=2}^{T} \underbrace{p_{\theta}\left (\boldsymbol{z}_{t-1}\mid \boldsymbol{z}_t\right )}_{\text{Decoder}} \\ \underbrace{q_{\phi}\left (z_{1:T}\mid x\right )}_{\text{Posterior Distribution}} =& q_{\phi}\left (z_1\mid x\right )\prod_{t=2}^{T} \underbrace{q_{\phi}\left (z_t\mid z_{t-1}\right )}_{\text{Encoder}} \end{align}\]

1.2 如何推导MHVAE的ELBo (Evidence Lower Bound)

\[\begin{align} \log{\underbrace{p\left (x\right )}_{\text{Evidence}}} &= \log{\int{p\left (x, z_{1:T}\right )\text{d}z_{1:T}}} \\&= \log{\int{\frac{p\left (x,z_{1:T}\right )q_{\phi}\left (z_{1:T}\mid x\right )}{q_{\phi}\left (z_{1:T}\mid x\right )}\text{d}z_{1:T}}} \\&= \log{\mathbb{E}_{q_{\phi}\left (z_{1:T}\mid x\right )}\left [\frac{p\left (x,z_{1:T}\right )}{q_{\phi}\left (z_{1:T}\mid x\right )}\right ]} \\&\geq \underbrace{\mathbb{E}_{q_{\phi}\left (z_{1:T}\mid x\right )}{\left [ \log{\frac{p\left (x,z_{1:T}\right )} {q_{\phi}\left (z_{1:T}\mid x\right )}}\right ]}}_{\text{ELBo of HVAE}} \quad \text{(apply Jensen's Inequality)} \end{align}\]

我们将等式 (1) (2) 应用到等式 (6)上，可得：

\[\begin{align} &\underbrace{\mathbb{E}_{q_{\phi}\left (z_{1:T}\mid x\right )}{\left [ \log{\frac{p\left (x,z_{1:T}\right )} {q_{\phi}\left (z_{1:T}\mid x\right )}}\right ]}}_{\text{ELBo of HVAE}} \\ = & \underbrace{\mathbb{E}_{q_{\phi}\left (z_{1:T}\mid x\right )}{\left [ \log{\frac{\overbrace{p\left (z_T\right )}^{\text{Prior}}p_{\theta}\left (x\mid z_1\right ) \prod_{t=2}^{T} \overbrace{p_{\theta}\left (z_{t-1}\mid z_t\right )}^{\text{Decoder}}}{q_{\phi}\left (z_1\mid x\right )\prod_{t=2}^{T} \underbrace{q_{\phi}\left (z_t\mid z_{t-1}\right )}_{\text{Encoder}}}} \right]}}_{\text{ELBo of MHVAE}} \end{align}\]