- DL divergence
$$DL = – \sum Q(z|x) log(\frac{P(z)}{Q(z|x)}) = \sum(\mu^2+\sigma^2-1-log(\simga^2))$$ -
Loss
The original paper is maximizing the loss/lower bound -
Variance non-negative issue
- softplus
- assume output is log(var)
-
Reparameterization technique
z = mu + sigma * normal(0, 1)