The Recurrent Temporal Restricted Boltzmann Machine

by Hya Sutskever, Geoffrey Hinton and Graham Taylor

This paper is succedent to the previous motion caption paper, which does not include much details about the model (at least I don't know anything about that dynamic biases). Anyway, I find more details in this paper. This dynamic bias is actually a cross term of h_t and h_t-1. So given h_t-1, it is functioning as the bias term.

The model proposed in this paper is a bit strange. I will read something about recurrent networks first. From the point of view of optimization, it uses BP instead of CD learning (exact vs stochastic approximation).

Anyway, I will return to this paper later. It is really weird to make a temporal model with RBM compared with HMM, CRF.