Sparse Deep Belief Net Model for Visual Area V2

by Honglak Lee, Chaintanya Ekanadham and Andrew Y. Ng

In fact the algorithm is quite simple. The sparsity is enforced by a norm regularizer (the conditional expectation of the hidden states, don't know why it works). This will not modify the CD learning process much. We simply add another update of the parameters, which is deterministic and very easy to compute.

The paper argues that the sparse RBM and DBN could learn Gabor-like filters from the natural images. And the corner-oriented filters are believed to be the V2 area in human cortex.