Towards interpretable pattern extraction from datasets using energy based models

Restricted Boltzmann Machines (RBMs) are shallow neural networks that are very similar to the Ising spin glass models from statistical physics. Despite their simplicity, they are capable of encoding the statistical properties of virtually any complex data set imaginable. RBMs are typically used as generative models, meaning that they are optimised to learn the most important features of a given dataset in an unsupervised way and use them to generate brand new samples that are statistically as close as possible to the original dataset. One can attempt to further explore these trained models using tools from the statistical physics of disordered systems to gain understandable and interpretable scientific insights hidden in the high-dimensional structure of large datasets. The possibilities are enormous and range from deriving effective interaction models for your data to identifying the critical features or components that characterise the label or function of a particular data point. I will describe some of these potential applications in my talk. However, for all these ideas to be successful, proper training of RBMs is essential. I will discuss the different modes of operation of RBMs and the impact of equilibrium and non-equilibrium training on feature extraction performance.

pie de foto: