Special Seminar: Dreaming with active dynamics: A correspondence between Hebbian unlearning and steady states generated by nonequilibrium activity

 Title: “Dreaming with active dynamics: A correspondence between Hebbian unlearning and steady states generated by nonequilibrium activity”

Venue: Rajarshi Bhattacharya Memorial Lecture Hall, Chemical Sciences Building

We previously showed how endowing an associative memory system with active dynamics can improve the system’s information storage and retrieval properties. In this work, we propose an equivalence between the Hebbian Unlearning algorithm (also known as “dreaming”) and modifying dynamics by driving the spins with exponentially correlated noise. Previous attempts to improve the capacity of associative memory models have focused on altering the connections between the neurons. Our method here provides a novel way to increase capacity by changing the dynamics of the neurons instead of altering connection strengths. In this talk I will give a brief overview of associative memory models, Hebbian unlearning and our proposed method. I will use the architecture of restricted Boltzmann machines to show how our analytical argument holds true for cases far away from equilibrium in a purely data-driven fashion. I will also connect out proposed method with how biological neurons might be communicating with one another to improve their information processing capabilities without explicitly tuning their synaptic strengths.