Statistical physics of disordered systems, information, and inference




Lectures: Johannes Berg
Exercises: Stephan Kleinbölting and Uli Michel


This lecture course gives a unified perspective on disordered systems, information theory, and the statistical theory of inference. These fields share a common methodology, namely the entropy of typical and of rare configurations. Applications will be taken from communication, machine learning, and statistical modelling. Topics include

• introduction to probability and information theory
• information theory and the foundations of statistical physics, the principle of maximum entropy
• typical and rare events, the source coding theorem
• disordered systems: introduction to spin glass theory
• statistical inference: the basics of data science
• message passing
• outlook: inverse problems, the inverse Ising problem

The course is part of the area of specialization "Statistical and biological physics" of the Master in physics. The course is self-contained; prior knowledge of advanced statistical physics (at the level of the Masters course) is useful but not required.

Times and places

Due to the ongoing outbreak of the coronavirus, this course will be given online, at least for the first few weeks. I will record the lectures and upload them to the university's ILIAS system. There will be a message board to ask questions, and most likely exercise classes will be via video conference calls. I am sure we will also find further, new, and effective modes of teaching together, so the crisis will at least also have some positive aspects.

If you are going to take this class for credit (this is most of you), sign in for the class on KLIPS2.0. This should automatically give you access to the relevant ILIAS page. Failing that, connect to this ILIAS page and sign up for membership (which I will then confirm). Use the rider "Sprache" (language) on the top right to set the language to English. Let me know via email if there are technical problems or if you have questions. Exercise classes and question & answer sessions will most likely be held via zoom. (You will need to set up a free account, and download a free app. On windows there is also a browser-based version.)

The problem sheets will also be available on Ilias. The lecture course will start on April 20th.

Literature

Cover and Thomas, Elements of Information Theory (Wiley)
MacKay, Information theory, Inference and Learning Algorithms (CUP)
Mézard and Montanari, Information, Physics, and Computation (OUP)


Picture: The amorphous structure of SiO2 (a glass), courtesy of User:127.0.0.l, licensed under GNU Free Documentation license 1.2 or later