Information theory and statistical physics

Lectures: Johannes Berg
Exercises: Stephan Kleinbölting

This lecture course gives an introduction to information theory and statistical inference from the perspective of statistical physics. Topics include

• introduction to probability and information theory
• information theory and the foundations of statistical physics, the principle of maximum entropy
• Maxwell's demon and Szilard's engine
• typical and rare events, the source coding theorem
• statistical inference
• inverse problems, the inverse Ising problem
• information processing in biology: sequence analysis, molecular structure prediction, regulation of gene expression

The course is part of the area of specialization "Statistical and biological physics" of the Master in physics. The course is self-contained; prior knowledge of advanced statistical physics
(at the level of the Masters course) is useful but not required.

Times and places

Lectures: Wednesday 16-17:30 st, lecture room 1, new theoretical physics building
Thursday 12-13:30 st, lecture room 1, new theoretical physics building
Exercises: Wednesday 16-17:30 st (alternating with lectures), lecture room 1, new theoretical physics building
Beginning: 19.4, 16:00,lecture room 1, new theoretical physics building


Cover and Thomas, Elements of Information Theory (Wiley)
MacKay, Information theory, Inference and Learning Algorithms (CUP)
Barber, Bayesian Reasoning and Machine Learning, (CUP)
Mézard and Montanari, Information, Physics, and Computation (OUP)

Picture: Structure of a motor protein, courtesy of