Department of Information and Computing Sciences

Departement Informatica Onderwijs
Bachelor Informatica Informatiekunde Kunstmatige intelligentie Master Computing Science Game&Media Technology Artifical Intelligence Human Computer Interaction Business Informatics

Onderwijs Informatica en Informatiekunde

Vak-informatie Informatica en Informatiekunde

Advanced machine learning

Course code:INFOMAML
Credits:7.5 ECTS
Period:period 2 (week 46 through 5, i.e., 9-11-2020 through 5-2-2021; retake week 16)
Participants:up till now 79 subscriptions
Schedule:Official schedule representation can be found in MyTimetable
lecture          Thijs van Ommen
Shihan Wang
tutorial          Olusanmi Hundogan
group 1        Samira Shirzadehhajimahmood

This course treats two advanced topics in machine learning: causal inference (the study of cause-effect relations), and reinforcement learning (learning to interact with an environment).

Learning goals: After completing the course, the student

  • understands the description of causal relations in terms of directed graphs and structural equation models;
  • can explain the implications of causal assumptions;
  • is able to test these implications using observational and interventional data;
  • can implement machine learning methods in such a way that causal information is taken into account;
  • knows the main concepts and algorithms of reinforcement learning;
  • can describe the role played by function approximation methods (such as neural networks) in reinforcement learning;
  • is able to explain the additional challenges that arise with off-policy methods.

The following knowledge will be assumed in this course:

  • solid proficiency in mathematics, in particular probability theory (e.g. ability to understand and manipulate formulas involving conditional probabilities and expectations), linear algebra, basic calculus
  • programming skill in Python
  • understanding of basic machine learning theory and methods, for example from the bachelor course Machine Learning (KI3V15001)

  • Judea Pearl, Madelyn Glymour, Nicholas P. Jewell. Causal Inference in Statistics: A Primer. Wiley, 2016.
  • Richard S. Sutton, Andrew G. Barto. Reinforcement Learning: An Introduction (second edition). MIT Press, 2018. (pdf available from authors' website:
  • additional material that will be made available online
Course form:Lectures; tutorials / practical sessions
Exam form:Written exam (60% of grade); programming assignments with written reports (40%). Theoretical exercises (ungraded) will be discussed during tutorial sessions as preparation for the exam.
Minimum effort to qualify for 2nd chance exam:Om aan de aanvullende toets te mogen meedoen moet de oorspronkelijke uitslag minstens 4 zijn.

Modern machine learning methods have achieved spectacular results on various tasks. Yet there are pitfalls and limitations that can't be overcome simply by increasing the amounts of data and computing power. For example, standard methods assume that the data are drawn from a single, unchanging probability distribution. The two main topics that we cover in this course both deal with situations where that is not the case.

The first topic, causal inference, is the subfield of machine learning that studies causes and effects: if we make a change to one random variable in a system, for which other variables does the distribution change? An understanding of these cause-and-effect relations allows us to predict the results of a change in the environment. We will also look at the problem of learning these relations from data.

Second, reinforcement learning is about the design of agents that can learn to interact with an unknown environment. Recent advances in supervised learning (such as deep learning) can be built on by reinforcement learning methods. This brings with it a unique set of challenges that we will cover in this course.