uu logo     

Multimedia Retrieval 2018-2019


News General Course elements Schedule
incl. slides + literature



Multimedia retrieval is about the search for and delivery of multimedia documents, such as images, video, audio, biometrics and the combination of these.

This course adopts a signal processing perspective on multimedia and data in general. This perspective allows us to see common aspects in multimedia (e.g., note that an image is a 2D signal) and explore the rich set of techniques, algorithms, and tools signal processing offers us. Where only one or a few signal types will be explored in the assignments and project, the vast majority of techniques, algorithms, and tools can used for other signal types as well. Moreover, this perspective aids the fusion of distinct signals or data types, in a non-conventional way. So, on of the course's main goals is to become aware of the similarities and differences between several types of media (e.g., video, audio, and biosignals) and how to handle this.

In real-world multimedia retrieval and in data mining in general, it becomes increasingly more important to adopt an interdisciplinary stance. This course prepares you on this by discussing core signal processing, modeling via machine learning, and psychological aspects.

Multimedia Retrieval is dominated by emperical computer science, not theoretical computer science. A theoretical foundation is valued; but, its value has to be shown in practice. For example, theoretical complexity can be low; however, emperical complexity can turn out much higher due to a specific hardware architecture. An image processing algorithms can claim an objective optimal retrieval results; however, if the person who assesses the results does not think so, it results will not be evaluated as such.

As a 2nd year MSc-course, the course has the meta goal to prepare its students on the MSc-graduation phase.

Basic info

Course goals

This course has objectives to train students in:

  1. General concepts of multimedia retrieval, including common and deviating characteristics between media (e.g., text, video, and biosignals), distance, and other complex concepts;
  2. Signal processing; and
  3. Conducting scientific research, as preparation for you MSc-project: i) define a research question, ii) execute, evaluate, and validate a research study, and iii) write a scientific article on it.
As such, the course aims to be complementary to other courses, in particular data-analyse en retrieval, pattern recognition and computer vision.


Compulsory (exam): The course's handbook: Eidenberger, H. (2012). Handbook of Multimedia Information Retrieval. Vienna, Austria: atpress. ISBN: 978-3-848-22283-4. The book can be ordered at A-Eskwadraat. Alternatively, order the book via Amazon.com or bol.com.

Optional (exam): Oppenheim, A.V. and Schafer, R.W. (2014). Discrete-time signal processing (3rd ed., Pearson New International Edition). Prentice-Hall Signal Processing Series. Harlow, Essex, England: Prentice Education Limited, Inc. ISBN: 978-1-292-02572-8.
Note. You can also use the 2nd ed. from 1999, which is freely available for download online.

Compulsory (project): Olsen, A. (2012). The Tobii I-VT Fixation Filter - Algorithm description. Technical Report. Danderyd, Sweden: Tobii Technology.

General academic guidelines


The course consists of lectures, practicals/work groups, assignments, and a project, including a presentation and a pitch. Our experience learned us that genuine active participation is needed to pass the course. Calculation grade:

Requirement: To pass the course, the weighted average of your assignments, exam, and project has to be at least 5.5 and the grades of the assignments, exam, and the project each have to be at least 4.0.

Retake: Only allowed if the grade of the assignments, exam, and the project is at least a 4.0.

Course elements: Assignments, exam, and project

The course consists of lectures, practicals/work groups, assignments, and a project, including a presentation and a pitch. The course's assignments, exam, and project determine your course grade, as is explained above. Below follows a short description of each of these course elements.


There will be two assignments in this course, one on signal processing basics, and one on eye-tracking, also see the Schedule below. The first homework will be handed out to students on 11th September, and the deadline is 25th September, 23:59 (local time). The second homework will be handed out to students on 18th September, and the deadline is 05th October, 23:59 (local time).

For assignment 1, you do not need any additional material. You do not even need computing machinery. However, for Assignment 2, you will need 6 data files, available in one zip-file: data.zip



The project is described in a separate file: MR project description (version 1.0). Additional information will be provided during the lectures and practicals.

Project data (in three parts):

Manual of the eye-tracker that was used to gather the data: BeGaze 2 Manual. Can be very usefull in case you don't understand some output (e.g., the units of the timestamps) or are curious about some background.

Presentation instructions

Within this course you will give two short presentations, in line with the two phases of the graduation project. These presentations will be on respectively Thursday 11 October 2018 and Thursday 01 November 2018, see also the Schedule. For both presentations, per group, your total presentation time is 15 minutes and consists of:

Your graduation project is devided in two parts, which can be roughly denoted as
  1. preparation (practical + theory) and
  2. execution + thesis.
For the first phase, it is important that you can explain:
  1. what,
  2. why, and
  3. how
you are going to do what you planned.

Moreover, it is strongly encouraged to make a semi-professional planning and decompose the project into workpackages (incl. dependencies), with group members made responsible for this. Such project schedules can be designed in many ways; for example, as a Gantt chart.

With respect to the project content, we refer to the MultiMedia Retrieval 2018-2019 project description for more information; see Project, under Course elements.

For the second phase, it is important that you can explain:

  1. what the results are;
  2. how the results can be interpreted;
  3. what has been learned;
  4. what went good, what went wrong, and what to do next; and
  5. what the core of the project is (reflected in both the abstract and the title).

All issues mentioned above and presented in the two presentations, also need to be grounded in the project report.

Lecture schedule and slides

The lecture schedule will be frequently updated during the course.

#lec. #week date time room topic slides literature
01 36 Tuesday 04 September 13.15 - 17.00 BBG-001 no lecture
01 36 Thursday 06 September 09.00 - 10.45 no lecture
02 37 Tuesday 11 September 13.15 - 17.00 Organization & Introduction + Explanation project
Signal processing (1): Basics (1)
Practical. Assignment 1 is provided.
Lecture 1
Lecture 2
Chapter 1, Sections 1 and 2
Chapter 2, Sections 1, 2, and 3
Chapter 3 until the bioinformation example on p. 56
Chapter 4
Chapter 12
Chapter 13
02 37 Thursday 13 September 09.00 - 10.45 Signal processing (2): Basics (2) Lecture 3
03 38 Tuesday 18 September 13.15 - 17.00 Signal processing (3): Basics (3)
Assignment 2 is provided.
Lecture 4
03 38 Thursday 20 September 09.00 - 10.45 Signal processing (4): The eye-tracking signal (1) Lecture 5
04 39 Tuesday 25 September 13.15 - 17.00 Signal processing (5): The eye-tracking signal (2)
Practical / Exam preparation
Lecture 6
Practical 1
Deadline Assignment 1
04 39 Thursday 27 September 09.00 - 10.45 Signal processing (6): ECG Practical / Exam preparation Practical 2 The accompanying MATLAB-code and ECG-data (1 ZIP-file).
05 40 Tuesday 02 October 13.15 - 17.00 Distance, Evaluation of Machine and Human, and
Practical / Exam preparation
Lecture 7a,
Lecture 7b
Lecture 7c
Practical 3
Chapters 7, 10, 20, and 28; recommended: Appendix B
also consider Stanford's material on Image Processing;
in particular, Handout 8, as shown during the lecture
05 40 Thursday 04 October 09.00 - 10.45 Practical 4: Feedback Assignment 1 / Exam preparation Practical 4
05 40 Friday 05 October Deadline Assignment 2
06 41 Tuesday 09 October 13.15 - 15.00 exam
15.15 - 17.00 free
06 41 Thursday 11 October 09.00 - 10.45 Presentations research plan
07 42 Tuesday 16 October 13.15 - 17.00 Writing a scientific report
Project practical
Lecture 8
Deadline Project WP1
07 42 Thursday 18 October 09.00 - 10.45 Project practical.
Individual feedback on Assignment 2
08 43 Tuesday 23 October 13.15 - 16.15 Project practical.
Feedback per group on WP1
08 43 Thursday 25 October 09.00 - 10.45 Project practical
Deadline Project WP2
09 44 Tuesday 30 October 13.15 - 17.00 Project practical.
Feedback per group on WP2
09 44 Thursday 01 November 09.00 - 10.45 Final project pitch
09 44 Tuesday 06 November 13.15 - 15.30 Project practical.
Deadline WP3 and final report

Valid HTML 4.01!Viewable With Any Browser
Link Checker
Last update on
Copyright © 2018 by Egon L. van den Broek