Multimedia Retrieval 2018-2019
incl. slides + literature
Multimedia retrieval is about the search for and delivery of multimedia documents, such as images, video, audio, biometrics and the combination of these.
This course adopts a signal processing perspective on multimedia and data in general. This perspective allows us to see common aspects in multimedia (e.g., note that an image is a 2D signal) and explore the rich set of techniques, algorithms, and tools signal processing offers us. Where only one or a few signal types will be explored in the assignments and project, the vast majority of techniques, algorithms, and tools can used for other signal types as well. Moreover, this perspective aids the fusion of distinct signals or data types, in a non-conventional way. So, on of the course's main goals is to become aware of the similarities and differences between several types of media (e.g., video, audio, and biosignals) and how to handle this.
In real-world multimedia retrieval and in data mining in general, it becomes increasingly more important to adopt an interdisciplinary stance. This course prepares you on this by discussing core signal processing, modeling via machine learning, and psychological aspects.
Multimedia Retrieval is dominated by emperical computer science, not theoretical computer science. A theoretical foundation is valued; but, its value has to be shown in practice. For example, theoretical complexity can be low; however, emperical complexity can turn out much higher due to a specific hardware architecture. An image processing algorithms can claim an objective optimal retrieval results; however, if the person who assesses the results does not think so, it results will not be evaluated as such.
As a 2nd year MSc-course, the course has the meta goal to prepare its students on the MSc-graduation phase.
This course has objectives to train students in:
Compulsory (exam): The course's handbook: Eidenberger, H. (2012). Handbook of Multimedia Information Retrieval. Vienna, Austria: atpress. ISBN: 978-3-848-22283-4. The book can be ordered at A-Eskwadraat. Alternatively, order the book via Amazon.com or bol.com.
Optional (exam): Oppenheim, A.V. and Schafer, R.W. (2014). Discrete-time signal processing (3rd ed., Pearson New International Edition). Prentice-Hall Signal Processing Series. Harlow, Essex, England: Prentice Education Limited, Inc. ISBN: 978-1-292-02572-8.
Note. You can also use the 2nd ed. from 1999, which is freely available for download online.
Compulsory (project): Olsen, A. (2012). The Tobii I-VT Fixation Filter - Algorithm description. Technical Report. Danderyd, Sweden: Tobii Technology.
Requirement: To pass the course, the weighted average of your assignments, exam, and project has to be at least 5.5 and the grades of the assignments, exam, and the project each have to be at least 4.0.
Retake: Only allowed if the grade of the assignments, exam, and the project is at least a 4.0.
There will be two assignments in this course, one on signal processing basics, and one on eye-tracking, also see the Schedule below. The first homework will be handed out to students on 11th September, and the deadline is 25th September, 23:59 (local time). The second homework will be handed out to students on 18th September, and the deadline is 05th October, 23:59 (local time).
For assignment 1, you do not need any additional material. You do not even need computing machinery. However, for Assignment 2, you will need 6 data files, available in one zip-file: data.zip
The project is described in a separate file: MR project description (version 1.0). Additional information will be provided during the lectures and practicals.
Project data (in three parts):
Manual of the eye-tracker that was used to gather the data: BeGaze 2 Manual. Can be very usefull in case you don't understand some output (e.g., the units of the timestamps) or are curious about some background.
Within this course you will give two short presentations, in line with the two phases of the graduation project. These presentations will be on respectively Thursday 11 October 2018 and Thursday 01 November 2018, see also the Schedule. For both presentations, per group, your total presentation time is 15 minutes and consists of:
Moreover, it is strongly encouraged to make a semi-professional planning and decompose the project into workpackages (incl. dependencies), with group members made responsible for this. Such project schedules can be designed in many ways; for example, as a Gantt chart.
With respect to the project content, we refer to the MultiMedia Retrieval 2018-2019 project description for more information; see Project, under Course elements.
For the second phase, it is important that you can explain:
All issues mentioned above and presented in the two presentations, also need to be grounded in the project report.
The lecture schedule will be frequently updated during the course.
|01||36||Tuesday 04 September||13.15 - 17.00||BBG-001||no lecture|
|01||36||Thursday 06 September||09.00 - 10.45||no lecture|
|02||37||Tuesday 11 September||13.15 - 17.00||Organization & Introduction + Explanation project
Signal processing (1): Basics (1)
Practical. Assignment 1 is provided.
|Chapter 1, Sections 1 and 2
Chapter 2, Sections 1, 2, and 3
Chapter 3 until the bioinformation example on p. 56
|02||37||Thursday 13 September||09.00 - 10.45||Signal processing (2): Basics (2)||Lecture 3|
|03||38||Tuesday 18 September||13.15 - 17.00||Signal processing (3): Basics (3)
Practical. Assignment 2 is provided.
|03||38||Thursday 20 September||09.00 - 10.45||Signal processing (4): The eye-tracking signal (1)||Lecture 5|
|04||39||Tuesday 25 September||13.15 - 17.00||Signal processing (5): The eye-tracking signal (2)
Practical / Exam preparation
|Deadline Assignment 1|
|04||39||Thursday 27 September||09.00 - 10.45||Signal processing (6): ECG Practical / Exam preparation||Practical 2||The accompanying MATLAB-code and ECG-data (1 ZIP-file).|
|05||40||Tuesday 02 October||13.15 - 17.00||Distance, Evaluation of Machine and Human, and
Practical / Exam preparation
|Chapters 7, 10, 20, and 28; recommended: Appendix B
also consider Stanford's material on Image Processing;
in particular, Handout 8, as shown during the lecture
|05||40||Thursday 04 October||09.00 - 10.45||Practical 4: Feedback Assignment 1 / Exam preparation||Practical 4|
|05||40||Friday 05 October||Deadline Assignment 2|
|06||41||Tuesday 09 October||13.15 - 15.00||exam|
|15.15 - 17.00||free|
|06||41||Thursday 11 October||09.00 - 10.45||Presentations research plan|
|07||42||Tuesday 16 October||13.15 - 17.00||Writing a scientific report
|Deadline Project WP1|
|07||42||Thursday 18 October||09.00 - 10.45||Project practical.
Individual feedback on Assignment 2
|08||43||Tuesday 23 October||13.15 - 16.15||Project practical.
Feedback per group on WP1
|08||43||Thursday 25 October||09.00 - 10.45||Project practical|
|Deadline Project WP2|
|09||44||Tuesday 30 October||13.15 - 17.00||Project practical.
Feedback per group on WP2
|09||44||Thursday 01 November||09.00 - 10.45||Final project pitch|
|09||44||Tuesday 06 November||13.15 - 15.30||Project practical.
Deadline WP3 and final report
|Last update on
Copyright © 2018 by Egon L. van den Broek