NWO-VIDI project MUSIVA

Modelling musical similarity over time

through the variation principle

 Principal investigator: Dr. Anja Volk

 

  Department of Computer and Information Sciences, Utrecht University



 


News

Special Issue Music similarity: Concepts, cognition and computation, Guest Editors: Anja Volk, Elaine Chew, Elizabeth Hellmuth Margulis, Christina Anagnostopoulou, Journal of New Music Research, to appear September 2016. Follow up to Lorentz workshop on Music similarity.

Marcelo Rodriguez Lopez successfully defended his PhD Thesis Automatic Melody Segmentation on June 20, 2016.

Peter Boot  received the runner up award of the Graduate School of Natural Sciences for his master thesis Using Discovered and Annotated Patterns as Compression Method for determining Similarity between Folk Songs. Download the corresponding journal article here.

Project Description:

Introduction

The aim of this project is to deliver cognition-based computational models on music similarity that ground in the variation principle employed in classical, folk and popular music. The project integrates knowledge and methods from Music Information Retrieval, Musicology and Cognitive Science.

The assessment of similarity is fundamental for cognitive processes. Cognitive Science provides formal models for the theoretical framework and empirical measurement of similarity, demonstrating its crucial influence on our conception of the world. However, no comprehensive theory exists on how listeners use similarity to predict, categorize or appreciate music. This is a major problem in the rapidly growing discipline of Music Information Retrieval (MIR). MIR researches methods that allow users to retrieve music that is similar to musical queries representing their needs via the Internet. Developing content-based search methods for music faces the challenge of relating features of the music to listeners' experience of similarity. Most methods developed in MIR extract low-level features from the music, severely limiting the cognitive plausibility of search results and hence impairing the usefulness of current retrieval systems.

This project investigates the fundamental principle of variation in music studied in Musicology and Cognitive Science as a means to establish similarity. The project researches computational modelling of music similarity in the symbolic domain (i.e. using perception-related notation, such as MIDI or **kern) based on the variation of structural elements (such as motives, rhythms and chord sequences). Specifically, it takes into account the interaction between global and local features of the music.

 

Significance

Modelling similarity based on the variation principle is urgently needed to establish a general theory of similarity in music. This is crucial for the successful design of MIR systems and opens up new research lines in music cognition research. Integrating the concept of variation (Musicology), formal models of similarity (Cognitive Science) and computational modelling (Information Retrieval) into a comprehensive approach to music similarity delivers important aspects for a general theory on similarity across different domains.

 

Music Information Retrieval: The largest part of modelling music similarity in Music Information Retrieval is currently in the audio domain. The extracted low-level features allow the comparison of broad categories of music, such as genres, but are less successful for establishing similarity for finer-grained data, such as for similar songs within the same genre. High-level features extracted in the symbolic domain from formats such as MIDI or **kern are closer to the way people perceive music. This project addresses the high-level processing in establishing similarity in music which delivers an essential requirement for building cognitively plausible search algorithms in Music Information Retrieval. The lack of established concepts of similarity in music causes the lack of suitable ground truths on which algorithms can be tested (Downie et al, 2009). For the three selected music styles, test cases based on the variation principle are established as a ground truth for the computational modelling.

 

Musicology: The musicological discourse on the variation principle is based on small numbers of selected musical examples and has not led to a general concept of variation. The computational modelling allows to formalize the concept and to explicitly test it on a large collection of music.

 

Cognitive Science: While research in music cognition has strongly focussed on the experienced listener of Western classical music (Peretz, 2006), research on music similarity contributes an excellent topic on basic music skills. Similarity is used as a default method to reason about a domain, even if we do not have specific knowledge about it (Goldstone & Son, 2005). Thus, understanding music similarity demonstrates that accessing music is not reserved to the highly trained specialist. Moreover, models of music similarity contribute an underrepresented domain to similarity research in Cognitive Science and hence contribute new aspects to the search for general principles of similarity across different domains.

Approach

Musicology provides analytic descriptions of music closely related to the phenomenon of similarity that are based on the variation principle. The term variation principle refers to the variation of musical patterns such that listeners are able to relate different components of a piece of music, or entire pieces, to each other and hence experience similarity relations. Although no formalized concept of the variation principle has been developed, studies on techniques to vary musical patterns such that they establish similarity in classical, folk and popular music strongly suggest that the variation principle is a universal principle in music. Recent studies in Cognitive Science show that similarity relations based on the variation of characteristic elements as described in Musicology are recognized both by experts and novices. This project takes into account different ways of interaction between global and local features of the music as a basis for modelling the concrete manifestations of the variation principle studied in Musicology and Cognitive Science. The data-rich approach of computational modelling within Music Information Retrieval allows to corroborate the generality of the variation principle in establishing similarity.

Research Team

The research team includes:

principal investigator: Dr. Anja Volk

Dr. Frans Wiering

Postdoc: Dr. Bas de Haas  

PhD student: Marcelo Rodriguez-Lopez

PhD student: Hendrik Vincent Koops

Publications

Boot, P., Volk, A., & de Haas, W.B. (2016). Evaluating the Role of Repeated Patterns in Folk Song Classification and Compression, Journal of New Music Research.

De Haas, W.B. & Volk, A. (2016). Meter Detection in Symbolic Music Using Inner Metric Analysis, Proceedings of the 17th Conference of the International Society for Music Information Retrieval (ISMIR), New York, USA, August 7-11.

Koops, H. V., de Haas, W.B., Bountouridis, D., & Volk, A. (2016). Integration and Quality Assessment of Heterogeneous Chord Sequences using Data Fusion. Proceedings of the 17th Conference of the International Society for Music Information Retrieval (ISMIR), New York, USA, August 7-11.

Janssen, B., van Kranenburg, P. & Volk, A. (2015). A comparison of symbolic similarity measures for finding occurrences of melodic segments. In: Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, (pp. 659-665). Malaga, Spain: ISMIR press.

Koops, H. V., Volk, A. & de Haas, W.B. (2015). Corpus-Based Rhythmic Pattern Analysis of Ragtime Syncopation. In  Proceedings of the 16th International Society for Music Information Retrieval Conference, ISMIR(pp. 483-489) (7 p.). Malaga, Spain: ISMIR press.

Rodriguez Lopez, M.E. & Volk, A. (2015). Location Constraints for Repetition-Based Segmentation of Melodies. Proceedings of the 5th International Conference in Mathematics and Computation in Music (MCM) (pp. 73-84) (12 p.). Springer.

Rodriguez Lopez, M.E., Bountouridis, D. & Volk, A. (2015). Novel Music Segmentation Interface and the Jazz Tune Collection. Proceedings of the 5th International Workshop on Folk Music Analysis (pp. 99-105) (7 p.). Paris: CNRS.

Rodriguez Lopez, M.E. & Volk, A. (2015). On the Evaluation of Automatic Segment Boundary Detection. Music, Mind, Embodiment - Proceedings of the 11th International Symposium on Computer Music Multidisciplinary Research (CMMR) (pp. 234-246) (14 p.). Plymouth: LMA publications.

Rodriguez Lopez, M.E. & Volk, A. (2015). Selective Acquisition Techniques for Enculturation-Based Melodic Phrase Segmentation. Proceedings of the 16th Conference of the International Society for Music Information Retrieval (ISMIR) (pp. 218-224) (7 p.). Malaga: ISMIR press.

Rodriguez Lopez, M.E., de Haas, Bas & Volk, A. (2014). Comparing repetition-based melody segmentation models. Proceedings of the 9th Conference on Interdisciplinary Musicology (CIM14) (pp. 143-148) (6 p.). Berlin: SIMPK and ICCMR.

Rodriguez Lopez, M.E., Volk, A. & Bountouridis, D. (2014). Multi-strategy Segmentation of Melodies. Proceedings of the 15th Conference of the International Society for Music Information Retrieval (ISMIR 2014) (pp. 207-212) (6 p.). Taipei: ISMIR press.

Rodriguez Lopez, M.E. & Volk, A. (2014). Symbolic Segmentation: A Corpus-Based Analysis of Melodic Phrases. Sound, Music, and Motion - Post Proceedings of the 10th international symposium, CMMR 2013 (pp. 548-557) (10 p.). Cham: Springer.

Janssen, B., De Haas, W.B., Volk, A., & Van Kranenburg, P. (2013). Discovering repeated patterns in music: state of knowledge, challenges, perspectives, Proceedings of the 10th International Symposium on Computer Music Modeling and Retrieval (CMMR 2013), CNRS - Laboratoire de Mecanique et d'Acoustique, Marseille, France.

Van Kranenburg, P.,  Volk, A., & Wiering, F. (2013). A Comparison between Global and Local Features for Computational Classification of Folk Song Melodies, Journal of New Music Research, Vol. 42 (1), 1-18.

De Haas, W.B., Volk, A. & Wiering, F. (2013). Structural segmentation of music based on repeated harmonies. Proceedings of the International Symposium on Multimedia (pp. 255-258) (4 p.). Anaheim: IEEE.

Rodriguez-Lopez, M.E.  & Volk, A. (2013). Symbolic Segmentation: A Corpus-Based Analysis of Melodic Phrases, Proceedings of the 10th International Symposium on Computer Music Modeling and Retrieval (CMMR 2013), CNRS - Laboratoire de Mecanique et d'Acoustique, Marseille, France.
   
Volk, A. & De Haas, W.B. (2013). A Corpus-based study on Ragtime syncopation. Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), 2013, Curitiba, Brazil.

Volk, A. & van Kranenburg, P. (2012), Melodic similarity among folk songs: An annotation study on similarity-based categorization in music, Musicae Scientiae, 16(3), 317-339.

Volk, A. & Honingh, A., (2012). Mathematical and computational approaches to music: challenges in an interdisciplinary enterprise. Journal of Mathematics and Music, Vol. 6, No. 2, pp. 73-81.

Rodriguez Lopez, M.E. & Volk, A. (2012). Melodic Segmentation Using the Jensen-Shannon Divergence, Proceedings of the 11th International Conference on Machine Learning and Applications, Boca Raton, USA.
   
Volk, A., De Haas, W.B.  &  Van Kranenburg, P. (2012). Towards Modeling Variation in Music as a Foundation of Similarity, Proceedings of the 12th International  Conference on Music Perception and Cognition, Thessaloniki, Greece.

Van Kranenburg, P., Volk, A., & Wiering, F. (2012). On Identifying Folk Song Melodies Employing Recurring Motifs, Proceedings of the 12th International  Conference on Music Perception and Cognition, Thessaloniki, Greece.

Biro, D., Van Kranenburg, P., Ness, S. G., Tzanetakis, G., & Volk, A. (2012). Stability and Variation in Cadence Formulas in Oral and Semi-Oral Chant Traditions - a Computational Approach, Proceedings of the 12th International  Conference on Music Perception and Cognition, Thessaloniki, Greece, 2012.

Volk, A., Wiering, F. & Van Kranenburg, P. (2011). Unfolding the Potential of Computational Musicology. Proceedings of the13th International Conference on Informatics and Semiotics in Organisations (ICISO), Leeuwarden, the Netherlands, 137-144, 2011.

Van Kranenburg, P., Wiering, F., &  Volk, A. (2011).  On Deconstructing the musicological concept of tune family for computational modeling. Proceedings of the Supporting Digital Humanities conference, Kopenhagen.

Contact for more detailed project description:

Anja Volk
Department of Information and Computing Sciences
Utrecht University
email and further contact details