Advances and Challenges in Multimodal Machine Learning 2nd Edition

Dear Colleagues,

The emerging field of multimodal machine learning has experienced much progress in the past few years; however, several core challenges remain. These challenges are mainly around learning how to represent and summarise multimodal data (representation); translating (mapping) data from one modality to another (translation); identifying direct relations between elements from different modalities (alignment); joining or fusing information from two or more modalities to perform a prediction task (fusion); and transferring knowledge between modalities, their representations, and predictive models (co-learning).

Within the field of information retrieval, the large and continually growing volume of data has given rise to the need for retrieval solutions that can deal with the search process of using one modality as a query to retrieve related information in another modality, known as cross-modal retrieval. In recent years, cross-modal retrieval methods have attracted considerable attention due to the learning capabilities of deep learning methods; however, most of these methods assume that data examples in different modalities are fully paired, when in reality, these data are not often paired.

Furthermore, the continually growing volume of data has given rise to the additional challenge of developing lifelong learning models than can continue to efficiently learn new volumes of data. Lifelong learning remains a challenge for machine learning models and most research on the topic focuses on classification tasks. There is a need to focus on lifelong learning for information retrieval and to propose methods for dealing with the continuous growth of information that can lead to catastrophic forgetting or interference. This limitation represents a major drawback for models that typically learn representations from batches of training data, when in reality, information becomes incrementally available over time. The challenge of lifelong learning increases when dealing with cross-modal learning.

We request contributions that present techniques that will contribute to addressing the multimodal machine learning challenges, and we strongly encourage contributions that propose advances in the field of continual lifelong learning for multimodal machine learning applications. 

Dr. Georgina Cosma
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Journal of Imaging is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• neural information retrieval
• multi-modal and cross-modal information retrieval
• relevance feedback and query expansion in multimodal retrieval
• sensitivity analysis of images or multi-modal data
• visual semantic embedding for information retrieval and other tasks
• continual lifelong learning for information retrieval
• temporal modelling of multi-modal data