DeepAI AI Chat
Log In Sign Up

Learning Multi-modal Similarity

08/30/2010
by   Brian McFee, et al.
University of California, San Diego
0

In many applications involving multi-media data, the definition of similarity between items is integral to several key tasks, e.g., nearest-neighbor retrieval, classification, and recommendation. Data in such regimes typically exhibits multiple modalities, such as acoustic and visual content of video. Integrating such heterogeneous data to form a holistic similarity space is therefore a key challenge to be overcome in many real-world applications. We present a novel multiple kernel learning technique for integrating heterogeneous data into a single, unified similarity space. Our algorithm learns an optimal ensemble of kernel transfor- mations which conform to measurements of human perceptual similarity, as expressed by relative comparisons. To cope with the ubiquitous problems of subjectivity and inconsistency in multi- media similarity, we develop graph-based techniques to filter similarity measurements, resulting in a simplified and robust training procedure.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/27/2014

A Topic Model Approach to Multi-Modal Similarity

Calculating similarities between objects defined by many heterogeneous d...
03/03/2023

Multi-modal Multi-kernel Graph Learning for Autism Prediction and Biomarker Discovery

Multi-modal integration and classification based on graph learning is am...
09/18/2017

Multi-modal analysis of genetically-related subjects using SIFT descriptors in brain MRI

So far, fingerprinting studies have focused on identifying features from...
06/05/2018

JTAV: Jointly Learning Social Media Content Representation by Fusing Textual, Acoustic, and Visual Features

Learning social media content is the basis of many real-world applicatio...
04/28/2020

Heterogeneous Representation Learning: A Review

The real-world data usually exhibits heterogeneous properties such as mo...
12/05/2022

Stars: Tera-Scale Graph Building for Clustering and Graph Learning

A fundamental procedure in the analysis of massive datasets is the const...
10/08/2017

Exploration of Heterogeneous Data Using Robust Similarity

Heterogeneous data pose serious challenges to data analysis tasks, inclu...