Tensorized Optical Multimodal Fusion Network

02/17/2023

∙

We propose the first tensorized optical multimodal fusion network architecture with a self-attention mechanism and low-rank tensor fusion. Simulation results show 51.3 × less hardware requirement and 3.7× 10^13 MAC/J energy efficiency.

READ FULL TEXT

Tensorized Optical Multimodal Fusion Network

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

Low Rank Fusion based Transformers for Multimodal Sequences

A novel multimodal fusion network based on a joint coding model for lane line segmentation

Multimodal Dual Attention Memory for Video Story Question Answering

Revealing the Illusion of Joint Multimodal Understanding in VideoQA Models

Incomplete Multimodal Learning for Remote Sensing Data Fusion

Multimodal sensor fusion in the latent representation space

Tensorized Optical Multimodal Fusion Network

Related Research

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

Low Rank Fusion based Transformers for Multimodal Sequences

A novel multimodal fusion network based on a joint coding model for lane line segmentation

Multimodal Dual Attention Memory for Video Story Question Answering

Revealing the Illusion of Joint Multimodal Understanding in VideoQA Models

Incomplete Multimodal Learning for Remote Sensing Data Fusion

Multimodal sensor fusion in the latent representation space