Cross-Modal Terrains: Navigating Sonic Space through Haptic Feedback

12/02/2020
by   Gabriella Isaac, et al.
0

This paper explores the idea of using virtual textural terrains as a means of generating haptic profiles for force-feedback controllers. This approach breaks from the para-digm established within audio-haptic research over the last few decades where physical models within virtual environments are designed to transduce gesture into sonic output. We outline a method for generating multimodal terrains using basis functions, which are rendered into monochromatic visual representations for inspection. This visual terrain is traversed using a haptic controller, the NovInt Falcon, which in turn receives force information based on the grayscale value of its location in this virtual space. As the image is traversed by a performer the levels of resistance vary, and the image is realized as a physical terrain. We discuss the potential of this approach to afford engaging musical experiences for both the performer and the audience as iterated through numerous performances.

READ FULL TEXT

page 1

page 4

research
04/26/2017

Deep Cross-Modal Audio-Visual Generation

Cross-modal audio-visual perception has been a long-lasting topic in psy...
research
06/21/2021

Attention-based cross-modal fusion for audio-visual voice activity detection in musical video streams

Many previous audio-visual voice-related works focus on speech, ignoring...
research
10/04/2022

HapticLever: Kinematic Force Feedback using a 3D Pantograph

HapticLever is a new kinematic approach for VR haptics which uses a 3D p...
research
10/10/2016

Matching of Images with Rotation Transformation Based on the Virtual Electromagnetic Interaction

A novel approach of image matching for rotating transformation is presen...
research
07/28/2022

Beyond the Blue Sky of Multimodal Interaction: A Centennial Vision of Interplanetary Virtual Spaces in Turn-based Metaverse

Human habitation across multiple planets requires communication and soci...
research
08/03/2023

Learning beyond sensations: how dreams organize neuronal representations

Semantic representations in higher sensory cortices form the basis for r...

Please sign up or login with your details

Forgot password? Click here to reset