A Human-Computer Duet System for Music Performance

by   Yuen-Jen Lin, et al.

Virtual musicians have become a remarkable phenomenon in the contemporary multimedia arts. However, most of the virtual musicians nowadays have not been endowed with abilities to create their own behaviors, or to perform music with human musicians. In this paper, we firstly create a virtual violinist, who can collaborate with a human pianist to perform chamber music automatically without any intervention. The system incorporates the techniques from various fields, including real-time music tracking, pose estimation, and body movement generation. In our system, the virtual musician's behavior is generated based on the given music audio alone, and such a system results in a low-cost, efficient and scalable way to produce human and virtual musicians' co-performance. The proposed system has been validated in public concerts. Objective quality assessment approaches and possible ways to systematically improve the system are also discussed.



There are no comments yet.


page 1

page 4

page 6


Enabling Embodied Analogies in Intelligent Music Systems

The present methodology is aimed at cross-modal machine learning and use...

Learning Embodied Semantics via Music and Dance Semiotic Correlations

Music semantics is embodied, in the sense that meaning is biologically m...

Aktuelle Entwicklungen in der Automatischen Musikverfolgung

In this paper we present current trends in real-time music tracking (a.k...

Listen to Dance: Music-driven choreography generation using Autoregressive Encoder-Decoder Network

Automatic choreography generation is a challenging task because it often...

Formal models of Structure Building in Music, Language and Animal Songs

Human language, music and a variety of animal vocalisations constitute w...

An ICT based Solution for Virtual Garment Fitting for Online Market Place: A Review of Related Literature

In this paper, we describe various technologies that are being used in v...

The Digital Synaptic Neural Substrate: A New Approach to Computational Creativity

We introduce a new artificial intelligence (AI) approach called, the 'Di...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.