Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

02/07/2021
by   Akinari Onishi, et al.
0

Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs. In the present study, we propose a rapid serial multimodal presentation (RSMP) BCI that incorporates artificial facial images and artificial voice stimuli. To clarify the effect of audiovisual stimuli on the RSMP BCI, scrambled images and masked sounds were applied instead of visual and auditory stimuli, respectively. Our findings indicated that the audiovisual stimuli improved the performance of the RSMP BCI, and that the late positive potential (LPP) at Pz contributed to classification accuracy. Taken together, these findings may aid in the development of better gaze-independent BCI systems.

READ FULL TEXT

page 3

page 7

research
11/14/2017

Reconstruction of the External Stimuli from Brain Signals

Despite the rapid advances in Brain-computer Interfacing (BCI) and conti...
research
11/03/2020

Evoking Places from Spaces. The application of multimodal narrative techniques in the creation of "U Modified"

Multimodal diegetic narrative tools, as applied in multimedia arts pract...
research
12/03/2018

Evaluation of flashing stimuli shape and colour heterogeneity using a P300 brain-computer interface speller

Objective: Previous works using a visual P300-based speller have reporte...
research
09/17/2018

Development of spatial suppression surrounding the focus of visual attention

The capacity to filter out irrelevant information from our environment i...
research
11/18/2022

Adversarial Stimuli: Attacking Brain-Computer Interfaces via Perturbed Sensory Events

Machine learning models are known to be vulnerable to adversarial pertur...
research
06/06/2023

Computation with Sequences in the Brain

Even as machine learning exceeds human-level performance on many applica...
research
11/25/2018

Nonlinear Dynamics of Binocular Rivalry: A Comparative Study

When our eyes are presented with the same image, the brain processes it ...

Please sign up or login with your details

Forgot password? Click here to reset