A deep Q-learning method for optimizing visual search strategies in backgrounds of dynamic noise

01/28/2022
by   Weimin Zhou, et al.
0

Humans process visual information with varying resolution (foveated visual system) and explore images by orienting through eye movements the high-resolution fovea to points of interest. The Bayesian ideal searcher (IS) that employs complete knowledge of task-relevant information optimizes eye movement strategy and achieves the optimal search performance. The IS can be employed as an important tool to evaluate the optimality of human eye movements, and potentially provide guidance to improve human observer visual search strategies. Najemnik and Geisler (2005) derived an IS for backgrounds of spatial 1/f noise. The corresponding template responses follow Gaussian distributions and the optimal search strategy can be analytically determined. However, the computation of the IS can be intractable when considering more realistic and complex backgrounds such as medical images. Modern reinforcement learning methods, successfully applied to obtain optimal policy for a variety of tasks, do not require complete knowledge of the background generating functions and can be potentially applied to anatomical backgrounds. An important first step is to validate the optimality of the reinforcement learning method. In this study, we investigate the ability of a reinforcement learning method that employs Q-network to approximate the IS. We demonstrate that the search strategy corresponding to the Q-network is consistent with the IS search strategy. The findings show the potential of the reinforcement learning with Q-network approach to estimate optimal eye movement planning with real anatomical backgrounds.

READ FULL TEXT
research
05/10/2020

Optimal control of eye-movements during visual search

We study the problem of optimal oculomotor control during the execution ...
research
08/12/2020

An ocular biomechanics environment for reinforcement learning

Reinforcement learning has been applied to human movement through physio...
research
10/25/2020

CLRGaze: Contrastive Learning of Representations for Eye Movement Signals

Eye movements are rich but ambiguous biosignals that usually require a m...
research
01/28/2021

Symmetric equilibrium of multi-agent reinforcement learning in repeated prisoner's dilemma

We investigate the repeated prisoner's dilemma game where both players a...
research
07/29/2019

Task Classification Model for Visual Fixation, Exploration, and Search

Yarbus' claim to decode the observer's task from eye movements has recei...
research
11/29/2014

Multiple Instance Reinforcement Learning for Efficient Weakly-Supervised Detection in Images

State-of-the-art visual recognition and detection systems increasingly r...
research
03/26/2022

Aggressive Quadrotor Flight Using Curiosity-Driven Reinforcement Learning

The ability to perform aggressive movements, which are called aggressive...

Please sign up or login with your details

Forgot password? Click here to reset