See Your Heart: Psychological states Interpretation through Visual Creations

02/11/2023
by   Likun Yang, et al.
0

In psychoanalysis, generating interpretations to one's psychological state through visual creations is facing significant demands. The two main tasks of existing studies in the field of computer vision, sentiment/emotion classification and affective captioning, can hardly satisfy the requirement of psychological interpreting. To meet the demands for psychoanalysis, we introduce a challenging task, Visual Emotion Interpretation Task (VEIT). VEIT requires AI to generate reasonable interpretations of creator's psychological state through visual creations. To support the task, we present a multimodal dataset termed SpyIn (Sandplay Interpretation Dataset), which is psychological theory supported and professional annotated. Dataset analysis illustrates that SpyIn is not only able to support VEIT, but also more challenging compared with other captioning datasets. Building on SpyIn, we conduct experiments of several image captioning method, and propose a visual-semantic combined model which obtains a SOTA result on SpyIn. The results indicate that VEIT is a more challenging task requiring scene graph information and psychological knowledge. Our work also show a promise for AI to analyze and explain inner world of humanity through visual creations.

READ FULL TEXT
research
08/05/2023

A Comprehensive Analysis of Real-World Image Captioning and Scene Identification

Image captioning is a computer vision task that involves generating natu...
research
06/12/2019

Vispi: Automatic Visual Perception and Interpretation of Chest X-rays

Medical imaging contains the essential information for rendering diagnos...
research
09/05/2023

NICE 2023 Zero-shot Image Captioning Challenge

In this report, we introduce NICE project[<https://nice.lgresearch.ai/>]...
research
04/13/2023

A-CAP: Anticipation Captioning with Commonsense Knowledge

Humans possess the capacity to reason about the future based on a sparse...
research
08/21/2023

Explore and Tell: Embodied Visual Captioning in 3D Environments

While current visual captioning models have achieved impressive performa...

Please sign up or login with your details

Forgot password? Click here to reset