Introducing the diagrammatic mode

01/30/2020
by   Tuomo Hiippala, et al.
0

In this article, we propose a multimodal perspective to diagrammatic representations by sketching a description of what may be tentatively termed the diagrammatic mode. We consider diagrammatic representations in the light of contemporary multimodality theory and explicate what enables diagrammatic representations to integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations.

READ FULL TEXT
research
03/08/2021

Semiotically-grounded distant viewing of diagrams: insights from two multimodal corpora

In this article, we bring together theories of multimodal communication ...
research
09/10/2019

Multimodal Embeddings from Language Models

Word embeddings such as ELMo have recently been shown to model word sema...
research
06/02/2021

Normalization for multimodal type theory

We consider the conversion problem for multimodal type theory (MTT) by c...
research
11/09/2017

Learning Multi-Modal Word Representation Grounded in Visual Context

Representing the semantics of words is a long-standing problem for the n...
research
04/04/2017

From Modal to Multimodal Ambiguities: a Classification Approach

This paper deals with classifying ambiguities for Multimodal Languages. ...
research
12/05/2019

Classifying Diagrams and Their Parts using Graph Neural Networks: A Comparison of Crowd-Sourced and Expert Annotations

This article compares two multimodal resources that consist of diagrams ...
research
09/15/2021

Comparing Text Representations: A Theory-Driven Approach

Much of the progress in contemporary NLP has come from learning represen...

Please sign up or login with your details

Forgot password? Click here to reset