Heterogeneous Line Graph Transformer for Math Word Problems

by   Zijian Hu, et al.

This paper describes the design and implementation of a new machine learning model for online learning systems. We aim at improving the intelligent level of the systems by enabling an automated math word problem solver which can support a wide range of functions such as homework correction, difficulty estimation, and priority recommendation. We originally planned to employ existing models but realized that they processed a math word problem as a sequence or a homogeneous graph of tokens. Relationships between the multiple types of tokens such as entity, unit, rate, and number were ignored. We decided to design and implement a novel model to use such relational data to bridge the information gap between human-readable language and machine-understandable logical form. We propose a heterogeneous line graph transformer (HLGT) model that constructs a heterogeneous line graph via semantic role labeling on math word problems and then perform node representation learning aware of edge types. We add numerical comparison as an auxiliary task to improve model training for real-world use. Experimental results show that the proposed model achieves a better performance than existing models and suggest that it is still far below human performance. Information utilization and knowledge discovery is continuously needed to improve the online learning systems.


page 1

page 2

page 3

page 4


Graph Propagation Transformer for Graph Representation Learning

This paper presents a novel transformer architecture for graph represent...

HINormer: Representation Learning On Heterogeneous Information Networks with Graph Transformer

Recent studies have highlighted the limitations of message-passing based...

RecipeRec: A Heterogeneous Graph Learning Model for Recipe Recommendation

Recipe recommendation systems play an essential role in helping people d...

LUKE-Graph: A Transformer-based Approach with Gated Relational Graph Attention for Cloze-style Reading Comprehension

Incorporating prior knowledge can improve existing pre-training models i...

An Attention-based Graph Neural Network for Heterogeneous Structural Learning

In this paper, we focus on graph representation learning of heterogeneou...

Label Enhanced Event Detection with Heterogeneous Graph Attention Networks

Event Detection (ED) aims to recognize instances of specified types of e...

Please sign up or login with your details

Forgot password? Click here to reset