DeepAI AI Chat
Log In Sign Up

Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems

10/16/2021
by   Zhongli Li, et al.
Tencent
0

Math Word Problem (MWP) solving needs to discover the quantitative relationships over natural language narratives. Recent work shows that existing models memorize procedures from context and rely on shallow heuristics to solve MWPs. In this paper, we look at this issue and argue that the cause is a lack of overall understanding of MWP patterns. We first investigate how a neural network understands patterns only from semantics, and observe that, if the prototype equations are the same, most problems get closer representations and those representations apart from them or close to other prototypes tend to produce wrong solutions. Inspired by it, we propose a contrastive learning approach, where the neural network perceives the divergence of patterns. We collect contrastive examples by converting the prototype equation into a tree and seeking similar tree structures. The solving model is trained with an auxiliary objective on the collected examples, resulting in the representations of problems with similar prototypes being pulled closer. We conduct experiments on the Chinese dataset Math23k and the English dataset MathQA. Our method greatly improves the performance in monolingual and multilingual settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/29/2022

Textual Enhanced Contrastive Learning for Solving Math Word Problems

Solving math word problems is the task that analyses the relation of qua...
02/14/2022

Learning Weakly-Supervised Contrastive Representations

We argue that a form of the valuable information provided by the auxilia...
09/11/2020

Solving Math Word Problems by Scoring Equations with Recursive Neural Networks

Solving math word problems is a cornerstone task in assessing language u...
06/29/2016

A Distributional Semantics Approach to Implicit Language Learning

In the present paper we show that distributional information is particul...
11/26/2022

Supervised Contrastive Prototype Learning: Augmentation Free Robust Neural Network

Transformations in the input space of Deep Neural Networks (DNN) lead to...