Measuring Mathematical Problem Solving With the MATH Dataset

03/05/2021
by   Dan Hendrycks, et al.
0

Many intellectual endeavors require mathematical problem solving, but this skill remains beyond the capabilities of computers. To measure this ability in machine learning models, we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems. Each problem in MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics. Even though we are able to increase accuracy on MATH, our results show that accuracy remains relatively low, even with enormous Transformer models. Moreover, we find that simply increasing budgets and model parameter counts will be impractical for achieving strong mathematical reasoning if scaling trends continue. While scaling Transformers is automatically solving most other text-based tasks, scaling is not currently solving MATH. To have more traction on mathematical problem solving we will likely need new algorithmic advancements from the broader research community.

READ FULL TEXT
research
03/24/2021

NaturalProofs: Mathematical Theorem Proving in Natural Language

Understanding and creating mathematics using natural mathematical langua...
research
12/20/2022

A Survey of Deep Learning for Mathematical Reasoning

Mathematical reasoning is a fundamental aspect of human intelligence and...
research
10/07/2021

Pretrained Language Models are Symbolic Mathematics Solvers too!

Solving symbolic mathematics has always been of in the arena of human in...
research
06/02/2023

An Empirical Study on Challenging Math Problem Solving with GPT-4

Employing Large Language Models (LLMs) to address mathematical problems ...
research
05/31/2022

Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers

From the latter half of the last decade, there has been a growing intere...
research
05/24/2023

Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective

Recent studies have discovered that Chain-of-Thought prompting (CoT) can...
research
12/02/2012

Problem Solving and Computational Thinking in a Learning Environment

Computational thinking is a new problem soling method named for its exte...

Please sign up or login with your details

Forgot password? Click here to reset