Clinical Text Prediction with Numerically Grounded Conditional Language Models

10/20/2016
by   Georgios P. Spithourakis, et al.
0

Assisted text input techniques can save time and effort and improve text quality. In this paper, we investigate how grounded and conditional extensions to standard neural language models can bring improvements in the tasks of word prediction and completion. These extensions incorporate a structured knowledge base and numerical values from the text into the context used to predict the next word. Our automated evaluation on a clinical dataset shows extended models significantly outperform standard models. Our best system uses both conditioning and grounding, because of their orthogonal benefits. For word prediction with a list of 5 suggestions, it improves recall from 25.03 71.28 44.81 qualitative investigation of how models with lower perplexity occasionally fare better at the tasks. We found that at test time numbers have more influence on the document level than on individual word probabilities.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2016

Numerically Grounded Language Models for Semantic Error Correction

Semantic error detection and correction is an important task for applica...
research
12/01/2022

Language models and brain alignment: beyond word-level semantics and prediction

Pretrained language models that have been trained to predict the next wo...
research
02/25/2016

Automated Word Prediction in Bangla Language Using Stochastic Language Models

Word completion and word prediction are two important phenomena in typin...
research
07/17/2017

A Simple Language Model based on PMI Matrix Approximations

In this study, we introduce a new approach for learning language models ...
research
08/17/2021

A Game Interface to Study Semantic Grounding in Text-Based Models

Can language models learn grounded representations from text distributio...
research
05/24/2023

Large Language Models are Few-Shot Health Learners

Large language models (LLMs) can capture rich representations of concept...
research
05/21/2018

Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers

Numeracy is the ability to understand and work with numbers. It is a nec...

Please sign up or login with your details

Forgot password? Click here to reset