edATLAS: An Efficient Disambiguation Algorithm for Texting in Languages with Abugida Scripts

01/05/2021
by   Sourav Ghosh, et al.
0

Abugida refers to a phonogram writing system where each syllable is represented using a single consonant or typographic ligature, along with a default vowel or optional diacritic(s) to denote other vowels. However, texting in these languages has some unique challenges in spite of the advent of devices with soft keyboard supporting custom key layouts. The number of characters in these languages is large enough to require characters to be spread over multiple views in the layout. Having to switch between views many times to type a single word hinders the natural thought process. This prevents popular usage of native keyboard layouts. On the other hand, supporting romanized scripts (native words transcribed using Latin characters) with language model based suggestions is also set back by the lack of uniform romanization rules. To this end, we propose a disambiguation algorithm and showcase its usefulness in two novel mutually non-exclusive input methods for languages natively using the abugida writing system: (a) disambiguation of ambiguous input for abugida scripts, and (b) disambiguation of word variants in romanized scripts. We benchmark these approaches using public datasets, and show an improvement in typing speed by 19.49 and Thai, respectively, using Ambiguous Input, owing to the human ease of locating keys combined with the efficiency of our inference method. Our Word Variant Disambiguation (WDA) maps valid variants of romanized words, previously treated as Out-of-Vocab, to a vocabulary of 100k words with high accuracy, leading to an increase in Error Correction F1 score by 10.03 Prediction (NWP) by 62.50

READ FULL TEXT

page 1

page 3

research
09/29/2021

Context based Roman-Urdu to Urdu Script Transliteration System

Now a day computer is necessary for human being and it is very useful in...
research
03/26/2018

Unsupervised Separation of Transliterable and Native Words for Malayalam

Differentiating intrinsic language words from transliterable words is a ...
research
03/26/2022

Joint Transformer/RNN Architecture for Gesture Typing in Indic Languages

Gesture typing is a method of typing words on a touch-based keyboard by ...
research
02/12/2020

Unsupervised Separation of Native and Loanwords for Malayalam and Telugu

Quite often, words from one language are adopted within a different lang...
research
10/01/2020

Joint Persian Word Segmentation Correction and Zero-Width Non-Joiner Recognition Using BERT

Words are properly segmented in the Persian writing system; in practice,...
research
09/30/2015

The "handedness" of language: Directional symmetry breaking of sign usage in words

Using large written corpora for many different scripts, we show that the...
research
11/28/2017

Treatment of Unicode canoncal decomposition among operating systems

This article shows how the text characters that have multiple representa...

Please sign up or login with your details

Forgot password? Click here to reset