Neural Compound-Word (Sandhi) Generation and Splitting in Sanskrit Language

10/24/2020
by   Sushant Dave, et al.
0

This paper describes neural network based approaches to the process of the formation and splitting of word-compounding, respectively known as the Sandhi and Vichchhed, in Sanskrit language. Sandhi is an important idea essential to morphological analysis of Sanskrit texts. Sandhi leads to word transformations at word boundaries. The rules of Sandhi formation are well defined but complex, sometimes optional and in some cases, require knowledge about the nature of the words being compounded. Sandhi split or Vichchhed is an even more difficult task given its non uniqueness and context dependence. In this work, we propose the route of formulating the problem as a sequence to sequence prediction task, using modern deep learning techniques. Being the first fully data driven technique, we demonstrate that our model has an accuracy better than the existing methods on multiple standard datasets, despite not using any additional lexical or morphological resources. The code is being made available at https://github.com/IITD-DataScience/Sandhi_Prakarana

READ FULL TEXT

Authors

page 5

01/01/2018

Sanskrit Sandhi Splitting using seq2(seq)^2

In Sanskrit, small words (morphemes) are combined through a morphophonol...
10/24/2020

A Benchmark Corpus and Neural Approach for Sanskrit Derivative Nouns Analysis

This paper presents first benchmark corpus of Sanskrit Pratyaya (suffix)...
03/23/2020

Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings

The number of senses of a given word, or polysemy, is a very subjective ...
02/17/2018

Building a Word Segmenter for Sanskrit Overnight

There is an abundance of digitised texts available in Sanskrit. However,...
04/01/2021

Do RNN States Encode Abstract Phonological Processes?

Sequence-to-sequence models have delivered impressive results in word fo...
04/07/2018

Simple Models for Word Formation in English Slang

We propose generative models for three types of extra-grammatical word f...
06/27/2019

Morphological Irregularity Correlates with Frequency

We present a study of morphological irregularity. Following recent work,...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.