Morphological Analysis of Japanese Hiragana Sentences using the BI-LSTM CRF Model

01/10/2022
by   Jun Izutsu, et al.
0

This study proposes a method to develop neural models of the morphological analyzer for Japanese Hiragana sentences using the Bi-LSTM CRF model. Morphological analysis is a technique that divides text data into words and assigns information such as parts of speech. This technique plays an essential role in downstream applications in Japanese natural language processing systems because the Japanese language does not have word delimiters between words. Hiragana is a type of Japanese phonogramic characters, which is used for texts for children or people who cannot read Chinese characters. Morphological analysis of Hiragana sentences is more difficult than that of ordinary Japanese sentences because there is less information for dividing. For morphological analysis of Hiragana sentences, we demonstrated the effectiveness of fine-tuning using a model based on ordinary Japanese text and examined the influence of training data on texts of various genres.

READ FULL TEXT

Authors

page 5

01/09/2013

Syntactic Analysis Based on Morphological Characteristic Features of the Romanian Language

This paper refers to the syntactic analysis of phrases in Romanian, as a...
04/26/2017

From Characters to Words to in Between: Do We Capture Morphology?

Words can be represented by composing the representations of subword uni...
12/18/2021

Morpheme Boundary Detection Grammatical Feature Prediction for Gujarati : Dataset Model

Developing Natural Language Processing resources for a low resource lang...
02/13/2020

Comparison of Turkish Word Representations Trained on Different Morphological Forms

Increased popularity of different text representations has also brought ...
10/24/2020

A Benchmark Corpus and Neural Approach for Sanskrit Derivative Nouns Analysis

This paper presents first benchmark corpus of Sanskrit Pratyaya (suffix)...
07/05/2018

Chinese Lexical Analysis with Deep Bi-GRU-CRF Network

Lexical analysis is believed to be a crucial step towards natural langua...
08/26/2019

uniblock: Scoring and Filtering Corpus with Unicode Block Information

The preprocessing pipelines in Natural Language Processing usually invol...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.