twitter_langid
A hierarchical character-word neural network for language identification
view repo
Social media messages' brevity and unconventional spelling pose a challenge to language identification. We introduce a hierarchical model that learns character and contextualized word-level representations for language identification. Our method performs well against strong base- lines, and can also reveal code-switching.
READ FULL TEXTA hierarchical character-word neural network for language identification