DeepAI AI Chat
Log In Sign Up

Syntactic representation learning for neural network based TTS with syntactic parse tree traversal

by   Changhe Song, et al.

Syntactic structure of a sentence text is correlated with the prosodic structure of the speech that is crucial for improving the prosody and naturalness of a text-to-speech (TTS) system. Nowadays TTS systems usually try to incorporate syntactic structure information with manually designed features based on expert knowledge. In this paper, we propose a syntactic representation learning method based on syntactic parse tree traversal to automatically utilize the syntactic structure information. Two constituent label sequences are linearized through left-first and right-first traversals from constituent parse tree. Syntactic representations are then extracted at word level from each constituent label sequence by a corresponding uni-directional gated recurrent unit (GRU) network. Meanwhile, nuclear-norm maximization loss is introduced to enhance the discriminability and diversity of the embeddings of constituent labels. Upsampled syntactic representations and phoneme embeddings are concatenated to serve as the encoder input of Tacotron2. Experimental results demonstrate the effectiveness of our proposed approach, with mean opinion score (MOS) increasing from 3.70 to 3.82 and ABX preference exceeding by 17 syntactic parse trees, prosodic differences can be clearly perceived from the synthesized speeches.


page 2

page 4


Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS

The end-to-end TTS, which can predict speech directly from a given seque...

Controllable Video Captioning with POS Sequence Guidance Based on Gated Fusion Network

In this paper, we propose to guide the video caption generation with Par...

A Self-supervised Representation Learning of Sentence Structure for Authorship Attribution

Syntactic structure of sentences in a document substantially informs abo...

SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech

The recent progress in non-autoregressive text-to-speech (NAR-TTS) has m...

A Character-level Span-based Model for Mandarin Prosodic Structure Prediction

The accuracy of prosodic structure prediction is crucial to the naturaln...

Word Representations, Tree Models and Syntactic Functions

Word representations induced from models with discrete latent variables ...