Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis

While generative adversarial networks (GANs) based neural text-to-speech (TTS) systems have shown significant improvement in speech synthesis, there is no TTS system to learn synthesize from text sequences with only feedback. Because feedback alone not sufficient train the generator, current models still require reconstruction loss compared ground-truth and generated mel-spectrogram directly. In this paper, we present Multi-SpectroGAN (MSG), which can multi-speaker model by conditioning a self-supervised hidden representation of generator conditional discriminator. This leads better guidance for training. Moreover, also propose style combination (ASC) generalization unseen speaking transcript, latent representations combined embedding multiple mel-spectrograms. Trained ASC feature matching, MSG synthesizes high-diversity controlling mixing individual styles (e.g., duration, pitch, energy). The result shows that high-fidelity mel-spectrogram, has almost same naturalness MOS score as mel-spectrogram.