Instructions to implement wavern and tacotron2 together

[mishav78]

here https://soundcloud.com/user-565970875/pocket-article-wavernn-and-tacotron2

The following is implemented Tacotron2 + WaveRN, but I can't find any documentation how I can do this. Please tell me the exact commands I have to type to do this. Thanks in advance. Mike

[Sadam1195]

Hey, you can train them separately once trained you can synthesis audio with WaveRNN and tacotron2 together. You can start from here https://github.com/coqui-ai/TTS/tree/main/recipes/ljspeech
@mishav78

[Sadam1195]

If you want to train a model on your own dataset then you will have to follow the steps described in the link that I have shared above.

But say you don't want to train nor you have own data and just want to generate speech from text input then you can follow the instructions here. https://tts.readthedocs.io/en/latest/inference.html

Now, coming back to the original question. Yes, you can use pretrained models to generate speech but not all TTS models are compatible with all vocoder models. Tacotron2 is a TTS model and WaveRNN is a vocoder model. For comparability TTS and vocoder model need to have similar config/audio parameter settings. You can check the compatibly by comparing audio parameters of released vocoder and TTS models. See the model list here https://github.com/coqui-ai/TTS/releases/tag/v0.0.13

Hope this clears things a bit.

[mishav78]

1. Brother, where are the audio parameters listed?
2. In the link you provided I don't see wavernn at all and Tacotron2 has several English versions. What is the difference between them?

[Sadam1195]

1. Brother, where are the audio parameters listed?

You can find these when you download a model from model list. Let's take this https://github.com/coqui-ai/TTS/releases/download/v0.0.12/tts_models--en--ljspeech--tacotron2-DDC.zip for an example. It has two files, model file and config which was used to train the tacotron2.

2. In the link you provided I don't see wavernn at all and Tacotron2 has several English versions. What is the difference between them?

Tacotron2 has been trained on different languages/datasets, you can choose on of your liking. On WarveRNN link : WaveRNN trained will be much likely incompatible with current code as a lot of new things have been implemented . So most likely you are going to need to train WaveRNN for yourself or you can try HifiGAN or Mel-GAN vocoder as well.

[mishav78]

So what parameters must be the same for Tacotron2 to work with a certain vocoder?

What combination do you think sounds the best?

[Sadam1195]

So what parameters must be the same for Tacotron2 to work with a certain vocoder?

Audio parameters and log functions have to be same.

What combination do you think sounds the best?

That's subjective to dataset' audio parameters. There are few notebooks here https://github.com/coqui-ai/TTS/tree/main/notebooks/dataset_analysis to help you choose what would work best according to your dataset's distribution.