Lowest Memory Models?

[stanleyshly]

What are some of the models that use the least amount of memory? I don't need an amazing output, I just want to show a demo of barely functioning model.

Also, is it possible to run inference on the CPU only?

[tekinek]

The Tacotron+MB-Melgan seem to use less memory (2~3GB) while the glow-tts synthesizes speech faster on CPU.

[arif334]

I've found the following model sizes for my 12-hour Bangla dataset. Sizes may vary slightly depending on datasets/languages.

Model	Size (MB)
GlowTTS	344
AlignTTS	279
Tacotron2-DDC	572
Tacotron2-DCA	344
FastPitch	458
VITS	998
MB-MelGAN	83
HifiGAN	1017
UnivNet	344
WaveRNN	51
WaveGrad	190

WaveRNN and WaveGrad vocoders are extremely slow on CPU. Other models can generate speech in real-time.

[arif334]

Sure. I'll give SpeedySpeech a try when my GPUs are available.

[WeberJulian]

One thing to keep in mind is that MelGAN HifiGAN and VITS have discriminator weights that can be discarded safely for inference. You can also further decrease the size of the checkpoint by deleting optimizer weights. This optimizations can make the weight for VITS go from almost 1GB to 350~MB

[KakoozaJerry]

Thanks @WeberJulian, would love to see how this can be done, thank you

[tekinek]

@KakoozaJerry create a python file (e.g. shrink.py) with the following code snippet. Excuteing it with a path to the checkpoint will generate a shrinked checkpoint named 'checkpoint_steps.inf.pth.tar'.

For example, # python shrink.py checkpoint_100000.pth.tar will generate checkpoint_100000.inf.pth.tar in the current directory.

import sys
file = sys.argv[1]
import torch
state = torch.load(file, map_location=torch.device('cpu'))
tmp = state.pop("optimizer", None)
torch.save(state, file.replace('.pth', '.inf.pth'))

[KakoozaJerry]

Thanks @tekinek

[erogol]

You should use SpeedySpeech model.