Fined-tuned model does not sound correct with the new voice

[Ca-ressemble-a-du-fake]

Hi

I fined-tuned a model a tested the voice during training after 83k steps. But result does not sound at all like the target voice and is slow :

savezVous.mp4

Is it because my dataset is too short (100 samples) compared to the MAILABS dataset that was used in the original model ? Or is it because I disabled the use_phoneme flag (as it was in the original config.json).

The other voices already trained in the original model seem to get better with fine-tuning but of course they don't sound as the target voice! Should I train a lot more ? I read somewhere that it should take 30 minutes on rtx3090, but here after 1,5 day on a V100 16GB (or A100 40GB) the result is not good.

What should I do ?

Thanks