Glitches and 'hallucinations' in clone voices

[Sambsop]

When using the clone feature the audio resulting comes (in 100% of my cases) with small (fast) audio glitches or more big deformed  intelligible rendered voice (really ghostly...).
I 'm working in french. I used several audio references (even a synthetic voice) with good quality and tried several file length. But the glitches are always  present.
I've done several tests fixin the randomness of the model (using a known seed value) to be reproducible, and in the best case I only got some fast audio peaks that could be candidates to be removed with classical methods.
I'll try Whisper + jiwer with WER & CER metrics (Gemini suggestion) to find the culprit chunks but without success.
I'm out of my depth so any suggestion is Welcome.
Thanks in advance.