Cn_hubert Instructions for use

Recently, we have noticed that some users have provided feedback on the model's performance in "articulation." After multiple comparative tests and technical analysis, we confirmed that in the Retrieval-based-Voice-Conversion-WebUI project, HuBERT_base It plays a crucial role in overall pronunciation clarity, articulation accuracy, and phonetic feature extraction.

After deeply tracing the problem, we further discovered:

• Differences in feature extraction among different versions of HuBERT (such as hubert_base, chinese-hubert, improved self-supervised versions) directly affect the model's recognition of consonants, plosives, and weaks.

• If the HuBERT weights used in the base mold are unstable, problems such as such as slurred speech, slurping words, inaccurate articulation, and incomplete consonants may occur.

• The quality of user-trained datasets (clarity, mood changes, noise, sentence segmentation) is also amplified or weakened by HuBERT, affecting the final articulation effect.

To this end, in earlier experiments, we tried to apply Retrieval-based-Voice-Conversion-WebUI HuBERT_base fine-tuning. To ensure the previous model remains compatible with the original self-supervised HuBERT_base weights, we specifically freezing part of the model's front-loading layers, fine-tuning only the upper layers, so that the basic features are preserved while adapting to new training data.

However, in actual testing, we found:
Although this method brings some improvements in certain scenarios, the overall effect remains unsatisfactory. Previously, all models trained on old features commonly exhibited resulting in confused enunciation, pronunciation drift, and even incoherent speech, which had a relatively serious impact.

Clearly, this fine-tuning based on the layer freeze strategy did not meet the expected goals and failed to meet users' requirements for encition stability. We have determined that this direction was a failed fine-tuning attempt and have adjusted subsequent optimization routes based on this experience.

After that, we had to change our strategy and directly use the chinese-hubert self-supervised model to train feature models.

How to determine which HuBERT self-supervised model your model is using

On the model detail page, you can see at the top the HuBERT type used for the model:

The

model parameter area at the top of the detail page will clearly display:

If it shows as: Cn_hubert, you must follow the tutorial below; otherwise, the model will encounter issues such as rambling incoherence / misspelling / confused pronunciation during reasoning.

---

HuBERT replacement tutorial (be sure to read carefully).

Please go to the installation directory of Retrieval-based-Voice-Conversion-WebUI and find the following file:

This file is the HuBERT model relied upon for WebUI inference.

When inference is done using the "Cn_hubert-trained model," this file must be replaced with the Cn_hubert version we provide; otherwise, inference errors may occur.

---

Cn_hubert Download link

123 Cloud Drive: https://www.123865.com/s/hufyjv-KphO3?pwd=OLNr

Overseas Direct Link: https://huggingface.co/Rvcmodel/hubert/resolve/main/hubert_base.pt?download=true

---

Replacement steps (please follow the exact sequence).

1. Download the Cn_hubert version of the hubert_base.pt file from the link above.
2. Go to your RVC WebUI directory and find:
   assets/hubert/hubert_base.pt
3. Be sure to manually back up the original file first.
4. Overwrite the original file with the newly downloaded hubert_base.pt.
5. Restart WebUI.

If the "Second Change" version or other versions have inconsistent directories, you can also search directly for hubert_base.pt in the root directory to replace it.

---

Important Notes (Must Read!) ）

(1) If you are using a model trained on "non-Cn_hubert":

If you use a model trained on En_hubert and replace it with Cn_hubert, it will cause inference to be rambling.

Solution: Replace your backup of the original hubert_base.pt.

(2) If you are using a "Cn_hubert-trained model":

If you don't replace it with the Cn_hubert version of hubert_base.pt, nonsense will also appear.

Solution: Install the Cn_hubert version we provide and restart WebUI.

---

Summary

One-sentence rule:

• Models trained with Cn_hubert → must use Cn_hubert version of hubert_base.pt
• Models trained with the original HuBERT → must use the original hubert_base.pt
• In other words, if you see the model parameters are using Cn_hubert you can just replace them according to this documentation. If it's En_hubert, you don't need to change any files—just load the model using the original rvc.

If HuBERT is inconsistent with the version used for model training, it will directly cause inference to be confused.

Later

After

, we will gradually release a batch of Chinese models trained based on the Cn_hubert self-supervised model to improve the clarity and stability of enunciation in Chinese scenarios.
It should be noted that this series of models is mainly optimized for Chinese speech features, so Unable to guarantee articulation accuracy in other languages.

> In the future, we will continue to use the original self-supervised HuBERT model from Retrieval-based-Voice-Conversion-WebUI for most model training to ensure overall ecosystem compatibility and cross-language stability.