baidu

/

ERNIE-4.5-21B-A3B-Thinking

like 756

Follow

BAIDU 1.43k

We are currently developing the fine-tuning capability for this model and plan to release it in ERNIE, which is estimated to take one to two weeks.

Just checking to see if it is possible to train this model with Llama Factory or unsloth

Really awesome model

This is a viable approach, but it requires implementing a dedicated template for this model in LLaMA-Factory.

I've successfully integrated and trained the ERNIE 4.5 text model on LLaMA-Factory—see the PR here: https://github.com/hiyouga/LLaMA-Factory/pull/9165

Any model that works in transformers will work in Unsloth for training as well including ERNIE: https://github.com/unslothai/unsloth

I just tried training ERNIE 4.5 by switching out the model name in one of our notebooks and it worked successfully: https://docs.unsloth.ai/get-started/unsloth-notebooks

Edit - a community developer from Baidu made this notebook: https://colab.research.google.com/github/shareAI-lab/lab-handbook/blob/main/LLM_SFT_for_ERNIE4_5_English.ipynb