SmolLM2-135M-Instruct-TaiwanChat / train_with_unsloth.py

Commit History

ajustement to avoid oom err
2340301

Luigi commited on

fix "train_on_responses_only not set correctly"
22d1cb2

Luigi commited on

bugfix: ValueError: Unsloth: instruction_part and response_part must be given!
20210b2

Luigi commited on

Re-apply unsloth following official guidiance
1f5b86c

Luigi commited on

fix for AttributeError
5d7244c

Luigi commited on

import automodel for inference test
eb278ee

Luigi commited on

disable evaluation to save ram as it did crash on v100 at eval phase
73aa00e

Luigi commited on

rename wandb project name for train with unsloth .py
3645d25

Luigi commited on

Add padding and truncation on examples to fix max length
8249056

Luigi commited on

tell trainer to mask user tokens to avoid llm to slef-talks during inference
bc9cad6

Luigi commited on

refs #30267, Add perplexity-based validation in unsloth-based train script
b91c58e

Luigi commited on

refs #30267, fine-tune with around 20M tokens subset of TaiwanChat
d31cdf6

Luigi commited on

Add script to efficiently train with unsloth
ab1f15c

Luigi commited on