End of training

README.md

@@ -1,109 +1,59 @@
-# 🧠 LoL_Build-Llama3B
-
-A fine-tuned version of the LLaMA 3.2B model using QLoRA on a custom League of Legends build suggestion dataset. This model generates champion-specific item build recommendations based on gameplay roles and current meta.
-
 ---
-
-## 📚 Dataset
-
-- **Source**: Custom JSONL dataset with `prompt` and `completion` fields.
-- **Train/Val Split**: 2 files – `train.jsonl` and `val.jsonl`
-- **Schema Example**:
-  ```json
-  {
-    "prompt": "Suggest a build for Ahri mid lane.",
-    "completion": "Luden's Tempest, Sorcerer's Shoes, Shadowflame..."
-  }
-  ```
-
----
-
-## 🏋️‍♂️ Training Configuration
-
-| Hyperparameter              | Value        |
-|----------------------------|--------------|
-| Base Model                 | unsloth/Llama-3.2-3B-bnb-4bit |
-| Batch Size                 | 16           |
-| Gradient Accumulation      | 1            |
-| Epochs                     | 1            |
-| Max Steps                  | 10000        |
-| Learning Rate              | 2e-4         |
-| Weight Decay               | 0.01         |
-| Max Sequence Length        | 512          |
-| Precision                  | BF16 (fallback to FP16) |
-| Optimizer                  | AdamW (8bit) |
-| LoRA Rank                  | 16           |
-| LoRA Alpha                 | 32           |
-| LoRA Dropout               | 0.05         |
-
+base_model: unsloth/llama-3.2-3b-bnb-4bit
+library_name: transformers
+model_name: LoL_Build-Llama3B
+tags:
+- generated_from_trainer
+- unsloth
+- trl
+- sft
+licence: license
 ---
 
-### 📊 Evaluation
+# Model Card for LoL_Build-Llama3B
 
-Trained on a single NVIDIA RTX 3060 GPU.
+This model is a fine-tuned version of [unsloth/llama-3.2-3b-bnb-4bit](https://huggingface.co/unsloth/llama-3.2-3b-bnb-4bit).
+It has been trained using [TRL](https://github.com/huggingface/trl).
 
-| Metric                     | Value              |
-|---------------------------|--------------------|
-| **Final Eval Loss**       | 0.1472             |
-| **Steps Completed**       | 2386               |
-| **Total Epochs Trained**  | 1.0                |
-| **Training Batch Size**   | 32 (effective)     |
-| **Final Learning Rate**   | 1.68e-7            |
-| **Final Grad Norm**       | 1.64               |
-| **Total FLOPs**           | 6.67e+17           |
-| **Eval Runtime**          | 1611.14 seconds    |
-| **Eval Samples/sec**      | 5.27               |
-| **Eval Steps/sec**        | 0.659              |
-
----
-
-## ⚙️ Usage
+## Quick start
 
 ```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
+from transformers import pipeline
 
-tokenizer = AutoTokenizer.from_pretrained("HatimF/LoL_Build-Llama3B")
-model = AutoModelForCausalLM.from_pretrained("HatimF/LoL_Build-Llama3B")
-
-prompt = "Suggest a build for Ahri in mid lane."
-inputs = tokenizer(prompt, return_tensors="pt")
-outputs = model.generate(**inputs, max_new_tokens=100)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="HatimF/LoL_Build-Llama3B", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
 ```
 
----
+## Training procedure
 
-## 🧠 Intended Use
+ 
 
-- **Primary**: Champion item build recommendation for League of Legends.
-- **Limitations**:
-  - May hallucinate outdated items or suggest invalid builds.
-  - Not trained on patch-specific data.
 
----
+This model was trained with SFT.
 
-## 📦 Repository Files
+### Framework versions
 
-| File                      | Description                     |
-|---------------------------|---------------------------------|
-| `adapter_model.safetensors` | LoRA adapter weights          |
-| `adapter_config.json`     | Configuration for LoRA          |
-| `generation_config.json`  | Decoding hyperparameters        |
-| `training_args.bin`       | TrainingArguments instance (Unsloth) |
-| `trainer_state.json`      | Logged evaluation metrics       |
-| `tokenizer.json`          | Tokenizer vocabulary            |
-| `special_tokens_map.json` | Special tokens                  |
-| `tokenizer_config.json`   | Tokenizer settings              |
+- TRL: 0.15.2
+- Transformers: 4.51.3
+- Pytorch: 2.6.0
+- Datasets: 3.5.0
+- Tokenizers: 0.21.1
 
----
+## Citations
 
-## 📄 Citation
 
+
+Cite TRL as:
+    
 ```bibtex
-@misc{hatimf2025lolbuildllama3b,
-  title={LoL_Build-Llama3B},
-  author={HatimF},
-  year={2025},
-  url={https://huggingface.co/HatimF/LoL_Build-Llama3B}
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
 }
-```
+```

adapter_config.json

@@ -24,13 +24,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
-    "v_proj",
-    "down_proj",
     "q_proj",
-    "up_proj",
+    "v_proj",
+    "k_proj",
     "gate_proj",
-    "k_proj"
+    "up_proj",
+    "down_proj",
+    "o_proj"
   ],
   "task_type": "CAUSAL_LM",
   "trainable_token_indices": null,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fb860557877b89ec4a77e2080ddfb5ea42f36bbb2d7b3272348bf6c21ba3e0a3
+oid sha256:2b95aabbb3b72cce017be0d45cd29a117ebc0a89be346814b9520f35e5bb4001
 size 48680136

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9c85066e7642934ed09b44155e6566b0b5dab2637fb9433439ba5c9c7f8b50d3
-size 17210018
+oid sha256:52716f60c3ad328509fa37cdded9a2f1196ecae463f5480f5d38c66a25e7a7dc
+size 17210019

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:013e435f2e29457d9392e4189b5bb0be9030270a541c5294766981b33966a016
+oid sha256:e97402967b333756f2c233024afbfee0e9b7c090c02c7548ac7f4bbc315d8f05
 size 5688