Upload README.md with huggingface_hub

README.md

@@ -1,62 +1,175 @@
 ---
 base_model: google/gemma-3-270m-it
 library_name: peft
-model_name: form-generator-gemma-adapters
 tags:
-- base_model:adapter:google/gemma-3-270m-it
+- text-generation
+- form-generation
+- json
+- gemma
 - lora
-- sft
-- transformers
-- trl
-licence: license
-pipeline_tag: text-generation
+- peft
+- bahasa-indonesia
+language:
+- id
+datasets:
+- bhismaperkasa/form_dinamis
+license: apache-2.0
 ---
 
-# Model Card for form-generator-gemma-adapters
+# Form Generator - Gemma 3 270M (LoRA Adapters)
 
-This model is a fine-tuned version of [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+Fine-tuned LoRA adapters for `google/gemma-3-270m-it` to generate form definitions in JSON format from natural language descriptions in Bahasa Indonesia.
 
-## Quick start
+## 🎯 Model Description
+
+This repository contains **LoRA adapters** (not a merged model) that can be loaded on top of the base Gemma 3 270M model. The adapters were trained to generate dynamic form definitions in JSON format.
+
+## ⚠️ Important Note
+
+**This is a LoRA adapter model, NOT a standalone model.** You need to load it together with the base model `google/gemma-3-270m-it`.
+
+## 🚀 Usage
+
+### Installation
+
+```bash
+pip install transformers peft torch
+```
+
+### Loading the Model
 
 ```python
-from transformers import pipeline
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    "google/gemma-3-270m-it",
+    device_map="auto",
+    torch_dtype=torch.bfloat16
+)
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+# Load LoRA adapters
+model = PeftModel.from_pretrained(base_model, "bhismaperkasa/form-generator-lora-adapters")
+tokenizer = AutoTokenizer.from_pretrained("bhismaperkasa/form-generator-lora-adapters")
+
+# Generate
+messages = [
+    {"role": "system", "content": "You are a helpful assistant that generates form definitions in JSON format based on user requests."},
+    {"role": "user", "content": "buatkan form pendaftaran event dengan nama dan email"}
+]
+
+input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    do_sample=True
+)
+
+result = tokenizer.decode(outputs[0], skip_special_tokens=True)
+generated = result.split("<start_of_turn>model\n")[-1].strip()
+print(generated)
 ```
 
-## Training procedure
+## 📊 Training Details
 
- 
+- **Base Model**: google/gemma-3-270m-it
+- **Method**: QLoRA (4-bit quantization + LoRA)
+- **Dataset**: bhismaperkasa/form_dinamis (10,000 samples)
+- **Training Samples**: ~9,000
+- **Validation Samples**: ~1,000
+- **Epochs**: 3
+- **Final Eval Loss**: 0.2256
+- **Token Accuracy**: 93.5%
 
+### Hyperparameters
 
-This model was trained with SFT.
+- LoRA Rank: 16
+- LoRA Alpha: 32
+- LoRA Dropout: 0.05
+- Learning Rate: 5e-5
+- Batch Size: 4
+- Max Length: 512 tokens
 
-### Framework versions
+## 📈 Performance
 
-- PEFT 0.17.1
-- TRL: 0.24.0
-- Transformers: 4.57.1
-- Pytorch: 2.9.0
-- Datasets: 4.3.0
-- Tokenizers: 0.22.1
+- **Evaluation Loss**: 0.2256
+- **Token Accuracy**: 93.50%
+- **Train-Eval Gap**: 4.9% (healthy, no overfitting)
+- **Entropy**: 0.1881 (high confidence)
 
-## Citations
+## 💡 Example Outputs
 
+**Input:**
+```
+buatkan form login sederhana
+```
 
+**Output:**
+```json
+{
+  "id": "form_login_sims",
+  "title": "Form Login",
+  "description": "Form untuk login akun",
+  "formDefinition": {
+    "fields": [
+      {"fieldId": "email", "label": "Email", "fieldType": "EMAIL", "required": true},
+      {"fieldId": "password", "label": "Password", "fieldType": "PASSWORD", "required": true}
+    ]
+  }
+}
+```
+
+## 🎓 Use Cases
+
+- Dynamic form generation for web applications
+- Survey and questionnaire creation
+- User registration forms
+- Data collection forms
+- Event registration forms
+
+## ⚙️ Technical Details
+
+### Why LoRA Adapters?
+
+- **Smaller size**: ~50MB vs ~500MB for merged model
+- **Better quality**: Works more reliably than merged model
+- **Flexibility**: Can be combined with different base models
+- **Efficiency**: Faster to download and deploy
+
+### Model Architecture
+
+- Base: Gemma 3 270M (270 million parameters)
+- Adapters: LoRA with rank 16 (few million trainable parameters)
+- Target modules: All linear layers
+- Quantization: 4-bit NormalFloat (NF4)
+
+## 📝 Citation
 
-Cite TRL as:
-    
 ```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
+@misc{form-generator-gemma-lora,
+  author = {bhismaperkasa},
+  title = {Form Generator - Gemma 3 270M LoRA Adapters},
+  year = {2025},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/bhismaperkasa/form-generator-lora-adapters}}
 }
-```
+```
+
+## 📄 License
+
+This model is based on Gemma 3 270M which is licensed under Apache 2.0.
+
+## 🙏 Acknowledgments
+
+- Google for Gemma 3 270M model
+- Hugging Face for transformers, PEFT, and TRL libraries
+- Dataset: bhismaperkasa/form_dinamis
+
+---
+
+**Note**: For production use, consider using these adapters instead of a merged model for better reliability and performance.