---
base_model: google/gemma-3-270m-it
library_name: peft
tags:
- text-generation
- form-generation
- json
- gemma
- lora
- peft
- bahasa-indonesia
language:
- id
datasets:
- bhismaperkasa/form_dinamis
license: apache-2.0
---

# Form Generator - Gemma 3 270M (LoRA Adapters)

Fine-tuned LoRA adapters for `google/gemma-3-270m-it` to generate form definitions in JSON format from natural language descriptions in Bahasa Indonesia.

## 🎯 Model Description

This repository contains **LoRA adapters** (not a merged model) that can be loaded on top of the base Gemma 3 270M model. The adapters were trained to generate dynamic form definitions in JSON format.

## ⚠️ Important Note

**This is a LoRA adapter model, NOT a standalone model.** You need to load it together with the base model `google/gemma-3-270m-it`.

## 🚀 Usage

### Installation

```bash
pip install transformers peft torch
```

### Loading the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-3-270m-it",
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "bhismaperkasa/form-generator-lora-adapters")
tokenizer = AutoTokenizer.from_pretrained("bhismaperkasa/form-generator-lora-adapters")

# Generate
messages = [
    {"role": "system", "content": "You are a helpful assistant that generates form definitions in JSON format based on user requests."},
    {"role": "user", "content": "buatkan form pendaftaran event dengan nama dan email"}
]

input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
generated = result.split("<start_of_turn>model\n")[-1].strip()
print(generated)
```

## 📊 Training Details

- **Base Model**: google/gemma-3-270m-it
- **Method**: QLoRA (4-bit quantization + LoRA)
- **Dataset**: bhismaperkasa/form_dinamis (10,000 samples)
- **Training Samples**: ~9,000
- **Validation Samples**: ~1,000
- **Epochs**: 3
- **Final Eval Loss**: 0.2256
- **Token Accuracy**: 93.5%

### Hyperparameters

- LoRA Rank: 16
- LoRA Alpha: 32
- LoRA Dropout: 0.05
- Learning Rate: 5e-5
- Batch Size: 4
- Max Length: 512 tokens

## 📈 Performance

- **Evaluation Loss**: 0.2256
- **Token Accuracy**: 93.50%
- **Train-Eval Gap**: 4.9% (healthy, no overfitting)
- **Entropy**: 0.1881 (high confidence)

## 💡 Example Outputs

**Input:**
```
buatkan form login sederhana
```

**Output:**
```json
{
  "id": "form_login_sims",
  "title": "Form Login",
  "description": "Form untuk login akun",
  "formDefinition": {
    "fields": [
      {"fieldId": "email", "label": "Email", "fieldType": "EMAIL", "required": true},
      {"fieldId": "password", "label": "Password", "fieldType": "PASSWORD", "required": true}
    ]
  }
}
```

## 🎓 Use Cases

- Dynamic form generation for web applications
- Survey and questionnaire creation
- User registration forms
- Data collection forms
- Event registration forms

## ⚙️ Technical Details

### Why LoRA Adapters?

- **Smaller size**: ~50MB vs ~500MB for merged model
- **Better quality**: Works more reliably than merged model
- **Flexibility**: Can be combined with different base models
- **Efficiency**: Faster to download and deploy

### Model Architecture

- Base: Gemma 3 270M (270 million parameters)
- Adapters: LoRA with rank 16 (few million trainable parameters)
- Target modules: All linear layers
- Quantization: 4-bit NormalFloat (NF4)

## 📝 Citation

```bibtex
@misc{form-generator-gemma-lora,
  author = {bhismaperkasa},
  title = {Form Generator - Gemma 3 270M LoRA Adapters},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bhismaperkasa/form-generator-lora-adapters}}
}
```

## 📄 License

This model is based on Gemma 3 270M which is licensed under Apache 2.0.

## 🙏 Acknowledgments

- Google for Gemma 3 270M model
- Hugging Face for transformers, PEFT, and TRL libraries
- Dataset: bhismaperkasa/form_dinamis

---

**Note**: For production use, consider using these adapters instead of a merged model for better reliability and performance.