Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: mlx-vlm
|
| 3 |
+
tags:
|
| 4 |
+
- mlx
|
| 5 |
+
- vision-language-model
|
| 6 |
+
- fine-tuned
|
| 7 |
+
- brake-components
|
| 8 |
+
- visual-ai
|
| 9 |
+
base_model: mlx-community/SmolVLM-256M-Instruct-bf16
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Hellow - MLX Fine-tuned Vision Language Model
|
| 13 |
+
|
| 14 |
+
This model was fine-tuned using the VisualAI platform with MLX (Apple Silicon optimization).
|
| 15 |
+
|
| 16 |
+
## π Model Details
|
| 17 |
+
- **Base Model**: `mlx-community/SmolVLM-256M-Instruct-bf16`
|
| 18 |
+
- **Training Platform**: VisualAI (MLX-optimized)
|
| 19 |
+
- **GPU Type**: MLX (Apple Silicon)
|
| 20 |
+
- **Training Job ID**: 1
|
| 21 |
+
- **Created**: 2025-06-03 04:52:55.384214
|
| 22 |
+
- **Training Completed**: β
Yes
|
| 23 |
+
|
| 24 |
+
## π Training Data
|
| 25 |
+
This model was trained on a combined dataset with visual examples and conversations.
|
| 26 |
+
|
| 27 |
+
## π οΈ Usage
|
| 28 |
+
|
| 29 |
+
### Installation
|
| 30 |
+
```bash
|
| 31 |
+
pip install mlx-vlm
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
### Loading the Model
|
| 35 |
+
```python
|
| 36 |
+
from mlx_vlm import load
|
| 37 |
+
import json
|
| 38 |
+
import os
|
| 39 |
+
|
| 40 |
+
# Load the base MLX model
|
| 41 |
+
model, processor = load("mlx-community/SmolVLM-256M-Instruct-bf16")
|
| 42 |
+
|
| 43 |
+
# Load the fine-tuned artifacts
|
| 44 |
+
model_info_path = "mlx_model_info.json"
|
| 45 |
+
if os.path.exists(model_info_path):
|
| 46 |
+
with open(model_info_path, 'r') as f:
|
| 47 |
+
model_info = json.load(f)
|
| 48 |
+
print(f"β
Loaded fine-tuned model with {model_info.get('training_examples_count', 0)} training examples")
|
| 49 |
+
|
| 50 |
+
# Check for adapter weights
|
| 51 |
+
adapters_path = "adapters/adapter_config.json"
|
| 52 |
+
if os.path.exists(adapters_path):
|
| 53 |
+
with open(adapters_path, 'r') as f:
|
| 54 |
+
adapter_config = json.load(f)
|
| 55 |
+
print(f"π― Found MLX adapters with {adapter_config.get('training_examples', 0)} training examples")
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
### Inference
|
| 59 |
+
```python
|
| 60 |
+
from mlx_vlm import generate
|
| 61 |
+
from mlx_vlm.prompt_utils import apply_chat_template
|
| 62 |
+
from mlx_vlm.utils import load_config
|
| 63 |
+
from PIL import Image
|
| 64 |
+
|
| 65 |
+
# Load your image
|
| 66 |
+
image = Image.open("your_image.jpg")
|
| 67 |
+
|
| 68 |
+
# Ask a question
|
| 69 |
+
question = "What type of brake component is this?"
|
| 70 |
+
|
| 71 |
+
# Format the prompt
|
| 72 |
+
config = load_config("mlx-community/SmolVLM-256M-Instruct-bf16")
|
| 73 |
+
formatted_prompt = apply_chat_template(processor, config, question, num_images=1)
|
| 74 |
+
|
| 75 |
+
# Generate response
|
| 76 |
+
response = generate(model, processor, formatted_prompt, [image], verbose=False, max_tokens=100)
|
| 77 |
+
print(f"Model response: {response}")
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
## π Model Artifacts
|
| 81 |
+
|
| 82 |
+
This repository contains:
|
| 83 |
+
- `mlx_model_info.json`: Training metadata and learned mappings
|
| 84 |
+
- `training_images/`: Reference images from training data
|
| 85 |
+
- `adapters/`: MLX LoRA adapter weights and configuration (if available)
|
| 86 |
+
- `README.md`: This documentation
|
| 87 |
+
|
| 88 |
+
## β οΈ Important Notes
|
| 89 |
+
|
| 90 |
+
- This model uses MLX format optimized for Apple Silicon
|
| 91 |
+
- The actual model weights remain in the base model (`mlx-community/SmolVLM-256M-Instruct-bf16`)
|
| 92 |
+
- The fine-tuning artifacts enhance the model's domain-specific knowledge
|
| 93 |
+
- **Check the `adapters/` folder for MLX-specific fine-tuned weights**
|
| 94 |
+
- For best results, use on Apple Silicon devices (M1/M2/M3)
|
| 95 |
+
|
| 96 |
+
## π― Training Statistics
|
| 97 |
+
|
| 98 |
+
- Training Examples: 3
|
| 99 |
+
- Learned Mappings: 2
|
| 100 |
+
- Domain Keywords: 79
|
| 101 |
+
|
| 102 |
+
## π Support
|
| 103 |
+
|
| 104 |
+
For questions about this model or the VisualAI platform, please refer to the training logs or contact support.
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
*This model was trained using VisualAI's MLX-optimized training pipeline.*
|