Update README.md

README.md

@@ -1,3 +1,117 @@
----
-license: apache-2.0
----
+---
+library_name: mlx
+pipeline_tag: text-generation
+inference: false
+license: apache-2.0
+base_model: openai/gpt-oss-120b
+base_model_relation: quantized
+language:
+- en
+- ro
+tags:
+- apple-silicon
+- metal
+- arm64
+- 6-bit
+- group-size-64
+- mlx
+- mlx-lm
+- openai
+- halley-ai
+---
+
+# gpt-oss-120b — MLX 6-bit (group size 64)
+
+**Summary.** This is a 6-bit MLX quantization of gpt-oss-120B with group size 64. It targets a smaller memory footprint and higher throughput than the 8-bit gs=32 build while keeping quality close to the bf16/8-bit references.
+
+- **Base model:** `openai/gpt-oss-120b` (Apache-2.0)
+- **Quantization:** MLX int6, `q_group_size=64` (some tensors may remain 16-bit for stability)
+- **Files:** MLX weight shards + `config.json`; tokenizer files included for drop-in use
+- **Intended use:** local inference / research on M-series Macs
+- **Not intended for:** safety-critical decisions; outputs may be inaccurate or biased
+
+## Requirements
+
+Runs on Apple Silicon (M1 or newer) with macOS ≥ 13.5 via MLX (Metal).
+
+- Not supported: Intel macOS / Linux / Windows (consider a GGUF build + llama.cpp instead).
+- Memory guidance: notably smaller footprint vs 8-bit/gs32; 64–96 GB recommended for comfortable headroom on 120B with moderate context sizes. The effective GPU working set is capped by Metal’s budget; keep 5–10% headroom.
+
+## How to use (MLX)
+
+```bash
+pip install mlx-lm
+```
+
+```python
+# Python API (uses tokenizer bundled with this repo)
+from mlx_lm import load, generate
+
+model, tokenizer = load("halley-ai/gpt-oss-120b-MLX-6bit-gs64")
+print(generate(
+    model, tokenizer,
+    prompt="Explain the Chudnovsky algorithm to compute π.",
+    max_tokens=256, max_kv_size=512
+))
+```
+
+```bash
+# CLI
+python -m mlx_lm generate --model halley-ai/gpt-oss-120b-MLX-6bit-gs64 \
+  --prompt "Explain the Chudnovsky algorithm to compute pi." \
+  --max-kv-size 512 --max-tokens 256
+```
+
+## Evaluation
+
+Perplexity (PPL) streaming evaluation on WikiText-2 (raw, test) is recommended with the fast preset (`window=stride=4096`, ~100k tokens, EOS inserted between docs):
+
+```bash
+python python/scripts/test_perplexity-mlx.py \
+  --model_path "/path/to/gpt-oss-120b-MLX-6bit-gs64" \
+  --fast --progress
+```
+
+For more sensitive comparisons, use overlapping windows (for example, `--stride 512`) and evaluate the full split.
+
+### Results
+
+| Variant              | PPL (ctx=4096, fast) |
+|----------------------|-----------------------|
+| MLX 6-bit (gs=64)    | 7.40                  |
+| MLX 8-bit (gs=32)    | 7.39                  |
+| MLX bf16 (reference) | 7.38                  |
+
+## Conversion details (provenance)
+
+```bash
+python -m mlx_lm convert \
+  --hf-path openai/gpt-oss-120b \
+  --mlx-path gpt-oss-120b-MLX-6bit-gs64 \
+  --q-bits 6 --q-group-size 64 -q
+```
+
+- Some tensors (for example, embeddings/norms/router) may remain 16-bit for numerical stability.
+
+## Footprint and speed tips
+
+- **Limit KV cache:** set `--max-kv-size` (CLI) or `max_kv_size` (Python) to the smallest context you need.
+- **Batching:** prefer single-stream generation; large batches increase memory pressure on 120B.
+- **Compute windowing:** when evaluating PPL, the provided script auto-clamps the compute window to avoid Metal’s per-buffer limits.
+- **Sampler settings:** top‑p/top‑k sampling with moderate temperature can improve throughput versus beam search.
+
+## Sibling and reference models
+
+- halley-ai/gpt-oss-120b-MLX-8bit-gs32 (reference 8-bit)
+- halley-ai/gpt-oss-120b-MLX-bf16 (non-quantized reference)
+
+## Limitations and biases
+
+Outputs may be factually wrong or unsafe. Do not use for medical, legal, or financial decisions without human review. Large models can be sensitive to prompt wording; prefer explicit instructions and structure.
+
+## License and credits
+
+- License: Apache-2.0 (inherits from base model)
+- Base model: OpenAI gpt-oss-120B
+- Quantization: Halley AI Lab (MLX int6, gs=64)
+- Please cite both the base model and this repository when you use the weights.