Update model card with benchmarking data.
Browse files
README.md
CHANGED
|
@@ -17,3 +17,10 @@ This model was converted to LiteRT (aka TFLite) format from [deepseek-ai/DeepSee
|
|
| 17 |
## Run the model on Android
|
| 18 |
|
| 19 |
Please follow the [instructions](https://github.com/google-ai-edge/mediapipe-samples/blob/main/examples/llm_inference/android/README.md).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
## Run the model on Android
|
| 18 |
|
| 19 |
Please follow the [instructions](https://github.com/google-ai-edge/mediapipe-samples/blob/main/examples/llm_inference/android/README.md).
|
| 20 |
+
|
| 21 |
+
## Benchmarking results
|
| 22 |
+
|
| 23 |
+
| Model | Params | | GGML tk/s (CPU, 4 threads) | GGML tk/s (CPU, 8 threads) | LiteRT tk/s (XNNPACK, 4 threads) | LiteRT tk/s (XNNPACK, 8 threads) |
|
| 24 |
+
| ---------------------------------------------- | ------ | ------------------ | -------------------------- | -------------------------- | -------------------------------- | -------------------------------- |
|
| 25 |
+
| DeepSeek-R1-Distill-Qwen-1.5B (Int8 quantized) | 1.78 B | Prefill 512 tokens | 64.66 | 87.18 | 260.95 | 299.15 |
|
| 26 |
+
| | | Decode 128 tokens | 23.85 | 15.37 | 23.126 | 10.486 |
|