Mungert

/

MiMo-VL-7B-RL-GGUF

like 3

Its works if you do this : llama-mtmd-cli -m MiMo-VL-7B-RL-q4_k_m.gguf --mmproj MiMo-VL-7B-RL-F16.mmproj
build: 6109 (1d72c841) with cc (Debian 12.2.0-14+deb12u1) 12.2.0 for x86_64-linux-gnu
llama_model_loader: loaded meta data with 37 key-value pairs and 435 tensors from MiMo-VL-7B-RL-q4_k_m.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen2vl
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = MiMo-VL-7B-RL
llama_model_loader: - kv 3: general.size_label str = 7.6B
llama_model_loader: - kv 4: general.license str = mit
llama_model_loader: - kv 5: general.base_model.count u32 = 1
llama_model_loader: - kv 6: general.base_model.0.name str = MiMo VL 7B RL
llama_model_loader: - kv 7: general.base_model.0.organization str = XiaomiMiMo
llama_model_loader: - kv 8: general.base_model.0.repo_url str = https://huggingface.co/XiaomiMiMo/MiM...
llama_model_loader: - kv 9: qwen2vl.block_count u32 = 36
llama_model_loader: - kv 10: qwen2vl.context_length u32 = 128000
llama_model_loader: - kv 11: qwen2vl.embedding_length u32 = 4096
llama_model_loader: - kv 12: qwen2vl.feed_forward_length u32 = 11008
llama_model_loader: - kv 13: qwen2vl.attention.head_count u32 = 32
llama_model_loader: - kv 14: qwen2vl.attention.head_count_kv u32 = 8
llama_model_loader: - kv 15: qwen2vl.rope.freq_base f32 = 640000.000000
llama_model_loader: - kv 16: qwen2vl.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 17: qwen2vl.rope.dimension_sections arr[i32,4] = [16, 24, 24, 0]
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,151680] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,151680] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 23: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 24: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 26: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 27: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 28: general.quantized_by str = Mungert
llama_model_loader: - kv 29: general.repo_url str = https://huggingface.co/mungert
llama_model_loader: - kv 30: general.sponsor_url str = https://readyforquantum.com
llama_model_loader: - kv 31: general.quantization_version u32 = 2
llama_model_loader: - kv 32: general.file_type u32 = 15
llama_model_loader: - kv 33: quantize.imatrix.file str = /home/mahadeva/code/models/MiMo-VL-7B...
llama_model_loader: - kv 34: quantize.imatrix.dataset str = /home/mahadeva/code/GGUFModelBuilder/...
llama_model_loader: - kv 35: quantize.imatrix.entries_count i32 = 252
llama_model_loader: - kv 36: quantize.imatrix.chunks_count i32 = 128
llama_model_loader: - type f32: 181 tensors
llama_model_loader: - type q4_K: 208 tensors
llama_model_loader: - type q5_K: 24 tensors
llama_model_loader: - type q6_K: 22 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 4.47 GiB (5.04 BPW)
load: printing all EOG tokens:
load: - 151643 ('<|endoftext|>')
load: - 151645 ('<|im_end|>')
load: - 151662 ('<|fim_pad|>')
load: - 151663 ('<|repo_name|>')
load: - 151664 ('<|file_sep|>')
load: special tokens cache size = 22
load: token to piece cache size = 0.9310 MB
print_info: arch = qwen2vl
print_info: vocab_only = 0
print_info: n_ctx_train = 128000
print_info: n_embd = 4096
print_info: n_layer = 36
print_info: n_head = 32
print_info: n_head_kv = 8
print_info: n_rot = 128
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 4
print_info: n_embd_k_gqa = 1024
print_info: n_embd_v_gqa = 1024
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 11008
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 8
print_info: rope scaling = linear
print_info: freq_base_train = 640000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 128000
print_info: rope_finetuned = unknown
print_info: model type = 3B
print_info: model params = 7.62 B
print_info: general.name = MiMo-VL-7B-RL
print_info: vocab type = BPE
print_info: n_vocab = 151680
print_info: n_merges = 151387
print_info: BOS token = 151643 '<|endoftext|>'
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 'Ċ'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: CPU_REPACK model buffer size = 2874.38 MiB
load_tensors: CPU_Mapped model buffer size = 4556.29 MiB
.................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 4096
llama_context: n_ctx_per_seq = 4096
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = 0
llama_context: kv_unified = false
llama_context: freq_base = 640000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (4096) < n_ctx_train (128000) -- the full capacity of the model will not be utilized
llama_context: CPU output buffer size = 0.58 MiB
llama_kv_cache_unified: CPU KV buffer size = 576.00 MiB
llama_kv_cache_unified: size = 576.00 MiB ( 4096 cells, 36 layers, 1/1 seqs), K (f16): 288.00 MiB, V (f16): 288.00 MiB
llama_context: CPU compute buffer size = 304.25 MiB
llama_context: graph nodes = 1374
llama_context: graph splits = 162 (with bs=512), 1 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 4096
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
mtmd_cli_context: chat template example:
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant

clip_model_loader: model name: Mimo-Vl-7B-Rl
clip_model_loader: description:
clip_model_loader: GGUF version: 3
clip_model_loader: alignment: 32
clip_model_loader: n_tensors: 519
clip_model_loader: n_kv: 30

clip_model_loader: has vision encoder
clip_ctx: CLIP using CPU backend
load_hparams: projector: qwen2.5vl_merger
load_hparams: n_embd: 1280
load_hparams: n_head: 16
load_hparams: n_ff: 3456
load_hparams: n_layer: 32
load_hparams: ffn_op: silu
load_hparams: projection_dim: 4096

--- vision hparams ---
load_hparams: image_size: 1024
load_hparams: patch_size: 14
load_hparams: has_llava_proj: 0
load_hparams: minicpmv_version: 0
load_hparams: proj_scale_factor: 0
load_hparams: n_wa_pattern: 8

load_hparams: model size: 1304.85 MiB
load_hparams: metadata size: 0.18 MiB
alloc_compute_meta: CPU compute buffer size = 3.63 MiB
main: loading model: MiMo-VL-7B-RL-q4_k_m.gguf

Running in chat mode, available commands:
/image

/image car-1.jpg
car-1.jpg image loaded

what is the image
encoding image slice...
image slice encoded in 97051 ms
decoding image batch 1/1, n_tokens_batch = 925
image decoded (batch 1/1) in 68650 ms

So, let's analyze the image. First, identify the main subject. The image shows a sleek, black Porsche Panamera driving on a highway. Let's check details: the car's make is Porsche, model Panamera (a luxury sedan). The background has a highway with blurred trees and other vehicles, indicating motion. The license plate is visible, and the car has a sporty design with custom wheels maybe. So the image is of a Porsche Panamera in motion on a road. The image depicts a **black Porsche Panamera** (a luxury sedan) driving on a highway. The car is in motion, as suggested by the blurred background (trees, other vehicles, and the road). Key details include the “Porsche” branding on the rear, the sleek design of the Panamera, custom wheels, and a visible license plate. The setting appears to be an urban or suburban highway during dusk or early evening, with a calm, overcast sky and blurred foliage in the background.

.... try updating your llama.cpp

It also works with latest llama-server. I think you should check your files names you cant run a blob file .