gguf_init_from_file_impl: tensor 'token_embd.weight'
#9
by
arichiardi
- opened
Hi there, I could not figure this out myself and apologies if this is a newbie mistake but I compiled ik-llama and download IQ2_KL. However, I get:
gguf_init_from_file_impl: tensor 'token_embd.weight' has invalid ggml type 139 (NONE)
when trying to load it with:
LLAMA_ARGS=--host 0.0.0.0 --port 10434 --alias GLM-4.5-Air --model models/ubergarm/GLM-4.5-GGUF/IQ2_KL/GLM-4.5-IQ2_KL-00001-of-00003.gguf --ctx-size 32768 --jinja --n-gpu-layers 99 --no-mmap --cache-type-k q8_0 --cache-type-v q8_0 -ot "blk.[0-6].ffn_up_exps=CUDA0,blk.[0-6].ffn_gate_exps=CUDA0,blk.[0-6].ffn_down_exps=CUDA0" -ot ".ffn_.*_exps.=CPU"
Am I doing something wrong here?
I am! :smile:
My systemd service was pointing to standard llama. Dumb mistake. Sorry for the noise!
arichiardi
changed discussion status to
closed
All good, thanks for stopping in and glad you got it going! This stuff is confusing and changes so quickly so make as much noise and take up some space all is well and all matter of thing shall be welle!