JunHowie

/

Qwen3-30B-A3B-Instruct-2507-GPTQ-Int4

like 0

Loading safetensors checkpoint shards: 0% Completed | 0/5 [00:00<?, ?it/s]
ERROR 12-01 21:56:09 [core.py:632] EngineCore failed to start.
ERROR 12-01 21:56:09 [core.py:632] Traceback (most recent call last):
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 623, in run_engine_core
ERROR 12-01 21:56:09 [core.py:632] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 441, in init
ERROR 12-01 21:56:09 [core.py:632] super().init(vllm_config, executor_class, log_stats,
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 77, in init
ERROR 12-01 21:56:09 [core.py:632] self.model_executor = executor_class(vllm_config)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in init
ERROR 12-01 21:56:09 [core.py:632] self._init_executor()
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 49, in _init_executor
ERROR 12-01 21:56:09 [core.py:632] self.collective_rpc("load_model")
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 58, in collective_rpc
ERROR 12-01 21:56:09 [core.py:632] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/utils/init.py", line 2985, in run_method
ERROR 12-01 21:56:09 [core.py:632] return func(*args, **kwargs)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 201, in load_model
ERROR 12-01 21:56:09 [core.py:632] self.model_runner.load_model(eep_scale_up=eep_scale_up)
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1876, in load_model
ERROR 12-01 21:56:09 [core.py:632] self.model = model_loader.load_model(
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 49, in load_model
ERROR 12-01 21:56:09 [core.py:632] self.load_weights(model, model_config)
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 259, in load_weights
ERROR 12-01 21:56:09 [core.py:632] loaded_weights = model.load_weights(
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 543, in load_weights
ERROR 12-01 21:56:09 [core.py:632] return loader.load_weights(weights)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 291, in load_weights
ERROR 12-01 21:56:09 [core.py:632] autoloaded_weights = set(self._load_module("", self.module, weights))
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 249, in _load_module
ERROR 12-01 21:56:09 [core.py:632] yield from self._load_module(prefix,
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 222, in _load_module
ERROR 12-01 21:56:09 [core.py:632] loaded_params = module_load_weights(weights)
ERROR 12-01 21:56:09 [core.py:632] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-01 21:56:09 [core.py:632] File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 477, in load_weights
ERROR 12-01 21:56:09 [core.py:632] param = params_dict[name]
ERROR 12-01 21:56:09 [core.py:632] ~~~~~~~~~~~^^^^^^
ERROR 12-01 21:56:09 [core.py:632] KeyError: 'layers.0.mlp.gate.g_idx'
Process EngineCore_0:
Traceback (most recent call last):
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 636, in run_engine_core
raise e
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 623, in run_engine_core
engine_core = EngineCoreProc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 441, in init
super().init(vllm_config, executor_class, log_stats,
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 77, in init
self.model_executor = executor_class(vllm_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in init
self._init_executor()
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 49, in _init_executor
self.collective_rpc("load_model")
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 58, in collective_rpc
answer = run_method(self.driver_worker, method, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/utils/init.py", line 2985, in run_method
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 201, in load_model
self.model_runner.load_model(eep_scale_up=eep_scale_up)
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1876, in load_model
self.model = model_loader.load_model(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 49, in load_model
self.load_weights(model, model_config)
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 259, in load_weights
loaded_weights = model.load_weights(
^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 543, in load_weights
return loader.load_weights(weights)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 291, in load_weights
autoloaded_weights = set(self._load_module("", self.module, weights))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 249, in _load_module
yield from self._load_module(prefix,
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 222, in _load_module
loaded_params = module_load_weights(weights)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/miniforge3/envs/vllm_jane/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 477, in load_weights
param = params_dict[name]
~~~~~~~~~~~^^^^^^
KeyError: 'layers.0.mlp.gate.g_idx'