PaddlePaddle

/

PaddleOCR-VL

like 1.03k

Follow

PaddlePaddle 654

docker run
-it
--rm
--gpus all
--network host
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --host 0.0.0.0 --port 8118 --backend vllm

Using official model (PaddleOCR-VL), the model files will be automatically downloaded and saved in /root/.paddlex/official_models/PaddleOCR-VL.
<Response [404]>
Encounter exception when download model from aistudio:
repo not found.
PaddleX would try to download from other model sources.
Using official model (PaddleOCR-VL), the model files will be automatically downloaded and saved in /root/.paddlex/official_models/PaddleOCR-VL.
Downloading Model from https://www.modelscope.cn to directory: /tmp/tmpqkkprmu2/temp_dir
2025-10-22 06:59:04,973 - modelscope - INFO - Got 21 files, start to download ...
Downloading [PP-DocLayoutV2/config.json]: 100%|████████████████████| 4.43k/4.43k [00:01<00:00, 3.02kB/s]
Downloading [configuration_paddleocr_vl.py]: 100%|█████████████████| 7.91k/7.91k [00:01<00:00, 5.12kB/s]
Downloading [generation_config.json]: 100%|█████████████████████████████| 112/112 [00:01<00:00, 63.4B/s]
Downloading [config.json]: 100%|███████████████████████████████████| 1.97k/1.97k [00:01<00:00, 1.04kB/s]
Downloading [chat_template.jinja]: 100%|█████████████████████████████████| 799/799 [00:02<00:00, 399B/s]
Downloading [image_processing.py]: 100%|███████████████████████████| 24.4k/24.4k [00:02<00:00, 12.3kB/s]
Downloading [added_tokens.json]: 100%|█████████████████████████████| 24.8k/24.8k [00:02<00:00, 10.5kB/s]
Downloading [inference.yml]: 100%|████████████████████████████████████| 39.0/39.0 [00:01<00:00, 32.6B/s]
Downloading [PP-DocLayoutV2/inference.yml]: 100%|████████████████████| 1.45k/1.45k [00:01<00:00, 935B/s]
Downloading [modeling_paddleocr_vl.py]: 100%|████████████████████████| 110k/110k [00:01<00:00, 80.1kB/s]
Downloading [preprocessor_config.json]: 100%|████████████████████████████| 687/687 [00:02<00:00, 340B/s]
Downloading [processor_config.json]: 100%|███████████████████████████████| 137/137 [00:01<00:00, 115B/s]
Downloading [README.md]: 100%|█████████████████████████████████████| 21.5k/21.5k [00:01<00:00, 15.8kB/s]
Downloading [special_tokens_map.json]: 100%|█████████████████████████| 1.12k/1.12k [00:01<00:00, 970B/s]
Downloading [PP-DocLayoutV2/inference.pdmodel]: 100%|███████████████| 1.44M/1.44M [00:03<00:00, 384kB/s]
Downloading [processing_paddleocr_vl.py]: 100%|████████████████████| 12.0k/12.0k [00:03<00:00, 3.66kB/s]
Downloading [tokenizer.json]: 100%|████████████████████████████████| 10.7M/10.7M [00:03<00:00, 3.55MB/s]
Downloading [tokenizer.model]: 100%|████████████████████████████████| 1.54M/1.54M [00:03<00:00, 467kB/s]
Downloading [tokenizer_config.json]: 100%|███████████████████████████| 181k/181k [00:03<00:00, 60.1kB/s]
Downloading [PP-DocLayoutV2/inference.pdiparams]: 100%|██████████████| 202M/202M [02:32<00:00, 1.39MB/s]
Downloading [model.safetensors]: 100%|█████████████████████████████| 1.79G/1.79G [14:01<00:00, 2.28MB/s]
Processing 21 items: 100%|███████████████████████████████████████████| 21.0/21.0 [14:03<00:00, 40.2s/it]
2025-10-22 07:13:08,833 - modelscope - INFO - Download model 'PaddlePaddle/PaddleOCR-VL' successfully.s]
INFO 10-22 07:13:11 [__init__.py:216] Automatically detected platform cuda./1.79G [02:27<53:41, 560kB/s]
(APIServer pid=1) INFO 10-22 07:13:12 [api_server.py:1896] vLLM API server version 0.10.20:00, 2.09MB/s]
(APIServer pid=1) INFO 10-22 07:13:12 [utils.py:328] non-default args: {'api_server_count': 4, 'host': '0.0.0.0', 'port': 8118, 'chat_template': '/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/chat_templates/PaddleOCR-VL-0.9B.jinja', 'model': '/root/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B', 'trust_remote_code': True, 'max_model_len': 16384, 'served_model_name': ['PaddleOCR-VL-0.9B'], 'gpu_memory_utilization': 0.5, 'max_num_batched_tokens': 131072}
(APIServer pid=1) The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=1) Traceback (most recent call last):
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 479, in cached_files
(APIServer pid=1) hf_hub_download(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
(APIServer pid=1) validate_repo_id(arg_value)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
(APIServer pid=1) raise HFValidationError(
(APIServer pid=1) huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B'. Use repo_type argument if needed.
(APIServer pid=1)
(APIServer pid=1) During handling of the above exception, another exception occurred:
(APIServer pid=1)
(APIServer pid=1) Traceback (most recent call last):
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 721, in _get_config_dict
(APIServer pid=1) resolved_config_file = cached_file(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 322, in cached_file
(APIServer pid=1) file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 531, in cached_files
(APIServer pid=1) resolved_files = [
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 532, in
(APIServer pid=1) _get_cache_file_to_return(path_or_repo_id, filename, cache_dir, revision, repo_type)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 143, in _get_cache_file_to_return
(APIServer pid=1) resolved_file = try_to_load_from_cache(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
(APIServer pid=1) validate_repo_id(arg_value)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
(APIServer pid=1) raise HFValidationError(
(APIServer pid=1) huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B'. Use repo_type argument if needed.
(APIServer pid=1)
(APIServer pid=1) During handling of the above exception, another exception occurred:
(APIServer pid=1)
(APIServer pid=1) Traceback (most recent call last):
(APIServer pid=1) File "/usr/local/bin/paddlex_genai_server", line 8, in
(APIServer pid=1) sys.exit(main())
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/server.py", line 113, in main
(APIServer pid=1) run_genai_server(args)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/server.py", line 100, in run_genai_server
(APIServer pid=1) run_server_func(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/backends/vllm.py", line 68, in run_vllm_server
(APIServer pid=1) uvloop.run(run_server(args))
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/uvloop/init.py", line 69, in run
(APIServer pid=1) return loop.run_until_complete(wrapper())
(APIServer pid=1) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=1) return await main
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 1941, in run_server
(APIServer pid=1) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 1961, in run_server_worker
(APIServer pid=1) async with build_async_engine_client(
(APIServer pid=1) File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=1) return await anext(self.gen)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 179, in build_async_engine_client
(APIServer pid=1) async with build_async_engine_client_from_engine_args(
(APIServer pid=1) File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=1) return await anext(self.gen)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 205, in build_async_engine_client_from_engine_args
(APIServer pid=1) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 1119, in create_engine_config
(APIServer pid=1) model_config = self.create_model_config()
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 963, in create_model_config
(APIServer pid=1) return ModelConfig(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/pydantic/_internal/_dataclasses.py", line 121, in init
(APIServer pid=1) s.pydantic_validator.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/config/init.py", line 603, in post_init
(APIServer pid=1) self.model, self.tokenizer = maybe_override_with_speculators_target_model( # noqa: E501
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/vllm/transformers_utils/config.py", line 487, in maybe_override_with_speculators_target_model
(APIServer pid=1) config_dict, _ = PretrainedConfig.get_config_dict(
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 662, in get_config_dict
(APIServer pid=1) config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
(APIServer pid=1) File "/usr/local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 744, in _get_config_dict
(APIServer pid=1) raise OSError(
(APIServer pid=1) OSError: Can't load the configuration of '/root/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/root/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B' is the correct path to a directory containing a config.json file