Docker Inference - Unknown model: PaddleOCR-VL
I tried to stand up the VLM Inference Service and got error "Unknown model: PaddleOCR-VL"
I am in WSL on a Windows 11 machine.
(paddleocr) aiuser@WINDOW11COMPUTER:~/paddleocr$ docker run \
-it \
--rm \
--gpus all \
--network host \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
Unable to find image 'ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest' locally
latest: Pulling from paddlepaddle/paddlex-genai-vllm-server
e335b9092447: Pull complete
32b550be6cb6: Pull complete
35af2a7690f2: Pull complete
7576b00d9bb1: Pull complete
ff1cff00b1dd: Pull complete
57e75b8db17c: Pull complete
e844e4b3e270: Pull complete
4ffff3de336e: Pull complete
d3432cb2d2d4: Pull complete
1590ff102653: Pull complete
Digest: sha256:14536af5e9d2574450a9bac907fc8632cc875094d3911d35ab1e27a3edce1443
Status: Downloaded newer image for ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest
Failed to get model directory
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/server.py", line 67, in run_genai_server
model_dir = get_model_dir(args.model_name, args.backend)
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/models/__init__.py", line 43, in get_model_dir
_check_model_name_and_backend(model_name, backend)
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/models/__init__.py", line 37, in _check_model_name_and_backend
raise ValueError(f"Unknown model: {model_name}")
ValueError: Unknown model: PaddleOCR-VL
It seems to be a network issue that has prevented the model from downloading successfully.
You can follow the steps below to check step by step whether the model downloaded successfully:
docker run \
-it \
--rm \
--gpus all \
--network host \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server \
/bin/bash
Then execute this command in the container to confirm whether the model has been successfully downloaded and the vllm service has been started.
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8118
command is: paddlex_genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8118
i think problem is in docker image. There is CMD ["paddlex_genai_server" "--model_name" "PaddleOCR-VL" " ...
should be CMD ["paddlex_genai_server" "--model_name" "PaddleOCR-VL-0.9B" ....
@kuopching As you mentioned, the Docker image will automatically execute:
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8080
@eddprogrammer You can also try pulling the latest image again:
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest
@xiaohei66 in docker image [1] for CMD command there is PaddleOCR-VL instead of PaddleOCR-VL-0.9B so error occurs [2] . I have to change docker image [3] to run it succesfully . Now it works like charm.
- ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest
- Unknown model: PaddleOCR-VL
- FROM ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest
CMD [ "paddlex_genai_server", "--model_name", "PaddleOCR-VL-0.9B", "--host", "0.0.0.0", "--port", "8080", "--backend", "vllm" ]
docker run \
-it \
--rm \
--gpus all \
--network host \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
and
docker run \
-it \
--rm \
--gpus all \
--network host \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server \
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --host 0.0.0.0 --port 8118 --backend vllm
both are correct methods. ValueError: Unknown model: PaddleOCR-VL The error might be caused by an outdated image—we pushed a new one yesterday.
Great. THis one works like a charm.
docker run
-it
--rm
--gpus all
--network host
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --host 0.0.0.0 --port 8118 --backend vllm
The other one still doesn't work..
We will push a new image today to fix this problem, then you can use this command to deploy a PaddleOCR-VL-0.9B vllm server:
docker run \
-it \
--rm \
--gpus all \
--network host \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
Thanks!