No longer works with vllm
#5
by
bgreene010
- opened
FYI, after an update in vllm, it crashes with this model, with the following error.
RuntimeError: No Qwen3Next layer found in the model.layers
This is the first commit in vllm where this happens:
commit e50c45467215f96068d95736b08d8a25f624e67d
Author: Ilya Markov <[email protected]>
Date: Wed Nov 5 16:22:17 2025 +0100
[BugFix] Support EP/DP + EPLB with MTP (#25311)
Signed-off-by: ilmarkov <[email protected]>
Signed-off-by: Sage Moore <[email protected]>
Co-authored-by: Sage Moore <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
https://github.com/vllm-project/vllm/pull/28960 - you can fix it in venv or in docker image manually. Very simple.