Update README and sample assets

README.md

@@ -2,79 +2,98 @@
 library_name: transformers
 license: apache-2.0
 base_model:
-- openbmb/MiniCPM-V-4.6
+  - openbmb/MiniCPM-V-4.6
 pipeline_tag: image-text-to-text
 tags:
-- minicpm-v
-- vlm
-- axera
-- AX650
+  - minicpm-v
+  - vlm
+  - video-understanding
+  - axera
+  - ax650
 language:
-- zh
-- en
+  - zh
+  - en
 ---
 
 # MiniCPM-V-4.6 on AXERA NPU
 
-Ready-to-run AX650 deployment package for `openbmb/MiniCPM-V-4.6`.
+Ready-to-run deployment package for `openbmb/MiniCPM-V-4.6` on AX650 / NPU3.
 
-- Runtime: `axllm serve` and `axllm run`
-- Target: `AX650 / AX650N aarch64`
-- Text backend: Qwen3.5 dense LLM, `bf16`, non-GPTQ weights
-- Vision backend: fixed-shape `448x448` MiniCPM-V-4.6 vision encoder
-- Context layout: `prefill_len=128`, `kv_cache_len=2047`, `prefill_max_token_num=1280`
-- Validated capabilities in this release:
-  - text-only chat
-  - long prompt multi-chunk prefill
-  - single-image understanding
-  - video understanding
-- Included board binary:
-  - `bin/axllm` for `AX650/AX650N aarch64`
-
-This package has been validated for text chat, single-image understanding, and video understanding through the OpenAI-compatible `axllm serve` API.
+- This release packages the AX650 `axllm` runtime together with the compiled text and vision `.axmodel` files.
+- The packaged text runtime uses the non-GPTQ BF16 build.
+- The packaged vision runtime uses a fixed-shape `448x448` MiniCPM-V-4.6 vision encoder.
+- The package supports text-only chat, single-image understanding, and video understanding through the OpenAI-compatible `axllm serve` API.
+- The package also includes board-side and server-side Python reference scripts for debugging and comparison.
 
 ## Supported Platform
 
-- [x] AX650 / AX650N
+- [x] AX650 / NPU3
+
+## Validated Devices
+
+This package has been validated on the following AX650-based device:
+
+- AX650 / NPU3 development board
 
-## Performance and Footprint
+## Performance
 
-Measurements below are from the preserved AX650 runtime log in [Inference Log](#inference-log), plus a repeated text-only TTFT recheck on `2026-06-05`. `TTFT` means time to first token.
-The text-only smoke prompt is intentionally kept within one `128`-token prefill group. Its TTFT is the 5-run average for the prompt `1+1等于几？只输出数字。`; the response is one token, so decode throughput is not reported for that row.
+All measurements below were taken on AX650 / NPU3. `TTFT` stands for time to first token.
+
+The text-only smoke prompt was kept within one `128`-token prefill chunk. The image and video rows were measured with the packaged fixed-shape `448x448` vision encoder.
 
 | Scenario | Input tokens | Prefill chunks | TTFT | Decode |
 |---|---:|---:|---:|---:|
 | Text-only smoke prompt | `22` | `1 x 128` | `240.45 ms avg` (`239.20-242.75 ms`) | `n/a` |
-| Video red-panda prompt | `1278` | `10 x 128` | `1795.28 ms` | `18.79 token/s` |
-| Image prompt | `358` | `3 x 128` | `518.23 ms` | `18.64 token/s` |
+| Image prompt | `88` | `1 x 128` | `232.54 ms avg` (`231.53-233.22 ms`) | `18.97 token/s avg` |
+| Video prompt | `1274` | `10 x 128` | `2468.50 ms` | `18.62 token/s` |
 
-### Flash Footprint
+The packaged runtime uses the following context layout:
 
-| Item | Value |
-|---|---:|
-| Text + post + vision axmodels | `1460.19 MiB` |
-| Package flash footprint, excluding `vision_cache/` | `1999.34 MiB` |
+- `prefill_len=128`
+- `kv_cache_len=2047`
+- `prefill_max_token_num=1280`
 
 ### Startup Runtime Footprint
 
-Measured on AX650 on `2026-06-05` by reading `/proc/ax_proc/mem_cmm_info` before startup and after the `/health` endpoint became ready. The measurement kept the existing GPTQ service on port `8000` running and records only the incremental CMM used by this extra service.
-
 | Item | Value |
 |---|---:|
-| Baseline CMM before startup | `1823 MB` |
-| CMM after `/health` ready | `3388 MB` |
-| Incremental runtime CMM | `1565 MB` |
-| Startup log `remain_cmm` after post axmodel init | `5388 MB` |
+| `Flash total (text + post + vision axmodels)` | `1.43 GiB` (`1460.19 MiB`) |
+| `Package flash total (excluding vision_cache/)` | `1.93 GiB` (`1980.33 MiB`) |
+| `Runtime CMM increment during board-side startup` | `1565 MB` |
 
-The preserved standalone inference log below records `remain_cmm(6933 MB)` for an earlier single-service startup; use the incremental CMM row above for the board-side footprint measured during this README refresh.
+The runtime CMM value above was measured during board-side startup on a shared AX650 system and should be treated as a practical reference value.
 
-The runtime log also records the video sampling decision:
+## Vision Encoder Latency
+
+Measured on AX650 / NPU3 with `/opt/bin/ax_run_model -m minicpmv4_6_vision_448.axmodel -g 0 -w 1 -r 5`.
+
+| Model | Resolution | Soft Tokens | Time (ms) |
+|---|---|---|---:|
+| `minicpmv4_6_vision_448.axmodel` | `448x448` | `64` | `526.717 ms avg` |
+
+For this packaged AX650 runtime, the visual token count is fixed by the shipped vision encoder configuration:
+
+- `vision_width = 448`
+- `vision_height = 448`
+- `vision_patch_size = 14`
+- patch grid = `(448 / 14) x (448 / 14) = 32 x 32`
+- raw patch tokens = `32 x 32 = 1024`
+- current packaged build uses the `16x` visual compression path
+- `Soft Tokens = 1024 / 16 = 64`
+
+So, for the fixed-shape runtime shipped in this repository, the relation is:
 
 ```text
-MiniCPM-V-4.6 video frames selected: 19/59 (configured_cap=0, tail_tokens=1278, max_tail=1280, precompute_len=0)
+Soft Tokens = (vision_width / patch_size) x (vision_height / patch_size) / 16
 ```
 
-## Repository Layout
+For the packaged sample image request used in this README, the board-side runtime log reports `input_num_token=88`. This is expected: the final request token count is larger than the visual soft-token count because the full prompt also includes user text and chat-template tokens. For this packaged sample, the complete image request still fits within a single `128`-token prefill chunk.
+
+`Soft Tokens` is not a runtime-configurable value in this package. This repository ships only `minicpmv4_6_vision_448.axmodel`, so the board-side AX650 runtime always uses `448x448 -> 64` soft tokens for image encoding.
+
+The upstream Hugging Face MiniCPM-V-4.6 processor supports other visual compression and slicing settings, but those are not exposed as a runtime option by this packaged fixed-shape AX650 release.
+
+## Package Layout
 
 ```text
 .
@@ -83,7 +102,9 @@ MiniCPM-V-4.6 video frames selected: 19/59 (configured_cap=0, tail_tokens=1278,
 │   ├── axllm
 │   └── axllm.version.json
 ├── assets/
-│   └── smoke_image.png
+│   ├── openai_api_demo.png
+│   ├── red-panda-openai.mp4
+│   └── sample.png
 ├── python/
 │   ├── infer_axmodel.py
 │   ├── infer_torch.py
@@ -99,7 +120,36 @@ MiniCPM-V-4.6 video frames selected: 19/59 (configured_cap=0, tail_tokens=1278,
 └── minicpm_v46_tokenizer.txt
 ```
 
-## Install `axllm`
+This package uses a hybrid layout: the packaged `axllm` runtime plus the compiled `.axmodel` files live at the repository root, while the Python reference scripts and the tokenizer directory used by those scripts stay under `python/`.
+
+## Sample Image
+
+Both the `axllm` flow and the packaged Python examples can use the sample image:
+`assets/sample.png`
+
+![sample](assets/sample.png)
+
+## Sample Video
+
+The package also includes a packaged sample video for board-side video understanding validation:
+
+- `assets/red-panda-openai.mp4`
+
+## Direct Inference with `axllm`
+
+> The `axllm` workflow is still being refined. The instructions below reflect the current validated flow and may be adjusted as the packaging continues to evolve.
+
+### Download the Model Package
+
+Download the release package from Hugging Face:
+
+```shell
+mkdir -p AXERA-TECH/MiniCPM-V-4.6
+cd AXERA-TECH/MiniCPM-V-4.6
+hf download AXERA-TECH/MiniCPM-V-4.6 --local-dir .
+```
+
+### Install `axllm`
 
 Option 1: use the validated binary included in this repository:
 
@@ -107,18 +157,53 @@ Option 1: use the validated binary included in this repository:
 chmod +x ./bin/axllm
 ```
 
-Option 2: build or install the latest `ax-llm` yourself:
+Option 2: install `axllm` from the public repository:
 
-- Repo: `https://github.com/AXERA-TECH/ax-llm`
-- The included binary metadata is recorded in `bin/axllm.version.json`.
+```shell
+git clone -b axllm https://github.com/AXERA-TECH/ax-llm.git
+cd ax-llm
+./install.sh
+```
 
-## Run on AX650
+Option 3: install with a one-line command:
 
-From the repository root on the board:
+```shell
+curl -fsSL https://raw.githubusercontent.com/AXERA-TECH/ax-llm/axllm/install.sh | bash
+```
+
+Option 4: download the prebuilt binary from GitHub Actions CI:
+
+If you do not have a local build environment, download the latest CI-generated `axllm` binary from GitHub Actions:
+`https://github.com/AXERA-TECH/ax-llm/actions?query=branch%3Aaxllm`
+Then run:
+
+```shell
+chmod +x axllm
+sudo mv axllm /usr/bin/axllm
+```
+
+### Run on the Board
+
+The package root is already arranged for `axllm`, so no extra runtime path arguments are required.
+
+For multimodal testing, you can use the packaged sample image shown above: `./assets/sample.png`, or the packaged sample video: `./assets/red-panda-openai.mp4`.
 
 ```bash
-chmod +x ./bin/axllm
-./bin/axllm serve . --port 18080
+./bin/axllm run .
+```
+
+In interactive mode:
+
+- press `Enter` directly for text-only chat
+- input an image path for single-image chat
+- input `video:/path/to/frames_dir` or `video:/path/to/video.mp4` for video chat
+
+### Serve with `axllm`
+
+From the package root on the board:
+
+```bash
+./bin/axllm serve . --port 8000
 ```
 
 Expected model id:
@@ -127,10 +212,72 @@ Expected model id:
 AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047
 ```
 
+Health check:
+
+```bash
+curl http://127.0.0.1:8000/health
+```
+
+A typical startup log looks like this:
+
+```text
+INF Init | LLM init start
+INF Init | mixed attention enabled: full_attention_interval=4 ref_full_layer_idx=3
+INF Init | attention config: layers=24 sliding=0 full=6 linear=18 sliding_window=0 ref_full_layer_idx=3
+tokenizer_type = 3
+huggingface tokenizer mode = gpt2_byte_bpe
+...
+INF Init | max_token_len : 2047
+INF Init | kv_cache_size : 512, kv_cache_num: 2047
+INF init_groups_from_model | prefill_token_num : 128
+INF init_groups_from_model | prefill_max_token_num : 1280
+INF Init | MiniCPM-V-4.6 token ids: image_pad=248056 video_pad=248057
+INF Init | VisionModule init ok: type=MiniCPMV46VL, tokens_per_block=64, embed_size=1024, out_dtype=fp32
+INF Init | LLM init ok
+Starting server on port 8000 with model 'AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047'...
+API URLs:
+  GET  http://127.0.0.1:8000/health
+  GET  http://127.0.0.1:8000/v1/models
+  POST http://127.0.0.1:8000/v1/chat/completions
+OpenAI API Server starting on http://0.0.0.0:8000
+Max concurrency: 1
+Models: AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047
+```
+
+You can then send requests to the server using the API endpoints shown in the log. For example, to check the health status and list the available models:
+
+```bash
+curl http://127.0.0.1:8000/health
+curl http://127.0.0.1:8000/v1/models
+```
+
+Example output:
+
+```json
+{
+  "concurrency": 0,
+  "max_concurrency": 1,
+  "status": "healthy"
+}
+{
+  "data": [
+    {
+      "created": 1780908633,
+      "id": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
+      "object": "model",
+      "owned_by": "openai-api"
+    }
+  ],
+  "object": "list"
+}
+```
+
+![openai_api_demo](assets/openai_api_demo.png)
+
 ### Text Request
 
 ```bash
-curl http://127.0.0.1:18080/v1/chat/completions \
+curl http://127.0.0.1:8000/v1/chat/completions \
   -H 'Content-Type: application/json' \
   -d '{
     "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
@@ -138,7 +285,7 @@ curl http://127.0.0.1:18080/v1/chat/completions \
       {
         "role": "user",
         "content": [
-          {"type": "text", "text": "1+1等于几？只输出数字。"}
+          {"type": "text", "text": "What is 1+1? Reply with the number only."}
         ]
       }
     ],
@@ -146,16 +293,34 @@ curl http://127.0.0.1:18080/v1/chat/completions \
   }'
 ```
 
+Example output:
+
+```json
+{
+  "choices": [
+    {
+      "message": {
+        "role": "assistant",
+        "content": "1+1=2"
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
+  "object": "chat.completion"
+}
+```
+
 ### Image Request
 
 ```bash
-python - <<'PY'
+python3 - <<'PY'
 import base64
 import json
 from pathlib import Path
 from urllib.request import Request, urlopen
 
-img = Path("assets/smoke_image.png").read_bytes()
+img = Path("assets/sample.png").read_bytes()
 payload = {
     "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
     "messages": [
@@ -175,7 +340,7 @@ payload = {
     "max_tokens": 64,
 }
 req = Request(
-    "http://127.0.0.1:18080/v1/chat/completions",
+    "http://127.0.0.1:8000/v1/chat/completions",
     data=json.dumps(payload).encode(),
     headers={"Content-Type": "application/json"},
 )
@@ -184,12 +349,30 @@ with urlopen(req, timeout=60) as resp:
 PY
 ```
 
+Example output:
+
+```json
+{
+  "choices": [
+    {
+      "message": {
+        "role": "assistant",
+        "content": "好的，这张图片是一个风格化的红色龙虾卡通形象。它有着夸张的表情和动态的姿势，显得非常活泼和有力。龙虾的肢体姿态显示出它正在准备出击或展示它的力量，整体设计充满了动感和趣味性。这个形象可能用于装饰或象征某种活力和力量。"
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
+  "object": "chat.completion"
+}
+```
+
 ### Video Request
 
-`axllm serve` supports a frames directory or a raw video file:
+`axllm serve` accepts either a frames directory or a raw video file:
 
 ```bash
-curl http://127.0.0.1:18080/v1/chat/completions \
+curl http://127.0.0.1:8000/v1/chat/completions \
   -H 'Content-Type: application/json' \
   -d '{
     "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
@@ -206,36 +389,56 @@ curl http://127.0.0.1:18080/v1/chat/completions \
   }'
 ```
 
-For a raw video file, use `video:/path/to/video.mp4` or `video:/path/to/video.mp4:2` to request a sampling FPS of `2`.
+For a raw video file, use `video:/path/to/video.mp4`. If you need to request a specific sampling FPS, use the form `video:/path/to/video.mp4:2`.
 
-### Interactive CLI
+To test the packaged sample video from the package root, you can set:
 
 ```bash
-./bin/axllm run .
+VIDEO_PATH="$(pwd)/assets/red-panda-openai.mp4"
 ```
 
-After each prompt:
+and then use `video:${VIDEO_PATH}:2` in the request payload.
+
+Example output:
+
+```json
+{
+  "choices": [
+    {
+      "message": {
+        "role": "assistant",
+        "content": "好的，这是当前对视频内容的详细描述：画面中，两只红熊猫正在一个由竹竿搭建的攀爬架周围活动。一只红熊猫正趴在竹竿上，身体伸展，尾巴自然垂落；另一只红熊猫则蹲在下方，抬头向上，似乎正在尝试攀爬或探索竹竿结���。背景是绿色的围栏和草地，整个场景展现了它们活泼、好奇的互动状态。"
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "model": "AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047",
+  "object": "chat.completion"
+}
+```
 
-- press `Enter` directly for text-only chat
-- input an image path for single-image chat
-- input `video:<frames_dir>` for video chat
+### Browser UI with `lite_webui`
 
-## Python Reference Scripts
+If you want a browser UI for the OpenAI-compatible service started by `axllm serve`, use [AXERA-TECH/lite_webui](https://huggingface.co/AXERA-TECH/lite_webui/tree/main).
 
-### Official PyTorch Reference
+Set the OpenAI base URL to `http://<board-ip>:8000` and the model name to `AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047`.
 
-`python/infer_torch.py` is a developer-side reference script. It requires the full upstream HuggingFace model with weights and is intended for x86/GPU validation.
+## Python Runtime Requirements
 
-```bash
-cd python
-python infer_torch.py \
-  --model-path /data/tmp/yongqiang/nfs/auto_model_deployment/Minicpm-V-4.6-hf-original/MiniCPM-V-4.6 \
-  --prompt "1+1等于几？请直接回答。"
-```
+Install the following packages before using the packaged Python reference scripts:
+
+- Board-side `infer_axmodel.py`:
+  `pyaxengine`, `transformers`, `numpy`, `ml_dtypes`
+- Server-side `infer_torch.py`:
+  `torch`, `transformers`
 
-### Python `.axmodel` Debug Runner
+The packaged Python scripts are reference utilities rather than the main runtime path.
 
-`python/infer_axmodel.py` is a text-only per-layer debug runner for the compiled LLM package:
+## Legacy Python Demo Flow
+
+### Text-Only Inference
+
+`python/infer_axmodel.py` is intended for board-side text debugging of the packaged runtime files:
 
 ```bash
 cd python
@@ -243,159 +446,43 @@ python3 infer_axmodel.py \
   --hf-model ./minicpm_v46_tokenizer \
   --axmodel-dir .. \
   --mode generate \
-  --prompt "1+1等于几？请直接回答。" \
+  --prompt "What is 1+1? Reply with the number only." \
   --prompt-mode prefill \
   --max-new-tokens 16 \
   --kv-cache-len 2047
 ```
 
-## Notes
+### Hugging Face Reference Inference
 
-1. `.axmodel` files are board-only. Do not run them on x86.
-2. This package was compiled with `--prefill_len 128 --kv_cache_len 2047`.
-3. The AX650 build keeps `FLOAT_MATMUL_USE_CONV_EU=1` enabled to reduce TTFT.
-4. `bin/axllm.version.json` records the exact runtime binary commit and verification metadata.
-5. The packaged vision encoder is fixed at `448x448`; arbitrary image/video sizes are resized by the runtime preprocessor.
-6. Video requests are isolated to the current user turn and drop KV cache after the response to avoid stale multimodal history reuse.
+`python/infer_torch.py` is intended for x86 or GPU-side comparison against the original Hugging Face model:
 
+```bash
+cd python
+python infer_torch.py \
+  --model-path /path/to/original/MiniCPM-V-4.6 \
+  --prompt "Please give a short self introduction."
+```
 
-## Inference Log
+## Packaged Python Runtime Paths
 
-```sh
-root@ax650 ~/yongqiang/auto_model_deployment/MiniCPM-V-4.6 # ./bin/axllm serve .
-14:59:39.577 INF Init:2239 | LLM init start
-14:59:39.577 INF Init:2248 | mixed attention enabled: full_attention_interval=4 ref_full_layer_idx=3
-14:59:39.577 INF Init:2270 | attention config: layers=24 sliding=0 full=6 linear=18 sliding_window=0 ref_full_layer_idx=3
-tokenizer_type = 3
-huggingface tokenizer mode = gpt2_byte_bpe
- 44% | ##############                   |  12 /  27 [4.73s<10.63s, 2.54 count/s] init 10 axmodel ok,remain_cmm(7587 MB                                                                                                                       96% | ##############################   |  26 /  27 [6.15s<6.39s, 4.23 count/s] init post axmodel ok,remain_cmm(6933 MB)
-14:59:45.729 INF Init:2420 | max_token_len : 2047
-14:59:45.729 INF Init:2423 | kv_cache_size : 512, kv_cache_num: 2047
-14:59:45.729 INF init_groups_from_model:1622 | prefill_token_num : 128
-14:59:45.729 INF init_groups_from_model:1870 | decode grp: 0, gid: 0, max_token_len : 2047
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 0, gid: 1, history_cap: 0, total_cap: 128, symbolic_cap: 1
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 1, gid: 2, history_cap: 128, total_cap: 256, symbolic_cap: 128
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 2, gid: 3, history_cap: 256, total_cap: 384, symbolic_cap: 256
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 3, gid: 4, history_cap: 384, total_cap: 512, symbolic_cap: 384
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 4, gid: 5, history_cap: 512, total_cap: 640, symbolic_cap: 512
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 5, gid: 6, history_cap: 640, total_cap: 768, symbolic_cap: 640
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 6, gid: 7, history_cap: 768, total_cap: 896, symbolic_cap: 768
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 7, gid: 8, history_cap: 896, total_cap: 1024, symbolic_cap: 896
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 8, gid: 9, history_cap: 1024, total_cap: 1152, symbolic_cap: 1024
-14:59:45.729 INF init_groups_from_model:1874 | prefill grp: 9, gid: 10, history_cap: 1152, total_cap: 1280, symbolic_cap: 1152
-14:59:45.729 INF init_groups_from_model:1881 | prefill_max_token_num : 1280
-14:59:45.729 INF init_layer_groups:1026 | layer 0 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 1 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 2 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 4 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 5 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 6 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 8 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 9 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 10 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 12 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 13 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.729 INF init_layer_groups:1026 | layer 14 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 16 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 17 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 18 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 20 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 21 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF init_layer_groups:1026 | layer 22 prefill groups=2 ref=10, reuse gid 2 for later prefill chunks
-14:59:45.730 INF Init:27 | LLaMaEmbedSelector use mmap
-100% | ################################ |  27 /  27 [6.16s<6.16s, 4.39 count/s] embed_selector init ok
-14:59:47.002 INF Init:1507 | MiniCPM-V-4.6 token ids: image_pad=248056 video_pad=248057
-14:59:47.002 INF Init:1514 | VisionModule init ok: type=MiniCPMV46VL, tokens_per_block=64, embed_size=1024, out_dtype=fp32
-14:59:47.002 WRN Init:1523 | Vision preprocess backend: SimpleCV (OpenCV not found at build time; minor differences vs OpenCV are possible)
-14:59:47.004 INF load_config:444 | load config:
-14:59:47.004 INF load_config:444 | {
-14:59:47.004 INF load_config:444 |     "enable_repetition_penalty": false,
-14:59:47.004 INF load_config:444 |     "enable_temperature": false,
-14:59:47.004 INF load_config:444 |     "enable_top_k_sampling": false,
-14:59:47.004 INF load_config:444 |     "enable_top_p_sampling": false,
-14:59:47.004 INF load_config:444 |     "penalty_window": 20,
-14:59:47.004 INF load_config:444 |     "repetition_penalty": 1.2,
-14:59:47.004 INF load_config:444 |     "temperature": 0.9,
-14:59:47.004 INF load_config:444 |     "top_k": 10,
-14:59:47.004 INF load_config:444 |     "top_p": 0.8
-14:59:47.004 INF load_config:444 | }
-14:59:47.004 INF Init:2532 | LLM init ok
-Starting server on port 8000 with model 'AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047'...
-API URLs:
-  GET  http://127.0.0.1:8000/health
-  GET  http://127.0.0.1:8000/v1/models
-  POST http://127.0.0.1:8000/v1/chat/completions
-  GET  http://10.168.232.217:8000/health
-  GET  http://10.168.232.217:8000/v1/models
-  POST http://10.168.232.217:8000/v1/chat/completions
-  GET  http://172.17.0.1:8000/health
-  GET  http://172.17.0.1:8000/v1/models
-  POST http://172.17.0.1:8000/v1/chat/completions
-Aliases:
-  GET  http://127.0.0.1:8000/models
-  POST http://127.0.0.1:8000/chat/completions
-  GET  http://10.168.232.217:8000/models
-  POST http://10.168.232.217:8000/chat/completions
-  GET  http://172.17.0.1:8000/models
-  POST http://172.17.0.1:8000/chat/completions
-OpenAI API Server starting on http://0.0.0.0:8000
-Max concurrency: 1
-Models: AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047
-15:00:35.571 INF operator():2136 | OpenAI chat request: model=AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047 stream=1 max_tokens=2048 has_temperature=0 temperature=1.0000 has_top_p=0 top_p=1.0000 messages=1 stop=0
-15:00:35.572 INF SetKVCache:3134 | decode_grpid:0 prefill_grpid:1 history_cap:0 total_cap:128 symbolic_cap:1 precompute_len:0 input_num_token:16 prefer_symbolic_group:0
-15:00:35.572 INF SetKVCache:3156 | current prefill_max_token_num:1280
-15:00:35.588 INF SetKVCache:3172 | first run
-15:00:35.602 INF Run:3376 | input token num : 16, prefill_split_num : 1
-15:00:35.602 INF Run:3387 | MiniCPM-V-4.6 short text prefill uses decode replay: input_tokens=16 prefill_tokens=128 precompute_len=0
-15:00:35.602 INF Run:3471 | prefill chunk p=0 history_len=0 grpid=1 kv_cache_num=0 input_tokens=16
-15:00:36.091 INF Run:3886 | ttft: 488.83 ms
-I'm a model from the MiniCPM series, developed by Modelbest and OpenBMB. For more details, you can visit https://github.com/OpenBMB/
-
-15:00:37.997 NTC Run:4298 | hit eos,decode avg 17.84 token/s
-15:00:37.997 INF GetKVCache:3087 | precompute_len:51, remaining:1229 (tracked)
-15:01:11.728 INF operator():2136 | OpenAI chat request: model=AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047 stream=1 max_tokens=2048 has_temperature=0 temperature=1.0000 has_top_p=0 top_p=1.0000 messages=3 stop=0
-15:01:11.941 WRN Run:4330 | video history is isolated to current user turn: old_history=3 new_history=1 old_media_inputs=1 new_media_inputs=1
-15:01:12.096 INF extract_video_frames_ffmpeg:386 | Extracting raw video container to frames: /tmp/axllm_media/media_863804170273309_0.mp4 -> /tmp/axllm_video_frames/video_863804326631725_0 fps=1.9907760708716282
-15:01:13.679 INF collect_video_frame_paths:466 | Video fps sampling: path=/tmp/axllm_media/media_863804170273309_0.mp4 fps=2 duration=30.139s target_frames=60 selected=59
-15:01:13.687 INF Prepare:2594 | MiniCPM-V-4.6 video frames selected: 19/59 (configured_cap=0, tail_tokens=1278, max_tail=1280, precompute_len=0)
-15:01:24.499 INF SetKVCache:3134 | decode_grpid:0 prefill_grpid:1 history_cap:0 total_cap:128 symbolic_cap:1 precompute_len:0 input_num_token:1278 prefer_symbolic_group:0
-15:01:24.499 INF SetKVCache:3156 | current prefill_max_token_num:1280
-15:01:24.516 INF SetKVCache:3172 | first run
-15:01:24.531 INF Run:3376 | input token num : 1278, prefill_split_num : 10
-15:01:24.531 INF Run:3471 | prefill chunk p=0 history_len=0 grpid=1 kv_cache_num=0 input_tokens=128
-15:01:24.689 INF Run:3471 | prefill chunk p=1 history_len=128 grpid=3 kv_cache_num=256 input_tokens=128
-15:01:24.865 INF Run:3471 | prefill chunk p=2 history_len=256 grpid=4 kv_cache_num=384 input_tokens=128
-15:01:25.041 INF Run:3471 | prefill chunk p=3 history_len=384 grpid=5 kv_cache_num=512 input_tokens=128
-15:01:25.219 INF Run:3471 | prefill chunk p=4 history_len=512 grpid=6 kv_cache_num=640 input_tokens=128
-15:01:25.398 INF Run:3471 | prefill chunk p=5 history_len=640 grpid=7 kv_cache_num=768 input_tokens=128
-15:01:25.578 INF Run:3471 | prefill chunk p=6 history_len=768 grpid=8 kv_cache_num=896 input_tokens=128
-15:01:25.759 INF Run:3471 | prefill chunk p=7 history_len=896 grpid=9 kv_cache_num=1024 input_tokens=128
-15:01:25.943 INF Run:3471 | prefill chunk p=8 history_len=1024 grpid=10 kv_cache_num=1152 input_tokens=128
-15:01:26.128 INF Run:3471 | prefill chunk p=9 history_len=1152 grpid=10 kv_cache_num=1152 input_tokens=126
-15:01:26.327 INF Run:3886 | ttft: 1795.28 ms
-好的，我很乐意为您描述这个视频的内容。
-
-这张图片展示了一个活泼可爱的红熊猫，它正积极地参与着某种游戏或探索活动。从画面来看，这只红熊猫有着标志性的红棕色和黑色相间的毛色，面部和胸部有白色的标记，非常醒目。它正用前爪紧紧抓住一根由竹子或类似材料搭建的木制梯子或支架，身体微微前倾，似乎正在用力或试图攀爬。它的姿态充满了动感和专注，显示出它对这个环境非常感兴趣。
-
-在画面的下方，还有一只较小的红熊猫，它正抬头向上看，似乎是在观察上方那只正在攀爬的同伴，或者是在等待自己的机会。这两只红熊猫被安置在一个绿色的草地上，背景中可以看到一些绿色的围栏或墙壁，暗示这可能是在一个动物园或野生动物保护区的围栏内。
-
-整个场景充满了自然和活力，红熊猫们通过互动和探索，展现了它们的天性和对环境的适应。这个视频捕捉到了它们 playful 和 curious 的瞬间，让人感受到它们活泼可爱的性格。
-
-15:01:37.610 NTC Run:4298 | hit eos,decode avg 18.79 token/s
-15:01:37.610 WRN Run:4631 | drop KV cache after isolated video-history request
-^@15:02:51.406 INF operator():2136 | OpenAI chat request: model=AXERA-TECH/MiniCPM-V-4.6-AX650-C128-P1152-CTX2047 stream=1 max_tokens=2048 has_temperature=0 temperature=1.0000 has_top_p=0 top_p=1.0000 messages=4 stop=0
-15:02:51.622 INF EncodeForContent:2080 | MiniCPM-V-4.6 pixel_values bytes=602112 min=2 max=250 (w=448 h=448 ps=14)
-15:02:52.151 INF EncodeForContent:2124 | vision cache store: /tmp/axllm_media/media_863903737595189_1.png
-15:02:52.225 INF SetKVCache:3134 | decode_grpid:0 prefill_grpid:1 history_cap:0 total_cap:128 symbolic_cap:1 precompute_len:0 input_num_token:358 prefer_symbolic_group:0
-15:02:52.225 INF SetKVCache:3156 | current prefill_max_token_num:1280
-15:02:52.242 INF SetKVCache:3172 | first run
-15:02:52.258 INF Run:3376 | input token num : 358, prefill_split_num : 3
-15:02:52.258 INF Run:3471 | prefill chunk p=0 history_len=0 grpid=1 kv_cache_num=0 input_tokens=128
-15:02:52.414 INF Run:3471 | prefill chunk p=1 history_len=128 grpid=3 kv_cache_num=256 input_tokens=128
-15:02:52.587 INF Run:3471 | prefill chunk p=2 history_len=256 grpid=4 kv_cache_num=384 input_tokens=102
-15:02:52.776 INF Run:3886 | ttft: 518.23 ms
-这张图片展示了一个年轻的女性，她有着长长的银灰色头发，头发上装饰着一些花朵。她穿着一件浅灰色的比基尼，比基尼上有褶皱和蕾丝细节，显得非常优雅。她的面部表情平静，眼神直视前方，给人一种温柔和自信的感觉。背景是海滩，可以看到海浪轻轻拍打着沙滩，远处有绿色的树木，整个场景显得非常宁静和自然。女性的姿态轻松，似乎在享受海滩上的美好时光。
-
-15:02:57.605 NTC Run:4298 | hit eos,decode avg 18.64 token/s
-15:02:57.605 INF GetKVCache:3087 | precompute_len:449, remaining:831 (tracked)
-```
+The packaged Python helper paths are:
+
+- `python/infer_axmodel.py`
+- `python/infer_torch.py`
+- `python/minicpm_v46_tokenizer/`
+
+The packaged `axllm` runtime does not depend on `python/minicpm_v46_tokenizer/`, but `python/infer_axmodel.py` uses it by default.
+
+These path arguments apply to the Python demo flow only. The `axllm` flow reads the same root-level runtime files packaged in this repository.
+
+## Conversion References
+
+If you need the original model files or want to rebuild the deployment artifacts, start with:
+
+- Original Hugging Face model: [openbmb/MiniCPM-V-4.6](https://huggingface.co/openbmb/MiniCPM-V-4.6)
+- AXERA conversion and deployment workflow: [AXERA-TECH/MiniCPM-V-4.6.axera](https://github.com/AXERA-TECH/MiniCPM-V-4.6.axera)
+
+## Discussion
+
+- GitHub Issues
+- QQ group: `139953715`

assets/red-panda-openai.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d921c07bb97224d65a37801541d246067f0d506f08723ffa1ad85c217907ccb8
+size 1867237