Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,78 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
library_name: onnx
|
| 4 |
+
tags:
|
| 5 |
+
- depth-estimation
|
| 6 |
+
- depth-anything-3
|
| 7 |
+
- dinov2
|
| 8 |
+
- camera-intrinsics
|
| 9 |
+
- onnx
|
| 10 |
+
base_model: depth-anything/DA3-BASE
|
| 11 |
+
pipeline_tag: depth-estimation
|
| 12 |
+
language:
|
| 13 |
+
- en
|
| 14 |
---
|
| 15 |
+
|
| 16 |
+
# Depth Anything 3 Base β Monocular Depth + Camera Intrinsics (ONNX)
|
| 17 |
+
|
| 18 |
+
Single-view ONNX export of [depth-anything/DA3-BASE](https://huggingface.co/depth-anything/DA3-BASE) β the Apache-2.0 any-view model of ByteDance's Depth Anything 3 family (DINOv2 ViT-B backbone, dual-DPT head, camera heads). This cut takes **one image per row** and emits depth, per-pixel confidence, and a per-image camera-intrinsics estimate.
|
| 19 |
+
|
| 20 |
+
Two things to know up front:
|
| 21 |
+
|
| 22 |
+
- **Depth is scale-ambiguous** (up-to-scale, not meters). For metric depth use [Heliosoph/da3metric-large-onnx](https://huggingface.co/Heliosoph/da3metric-large-onnx), or anchor this model's scale against it.
|
| 23 |
+
- **The pose output is not useful here.** DA3 predicts camera pose *relative to the other views in the same forward pass*; with a single view the `extrinsics` output is near-identity by construction. For real pose recovery use the multi-view sibling: [Heliosoph/da3-base-4view-onnx](https://huggingface.co/Heliosoph/da3-base-4view-onnx) (same weights, 4-frame window). The `intrinsics` output **is** meaningful for single images β a per-image focal-length estimate.
|
| 24 |
+
|
| 25 |
+
Re-exported from upstream safetensors via the official `depth-anything-3` package. Provenance trail: Lin et al. β depth-anything/DA3-BASE safetensors β `depth_anything_3.api.DepthAnything3` + thin wrapper β `torch.onnx.export` β these files. The any-view checkpoints need two exporter workarounds (baked into the script): `torch.cartesian_prod` (RoPE position grid, no ONNX symbolic) replaced with meshgrid+stack, and the TorchScript-compiled `affine_inverse` (whose `aten::mT` is unexportable) rebound to a `transpose(-2, -1)` equivalent. fp16 sibling via onnxconverter-common with a Cast-node type realignment.
|
| 26 |
+
|
| 27 |
+
Toolchain: `torch 2.4.x` (CUDA 12.4), `depth-anything-3 0.1.1`, opset 17, legacy TorchScript exporter, fp32 trace (upstream bf16 autocast disabled). Conversion script: [`scripts/export-da3metric.ps1`](https://github.com/HeliosophLLC/DatumV/blob/main/scripts/export-da3metric.ps1) in the Heliosoph repo. Export validation: fp32 ONNX matches PyTorch to 4.1e-07 max relative error across all four heads; fp16 matches fp32 to β€4.9e-04; batch>1 verified item-wise against batch=1.
|
| 28 |
+
|
| 29 |
+
Credit: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, Bingyi Kang (ByteDance Seed). Paper: *"Depth Anything 3: Recovering the Visual Space from Any Views"*, 2025.
|
| 30 |
+
|
| 31 |
+
## What this repo contains
|
| 32 |
+
|
| 33 |
+
| File | Variant | Size | Use |
|
| 34 |
+
|---|---|---|---|
|
| 35 |
+
| `model.onnx` | fp32 | ~394 MB | Default β matches the PyTorch upstream to ~1e-6. |
|
| 36 |
+
| `model_fp16.onnx` | fp16 | ~198 MB | Half precision, **I/O stays fp32** (`keep_io_types`) β drop-in swap. |
|
| 37 |
+
| `config.json` | β | <1 KB | Upstream DA3 model config (provenance / re-instantiation). |
|
| 38 |
+
|
| 39 |
+
## Input / output
|
| 40 |
+
|
| 41 |
+
| | Spec |
|
| 42 |
+
|---|---|
|
| 43 |
+
| Input name | `image` |
|
| 44 |
+
| Input shape | `[batch, 3, 504, 504]` (NCHW) |
|
| 45 |
+
| Input dtype | float32 (both variants) |
|
| 46 |
+
| Preprocessing | RGB, scale to [0,1], ImageNet mean/std (`[0.485, 0.456, 0.406]` / `[0.229, 0.224, 0.225]`) |
|
| 47 |
+
| Output `depth` | `[batch, 1, 504, 504]` β up-to-scale depth, bigger = farther |
|
| 48 |
+
| Output `depth_conf` | `[batch, 1, 504, 504]` β per-pixel confidence |
|
| 49 |
+
| Output `extrinsics` | `[batch, 1, 3, 4]` β `[R \| t]`; **near-identity for single view** (see above) |
|
| 50 |
+
| Output `intrinsics` | `[batch, 1, 3, 3]` β estimated K **at the 504Γ504 input grid** (principal point at 252, 252); rescale to source dims via `K' = diag(W/504, H/504, 1) Β· K` |
|
| 51 |
+
| Dynamic axes | batch only |
|
| 52 |
+
|
| 53 |
+
**Resolution is fixed at 504Γ504** β the ViT position-embedding interpolation bakes the patch count into the trace (inherent to DA3 ONNX exports, not a choice). Resize inputs to match; re-run the conversion script with `-Height`/`-Width` for a different fixed resolution (multiples of 14).
|
| 54 |
+
|
| 55 |
+
## How to use
|
| 56 |
+
|
| 57 |
+
```python
|
| 58 |
+
import numpy as np
|
| 59 |
+
import onnxruntime as ort
|
| 60 |
+
from PIL import Image
|
| 61 |
+
|
| 62 |
+
sess = ort.InferenceSession("model.onnx")
|
| 63 |
+
|
| 64 |
+
img = Image.open("photo.jpg").convert("RGB")
|
| 65 |
+
x = np.asarray(img.resize((504, 504), Image.BILINEAR), dtype=np.float32) / 255.0
|
| 66 |
+
x = ((x - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]).transpose(2, 0, 1)[None].astype(np.float32)
|
| 67 |
+
|
| 68 |
+
depth, conf, _ext, K = sess.run(["depth", "depth_conf", "extrinsics", "intrinsics"], {"image": x})
|
| 69 |
+
depth, conf, K = depth[0, 0], conf[0, 0], K[0, 0]
|
| 70 |
+
|
| 71 |
+
# K is at the 504x504 grid; rescale to the original image if needed:
|
| 72 |
+
w, h = img.size
|
| 73 |
+
K_src = np.diag([w / 504, h / 504, 1.0]) @ K
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
## License
|
| 77 |
+
|
| 78 |
+
**Apache-2.0** β same as upstream [depth-anything/DA3-BASE](https://huggingface.co/depth-anything/DA3-BASE). (The DA3 any-view Large/Giant checkpoints are CC-BY-NC 4.0 and are **not** part of this export; Base is the largest permissively-licensed any-view variant.) The ONNX-export step doesn't change licensing β same model, different serialization.
|