flyingbertman commited on
Commit
9535e60
Β·
verified Β·
1 Parent(s): c3e6ba1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md CHANGED
@@ -1,3 +1,78 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: onnx
4
+ tags:
5
+ - depth-estimation
6
+ - depth-anything-3
7
+ - dinov2
8
+ - camera-intrinsics
9
+ - onnx
10
+ base_model: depth-anything/DA3-BASE
11
+ pipeline_tag: depth-estimation
12
+ language:
13
+ - en
14
  ---
15
+
16
+ # Depth Anything 3 Base β€” Monocular Depth + Camera Intrinsics (ONNX)
17
+
18
+ Single-view ONNX export of [depth-anything/DA3-BASE](https://huggingface.co/depth-anything/DA3-BASE) β€” the Apache-2.0 any-view model of ByteDance's Depth Anything 3 family (DINOv2 ViT-B backbone, dual-DPT head, camera heads). This cut takes **one image per row** and emits depth, per-pixel confidence, and a per-image camera-intrinsics estimate.
19
+
20
+ Two things to know up front:
21
+
22
+ - **Depth is scale-ambiguous** (up-to-scale, not meters). For metric depth use [Heliosoph/da3metric-large-onnx](https://huggingface.co/Heliosoph/da3metric-large-onnx), or anchor this model's scale against it.
23
+ - **The pose output is not useful here.** DA3 predicts camera pose *relative to the other views in the same forward pass*; with a single view the `extrinsics` output is near-identity by construction. For real pose recovery use the multi-view sibling: [Heliosoph/da3-base-4view-onnx](https://huggingface.co/Heliosoph/da3-base-4view-onnx) (same weights, 4-frame window). The `intrinsics` output **is** meaningful for single images β€” a per-image focal-length estimate.
24
+
25
+ Re-exported from upstream safetensors via the official `depth-anything-3` package. Provenance trail: Lin et al. β†’ depth-anything/DA3-BASE safetensors β†’ `depth_anything_3.api.DepthAnything3` + thin wrapper β†’ `torch.onnx.export` β†’ these files. The any-view checkpoints need two exporter workarounds (baked into the script): `torch.cartesian_prod` (RoPE position grid, no ONNX symbolic) replaced with meshgrid+stack, and the TorchScript-compiled `affine_inverse` (whose `aten::mT` is unexportable) rebound to a `transpose(-2, -1)` equivalent. fp16 sibling via onnxconverter-common with a Cast-node type realignment.
26
+
27
+ Toolchain: `torch 2.4.x` (CUDA 12.4), `depth-anything-3 0.1.1`, opset 17, legacy TorchScript exporter, fp32 trace (upstream bf16 autocast disabled). Conversion script: [`scripts/export-da3metric.ps1`](https://github.com/HeliosophLLC/DatumV/blob/main/scripts/export-da3metric.ps1) in the Heliosoph repo. Export validation: fp32 ONNX matches PyTorch to 4.1e-07 max relative error across all four heads; fp16 matches fp32 to ≀4.9e-04; batch>1 verified item-wise against batch=1.
28
+
29
+ Credit: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, Bingyi Kang (ByteDance Seed). Paper: *"Depth Anything 3: Recovering the Visual Space from Any Views"*, 2025.
30
+
31
+ ## What this repo contains
32
+
33
+ | File | Variant | Size | Use |
34
+ |---|---|---|---|
35
+ | `model.onnx` | fp32 | ~394 MB | Default β€” matches the PyTorch upstream to ~1e-6. |
36
+ | `model_fp16.onnx` | fp16 | ~198 MB | Half precision, **I/O stays fp32** (`keep_io_types`) β€” drop-in swap. |
37
+ | `config.json` | β€” | <1 KB | Upstream DA3 model config (provenance / re-instantiation). |
38
+
39
+ ## Input / output
40
+
41
+ | | Spec |
42
+ |---|---|
43
+ | Input name | `image` |
44
+ | Input shape | `[batch, 3, 504, 504]` (NCHW) |
45
+ | Input dtype | float32 (both variants) |
46
+ | Preprocessing | RGB, scale to [0,1], ImageNet mean/std (`[0.485, 0.456, 0.406]` / `[0.229, 0.224, 0.225]`) |
47
+ | Output `depth` | `[batch, 1, 504, 504]` β€” up-to-scale depth, bigger = farther |
48
+ | Output `depth_conf` | `[batch, 1, 504, 504]` β€” per-pixel confidence |
49
+ | Output `extrinsics` | `[batch, 1, 3, 4]` β€” `[R \| t]`; **near-identity for single view** (see above) |
50
+ | Output `intrinsics` | `[batch, 1, 3, 3]` β€” estimated K **at the 504Γ—504 input grid** (principal point at 252, 252); rescale to source dims via `K' = diag(W/504, H/504, 1) Β· K` |
51
+ | Dynamic axes | batch only |
52
+
53
+ **Resolution is fixed at 504Γ—504** β€” the ViT position-embedding interpolation bakes the patch count into the trace (inherent to DA3 ONNX exports, not a choice). Resize inputs to match; re-run the conversion script with `-Height`/`-Width` for a different fixed resolution (multiples of 14).
54
+
55
+ ## How to use
56
+
57
+ ```python
58
+ import numpy as np
59
+ import onnxruntime as ort
60
+ from PIL import Image
61
+
62
+ sess = ort.InferenceSession("model.onnx")
63
+
64
+ img = Image.open("photo.jpg").convert("RGB")
65
+ x = np.asarray(img.resize((504, 504), Image.BILINEAR), dtype=np.float32) / 255.0
66
+ x = ((x - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]).transpose(2, 0, 1)[None].astype(np.float32)
67
+
68
+ depth, conf, _ext, K = sess.run(["depth", "depth_conf", "extrinsics", "intrinsics"], {"image": x})
69
+ depth, conf, K = depth[0, 0], conf[0, 0], K[0, 0]
70
+
71
+ # K is at the 504x504 grid; rescale to the original image if needed:
72
+ w, h = img.size
73
+ K_src = np.diag([w / 504, h / 504, 1.0]) @ K
74
+ ```
75
+
76
+ ## License
77
+
78
+ **Apache-2.0** β€” same as upstream [depth-anything/DA3-BASE](https://huggingface.co/depth-anything/DA3-BASE). (The DA3 any-view Large/Giant checkpoints are CC-BY-NC 4.0 and are **not** part of this export; Base is the largest permissively-licensed any-view variant.) The ONNX-export step doesn't change licensing β€” same model, different serialization.