Duplicate from wanglab/MedSAM2

Co-authored-by: Adibvafa Fallahpour <adibvafa@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

MedSAM2_2411.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcd946a4d934f553236866fc7e8af77f7e931430e9c044f4ac9d6a723630a870
+size 156039179

MedSAM2_CTLesion.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78f7e125418dfd6fec22f4afe90bcd85cb1d4423d0a9df36f7a87ed63aa1a5f5
+size 156041079

MedSAM2_MRI_LiverLesion.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3632fc77def3a136d7ae8d734613472d187a803b4a1846370b45419622072b2b
+size 156044532

MedSAM2_US_Heart.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:295c0ff8912c99947c364287bbecd1cd36963f0c0ac67a042d292f0dedf8d933
+size 156041079

MedSAM2_latest.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c92743b99f00d078bf32a3afcc38aaa9faf1c1692dffe3eaa7a90938c1991060
+size 156040129

README.md

@@ -0,0 +1,161 @@
+---
+datasets:
+- medical
+language: en
+library_name: torch
+license: cc-by-sa-4.0
+pipeline_tag: image-segmentation
+tags:
+- medical
+- segmentation
+- sam
+- medical-imaging
+- ct
+- mri
+- ultrasound
+---
+
+<div align="center">
+
+<img src="https://github.com/user-attachments/assets/18937bf5-619d-4ae6-a64c-d9900369a7e0" alt="MedSAM2 - Logo" width="30%">
+
+**Segment Anything in 3D Medical Images and Videos**
+
+</div>
+
+<div align="center">
+ <table align="center">
+   <tr>
+     <td><a href="https://arxiv.org/abs/2504.03600" target="_blank"><img src="https://img.shields.io/badge/arXiv-Paper-FF6B6B?style=for-the-badge&logo=arxiv&logoColor=white" alt="Paper"></a></td>
+     <td><a href="https://medsam2.github.io/" target="_blank"><img src="https://img.shields.io/badge/Project-Page-4285F4?style=for-the-badge&logoColor=white" alt="Project"></a></td>
+     <td><a href="https://github.com/bowang-lab/MedSAM2" target="_blank"><img src="https://img.shields.io/badge/GitHub-Code-181717?style=for-the-badge&logo=github&logoColor=white" alt="Code"></a></td>
+     <td><a href="https://huggingface.co/wanglab/MedSAM2" target="_blank"><img src="https://img.shields.io/badge/HuggingFace-Model-FFBF00?style=for-the-badge&logo=huggingface&logoColor=white" alt="HuggingFace Model"></a></td>
+   </tr>
+   <tr>
+     <td><a href="https://medsam-datasetlist.github.io/" target="_blank"><img src="https://img.shields.io/badge/Dataset-List-00B89E?style=for-the-badge" alt="Dataset List"></a></td>
+     <td><a href="https://huggingface.co/datasets/wanglab/CT_DeepLesion-MedSAM2" target="_blank"><img src="https://img.shields.io/badge/Dataset-CT__DeepLesion-28A745?style=for-the-badge" alt="CT_DeepLesion-MedSAM2"></a></td>
+     <td><a href="https://huggingface.co/datasets/wanglab/LLD-MMRI-MedSAM2" target="_blank"><img src="https://img.shields.io/badge/Dataset-LLD--MMRI-FF6B6B?style=for-the-badge" alt="LLD-MMRI-MedSAM2"></a></td>
+     <td><a href="https://github.com/bowang-lab/MedSAMSlicer/tree/MedSAM2" target="_blank"><img src="https://img.shields.io/badge/3D_Slicer-Plugin-e2006a?style=for-the-badge" alt="3D Slicer"></a></td>
+   </tr>
+   <tr>
+     <td><a href="https://github.com/bowang-lab/MedSAM2/blob/main/app.py" target="_blank"><img src="https://img.shields.io/badge/Gradio-Demo-F9D371?style=for-the-badge&logo=gradio&logoColor=white" alt="Gradio App"></a></td>
+     <td><a href="https://colab.research.google.com/drive/1MKna9Sg9c78LNcrVyG58cQQmaePZq2k2?usp=sharing" target="_blank"><img src="https://img.shields.io/badge/Colab-CT--Seg--Demo-F9AB00?style=for-the-badge&logo=googlecolab&logoColor=white" alt="CT-Seg-Demo"></a></td>
+     <td><a href="https://colab.research.google.com/drive/16niRHqdDZMCGV7lKuagNq_r_CEHtKY1f?usp=sharing" target="_blank"><img src="https://img.shields.io/badge/Colab-Video--Seg--Demo-F9AB00?style=for-the-badge&logo=googlecolab&logoColor=white" alt="Video-Seg-Demo"></a></td>
+     <td><a href="https://github.com/bowang-lab/MedSAM2?tab=readme-ov-file#bibtex" target="_blank"><img src="https://img.shields.io/badge/Paper-BibTeX-9370DB?style=for-the-badge&logoColor=white" alt="BibTeX"></a></td>
+   </tr>
+ </table>
+</div>
+
+
+
+## Authors
+
+<p align="center">
+  <a href="https://scholar.google.com.hk/citations?hl=en&user=bW1UV4IAAAAJ&view_op=list_works&sortby=pubdate">Jun Ma</a><sup>* 1,2</sup>, 
+  <a href="https://scholar.google.com/citations?user=8IE0CfwAAAAJ&hl=en">Zongxin Yang</a><sup>* 3</sup>, 
+  Sumin Kim<sup>2,4,5</sup>, 
+  Bihui Chen<sup>2,4,5</sup>, 
+  <a href="https://scholar.google.com.hk/citations?user=U-LgNOwAAAAJ&hl=en&oi=sra">Mohammed Baharoon</a><sup>2,3,5</sup>,<br>
+  <a href="https://scholar.google.com.hk/citations?user=4qvKTooAAAAJ&hl=en&oi=sra">Adibvafa Fallahpour</a><sup>2,4,5</sup>, 
+  <a href="https://scholar.google.com.hk/citations?user=UlTJ-pAAAAAJ&hl=en&oi=sra">Reza Asakereh</a><sup>4,7</sup>, 
+  Hongwei Lyu<sup>4</sup>, 
+  <a href="https://wanglab.ai/index.html">Bo Wang</a><sup>† 1,2,4,5,6</sup>
+</p>
+
+<p align="center">
+  <sup>*</sup> Equal contribution &nbsp;&nbsp;&nbsp; <sup>†</sup> Corresponding author
+</p>
+
+<p align="center">
+  <sup>1</sup>AI Collaborative Centre, University Health Network, Toronto, Canada<br>
+  <sup>2</sup>Vector Institute for Artificial Intelligence, Toronto, Canada<br>
+  <sup>3</sup>Department of Biomedical Informatics, Harvard Medical School, Harvard University, Boston, USA<br>
+  <sup>4</sup>Peter Munk Cardiac Centre, University Health Network, Toronto, Canada<br>
+  <sup>5</sup>Department of Computer Science, University of Toronto, Toronto, Canada<br>
+  <sup>6</sup>Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada<br>
+  <sup>7</sup>Roche Canada and Genentech
+</p>
+
+
+## Highlights
+
+- A promptable foundation model for 3D medical image and video segmentation
+- Trained on 455,000+ 3D image-mask pairs and 76,000+ annotated video frames
+- Versatile segmentation capability across diverse organs and pathologies
+- Extensive user studies in large-scale lesion and video datasets demonstrate that MedSAM2 substantially facilitates annotation workflows
+
+## Model Overview
+MedSAM2 is a promptable segmentation segmentation model tailored for medical imaging applications. Built upon the foundation of the [Segment Anything Model (SAM) 2.1](https://github.com/facebookresearch/sam2), MedSAM2 has been specifically adapted and fine-tuned for various 3D medical images and videos.
+
+## Available Models
+
+- **MedSAM2_2411.pt**: Base model trained in November 2024
+- **MedSAM2_US_Heart.pt**: Fine-tuned model specialized for heart ultrasound video segmentation
+- **MedSAM2_MRI_LiverLesion.pt**: Fine-tuned model for liver lesion segmentation in MRI scans
+- **MedSAM2_CTLesion.pt**: Fine-tuned model for general lesion segmentation in CT scans
+- **MedSAM2_latest.pt** (recommended): Latest version trained on the combination of public datasets and newly annotated medical imaging data
+
+## Downloading Models
+
+### Option 1: Download individual models
+You can download the models directly from the Hugging Face repository:
+
+```python
+# Using huggingface_hub
+from huggingface_hub import hf_hub_download
+
+# Download the recommended latest model
+model_path = hf_hub_download(repo_id="wanglab/MedSAM2", filename="MedSAM2_latest.pt")
+
+# Or download a specific fine-tuned model
+heart_us_model_path = hf_hub_download(repo_id="wanglab/MedSAM2", filename="MedSAM2_US_Heart.pt")
+liver_model_path = hf_hub_download(repo_id="wanglab/MedSAM2", filename="MedSAM2_MRI_LiverLesion.pt")
+```
+
+### Option 2: Download all models to a specific folder
+```python
+from huggingface_hub import hf_hub_download
+import os
+
+# Create checkpoints directory if it doesn't exist
+os.makedirs("checkpoints", exist_ok=True)
+
+# List of model filenames
+model_files = [
+    "MedSAM2_2411.pt",
+    "MedSAM2_US_Heart.pt",
+    "MedSAM2_MRI_LiverLesion.pt",
+    "MedSAM2_CTLesion.pt",
+    "MedSAM2_latest.pt"
+]
+
+# Download all models
+for model_file in model_files:
+    local_path = os.path.join("checkpoints", model_file)
+    hf_hub_download(
+        repo_id="wanglab/MedSAM2",
+        filename=model_file,
+        local_dir="checkpoints",
+        local_dir_use_symlinks=False
+    )
+    print(f"Downloaded {model_file} to {local_path}")
+```
+
+Alternatively, you can manually download the models from the [Hugging Face repository page](https://huggingface.co/wanglab/MedSAM2).
+
+
+
+## Citations
+
+```
+@article{MedSAM2,
+    title={MedSAM2: Segment Anything in 3D Medical Images and Videos},
+    author={Ma, Jun and Yang, Zongxin and Kim, Sumin and Chen, Bihui and Baharoon, Mohammed and Fallahpour, Adibvafa and Asakereh, Reza and Lyu, Hongwei and Wang, Bo},
+    journal={arXiv preprint arXiv:2504.03600},
+    year={2025}
+}
+```
+
+## License
+
+The model weights can only be used for research and education purposes.

config.json

@@ -0,0 +1,3 @@
+{
+  "_name_or_path": "wanglab/MedSAM2"
+}

eff_medsam2_small_FLARE25_RECIST_baseline.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1a54157f93df34f3eb702d19e0a03b2159be8c1a12beb1972088123aa70e073
+size 136422106

eff_medsam2_tiny_FLARE25_RECIST_baseline.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78cd67f41c1c2953026bd6e70a7648998079d08cb6ab0f75bbe289039695c103
+size 71662031

medsam2_FLARE25_RECIST_baseline.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:890e6e10924984c7b704a1984362c53a56dc0ee27c796a34eb49c83a1dc4a144
+size 156048332