zzl
commited on
Commit
·
f4ada59
0
Parent(s):
Initial commit (clean history)
Browse files- .gitattributes +5 -0
- .gitignore +3 -0
- LICENSE.md +35 -0
- README.md +93 -0
- ram/ram_promptir_finetune.pth +3 -0
- ram/ram_promptir_pretrain.pth +3 -0
- ram/ram_swinir_finetune.pth +3 -0
- ram/ram_swinir_pretrain.pth +3 -0
- ram_plus/3task/AdaSAM_pretrained.pth +3 -0
- ram_plus/3task/RestormerRFR.pth +3 -0
- ram_plus/3task/RestormerRFR_ratio0.3.pth +3 -0
- ram_plus/3task/RestormerWoSkip.pth +3 -0
- ram_plus/5task/AdaSAM_pretrained.pth +3 -0
- ram_plus/5task/RestormerRFR.pth +3 -0
- ram_plus/5task/RestormerWoSkip.pth +3 -0
- ram_plus/7task/AdaSAM_pretrained.pth +3 -0
- ram_plus/7task/RestormerRFR.pth +3 -0
- ram_plus/7task/RestormerRFR_ratio0.3.pth +3 -0
- ram_plus/7task/RestormerWoSkip.pth +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.DS_Store
|
| 2 |
+
Icon?
|
| 3 |
+
.DS_Store
|
LICENSE.md
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Pi-Lab License 1.0
|
| 2 |
+
|
| 3 |
+
Copyright 2024 Pi-Lab
|
| 4 |
+
|
| 5 |
+
Redistribution and use for non-commercial purpose in source and
|
| 6 |
+
binary forms, with or without modification, are permitted provided
|
| 7 |
+
that the following conditions are met:
|
| 8 |
+
|
| 9 |
+
1. Redistributions of source code must retain the above copyright
|
| 10 |
+
notice, this list of conditions and the following disclaimer.
|
| 11 |
+
|
| 12 |
+
2. Redistributions in binary form must reproduce the above copyright
|
| 13 |
+
notice, this list of conditions and the following disclaimer in
|
| 14 |
+
the documentation and/or other materials provided with the
|
| 15 |
+
distribution.
|
| 16 |
+
|
| 17 |
+
3. Neither the name of the copyright holder nor the names of its
|
| 18 |
+
contributors may be used to endorse or promote products derived
|
| 19 |
+
from this software without specific prior written permission.
|
| 20 |
+
|
| 21 |
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
| 22 |
+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
| 23 |
+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
| 24 |
+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
| 25 |
+
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
| 26 |
+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
| 27 |
+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
| 28 |
+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
| 29 |
+
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
| 30 |
+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
| 31 |
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
| 32 |
+
|
| 33 |
+
In the event that redistribution and/or use for commercial purpose in
|
| 34 |
+
source or binary forms, with or without modification is required,
|
| 35 |
+
please contact the contributor(s) of the work.
|
README.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: pytorch
|
| 3 |
+
license: other
|
| 4 |
+
tags:
|
| 5 |
+
- low-level-vision
|
| 6 |
+
- all-in-one image-restoration
|
| 7 |
+
language:
|
| 8 |
+
- en
|
| 9 |
+
pipeline_tag: image-to-image
|
| 10 |
+
model-index:
|
| 11 |
+
- name: RAM / RAM++
|
| 12 |
+
results:
|
| 13 |
+
- task:
|
| 14 |
+
type: image-to-image
|
| 15 |
+
name: All-in-One Image Restoration
|
| 16 |
+
dataset:
|
| 17 |
+
name: placeholder
|
| 18 |
+
type: image
|
| 19 |
+
metrics:
|
| 20 |
+
- name: PSNR
|
| 21 |
+
type: psnr
|
| 22 |
+
value: 0.0
|
| 23 |
+
---
|
| 24 |
+
This is the official pretrained models for the paper.
|
| 25 |
+
>**Restore Anything with Masks:Leveraging Mask Image Modeling for Blind All-in-One Image Restoration**<br> [Chujie Qin](https://github.com/Dragonisss), [Ruiqi Wu](https://rq-wu.github.io/), [Zikun Liu](), [Xin Lin](https://linxin0.github.io/), [Chunle Guo](https://scholar.google.com/citations?user=RZLYwR0AAAAJ&hl=en), [Hyun Hee Park](s), [Chongyi Li<sup>†</sup>](https://li-chongyi.github.io/)<br/>
|
| 26 |
+
> ( † indicates corresponding author )<br/>
|
| 27 |
+
> In ECCV 2024, \[[HomePage](https://rq-wu.github.io/projects/RAM/index.html)\], \[[Paper Link](https://arxiv.org/abs/2409.19403v1)\]
|
| 28 |
+
|
| 29 |
+
> **RAM++: <u>R</u>obust Representation Learning via <u>A</u>daptive <u>M</u>ask for All-in-One Image Restoration**<br>
|
| 30 |
+
> [Zilong Zhang<sup>*</sup>](https://github.com/Zilong-Zhang003), [Chujie Qin<sup>*</sup>](https://github.com/DragonisCV), [Chunle Guo](https://mmcheng.net/clguo/), [Yong Zhang](), [Chao Xue](), [Ming-Ming Cheng](https://mmcheng.net/cmm/), [Chongyi Li<sup>†</sup>](https://li-chongyi.github.io/)<br/>
|
| 31 |
+
> (<sup>*</sup>indicates equal contribution; <sup>†</sup> indicates corresponding author)<br/>
|
| 32 |
+
> arxiv preprint, \[[HomePage](https://zilong-zhang003.github.io/RAM2.0/)\], \[[Paper Link](https://arxiv.org/abs/2509.12039)\]
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
# Model description
|
| 36 |
+
## RAM
|
| 37 |
+
This method is architecture-agnostic and can be trained with any model. \
|
| 38 |
+
Here we provide the pre-trained and fine-tuned weights for two representative models: <strong>[PromptIR](https://github.com/va1shn9v/PromptIR)</strong> and <strong>[SwinIR](https://github.com/JingyunLiang/SwinIR)</strong>.
|
| 39 |
+
## RAM_plus
|
| 40 |
+
<strong>AdaSAM</strong> is a ViT-based, pixel-level mask generator. It analyzes correlations between image tokens and applies masks to regions that are semantically and texturally rich.
|
| 41 |
+
|
| 42 |
+
<strong>RestormerWoSkip</strong> is built on <strong>[Restormer](https://github.com/swz30/Restormer)</strong>; it differs by removing the long-range residual connections.
|
| 43 |
+
|
| 44 |
+
<strong>RestormerRFR</strong> regularizes via an efficient feature-fusion strategy that leverages DINOv2’s semantic consistency and degradation invariance.
|
| 45 |
+
|
| 46 |
+
<strong>Different folders</strong> contain model weights trained under configurations with different numbers of tasks.
|
| 47 |
+
|
| 48 |
+
# How to use
|
| 49 |
+
For full instructions and runnable scripts, see the [code repository](https://github.com/DragonisCV/RAM/)
|
| 50 |
+
## RAM
|
| 51 |
+
### Pre-training:
|
| 52 |
+
```python
|
| 53 |
+
mask, mask_token = Random(img) #pixel_level
|
| 54 |
+
output = PromptIR(img, mask, mask_token)
|
| 55 |
+
```
|
| 56 |
+
### Fine-tuning:
|
| 57 |
+
```python
|
| 58 |
+
output = PromptIR(img, mask=None, mask_token=None)
|
| 59 |
+
```
|
| 60 |
+
## RAM_plus
|
| 61 |
+
### Pre-training:
|
| 62 |
+
```python
|
| 63 |
+
mask, mask_token = AdaSAM(img)
|
| 64 |
+
output = RestormerWoSkip(img, mask, mask_token)
|
| 65 |
+
```
|
| 66 |
+
### Fine-tuning:
|
| 67 |
+
```python
|
| 68 |
+
dino_features = DINOv2(img)
|
| 69 |
+
output = RestormerRFR(img, mask=None, mask_token=None, dino_features)
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
# Citation
|
| 73 |
+
If you find our repo useful for your research, please consider citing our paper:
|
| 74 |
+
```bibtex
|
| 75 |
+
@inproceedings{qin2024restore,
|
| 76 |
+
title={Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration},
|
| 77 |
+
author={Qin, Chu-Jie and Wu, Rui-Qi and Liu, Zikun and Lin, Xin and Guo, Chun-Le and Park, Hyun Hee and Li, Chongyi},
|
| 78 |
+
booktitle={European Conference on Computer Vision},
|
| 79 |
+
pages={364--380},
|
| 80 |
+
year={2024},
|
| 81 |
+
organization={Springer}
|
| 82 |
+
}
|
| 83 |
+
|
| 84 |
+
@misc{zhang2025ramrobustrepresentationlearning,
|
| 85 |
+
title={RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration},
|
| 86 |
+
author={Zilong Zhang and Chujie Qin and Chunle Guo and Yong Zhang and Chao Xue and Ming-Ming Cheng and Chongyi Li},
|
| 87 |
+
year={2025},
|
| 88 |
+
eprint={2509.12039},
|
| 89 |
+
archivePrefix={arXiv},
|
| 90 |
+
primaryClass={cs.CV},
|
| 91 |
+
url={https://arxiv.org/abs/2509.12039},
|
| 92 |
+
}
|
| 93 |
+
```
|
ram/ram_promptir_finetune.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f5497063955d8a8285b53f5cb39f72e446296e3dc8df363392fbe498c3ac86c8
|
| 3 |
+
size 141708965
|
ram/ram_promptir_pretrain.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:050355603860b56654d9dabcd963e9157909ee4fa38eed95c8dca1ff1834175a
|
| 3 |
+
size 283418451
|
ram/ram_swinir_finetune.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8ca5261ea6e48c099d05664b34385f0701ee7fc2cc6ec1a06c58fa10d17e96a0
|
| 3 |
+
size 66309678
|
ram/ram_swinir_pretrain.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:54e076ac7a7bf1286a5613f8473156975c508eb3e5462f3d0486349d7d41b577
|
| 3 |
+
size 132619947
|
ram_plus/3task/AdaSAM_pretrained.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6272bf3c794f2c938f29a72c153ceb1b37ae4a1db43c8a364c0fb61a9c617c42
|
| 3 |
+
size 59467679
|
ram_plus/3task/RestormerRFR.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:841f6de18d888e61e7c6bc74cb683b57efb25365fa4cd6a7502255580cb7d06c
|
| 3 |
+
size 327231278
|
ram_plus/3task/RestormerRFR_ratio0.3.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dd9af2db2ca6e7745de7ad96ffa1bf9f86944208211b8e8918a125edb77981b9
|
| 3 |
+
size 327231278
|
ram_plus/3task/RestormerWoSkip.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a793b2907ca760261c02e14e684122fce17b2115f6b3a6e788af023e72a3fbb3
|
| 3 |
+
size 209380959
|
ram_plus/5task/AdaSAM_pretrained.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b2d1596161c81a88fa2df466e5e51454ad1aa316dc890e61e8e379c8eec9c96
|
| 3 |
+
size 59467679
|
ram_plus/5task/RestormerRFR.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:54a40ea868e580e9d7c65ab02a2f783c57423dc18fc86c443acb43cf3eaf22db
|
| 3 |
+
size 327231721
|
ram_plus/5task/RestormerWoSkip.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b6eb8bffc17f761376daadfb055d890909e111f992d6b5b6f011937e4ffc924
|
| 3 |
+
size 209380959
|
ram_plus/7task/AdaSAM_pretrained.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:20e5ecc128b658c21a02edbfdb5d0c36689ebffd13cd2d3ad2c89ffa94834193
|
| 3 |
+
size 59467679
|
ram_plus/7task/RestormerRFR.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:020d66ea48ce11d7ad1ac1e62737466e705f950c62ecf0ddd48701ed6e3d3cdd
|
| 3 |
+
size 327231278
|
ram_plus/7task/RestormerRFR_ratio0.3.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6b14736c5982ebb988857deb8ff34f2beff4720df8e38c5f165021a9aaadfe59
|
| 3 |
+
size 327231721
|
ram_plus/7task/RestormerWoSkip.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d59a7afc9a0d9753e5b506fcf5e0344191790a4d65117dfc9b277a21ffc84778
|
| 3 |
+
size 209380959
|