README.md

---
license: mit
---

# **AeroReformer: Aerial Referring Transformer for UAV-Based Referring Image Segmentation**

**This repository hosts pre-trained checkpoints for AeroReformer.**
For the **full implementation, training and evaluation scripts**, please see the official codebase:

**GitHub:** [https://github.com/lironui/AeroReformer](https://github.com/lironui/AeroReformer)

> Paper: [AeroReformer: Aerial Referring Transformer for UAV-Based Referring Image Segmentation](https://arxiv.org/pdf/2502.16680)

AeroReformer is a vision-language framework for UAV-based referring image segmentation (UAV-RIS), featuring:

* **VLCAM**: Vision-Language Cross-Attention Module for robust multimodal grounding.
* **RAMSF**: Rotation-Aware Multi-Scale Fusion decoder for aerial scenes with large scale/rotation variance.

---

## 📦 What’s in this repo?

Pre-trained model weights (PyTorch `.pth`) for:

* **UAVid-RIS** (best checkpoint)
* **VDD-RIS** (best checkpoint)

All training/inference code lives in the GitHub repo.

---

## 🚀 How to use these weights

### 1) Download with `huggingface_hub`


### 2) Load into the official implementation

Follow the **loading utilities and model definitions** from the GitHub repo:

* Code & instructions: [https://github.com/lironui/AeroReformer](https://github.com/lironui/AeroReformer)
* Typical usage: run their `test.py` / `train.py` while pointing `--resume` to the checkpoint path you just downloaded.

Example (UAVid-RIS testing) — **run in the code repo**:

```bash
python test.py --swin_type base --dataset uavid_ris \
  --resume /path/to/model_best_AeroReformer.pth \
  --model_id AeroReformer --split test --workers 4 --window12 \
  --img_size 480 --refer_data_root ./data/UAVid_RIS/ --mha 4-4-4-4
```

---

## ✅ Recommended environments

* **Python:** 3.10
* **PyTorch:** 2.3.1
* **CUDA:** 12.4 (or adapt to your setup)
* **OS:** Ubuntu (verified by authors)

> Exact dependency setup and `requirements.txt` are maintained in the GitHub repo.

---

## 📂 Datasets

Experiments use **UAVid-RIS** and **VDD-RIS**.
Referring expressions were generated by **Qwen** and **LLaMA** and may include **errors/inconsistencies**.

* Text labels:

  * [lironui/UAVid-RIS](https://huggingface.co/datasets/lironui/UAVid-RIS)
  * [lironui/VDD-RIS](https://huggingface.co/datasets/lironui/VDD-RIS)
* Raw imagery:

  * UAVid: [https://uavid.nl/](https://uavid.nl/)
  * VDD: [https://huggingface.co/datasets/RussRobin/VDD](https://huggingface.co/datasets/RussRobin/VDD)

Preprocessing and split scripts are provided in the GitHub repo.

---

## ⚠️ Notes & Limitations

* Checkpoints here are **frozen artifacts**; for training/fine-tuning or code changes, use the GitHub repo.
* Performance depends on preprocessing, image size, and split strategy; follow the original scripts for reproducibility.
* Auto-generated text expressions may contain **ambiguity/noise**.

---

## 🙏 Acknowledgements

AeroReformer builds upon:

* [LAVT](https://github.com/yz93/LAVT-RIS)
* [RMSIN](https://github.com/Lsan2401/RMSIN)

---

## 📜 Citation

```bibtex
@article{AeroReformer2025,
  title   = {AeroReformer: Aerial Referring Transformer for UAV-Based Referring Image Segmentation},
  author  = {<Author List>},
  journal = {arXiv preprint arXiv:2502.16680},
  year    = {2025}
}
```

---

## 🔐 License

Unless stated otherwise, weights are distributed under **MIT**.
Please also respect the licenses/terms of the underlying datasets and the Swin-Transformer weights.

---