Enhance model card for SRUM

README.md

@@ -1,47 +1,75 @@
 ---
-license: apache-2.0
+base_model:
+- ByteDance-Seed/BAGEL-7B-MoT
 language:
 - en
+license: apache-2.0
 metrics:
 - accuracy
-base_model:
-- ByteDance-Seed/BAGEL-7B-MoT
 pipeline_tag: text-to-image
+library_name: transformers
 ---
 
 <p align="center">
+  <img src="https://github.com/WayneJin0918/SRUM/raw/main/assets/srum_log_2.png" alt="SRUM" width="220"/>
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/papers/2510.12784">
+    <img
+      src="https://img.shields.io/badge/SRUM-Paper-red"
+      alt="SRUM Paper on Hugging Face"
+    />
+  </a>
+  <a href="https://github.com/WayneJin0918/SRUM">
+    <img
+      src="https://img.shields.io/badge/GitHub-Code-black"
+      alt="GitHub Repository"
+    />
+  </a>
   <a href="https://waynejin0918.github.io/srum_web/">
     <img
       src="https://img.shields.io/badge/SRUM-Website-blue"
       alt="SRUM Website"
     />
   </a>
-
+  <a href="https://huggingface.co/Wayne-King/SRUM_BAGEL_7B_MoT">
+    <img
+        src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue"
+        alt="SRUM Model"
+    />
+  </a>
+  <a href="https://huggingface.co/datasets/Wayne-King/SRUM_6k_CompBench_Train">
+    <img
+        src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Datasets-blue"
+        alt="SRUM Data"
+    />
+  </a>
+  <a href="https://huggingface.co/spaces/Wayne-King/SRUM_Bagel_MoT-7B">
+    <img
+        src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue"
+        alt="SRUM Demo"
+    />
+  </a>
 </p>
 
 # SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
 > [Weiyang Jin*](https://github.com/WayneJin0918), [Yuwei Niu*](https://purshow.github.io/), Jiaqi Liao, [Chengqi Duan](https://scholar.google.com/citations?user=r9qb4ZwAAAAJ&hl=en), Aoxue Li, [Shenghua Gao](https://scholar.google.com/citations?user=fe-1v0MAAAAJ&hl=en), [Xihui Liu :email: ](https://xh-liu.github.io/)
 >
 > contact: [email protected]
-> 
-> We present **SRUM**, a post-training reward fine-tuning method based on Unified Multimodal Models (UMMs) leverages UMMs' inherent understanding capabilities to boost their generative abilities, bridging the gaps in performance caused by conflicts during the previous training phase. SRUM demonstrates exceptional generalization across both common positions and world knowledge..
-The figure below showcases SRUM's qualitative performance compared with SFT and Base Model.
-
-<!-- ## 🧠 Method
-BAGEL adopts a Mixture-of-Transformer-Experts (MoT) architecture to maximize the model’s capacity to learn from richly diverse multimodal information. Following the same principle of capacity maximization, it utilizes two separate encoders to capture pixel-level and semantic-level features of an image. The overall framework follows a Next Group of Token Prediction paradigm, where the model is trained to predict the next group of language or visual tokens as a compression target.
-
-BAGEL scales MoT’s capacity through Pre-training, Continued Training, and Supervised Finetuning on trillions of interleaved multimodal tokens spanning language, image, video, and web data. It surpasses open models on standard understanding and generation benchmarks and demonstrates advanced in-context multimodal abilities like free-form image editing, future frame prediction, 3D manipulation, world navigation, and sequential reasoning.
+>
+> **Abstract:** Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding capabilities within a single framework. However, a significant gap exists where a model's strong visual understanding often fails to transfer to its visual generation. A model might correctly understand an image based on user instructions, yet be unable to generate a faithful image from text prompts. This phenomenon directly raises a compelling question: Can a model achieve self-improvement by using its understanding module to reward its generation module? To bridge this gap and achieve self-improvement, we introduce SRUM, a self-rewarding post-training framework that can be directly applied to existing UMMs of various designs. SRUM creates a feedback loop where the model's own understanding module acts as an internal `evaluator`, providing corrective signals to improve its generation module, without requiring additional human-labeled data. To ensure this feedback is comprehensive, we designed a global-local dual reward system. To tackle the inherent structural complexity of images, this system offers multi-scale guidance: a **global reward** ensures the correctness of the overall visual semantics and layout, while a **local reward** refines fine-grained, object-level fidelity. SRUM leads to powerful capabilities and shows strong generalization, boosting performance on T2I-CompBench from 82.18 to **88.37** and on T2I-ReasonBench from 43.82 to **46.75**. Overall, our work establishes a powerful new paradigm for enabling a UMMs' understanding module to guide and enhance its own generation via self-rewarding.
 
-<p align="center"><img src="assets/arch.png" width="95%"></p>
+We present **SRUM**, a post-training reward fine-tuning method based on Unified Multimodal Models (UMMs) leverages UMMs' inherent understanding capabilities to boost their generative abilities, bridging the gaps in performance caused by conflicts during the previous training phase. SRUM demonstrates exceptional generalization across both common positions and world knowledge.
+The figure below showcases SRUM's qualitative performance compared with SFT and Base Model.
 
+## 📢 News
 
-## 🌱 Emerging Properties
-<p align="center"><img src="assets/emerging_curves.png" width="95%"></p>
+We sincerely thank all contributors from the open community for their valuable support.
 
-As we scale up BAGEL’s pretraining with more multimodal tokens, we observe consistent performance gains across understanding, generation, and editing tasks. Different capabilities emerge at distinct training stages—multimodal understanding and generation appear early, followed by basic editing, while complex, intelligent editing emerges later. This staged progression suggests an emergent pattern, where advanced multimodal reasoning builds on well-formed foundational skills. Ablation studies further show that combining VAE and ViT features significantly improves intelligent editing, underscoring the importance of visual-semantic context in enabling complex multimodal reasoning and further supporting its role in the emergence of advanced capabilities. -->
+- **Nov. 15, 2025:** We released the official [website](https://waynejin0918.github.io/srum_web/), [model](https://huggingface.co/Wayne-King/SRUM_BAGEL_7B_MoT), and [report](https://arxiv.org/abs/2510.12784) for SRUM. And please upvote for our [huggingface daily paper](https://huggingface.co/papers/2510.12784) as well as try the [demo](https://huggingface.co/spaces/Wayne-King/SRUM_Bagel_MoT-7B)
 
 ## 📮 Notice
-<!-- **Call for Bad Cases:** If you have encountered any cases where the model performs poorly, we would greatly appreciate it if you could share them in the [issue#11](https://github.com/ByteDance-Seed/Bagel/issues/11) or [Discord](https://discord.gg/Z836xxzy). -->
 Follow the Bagel's original settings, you should focus:
 
 **About Inference Hyperparameters:**
@@ -57,6 +85,62 @@ Follow the Bagel's original settings, you should focus:
   - `text_channel`: Like `channel`, but only applies to text condition (good for editing, may cause blur).
 - **If edited images appear blurry, try `global` CFG-Renorm, decrease `cfg_renorm_min` or decrease `cfg_scale`.**
 
+## 🔥 Quick Start
+
+1️⃣  Set up environment
+```bash
+git clone https://github.com/WayneJin0918/SRUM
+cd SRUM
+conda env create -f environment.yaml
+conda activate SRUM
+pip install -r requirements.txt
+```
+if flash attention is hard to pip, please follow:
+
+```bash
+wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.0.post2/flash_attn-2.7.0.post2+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+pip install flash_attn-2.7.0.post2+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+```
+
+Or you can follow the settings of Bagel
+
+2️⃣  Download Bagel pretrained or our SRUM checkpoint
+```python
+#bagel
+from huggingface_hub import snapshot_download
+
+save_dir = "models/BAGEL-7B-MoT"
+repo_id = "ByteDance-Seed/BAGEL-7B-MoT"
+cache_dir = save_dir + "/cache"
+
+snapshot_download(cache_dir=cache_dir,
+  local_dir=save_dir,
+  repo_id=repo_id,
+  local_dir_use_symlinks=False,
+  resume_download=True,
+  allow_patterns=["*.json", "*.safetensors", "*.bin", "*.py", "*.md", "*.txt"],
+)
+
+```
+
+```python
+#SRUM
+from huggingface_hub import snapshot_download
+
+save_dir = "models/SRUM_BAGEL_7B_MoT"
+repo_id = "Wayne-King/SRUM_BAGEL_7B_MoT"
+cache_dir = save_dir + "/cache"
+
+snapshot_download(cache_dir=cache_dir,
+  local_dir=save_dir,
+  repo_id=repo_id,
+  local_dir_use_symlinks=False,
+  resume_download=True,
+  allow_patterns=["*.json", "*.safetensors", "*.bin", "*.py", "*.md", "*.txt"],
+)
+
+```
+
 ## 📊 Benchmarks
 
 ### 1. Composition
@@ -91,5 +175,16 @@ Follow the Bagel's original settings, you should focus:
 
 *Performance comparison of Bagel models across four categories and their average scores. **Bold values** indicate the best performance in each column.*
 
+## ✍️ Citation
+
+```bibtex
+@article{deng2025bagel,
+  title   = {SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models},
+  author  = {Jin, Weiyang and Niu, Yuwei and Liao, Jiaqi and Duan, Chengqi and Li, Aoxue and Gao, Shenghua and Liu, Xihui},
+  journal = {arXiv preprint arXiv:2510.12784},
+  year    = {2025}
+}
+```
+
 ## 📜 License
 SRUM is licensed under the Apache 2.0.