README.md

---
license: mit
datasets:
- chengpingan/CoConflictQA
language:
- en
base_model:
- meta-llama/Meta-Llama-3-8B
library_name: transformers
---

# 🤖 ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

This is the official model for **[ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation](https://arxiv.org/pdf/2502.15543)**.  

We investigate the internal mechanisms behind unfaithful generation and identify a subset of **mid-to-deep (70%–90% relative depth range) FFNs** that are disproportionately activated in such cases. Building on this insight, we propose Parametric Knowledge Muting through FFN Suppression (**ParamMute**), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs and calibrating the model toward retrieved knowledge. Experimental results on CoConflictQA and ConFiQA demonstrate that ParamMute significantly reduces knowledge conflicts and improves context fidelity.

---

## 📚 **Paper**
For a detailed explanation of the methodology and experiments, please refer to our paper:  
[**ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation**](https://arxiv.org/abs/2502.15543)

---

## 📊 Reproduce the Results
To reproduce the experiments and benchmarks from the paper, follow the instructions provided in the official GitHub repository:
[👉 GitHub: OpenBMB/ParamMute](https://github.com/OpenBMB/ParamMute).

## 📁 Model Details
- Model Name: ParamMute-8B-SFT
- Architecture: LLaMA3-8B-Instruct trained with the SFT
- Training Data: [CoConflictQA](https://huggingface.co/datasets/chengpingan/CoConflictQA) Dataset
- Pretrained Tasks: Knowledge-Augmented Generation, Contextual Faithfulness Evaluation

## 🔖 Citation
If you use PIP-KAG in your work, please consider citing our paper:
```
@misc{huang2025parammutesuppressingknowledgecriticalffns,
      title={ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation}, 
      author={Pengcheng Huang and Zhenghao Liu and Yukun Yan and Haiyan Zhao and Xiaoyuan Yi and Hao Chen and Zhiyuan Liu and Maosong Sun and Tong Xiao and Ge Yu and Chenyan Xiong},
      year={2025},
      eprint={2502.15543},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.15543}, 
}

```