Yasmine97/BiomedCLIP_for_AD

BiomedCLIP MRI + Clinical Text Classifier

This model fine-tunes BiomedCLIP (PubMedBERT ViT-B/16) for Alzheimer’s disease classification from MRI (3D volumes) and synthetic clinical text.

---

🧩 Model Details

• Backbone: BiomedCLIP (image + text encoders)
• Input MRI: 3D NIfTI → reduced to 3 mid-slices (axial, coronal, sagittal) → stacked into RGB
• Input Text: Synthetic patient note (tokenized with PubMedBERT)
• Fusion: Concatenate image & text embeddings
• Head: MLP (Linear → ReLU → Dropout → Linear) → 3-way classification
• Labels:
  • CN – Cognitively Normal
  • MCI – Mild Cognitive Impairment
  • Dementia

---

🚀 Usage

Install

pip install open_clip_torch nibabel torch torchvision


##Load Pretrained Model

import torch from model import BiomedClipClassifier, predict_from_paths

device = "cuda" if torch.cuda.is_available() else "cpu"

Load from repo (assuming you pushed pytorch_model.bin + config.json here)

model = BiomedClipClassifier.from_pretrained(".", device=device)

Example inference

pred, probs = predict_from_paths( model, "/path/to/sample_brain.nii.gz", "Patient shows mild memory impairment and hippocampal atrophy.", device=device )

print("Prediction:", pred) print("Probabilities:", probs) # [CN, MCI, Dementia]

##Run Inference 

python inference.py --weights . --mri /path/to/sample.nii.gz --text "Patient shows memory issues"