Ningze123/cis5190-news-source-deberta-v3

DeBERTa-V3-Base — News Source Classifier (FoxNews vs NBC)

Fine-tuned from microsoft/deberta-v3-base on the CIS 5190 final-project dataset of ~3,800 headlines from FoxNews and NBC News.

Test-set performance (757 holdout samples, stratified split, seed=42)

Metric	Value
Accuracy	0.8653
F1	0.8774
ROC-AUC	0.9448

Labels

• 0 → NBC
• 1 → FoxNews

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo = "Ningze123/cis5190-news-source-deberta-v3"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

text = "Trump rallies supporters in key swing state"
inputs = tok(text, return_tensors="pt", truncation=True, max_length=64)
with torch.no_grad():
    probs = torch.softmax(model(**inputs).logits, dim=-1)[0]
label = "FoxNews" if probs.argmax() == 1 else "NBC"
print(label, float(probs.max()))

Training summary

• Per-model learning-rate sweep across {5e-6, 1e-5, 2e-5, 3e-5, 5e-5, 1e-4}
• Linear warmup (10% of steps), weight decay 0.01, max 8 epochs, early stopping with patience=2 on validation F1.
• DeBERTa-V3 stability fixes: StableDropout replaced with nn.Dropout, classifier and pooler re-initialised with std=0.02.

See the project report for the full ablation study, including comparisons against DistilBERT, DistilRoBERTa, RoBERTa-Base, ELECTRA, and traditional ML baselines (TF-IDF + LR / SVM / NB / RF / XGBoost).