Arko007/fake-news-liar-political

LIAR Political Fact-Checker

Short description A RoBERTa-base model fine-tuned to perform binary classification (FAKE vs REAL) on short political statements (PolitiFact-style). Intended as an assistive tool for political fact-checking, claim triage, and analyst workflows. Not intended as a standalone authoritative verdict.

Model repository: Arko007/fake-news-liar-political

Model snapshot / overview

• Base model: RoBERTa-base (125M parameters)
• Task: Binary classification — FAKE (0) vs REAL (1)
• Domain: Short political statements (US-centric PolitiFact / LIAR)
• Intended uses: claim triage, researcher experiments, human-in-loop fact-checking
• Not intended for: medical/scientific fact-checking or automated high-stakes enforcement

Key performance (reported)

• Test accuracy: 71.25%
• Validation accuracy: 71.44%
• F1 (binary): 71.42%

Benchmarks for context:

• FakeStack (ensemble, 2023): 75.58% (ensemble)
• Standard RoBERTa baseline: ~68%
• BERT baseline: ~67%
• DeBERTa (6-class attempt): ~40% (multi-class differences)

Training & fine-tuning pipeline

This model used a 3-stage continual fine-tuning approach:

1. RoBERTa pretraining on general corpora (official RoBERTa)
2. Domain-adaptive fine-tuning on a 5M curated fake-news corpus (Arko007/fake-news-roberta-5M)
3. Final fine-tune on the LIAR dataset (converted to binary)

LIAR split (binary conversion):

• Train: 18,369 statements
• Validation: 2,297 statements
• Test: 2,296 statements

6→2 label mapping used for training:

• FAKE: pants-fire (0), false (1), barely-true (2)
• REAL: half-true (3), mostly-true (4), true (5)

Training hyperparameters:

• Optimizer: AdamW
• Learning rate: 5e-6
• Weight decay: 0.02
• Batch size: 96 (effective 192 with gradient accumulation)
• Epochs: up to 30 (early stopped at ~17)
• Class weights: FAKE = 1.0, REAL = 1.3
• Precision: BF16 mixed precision
• Hardware: NVIDIA L4 (24 GB VRAM)

Notes:

• Class weighting and sampling were applied to address imbalance in LIAR after mapping.
• Preprocessing: standard tokenization, minimal text normalization; see training scripts for exact sequence length/truncation policy.

Data

Primary data for final tuning:

• LIAR (chengxuphd/liar2) — political fact-check statements (PolitiFact). The dataset contains 6-way labels; this model uses the binary mapping shown above.

Upstream domain-adaptive data:

• Arko007/fake-news-roberta-5M — ~5M curated fake/real news samples used to adapt RoBERTa to misinformation-style news language before LIAR fine-tune.

Data caveats:

• LIAR / PolitiFact annotations reflect annotator and editorial judgment and are US-centric.
• Binary conversion flattens nuance; use caution when interpreting borderline cases.

Evaluation & metrics

Evaluation performed on withheld LIAR test split (2,296 statements). Metrics reported:

• Accuracy: 71.25%
• Binary F1: 71.42%

When comparing to other models, ensure consistent 6→2 mapping and preprocessing.

Usage example

Example using Hugging Face Transformers (PyTorch):

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

model_id = "Arko007/fake-news-liar-political"  # replace with HF repo id when uploaded
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

clf = pipeline("text-classification", model=model, tokenizer=tokenizer, return_all_scores=False, device=0)  # device=-1 for CPU

examples = [
    "The government passed a law guaranteeing free college for all citizens.",
    "Candidate A voted to privatize Social Security in 2017."
]

results = clf(examples, truncation=True)
print(results)  # check label names in model.config if needed

Label mapping in outputs:

• "FAKE" → predicted fake
• "REAL" → predicted real

Intended uses & limitations

Appropriate uses:

• Research experiments and baselines for political misinformation
• Human-in-the-loop claim triage and prioritization
• Tools to assist fact-checkers (not to replace them)

Limitations & risks:

• Domain-specific: optimized for short political statements; may underperform on long-form articles or non-political claims.
• Binary labels lose nuance from the original 6-class annotation.
• Biases from PolitiFact / LIAR (US-centric) will reflect in model outputs.
• Avoid using as the sole decision-maker for moderation, legal, or medical decisions.

Responsible deployment recommendations

• Always use with human oversight, especially for high-impact decisions.
• Report uncertainty and provide sources/evidence; do not surface model output as final judgment.
• Audit outputs for demographic and topical biases before deployment.
• Respect dataset licensing and attribution requirements when redistributing or publishing models.

Reproducibility & code

See repository training scripts for:

• Exact 6→2 mapping and preprocessing
• Checkpointing & evaluation code
• Hyperparameter schedules and scheduler settings

Citation

If you use this model, cite the model and primary datasets:

Suggested model citation:

@misc{liar-political-2025,
  title = {LIAR Political Fact-Checker},
  author = {Arko007},
  year = {2025},
  howpublished = {Hugging Face model hub: Arko007/fake-news-liar-political},
  note = {RoBERTa-base fine-tuned on 5M corpus and LIAR (binary)}
}

Also cite: Wang, W. Y. (2017) "Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection (LIAR).

Contact & maintainer

Maintainer: Arko007 (https://huggingface.co/Arko007)
Repository: https://github.com/Arko007/fake-news-liar-political

If you find licensing issues, data-provenance errors, or safety concerns, please open an issue in the repo.