🔍 DeepSeek-PRCT

A LoRA fine-tuned DeepSeek-R1-14B model for detecting Population Replacement Conspiracy Theory (PRCT) content, optimized for formal news discourse with good cross-domain performance.

Overview

DeepSeek-PRCT is a LoRA adapter fine-tuned on Portuguese Telegram messages for detecting Population Replacement Conspiracy Theories. Despite training on informal social media, the model achieves exceptional performance on formal Italian news (0.892 F1-macro), demonstrating remarkable cross-domain transfer from informal to formal discourse.

Key Metrics

Dataset	F1-Macro	F1-Binary	Accuracy
News ITA (cross-domain)	0.892	0.850	0.908
Telegram PT (in-domain)	0.655	0.393	0.853

⚠️ Important: This model exhibits unusual behavior - it performs better on cross-domain (formal news) than on its training domain (informal Telegram), suggesting architectural sensitivity to reasoning-intensive formal text.

Model Description

DeepSeek-PRCT is a LoRA fine-tuned version of DeepSeek-R1-Distill-Qwen-14B, a reasoning-optimized language model. Trained on Portuguese Telegram messages, it demonstrates exceptional cross-domain transfer to Italian news headlines, achieving the highest F1-macro (0.892) among all evaluated configurations.

What are PRCTs?

Population Replacement Conspiracy Theories are false narratives claiming deliberate orchestration of demographic substitution through immigration. Main variants include:

• The Great Replacement Theory
• White Genocide
• Kalergi Plan
• Eurabia

These narratives are linked to extremist violence (Christchurch 2019, Utøya 2011) and pose serious threats to democratic discourse.

Model Configuration

Architecture

• Base Model: DeepSeek-R1-Distill-Qwen-14B (14B parameters)
• Adapter Type: LoRA (Low-Rank Adaptation)
• LoRA Rank: 16
• LoRA Alpha: 32
• Target Modules: q_proj, v_proj, k_proj, o_proj
• Special Feature: Reasoning chains with <think> tags

Label Mapping

• 0: Non-PRCT content
• 1: PRCT content (supports/mentions replacement narratives)

Input Requirements

• Maximum sequence length: 4096 tokens
• Input type: Text (Portuguese, Italian, Spanish)
• Preprocessing: DeepSeek tokenization

Intended Uses & Limitations

✅ Intended Uses

• Formal news analysis: Optimized for detecting PRCT in journalistic content
• Cross-domain deployment: Training on informal data, deployment on formal text
• Research: Understanding domain transfer in conspiracy detection
• High-accuracy applications: Where F1-macro >0.85 is required

⚠️ Limitations

• Inverse domain performance: Worse on training domain (Telegram) than test domain (News)
• Catastrophic forgetting: Fine-tuning degrades informal discourse understanding
• Inference cost: 16.3s per sample (slowest among evaluated models)
• Memory requirements: 14B parameters require substantial GPU memory
• Reasoning overhead: <think> tags increase token usage

Critical Note: This model's unusual performance profile (cross-domain > in-domain) makes it ideal for formal text analysis but not recommended for social media monitoring.

Training Data

• Primary training: Portuguese Telegram messages (n=919)
• Domain: Informal social media discourse, conspiracy-oriented channels
• PRCT prevalence: 15.7%
• Annotation: Expert annotators (Krippendorff's α=0.58)
• Time period: 2020-2024

Training Procedure

Hyperparameters

• Learning rate: 2e-5
• Batch size: 2 (with gradient accumulation)
• Training steps: 600
• Optimizer: AdamW 8-bit
• LoRA dropout: 0.05
• Weight decay: 0.01

Hardware

• GPU: NVIDIA A100 40GB
• Training time: ~4 hours
• Framework: PyTorch + PEFT

Results

Cross-Domain Performance (News ITA) ⭐

Metric	Score
Accuracy	0.908
Precision (Macro)	0.892
Recall (Macro)	0.892
F1-Macro	0.892
F1-Binary	0.850
Inference Time	16.67s/sample

Best-in-class performance: Highest F1-macro among all evaluated models on formal news text.

In-Domain Performance (Telegram PT)

Metric	Score
Accuracy	0.853
Precision (Macro)	0.716
Recall (Macro)	0.630
F1-Macro	0.655
F1-Binary	0.393
Inference Time	15.94s/sample

Performance degradation: 23.7pp drop from cross-domain to in-domain, indicating architectural sensitivity to text formality.

Usage

Installation

pip install transformers peft torch

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
import re

# Load base model and tokenizer
base_model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B"
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "erikbranmarino/DeepSeek-PRCT")

# Prepare prompt
text = "Your Italian or Portuguese text here"
prompt = f"""Classify if the following text contains Population Replacement Conspiracy Theory (PRCT) content.

Text: {text}

Classification (YES/NO):"""

# Generate prediction
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.0,
    do_sample=False
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Parse response (remove <think> tags if present)
response = re.sub(r'<think>.*?</think>', '', response, flags=re.DOTALL)
print(response)

Processing with Reasoning Chains

DeepSeek-R1 outputs reasoning chains in <think> tags. You can preserve or remove them:

def extract_classification(response):
    """Extract classification from DeepSeek response"""
    # Remove thinking process
    cleaned = re.sub(r'<think>.*?</think>', '', response, flags=re.DOTALL)
    
    # Extract YES/NO
    if "YES" in cleaned.upper():
        return "PRCT"
    elif "NO" in cleaned.upper():
        return "Non-PRCT"
    else:
        return "Uncertain"

# With reasoning visible
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("Reasoning:", full_response)
print("Classification:", extract_classification(full_response))

Architectural Insights

Why Cross-Domain > In-Domain?

DeepSeek-R1's reasoning architecture benefits from formal text structure:

1. Explicit logic: News headlines state claims directly
2. Reasoning alignment: Formal text matches model's reasoning chains
3. Catastrophic forgetting: Fine-tuning on informal text disrupts implicit pattern recognition

Recommendation: Use this model for formal news analysis, not social media monitoring.

Bias and Ethical Considerations

Known Biases

• Formality bias: Optimized for formal journalistic discourse
• Domain paradox: Underperforms on training domain
• Language bias: Best on Italian, weaker on Portuguese
• Reasoning bias: May over-analyze simple informal text

Ethical Use

• ⚠️ Not for automated censorship: Requires human review
• ✅ News monitoring: Ideal for tracking PRCTs in media
• ✅ Research purposes: Understanding cross-domain transfer
• ❌ Social media: Better alternatives exist (Mistral-PRCT)

We advocate for freedom of speech and constitutional rights.

Comparison with Mistral-PRCT

Feature	DeepSeek-PRCT	Mistral-PRCT
Best for	Formal news	Informal social media
News F1-Macro	0.892 ✅	0.753
Telegram F1-Macro	0.655	0.819 ✅
Inference Speed	16.3s ⚠️	4.6s ✅
Memory	14B params	7B params ✅

Choose DeepSeek-PRCT if: High accuracy on news, can afford slow inference Choose Mistral-PRCT if: Social media focus, need faster processing

Citation (to appear)

@inproceedings{marino2025prct,
  title={Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: 
         benchmarking LLMs and BERT models in Portuguese and Italian},
  author={Marino, Erik Bran and Vieira, Renata},
  booktitle={Proceedings of PROPOR 2026},
  year={2026}
}

Model Card Authors

Erik Bran Marino (Universidade de Évora, HYBRIDS Project)

Contact

• Email: [email protected]
• Project: MSCA HYBRIDS (Grant Agreement No. 101073351)
• Institution: Universidade de Évora, Portugal

License

MIT License - Free for research and educational purposes.

---

Developed as part of the HYBRIDS Marie Skłodowska-Curie Actions project