ShantanuT01/dactyl-bert-tiny-finetuned

DACTYL Text Detector

Training Configuration

{
    "training_split": "training",
    "evaluation_split": "testing",
    "results_path": "libauc-bert-tiny.csv",
    "num_epochs": 1,
    "model_path": "ShantanuT01/dactyl-bert-tiny-pretrained",
    "tokenizer": "prajjwal1/bert-tiny",
    "optimizer": "SOTAs",
    "optimizer_type": "libauc",
    "optimizer_args": {
        "lr": 1e-05
    },
    "loss_fn": "tpAUC_KL_Loss",
    "reset_classification_head": true,
    "loss_type": "libauc",
    "loss_fn_args": {
        "data_len": 466005
    },
    "needs_loss_fn_as_parameter": false,
    "save_path": "ShantanuT01/dactyl-bert-tiny-finetuned",
    "training_args": {
        "batch_size": 64,
        "needs_sampler": true,
        "needs_index": true,
        "shuffle": false,
        "sampling_rate": 0.5,
        "apply_sigmoid": true
    },
    "best_model_path": "best-tpauc-model-bert-tiny"
}

Results

model	AP Score	AUC Score	OPAUC Score	TPAUC Score
DeepSeek-V3	0.986177	0.997357	0.991422	0.916469
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-RedditWritingPrompts-testing	0.0242264	0.805738	0.619715	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-abstracts-testing	0.473122	0.955828	0.857774	0.0525074
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-news-testing	0.0223995	0.776301	0.579092	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-reviews-testing	0.0167409	0.861497	0.526359	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-student_essays-testing	0.0102312	0.621629	0.51884	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-tweets-testing	0.200169	0.949163	0.728122	0
claude-3-5-haiku-20241022	0.96485	0.993769	0.977445	0.780686
claude-3-5-sonnet-20241022	0.981121	0.997236	0.987999	0.883237
gemini-1.5-flash	0.93374	0.989888	0.955699	0.57296
gemini-1.5-pro	0.856575	0.96967	0.909663	0.190672
gpt-4o-2024-11-20	0.950062	0.990051	0.968528	0.695722
gpt-4o-mini	0.992634	0.999149	0.995225	0.953515
llama-3.2-90b	0.895571	0.978201	0.933229	0.366601
llama-3.3-70b	0.952439	0.992252	0.968929	0.699379
mistral-large-latest	0.979345	0.996937	0.986758	0.871375
mistral-small-latest	0.986836	0.997499	0.991305	0.915284
overall	0.976784	0.980759	0.947371	0.492028