multilingual_go_emotions_V1.1 / README.md

AnasAlokla

Update README.md

4dd0af9 verified 6 months ago

preview code

raw

history blame

7.21 kB

metadata

datasets:
  - AnasAlokla/multilingual_go_emotions
language:
  - ar
  - en
  - fr
  - es
  - de
  - tr
metrics:
  - accuracy
  - f1
  - recall
base_model:
  - google-bert/bert-base-multilingual-cased
new_version: AnasAlokla/multilingual_go_emotions
pipeline_tag: text-classification

Overview

Model trained from bert-base-multilingual-cased on the multilingual_go_emotions dataset for multi-label classification.

Multilingual GoEmotions Chatbot Task Report

Author: Anas Hamid Alokla

This repository/space contains details about a multilingual emotion classification model and chatbot based on the GoEmotions dataset.

Test Set Performance

The following table shows the performance metrics of the fine-tuned model on the test set, broken down by emotion category.

The table below shows the performance of the test model:

Labels	Accuracy	Precision	Recall	F1	MCC	Support	Threshold
admiration	0.935	0.616	0.629	0.622	0.587	2790	0.20
amusement	0.969	0.706	0.800	0.750	0.735	1866	0.10
anger	0.956	0.360	0.344	0.352	0.329	1128	0.10
annoyance	0.925	0.274	0.263	0.268	0.229	1704	0.10
approval	0.904	0.286	0.330	0.306	0.256	2094	0.10
caring	0.962	0.299	0.370	0.331	0.314	816	0.10
confusion	0.961	0.369	0.333	0.350	0.331	1020	0.35
curiosity	0.932	0.404	0.569	0.473	0.445	1734	0.10
desire	0.985	0.412	0.418	0.415	0.407	414	0.35
disappointment	0.959	0.276	0.204	0.235	0.217	1014	0.25
disapproval	0.926	0.238	0.324	0.275	0.240	1398	0.10
disgust	0.977	0.364	0.297	0.327	0.317	600	0.25
embarrassment	0.993	0.538	0.292	0.378	0.393	240	0.85
excitement	0.971	0.285	0.327	0.305	0.291	624	0.20
fear	0.986	0.562	0.474	0.514	0.509	498	0.40
gratitude	0.989	0.928	0.890	0.909	0.903	2004	0.75
grief	0.998	0.273	0.333	0.300	0.301	36	0.15
joy	0.959	0.382	0.453	0.415	0.396	1032	0.20
love	0.972	0.734	0.780	0.757	0.742	1812	0.20
nervousness	0.995	0.339	0.358	0.348	0.346	120	0.20
optimism	0.970	0.553	0.427	0.482	0.471	1062	0.45
pride	0.997	0.387	0.286	0.329	0.331	84	0.15
realization	0.964	0.191	0.152	0.169	0.152	792	0.25
relief	0.992	0.129	0.145	0.137	0.133	138	0.10
remorse	0.986	0.566	0.614	0.589	0.583	516	0.25
sadness	0.968	0.506	0.444	0.473	0.457	1062	0.40
surprise	0.973	0.465	0.450	0.457	0.444	828	0.45
neutral	0.731	0.577	0.630	0.602	0.400	10524	0.10

Test Model Performance (Threshold = 0.5)

The table below shows the performance of the test model with a threshold of 0.5:

Label	Accuracy	Precision	Recall	F1	MCC	Support	Threshold
admiration	0.939	0.678	0.542	0.603	0.574	2790	0.5
amusement	0.970	0.760	0.693	0.725	0.710	1866	0.5
anger	0.964	0.471	0.210	0.291	0.299	1128	0.5
annoyance	0.944	0.383	0.100	0.158	0.175	1704	0.5
approval	0.934	0.472	0.176	0.256	0.260	2094	0.5
caring	0.973	0.424	0.246	0.312	0.310	816	0.5
confusion	0.964	0.399	0.310	0.349	0.333	1020	0.5
curiosity	0.945	0.480	0.375	0.421	0.396	1734	0.5
desire	0.985	0.420	0.394	0.406	0.399	414	0.5
disappointment	0.964	0.333	0.169	0.224	0.220	1014	0.5
disapproval	0.950	0.338	0.163	0.220	0.212	1398	0.5
disgust	0.979	0.390	0.252	0.306	0.303	600	0.5
embarrassment	0.992	0.430	0.333	0.376	0.375	240	0.5
excitement	0.976	0.324	0.237	0.274	0.265	624	0.5
fear	0.987	0.576	0.458	0.510	0.507	498	0.5
gratitude	0.988	0.905	0.903	0.904	0.898	2004	0.5
grief	0.999	0.294	0.139	0.189	0.202	36	0.5
joy	0.964	0.422	0.355	0.385	0.369	1032	0.5
love	0.972	0.771	0.714	0.741	0.727	1812	0.5
nervousness	0.995	0.351	0.275	0.308	0.308	120	0.5
optimism	0.971	0.566	0.416	0.480	0.471	1062	0.5
pride	0.997	0.432	0.226	0.297	0.311	84	0.5
realization	0.969	0.231	0.117	0.156	0.150	792	0.5
relief	0.994	0.143	0.065	0.090	0.094	138	0.5
remorse	0.987	0.603	0.562	0.582	0.576	516	0.5
sadness	0.969	0.531	0.424	0.471	0.458	1062	0.5
surprise	0.973	0.465	0.436	0.450	0.436	828	0.5
neutral	0.742	0.670	0.400	0.501	0.362	10524	0.5

AnasAlokla
/

multilingual_go_emotions_V1.1

Overview

Multilingual GoEmotions Chatbot Task Report

Links

Test Set Performance

Test Model Performance (Threshold = 0.5)