Create README.md

README.md

@@ -0,0 +1,106 @@
+---
+datasets:
+- psytechlab/rus_rudeft_wcl-wiki
+language:
+- ru
+base_model:
+- DeepPavlov/rubert-base-cased
+---
+
+# RuBERT base fine-tuned on ruDEFT and WCL Wiki Ru datasets for NER
+The model aims to extract terms and defenitions in a text. 
+Labels:
+- Term
+- Definition
+
+```python
+import torch
+import numpy as np
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+
+tokenizer = AutoTokenizer.from_pretrained("psytechlab/wcl-wiki_rudeft__ner-model")
+model = AutoModelForTokenClassification.from_pretrained("psytechlab/wcl-wiki_rudeft__ner-model")
+model.eval()
+
+inputs = tokenizer('оромо — это африканская этническая группа, проживающая в эфиопии и в меньшей степени в кении.', return_tensors="pt")
+with torch.no_grad():
+    outputs = model(**inputs)
+
+logits = outputs.logits
+predictions = torch.argmax(logits, dim=-1)[0].tolist() 
+
+tokens = inputs["input_ids"][0]
+word_ids = inputs.word_ids(batch_index=0)
+
+word_to_labels = {}
+for token_id, word_id, label_id in zip(tokens, word_ids, predictions):
+    if word_id is None:
+        continue
+    if word_id not in word_to_labels:
+        word_to_labels[word_id] = []
+    word_to_labels[word_id].append(label_id)
+
+word_level_predictions = [model.config.id2label[labels[0]] for labels in word_to_labels.values()]
+
+print(word_level_predictions)
+# ['B-Term', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O']
+```
+
+## Training procedure
+
+### Training
+The training was done with Trainier class that has next parameters:
+```python
+training_args = TrainingArguments(
+    eval_strategy="epoch",
+    save_strategy="epoch",
+    learning_rate=2e-5,
+    num_train_epochs=7,
+    weight_decay=0.01,
+)
+```
+
+### Metrics
+Metrics on combined set (ruDEFT + WCL Wiki Ru) `psytechlab/rus_rudeft_wcl-wiki`:
+```python
+              precision    recall  f1-score   support
+
+I-Definition       0.75      0.90      0.82      3344
+B-Definition       0.62      0.73      0.67       230
+      I-Term       0.80      0.85      0.82       524
+           O       0.97      0.91      0.94     11359
+      B-Term       0.96      0.93      0.94      2977
+
+    accuracy                           0.91     18434
+   macro avg       0.82      0.87      0.84     18434
+weighted avg       0.92      0.91      0.91     18434
+```
+
+Metrics only on `astromis/ruDEFT`:
+```python
+              precision    recall  f1-score   support
+
+           0       0.87      0.95      0.91       836
+           1       0.84      0.67      0.74       353
+
+    accuracy                           0.86      1189
+   macro avg       0.85      0.81      0.82      1189
+weighted avg       0.86      0.86      0.86      1189
+```
+
+Metrics only on `astromis/WCL_Wiki_Ru`:
+```python
+              precision    recall  f1-score   support
+
+I-Definition       0.00      0.00      0.00         0
+B-Definition       0.00      0.00      0.00         0
+      I-Term       0.72      0.78      0.75       135
+           O       1.00      0.93      0.96      9137
+      B-Term       0.99      0.95      0.97      2339
+
+    accuracy                           0.93     11611
+   macro avg       0.54      0.53      0.54     11611
+weighted avg       0.99      0.93      0.96     11611
+
+
+```