Create README.md

README.md

@@ -0,0 +1,106 @@
+---
+license: mit
+language: en
+tags:
+- Pre-CoFactv3
+- Question Answering
+datasets:
+- FACTIFY5WQA
+metrics:
+- bleu
+pipeline_tag: question-answering
+library_name: transformers
+base_model: microsoft/deberta-v3-large
+widget:
+- text: "Who spent an entire season at aston vila without playing a single game?"
+  context: "Micah Richards spent an entire season at Aston Vila without playing a single game."
+  example_title: "Claim"
+- text: "Who spent an entire season at aston vila without playing a single game?"
+  context: "Despite speculation that Richards would leave Aston Villa before the transfer deadline for the 2018~19 season , he remained at the club , although he is not being considered for first team selection."
+  example_title: "Evidence"
+---
+
+# Pre-CoFactv3-Question-Answering
+
+## Model description
+
+This is a Question Answering model for **AAAI 2024 Workshop Paper: “Team Trifecta at Factify5WQA: Setting the Standard in Fact Verification with Fine-Tuning”**
+
+Its input are question and context, and output is the answers derived from the context. It is fine-tuned by **FACTIFY5WQA** dataset based on [**microsoft/deberta-v3-large**](https://huggingface.co/microsoft/deberta-v3-large) model.
+
+For more details, you can see our **paper** or [**GitHub**](https://github.com/AndyChiangSH/Pre-CoFactv3).
+
+## How to use?
+
+1. Download the model by hugging face transformers.
+```python
+from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
+
+model = AutoModelForQuestionAnswering.from_pretrained("AndyChiang/Pre-CoFactv3-Question-Answering")
+tokenizer = AutoTokenizer.from_pretrained("AndyChiang/Pre-CoFactv3-Question-Answering")
+```
+
+2. Create a pipeline.
+```python
+QA = pipeline("question-answering", model=model, tokenizer=tokenizer)
+```
+
+3. Use the pipeline to answer the question by context.
+```python
+QA_input = {
+    'context': "Micah Richards spent an entire season at Aston Vila without playing a single game.",
+    'question': "Who spent an entire season at aston vila without playing a single game?",
+}
+answer = QA(QA_input)
+print(answer)
+```
+
+## Dataset
+
+We utilize the dataset FACTIFY5WQA provided by the AAAI-24 Workshop Factify 3.0.
+
+This dataset is designed for fact verification, with the task of determining the veracity of a claim based on the given evidence.
+
+- **claim:** the statement to be verified.
+- **evidence:** the facts to verify the claim.
+- **question:** the questions generated from the claim by the 5W framework (who, what, when, where, and why).
+- **claim_answer:** the answers derived from the claim.
+- **evidence_answer:** the answers derived from the evidence.
+- **label:** the veracity of the claim based on the given evidence, which is one of three categories: Support, Neutral, or Refute.
+
+|  | Training | Validation | Testing | Total |
+| --- | --- | --- | --- | --- |
+| Support | 3500 | 750 | 750 | 5000 |
+| Neutral | 3500 | 750 | 750 | 5000 |
+| Refute | 3500 | 750 | 750 | 5000 |
+| Total | 10500 | 2250 | 2250 | 15000 |
+
+## Fine-tuning
+
+Fine-tuning is conducted by the Hugging Face Trainer API on the [Question Answering](https://huggingface.co/docs/transformers/tasks/question_answering) task.
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+
+- Pre-train language model: [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large)
+- Optimizer: adam
+- Learning rate: 0.00001
+- Max length of input: 3200
+- Batch size: 4
+- Epoch: 3
+- Device: NVIDIA RTX A5000
+
+## Testing
+
+We employ BLEU scores for both claim answer and evidence answer, taking the average of the two as the metric.
+
+| Claim Answer | Evidence Answer | Average |
+| ----- | ----- | ----- |
+| 0.5248 | 0.3963 | 0.4605 |
+
+## Other models
+
+[AndyChiang/Pre-CoFactv3-Text-Classification](https://huggingface.co/AndyChiang/Pre-CoFactv3-Text-Classification)
+
+## Citation