Update README.md
Browse files
README.md
CHANGED
|
@@ -86,15 +86,15 @@ We clean the content of the remaining dataset entries according to the following
|
|
| 86 |
|
| 87 |
# Evaluation
|
| 88 |
|
| 89 |
-
To evaluate we used Kotlin Humaneval
|
| 90 |
|
| 91 |
Fine-tuned model:
|
| 92 |
|
| 93 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
| 94 |
|:---------------------------:|:----------------------------------------:|
|
| 95 |
| `base model` | 26.09 |
|
| 96 |
-
| `fine-tuned model` | 29.19 |
|
| 97 |
|
| 98 |
# Ethical Considerations and Limitations
|
| 99 |
|
| 100 |
-
|
|
|
|
| 86 |
|
| 87 |
# Evaluation
|
| 88 |
|
| 89 |
+
To evaluate we used [Kotlin Humaneval](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval)
|
| 90 |
|
| 91 |
Fine-tuned model:
|
| 92 |
|
| 93 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
| 94 |
|:---------------------------:|:----------------------------------------:|
|
| 95 |
| `base model` | 26.09 |
|
| 96 |
+
| `fine-tuned model` | **29.19** |
|
| 97 |
|
| 98 |
# Ethical Considerations and Limitations
|
| 99 |
|
| 100 |
+
CodeLlama-7B-KStack-full and its variants are a new technology that carries risks with use. The testing conducted to date could not cover all scenarios. For these reasons, as with all LLMs, CodeLlama-7B-KStack-full's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. The model was fine-tuned on a specific data format (Kotlin tasks), and deviation from this format can also lead to inaccurate or undesirable responses to user queries. Therefore, before deploying any applications of CodeLlama-7B-KStack-full, developers should perform safety testing and tuning tailored to their specific applications of the model.
|