Update README.md
Browse files
README.md
CHANGED
|
@@ -1,11 +1,5 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
-
datasets:
|
| 4 |
-
- ncbi/pubmed
|
| 5 |
-
- starmpcc/Asclepius-Synthetic-Clinical-Notes
|
| 6 |
-
- akemiH/NoteChat
|
| 7 |
-
- zhengyun21/PMC-Patients
|
| 8 |
-
- jpcorb20/medical_wikipedia
|
| 9 |
language:
|
| 10 |
- en
|
| 11 |
base_model:
|
|
@@ -26,7 +20,7 @@ The MediPhi Model Collection comprises 7 small language models of 3.8B parameter
|
|
| 26 |
## Model Details
|
| 27 |
### Model Description
|
| 28 |
|
| 29 |
-
This model is `MediPhi` obtained by merging
|
| 30 |
|
| 31 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
| 32 |
- **Model type:** Phi3
|
|
@@ -90,7 +84,7 @@ Researchers should apply responsible AI best practices, including mapping, measu
|
|
| 90 |
|
| 91 |
torch.random.manual_seed(0)
|
| 92 |
|
| 93 |
-
model_name = "microsoft/MediPhi"
|
| 94 |
model = AutoModelForCausalLM.from_pretrained(
|
| 95 |
model_name,
|
| 96 |
device_map="cuda",
|
|
@@ -134,13 +128,7 @@ Check `microsoft/Phi-3.5-mini-instruct` for details about the tokenizer, require
|
|
| 134 |
### Training Data
|
| 135 |
|
| 136 |
Continual Pre-training:
|
| 137 |
-
- PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
|
| 138 |
-
- Medical Guideline `epfl-llm/guidelines`.
|
| 139 |
-
- Medical Wikipedia `jpcorb20/medical_wikipedia`.
|
| 140 |
- Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
|
| 141 |
-
- Clinical documents:
|
| 142 |
-
- `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
|
| 143 |
-
- mtsamples
|
| 144 |
|
| 145 |
See paper for details.
|
| 146 |
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
base_model:
|
|
|
|
| 20 |
## Model Details
|
| 21 |
### Model Description
|
| 22 |
|
| 23 |
+
This model is `MediPhi-MedCode` obtained by merging the fine-tuned MedCode expert with the SLERP technique into its base model at 50%.
|
| 24 |
|
| 25 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
| 26 |
- **Model type:** Phi3
|
|
|
|
| 84 |
|
| 85 |
torch.random.manual_seed(0)
|
| 86 |
|
| 87 |
+
model_name = "microsoft/MediPhi-MedCode"
|
| 88 |
model = AutoModelForCausalLM.from_pretrained(
|
| 89 |
model_name,
|
| 90 |
device_map="cuda",
|
|
|
|
| 128 |
### Training Data
|
| 129 |
|
| 130 |
Continual Pre-training:
|
|
|
|
|
|
|
|
|
|
| 131 |
- Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
|
|
|
|
|
|
|
|
|
|
| 132 |
|
| 133 |
See paper for details.
|
| 134 |
|