End of training

README.md

@@ -1,78 +1,17 @@
 ---
-license: other
+base_model: alpindale/Mistral-7B-v0.2-hf
 tags:
 - axolotl
 - generated_from_trainer
-- Mistral
-- instruct
-- finetune
-- chatml
-- gpt4
-- synthetic data
-- science
-- physics
-- chemistry
-- biology
-- math
-base_model: alpindale/Mistral-7B-v0.2-hf
-datasets:
-- allenai/ai2_arc
-- camel-ai/physics
-- camel-ai/chemistry
-- camel-ai/biology
-- camel-ai/math
-- metaeval/reclor
-- openbookqa
-- mandyyyyii/scibench
-- derek-thomas/ScienceQA
-- TIGER-Lab/ScienceEval
-- jondurbin/airoboros-3.2
-- LDJnr/Capybara
-- Cot-Alpaca-GPT4-From-OpenHermes-2.5
-- STEM-AI-mtl/Electrical-engineering
-- knowrohit07/saraswati-stem
-- sablo/oasst2_curated
-- lmsys/lmsys-chat-1m
-- TIGER-Lab/MathInstruct
-- bigbio/med_qa
-- meta-math/MetaMathQA-40K
-- openbookqa
-- piqa
-- metaeval/reclor
-- derek-thomas/ScienceQA
-- scibench
-- sciq
-- Open-Orca/SlimOrca
-- migtissera/Synthia-v1.3
-- TIGER-Lab/ScienceEval
-- allenai/WildChat
-- microsoft/orca-math-word-problems-200k
-- openchat/openchat_sharegpt4_dataset
-- teknium/GPTeacher-General-Instruct
-- m-a-p/CodeFeedback-Filtered-Instruction
-- totally-not-an-llm/EverythingLM-data-V3
-- HuggingFaceH4/no_robots
-- OpenAssistant/oasst_top1_2023-08-25
-- WizardLM/WizardLM_evol_instruct_70k
-language:
-- en
+model-index:
+- name: Einstein-v6-7B
+  results: []
 ---
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/6468ce47e134d050a58aa89c/CxDk4KKhQqL-Pg0AMn1gb.png)
-
-<center><h1>📝 Note 📝</h1></center>
-
-📢 This model is currently in 1.5 epoch and this is a pre release. Main release will be available in 1 days. 
-
--------------
-
-# 🔬 Einstein-v6-7B
 
-This model is a full fine-tuned version of [alpindale/Mistral-7B-v0.2-hf](https://huggingface.co/alpindale/Mistral-7B-v0.2-hf) on diverse datasets.
-
-This model is finetuned using `8xRTX3090` + `1xRTXA6000` using [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl).
-
-This model's training was sponsored by [sablo.ai](https://sablo.ai). 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 <details><summary>See axolotl config</summary>
 
 axolotl version: `0.4.0`
@@ -227,73 +166,53 @@ special_tokens:
   unk_token: "<unk>"
 tokens:
   - "<|im_start|>"
-```
-
-</details><br>
-
-# 💬 Prompt Template
-
-You can use this prompt template while using the model:
-
-### ChatML
 
 ```
-<|im_start|>system
-{system}<|im_end|>
-<|im_start|>user
-{user}<|im_end|>
-<|im_start|>assistant
-{asistant}<|im_end|>
-```
-
-This prompt template is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating), which means you can format messages using the
-`tokenizer.apply_chat_template()` method:
-
-```python
-messages = [
-    {"role": "system", "content": "You are helpful AI asistant."},
-    {"role": "user", "content": "Hello!"}
-]
-gen_input = tokenizer.apply_chat_template(message, return_tensors="pt")
-model.generate(**gen_input)
-```
-
-# 🔄 Quantizationed versions
 
-## GGUF [@bartowski](https://huggingface.co/bartowski)
-
-- https://huggingface.co/bartowski/Einstein-v6-7B-GGUF
-
-## ExLlamaV2 [@bartowski](https://huggingface.co/bartowski)
-
-- https://huggingface.co/bartowski/Einstein-v6-7B-exl2
+</details><br>
 
-# 🎯 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+# Einstein-v6-7B
 
+This model is a fine-tuned version of [alpindale/Mistral-7B-v0.2-hf](https://huggingface.co/alpindale/Mistral-7B-v0.2-hf) on the None dataset.
 
-# 🤖 Additional information about training
+## Model description
 
-This model is full fine-tuned for 2 epoch. 
+More information needed
 
-Total number of steps was 2412.
+## Intended uses & limitations
 
-<details><summary>Loss graph</summary>
+More information needed
 
+## Training and evaluation data
 
-</details><br>
+More information needed
 
-# 🤝 Acknowledgments
+## Training procedure
 
-Thanks to [sablo.ai](https://sablo.ai) for sponsoring this model.
+### Training hyperparameters
 
-Thanks to all the dataset authors mentioned in the datasets section.
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 9
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 36
+- total_eval_batch_size: 9
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 2
 
-Thanks to [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) for making the repository I used to make this model.
+### Training results
 
-Thanks to all open source AI community.
 
-[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 
-If you would like to support me:
+### Framework versions
 
-[☕ Buy Me a Coffee](https://www.buymeacoffee.com/weyaxi)
+- Transformers 4.38.2
+- Pytorch 2.1.2+cu118
+- Datasets 2.18.0
+- Tokenizers 0.15.0

model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4aa419e433b185323444a3b8350d979b45a038e6887330b3a1edaacf48ac9f2d
+oid sha256:c89fd0fface188ca3f7988aa53f25e087292d72ca99cd52ef8cb52cf180ad2ff
 size 4943178720

model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:73c5a608fc2645deb20b706f73174b5ddc9df7a86e31b670b4ea896c064afb27
+oid sha256:49dd97160e0a8ff75303f02969df38307407c8800ce94aaa86611ceb6727bca0
 size 4999819336

model-00003-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91a448af004507aa23616541e844c83722dc86610112b69ad59f13b4dc59b466
+oid sha256:03098a839ef612f1efe325b376aa90bc8311a01c1236120d9ca7934eb9b12fed
 size 4540532728

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52510a040ad00eb50bcdf98721ac331fdb2d3b22b03a27088ed90f48debc4104
+size 539576