lemms commited on
Commit
c2a7297
·
verified ·
1 Parent(s): d66fb15

Add OpenLLM Small Extended 6k model

Browse files

OpenLLM Small Extended model trained for 6,000 steps.

- Model: GPT-style transformer (35.8M parameters)
- Training: 6,000 steps on SQUAD Wikipedia passages
- Tokenizer: SentencePiece BPE (32k vocabulary)
- License: GPL-3.0 / Commercial available

For more details, see: https://github.com/louischua/openllm

Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # OpenLLM Small Extended 6k
2
 
3
  This is the OpenLLM Small Extended model trained for 6,000 steps on Wikipedia passages from the SQUAD dataset.
 
1
+ ---
2
+ language:
3
+ - en
4
+ license:
5
+ - gpl-3.0
6
+ - other
7
+ tags:
8
+ - text-generation
9
+ - language-model
10
+ - gpt
11
+ - transformer
12
+ - open-source
13
+ - squad
14
+ - wikipedia
15
+ datasets:
16
+ - squad
17
+ metrics:
18
+ - perplexity
19
+ - text-generation-quality
20
+ library_name: transformers
21
+ pipeline_tag: text-generation
22
+ model-index:
23
+ - name: OpenLLM Small Extended 6k
24
+ results:
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ type: squad
29
+ name: SQUAD Wikipedia Passages
30
+ metrics:
31
+ - type: perplexity
32
+ value: 816.04
33
+ - type: training_loss
34
+ value: 5.4302
35
+ ---
36
+
37
  # OpenLLM Small Extended 6k
38
 
39
  This is the OpenLLM Small Extended model trained for 6,000 steps on Wikipedia passages from the SQUAD dataset.