add models

.gitattributes

@@ -0,0 +1,17 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,114 @@
+---
+language: pl
+tags:
+- fastText
+datasets:
+- kgr10
+---
+
+# KGR10 FastText Polish word embeddings
+
+Distributional language model (both textual and binary) for Polish (word embeddings) trained on KGR10 corpus (over 4 billion of words) using Fasttext with the following variants (all possible combinations):
+- dimension: 100, 300
+- method: skipgram, cbow
+- tool: FastText, Magnitude
+- source text: plain, plain.lower, plain.lemma, plain.lemma.lower
+
+## Models
+
+In the repository you can find 4 selected models, that were examined in the paper (see Citation). 
+A model that performed the best is the default model/config (see `default_config.json`).
+
+## Usage
+
+To use these embedding models easily, it is required to install [embeddings](https://github.com/CLARIN-PL/embeddings).
+
+```bash
+pip install clarinpl-embeddings
+```
+
+### Utilising the default model (the easiest way)
+
+Word embedding:
+
+```python
+from embeddings.embedding.auto_flair import AutoFlairWordEmbedding
+from flair.data import Sentence
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+
+embedding = AutoFlairWordEmbedding.from_hub("clarin-pl/fastText-kgr10")
+embedding.embed([sentence])
+
+for token in sentence:
+    print(token)
+    print(token.embedding)
+```
+
+Document embedding (averaged over words):
+
+```python
+from embeddings.embedding.auto_flair import AutoFlairDocumentEmbedding
+from flair.data import Sentence
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+
+embedding = AutoFlairDocumentEmbedding.from_hub("clarin-pl/fastText-kgr10")
+embedding.embed([sentence])
+
+print(sentence.embedding)
+```
+
+### Customisable way
+
+Word embedding:
+
+```python
+from embeddings.embedding.static.embedding import AutoStaticWordEmbedding
+from embeddings.embedding.static.fasttext import KGR10FastTextConfig
+from flair.data import Sentence
+
+config = KGR10FastTextConfig(method='cbow', dimension=100)
+embedding = AutoStaticWordEmbedding.from_config(config)
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+embedding.embed([sentence])
+
+for token in sentence:
+    print(token)
+    print(token.embedding)
+```
+
+Document embedding (averaged over words):
+
+```python
+from embeddings.embedding.static.embedding import AutoStaticDocumentEmbedding
+from embeddings.embedding.static.fasttext import KGR10FastTextConfig
+from flair.data import Sentence
+
+config = KGR10FastTextConfig(method='cbow', dimension=100)
+embedding = AutoStaticDocumentEmbedding.from_config(config)
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+embedding.embed([sentence])
+
+print(sentence.embedding)
+```
+
+
+## Citation
+
+The link below leads to the NextCloud directory with all variants of embeddings. If you use it, please cite the following article:
+
+```
+@article{kocon2018embeddings,
+author = {Koco\'{n}, Jan and Gawor, Micha{\l}},
+title = {Evaluating {KGR10} {P}olish word embeddings in the recognition of temporal
+expressions using {BiLSTM-CRF}},
+journal = {Schedae Informaticae},
+volume = {27},
+year = {2018},
+url = {http://www.ejournals.eu/Schedae-Informaticae/2018/Volume-27/art/13931/},
+doi = {10.4467/20838476SI.18.008.10413}
+}
+```

default_config.json

@@ -0,0 +1,4 @@
+{
+  "method": "skipgram",
+  "dimension": 300
+}

kgr10.plain.cbow.dim100.neg10.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43a99e3ba9f91e50d82d1fa30d78fa2b8663bae2571a8ceb60f1b86c9fe587c4
+size 3657638661

kgr10.plain.cbow.dim300.neg10.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1dfe6ad69103ce48d2b5f24d4ad3ba6771b8a892ea2b01f0847fc367d64add6
+size 10839393861

kgr10.plain.skipgram.dim100.neg10.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56a7b5bb1eb817ccf6dc229988913af9cfb6fd1ca1d8ae331fd7e07d7a0e2c62
+size 3657638661

kgr10.plain.skipgram.dim300.neg10.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:664fdeba47a5aa694edc2360ad89deeb069c7ef1875d7d3782205c37a2dcc072
+size 10839393861

module.json

@@ -0,0 +1,3 @@
+{
+  "type": "embeddings.embedding.static.fasttext.KGR10FastTextEmbedding"
+}

test/dummy.model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c49c8b8e85de626b44d1c170b6d66f7d7e6fdf0b692039959e9e7e97b5cad2e9
+size 90705