Update README.md
Browse files
README.md
CHANGED
|
@@ -13,18 +13,18 @@ tags:
|
|
| 13 |
- mteb
|
| 14 |
---
|
| 15 |
|
| 16 |
-
# granite-embedding-english-
|
| 17 |
<!-- Provide a quick summary of what the model is/does. -->
|
| 18 |
|
| 19 |
-
**Model Summary:** _granite-embedding-english-
|
| 20 |
|
| 21 |
-
The _granite-embedding-english-
|
| 22 |
The model is trained with ranking-specific loss functions such as ListMLE, along with model merging techniques to enhance performance. The reranker model shows strong performance on standard information retrieval benchmark (BEIR, MIRACL), long-document search benchmarks (MLDR), and on many enterprise use cases.
|
| 23 |
|
| 24 |
The latest granite embedding r2 release introduces two English embedding models, and one English reranking all based on the ModernBERT architecture:
|
| 25 |
- _granite-embedding-english-r2_ (**149M** parameters): with an output embedding size of _768_, replacing _granite-embedding-125m-english_.
|
| 26 |
- _granite-embedding-small-english-r2_ (**47M** parameters): A _first-of-its-kind_ reduced-size model, with 8192 context length support, fewer layers and a smaller output embedding size (_384_), replacing _granite-embedding-30m-english_.
|
| 27 |
-
- **_granite-embedding-english-
|
| 28 |
|
| 29 |
## Model Details
|
| 30 |
|
|
@@ -53,7 +53,7 @@ The model can then be used to jointly encode pairs of text to compute a relevanc
|
|
| 53 |
```python
|
| 54 |
from sentence_transformers import CrossEncoder, util
|
| 55 |
|
| 56 |
-
model_path = "ibm-granite/granite-embedding-english-
|
| 57 |
# Load the Sentence Transformer model
|
| 58 |
model = CrossEncoder(model_path)
|
| 59 |
|
|
@@ -89,7 +89,7 @@ The model can then be used to encode pairs of text
|
|
| 89 |
import torch
|
| 90 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 91 |
|
| 92 |
-
model_path = "ibm-granite/granite-embedding-english-
|
| 93 |
|
| 94 |
# Load the model and tokenizer
|
| 95 |
model = AutoModelForSequenceClassification.from_pretrained(model_path).eval()
|
|
@@ -164,7 +164,7 @@ for doc, score in retriever_ranked:
|
|
| 164 |
# --------------------------
|
| 165 |
# 2. Load reranker (149M)
|
| 166 |
# --------------------------
|
| 167 |
-
reranker_model_path = "ibm-granite/granite-embedding-english-
|
| 168 |
reranker = AutoModelForSequenceClassification.from_pretrained(reranker_model_path).eval()
|
| 169 |
reranker_tokenizer = AutoTokenizer.from_pretrained(reranker_model_path)
|
| 170 |
|
|
@@ -215,11 +215,11 @@ Each reranking model is evaluated with its maximum supported sequence length, wh
|
|
| 215 |
### Model Architecture and Key Features
|
| 216 |
|
| 217 |
The latest Granite Reranking r2 release introduces an English ranking model, based on the ModernBERT architecture:
|
| 218 |
-
- _granite-embedding-english-
|
| 219 |
|
| 220 |
The following table shows the structure of the two R2 models:
|
| 221 |
|
| 222 |
-
| Model | granite-embedding-english-
|
| 223 |
| :--------- |:--------:|
|
| 224 |
| Embedding size | 768 |
|
| 225 |
| Number of layers | 22 |
|
|
@@ -258,7 +258,7 @@ This comprehensive process captures critical information about the data, includi
|
|
| 258 |
We train Granite Reranking Model using IBM's computing cluster, BlueVela Cluster, which is outfitted with NVIDIA H100 80gb GPUs. This cluster provides a scalable and efficient infrastructure for training our models over multiple GPUs.
|
| 259 |
|
| 260 |
## Ethical Considerations and Limitations
|
| 261 |
-
The data used to train the base language model was filtered to remove text containing hate, abuse, and profanity. granite-embedding-english-
|
| 262 |
|
| 263 |
## Resources
|
| 264 |
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
|
|
|
|
| 13 |
- mteb
|
| 14 |
---
|
| 15 |
|
| 16 |
+
# granite-embedding-reranker-english-r2
|
| 17 |
<!-- Provide a quick summary of what the model is/does. -->
|
| 18 |
|
| 19 |
+
**Model Summary:** _granite-embedding-reranker-english-r2_ is a 149M parameter dense cross-encoder model from the Granite Embeddings collection that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768 based on context length of upto 8192 tokens. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets.
|
| 20 |
|
| 21 |
+
The _granite-embedding-reranker-english-r2_ model uses a cross-encoder architecture to compute high-quality relevance scores between queries and documents by jointly encoding their text, enabling precise reranking based on contextual alignment.
|
| 22 |
The model is trained with ranking-specific loss functions such as ListMLE, along with model merging techniques to enhance performance. The reranker model shows strong performance on standard information retrieval benchmark (BEIR, MIRACL), long-document search benchmarks (MLDR), and on many enterprise use cases.
|
| 23 |
|
| 24 |
The latest granite embedding r2 release introduces two English embedding models, and one English reranking all based on the ModernBERT architecture:
|
| 25 |
- _granite-embedding-english-r2_ (**149M** parameters): with an output embedding size of _768_, replacing _granite-embedding-125m-english_.
|
| 26 |
- _granite-embedding-small-english-r2_ (**47M** parameters): A _first-of-its-kind_ reduced-size model, with 8192 context length support, fewer layers and a smaller output embedding size (_384_), replacing _granite-embedding-30m-english_.
|
| 27 |
+
- **_granite-embedding-reranker-english-r2_** (**149M** parameters): reranker model based on _granite-embedding-english-r2_, with an output embedding size of _768_.
|
| 28 |
|
| 29 |
## Model Details
|
| 30 |
|
|
|
|
| 53 |
```python
|
| 54 |
from sentence_transformers import CrossEncoder, util
|
| 55 |
|
| 56 |
+
model_path = "ibm-granite/granite-embedding-reranker-english-r2"
|
| 57 |
# Load the Sentence Transformer model
|
| 58 |
model = CrossEncoder(model_path)
|
| 59 |
|
|
|
|
| 89 |
import torch
|
| 90 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 91 |
|
| 92 |
+
model_path = "ibm-granite/granite-embedding-reranker-english-r2"
|
| 93 |
|
| 94 |
# Load the model and tokenizer
|
| 95 |
model = AutoModelForSequenceClassification.from_pretrained(model_path).eval()
|
|
|
|
| 164 |
# --------------------------
|
| 165 |
# 2. Load reranker (149M)
|
| 166 |
# --------------------------
|
| 167 |
+
reranker_model_path = "ibm-granite/granite-embedding-reranker-english-r2"
|
| 168 |
reranker = AutoModelForSequenceClassification.from_pretrained(reranker_model_path).eval()
|
| 169 |
reranker_tokenizer = AutoTokenizer.from_pretrained(reranker_model_path)
|
| 170 |
|
|
|
|
| 215 |
### Model Architecture and Key Features
|
| 216 |
|
| 217 |
The latest Granite Reranking r2 release introduces an English ranking model, based on the ModernBERT architecture:
|
| 218 |
+
- _granite-embedding-reranker-english-r2_ (**149M** parameters): with an output embedding size of _768_.
|
| 219 |
|
| 220 |
The following table shows the structure of the two R2 models:
|
| 221 |
|
| 222 |
+
| Model | granite-embedding-reranker-english-r2 |
|
| 223 |
| :--------- |:--------:|
|
| 224 |
| Embedding size | 768 |
|
| 225 |
| Number of layers | 22 |
|
|
|
|
| 258 |
We train Granite Reranking Model using IBM's computing cluster, BlueVela Cluster, which is outfitted with NVIDIA H100 80gb GPUs. This cluster provides a scalable and efficient infrastructure for training our models over multiple GPUs.
|
| 259 |
|
| 260 |
## Ethical Considerations and Limitations
|
| 261 |
+
The data used to train the base language model was filtered to remove text containing hate, abuse, and profanity. granite-embedding-reranker-english-r2 is finetuned on English, and has a context length of 8192 tokens (longer texts will be truncated to this size).
|
| 262 |
|
| 263 |
## Resources
|
| 264 |
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
|