ibm-granite

/

granite-embedding-reranker-english-r2

like 19

Follow

IBM Granite 3.55k

Hi,
Thanks for the great work! I was trying out the model with the example provided and it seems the scores for clearly non-relevant documents are super close to 1. Is this expected behavior?

from sentence_transformers import CrossEncoder, util

model_path = "ibm-granite/granite-embedding-reranker-english-r2"
# Load the Sentence Transformer model
model = CrossEncoder(model_path)

passages = [
               "Romeo and Juliet is a play by William Shakespeare.",
               "Climate change refers to long-term shifts in temperatures.",
               "Shakespeare also wrote Hamlet and Macbeth.",
               "Water is an inorganic compound with the chemical formula H2O.",
               "In liquid form, H2O is also called 'water' at standard temperature and pressure."
            ]

query = "what is the chemical formula of water?"

# encodes query and passages jointly and computes relevance score.
ranks = model.rank(query, passages, return_documents=True)

# Print document rank and relevance score
for rank in ranks:
    print(f"- #{rank['corpus_id']} ({rank['score']}): {rank['text']}")
###############################
- #3 (1.0): Water is an inorganic compound with the chemical formula H2O.
- #4 (1.0): In liquid form, H2O is also called 'water' at standard temperature and pressure.
- #1 (0.9997370839118958): Climate change refers to long-term shifts in temperatures.
- #0 (0.997329831123352): Romeo and Juliet is a play by William Shakespeare.
- #2 (0.9960688352584839): Shakespeare also wrote Hamlet and Macbeth.