Hugging Face's logo

bod9
/
stella_en_1.5B_v5-ecom-retrieval

Sentence Similarity
sentence-transformers
Safetensors
feature-extraction
Eval Results
Model card Files
xet
Community
stella_en_1.5B_v5-ecom-retrieval / README.md
bod9's picture
bod9
Update README.md
8fb63a0 verified
preview code
|
raw
history blame contribute delete
13.7 kB
metadata
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
base_model: NovaSearch/stella_en_1.5B_v5
widget:
  - source_sentence: $230 pool for outdoors not plastic
    sentences:
      - >-
        Wireless Outdoor Security Camera, WiFi Solar Rechargeable Battery Power
        IP Surveillance Home Cameras, 1080P, Human Motion Detection, Night
        Vision, 2-Way Audio, 4dbi Antenna, IP65 Waterproof, Cloud/SD
      - Bestway SaluSpa Miami Inflatable Hot Tub, 4-Person AirJet Spa
      - Mens Plush Robe - Fleece Robe, Mens Bathrobe - Fig -Small/Medium
  - source_sentence: (hearing aid not amplifer)
    sentences:
      - Hearing Aid Cleaning Wire for Sound Tubes (2 Packs of 5)
      - >-
        Hearing Aids, Enjoyee Hearing Aids for Seniors Rechargeable Hearing
        Amplifier with Noise Cancelling for Adults Hearing Loss, Digital Ear
        Hearing Assist Devices with Volume Control
      - >-
        24 Pieces Checking Erasable Pencils Red Pencils Pre-Sharpened #2 HB with
        Erasable Tops for Checking Map Coloring Tests Grading
  - source_sentence: (can not use in the usa) european 220voltage hair tools
    sentences:
      - Umarex 2252109 Brodax Air Pistol .177 BB
      - >-
        One-Step Hair Dryer & Volumizer Hot Air Brush, 3-in-1 Hair Dryer Brush
        Styler for Straightening, Curling, Salon Negative Ion Ceramic
        Lightweight Blow Dryers Straightener Curl Hair Brush
      - >-
        Mini Portable Flat Iron Tourmaline Ceramic Dual Voltage Travel Iron for
        Worldwide Use LED Indicator LOVANI Hair Straightener (Ceramic Mini)
  - source_sentence: '''not my circus not my monkeys my monkeys flyshirt'''
    sentences:
      - >-
        Dresswel Women This is My Circus These are My Monkeys T-Shirt Mom Life
        Graphic Tee Pocket Shirt Casual Tops
      - >-
        Goyunwell Nylon Black Zippers by The Yard #5 10 Yards Nylon Black Long
        Zipper Tape for Sewing 20Pcs Gunmetal Pulls Slider Zipper by The Yard
        Black Zipper Roll for Craft Bag Purse Sewing Black Tape
      - Not My Circus Not My Monkeys Party T-Shirt
  - source_sentence: '#1 small corded treadmill without remote control'
    sentences:
      - >-
        Goplus Under Desk Treadmill, with Touchable LED Display and Wireless
        Remote Control, Built-in 3 Workout Modes and 12 Programs, Walking
        Jogging Machine, Superfit Electric Treadmill for Home Office
      - Pencil Guy Untipped white round pencil, no eraser 144 to a box
      - >-
        SUNNY HEALTH & FITNESS ASUNA Space Saving Treadmill, Motorized with
        Speakers for AUX Audio Connection - 8730G
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@5
  - cosine_ndcg@10
  - cosine_mrr@1
  - cosine_mrr@5
  - cosine_mrr@10
  - cosine_map@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on NovaSearch/stella_en_1.5B_v5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: ir evaluation
          type: ir_evaluation
        metrics:
          - type: cosine_accuracy@1
            value: 0.5243984708792444
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7189116258151563
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.782325163031257
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8452889588486621
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5243984708792444
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.44764260550183643
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4002248706993479
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.3198335956824826
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.09367303103726933
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.21358059074273028
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.2980042886250134
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.43181596262310956
            name: Cosine Recall@10
          - type: cosine_ndcg@5
            value: 0.46762394517912087
            name: Cosine Ndcg@5
          - type: cosine_ndcg@10
            value: 0.46811760590873697
            name: Cosine Ndcg@10
          - type: cosine_mrr@1
            value: 0.5243984708792444
            name: Cosine Mrr@1
          - type: cosine_mrr@5
            value: 0.625680233865528
            name: Cosine Mrr@5
          - type: cosine_mrr@10
            value: 0.6341315350816145
            name: Cosine Mrr@10
          - type: cosine_map@10
            value: 0.34926714503106293
            name: Cosine Map@10
          - type: cosine_map@100
            value: 0.4065326888005573
            name: Cosine Map@100
      - task:
          type: graded-ir
          name: Graded IR
        dataset:
          name: gr evaluation
          type: gr_evaluation
        metrics:
          - type: cosine_accuracy@1
            value: 0.708792444344502
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9037553406791096
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9462559028558579
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9840341803463009
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.708792444344502
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.6468780451240538
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.6044974139869574
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.5283786822577018
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.12721101888273206
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.30930165915538804
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.4489712213025254
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7098551817763595
            name: Cosine Recall@10
          - type: cosine_ndcg@5
            value: 0.7127144186187505
            name: Cosine Ndcg@5
          - type: cosine_ndcg@10
            value: 0.7543447490248549
            name: Cosine Ndcg@10
          - type: cosine_mrr@1
            value: 0.708792444344502
            name: Cosine Mrr@1
          - type: cosine_mrr@5
            value: 0.8065999550258622
            name: Cosine Mrr@5
          - type: cosine_mrr@10
            value: 0.8118267710352285
            name: Cosine Mrr@10
          - type: cosine_map@10
            value: 0.6051494969120079
            name: Cosine Map@10
          - type: cosine_map@100
            value: 0.6944358205631005
            name: Cosine Map@100

SentenceTransformer based on NovaSearch/stella_en_1.5B_v5

This is a sentence-transformers model finetuned from NovaSearch/stella_en_1.5B_v5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: NovaSearch/stella_en_1.5B_v5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 1024 dimensions
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model 
  (1): Pooling({'word_embedding_dimension': 1536, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 1536, 'out_features': 1024, 'bias': True, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bod9/fulldshardneg")
# Run inference
sentences = [
    '#1 small corded treadmill without remote control',
    'SUNNY HEALTH & FITNESS ASUNA Space Saving Treadmill, Motorized with Speakers for AUX Audio Connection - 8730G',
    'Goplus Under Desk Treadmill, with Touchable LED Display and Wireless Remote Control, Built-in 3 Workout Modes and 12 Programs, Walking Jogging Machine, Superfit Electric Treadmill for Home Office',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.5244
cosine_accuracy@3 0.7189
cosine_accuracy@5 0.7823
cosine_accuracy@10 0.8453
cosine_precision@1 0.5244
cosine_precision@3 0.4476
cosine_precision@5 0.4002
cosine_precision@10 0.3198
cosine_recall@1 0.0937
cosine_recall@3 0.2136
cosine_recall@5 0.298
cosine_recall@10 0.4318
cosine_ndcg@5 0.4676
cosine_ndcg@10 0.4681
cosine_mrr@1 0.5244
cosine_mrr@5 0.6257
cosine_mrr@10 0.6341
cosine_map@10 0.3493
cosine_map@100 0.4065

Graded IR

  • Dataset: gr_evaluation
  • Evaluated with GradedIREvaluator.GradedIREvaluator
Metric Value
cosine_accuracy@1 0.7088
cosine_accuracy@3 0.9038
cosine_accuracy@5 0.9463
cosine_accuracy@10 0.984
cosine_precision@1 0.7088
cosine_precision@3 0.6469
cosine_precision@5 0.6045
cosine_precision@10 0.5284
cosine_recall@1 0.1272
cosine_recall@3 0.3093
cosine_recall@5 0.449
cosine_recall@10 0.7099
cosine_ndcg@5 0.7127
cosine_ndcg@10 0.7543
cosine_mrr@1 0.7088
cosine_mrr@5 0.8066
cosine_mrr@10 0.8118
cosine_map@10 0.6051
cosine_map@100 0.6944

Framework Versions

  • Python: 3.11.12
  • Sentence Transformers: 4.1.0
  • Transformers: 4.51.3
  • PyTorch: 2.6.0+cu124
  • Accelerate: 1.6.0
  • Datasets: 3.6.0
  • Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}