README.md

---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:321
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: nomic-ai/modernbert-embed-base
widget:
- source_sentence: Since what year have they been married?
  sentences:
  - 'Graph: Team Coco Knowledge Graph

    Node ID: 2015_conan_cuba

    Category: events

    Name: Conan in Cuba

    Type: Event


    Description: Conan O''Brien traveled to Havana to film a historic episode—the
    first by an American late-night host in over 50 years—part of his ''Conan Without
    Borders'' specials.


    Relationships:

    - Host conan_obrien

    - Occurred during conan_tbs'
  - 'Description: Liza Powel O''Brien is an American playwright and podcast host.
    She met Conan O''Brien in 2000 while working at an advertising agency, and they
    married in 2002. She has written numerous plays staged at theaters like the Geffen
    Playhouse and Ojai Playwrights Conference, and in 2022 she launched the history
    podcast "Significant Others" on Conan''s Team Coco network.'
  - "Relationships:\n- Spouse conan_obrien (Strength: very strong)\n  Description:\
    \ Married since 2002; they have two children together.\n- Podcast host team_coco\
    \ (Strength: moderate)\n  Description: Hosts the \"Significant Others\" podcast\
    \ under the Team Coco banner."
- source_sentence: Which team produced Conan's final late night episode?
  sentences:
  - 'Graph: Team Coco Knowledge Graph

    Node ID: 2021_conan_finale

    Category: events

    Name: Conan''s Final Late Night Episode

    Type: Event


    Description: The final episode of ''Conan'' on TBS, marking the end of Conan O''Brien''s
    28-year run as a late-night host with heartfelt goodbyes and memorable comedy
    moments.


    Relationships:

    - Honoree conan_obrien

    - Participant andy_richter

    - Producer team_coco'
  - 'References:

    - ([Conan O''Brien - Wikipedia](https://en.wikipedia.org/wiki/Conan_O%27Brien))

    - ([Andy Richter Net Worth | Celebrity Net Worth](https://www.celebritynetworth.com))'
  - 'Description: Airing on SiriusXM''s Team Coco Radio channel.'
- source_sentence: What type of document is referenced for the tour?
  sentences:
  - "Relationships:\n- Late-night host conan_obrien (Strength: core talent)\n  Description:\
    \ Conan's break in late night came through NBC.\n- Production partner conaco (Strength:\
    \ strong)\n  Description: NBC worked with Conaco on Conan's shows.\n\nAwards and\
    \ Recognitions:\n- Legacy of late-night programming"
  - 'Major Events:

    - 1993 Joined ''Late Night'' with Conan

    - 2009 Transitioned to ''The Tonight Show''

    - 2010 Concluded run as Conan''s bandleader'
  - 'References:

    - ([The Legally Prohibited from Being Funny on Television Tour - Wikipedia](https://en.wikipedia.org/wiki/The_Legally_Prohibited_from_Being_Funny_on_Television_Tour))'
- source_sentence: In what year did Triumph the Insult Comic Dog debut?
  sentences:
  - "Relationships:\n- Host-guest (Prankster) conan_obrien (Strength: moderate)\n\
    \  Description: Repeatedly played the 'Mac and Me' gag, to Conan's feigned exasperation.\n\
    \nMajor Events:\n- 2004 First Mac and Me Gag on 'Late Night'\n- 2021 Final TBS\
    \ Show Prank cameo"
  - 'Awards and Recognitions:

    - MFA in Fiction Writing from Columbia University

    - Playwright with works at the Geffen Playhouse and Ojai Playwrights Conference

    - Host of the "Significant Others" podcast (2022–present)'
  - 'Graph: Team Coco Knowledge Graph

    Node ID: triumph_insult_comic_dog

    Category: creative works

    Name: Triumph the Insult Comic Dog

    Type: Puppet character


    Description: A recurring canine puppet character, voiced by Robert Smigel, that
    debuted on Conan''s ''Late Night'' in 1997, known for roasting celebrities and
    absurd humor.


    Relationships:

    - Creator/performer robert_smigel

    - Host platform conan_obrien'
- source_sentence: Who are the hosts of The Conan & Jordan Show?
  sentences:
  - 'Awards and Recognitions:

    - 7 Primetime Emmy nominations for writing on Conan''s shows

    - 10 WGA Award nominations (with 2 wins)

    - 2 Daytime Emmy nominations for Animated Program performance


    Major Events:

    - 1993 Late Night Debut – Joined Conan''s first show as sidekick.

    - 2000 Departure – Left ''Late Night'' to pursue acting.

    - 2010 Tour & TBS Move – Reunited with Conan on the live tour and TBS.'
  - 'Graph: Team Coco Knowledge Graph

    Node ID: the_conan_and_jordan_show

    Category: shows

    Name: The Conan & Jordan Show (radio program)

    Type: Show


    Description: A spin-off audio series on SiriusXM''s Team Coco Radio, launched
    in 2023, featuring Conan O''Brien and Jordan Schlansky continuing their comedic
    odd-couple dynamic.'
  - 'Major Events:

    - 2010 Premiere – ''Conan'' debuted on TBS.

    - 2015 ''Conan Without Borders'' – International travel specials aired.

    - 2021 Finale – Conan ended his TBS run.


    References:

    - ([Conan O''Brien - Wikipedia](https://en.wikipedia.org/wiki/Conan_O%27Brien))'
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB)
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.7222222222222222
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8611111111111112
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9166666666666666
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9444444444444444
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.7222222222222222
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2870370370370371
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18333333333333338
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09444444444444446
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7222222222222222
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8611111111111112
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9166666666666666
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9444444444444444
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8363985989991439
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.800925925925926
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.8041634291634291
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.6944444444444444
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8888888888888888
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9166666666666666
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9722222222222222
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6944444444444444
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.29629629629629634
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18333333333333335
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09722222222222224
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6944444444444444
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8888888888888888
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9166666666666666
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9722222222222222
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8349701465406345
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7909722222222222
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.791703216374269
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.6666666666666666
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8611111111111112
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9166666666666666
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9444444444444444
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6666666666666666
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.28703703703703703
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18333333333333335
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09444444444444446
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6666666666666666
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8611111111111112
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9166666666666666
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9444444444444444
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.8074890903790802
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7627314814814814
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7662037037037037
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.6388888888888888
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8611111111111112
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.9166666666666666
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9444444444444444
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6388888888888888
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2870370370370371
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.18333333333333338
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09444444444444446
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6388888888888888
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8611111111111112
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9166666666666666
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9444444444444444
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.803777679552595
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7574074074074074
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7597654530591711
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.6111111111111112
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7777777777777778
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8333333333333334
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9166666666666666
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6111111111111112
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2592592592592593
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16666666666666669
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09166666666666669
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6111111111111112
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7777777777777778
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8333333333333334
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9166666666666666
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7608354868794361
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7111441798941799
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7139831037236697
      name: Cosine Map@100
---

# Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB)

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision d556a88e332558790b210f7bdbe87da2fa94a8d8 -->
- **Maximum Sequence Length:** 1024 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("densonsmith/modernbert-embed-quickb")
# Run inference
sentences = [
    'Who are the hosts of The Conan & Jordan Show?',
    "Graph: Team Coco Knowledge Graph\nNode ID: the_conan_and_jordan_show\nCategory: shows\nName: The Conan & Jordan Show (radio program)\nType: Show\n\nDescription: A spin-off audio series on SiriusXM's Team Coco Radio, launched in 2023, featuring Conan O'Brien and Jordan Schlansky continuing their comedic odd-couple dynamic.",
    "Awards and Recognitions:\n- 7 Primetime Emmy nominations for writing on Conan's shows\n- 10 WGA Award nominations (with 2 wins)\n- 2 Daytime Emmy nominations for Animated Program performance\n\nMajor Events:\n- 1993 Late Night Debut – Joined Conan's first show as sidekick.\n- 2000 Departure – Left 'Late Night' to pursue acting.\n- 2010 Tour & TBS Move – Reunited with Conan on the live tour and TBS.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | dim_768    | dim_512   | dim_256    | dim_128    | dim_64     |
|:--------------------|:-----------|:----------|:-----------|:-----------|:-----------|
| cosine_accuracy@1   | 0.7222     | 0.6944    | 0.6667     | 0.6389     | 0.6111     |
| cosine_accuracy@3   | 0.8611     | 0.8889    | 0.8611     | 0.8611     | 0.7778     |
| cosine_accuracy@5   | 0.9167     | 0.9167    | 0.9167     | 0.9167     | 0.8333     |
| cosine_accuracy@10  | 0.9444     | 0.9722    | 0.9444     | 0.9444     | 0.9167     |
| cosine_precision@1  | 0.7222     | 0.6944    | 0.6667     | 0.6389     | 0.6111     |
| cosine_precision@3  | 0.287      | 0.2963    | 0.287      | 0.287      | 0.2593     |
| cosine_precision@5  | 0.1833     | 0.1833    | 0.1833     | 0.1833     | 0.1667     |
| cosine_precision@10 | 0.0944     | 0.0972    | 0.0944     | 0.0944     | 0.0917     |
| cosine_recall@1     | 0.7222     | 0.6944    | 0.6667     | 0.6389     | 0.6111     |
| cosine_recall@3     | 0.8611     | 0.8889    | 0.8611     | 0.8611     | 0.7778     |
| cosine_recall@5     | 0.9167     | 0.9167    | 0.9167     | 0.9167     | 0.8333     |
| cosine_recall@10    | 0.9444     | 0.9722    | 0.9444     | 0.9444     | 0.9167     |
| **cosine_ndcg@10**  | **0.8364** | **0.835** | **0.8075** | **0.8038** | **0.7608** |
| cosine_mrr@10       | 0.8009     | 0.791     | 0.7627     | 0.7574     | 0.7111     |
| cosine_map@100      | 0.8042     | 0.7917    | 0.7662     | 0.7598     | 0.714      |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 321 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 321 samples:
  |         | anchor                                                                            | positive                                                                            |
  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                              |
  | details | <ul><li>min: 7 tokens</li><li>mean: 14.03 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 74.79 tokens</li><li>max: 117 tokens</li></ul> |
* Samples:
  | anchor                                                                                      | positive                                                                                                                                                                                                                                                                                                                                                                              |
  |:--------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>What brand did Jeff Ross help establish?</code>                                       | <code>Graph: Team Coco Knowledge Graph<br>Node ID: jeff_ross_producer<br>Category: people<br>Name: Jeff Ross (Producer)<br>Type: Person<br><br>Description: Jeff Ross is a television producer who has served as Conan O'Brien's executive producer since 1993. He is a key business partner in Conan's media ventures and helped establish the Team Coco brand.</code>               |
  | <code>In what year did Conan O'Brien launch the travel show 'Conan O'Brien Must Go'?</code> | <code>Description: Conan O'Brien is an American television host, comedian, writer, actor, and producer, best known for hosting late-night shows including "Late Night with Conan O'Brien", "The Tonight Show with Conan O'Brien", and "Conan". He also hosts the podcast "Conan O'Brien Needs a Friend" and, in 2024, launched the travel show "Conan O'Brien Must Go" on Max.</code> |
  | <code>What is the strength of the network TBS?</code>                                       | <code>- Network tbs (Strength: parent)<br>  Description: TBS provided the platform for the show.</code>                                                                                                                                                                                                                                                                               |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 4
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch   | Step   | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:-------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 1.0     | 6      | -             | 0.7909                 | 0.8034                 | 0.7711                 | 0.7992                 | 0.6908                |
| 1.7901  | 10     | 16.3044       | -                      | -                      | -                      | -                      | -                     |
| **2.0** | **12** | **-**         | **0.8364**             | **0.8294**             | **0.8022**             | **0.8038**             | **0.7691**            |
| 3.0     | 18     | -             | 0.8364                 | 0.8313                 | 0.8059                 | 0.7938                 | 0.7599                |
| 3.3951  | 20     | 5.6348        | 0.8364                 | 0.8350                 | 0.8075                 | 0.8038                 | 0.7608                |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.12.4
- Sentence Transformers: 3.4.0
- Transformers: 4.48.1
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.2.0
- Tokenizers: 0.21.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->