CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_2025-04-05_23-00-04

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_2025-04-05_23-00-04")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: Who is the head of Slytherin house?',
    'Lots of people had come from Muggle families and, like him, hadn\'t had any idea that they were witches and wizards. There was so much to learn that even people like Ron didn\'t have much of a head start. Friday was an important day for Harry and Ron. They finally managed to find their way down to the Great Hall for breakfast without getting lost once. "What have we got today?" Harry asked Ron as he poured sugar on his porridge. "Double Potions with the Slytherins," said Ron. "Snape\'s Head of Slytherin House. They say he always favors them - we\'ll be able to see if it\'s true." "Wish McGonagall favored us," said Harry. Professor McGonagall was head of Gryffindor House, but it hadn\'t stopped her from giving them a huge pile of homework the day before. Just then, the mail arrived.',
    'It ought to reinforce the lesson I am trying to teach you." Harry felt the blood surge to his head and heard a thumping noise in his ears. So he told evil, nasty, attention-seeking stories, did he? She was watching him with her head slightly to one side, still smiling widely, as though she knew exactly what he was thinking and was waiting to see whether he would start shouting again. With a massive effort Harry looked away from her, dropped his schoolbag beside the straight-backed chair, and sat down. "There," said Umbridge sweetly, "we\'re getting better at controlling our temper already, aren\'t we? Now, you are going to be doing some lines for me, Mr. Potter. No, not with your quill," she added, as Harry bent down to open his bag. "You\'re going to be using a rather special one of mine. Here you are."',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1975
cosine_accuracy@3	0.284
cosine_accuracy@5	0.321
cosine_accuracy@10	0.3827
cosine_precision@1	0.1975
cosine_precision@3	0.0947
cosine_precision@5	0.0642
cosine_precision@10	0.0395
cosine_recall@1	0.1626
cosine_recall@3	0.2202
cosine_recall@5	0.251
cosine_recall@10	0.3004
cosine_ndcg@10	0.2398
cosine_mrr@10	0.2546
cosine_map@100	0.2143

Training Details

Training Dataset

Unnamed Dataset

• Size: 470 training samples
• Columns: sentence_0, sentence_1, sentence_2, sentence_3, sentence_4, sentence_5, and sentence_6
• Approximate statistics based on the first 470 samples:

  	sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  type	string	string	string	string	string	string	string
  details	min: 17 tokens mean: 27.46 tokens max: 59 tokens	min: 37 tokens mean: 199.03 tokens max: 512 tokens	min: 37 tokens mean: 172.8 tokens max: 512 tokens	min: 39 tokens mean: 168.26 tokens max: 512 tokens	min: 35 tokens mean: 158.22 tokens max: 423 tokens	min: 22 tokens mean: 171.01 tokens max: 512 tokens	min: 36 tokens mean: 175.27 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  Represent this sentence for searching relevant passages: What object does Dumbledore leave to Hermione in his will in the seventh book, and how does it connect to a lesson from the first book?	"What else could I do with it?" Evidently Scrimgeour had no suggestions. After squinting at Ron for a moment or two, he turned back to Dumbledore's will. " 'To Miss Hermione Jean Granger, I leave my copy of The Tales of Beedle the Bard, in the hope that she will find it entertaining and instructive.' " Scrimgeour now pulled out of the bag a small book that looked as ancient as the copy of Secrets of the Darkest Art upstairs. Its binding was stained and peeling in places.	He was five, watching Dudley riding a new red bicycle, and his heart was bursting with jealousy. ... He was nine, and Ripper the bulldog was chasing him up a tree and the Dursleys were laughing below on the lawn. ... He was sitting under the Sorting Hat, and it was telling him he would do well in Slytherin. ... Hermione was lying in the hospital wing, her face covered with thick black hair. ... A hundred dementors were closing in on him beside the dark lake. ... Cho Chang was drawing nearer to him under the mistletoe. ... No, said a voice in Harry's head, as the memory of Cho drew nearer, you're not watching that, you're not watching it, it's private - He felt a sharp pain in his knee. Snape's office had come back into view and he realized that he had fallen to the floor; one of his knees had collided painfully with the leg of Snape's desk. He looked up at Snape, who had lowered his wand and was rubbing his wrist.	"I still don't get why he didn't turn you in for using that book," said Ron. "He must've known where you were getting it all from." "He knew," said Harry bitterly. "He knew when I used Sectumsempra. He didn't really need Legilimency. ... He might even have known before then, with Slughorn talking about how brilliant I was at Potions. ... Shouldn't have left his old book in the bottom of that cupboard, should he?" "But why didn't he turn you in?" "I don't think he wanted to associate himself with that book," said Hermione.	Harry stood rooted to the spot. The letters from the Ministry, Mr. Weasley, and Sirius were crushed in his left hand. Don't leave the house again, whatever you do. DO NOT LEAVE YOUR AUNT AND UNCLE'S HOUSE. "You heard me!" said Uncle Vernon, bending forward now, so that his massive purple face came closer to Harry's, so that Harry actually felt flecks of spit hit his face. "Get going! You were all keen to leave half an hour ago! I'm right behind you! Get out and never darken our doorstep again!	Thought she should. She hasn't had the pleasure yet." "Genius!" yelled Harry. "It was nothing," said Ron, though he looked delighted with himself. "So what's new with you?"	Then he saw Hermione's foot swing out of nowhere. Black let go of Harry with a grunt of pain; Ron had thrown himself on Black's wand hand and Harry heard a faint clatter - He fought free of the tangle of bodies and saw his own wand rolling across the floor; he threw himself toward it but - "Argh!" Crookshanks had joined the fray; both sets of front claws had sunk themselves deep into Harry's arm; Harry threw him off, but Crookshanks now darted toward Harry's wand - "NO YOU DON'T!" roared Harry, and he aimed a kick at Crookshanks that made the cat leap aside, spitting; Harry snatched up his wand and turned - "Get out of the way!" he shouted at Ron and Hermione. They didn't need telling twice. Hermione, gasping for breath, her lip bleeding, scrambled aside, snatching up her and Ron's wands. Ron crawled to the four-poster and collapsed onto it, panting, his white face now tinged with green, both hands clutching his broken leg.
  Represent this sentence for searching relevant passages: Who teaches History of Magic at Hogwarts?	Three times a week they went out to the greenhouses behind the castle to study Herbology, with a dumpy little witch called Professor Sprout, where they learned how to take care of all the strange plants and fungi, and found out what they were used for. Easily the most boring class was History of Magic, which was the only one taught by a ghost. Professor Binns had been very old indeed when he had fallen asleep in front of the staff room fire and got up next morning to teach, leaving his body behind him. Binns droned on and on while they scribbled down names and dates, and got Emeric the Evil and Uric the Oddball mixed up. Professor Flitwick, the Charms teacher, was a tiny little wizard who had to stand on a pile of books to see over his desk. At the start of their first class he took the roll call, and when he reached Harry's name he gave an excited squeak and toppled out of sight. Professor McGonagall was again different.	A lantern was bobbing at the distant foot of the castle. Harry was so pleased to see it he felt he could even endure Filch's wheezy criticisms of his tardiness and rants about how his timekeeping would improve with the regular application of thumbscrews. It was not until the glowing yellow light was ten feet away from them, and Harry had pulled off his Invisibility Cloak so that he could be seen, that he recognized, with a rush of pure loathing, the uplit hooked nose and long, black, greasy hair of Severus Snape. "Well, well, well," sneered Snape, taking out his wand and tapping the padlock once, so that the chains snaked backward and the gates creaked open. "Nice of you to turn up, Potter, although you have evidently decided that the wearing of school robes would detract from your appearance."	"It's got to be her - d'you know where she is?" "Let's see. ..." Nick's head wobbled a little on his ruff as he turned hither and thither, peering over the heads of the swarming students. "That's her over there, Harry, the young woman with the long hair." Harry looked in the direction of Nick's transparent, pointing finger and saw a tall ghost who caught sight of Harry looking at her, raised her eyebrows, and drifted away through a solid wall.	The man broke it. "You speak it?" "Yes, I speak it," said Riddle. He moved forward into the room, allowing the door to swing shut behind him. Harry could not help but feel a resentful admiration for Voldemort's complete lack of fear. His face merely expressed disgust and, perhaps, disappointment. "Where is Marvolo?"	"Harry, dear, if you'll just tiptoe across the hall, it's through this door here -" CRASH. "Tonks!" cried Mrs. Weasley exasperatedly, turning to look behind her. "I'm sorry!" wailed Tonks, who was lying flat on the floor.	grades. "So top grade's O for 'Outstanding,' " she was saying, "and then there's A -" "No, E," George corrected her, "E for 'Exceeds Expectations.' And I've always thought Fred and I should've got E in everything, because we exceeded expectations just by turning up for the exams." They all laughed except Hermione, who plowed on, "So after E, it's A for 'Acceptable,' and that's the last pass grade, isn't it?" "Yep," said Fred, dunking an entire roll in his soup, transferring it to his mouth, and swallowing it whole. "Then you get P for 'Poor' " - Ron raised both his arms in mock celebration - "and D for 'Dreadful.' " "And then T," George reminded him. "T?" asked Hermione, looking appalled. "Even lower than a D? What on earth does that stand for?" " 'Troll,' " said George promptly. Harry laughed again, though he was not sure whether or not George was joking. He imagined trying to conceal from Hermione that he had received T's in all his O.W.L.s and immediately resolved to work harder fro...
  Represent this sentence for searching relevant passages: Which object are Harry, Ron, and Hermione searching for inside Bellatrix Lestrange's vault at Gringotts?	Half buried in a rising tide of red-hot treasure, they struggled and yelled as Harry thrust the sword through the handle of Hufflepuff's cup, hooking it onto the blade. "Impervius!" screeched Hermione in an attempt to protect herself, Ron, and the goblins from the burning metal. Then the worst scream yet made Harry look down: Ron and Hermione were waist-deep in treasure, struggling to keep Bogrod from slipping beneath the rising tide, but Griphook had sunk out of sight and nothing but the tips of a few long fingers were left in view. Harry seized Griphook's fingers and pulled. The blistered goblin emerged by degrees, howling. "Liberacorpus!" yelled Harry, and with a crash he and Griphook landed on the surface of the swelling treasure, and the sword flew out of Harry's hand.	They turned left, streaked down the stairs, and stopped sharply in front of the twins, the chain clattering loudly on the flagged stone floor. "We won't be seeing you," Fred told Professor Umbridge, swinging his leg over his broomstick. "Yeah, don't bother to keep in touch," said George, mounting his own. Fred looked around at the assembled students, and at the silent, watchful crowd. "If anyone fancies buying a Portable Swamp, as demonstrated upstairs, come to number ninety-three, Diagon Alley - Weasleys' Wizarding Wheezes," he said in a loud voice. "Our new premises!" "Special discounts to Hogwarts students who swear they're going to use our products to get rid of this old bat," added George, pointing at Professor Umbridge.	"Sh - she's not coming here, is she?" Aunt Marge was Uncle Vernon's sister. Even though she was not a blood relative of Harry's (whose mother had been Aunt Petunia's sister), he had been forced to call her "Aunt" all his life. Aunt Marge lived in the country, in a house with a large garden, where she bred bulldogs. She didn't often stay at Privet Drive, because she couldn't bear to leave her precious dogs, but each of her visits stood out horribly vividly in Harry's mind. At Dudley's fifth birthday party, Aunt Marge had whacked Harry around the shins with her walking stick to stop him from beating Dudley at musical statues. A few years later, she had turned up at Christmas with a computerized robot for Dudley and a box of dog biscuits for Harry. On her last visit, the year before Harry started at Hogwarts, Harry had accidentally trodden on the tail of her favorite dog. Ripper had chased Harry out into the garden and up a tree, and Aunt Marge had refused to call him off until past midni...	Harry glanced over at Ron, who had always had a problem with nerves; Harry had hoped that winning their final match last term might have cured it, but apparently not: Ron was a delicate shade of green. None of the first five applicants saved more than two goals apiece. To Harry's great disappointment, Cormac McLaggen saved four penalties out of five. On the last one, however, he shot off in completely the wrong direction; the crowd laughed and booed and McLaggen returned to the ground grinding his teeth. Ron looked ready to pass out as he mounted his Cleansweep Eleven. "Good luck!" cried a voice from the stands. Harry looked around, expecting to see Hermione, but it was Lavender Brown.	He can't avoid us forever." "Who's avoiding you?" said Ron, sitting down next to them. "Wish you would," said Fred, looking irritated at the interruption. "What's a bummer?" Ron asked George. "Having a nosy git like you for a brother," said George. "You two got any ideas on the Triwizard Tournament yet?" Harry asked. "Thought any more about trying to enter?" "I asked McGonagall how the champions are chosen but she wasn't telling," said George bitterly. "She just told me to shut up and get on with transfiguring my raccoon." "Wonder what the tasks are going to be?" said Ron thoughtfully. "You know, I bet we could do them, Harry. We've done dangerous stuff before. ..." "Not in front of a panel of judges, you haven't," said Fred. "McGonagall says the champions get awarded points according to how well they've done the tasks." "Who are the judges?"	Harry crept past them to the portrait hole and waited for a minute or so, keeping an eye on his watch. Then Hermione opened the Fat Lady for him from outside as they had planned. He slipped past her with a whispered "Thanks!" and set off through the castle. The grounds were very dark. Harry walked down the lawn toward the lights shining in Hagrid's cabin. The inside of the enormous Beauxbatons carriage was also lit up; Harry could hear Madame Maxime talking inside it as he knocked on Hagrid's front door. "You there, Harry?"

• Loss: main.MultipleNegativesRankingLoss_self with these parameters:

  {
      "scale": 5,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 5
• fp16: True
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
0.1064	5	0.1626
0.2128	10	0.1799
0.3191	15	0.1720
0.4255	20	0.1769
0.5319	25	0.1793
0.6383	30	0.1678
0.7447	35	0.1872
0.8511	40	0.2052
0.9574	45	0.2272
1.0	47	0.2280
1.0638	50	0.2348
1.1702	55	0.2357
1.2766	60	0.2407
1.3830	65	0.2477
1.4894	70	0.2481
1.5957	75	0.2497
1.7021	80	0.2467
1.8085	85	0.2505
1.9149	90	0.2612
2.0	94	0.2600
2.0213	95	0.2607
2.1277	100	0.2558
2.2340	105	0.2561
2.3404	110	0.2566
2.4468	115	0.2588
2.5532	120	0.2556
2.6596	125	0.2527
2.7660	130	0.2527
2.8723	135	0.2490
2.9787	140	0.2499
3.0	141	0.2454
3.0851	145	0.2514
3.1915	150	0.2434
3.2979	155	0.2436
3.4043	160	0.2412
3.5106	165	0.2482
3.6170	170	0.2478
3.7234	175	0.2417
3.8298	180	0.2406
3.9362	185	0.2416
4.0	188	0.2420
4.0426	190	0.2420
4.1489	195	0.2430
4.2553	200	0.2454
4.3617	205	0.2431
4.4681	210	0.2442
4.5745	215	0.2468
4.6809	220	0.2403
4.7872	225	0.2406
4.8936	230	0.2398
5.0	235	0.2398
-1	-1	0.2398

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss_self

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}