CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_2025-04-07_21-26-50

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_2025-04-07_21-26-50")
# Run inference
sentences = [
    'Who replaces Hagrid as Care of Magical Creatures teacher?',
    '"... I don\' reckon it\'d be safe fer anyone but me ter go near the colony at the mo\'," Hagrid finished, blowing his nose hard on his apron and looking up. "But thanks fer offerin\', Hermione. ... It means a lot. ..."\nAfter that, the atmosphere lightened considerably, for although neither Harry nor Ron had shown any inclination to go and feed giant grubs to a murderous, gargantuan spider, Hagrid seemed to take it for granted that they would have liked to have done and became his usual self once more. "Ar, I always knew yeh\'d find it hard ter squeeze me inter yer timetables," he said gruffly, pouring them more tea. "Even if yeh applied fer Time-Turners -"\n"We couldn\'t have done," said Hermione. "We smashed the entire stock of Ministry Time-Turners when we were there last summer.',
    'This pair I\'m talking about went and splinched themselves." Everyone around the table except Harry winced. "Er - splinched?" said Harry. "They left half of themselves behind," said Mr. Weasley, now spooning large amounts of treacle onto his porridge. "So, of course, they were stuck. Couldn\'t move either way. Had to wait for the Accidental Magic Reversal Squad to sort them out. Meant a fair old bit of paperwork, I can tell you, what with the Muggles who spotted the body parts they\'d left behind. ..."\nHarry had a sudden vision of a pair of legs and an eyeball lying abandoned on the pavement of Privet Drive.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2346
cosine_accuracy@3	0.358
cosine_accuracy@5	0.3704
cosine_accuracy@10	0.4074
cosine_precision@1	0.2346
cosine_precision@3	0.1235
cosine_precision@5	0.0765
cosine_precision@10	0.0457
cosine_recall@1	0.1893
cosine_recall@3	0.2922
cosine_recall@5	0.2984
cosine_recall@10	0.3395
cosine_ndcg@10	0.281
cosine_mrr@10	0.2948
cosine_map@100	0.2538

Training Details

Training Dataset

Unnamed Dataset

• Size: 464 training samples
• Columns: sentence_0, sentence_1, sentence_2, sentence_3, sentence_4, sentence_5, and sentence_6
• Approximate statistics based on the first 464 samples:

  	sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  type	string	string	string	string	string	string	string
  details	min: 9 tokens mean: 19.25 tokens max: 51 tokens	min: 37 tokens mean: 199.44 tokens max: 512 tokens	min: 29 tokens mean: 166.64 tokens max: 512 tokens	min: 25 tokens mean: 167.34 tokens max: 512 tokens	min: 40 tokens mean: 178.74 tokens max: 512 tokens	min: 31 tokens mean: 174.04 tokens max: 512 tokens	min: 26 tokens mean: 178.89 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  What creature does Hagrid introduce in the third book, and how does it play a role in the fifth book?	Everyone drew back slightly as Hagrid reached them and tethered the creatures to the fence. "Hippogriffs!" Hagrid roared happily, waving a hand at them. "Beau'iful, aren' they?" Harry could sort of see what Hagrid meant. Once you got over the first shock of seeing something that was half horse, half bird, you started to appreciate the hippogriffs' gleaming coats, changing smoothly from feather to hair, each of them a different color: stormy gray, bronze, pinkish roan, gleaming chestnut, and inky black. "So," said Hagrid, rubbing his hands together and beaming around, "if yeh wan' ter come a bit nearer -" No one seemed to want to. Harry, Ron, and Hermione, however, approached the fence cautiously. "Now, firs' thing yeh gotta know abou' hippogriffs is, they're proud," said Hagrid. "Easily offended, hippogriffs are. Don't never insult one, 'cause it might be the last thing yeh do." Malfoy, Crabbe, and Goyle weren't listening; they were talking in an undertone and Harry had a nasty feeling...	Ron asked Hermione, but it was Ginny who answered. "Michael Corner," she said. "Michael - but -" said Ron, craning around in his seat to stare at her. "But you were going out with him!" "Not anymore," said Ginny resolutely. "He didn't like Gryffindor beating Ravenclaw at Quidditch and got really sulky, so I ditched him and he ran off to comfort Cho instead." She scratched her nose absently with the end of her quill, turned The Quibbler upside down, and began marking her answers. Ron looked highly delighted. "Well, I always thought he was a bit of an idiot," he said, prodding his queen forward toward Harry's quivering castle. "Good for you.	It suddenly occurred to Harry how odd this would look if a Muggle were to walk up here now ... nine people, two of them grown men, clutching this manky old boot in the semidarkness, waiting. ... "Three ..." muttered Mr. Weasley, one eye still on his watch, "two ... one ..." It happened immediately: Harry felt as though a hook just behind his navel had been suddenly jerked irresistibly forward. His feet left the ground; he could feel Ron and Hermione on either side of him, their shoulders banging into his; they were all speeding forward in a howl of wind and swirling color; his forefinger was stuck to the boot as though it was pulling him magnetically onward and then - His feet slammed into the ground; Ron staggered into him and he fell over; the Portkey hit the ground near his head with a heavy thud. Harry looked up. Mr. Weasley, Mr. Diggory, and Cedric were still standing, though looking very windswept; everybody else was on the ground. "Seven past five from Stoatshead Hill," said a v...	"I was saying, what are those horse things?" Harry said, as he, Ron, and Luna made for the carriage in which Hermione and Ginny were already sitting. "What horse things?" "The horse things pulling the carriages!" said Harry impatiently; they were, after all, about three feet from the nearest one; it was watching them with empty white eyes. Ron, however, gave Harry a perplexed look. "What are you talking about?" "I'm talking about - look!" Harry grabbed Ron's arm and wheeled him about so that he was face-to-face with the winged horse.	Here, take a napkin. I've packed my own lunch; the trolley, as I remember it, is heavy on licorice wands, and a poor old man's digestive system isn't quite up to such things. ... Pheasant, Belby?" Belby started and accepted what looked like half a cold pheasant. "I was just telling young Marcus here that I had the pleasure of teaching his Uncle Damocles," Slughorn told Harry and Neville, now passing around a basket of rolls. "Outstanding wizard, outstanding, and his Order of Merlin most well-deserved. Do you see much of your uncle, Marcus?" Unfortunately, Belby had just taken a large mouthful of pheasant; in his haste to answer Slughorn he swallowed too fast, turned purple, and began to choke. "Anapneo," said Slughorn calmly, pointing his wand at Belby, whose airway seemed to clear at once. "Not ... not much of him, no," gasped Belby, his eyes streaming.	"Hagrid!" Hermione shouted, pounding on his front door. "Hagrid, that's enough! We know you're in there! Nobody cares if your mum was a giantess, Hagrid! You can't let that foul Skeeter woman do this to you! Hagrid, get out here, you're just being -" The door opened. Hermione said, "About t - !" and then stopped, very suddenly, because she had found herself face-to-face, not with Hagrid, but with Albus Dumbledore. "Good afternoon," he said pleasantly, smiling down at them. "We - er - we wanted to see Hagrid," said Hermione in a rather small voice. "Yes, I surmised as much," said Dumbledore, his eyes twinkling. "Why don't you come in?" "Oh ... um ... okay," said Hermione. She, Ron, and Harry went into the cabin; Fang launched himself upon Harry the moment he entered, barking madly and trying to lick his ears. Harry fended off Fang and looked around. Hagrid was sitting at his table, where there were two large mugs of tea. He looked a real mess. His face was blotchy, his eyes swollen, and...
  What spell does Harry use to save his cousin Dudley from danger in the fifth book, and how does this connect to an event in the second book?	..." He was never going to see Ron and Hermione again - And their faces burst clearly into his mind as he fought for breath - "EXPECTO PATRONUM!" An enormous silver stag erupted from the tip of Harry's wand; its antlers caught the dementor in the place where the heart should have been; it was thrown backward, weightless as darkness, and as the stag charged, the dementor swooped away, batlike and defeated. "THIS WAY!" Harry shouted at the stag. Wheeling around, he sprinted down the alleyway, holding the lit wand aloft. "DUDLEY? DUDLEY!" He had run barely a dozen steps when he reached them: Dudley was curled on the ground, his arms clamped over his face; a second dementor was crouching low over him, gripping his wrists in its slimy hands, prizing them slowly, almost lovingly apart, lowering its hooded head toward Dudley's face as though about to kiss him. ... "GET IT!"	Snape said something about Crouch -" "Crouch?" said Hagrid blankly. "Karkaroff, please, Hagrid!" said Dumbledore sharply. "Oh yeah ... right y'are, Professor ..." said Hagrid, and he turned and disappeared into the dark trees, Fang trotting after him.	But Harry had suddenly remembered. He should have told Dumbledore, he should have said it straightaway - "There's a Death Eater at Hogwarts! There's a Death Eater here - they put my name in the Goblet of Fire, they made sure I got through to the end -" Harry tried to get up, but Moody pushed him back down. "I know who the Death Eater is," he said quietly. "Karkaroff?" said Harry wildly. "Where is he? Have you got him? Is he locked up?" "Karkaroff?"	"Looking for Crouch, according to Bagman," said Harry. "He's still ill. Hasn't been into work." "Maybe Percy's poisoning him," said Ron. "Probably thinks if Crouch snuffs it he'll be made head of the Department of International Magical Cooperation." Hermione gave Ron a don't-joke-about-things-like-that look, and said, "Funny, goblins looking for Mr. Crouch. ... They'd normally deal with the Department for the Regulation and Control of Magical Creatures." "Crouch can speak loads of different languages, though," said Harry. "Maybe they need an interpreter."	Harry repeated, tearing his gaze from the water to look at Dumbledore. "Not while they are merely drifting peacefully below us," said Dumbledore. "There is nothing to be feared from a body, Harry, any more than there is anything to be feared from the darkness. Lord Voldemort, who of course secretly fears both, disagrees. But once again he reveals his own lack of wisdom. It is the unknown we fear when we look upon death and darkness, nothing more." Harry said nothing; he did not want to argue, but he found the idea that there were bodies floating around them and beneath them horrible and, what was more, he did not believe that they were not dangerous. "But one of them jumped," he said, trying to make his voice as level and calm as Dumbledore's.	"I've got loads. But don't touch my hand, now. I've just washed it, you see; don't want a Mudblood sliming it up." Some of the anger Harry had been feeling for days and days seemed to burst through a dam in his chest. He had reached for his wand before he'd thought what he was doing.
  What is the name of the spell that causes a person to be bound?	She let out an earsplitting scream and threw her hands over her head while some of the centaurs bellowed their approval and others laughed raucously. The sound of their wild, neighing laughter echoing around the dimly lit clearing and the sight of their pawing hooves was extremely unnerving. "Whose forest is it now, human?" bellowed Bane. "Filthy half-breeds!" she screamed, her hands still tight over her head. "Beasts! Uncontrolled animals!" "Be quiet!" shouted Hermione, but it was too late - Umbridge pointed her wand at Magorian and screamed, "Incarcerous!"	"Sir - Professor Dumbledore? Can I ask you something?" "Obviously, you've just done so," Dumbledore smiled. "You may ask me one more thing, however." "What do you see when you look in the mirror?" "I? I see myself holding a pair of thick, woolen socks." Harry stared. "One can never have enough socks," said Dumbledore. "Another Christmas has come and gone and I didn't get a single pair. People will insist on giving me books." It was only when he was back in bed that it struck Harry that Dumbledore might not have been quite truthful. But then, he thought, as he shoved Scabbers off his pillow, it had been quite a personal question.	"Settle down," said Snape coldly, shutting the door behind him. There was no real need for the call to order; the moment the class had heard the door close, quiet had fallen and all fidgeting stopped. Snape's mere presence was usually enough to ensure a class's silence. "Before we begin today's lesson," said Snape, sweeping over to his desk and staring around at them all, "I think it appropriate to remind you that next June you will be sitting an important examination, during which you will prove how much you have learned about the composition and use of magical potions. Moronic though some of this class undoubtedly are, I expect you to scrape an 'Acceptable' in your O.W.L., or suffer my ... displeasure." His gaze lingered this time upon Neville, who gulped. "After this year, of course, many of you will cease studying with me," Snape went on. "I take only the very best into my N.E.W.T.	Harry heard the sarcasm in his voice, but he was not sure that anyone else did. Opposite Harry, Tonks was entertaining Hermione and Ginny by transforming her nose between mouthfuls. Screwing up her eyes each time with the same pained expression she had worn back in Harry's bedroom, her nose swelled to a beaklike protuberance like Snape's, shrank to something resembling a button mushroom, and then sprouted a great deal of hair from each nostril. Apparently this was a regular mealtime entertainment, because after a while Hermione and Ginny started requesting their favorite noses. "Do that one like a pig snout, Tonks ..." Tonks obliged, and Harry, looking up, had the fleeting impression that a female Dudley was grinning at him from across the table. Mr. Weasley, Bill, and Lupin were having an intense discussion about goblins. "They're not giving anything away yet," said Bill. "I still can't work out whether they believe he's back or not. 'Course, they might prefer not to take sides at all...	... Since then, I have served him faithfully, although I have let him down many times. He has had to be very hard on me." Quirrell shivered suddenly. "He does not forgive mistakes easily. When I failed to steal the Stone from Gringotts, he was most displeased.	Despite his determination to catch Malfoy out, Harry had no luck at all over the next couple of weeks. Although he consulted the map as often as he could, sometimes making unnecessary visits to the bathroom between lessons to search it, he did not once see Malfoy anywhere suspicious. Admittedly, he spotted Crabbe and Goyle moving around the castle on their own more often than usual, sometimes remaining stationary in deserted corridors, but at these times Malfoy was not only nowhere near them, but impossible to locate on the map at all. This was most mysterious. Harry toyed with the possibility that Malfoy was actually leaving the school grounds, but could not see how he could be doing it, given the very high level of security now operating within the castle. He could only suppose that he was missing Malfoy amongst the hundreds of tiny black dots upon the map. As for the fact that Malfoy, Crabbe, and Goyle appeared to be going their different ways when they were usually inseparable, the...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 10,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• fp16: True
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
0.1724	5	0.1746
0.3448	10	0.1983
0.5172	15	0.2130
0.6897	20	0.2324
0.8621	25	0.2290
1.0	29	0.2324
1.0345	30	0.2332
1.2069	35	0.2464
1.3793	40	0.2474
1.5517	45	0.2565
1.7241	50	0.2580
1.8966	55	0.2616
2.0	58	0.2642
2.0690	60	0.2651
2.2414	65	0.2716
2.4138	70	0.2769
2.5862	75	0.2734
2.7586	80	0.2739
2.9310	85	0.2757
3.0	87	0.2753
3.1034	90	0.2758
3.2759	95	0.2792
3.4483	100	0.2804
3.6207	105	0.2806
3.7931	110	0.2863
3.9655	115	0.2853
4.0	116	0.2842
4.1379	120	0.2843
4.3103	125	0.2832
4.4828	130	0.2787
4.6552	135	0.2810
4.8276	140	0.2810
5.0	145	0.2810
-1	-1	0.2810

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}