CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_v1

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("CatkinChen/BAAI_bge-base-en-v1.5_retrieval_finetuned_2025-04-03_23-18-57")
# Run inference
sentences = [
    'Who is the Slytherin boy who loves fame?',
    'There was also a list of the new books he\'d need for the coming year. SECOND-YEAR STUDENTS WILL REQUIRE:\nThe Standard Book of Spells, Grade 2\nby Miranda Goshawk\nBreak with a Banshee by Gilderoy Lockhart\nGadding with Ghouls by Gilderoy Lockhart\nHolidays with Hags by Gilderoy Lockhart\nTravels with Trolls by Gilderoy Lockhart\nVoyages with Vampires by Gilderoy Lockhart\nWanderings with Werewolves by Gilderoy Lockhart\nYear with the Yeti by Gilderoy Lockhart\nFred, who had finished his own list, peered over at Harry\'s. "You\'ve been told to get all Lockhart\'s books, too!" he said. "The new Defense Against the Dark Arts teacher must be a fan - bet it\'s a witch." At this point, Fred caught his mother\'s eye and quickly busied himself with the marmalade.',
    'Nobody could quite believe their ears until she looked directly at Harry and Ron and said grimly, "I\'ve become accustomed to seeing the Quidditch Cup in my study, boys, and I really don\'t want to have to hand it over to Professor Snape, so use the extra time to practice, won\'t you?" Snape was no less obviously partisan: He had booked the Quidditch pitch for Slytherin practice so often that the Gryffindors had difficulty getting on it to play. He was also turning a deaf ear to the many reports of Slytherin attempts to hex Gryffindor players in the corridors. When Alicia Spinnet turned up in the hospital wing with her eyebrows growing so thick and fast that they obscured her vision and obstructed her mouth, Snape insisted that she must have attempted a Hair-Thickening Charm on herself and refused to listen to the fourteen eyewitnesses who insisted that they had seen the Slytherin Keeper, Miles Bletchley, hit her from behind with a jinx while she worked in the library. Harry felt optimistic about Gryffindor\'s chances; they had, after all, never lost to Malfoy\'s team. Admittedly Ron was still not performing to Wood\'s standard, but he was working extremely hard to improve. His greatest weakness was a tendency to lose confidence when he made a blunder; if he let in one goal he became flustered and was therefore likely to miss more. On the other hand, Harry had seen Ron make some truly spectacular saves when he was on form: During one memorable practice, he had hung one-handed from his broom and kicked the Quaffle so hard away from the goal hoop that it soared the length of the pitch and through the center hoop at the other end. The rest of the team felt this save compared favorably with one made recently by Barry Ryan, the Irish International Keeper, against Poland\'s top Chaser, Ladislaw Zamojski.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2222
cosine_accuracy@3	0.358
cosine_accuracy@5	0.3704
cosine_accuracy@10	0.4074
cosine_precision@1	0.2222
cosine_precision@3	0.1235
cosine_precision@5	0.079
cosine_precision@10	0.0444
cosine_recall@1	0.1708
cosine_recall@3	0.2798
cosine_recall@5	0.2922
cosine_recall@10	0.3354
cosine_ndcg@10	0.2692
cosine_mrr@10	0.2875
cosine_map@100	0.2392

Training Details

Training Dataset

Unnamed Dataset

• Size: 464 training samples
• Columns: sentence_0, sentence_1, sentence_2, sentence_3, sentence_4, sentence_5, and sentence_6
• Approximate statistics based on the first 464 samples:

  	sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  type	string	string	string	string	string	string	string
  details	min: 9 tokens mean: 19.47 tokens max: 51 tokens	min: 37 tokens mean: 199.31 tokens max: 512 tokens	min: 43 tokens mean: 170.61 tokens max: 512 tokens	min: 39 tokens mean: 176.24 tokens max: 512 tokens	min: 28 tokens mean: 170.23 tokens max: 512 tokens	min: 45 tokens mean: 167.81 tokens max: 512 tokens	min: 29 tokens mean: 165.47 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2	sentence_3	sentence_4	sentence_5	sentence_6
  What spell does Harry use to save his cousin Dudley from danger in the fifth book, and how does this connect to an event in the second book?	It did not contain birthday greetings. Dear Mr. Potter, We have received intelligence that a Hover Charm was used at your place of residence this evening at twelve minutes past nine. As you know, underage wizards are not permitted to perform spells outside school, and further spellwork on your part may lead to expulsion from said school (Decree for the Reasonable Restriction of Underage Sorcery, 1875, Paragraph C). We would also ask you to remember that any magical activity that risks notice by members of the non-magical community (Muggles) is a serious offense under section 13 of the International Confederation of Warlocks' Statute of Secrecy. Enjoy your holidays! Yours sincerely, Mafalda Hopkirk IMPROPER USE OF MAGIC OFFICE Ministry of Magic Harry looked up from the letter and gulped. "You didn't tell us you weren't allowed to use magic outside school," said Uncle Vernon, a mad gleam dancing in his eyes. "Forgot to mention it. ... Slipped your mind, I daresay.	"They sealed off all of those before the start of the year," said Neville. "There's no chance of getting through any of them now, not with curses over the entrances and Death Eaters and dementors waiting at the exits." He started walking backward, beaming, drinking them in. "Never mind that stuff. ... Is it true? Did you break into Gringotts? Did you escape on a dragon?	A large plate of sandwiches, two silver goblets, and a jug of iced pumpkin juice appeared with a pop. "You will eat in here and then go straight up to your dormitory," she said. "I must also return to the feast." When the door had closed behind her, Ron let out a long, low whistle. "I thought we'd had it," he said, grabbing a sandwich. "So did I," said Harry, taking one, too. "Can you believe our luck, though?" said Ron thickly through a mouthful of chicken and ham. "Fred and George must've flown that car five or six times and no Muggle ever saw them." He swallowed and took another huge bite. "Why couldn't we get through the barrier?" Harry shrugged. "We'll have to watch our step from now on, though," he said, taking a grateful swig of pumpkin juice. "Wish we could've gone up to the feast. ..." "She didn't want us showing off," said Ron sagely.	"Not by magic. Have you got a spade?" And shortly afterward he had set to work, alone, digging the grave in the place that Bill had shown him at the end of the garden, between bushes. He dug with a kind of fury, relishing the manual work, glorying in the non-magic of it, for every drop of his sweat and every blister felt like a gift to the elf who had saved their lives. His scar burned, but he was master of the pain; he felt it, yet was apart from it.	Apparently the Peverells were one of the earliest families to vanish." " 'Extinct in the male line'?" repeated Ron. "It means the name's died out," said Hermione, "centuries ago, in the case of the Peverells. They could still have descendants, though, they'd just be called something different." And then it came to Harry in one shining piece, the memory that had stirred at the sound of the name "Peverell": a filthy old man brandishing an ugly ring in the face of a Ministry official, and he cried aloud, "Marvolo Gaunt!" "Sorry?" said Ron and Hermione together. "Marvolo Gaunt! You-Know-Who's grandfather!	... Well, Filch ran over all of us with Secrecy Sensors when we got into the entrance hall. Any Dark object would have been found, I know for a fact Crabbe had a shrunken head confiscated. So you see, Malfoy can't have brought in anything dangerous!" Momentarily stymied, Harry watched Ginny Weasley playing with Arnold the Pygmy Puff for a while before seeing a way around this objection. "Someone's sent it to him by owl, then," he said. "His mother or someone." "All the owls are being checked too," said Hermione. "Filch told us so when he was jabbing those Secrecy Sensors everywhere he could reach." Really stumped this time, Harry found nothing else to say. There did not seem to be any way Malfoy could have brought a dangerous or Dark object into the school. He looked hopefully at Ron, who was sitting with his arms folded, staring over at Lavender Brown. "Can you think of any way Malfoy - ?"
  Who becomes Headmistress of Hogwarts immediately following the final Battle of Hogwarts?	Tom Riddle hit the floor with a mundane finality, his body feeble and shrunken, the white hands empty, the snakelike face vacant and unknowing. Voldemort was dead, killed by his own rebounding curse, and Harry stood with two wands in his hand, staring down at his enemy's shell. One shivering second of silence, the shock of the moment suspended: and then the tumult broke around Harry as the screams and the cheers and the roars of the watchers rent the air. The fierce new sun dazzled the windows as they thundered toward him, and the first to reach him were Ron and Hermione, and it was their arms that were wrapped around him, their incomprehensible shouts that deafened him. Then Ginny, Neville, and Luna were there, and then all the Weasleys and Hagrid, and Kingsley and McGonagall and Flitwick and Sprout, and Harry could not hear a word that anyone was shouting, nor tell whose hands were seizing him, pulling him, trying to hug some part of him, hundreds of them pressing in, all of them det...	Dumbledore's future career seemed likely to be meteoric, and the only question that remained was when he would become Minister of Magic. Though it was often predicted in later years that he was on the point of taking the job, however, he never had Ministerial ambitions. Three years after we had started at Hogwarts, Albus's brother, Aberforth, arrived at school. They were not alike; Aberforth was never bookish and, unlike Albus, preferred to settle arguments by dueling rather than through reasoned discussion. However, it is quite wrong to suggest, as some have, that the brothers were not friends. They rubbed along as comfortably as two such different boys could do. In fairness to Aberforth, it must be admitted that living in Albus's shadow cannot have been an altogether comfortable experience. Being continually outshone was an occupational hazard of being his friend and cannot have been any more pleasurable as a brother.	"Soon?" "Almost at once," said Lupin, "we're just waiting for the all-clear." "Where are we going? The Burrow?" Harry asked hopefully. "Not the Burrow, no," said Lupin, motioning Harry toward the kitchen; the little knot of wizards followed, all still eyeing Harry curiously. "Too risky. We've set up headquarters somewhere undetectable. It's taken a while. ..." Mad-Eye Moody was now sitting at the kitchen table swigging from a hip flask, his magical eye spinning in all directions, taking in the Dursleys' many labor-saving appliances.	said Hermione. Crookshanks slowly chewed up the spider, his yellow eyes fixed insolently on Ron. "Just keep him over there, that's all," said Ron irritably, turning back to his star chart. "I've got Scabbers asleep in my bag." Harry yawned. He really wanted to go to bed, but he still had his own star chart to complete. He pulled his bag toward him, took out parchment, ink, and quill, and started work. "You can copy mine, if you like," said Ron, labeling his last star with a flourish and shoving the chart toward Harry.	Harry said, looking around at the door to the girls' dormitories. "Let's go and tell her," said Ron. He bounded forward, pulled open the door, and set off up the spiral staircase. He was on the sixth stair when it happened. There was a loud, wailing, klaxonlike sound and the steps melted together to make a long, smooth stone slide. There was a brief moment when Ron tried to keep running, arms working madly like windmills, then he toppled over backward and shot down the newly created slide, coming to rest on his back at Harry's feet. "Er - I don't think we're allowed in the girls' dormitories," said Harry, pulling Ron to his feet and trying not to laugh. Two fourth-year girls came zooming gleefully down the stone slide. "Oooh, who tried to get upstairs?" they giggled happily, leaping to their feet and ogling Harry and Ron. "Me," said Ron, who was still rather disheveled. "I didn't realize that would happen. It's not fair!" he added to Harry, as the girls headed off for the portrait hole...	..." "Cool," muttered Harry, sparing the watch a glance before peering more closely at the map. Where was Malfoy? He did not seem to be at the Slytherin table in the Great Hall, eating breakfast. ... He was nowhere near Snape, who was sitting in his study.
  In the sixth book, what memory does Harry retrieve from Slughorn, and how does it connect to the second book?	"W-what's that?" said Mr. Weasley in a stunned voice. "You-Know-Who? En-enchant Ginny? But Ginny's not ... Ginny hasn't been ... has she?" "It was this diary," said Harry quickly, picking it up and showing it to Dumbledore. "Riddle wrote it when he was sixteen. ..." Dumbledore took the diary from Harry and peered keenly down his long, crooked nose at its burnt and soggy pages. "Brilliant," he said softly. "Of course, he was probably the most brilliant student Hogwarts has ever seen." He turned around to the Weasleys, who were looking utterly bewildered. "Very few people know that Lord Voldemort was once called Tom Riddle.	..." They hurried through the portrait hole and into the crowd, still discussing Snape. "But if he - you know" - Hermione dropped her voice, glancing nervously around - "if he was trying to - to poison Lupin - he wouldn't have done it in front of Harry." "Yeah, maybe," said Harry as they reached the entrance hall and crossed into the Great Hall. It had been decorated with hundreds and hundreds of candle-filled pumpkins, a cloud of fluttering live bats, and many flaming orange streamers, which were swimming lazily across the stormy ceiling like brilliant watersnakes. The food was delicious; even Hermione and Ron, who were full to bursting with Honeydukes sweets, managed second helpings of everything. Harry kept glancing at the staff table.	said Dumbledore calmly, and Peeves's grin faded a little. He didn't dare taunt Dumbledore. Instead he adopted an oily voice that was no better than his cackle. "Ashamed, Your Headship, sir. Doesn't want to be seen. She's a horrible mess. Saw her running through the landscape up on the fourth floor, sir, dodging between the trees. Crying something dreadful," he said happily.	I knew that the Wizarding world was clamoring for the full story and I wanted to be the first to meet that need." I mention the recent, widely publicized remarks of Elphias Doge, Special Advisor to the Wizengamot and longstanding friend of Albus Dumbledore's, that "Skeeter's book contains less fact than a Chocolate Frog card." Skeeter throws back her head and laughs. "Darling Dodgy! I remember interviewing him a few years back about merpeople rights, bless him. Completely gaga, seemed to think we were sitting at the bottom of Lake Windermere, kept telling me to watch out for trout." And yet Elphias Doge's accusations of inaccuracy have been echoed in many places.	"Harry, guess what?" said Tonks from her perch on top of the washing machine, and she wiggled her left hand at him; a ring glittered there. "You got married?" Harry yelped, looking from her to Lupin. "I'm sorry you couldn't be there, Harry, it was very quiet." "That's brilliant, congrat -" "All right, all right, we'll have time for a cozy catch-up later!" roared Moody over the hubbub, and silence fell in the kitchen. Moody dropped his sacks at his feet and turned to Harry. "As Dedalus probably told you, we had to abandon Plan A. Pius Thicknesse has gone over, which gives us a big problem.	It was a very sunny Saturday and the zoo was crowded with families. The Dursleys bought Dudley and Piers large chocolate ice creams at the entrance and then, because the smiling lady in the van had asked Harry what he wanted before they could hurry him away, they bought him a cheap lemon ice pop. It wasn't bad, either, Harry thought, licking it as they watched a gorilla scratching its head who looked remarkably like Dudley, except that it wasn't blond. Harry had the best morning he'd had in a long time. He was careful to walk a little way apart from the Dursleys so that Dudley and Piers, who were starting to get bored with the animals by lunchtime, wouldn't fall back on their favorite hobby of hitting him. They ate in the zoo restaurant, and when Dudley had a tantrum because his knickerbocker glory didn't have enough ice cream on top, Uncle Vernon bought him another one and Harry was allowed to finish the first. Harry felt, afterward, that he should have known it was all too good to la...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 10,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 6
• fp16: True
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 6
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
0.1724	5	0.1644
0.3448	10	0.1969
0.5172	15	0.2073
0.6897	20	0.2215
0.8621	25	0.2352
1.0	29	0.2418
1.0345	30	0.2380
1.2069	35	0.2452
1.3793	40	0.2610
1.5517	45	0.2655
1.7241	50	0.2627
1.8966	55	0.2679
2.0	58	0.2675
2.0690	60	0.2662
2.2414	65	0.2605
2.4138	70	0.2625
2.5862	75	0.2571
2.7586	80	0.2705
2.9310	85	0.2737
3.0	87	0.2737
3.1034	90	0.2702
3.2759	95	0.2744
3.4483	100	0.2742
3.6207	105	0.2716
3.7931	110	0.2694
3.9655	115	0.2649
4.0	116	0.2649
4.1379	120	0.2705
4.3103	125	0.2705
4.4828	130	0.2656
4.6552	135	0.2688
4.8276	140	0.2677
5.0	145	0.2689
5.1724	150	0.2689
5.3448	155	0.2661
5.5172	160	0.2653
5.6897	165	0.2697
5.8621	170	0.2692
6.0	174	0.2692
-1	-1	0.2692

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}