ayushexel/emb-bge-base-en-v1.5-squad-1-epochs

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ayushexel/emb-bge-base-en-v1.5-squad-1-epochs")
# Run inference
sentences = [
    'What does it mean for the LGBT people who are in a double monority?',
    'In the United States, non-Caucasian LGBT individuals may find themselves in a double minority, where they are neither fully accepted or understood by mainly Caucasian LGBT communities, nor are they accepted by their own ethnic group. Many people experience racism in the dominant LGBT community where racial stereotypes merge with gender stereotypes, such that Asian-American LGBTs are viewed as more passive and feminine, while African-American LGBTs are viewed as more masculine and aggressive. There are a number of culturally specific support networks for LGBT individuals active in the United States. For example, "Ô-Môi" for Vietnamese American queer females.',
    'In the United States, non-Caucasian LGBT individuals may find themselves in a double minority, where they are neither fully accepted or understood by mainly Caucasian LGBT communities, nor are they accepted by their own ethnic group. Many people experience racism in the dominant LGBT community where racial stereotypes merge with gender stereotypes, such that Asian-American LGBTs are viewed as more passive and feminine, while African-American LGBTs are viewed as more masculine and aggressive. There are a number of culturally specific support networks for LGBT individuals active in the United States. For example, "Ô-Môi" for Vietnamese American queer females.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Dataset: gooqa-dev
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.4096

Training Details

Training Dataset

Unnamed Dataset

• Size: 44,286 training samples
• Columns: question, context, and negative
• Approximate statistics based on the first 1000 samples:

  	question	context	negative
  type	string	string	string
  details	min: 6 tokens mean: 14.5 tokens max: 33 tokens	min: 29 tokens mean: 150.96 tokens max: 469 tokens	min: 29 tokens mean: 154.7 tokens max: 512 tokens

• Samples:

  question	context	negative
  Suga Mama is also a song on which Beyoncé album?	In 2006, Beyoncé introduced her all-female tour band Suga Mama (also the name of a song in B'Day) which includes bassists, drummers, guitarists, horn players, keyboardists and percussionists. Her background singers, The Mamas, consist of Montina Cooper-Donnell, Crystal Collins and Tiffany Moniqué Riddick. They made their debut appearance at the 2006 BET Awards and re-appeared in the music videos for "Irreplaceable" and "Green Light". The band have supported Beyoncé in most subsequent live performances, including her 2007 concert tour The Beyoncé Experience, 2009–2010 I Am... World Tour and 2013–2014 The Mrs. Carter Show World Tour.	On December 13, 2013, Beyoncé unexpectedly released her eponymous fifth studio album on the iTunes Store without any prior announcement or promotion. The album debuted atop the Billboard 200 chart, giving Beyoncé her fifth consecutive number-one album in the US. This made her the first woman in the chart's history to have her first five studio albums debut at number one. Beyoncé received critical acclaim and commercial success, selling one million digital copies worldwide in six days; The New York Times noted the album's unconventional, unexpected release as significant. Musically an electro-R&B album, it concerns darker themes previously unexplored in her work, such as "bulimia, postnatal depression [and] the fears and insecurities of marriage and motherhood". The single "Drunk in Love", featuring Jay Z, peaked at number two on the Billboard Hot 100 chart. In April 2014, after much speculation in the weeks before, Beyoncé and Jay Z officially announced their On the Run Tour. It served...
  How were residents listed in the census in 1890?	For 1890, the Census Office changed the design of the population questionnaire. Residents were still listed individually, but a new questionnaire sheet was used for each family. Additionally, this was the first year that the census distinguished between different East Asian races, such as Japanese and Chinese, due to increased immigration. This census also marked the beginning of the term "race" in the questionnaires. Enumerators were instructed to write "White," "Black," "Mulatto," "Quadroon," "Octoroon," "Chinese," "Japanese," or "Indian."	The 2011 census recorded that 2,998,264 people or 36.7% of London's population are foreign-born making London the city with the second largest immigrant population, behind New York City, in terms of absolute numbers. The table to the right shows the most common countries of birth of London residents. Note that some of the German-born population, in 18th position, are British citizens from birth born to parents serving in the British Armed Forces in Germany. With increasing industrialisation, London's population grew rapidly throughout the 19th and early 20th centuries, and it was for some time in the late 19th and early 20th centuries the most populous city in the world. Its population peaked at 8,615,245 in 1939 immediately before the outbreak of the Second World War, but had declined to 7,192,091 at the 2001 Census. However, the population then grew by just over a million between the 2001 and 2011 Censuses, to reach 8,173,941 in the latter enumeration.
  Parliamentary statutes are put into effect by who?	Legislative powers are exercised by a 300-member elective unicameral Parliament. Statutes passed by the Parliament are promulgated by the President of the Republic. Parliamentary elections are held every four years, but the President of the Republic is obliged to dissolve the Parliament earlier on the proposal of the Cabinet, in view of dealing with a national issue of exceptional importance. The President is also obliged to dissolve the Parliament earlier, if the opposition manages to pass a motion of no confidence.	London is the seat of the Government of the United Kingdom. Many government departments are based close to the Palace of Westminster, particularly along Whitehall, including the Prime Minister's residence at 10 Downing Street. The British Parliament is often referred to as the "Mother of Parliaments" (although this sobriquet was first applied to England itself by John Bright) because it has been the model for most other parliamentary systems. There are 73 Members of Parliament (MPs) from London, who correspond to local parliamentary constituencies in the national Parliament. As of May 2015, 45 are from the Labour Party, 27 are Conservatives, and one is a Liberal Democrat.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 5,000 evaluation samples
• Columns: question, context, and negative_1
• Approximate statistics based on the first 1000 samples:

  	question	context	negative_1
  type	string	string	string
  details	min: 7 tokens mean: 14.4 tokens max: 36 tokens	min: 28 tokens mean: 154.21 tokens max: 512 tokens	min: 28 tokens mean: 151.03 tokens max: 512 tokens

• Samples:

  question	context	negative_1
  Are groups who represent workers allowed to express their opinions in protest in Burma?	Since the 2010 election, the government has embarked on a series of reforms to direct the country towards liberal democracy, a mixed economy, and reconciliation, although doubts persist about the motives that underpin such reforms. The series of reforms includes the release of pro-democracy leader Aung San Suu Kyi from house arrest, the establishment of the National Human Rights Commission, the granting of general amnesties for more than 200 political prisoners, new labour laws that permit labour unions and strikes, a relaxation of press censorship, and the regulation of currency practices.	In response to the demonstrations, an editorial in the People's Daily urged Chinese people to "express [their] patriotic enthusiasm calmly and rationally, and express patriotic aspiration in an orderly and legal manner".
  A heavy penalty for violating FAA reulations could said to be do what?	The FAA has been cited as an example of regulatory capture, "in which the airline industry openly dictates to its regulators its governing rules, arranging for not only beneficial regulation, but placing key people to head these regulators." Retired NASA Office of Inspector General Senior Special Agent Joseph Gutheinz, who used to be a Special Agent with the Office of Inspector General for the Department of Transportation and with FAA Security, is one of the most outspoken critics of FAA. Rather than commend the agency for proposing a $10.2 million fine against Southwest Airlines for its failure to conduct mandatory inspections in 2008, he was quoted as saying the following in an Associated Press story: "Penalties against airlines that violate FAA directives should be stiffer. At $25,000 per violation, Gutheinz said, airlines can justify rolling the dice and taking the chance on getting caught. He also said the FAA is often too quick to bend to pressure from airlines and pilots." Other...	The FAA has been cited as an example of regulatory capture, "in which the airline industry openly dictates to its regulators its governing rules, arranging for not only beneficial regulation, but placing key people to head these regulators." Retired NASA Office of Inspector General Senior Special Agent Joseph Gutheinz, who used to be a Special Agent with the Office of Inspector General for the Department of Transportation and with FAA Security, is one of the most outspoken critics of FAA. Rather than commend the agency for proposing a $10.2 million fine against Southwest Airlines for its failure to conduct mandatory inspections in 2008, he was quoted as saying the following in an Associated Press story: "Penalties against airlines that violate FAA directives should be stiffer. At $25,000 per violation, Gutheinz said, airlines can justify rolling the dice and taking the chance on getting caught. He also said the FAA is often too quick to bend to pressure from airlines and pilots." Other...
  Which three actors reprised supporting roles for Spectre?	The main cast was revealed in December 2014 at the 007 Stage at Pinewood Studios. Daniel Craig returned for his fourth appearance as James Bond, while Ralph Fiennes, Naomie Harris and Ben Whishaw reprised their roles as M, Eve Moneypenny and Q respectively, having been established in Skyfall. Rory Kinnear also reprised his role as Bill Tanner in his third appearance in the series.	In November 2013 MGM and the McClory estate formally settled the issue with Danjaq, LLC—sister company of Eon Productions—with MGM acquiring the full copyright film rights to the concept of Spectre and all of the characters associated with it. With the acquisition of the film rights and the organisation's re-introduction to the series' continuity, the SPECTRE acronym was discarded and the organisation reimagined as "Spectre".

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	gooqa-dev_cosine_accuracy
-1	-1	-	-	0.3520
0.2890	100	0.5832	0.7851	0.3854
0.5780	200	0.4321	0.7335	0.4092
0.8671	300	0.4017	0.7273	0.4124
-1	-1	-	-	0.4096

Framework Versions

• Python: 3.11.0
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}