--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:90000 - loss:CosineSimilarityLoss base_model: sentence-transformers/msmarco-distilbert-base-v4 widget: - source_sentence: roast lamb internal temperature celsius sentences: - For a moderated subform of flagellation, described as bastinado, the soles of a person's bare feet are used as a target for beating (see foot whipping). In some circumstances the word flogging is used loosely to include any sort of corporal punishment, including birching and caning. - The periodic table provides a useful framework for analyzing chemical behaviour, and is widely used in chemistry and other sciences. Dmitri Mendeleev published in 1869 the first widely recognized periodic table. He developed his table to illustrate periodic trends in the properties of the then-known elements. - Service & Support. Key to COROB's success, is the extensive direct and partner network. The highly appreciated network sets COROB apart from competition as its main target is clearly to deliver the highest standards of customer satisfaction. - source_sentence: ups danville va phone number sentences: - A polysaccharide is a type of sugar composed of a long chain of monosaccharides. Examples are cellulose, starch, glycogen, and chitin. - '$7.99. Get an INSTANT estimate of the cost to Refinish Countertops! Our free calculator uses recent, trusted data to estimate costs for your Countertop Refinishing project. For a basic 120 square feet project in zip code 47474, the benchmark cost to Refinish Countertops ranges between $3.60 - $7.99 per square foot* .To estimate costs for your project: 1.or a basic 120 square feet project in zip code 47474, the benchmark cost to Refinish Countertops ranges between $3.60 - $7.99 per square foot* .' - 'UPS Drop Box - Danville 370 Mt Cross Rd, Danville VA 24541 Phone Number: (800) 742-5877' - source_sentence: incubation period of mallard duck eggs sentences: - 'Havana Climate & Temperature. 1 Havana, Cuba is at 2258''N, 8224''W, 75 m (246 ft). 2 Havana has a tropical wet and dry/ savanna climate (Kppen-Geiger classification: Aw) with a pronounced dry season in the low-sun months , no cold season, wet season is in the high-sun months.' - compression injuries such as carpal tunnel syndrome and compression neuropathy. A nerve conduction velocity test is often done at the same time as an electromyogram (EMG). Pinched Nerve Overview A pinched nerve causes pain, numbness, or tingling in the affected area due to pressure on a nerve. - 1 57% - How long is mail from kamloops to ontario? 2 62% - How long does mail take to go from ontario to montreal? 3 74% - How long does it take for mail from kentucky to ontario? 4 64% - How long would it take to mail a package from ontario to georgia? 71% - How long does it take to get mail from vermont to ontario canada? 2 62% - How long does mail take to travel from ontario to new brunswick? 3 66% - How long does it take for mail to get from ontario to washington sate? - source_sentence: how to cook a rack of lamb well done sentences: - As the 20th anniversary of Freddie's death looms — he died on November 24, 1991 — his friends have been celebrating what would have been his 65th birthday this week with an AIDS charity dinner for the Mercury Phoenix Trust at London's Savoy Hotel, marking the start of a fund-raising year. - Medication. Medication for ulcerative colitis can suppress the inflammation of the colon and allow for tissues to heal. Symptoms including diarrhea, bleeding, and abdominal pain can also be reduced and controlled with effective medication. - Rack Opinions. To roast a lamb rack with the traditional method, you place your seasoned racks in a ripping hot oven (around 450°F), and basically hope that the center reaches the perfect medium rare (around 125 to 130°F), just as the exterior hits that deep-golden-brown-but-not-yet-black sweet spot.ack Opinions. To roast a lamb rack with the traditional method, you place your seasoned racks in a ripping hot oven (around 450°F), and basically hope that the center reaches the perfect medium rare (around 125 to 130°F), just as the exterior hits that deep-golden-brown-but-not-yet-black sweet spot. - source_sentence: when is best time of year to visit vegas sentences: - Extended Description of this Name (viewable to Premium Members only). You enjoy freedom from monotony and are stimulated by unexpected opportunities for meeting people, entertaining, or pursuing activities of a carefree nature. In your work, you find it difficult to be neat and orderly.viewable to Premium Members only). Below is a brief analysis of the first name only. For an analysis of your full name and destiny, see our full free Name and Birth Date Report service for further details. - Create anything you can imagine. Anywhere you are. The world's best imaging and design app is at the core of almost every creative project. Work across desktop and mobile devices to create and enhance your photographs, web and mobile app designs, 3D artwork, videos, and more. - Of all the Tiger subspecies, P. t. corbetti showed the highest genetic diversity in microsatellite alleles, indicating a fairly stable evolutionary history, and alleles found in other subspecies were almost always a subset of those found in P. t. corbetti. Luo et al. pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - pearson_cosine - spearman_cosine model-index: - name: SentenceTransformer based on sentence-transformers/msmarco-distilbert-base-v4 results: - task: type: semantic-similarity name: Semantic Similarity dataset: name: val eval type: val-eval metrics: - type: pearson_cosine value: 0.9685934106430019 name: Pearson Cosine - type: spearman_cosine value: 0.8659063123134304 name: Spearman Cosine --- # SentenceTransformer based on sentence-transformers/msmarco-distilbert-base-v4 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/msmarco-distilbert-base-v4](https://huggingface.co/sentence-transformers/msmarco-distilbert-base-v4). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/msmarco-distilbert-base-v4](https://huggingface.co/sentence-transformers/msmarco-distilbert-base-v4) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DistilBertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("C-KAI/sbert-academic-group44") # Run inference sentences = [ 'when is best time of year to visit vegas', 'Of all the Tiger subspecies, P. t. corbetti showed the highest genetic diversity in microsatellite alleles, indicating a fairly stable evolutionary history, and alleles found in other subspecies were almost always a subset of those found in P. t. corbetti. Luo et al.', 'Extended Description of this Name (viewable to Premium Members only). You enjoy freedom from monotony and are stimulated by unexpected opportunities for meeting people, entertaining, or pursuing activities of a carefree nature. In your work, you find it difficult to be neat and orderly.viewable to Premium Members only). Below is a brief analysis of the first name only. For an analysis of your full name and destiny, see our full free Name and Birth Date Report service for further details.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Semantic Similarity * Dataset: `val-eval` * Evaluated with [EmbeddingSimilarityEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator) | Metric | Value | |:--------------------|:-----------| | pearson_cosine | 0.9686 | | **spearman_cosine** | **0.8659** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 90,000 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | label | |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:-------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | where is sky zone located canada | In the 1950s, a dryer cost the equivalent of $1600 today and only ten percent of households used them. By 1997 seventy-eight percent of households owned a dry as the cost had fallen to an average $370. The report also includes a table of Brief Chronology of Clothes Dryer Improvements. | 0.0 | | what is syracuse new york population | East Syracuse, New York. East Syracuse is an incorporated village and a suburb of the City of Syracuse in eastern Onondaga County, New York. United States. As of the 2010 U.S. census, the village had a population of 3,084. It is located immediately east of Syracuse, in the town of DeWitt. | 1.0 | | what are the tiny blood vessels that surround alveoli | What is 'people-first language'? People-first or person-first language is a way of describing disability that involves putting the word person or people before the word disability or the name of a disability, rather than placing the disability first and using it as an adjective. | 0.0 | * Loss: [CosineSimilarityLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters: ```json { "loss_fct": "torch.nn.modules.loss.MSELoss" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `num_train_epochs`: 2 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 2 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `tp_size`: 0 - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | Training Loss | val-eval_spearman_cosine | |:------:|:----:|:-------------:|:------------------------:| | 0.0355 | 100 | - | 0.8657 | | 0.0711 | 200 | - | 0.8657 | | 0.1066 | 300 | - | 0.8658 | | 0.1422 | 400 | - | 0.8658 | | 0.1777 | 500 | 0.06 | 0.8659 | | 0.2133 | 600 | - | 0.8659 | | 0.2488 | 700 | - | 0.8659 | | 0.2844 | 800 | - | 0.8659 | ### Framework Versions - Python: 3.11.12 - Sentence Transformers: 4.0.2 - Transformers: 4.50.3 - PyTorch: 2.6.0+cu124 - Accelerate: 1.6.0 - Datasets: 3.5.0 - Tokenizers: 0.21.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```