iambestfeed/phobert-base-v2-wiki-data-filter_0_dot_7_wseg-lr2e-05-1-epochs-bs-48

SentenceTransformer based on vinai/phobert-base-v2

This is a sentence-transformers model finetuned from vinai/phobert-base-v2 on the wiki-data dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: vinai/phobert-base-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • wiki-data

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("iambestfeed/phobert-base-v2-wiki-data-filter_0_dot_7_wseg-lr2e-05-1-epochs-bs-48")
# Run inference
sentences = [
    'Hình_ảnh của Sarcoglottis sceptrodes',
    'Tập_tin : Sarcoglottis sceptrodes - Flickr 004.jpg Tập_tin : Sarcoglottis sceptrodes - Flickr 005.jpg',
    'Spinipochira excavata là một loài bọ cánh_cứng trong họ Cerambycidae .( )',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

wiki-data

• Dataset: wiki-data at 4567021
• Size: 570,310 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 5 tokens mean: 10.5 tokens max: 26 tokens	min: 11 tokens mean: 49.28 tokens max: 256 tokens

• Samples:

  anchor	positive
  Giới_thiệu của Munna parvituberculata	Munna parvituberculata là một loài chân đều trong họ Munnidae .Loài này được Kussakin miêu_tả khoa_học năm 1962 .( Schotte , M. ( 2010 ) .Munna parvituberculata Kussakin , 1962B .In :Schotte , M. , Boyko , C.B , Bruce , N.L. , Poore , G.C.B. , Taiti , S. , Wilson , G.D.F. ( Eds ) ( 2010 ) .World_Marine , Freshwater and Terrestrial_Isopod_Crustaceans database .Gebaseerd op informatie uit het Cơ_sở_dữ_liệu sinh_vật biển , te vinden op http://www.marinespecies.org/aphia.php?p=taxdetails&id=256237)
  Giới_thiệu của Goniophlebium tweedieanum	Goniophlebium tweedieanum là một loài dương_xỉ trong họ Polypodiaceae .Loài này được J.Sm. mô_tả khoa_học đầu_tiên năm 1841 .( ) Danh_pháp khoa_học của loài này chưa được làm sáng_tỏ .
  Giới_thiệu của Acanthorrhynchium grosso-papillatum	Acanthorrhynchium grosso-papillatum là một loài Rêu trong họ Sematophyllaceae .Loài này được ( Broth . ) M. Fleisch . mô_tả khoa_học đầu_tiên năm 1923 .( )

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 48
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• save_safetensors: False
• fp16: True
• push_to_hub: True
• hub_model_id: iambestfeed/phobert-base-v2-wiki-data-filter_0_dot_7_wseg-lr2e-05-1-epochs-bs-48
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 48
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: iambestfeed/phobert-base-v2-wiki-data-filter_0_dot_7_wseg-lr2e-05-1-epochs-bs-48
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0.0017	10	3.7921
0.0034	20	3.6038
0.0051	30	3.1217
0.0067	40	2.7701
0.0084	50	2.4196
0.0101	60	2.0527
0.0118	70	1.6274
0.0135	80	1.0704
0.0152	90	0.5303
0.0168	100	0.2232
0.0185	110	0.1123
0.0202	120	0.0535
0.0219	130	0.0244
0.0236	140	0.0135
0.0253	150	0.0115
0.0269	160	0.0119
0.0286	170	0.0098
0.0303	180	0.0069
0.0320	190	0.0036
0.0337	200	0.0054
0.0354	210	0.0029
0.0370	220	0.0066
0.0387	230	0.0042
0.0404	240	0.009
0.0421	250	0.0033
0.0438	260	0.0034
0.0455	270	0.0055
0.0471	280	0.0038
0.0488	290	0.0037
0.0505	300	0.0027
0.0522	310	0.0024
0.0539	320	0.0034
0.0556	330	0.0027
0.0572	340	0.0021
0.0589	350	0.0046
0.0606	360	0.002
0.0623	370	0.0014
0.0640	380	0.0012
0.0657	390	0.0024
0.0673	400	0.0019
0.0690	410	0.0034
0.0707	420	0.002
0.0724	430	0.0017
0.0741	440	0.0013
0.0758	450	0.001
0.0774	460	0.0009
0.0791	470	0.001
0.0808	480	0.0021
0.0825	490	0.0008
0.0842	500	0.0021
0.0859	510	0.001
0.0875	520	0.0028
0.0892	530	0.0008
0.0909	540	0.0008
0.0926	550	0.0007
0.0943	560	0.0013
0.0960	570	0.0009
0.0976	580	0.002
0.0993	590	0.0007
0.1010	600	0.0037
0.1027	610	0.0024
0.1044	620	0.0011
0.1061	630	0.001
0.1077	640	0.001
0.1094	650	0.001
0.1111	660	0.0005
0.1128	670	0.0005
0.1145	680	0.0033
0.1162	690	0.0014
0.1178	700	0.001
0.1195	710	0.0008
0.1212	720	0.0005
0.1229	730	0.0005
0.1246	740	0.001
0.1263	750	0.0005
0.1279	760	0.0018
0.1296	770	0.0005
0.1313	780	0.0006
0.1330	790	0.0004
0.1347	800	0.0005
0.1364	810	0.0003
0.1380	820	0.0003
0.1397	830	0.0015
0.1414	840	0.0027
0.1431	850	0.0019
0.1448	860	0.0005
0.1465	870	0.0008
0.1481	880	0.001
0.1498	890	0.0005
0.1515	900	0.0004
0.1532	910	0.0005
0.1549	920	0.0008
0.1566	930	0.0005
0.1582	940	0.0003
0.1599	950	0.0009
0.1616	960	0.0008
0.1633	970	0.0004
0.1650	980	0.0008
0.1667	990	0.001
0.1684	1000	0.0009
0.1700	1010	0.0011
0.1717	1020	0.0005
0.1734	1030	0.0017
0.1751	1040	0.0006
0.1768	1050	0.0005
0.1785	1060	0.0004
0.1801	1070	0.0006
0.1818	1080	0.0003
0.1835	1090	0.0005
0.1852	1100	0.0005
0.1869	1110	0.0004
0.1886	1120	0.0004
0.1902	1130	0.0006
0.1919	1140	0.0005
0.1936	1150	0.0003
0.1953	1160	0.0003
0.1970	1170	0.0014
0.1987	1180	0.0003
0.2003	1190	0.0009
0.2020	1200	0.0016
0.2037	1210	0.0004
0.2054	1220	0.0005
0.2071	1230	0.0003
0.2088	1240	0.0003
0.2104	1250	0.0002
0.2121	1260	0.0003
0.2138	1270	0.0006
0.2155	1280	0.0002
0.2172	1290	0.0002
0.2189	1300	0.0003
0.2205	1310	0.0002
0.2222	1320	0.0002
0.2239	1330	0.0007
0.2256	1340	0.0004
0.2273	1350	0.0004
0.2290	1360	0.0004
0.2306	1370	0.0003
0.2323	1380	0.0005
0.2340	1390	0.0004
0.2357	1400	0.0046
0.2374	1410	0.0007
0.2391	1420	0.0005
0.2407	1430	0.0004
0.2424	1440	0.001
0.2441	1450	0.0004
0.2458	1460	0.0006
0.2475	1470	0.0004
0.2492	1480	0.0002
0.2508	1490	0.0002
0.2525	1500	0.0003
0.2542	1510	0.0003
0.2559	1520	0.0003
0.2576	1530	0.0011
0.2593	1540	0.0012
0.2609	1550	0.0004
0.2626	1560	0.0004
0.2643	1570	0.0004
0.2660	1580	0.0002
0.2677	1590	0.0002
0.2694	1600	0.0004
0.2710	1610	0.0006
0.2727	1620	0.0002
0.2744	1630	0.0003
0.2761	1640	0.0003
0.2778	1650	0.0003
0.2795	1660	0.0003
0.2811	1670	0.0008
0.2828	1680	0.0011
0.2845	1690	0.0008
0.2862	1700	0.0003
0.2879	1710	0.0006
0.2896	1720	0.0003
0.2912	1730	0.0003
0.2929	1740	0.0026
0.2946	1750	0.0006
0.2963	1760	0.0004
0.2980	1770	0.0007
0.2997	1780	0.0002
0.3013	1790	0.0005
0.3030	1800	0.0003
0.3047	1810	0.0003
0.3064	1820	0.0004
0.3081	1830	0.0002
0.3098	1840	0.0002
0.3114	1850	0.0003
0.3131	1860	0.0013
0.3148	1870	0.0002
0.3165	1880	0.0003
0.3182	1890	0.0003
0.3199	1900	0.0015
0.3215	1910	0.0005
0.3232	1920	0.0003
0.3249	1930	0.0005
0.3266	1940	0.0003
0.3283	1950	0.0003
0.3300	1960	0.0012
0.3316	1970	0.0006
0.3333	1980	0.001
0.3350	1990	0.0005
0.3367	2000	0.0002
0.3384	2010	0.0003
0.3401	2020	0.0003
0.3418	2030	0.0002
0.3434	2040	0.0003
0.3451	2050	0.0003
0.3468	2060	0.0002
0.3485	2070	0.0002
0.3502	2080	0.0003
0.3519	2090	0.0002
0.3535	2100	0.0002
0.3552	2110	0.0007
0.3569	2120	0.0003
0.3586	2130	0.0002
0.3603	2140	0.0002
0.3620	2150	0.0003
0.3636	2160	0.0001
0.3653	2170	0.0002
0.3670	2180	0.001
0.3687	2190	0.0008
0.3704	2200	0.001
0.3721	2210	0.0005
0.3737	2220	0.0006
0.3754	2230	0.0005
0.3771	2240	0.0002
0.3788	2250	0.0003
0.3805	2260	0.0003
0.3822	2270	0.0002
0.3838	2280	0.0002
0.3855	2290	0.0002
0.3872	2300	0.0004
0.3889	2310	0.0002
0.3906	2320	0.0002
0.3923	2330	0.0003
0.3939	2340	0.0002
0.3956	2350	0.0004
0.3973	2360	0.0016
0.3990	2370	0.0003
0.4007	2380	0.0004
0.4024	2390	0.0005
0.4040	2400	0.0003
0.4057	2410	0.0005
0.4074	2420	0.0012
0.4091	2430	0.0004
0.4108	2440	0.0004
0.4125	2450	0.0012
0.4141	2460	0.0005
0.4158	2470	0.0004
0.4175	2480	0.0004
0.4192	2490	0.0003
0.4209	2500	0.0005
0.4226	2510	0.0002
0.4242	2520	0.0003
0.4259	2530	0.0003
0.4276	2540	0.0006
0.4293	2550	0.0003
0.4310	2560	0.0017
0.4327	2570	0.0009
0.4343	2580	0.0003
0.4360	2590	0.0005
0.4377	2600	0.0003
0.4394	2610	0.0005
0.4411	2620	0.0005
0.4428	2630	0.0003
0.4444	2640	0.0009
0.4461	2650	0.0027
0.4478	2660	0.0004
0.4495	2670	0.0002
0.4512	2680	0.0002
0.4529	2690	0.0004
0.4545	2700	0.0003
0.4562	2710	0.0002
0.4579	2720	0.0003
0.4596	2730	0.0004
0.4613	2740	0.0002
0.4630	2750	0.0003
0.4646	2760	0.0002
0.4663	2770	0.0003
0.4680	2780	0.0002
0.4697	2790	0.0003
0.4714	2800	0.0001
0.4731	2810	0.0004
0.4747	2820	0.0002
0.4764	2830	0.0002
0.4781	2840	0.0002
0.4798	2850	0.0002
0.4815	2860	0.0001
0.4832	2870	0.0001
0.4848	2880	0.0004
0.4865	2890	0.0001
0.4882	2900	0.0005
0.4899	2910	0.0003
0.4916	2920	0.0001
0.4933	2930	0.0003
0.4949	2940	0.0008
0.4966	2950	0.0007
0.4983	2960	0.0007
0.5	2970	0.0005
0.5017	2980	0.0002
0.5034	2990	0.0002
0.5051	3000	0.0002
0.5067	3010	0.0004
0.5084	3020	0.0002
0.5101	3030	0.0002
0.5118	3040	0.0026
0.5135	3050	0.0003
0.5152	3060	0.0003
0.5168	3070	0.0002
0.5185	3080	0.0011
0.5202	3090	0.0003
0.5219	3100	0.0004
0.5236	3110	0.0004
0.5253	3120	0.0002
0.5269	3130	0.0004
0.5286	3140	0.0002
0.5303	3150	0.0003
0.5320	3160	0.0003
0.5337	3170	0.0004
0.5354	3180	0.0004
0.5370	3190	0.0002
0.5387	3200	0.0003
0.5404	3210	0.0003
0.5421	3220	0.0001
0.5438	3230	0.0002
0.5455	3240	0.0002
0.5471	3250	0.0009
0.5488	3260	0.0001
0.5505	3270	0.0004
0.5522	3280	0.0005
0.5539	3290	0.0003
0.5556	3300	0.0002
0.5572	3310	0.0002
0.5589	3320	0.0004
0.5606	3330	0.0002
0.5623	3340	0.0003
0.5640	3350	0.0002
0.5657	3360	0.0006
0.5673	3370	0.0004
0.5690	3380	0.0002
0.5707	3390	0.0002
0.5724	3400	0.0002
0.5741	3410	0.0002
0.5758	3420	0.0004
0.5774	3430	0.0002
0.5791	3440	0.0009
0.5808	3450	0.0003
0.5825	3460	0.0003
0.5842	3470	0.0002
0.5859	3480	0.0002
0.5875	3490	0.0002
0.5892	3500	0.0002
0.5909	3510	0.0002
0.5926	3520	0.0002
0.5943	3530	0.0003
0.5960	3540	0.0003
0.5976	3550	0.0001
0.5993	3560	0.0002
0.6010	3570	0.0002
0.6027	3580	0.0002
0.6044	3590	0.0002
0.6061	3600	0.0002
0.6077	3610	0.0002
0.6094	3620	0.0002
0.6111	3630	0.0002
0.6128	3640	0.0001
0.6145	3650	0.0002
0.6162	3660	0.0003
0.6178	3670	0.0002
0.6195	3680	0.0002
0.6212	3690	0.0001
0.6229	3700	0.0001
0.6246	3710	0.0001
0.6263	3720	0.0002
0.6279	3730	0.0001
0.6296	3740	0.0001
0.6313	3750	0.0005
0.6330	3760	0.0002
0.6347	3770	0.0003
0.6364	3780	0.0001
0.6380	3790	0.0001
0.6397	3800	0.0003
0.6414	3810	0.0008
0.6431	3820	0.0002
0.6448	3830	0.0002
0.6465	3840	0.0002
0.6481	3850	0.0002
0.6498	3860	0.0002
0.6515	3870	0.0001
0.6532	3880	0.0002
0.6549	3890	0.0002
0.6566	3900	0.0001
0.6582	3910	0.0003
0.6599	3920	0.0011
0.6616	3930	0.0002
0.6633	3940	0.0002
0.6650	3950	0.0001
0.6667	3960	0.0002
0.6684	3970	0.0001
0.6700	3980	0.0002
0.6717	3990	0.0002
0.6734	4000	0.0001
0.6751	4010	0.0002
0.6768	4020	0.0001
0.6785	4030	0.0001
0.6801	4040	0.0002
0.6818	4050	0.0002
0.6835	4060	0.0001
0.6852	4070	0.0003
0.6869	4080	0.0003
0.6886	4090	0.0005
0.6902	4100	0.0002
0.6919	4110	0.0004
0.6936	4120	0.0005
0.6953	4130	0.0002
0.6970	4140	0.0002
0.6987	4150	0.0002
0.7003	4160	0.0003
0.7020	4170	0.0008
0.7037	4180	0.0002
0.7054	4190	0.0003
0.7071	4200	0.0002
0.7088	4210	0.0003
0.7104	4220	0.0002
0.7121	4230	0.0002
0.7138	4240	0.0002
0.7155	4250	0.0001
0.7172	4260	0.0003
0.7189	4270	0.0004
0.7205	4280	0.0002
0.7222	4290	0.0002
0.7239	4300	0.0002
0.7256	4310	0.0011
0.7273	4320	0.0003
0.7290	4330	0.0003
0.7306	4340	0.0002
0.7323	4350	0.0002
0.7340	4360	0.0002
0.7357	4370	0.0003
0.7374	4380	0.0001
0.7391	4390	0.0004
0.7407	4400	0.0002
0.7424	4410	0.0002
0.7441	4420	0.0002
0.7458	4430	0.0001
0.7475	4440	0.0001
0.7492	4450	0.0001
0.7508	4460	0.0002
0.7525	4470	0.0004
0.7542	4480	0.0003
0.7559	4490	0.0002
0.7576	4500	0.0001
0.7593	4510	0.0005
0.7609	4520	0.0002
0.7626	4530	0.0002
0.7643	4540	0.0002
0.7660	4550	0.0001
0.7677	4560	0.0002
0.7694	4570	0.0001
0.7710	4580	0.0002
0.7727	4590	0.0008
0.7744	4600	0.0002
0.7761	4610	0.0002
0.7778	4620	0.0002
0.7795	4630	0.0002
0.7811	4640	0.0002
0.7828	4650	0.0013
0.7845	4660	0.0001
0.7862	4670	0.0002
0.7879	4680	0.0022
0.7896	4690	0.0001
0.7912	4700	0.0003
0.7929	4710	0.0002
0.7946	4720	0.0002
0.7963	4730	0.0004
0.7980	4740	0.0001
0.7997	4750	0.0003
0.8013	4760	0.0001
0.8030	4770	0.0001
0.8047	4780	0.0002
0.8064	4790	0.0001
0.8081	4800	0.0001
0.8098	4810	0.0004
0.8114	4820	0.0001
0.8131	4830	0.0004
0.8148	4840	0.0001
0.8165	4850	0.0001
0.8182	4860	0.0003
0.8199	4870	0.0001
0.8215	4880	0.0003
0.8232	4890	0.0001
0.8249	4900	0.0001
0.8266	4910	0.0002
0.8283	4920	0.0025
0.8300	4930	0.0018
0.8316	4940	0.0011
0.8333	4950	0.0003
0.8350	4960	0.0003
0.8367	4970	0.0004
0.8384	4980	0.0002
0.8401	4990	0.0003
0.8418	5000	0.0002
0.8434	5010	0.0002
0.8451	5020	0.0001
0.8468	5030	0.0002
0.8485	5040	0.0006
0.8502	5050	0.0003
0.8519	5060	0.001
0.8535	5070	0.0001
0.8552	5080	0.0001
0.8569	5090	0.0001
0.8586	5100	0.0001
0.8603	5110	0.0001
0.8620	5120	0.0005
0.8636	5130	0.0004
0.8653	5140	0.0002
0.8670	5150	0.0002
0.8687	5160	0.0001
0.8704	5170	0.0003
0.8721	5180	0.0004
0.8737	5190	0.0002
0.8754	5200	0.0002
0.8771	5210	0.0002
0.8788	5220	0.0001
0.8805	5230	0.0001
0.8822	5240	0.0003
0.8838	5250	0.0001
0.8855	5260	0.0004
0.8872	5270	0.0001
0.8889	5280	0.0002
0.8906	5290	0.0003
0.8923	5300	0.0004
0.8939	5310	0.0001
0.8956	5320	0.0001
0.8973	5330	0.0003
0.8990	5340	0.0001
0.9007	5350	0.0001
0.9024	5360	0.0001
0.9040	5370	0.0001
0.9057	5380	0.0001
0.9074	5390	0.0001
0.9091	5400	0.0001
0.9108	5410	0.0001
0.9125	5420	0.0005
0.9141	5430	0.0006
0.9158	5440	0.0001
0.9175	5450	0.0001
0.9192	5460	0.0001
0.9209	5470	0.0001
0.9226	5480	0.0001
0.9242	5490	0.0001
0.9259	5500	0.0002
0.9276	5510	0.0002
0.9293	5520	0.0002
0.9310	5530	0.0001
0.9327	5540	0.0002
0.9343	5550	0.0001
0.9360	5560	0.0002
0.9377	5570	0.0001
0.9394	5580	0.0002
0.9411	5590	0.0001
0.9428	5600	0.0001
0.9444	5610	0.0001
0.9461	5620	0.0001
0.9478	5630	0.0001
0.9495	5640	0.0002
0.9512	5650	0.0001
0.9529	5660	0.0001
0.9545	5670	0.0001
0.9562	5680	0.0001
0.9579	5690	0.0001
0.9596	5700	0.0002
0.9613	5710	0.0001
0.9630	5720	0.0001
0.9646	5730	0.0002
0.9663	5740	0.0002
0.9680	5750	0.0001
0.9697	5760	0.0001
0.9714	5770	0.0001
0.9731	5780	0.0002
0.9747	5790	0.0004
0.9764	5800	0.0001
0.9781	5810	0.0012
0.9798	5820	0.0003
0.9815	5830	0.0002
0.9832	5840	0.0001
0.9848	5850	0.0001
0.9865	5860	0.0001
0.9882	5870	0.0001
0.9899	5880	0.0001
0.9916	5890	0.0002
0.9933	5900	0.0001
0.9949	5910	0.0003
0.9966	5920	0.0001
0.9983	5930	0.0002
1.0	5940	0.0001

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.0
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.3.1
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}