codersan/FaLaBSE_Mizan2

SentenceTransformer based on codersan/FaLabse

This is a sentence-transformers model finetuned from codersan/FaLabse. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: codersan/FaLabse
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("codersan/FaLaBSE_Mizan2")
# Run inference
sentences = [
    'اگر این کار مداومت می\u200cیافت، سنگر قادر به مقاومت نمی\u200cبود.',
    'If this were continued, the barricade was no longer tenable.',
    'Well, for this moment she had a protector.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,021,596 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 18.63 tokens max: 81 tokens	min: 3 tokens mean: 16.37 tokens max: 85 tokens

• Samples:

  anchor	positive
  دختران برای اطاعت امر پدر از جا برخاستند.	They arose to obey.
  همه چیز را بم وقع خواهی دانست.	You'll know it all in time
  او هر لحظه گرفتار یک‌ وضع است، زارزار گریه می‌کند. می‌گوید به ما توهین کرده‌اند، حیثیتمان را لکه‌دار نمودند.	She is in hysterics up there, and moans and says that we have been 'shamed and disgraced.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 12
• learning_rate: 5e-06
• weight_decay: 0.01
• max_grad_norm: 5
• num_train_epochs: 1
• warmup_ratio: 0.1
• push_to_hub: True
• hub_model_id: codersan/FaLaBSE_Mizan2
• eval_on_start: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 12
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-06
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 5
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: codersan/FaLaBSE_Mizan2
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0	0	-
0.0012	100	0.054
0.0023	200	0.0442
0.0035	300	0.0714
0.0047	400	0.0715
0.0059	500	0.0642
0.0070	600	0.058
0.0082	700	0.062
0.0094	800	0.0626
0.0106	900	0.0466
0.0117	1000	0.0617
0.0129	1100	0.0464
0.0141	1200	0.0532
0.0153	1300	0.0472
0.0164	1400	0.0396
0.0176	1500	0.0587
0.0188	1600	0.0378
0.0200	1700	0.0448
0.0211	1800	0.0475
0.0223	1900	0.0533
0.0235	2000	0.0693
0.0247	2100	0.0451
0.0258	2200	0.0397
0.0270	2300	0.0392
0.0282	2400	0.0437
0.0294	2500	0.0467
0.0305	2600	0.0456
0.0317	2700	0.0274
0.0329	2800	0.0379
0.0341	2900	0.0412
0.0352	3000	0.0445
0.0364	3100	0.0419
0.0376	3200	0.032
0.0388	3300	0.0351
0.0399	3400	0.0442
0.0411	3500	0.0434
0.0423	3600	0.0331
0.0435	3700	0.0398
0.0446	3800	0.0518
0.0458	3900	0.0287
0.0470	4000	0.0322
0.0482	4100	0.0389
0.0493	4200	0.0268
0.0505	4300	0.0352
0.0517	4400	0.021
0.0529	4500	0.0322
0.0540	4600	0.0228
0.0552	4700	0.0396
0.0564	4800	0.033
0.0576	4900	0.0444
0.0587	5000	0.0392
0.0599	5100	0.033
0.0611	5200	0.0401
0.0623	5300	0.0397
0.0634	5400	0.0327
0.0646	5500	0.0346
0.0658	5600	0.0315
0.0670	5700	0.0315
0.0681	5800	0.0234
0.0693	5900	0.0311
0.0705	6000	0.0323
0.0717	6100	0.0248
0.0728	6200	0.0384
0.0740	6300	0.0394
0.0752	6400	0.0299
0.0764	6500	0.0479
0.0775	6600	0.0253
0.0787	6700	0.0424
0.0799	6800	0.0269
0.0810	6900	0.035
0.0822	7000	0.0349
0.0834	7100	0.0302
0.0846	7200	0.0426
0.0857	7300	0.0287
0.0869	7400	0.0254
0.0881	7500	0.0306
0.0893	7600	0.0356
0.0904	7700	0.0393
0.0916	7800	0.035
0.0928	7900	0.0449
0.0940	8000	0.0228
0.0951	8100	0.0342
0.0963	8200	0.0233
0.0975	8300	0.0259
0.0987	8400	0.0402
0.0998	8500	0.0277
0.1010	8600	0.0345
0.1022	8700	0.0361
0.1034	8800	0.0326
0.1045	8900	0.0367
0.1057	9000	0.0408
0.1069	9100	0.0289
0.1081	9200	0.026
0.1092	9300	0.0367
0.1104	9400	0.0327
0.1116	9500	0.0273
0.1128	9600	0.0545
0.1139	9700	0.0395
0.1151	9800	0.0394
0.1163	9900	0.0293
0.1175	10000	0.0411
0.1186	10100	0.0353
0.1198	10200	0.0369
0.1210	10300	0.0222
0.1222	10400	0.0418
0.1233	10500	0.039
0.1245	10600	0.041
0.1257	10700	0.0316
0.1269	10800	0.0351
0.1280	10900	0.0258
0.1292	11000	0.0481
0.1304	11100	0.027
0.1316	11200	0.0357
0.1327	11300	0.0366
0.1339	11400	0.0345
0.1351	11500	0.0311
0.1363	11600	0.0335
0.1374	11700	0.0268
0.1386	11800	0.0272
0.1398	11900	0.0317
0.1410	12000	0.052
0.1421	12100	0.027
0.1433	12200	0.028
0.1445	12300	0.0435
0.1457	12400	0.0335
0.1468	12500	0.0506
0.1480	12600	0.033
0.1492	12700	0.0278
0.1504	12800	0.0298
0.1515	12900	0.0317
0.1527	13000	0.0157
0.1539	13100	0.0252
0.1551	13200	0.0214
0.1562	13300	0.0269
0.1574	13400	0.0287
0.1586	13500	0.0261
0.1598	13600	0.0195
0.1609	13700	0.0262
0.1621	13800	0.0446
0.1633	13900	0.0402
0.1644	14000	0.0318
0.1656	14100	0.039
0.1668	14200	0.0227
0.1680	14300	0.0247
0.1691	14400	0.0236
0.1703	14500	0.0213
0.1715	14600	0.0434
0.1727	14700	0.0486
0.1738	14800	0.0537
0.1750	14900	0.033
0.1762	15000	0.0289
0.1774	15100	0.0389
0.1785	15200	0.0267
0.1797	15300	0.031
0.1809	15400	0.029
0.1821	15500	0.0357
0.1832	15600	0.0231
0.1844	15700	0.035
0.1856	15800	0.0201
0.1868	15900	0.0361
0.1879	16000	0.0297
0.1891	16100	0.0216
0.1903	16200	0.0283
0.1915	16300	0.0205
0.1926	16400	0.0318
0.1938	16500	0.0385
0.1950	16600	0.0363
0.1962	16700	0.0462
0.1973	16800	0.0342
0.1985	16900	0.0213
0.1997	17000	0.0492
0.2009	17100	0.0354
0.2020	17200	0.0219
0.2032	17300	0.0338
0.2044	17400	0.0322
0.2056	17500	0.0283
0.2067	17600	0.024
0.2079	17700	0.0206
0.2091	17800	0.0416
0.2103	17900	0.0284
0.2114	18000	0.0305
0.2126	18100	0.0261
0.2138	18200	0.0228
0.2150	18300	0.048
0.2161	18400	0.0241
0.2173	18500	0.0484
0.2185	18600	0.0362
0.2197	18700	0.0296
0.2208	18800	0.0335
0.2220	18900	0.0383
0.2232	19000	0.0378
0.2244	19100	0.042
0.2255	19200	0.0405
0.2267	19300	0.0369
0.2279	19400	0.0238
0.2291	19500	0.0226
0.2302	19600	0.0338
0.2314	19700	0.0299
0.2326	19800	0.0436
0.2338	19900	0.0302
0.2349	20000	0.0253
0.2361	20100	0.0233
0.2373	20200	0.0427
0.2385	20300	0.0328
0.2396	20400	0.0366
0.2408	20500	0.0231
0.2420	20600	0.0467
0.2431	20700	0.0287
0.2443	20800	0.0393
0.2455	20900	0.0276
0.2467	21000	0.0355
0.2478	21100	0.0189
0.2490	21200	0.0152
0.2502	21300	0.0272
0.2514	21400	0.0267
0.2525	21500	0.044
0.2537	21600	0.024
0.2549	21700	0.0142
0.2561	21800	0.0263
0.2572	21900	0.0273
0.2584	22000	0.0238
0.2596	22100	0.0185
0.2608	22200	0.0459
0.2619	22300	0.0351
0.2631	22400	0.0498
0.2643	22500	0.0478
0.2655	22600	0.0331
0.2666	22700	0.0276
0.2678	22800	0.025
0.2690	22900	0.0424
0.2702	23000	0.0335
0.2713	23100	0.0401
0.2725	23200	0.038
0.2737	23300	0.0184
0.2749	23400	0.0235
0.2760	23500	0.0361
0.2772	23600	0.0359
0.2784	23700	0.0279
0.2796	23800	0.038
0.2807	23900	0.0198
0.2819	24000	0.0466
0.2831	24100	0.0297
0.2843	24200	0.0189
0.2854	24300	0.0418
0.2866	24400	0.0247
0.2878	24500	0.054
0.2890	24600	0.0449
0.2901	24700	0.0532
0.2913	24800	0.0317
0.2925	24900	0.0427
0.2937	25000	0.0282
0.2948	25100	0.029
0.2960	25200	0.0298
0.2972	25300	0.0297
0.2984	25400	0.0414
0.2995	25500	0.0297
0.3007	25600	0.0525
0.3019	25700	0.0478
0.3031	25800	0.0287
0.3042	25900	0.0235
0.3054	26000	0.0344
0.3066	26100	0.041
0.3078	26200	0.0325
0.3089	26300	0.0334
0.3101	26400	0.0211
0.3113	26500	0.0461
0.3125	26600	0.025
0.3136	26700	0.0276
0.3148	26800	0.0322
0.3160	26900	0.0261
0.3172	27000	0.0268
0.3183	27100	0.0349
0.3195	27200	0.0303
0.3207	27300	0.026
0.3218	27400	0.0328
0.3230	27500	0.0294
0.3242	27600	0.0275
0.3254	27700	0.0343
0.3265	27800	0.0294
0.3277	27900	0.032
0.3289	28000	0.0221
0.3301	28100	0.0249
0.3312	28200	0.0311
0.3324	28300	0.0257
0.3336	28400	0.0424
0.3348	28500	0.0394
0.3359	28600	0.044
0.3371	28700	0.0271
0.3383	28800	0.0363
0.3395	28900	0.0329
0.3406	29000	0.0383
0.3418	29100	0.0414
0.3430	29200	0.0219
0.3442	29300	0.0137
0.3453	29400	0.0389
0.3465	29500	0.0355
0.3477	29600	0.0105
0.3489	29700	0.0347
0.3500	29800	0.037
0.3512	29900	0.0333
0.3524	30000	0.0164
0.3536	30100	0.0336
0.3547	30200	0.0345
0.3559	30300	0.0359
0.3571	30400	0.0343
0.3583	30500	0.0528
0.3594	30600	0.0332
0.3606	30700	0.0487
0.3618	30800	0.0302
0.3630	30900	0.037
0.3641	31000	0.0339
0.3653	31100	0.0359
0.3665	31200	0.0403
0.3677	31300	0.0376
0.3688	31400	0.0367
0.3700	31500	0.0452
0.3712	31600	0.023
0.3724	31700	0.0281
0.3735	31800	0.0297
0.3747	31900	0.0353
0.3759	32000	0.0215
0.3771	32100	0.0234
0.3782	32200	0.0245
0.3794	32300	0.0485
0.3806	32400	0.0249
0.3818	32500	0.021
0.3829	32600	0.0381
0.3841	32700	0.0332
0.3853	32800	0.0263
0.3865	32900	0.0346
0.3876	33000	0.0401
0.3888	33100	0.0318
0.3900	33200	0.0224
0.3912	33300	0.0225
0.3923	33400	0.0265
0.3935	33500	0.0204
0.3947	33600	0.0321
0.3959	33700	0.0188
0.3970	33800	0.0338
0.3982	33900	0.0309
0.3994	34000	0.0233
0.4005	34100	0.0303
0.4017	34200	0.0387
0.4029	34300	0.0255
0.4041	34400	0.0212
0.4052	34500	0.0324
0.4064	34600	0.0412
0.4076	34700	0.0203
0.4088	34800	0.0211
0.4099	34900	0.031
0.4111	35000	0.0178
0.4123	35100	0.0222
0.4135	35200	0.018
0.4146	35300	0.0274
0.4158	35400	0.0364
0.4170	35500	0.0254
0.4182	35600	0.0219
0.4193	35700	0.0352
0.4205	35800	0.0324
0.4217	35900	0.026
0.4229	36000	0.0212
0.4240	36100	0.0326
0.4252	36200	0.0332
0.4264	36300	0.0358
0.4276	36400	0.0301
0.4287	36500	0.0328
0.4299	36600	0.0289
0.4311	36700	0.0351
0.4323	36800	0.0331
0.4334	36900	0.0209
0.4346	37000	0.0392
0.4358	37100	0.0171
0.4370	37200	0.035
0.4381	37300	0.0395
0.4393	37400	0.0437
0.4405	37500	0.0355
0.4417	37600	0.0383
0.4428	37700	0.0227
0.4440	37800	0.0286
0.4452	37900	0.0337
0.4464	38000	0.0514
0.4475	38100	0.0299
0.4487	38200	0.0343
0.4499	38300	0.025
0.4511	38400	0.0193
0.4522	38500	0.0293
0.4534	38600	0.0159
0.4546	38700	0.0183
0.4558	38800	0.0226
0.4569	38900	0.0437
0.4581	39000	0.0242
0.4593	39100	0.0396
0.4605	39200	0.0414
0.4616	39300	0.0337
0.4628	39400	0.035
0.4640	39500	0.0175
0.4652	39600	0.0228
0.4663	39700	0.019
0.4675	39800	0.0402
0.4687	39900	0.0177
0.4699	40000	0.0287
0.4710	40100	0.0262
0.4722	40200	0.0347
0.4734	40300	0.0249
0.4746	40400	0.0217
0.4757	40500	0.0258
0.4769	40600	0.0336
0.4781	40700	0.0391
0.4793	40800	0.042
0.4804	40900	0.03
0.4816	41000	0.0205
0.4828	41100	0.0273
0.4839	41200	0.0564
0.4851	41300	0.0311
0.4863	41400	0.0333
0.4875	41500	0.0162
0.4886	41600	0.0414
0.4898	41700	0.044
0.4910	41800	0.0411
0.4922	41900	0.0384
0.4933	42000	0.0269
0.4945	42100	0.0414
0.4957	42200	0.0175
0.4969	42300	0.0223
0.4980	42400	0.0354
0.4992	42500	0.0338
0.5004	42600	0.0182
0.5016	42700	0.0217
0.5027	42800	0.0227
0.5039	42900	0.0247
0.5051	43000	0.0238
0.5063	43100	0.0357
0.5074	43200	0.0237
0.5086	43300	0.0308
0.5098	43400	0.0294
0.5110	43500	0.0258
0.5121	43600	0.0234
0.5133	43700	0.0324
0.5145	43800	0.0334
0.5157	43900	0.0256
0.5168	44000	0.0243
0.5180	44100	0.0231
0.5192	44200	0.0312
0.5204	44300	0.0278
0.5215	44400	0.0432
0.5227	44500	0.0413
0.5239	44600	0.047
0.5251	44700	0.0384
0.5262	44800	0.0181
0.5274	44900	0.0303
0.5286	45000	0.0297
0.5298	45100	0.0292
0.5309	45200	0.033
0.5321	45300	0.0299
0.5333	45400	0.0269
0.5345	45500	0.0255
0.5356	45600	0.0395
0.5368	45700	0.0302
0.5380	45800	0.0237
0.5392	45900	0.0228
0.5403	46000	0.0329
0.5415	46100	0.0265
0.5427	46200	0.0187
0.5439	46300	0.0358
0.5450	46400	0.0319
0.5462	46500	0.0292
0.5474	46600	0.0366
0.5486	46700	0.0369
0.5497	46800	0.0219
0.5509	46900	0.0339
0.5521	47000	0.0383
0.5533	47100	0.0316
0.5544	47200	0.0374
0.5556	47300	0.0199
0.5568	47400	0.0279
0.5580	47500	0.04
0.5591	47600	0.0276
0.5603	47700	0.0281
0.5615	47800	0.0288
0.5626	47900	0.0256
0.5638	48000	0.0262
0.5650	48100	0.0264
0.5662	48200	0.0222
0.5673	48300	0.0202
0.5685	48400	0.0233
0.5697	48500	0.034
0.5709	48600	0.0354
0.5720	48700	0.0455
0.5732	48800	0.0384
0.5744	48900	0.0362
0.5756	49000	0.0249
0.5767	49100	0.0392
0.5779	49200	0.0279
0.5791	49300	0.0255
0.5803	49400	0.0254
0.5814	49500	0.0187
0.5826	49600	0.0215
0.5838	49700	0.0407
0.5850	49800	0.0158
0.5861	49900	0.0404
0.5873	50000	0.0303
0.5885	50100	0.0296
0.5897	50200	0.0307
0.5908	50300	0.0217
0.5920	50400	0.0436
0.5932	50500	0.0253
0.5944	50600	0.0191
0.5955	50700	0.032
0.5967	50800	0.0399
0.5979	50900	0.0346
0.5991	51000	0.031
0.6002	51100	0.0214
0.6014	51200	0.0134
0.6026	51300	0.0337
0.6038	51400	0.0394
0.6049	51500	0.0359
0.6061	51600	0.019
0.6073	51700	0.0145
0.6085	51800	0.0157
0.6096	51900	0.0496
0.6108	52000	0.0113
0.6120	52100	0.0293
0.6132	52200	0.0165
0.6143	52300	0.03
0.6155	52400	0.0266
0.6167	52500	0.0244
0.6179	52600	0.0234
0.6190	52700	0.0354
0.6202	52800	0.0176
0.6214	52900	0.0377
0.6226	53000	0.0374
0.6237	53100	0.0147
0.6249	53200	0.0408
0.6261	53300	0.0281
0.6273	53400	0.0474
0.6284	53500	0.0389
0.6296	53600	0.0294
0.6308	53700	0.0316
0.6320	53800	0.0314
0.6331	53900	0.0246
0.6343	54000	0.0243
0.6355	54100	0.0138
0.6367	54200	0.0242
0.6378	54300	0.0164
0.6390	54400	0.0279
0.6402	54500	0.0195
0.6413	54600	0.0202
0.6425	54700	0.0239
0.6437	54800	0.0287
0.6449	54900	0.0186
0.6460	55000	0.0141
0.6472	55100	0.0182
0.6484	55200	0.0275
0.6496	55300	0.0227
0.6507	55400	0.027
0.6519	55500	0.0242
0.6531	55600	0.0179
0.6543	55700	0.0245
0.6554	55800	0.0288
0.6566	55900	0.0189
0.6578	56000	0.0336
0.6590	56100	0.0328
0.6601	56200	0.0295
0.6613	56300	0.032
0.6625	56400	0.0256
0.6637	56500	0.0387
0.6648	56600	0.031
0.6660	56700	0.0188
0.6672	56800	0.028
0.6684	56900	0.0397
0.6695	57000	0.0285
0.6707	57100	0.033
0.6719	57200	0.0281
0.6731	57300	0.0196
0.6742	57400	0.025
0.6754	57500	0.0397
0.6766	57600	0.0403
0.6778	57700	0.022
0.6789	57800	0.0392
0.6801	57900	0.0254
0.6813	58000	0.0316
0.6825	58100	0.0186
0.6836	58200	0.0271
0.6848	58300	0.035
0.6860	58400	0.0322
0.6872	58500	0.0147
0.6883	58600	0.0314
0.6895	58700	0.0214
0.6907	58800	0.0177
0.6919	58900	0.0307
0.6930	59000	0.0246
0.6942	59100	0.0125
0.6954	59200	0.0232
0.6966	59300	0.0325
0.6977	59400	0.0253
0.6989	59500	0.0151
0.7001	59600	0.0261
0.7013	59700	0.0253
0.7024	59800	0.0124
0.7036	59900	0.0298
0.7048	60000	0.0254
0.7060	60100	0.0262
0.7071	60200	0.0274
0.7083	60300	0.0344
0.7095	60400	0.03
0.7107	60500	0.0312
0.7118	60600	0.0354
0.7130	60700	0.0334
0.7142	60800	0.0325
0.7154	60900	0.0236
0.7165	61000	0.0266
0.7177	61100	0.0183
0.7189	61200	0.045
0.7200	61300	0.0174
0.7212	61400	0.0518
0.7224	61500	0.0247
0.7236	61600	0.0255
0.7247	61700	0.0209
0.7259	61800	0.0206
0.7271	61900	0.0306
0.7283	62000	0.0215
0.7294	62100	0.0241
0.7306	62200	0.0324
0.7318	62300	0.0433
0.7330	62400	0.0238
0.7341	62500	0.0302
0.7353	62600	0.0282
0.7365	62700	0.0371
0.7377	62800	0.0397
0.7388	62900	0.0488
0.7400	63000	0.032
0.7412	63100	0.0161
0.7424	63200	0.0351
0.7435	63300	0.0282
0.7447	63400	0.0221
0.7459	63500	0.0275
0.7471	63600	0.0198
0.7482	63700	0.0339
0.7494	63800	0.0285
0.7506	63900	0.0314
0.7518	64000	0.0216
0.7529	64100	0.0383
0.7541	64200	0.0386
0.7553	64300	0.0305
0.7565	64400	0.0265
0.7576	64500	0.0288
0.7588	64600	0.0125
0.7600	64700	0.0212
0.7612	64800	0.0242
0.7623	64900	0.0384
0.7635	65000	0.0163
0.7647	65100	0.0132
0.7659	65200	0.0209
0.7670	65300	0.0408
0.7682	65400	0.0312
0.7694	65500	0.0382
0.7706	65600	0.0217
0.7717	65700	0.0384
0.7729	65800	0.0267
0.7741	65900	0.047
0.7753	66000	0.021
0.7764	66100	0.0138
0.7776	66200	0.0308
0.7788	66300	0.0193
0.7800	66400	0.0285
0.7811	66500	0.0235
0.7823	66600	0.0281
0.7835	66700	0.0407
0.7847	66800	0.0269
0.7858	66900	0.0346
0.7870	67000	0.0223
0.7882	67100	0.0278
0.7894	67200	0.0255
0.7905	67300	0.014
0.7917	67400	0.0248
0.7929	67500	0.022
0.7941	67600	0.0292
0.7952	67700	0.038
0.7964	67800	0.0158
0.7976	67900	0.0212
0.7988	68000	0.0405
0.7999	68100	0.029
0.8011	68200	0.0379
0.8023	68300	0.0256
0.8034	68400	0.0263
0.8046	68500	0.0214
0.8058	68600	0.0224
0.8070	68700	0.0159
0.8081	68800	0.0302
0.8093	68900	0.0313
0.8105	69000	0.0395
0.8117	69100	0.0296
0.8128	69200	0.0353
0.8140	69300	0.025
0.8152	69400	0.0246
0.8164	69500	0.0312
0.8175	69600	0.0199
0.8187	69700	0.0225
0.8199	69800	0.0254
0.8211	69900	0.0095
0.8222	70000	0.0326
0.8234	70100	0.0355
0.8246	70200	0.0368
0.8258	70300	0.0339
0.8269	70400	0.0278
0.8281	70500	0.0249
0.8293	70600	0.0379
0.8305	70700	0.0368
0.8316	70800	0.0146
0.8328	70900	0.0153
0.8340	71000	0.0352
0.8352	71100	0.0248
0.8363	71200	0.0255
0.8375	71300	0.0306
0.8387	71400	0.0293
0.8399	71500	0.0303
0.8410	71600	0.0244
0.8422	71700	0.0174
0.8434	71800	0.0241
0.8446	71900	0.0276
0.8457	72000	0.0359
0.8469	72100	0.0257
0.8481	72200	0.0344
0.8493	72300	0.0275
0.8504	72400	0.022
0.8516	72500	0.0275
0.8528	72600	0.0317
0.8540	72700	0.0386
0.8551	72800	0.0421
0.8563	72900	0.0259
0.8575	73000	0.0244
0.8587	73100	0.0231
0.8598	73200	0.0373
0.8610	73300	0.0296
0.8622	73400	0.024
0.8634	73500	0.0382
0.8645	73600	0.0223
0.8657	73700	0.0254
0.8669	73800	0.0259
0.8681	73900	0.0171
0.8692	74000	0.0268
0.8704	74100	0.0196
0.8716	74200	0.0206
0.8728	74300	0.0411
0.8739	74400	0.039
0.8751	74500	0.0197
0.8763	74600	0.0144
0.8775	74700	0.0231
0.8786	74800	0.0217
0.8798	74900	0.0244
0.8810	75000	0.0291
0.8821	75100	0.0243
0.8833	75200	0.0294
0.8845	75300	0.0129
0.8857	75400	0.0291
0.8868	75500	0.0273
0.8880	75600	0.0297
0.8892	75700	0.0266
0.8904	75800	0.0374
0.8915	75900	0.0225
0.8927	76000	0.0223
0.8939	76100	0.0229
0.8951	76200	0.0306
0.8962	76300	0.0238
0.8974	76400	0.0197
0.8986	76500	0.0265
0.8998	76600	0.0411
0.9009	76700	0.022
0.9021	76800	0.0151
0.9033	76900	0.0251
0.9045	77000	0.0211
0.9056	77100	0.0302
0.9068	77200	0.0229
0.9080	77300	0.0398
0.9092	77400	0.0174
0.9103	77500	0.0327
0.9115	77600	0.0258
0.9127	77700	0.026
0.9139	77800	0.0251
0.9150	77900	0.0351
0.9162	78000	0.0315
0.9174	78100	0.0342
0.9186	78200	0.0244
0.9197	78300	0.0171
0.9209	78400	0.043
0.9221	78500	0.0189
0.9233	78600	0.0241
0.9244	78700	0.0266
0.9256	78800	0.0173
0.9268	78900	0.0238
0.9280	79000	0.0222
0.9291	79100	0.0416
0.9303	79200	0.0377
0.9315	79300	0.0311
0.9327	79400	0.0251
0.9338	79500	0.0208
0.9350	79600	0.0274
0.9362	79700	0.0327
0.9374	79800	0.0258
0.9385	79900	0.0175
0.9397	80000	0.0297
0.9409	80100	0.0182
0.9421	80200	0.0279
0.9432	80300	0.0197
0.9444	80400	0.0122
0.9456	80500	0.0293
0.9468	80600	0.0126
0.9479	80700	0.0317
0.9491	80800	0.0276
0.9503	80900	0.025
0.9515	81000	0.0264
0.9526	81100	0.0236
0.9538	81200	0.0273
0.9550	81300	0.0276
0.9562	81400	0.0262
0.9573	81500	0.0167
0.9585	81600	0.0313
0.9597	81700	0.0253
0.9608	81800	0.0207
0.9620	81900	0.0199
0.9632	82000	0.0411
0.9644	82100	0.0302
0.9655	82200	0.0244
0.9667	82300	0.0264
0.9679	82400	0.0213
0.9691	82500	0.0137
0.9702	82600	0.019
0.9714	82700	0.0318
0.9726	82800	0.037
0.9738	82900	0.0249
0.9749	83000	0.0193
0.9761	83100	0.0243
0.9773	83200	0.0222
0.9785	83300	0.0271
0.9796	83400	0.0147
0.9808	83500	0.0211
0.9820	83600	0.0248
0.9832	83700	0.0216
0.9843	83800	0.0238
0.9855	83900	0.0231
0.9867	84000	0.0308
0.9879	84100	0.0282
0.9890	84200	0.0217
0.9902	84300	0.021
0.9914	84400	0.0162
0.9926	84500	0.0288
0.9937	84600	0.0343
0.9949	84700	0.0192
0.9961	84800	0.0256
0.9973	84900	0.0181
0.9984	85000	0.0186
0.9996	85100	0.0206

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.0
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}