train_qnli_101112_1760638090

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.1869
• Num Input Tokens Seen: 207147488

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1537	1.0	23567	0.1973	10356896
0.4046	2.0	47134	0.1900	20715296
0.1866	3.0	70701	0.1885	31065184
0.0728	4.0	94268	0.1873	41428128
0.1521	5.0	117835	0.1885	51784320
0.0744	6.0	141402	0.1887	62144160
0.2178	7.0	164969	0.1876	72511552
0.2643	8.0	188536	0.1871	82864256
0.0972	9.0	212103	0.1884	93220320
0.3013	10.0	235670	0.1869	103572992
0.1516	11.0	259237	0.1869	113924768
0.2913	12.0	282804	0.1884	124282240
0.1283	13.0	306371	0.1878	134645600
0.0657	14.0	329938	0.1882	145000704
0.053	15.0	353505	0.1877	155349152
0.2966	16.0	377072	0.1882	165706304
0.2382	17.0	400639	0.1870	176064704
0.2344	18.0	424206	0.1870	186423936
0.2151	19.0	447773	0.1870	196786464
0.2732	20.0	471340	0.1870	207147488

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4