train_qnli_1755694488

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.1079
• Num Input Tokens Seen: 94426336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0296	0.5	23567	0.0760	4726160
0.0621	1.0	47134	0.0588	9443856
0.137	1.5	70701	0.0478	14171744
0.0083	2.0	94268	0.0421	18885616
0.0141	2.5	117835	0.0430	23594480
0.0028	3.0	141402	0.0553	28322288
0.0069	3.5	164969	0.0423	33044192
0.1483	4.0	188536	0.0451	37765424
0.07	4.5	212103	0.0431	42484624
0.0074	5.0	235670	0.0649	47208432
0.0032	5.5	259237	0.0495	51927872
0.1186	6.0	282804	0.0562	56653952
0.0011	6.5	306371	0.0570	61379616
0.0028	7.0	329938	0.0591	66100352
0.0825	7.5	353505	0.0679	70821872
0.0006	8.0	377072	0.0649	75542800
0.0001	8.5	400639	0.0915	80264848
0.0	9.0	424206	0.0899	84986304
0.0001	9.5	447773	0.1052	89703232
0.0	10.0	471340	0.1079	94426336

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1