train_mrpc_101112_1760638020

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.0804
• Num Input Tokens Seen: 6767120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1407	1.0	826	0.1059	339144
0.134	2.0	1652	0.1024	677000
0.0748	3.0	2478	0.0804	1015816
0.0826	4.0	3304	0.1039	1354072
0.0818	5.0	4130	0.1681	1692608
0.1482	6.0	4956	0.1787	2030416
0.0	7.0	5782	0.2629	2369136
0.0002	8.0	6608	0.2578	2707920
0.0	9.0	7434	0.2887	3046392
0.0	10.0	8260	0.3122	3383592
0.0	11.0	9086	0.3385	3722064
0.0	12.0	9912	0.3629	4060512
0.0	13.0	10738	0.3364	4398224
0.0	14.0	11564	0.2845	4737184
0.0	15.0	12390	0.4198	5075136
0.0	16.0	13216	0.4235	5413384
0.0	17.0	14042	0.4661	5751600
0.0	18.0	14868	0.4560	6090384
0.0	19.0	15694	0.4506	6429416
0.0	20.0	16520	0.4480	6767120

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4