train_mrpc_101112_1760638023

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1057
• Num Input Tokens Seen: 6767120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2113	1.0	826	0.1636	339144
0.1104	2.0	1652	0.1466	677000
0.1125	3.0	2478	0.1352	1015816
0.1356	4.0	3304	0.1293	1354072
0.1318	5.0	4130	0.1245	1692608
0.1543	6.0	4956	0.1183	2030416
0.1004	7.0	5782	0.1145	2369136
0.1011	8.0	6608	0.1124	2707920
0.132	9.0	7434	0.1109	3046392
0.1628	10.0	8260	0.1099	3383592
0.1131	11.0	9086	0.1080	3722064
0.0903	12.0	9912	0.1082	4060512
0.1352	13.0	10738	0.1082	4398224
0.0789	14.0	11564	0.1077	4737184
0.0663	15.0	12390	0.1061	5075136
0.1096	16.0	13216	0.1057	5413384
0.1921	17.0	14042	0.1068	5751600
0.0834	18.0	14868	0.1066	6090384
0.0692	19.0	15694	0.1068	6429416
0.0673	20.0	16520	0.1069	6767120

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4