train_mrpc_1755694491

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.5279
• Num Input Tokens Seen: 3186272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2627	0.5003	826	0.2115	159824
0.1815	1.0006	1652	0.1709	318992
0.188	1.5009	2478	0.2088	478672
0.1187	2.0012	3304	0.1502	637952
0.2891	2.5015	4130	0.1866	795872
0.0183	3.0018	4956	0.1473	956728
0.2041	3.5021	5782	0.2004	1116152
0.1307	4.0024	6608	0.1742	1275352
0.041	4.5027	7434	0.1583	1434648
0.2473	5.0030	8260	0.1349	1593600
0.0148	5.5033	9086	0.1791	1752592
0.2325	6.0036	9912	0.1774	1912280
0.0005	6.5039	10738	0.2380	2072056
0.0948	7.0042	11564	0.2457	2231152
0.0001	7.5045	12390	0.3442	2390560
0.0003	8.0048	13216	0.3302	2550320
0.0377	8.5051	14042	0.4339	2709536
0.0	9.0055	14868	0.4908	2869056
0.0	9.5058	15694	0.5287	3028464

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1