train_copa_1757340182

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0386
• Num Input Tokens Seen: 282368

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0771	0.5	45	0.1575	14176
0.0904	1.0	90	0.1057	28256
0.1193	1.5	135	0.0855	42240
0.0732	2.0	180	0.0704	56480
0.1398	2.5	225	0.0614	70688
0.1844	3.0	270	0.0523	84736
0.01	3.5	315	0.0443	98784
0.116	4.0	360	0.0458	113024
0.0033	4.5	405	0.0412	127264
0.1303	5.0	450	0.0445	141440
0.0177	5.5	495	0.0422	155488
0.0035	6.0	540	0.0397	169600
0.0757	6.5	585	0.0419	183648
0.0091	7.0	630	0.0386	197792
0.0243	7.5	675	0.0430	211936
0.0013	8.0	720	0.0413	225984
0.0185	8.5	765	0.0391	239936
0.1051	9.0	810	0.0393	254112
0.0014	9.5	855	0.0405	268256
0.0008	10.0	900	0.0395	282368

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1