train_copa_1757340203

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 1.0366
• Num Input Tokens Seen: 547136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2307	1.0	180	0.2443	27408
0.2399	2.0	360	0.2335	54752
0.2437	3.0	540	0.2357	82176
0.2275	4.0	720	0.2330	109584
0.2341	5.0	900	0.2357	137008
0.236	6.0	1080	0.2312	164336
0.2103	7.0	1260	0.2379	191712
0.24	8.0	1440	0.2418	219072
0.231	9.0	1620	0.2355	246416
0.2325	10.0	1800	0.2340	273712
0.2153	11.0	1980	0.2357	301088
0.3359	12.0	2160	0.3374	328384
0.2397	13.0	2340	0.2423	355760
0.1748	14.0	2520	0.2951	383088
0.0885	15.0	2700	0.3874	410448
0.1848	16.0	2880	0.6436	437776
0.0092	17.0	3060	0.8915	465168
0.0543	18.0	3240	1.0006	492560
0.1114	19.0	3420	1.0362	519840
0.0002	20.0	3600	1.0366	547136

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1