train_multirc_123_1765085968

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1493
• Num Input Tokens Seen: 264547520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2291	1.0	6130	0.1683	13255424
0.1362	2.0	12260	0.1537	26471216
0.1127	3.0	18390	0.1624	39694112
0.006	4.0	24520	0.1493	52929744
0.1242	5.0	30650	0.1528	66152480
0.0787	6.0	36780	0.1757	79389648
0.0334	7.0	42910	0.1998	92621824
0.0037	8.0	49040	0.1987	105830544
0.0024	9.0	55170	0.2279	119047920
0.0016	10.0	61300	0.2657	132272272
0.0802	11.0	67430	0.2837	145487264
0.1503	12.0	73560	0.3020	158737232
0.0007	13.0	79690	0.3717	171979232
0.0002	14.0	85820	0.3937	185199728
0.0005	15.0	91950	0.4193	198426688
0.0002	16.0	98080	0.4258	211640976
0.0001	17.0	104210	0.4539	224870720
0.0005	18.0	110340	0.4676	238102672
0.0001	19.0	116470	0.4672	251320768
0.0001	20.0	122600	0.4679	264547520

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1