train_piqa_123_1762687058

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1131
• Num Input Tokens Seen: 44193480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0947	1.0	3626	0.1277	2216600
0.0548	2.0	7252	0.1158	4419000
0.11	3.0	10878	0.1131	6628280
0.0944	4.0	14504	0.1137	8844408
0.0336	5.0	18130	0.1201	11048200
0.0045	6.0	21756	0.1497	13257624
0.1283	7.0	25382	0.1657	15468632
0.0497	8.0	29008	0.2049	17678024
0.0006	9.0	32634	0.2118	19894712
0.0003	10.0	36260	0.2350	22103448
0.0001	11.0	39886	0.2672	24314040
0.0001	12.0	43512	0.3277	26522184
0.0001	13.0	47138	0.4018	28731152
0.0002	14.0	50764	0.4293	30934032
0.0	15.0	54390	0.4618	33147696
0.0	16.0	58016	0.5101	35360272
0.0	17.0	61642	0.5407	37574896
0.0	18.0	65268	0.5613	39772600
0.0162	19.0	68894	0.5676	41981688
0.0001	20.0	72520	0.5704	44193480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1