train_piqa_123_1762699637

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1047
• Num Input Tokens Seen: 44193480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1076	1.0	3626	0.1334	2216600
0.0721	2.0	7252	0.1140	4419000
0.0754	3.0	10878	0.1083	6628280
0.1419	4.0	14504	0.1050	8844408
0.1238	5.0	18130	0.1047	11048200
0.0809	6.0	21756	0.1055	13257624
0.103	7.0	25382	0.1048	15468632
0.0655	8.0	29008	0.1064	17678024
0.0958	9.0	32634	0.1064	19894712
0.0198	10.0	36260	0.1049	22103448
0.0569	11.0	39886	0.1067	24314040
0.048	12.0	43512	0.1086	26522184
0.1129	13.0	47138	0.1100	28731152
0.1454	14.0	50764	0.1107	30934032
0.0707	15.0	54390	0.1117	33147696
0.0748	16.0	58016	0.1124	35360272
0.0372	17.0	61642	0.1130	37574896
0.0309	18.0	65268	0.1124	39772600
0.1315	19.0	68894	0.1136	41981688
0.0742	20.0	72520	0.1132	44193480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1