train_piqa_456_1765483358

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0949
• Num Input Tokens Seen: 44177928

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1549	1.0	3626	0.1288	2208216
0.0624	2.0	7252	0.1103	4420664
0.0538	3.0	10878	0.1016	6629696
0.0451	4.0	14504	0.0979	8840800
0.0751	5.0	18130	0.0957	11045752
0.0352	6.0	21756	0.0952	13254840
0.06	7.0	25382	0.0952	15458512
0.1287	8.0	29008	0.0949	17666816
0.06	9.0	32634	0.0953	19878664
0.0944	10.0	36260	0.0967	22082280
0.105	11.0	39886	0.0973	24300584
0.0962	12.0	43512	0.0988	26515920
0.0905	13.0	47138	0.0989	28721912
0.0361	14.0	50764	0.1008	30927016
0.0971	15.0	54390	0.1019	33135160
0.0289	16.0	58016	0.1026	35347688
0.0206	17.0	61642	0.1031	37560560
0.0127	18.0	65268	0.1032	39771536
0.0944	19.0	68894	0.1037	41974792
0.0856	20.0	72520	0.1035	44177928

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1