train_piqa_123_1762645765

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2314
• Num Input Tokens Seen: 44193480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2308	1.0	3626	0.2315	2216600
0.2307	2.0	7252	0.2318	4419000
0.2278	3.0	10878	0.2315	6628280
0.2315	4.0	14504	0.2315	8844408
0.2319	5.0	18130	0.2317	11048200
0.2299	6.0	21756	0.2315	13257624
0.233	7.0	25382	0.2316	15468632
0.2309	8.0	29008	0.2315	17678024
0.233	9.0	32634	0.2316	19894712
0.2288	10.0	36260	0.2314	22103448
0.2314	11.0	39886	0.2314	24314040
0.2293	12.0	43512	0.2317	26522184
0.2293	13.0	47138	0.2316	28731152
0.231	14.0	50764	0.2319	30934032
0.2313	15.0	54390	0.2316	33147696
0.2329	16.0	58016	0.2317	35360272
0.2288	17.0	61642	0.2316	37574896
0.2345	18.0	65268	0.2315	39772600
0.2319	19.0	68894	0.2317	41981688
0.2298	20.0	72520	0.2319	44193480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1