train_piqa_1755545132

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2341
• Num Input Tokens Seen: 22103448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5272	0.5	1813	0.5126	1118368
0.229	1.0	3626	0.2631	2216600
0.2295	1.5	5439	0.2451	3320792
0.2232	2.0	7252	0.2398	4419000
0.2235	2.5	9065	0.2405	5525176
0.2368	3.0	10878	0.2376	6628280
0.2241	3.5	12691	0.2357	7736376
0.23	4.0	14504	0.2345	8844408
0.2303	4.5	16317	0.2359	9951832
0.2353	5.0	18130	0.2341	11048200
0.2257	5.5	19943	0.2350	12157032
0.2334	6.0	21756	0.2360	13257624
0.2314	6.5	23569	0.2350	14360952
0.2249	7.0	25382	0.2356	15468632
0.2277	7.5	27195	0.2347	16574840
0.2304	8.0	29008	0.2351	17678024
0.2312	8.5	30821	0.2350	18780040
0.2237	9.0	32634	0.2355	19894712
0.2352	9.5	34447	0.2353	21014840
0.2253	10.0	36260	0.2347	22103448

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1