train_piqa_123_1762657736

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2335
• Num Input Tokens Seen: 44193480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2309	1.0	3626	0.2313	2216600
0.2301	2.0	7252	0.2324	4419000
0.2325	3.0	10878	0.2316	6628280
0.2325	4.0	14504	0.2313	8844408
0.2315	5.0	18130	0.2315	11048200
0.2289	6.0	21756	0.2316	13257624
0.2334	7.0	25382	0.2314	15468632
0.2302	8.0	29008	0.2315	17678024
0.2308	9.0	32634	0.2315	19894712
0.2312	10.0	36260	0.2318	22103448
0.239	11.0	39886	0.2323	24314040
0.2355	12.0	43512	0.2325	26522184
0.2098	13.0	47138	0.2338	28731152
0.2307	14.0	50764	0.2360	30934032
0.2381	15.0	54390	0.2360	33147696
0.2294	16.0	58016	0.2366	35360272
0.2106	17.0	61642	0.2378	37574896
0.2123	18.0	65268	0.2382	39772600
0.1992	19.0	68894	0.2387	41981688
0.2025	20.0	72520	0.2389	44193480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1