train_gsm8k_1754652178

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the gsm8k dataset. It achieves the following results on the evaluation set:

• Loss: 3.5647
• Num Input Tokens Seen: 17277648

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
6.8827	0.5	841	6.9697	865376
5.83	1.0	1682	5.7327	1731768
5.0396	1.5	2523	5.1637	2596664
4.7207	2.0	3364	4.8774	3464008
4.7156	2.5	4205	4.7102	4329160
4.4299	3.0	5046	4.5935	5197240
4.3705	3.5	5887	4.4639	6061624
4.4118	4.0	6728	4.3314	6920632
4.162	4.5	7569	4.2099	7784408
3.9714	5.0	8410	4.0998	8646936
4.0372	5.5	9251	3.9920	9505560
3.7467	6.0	10092	3.8812	10374192
3.8546	6.5	10933	3.7810	11237008
3.9448	7.0	11774	3.7024	12101200
3.5379	7.5	12615	3.6452	12959728
3.5817	8.0	13456	3.6064	13828800
3.7709	8.5	14297	3.5823	14696832
3.6574	9.0	15138	3.5699	15552184
3.4324	9.5	15979	3.5658	16413528
3.7105	10.0	16820	3.5647	17277648

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1