train_math_qa_1756735780

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.8016
• Num Input Tokens Seen: 35721744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.819	0.5000	6714	0.8030	1788512
0.7883	1.0001	13428	0.8025	3570752
0.7934	1.5001	20142	0.8039	5352896
0.798	2.0001	26856	0.8023	7143432
0.7671	2.5002	33570	0.8028	8930936
0.7541	3.0002	40284	0.8044	10716624
0.8174	3.5003	46998	0.8037	12503440
0.7763	4.0003	53712	0.8060	14291264
0.7698	4.5003	60426	0.8022	16073776
0.8005	5.0004	67140	0.8015	17863392
0.7941	5.5004	73854	0.8022	19647376
0.8181	6.0004	80568	0.8027	21435032
0.7893	6.5005	87282	0.8030	23219624
0.7838	7.0005	93996	0.8023	25008304
0.7948	7.5006	100710	0.8028	26793376
0.7865	8.0006	107424	0.8009	28582664
0.8228	8.5006	114138	0.8016	30372824
0.7991	9.0007	120852	0.8013	32154488
0.7905	9.5007	127566	0.8015	33936616

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1