train_math_qa_1755694508

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.8013
• Num Input Tokens Seen: 35721744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8164	0.5000	6714	0.8034	1788512
0.809	1.0001	13428	0.8022	3570752
0.7974	1.5001	20142	0.8032	5352896
0.7931	2.0001	26856	0.8040	7143432
0.7631	2.5002	33570	0.8025	8930936
0.7596	3.0002	40284	0.8030	10716624
0.8258	3.5003	46998	0.8032	12503440
0.7852	4.0003	53712	0.8018	14291264
0.7626	4.5003	60426	0.8027	16073776
0.7895	5.0004	67140	0.8019	17863392
0.7574	5.5004	73854	0.8017	19647376
0.8196	6.0004	80568	0.8026	21435032
0.799	6.5005	87282	0.8106	23219624
0.7899	7.0005	93996	0.8036	25008304
0.8099	7.5006	100710	0.8040	26793376
0.814	8.0006	107424	0.8032	28582664
0.846	8.5006	114138	0.8028	30372824
0.8158	9.0007	120852	0.8017	32154488
0.7827	9.5007	127566	0.8014	33936616

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1