train_svamp_1757340196

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.2017
• Num Input Tokens Seen: 1351952

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4684	1.0	315	0.4826	67680
0.0372	2.0	630	0.1443	135360
0.04	3.0	945	0.0852	203040
0.0116	4.0	1260	0.0968	270576
0.0086	5.0	1575	0.1172	338192
0.0211	6.0	1890	0.1189	405888
0.0025	7.0	2205	0.1951	473488
0.0033	8.0	2520	0.1472	541008
0.0583	9.0	2835	0.1560	608624
0.0	10.0	3150	0.1880	676320
0.0	11.0	3465	0.1802	743888
0.0001	12.0	3780	0.1756	811536
0.0	13.0	4095	0.2002	879168
0.0	14.0	4410	0.2022	946656
0.0	15.0	4725	0.2031	1014256
0.0	16.0	5040	0.2034	1081760
0.0	17.0	5355	0.2035	1149200
0.0	18.0	5670	0.2041	1216832
0.0	19.0	5985	0.2039	1284336
0.0	20.0	6300	0.2017	1351952

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1