train_svamp_1757340176

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0833
• Num Input Tokens Seen: 704336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.0626	0.5	79	1.9812	35680
1.1256	1.0	158	1.1086	70512
0.2612	1.5	237	0.2784	105904
0.0956	2.0	316	0.1415	140960
0.0752	2.5	395	0.1209	176096
0.0711	3.0	474	0.1086	211424
0.0336	3.5	553	0.1012	246784
0.1394	4.0	632	0.0968	281968
0.0483	4.5	711	0.0930	317232
0.0396	5.0	790	0.0909	352368
0.0635	5.5	869	0.0879	387824
0.0596	6.0	948	0.0855	422704
0.0235	6.5	1027	0.0860	457744
0.0522	7.0	1106	0.0853	493200
0.0399	7.5	1185	0.0843	528304
0.1102	8.0	1264	0.0836	563520
0.0638	8.5	1343	0.0843	599072
0.0324	9.0	1422	0.0833	634176
0.0851	9.5	1501	0.0837	669440
0.0526	10.0	1580	0.0833	704336

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1