train_svamp_42_1763630700

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0889
• Num Input Tokens Seen: 686560

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.4882	0.5016	158	1.5093	34720
0.2632	1.0032	316	0.2166	68736
0.1855	1.5048	474	0.1328	103216
0.1355	2.0063	632	0.1132	137648
0.0067	2.5079	790	0.0988	171744
0.0576	3.0095	948	0.0944	206608
0.1071	3.5111	1106	0.0933	240992
0.1266	4.0127	1264	0.0889	275328
0.0368	4.5143	1422	0.0901	309648
0.0377	5.0159	1580	0.0897	344256
0.0464	5.5175	1738	0.0963	379024
0.0265	6.0190	1896	0.0922	413280
0.0486	6.5206	2054	0.1014	447504
0.0311	7.0222	2212	0.1011	482256
0.006	7.5238	2370	0.1041	516448
0.0358	8.0254	2528	0.1052	550928
0.0837	8.5270	2686	0.1081	585568
0.0311	9.0286	2844	0.1078	619904
0.0017	9.5302	3002	0.1079	654176

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4