train_conala_1756729619

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the conala dataset. It achieves the following results on the evaluation set:

• Loss: 1.2345
• Num Input Tokens Seen: 1382584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9926	0.5005	536	0.8520	68880
1.0361	1.0009	1072	0.7261	138320
0.6059	1.5014	1608	0.6730	207744
0.4097	2.0019	2144	0.6240	276856
0.6925	2.5023	2680	0.6522	346040
0.5218	3.0028	3216	0.6635	415184
0.3896	3.5033	3752	0.6632	484576
0.1977	4.0037	4288	0.6992	553632
0.2292	4.5042	4824	0.7413	623280
0.2427	5.0047	5360	0.7145	691912
0.2383	5.5051	5896	0.8535	762008
0.1354	6.0056	6432	0.8560	830744
0.0319	6.5061	6968	0.9736	900568
0.0472	7.0065	7504	0.9691	969200
0.1088	7.5070	8040	1.0775	1037856
0.0452	8.0075	8576	1.0524	1107480
0.1948	8.5079	9112	1.1915	1176200
0.0369	9.0084	9648	1.1851	1245744
0.0468	9.5089	10184	1.2355	1314112

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1