train_codealpacapy_1754507519

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.5095
• Num Input Tokens Seen: 12472912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.2063	0.5	954	0.8954	616992
0.6751	1.0	1908	0.6183	1248304
0.6012	1.5	2862	0.5800	1877040
0.5251	2.0	3816	0.5593	2497016
0.6714	2.5	4770	0.5456	3129368
0.479	3.0	5724	0.5357	3742552
0.5164	3.5	6678	0.5292	4361944
0.466	4.0	7632	0.5243	4985200
0.5503	4.5	8586	0.5204	5611760
0.5104	5.0	9540	0.5176	6233920
0.4934	5.5	10494	0.5154	6849184
0.5247	6.0	11448	0.5138	7478504
0.3868	6.5	12402	0.5123	8083560
0.4867	7.0	13356	0.5115	8722744
0.4567	7.5	14310	0.5105	9345976
0.8155	8.0	15264	0.5102	9977520
0.278	8.5	16218	0.5096	10604656
0.5267	9.0	17172	0.5098	11225416
0.4248	9.5	18126	0.5096	11845704
0.6968	10.0	19080	0.5095	12472912

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1