train_cola_1757340237

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1651
• Num Input Tokens Seen: 3663512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0727	0.5	962	0.2056	182656
0.28	1.0	1924	0.1750	365728
0.2319	1.5	2886	0.2058	548992
0.1678	2.0	3848	0.1835	731984
0.068	2.5	4810	0.2135	915792
0.4099	3.0	5772	0.1894	1098920
0.043	3.5	6734	0.1944	1281640
0.0726	4.0	7696	0.1651	1465464
0.1162	4.5	8658	0.1846	1649720
0.0194	5.0	9620	0.1789	1831920
0.0803	5.5	10582	0.1859	2014928
0.2613	6.0	11544	0.1869	2198176
0.1435	6.5	12506	0.1877	2381440
0.1227	7.0	13468	0.1890	2564952
0.288	7.5	14430	0.1912	2748568
0.2387	8.0	15392	0.1974	2931096
0.0504	8.5	16354	0.1940	3113624
0.0763	9.0	17316	0.1961	3296808
0.0386	9.5	18278	0.1963	3480168
0.094	10.0	19240	0.1969	3663512

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1