train_rte_1755694492

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.6713
• Num Input Tokens Seen: 2923240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1576	0.5004	561	0.1624	148000
0.1804	1.0009	1122	0.1550	292608
0.1808	1.5013	1683	0.1861	440304
0.1418	2.0018	2244	0.1839	586640
0.2013	2.5022	2805	0.1415	733968
0.1761	3.0027	3366	0.1492	879160
0.1022	3.5031	3927	0.1411	1025720
0.1388	4.0036	4488	0.1517	1171832
0.1175	4.5040	5049	0.1754	1317624
0.1502	5.0045	5610	0.1731	1464496
0.1553	5.5049	6171	0.1697	1612464
0.2036	6.0054	6732	0.1878	1755968
0.0244	6.5058	7293	0.4073	1901984
0.0017	7.0062	7854	0.3555	2048856
0.0002	7.5067	8415	0.5187	2193608
0.0796	8.0071	8976	0.5269	2340704
0.0011	8.5076	9537	0.6588	2486032
0.0001	9.0080	10098	0.6575	2632408
0.0001	9.5085	10659	0.6688	2780920

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1