train_rte_101112_1760638016

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.0588
• Num Input Tokens Seen: 6980984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2449	1.0	561	0.1562	350480
0.1225	2.0	1122	0.0995	700992
0.0818	3.0	1683	0.0855	1050848
0.0678	4.0	2244	0.0770	1400856
0.1252	5.0	2805	0.0713	1749544
0.02	6.0	3366	0.0673	2099368
0.0831	7.0	3927	0.0641	2447504
0.048	8.0	4488	0.0638	2794592
0.0365	9.0	5049	0.0612	3145760
0.0347	10.0	5610	0.0610	3495600
0.0404	11.0	6171	0.0606	3844488
0.0366	12.0	6732	0.0598	4191800
0.027	13.0	7293	0.0592	4538416
0.0183	14.0	7854	0.0588	4888904
0.0321	15.0	8415	0.0593	5236560
0.0461	16.0	8976	0.0601	5587768
0.0446	17.0	9537	0.0588	5935088
0.0304	18.0	10098	0.0592	6283144
0.0534	19.0	10659	0.0592	6632504
0.1391	20.0	11220	0.0592	6980984

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4