train_qqp_1756729596

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qqp dataset. It achieves the following results on the evaluation set:

• Loss: 0.2117
• Num Input Tokens Seen: 227659432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1194	0.5000	81866	0.2017	11386496
0.3966	1.0000	163732	0.1633	22764472
0.0843	1.5000	245598	0.0992	34144408
0.2298	2.0000	327464	0.2374	45529424
0.3428	2.5000	409330	0.2306	56915424
0.28	3.0000	491196	0.2266	68299488
0.2561	3.5000	573062	0.2308	79670992
0.1879	4.0000	654928	0.2167	91066456
0.2034	4.5000	736794	0.2223	102449336
0.2342	5.0000	818660	0.2082	113829176
0.2252	5.5000	900526	0.2078	125219848
0.1813	6.0000	982392	0.2041	136600616
0.2893	6.5000	1064258	0.2011	147981640
0.1523	7.0000	1146124	0.2053	159365688
0.1371	7.5000	1227990	0.2020	170758584
0.1622	8.0000	1309856	0.2011	182133096
0.1149	8.5001	1391722	0.2092	193504584
0.2538	9.0001	1473588	0.2079	204895744
0.2284	9.5001	1555454	0.2120	216280368

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1