train_wsc_456_1760444955

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3601
• Num Input Tokens Seen: 1457072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3505	1.504	188	0.3326	73040
0.3062	3.008	376	0.3693	145504
0.3547	4.5120	564	0.3735	219728
0.3581	6.016	752	0.3446	291856
0.3482	7.52	940	0.3529	364400
0.3531	9.024	1128	0.3424	437840
0.3596	10.528	1316	0.3497	510976
0.351	12.032	1504	0.3582	583328
0.3457	13.536	1692	0.3369	655600
0.3582	15.04	1880	0.3661	728944
0.3566	16.544	2068	0.3375	801728
0.3477	18.048	2256	0.3403	875104
0.3511	19.552	2444	0.3503	948912
0.3286	21.056	2632	0.3433	1021088
0.3464	22.56	2820	0.3556	1093760
0.35	24.064	3008	0.3474	1167376
0.3229	25.568	3196	0.3539	1241056
0.3347	27.072	3384	0.3573	1314832
0.3673	28.576	3572	0.3585	1388480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1