train_wsc_42_1760619133

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.4196
• Num Input Tokens Seen: 1308280

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4599	3.0	333	0.4038	130808
0.3618	6.0	666	0.3551	260688
0.3542	9.0	999	0.3522	390936
0.3678	12.0	1332	0.3531	521840
0.337	15.0	1665	0.3619	653216
0.3321	18.0	1998	0.3597	784328
0.3729	21.0	2331	0.3777	916192
0.315	24.0	2664	0.3979	1046952
0.3353	27.0	2997	0.4137	1177504
0.3193	30.0	3330	0.4196	1308280

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1