train_boolq_123_1762589801

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3213
• Num Input Tokens Seen: 42678144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2872	1.0	2121	0.3391	2131904
0.3153	2.0	4242	0.3294	4264768
0.3251	3.0	6363	0.3346	6404896
0.3535	4.0	8484	0.3308	8537088
0.3267	5.0	10605	0.3278	10677088
0.3159	6.0	12726	0.3298	12814080
0.2604	7.0	14847	0.3271	14950432
0.3271	8.0	16968	0.3267	17082336
0.3257	9.0	19089	0.3424	19211360
0.3441	10.0	21210	0.3264	21342336
0.3417	11.0	23331	0.3277	23472352
0.3086	12.0	25452	0.3271	25602144
0.3027	13.0	27573	0.3251	27739072
0.3239	14.0	29694	0.3250	29880544
0.3697	15.0	31815	0.3236	32013760
0.3328	16.0	33936	0.3237	34138272
0.3275	17.0	36057	0.3222	36269152
0.3746	18.0	38178	0.3221	38408800
0.339	19.0	40299	0.3219	40541312
0.2792	20.0	42420	0.3213	42678144

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1