train_boolq_123_1762607523

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1726
• Num Input Tokens Seen: 42678144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0239	1.0	2121	0.1726	2131904
0.0891	2.0	4242	0.2051	4264768
0.3352	3.0	6363	0.1879	6404896
0.1306	4.0	8484	0.2322	8537088
0.0016	5.0	10605	0.2283	10677088
0.0041	6.0	12726	0.3797	12814080
0.0	7.0	14847	0.3552	14950432
0.0	8.0	16968	0.3507	17082336
0.1038	9.0	19089	0.3142	19211360
0.0	10.0	21210	0.4128	21342336
0.0007	11.0	23331	0.3758	23472352
0.0	12.0	25452	0.5214	25602144
0.0	13.0	27573	0.6250	27739072
0.0	14.0	29694	0.6645	29880544
0.0	15.0	31815	0.6878	32013760
0.0	16.0	33936	0.7089	34138272
0.0	17.0	36057	0.7214	36269152
0.0	18.0	38178	0.7352	38408800
0.0	19.0	40299	0.7364	40541312
0.0	20.0	42420	0.7314	42678144

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1