train_boolq_123_1762583754

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 1.0732
• Num Input Tokens Seen: 37859408

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4523	2.0	3772	0.3295	3786792
0.3608	4.0	7544	0.3171	7579824
0.2307	6.0	11316	0.3416	11360672
0.2267	8.0	15088	0.3523	15150888
0.1659	10.0	18860	0.4134	18940064
0.2084	12.0	22632	0.5529	22734144
0.1017	14.0	26404	0.7290	26510480
0.0012	16.0	30176	0.9381	30297384
0.0006	18.0	33948	1.0485	34080816
0.0009	20.0	37720	1.0732	37859408

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1