train_boolq_456_1765386945

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1715
• Num Input Tokens Seen: 42758400

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0939	1.0	2121	0.2470	2134176
0.0918	2.0	4242	0.2088	4262848
0.2719	3.0	6363	0.1910	6393376
0.2535	4.0	8484	0.1835	8527168
0.1725	5.0	10605	0.1774	10663616
0.2312	6.0	12726	0.1743	12804064
0.1922	7.0	14847	0.1715	14945088
0.2345	8.0	16968	0.1721	17092000
0.1187	9.0	19089	0.1729	19221568
0.1793	10.0	21210	0.1745	21363392
0.174	11.0	23331	0.1720	23510432
0.0468	12.0	25452	0.1736	25647872
0.0497	13.0	27573	0.1744	27792544
0.0608	14.0	29694	0.1732	29926112
0.1619	15.0	31815	0.1738	32059584
0.1059	16.0	33936	0.1752	34199392
0.116	17.0	36057	0.1756	36344896
0.0197	18.0	38178	0.1773	38482112
0.0114	19.0	40299	0.1754	40624896
0.1493	20.0	42420	0.1750	42758400

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1