train_boolq_789_1767816931

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1710
• Num Input Tokens Seen: 42723072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1051	1.0	2121	0.2393	2132416
0.1926	2.0	4242	0.2046	4255296
0.0422	3.0	6363	0.1902	6393664
0.0423	4.0	8484	0.1787	8532736
0.1779	5.0	10605	0.1728	10659936
0.1118	6.0	12726	0.1728	12794688
0.2214	7.0	14847	0.1743	14935648
0.1	8.0	16968	0.1711	17068640
0.1641	9.0	19089	0.1732	19208672
0.1169	10.0	21210	0.1710	21357280
0.1458	11.0	23331	0.1745	23492192
0.4114	12.0	25452	0.1733	25634336
0.1151	13.0	27573	0.1736	27764736
0.0163	14.0	29694	0.1749	29902592
0.1206	15.0	31815	0.1767	32041216
0.022	16.0	33936	0.1770	34178752
0.2884	17.0	36057	0.1765	36309792
0.0796	18.0	38178	0.1781	38446560
0.2409	19.0	40299	0.1783	40588096
0.0937	20.0	42420	0.1776	42723072

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1