train_boolq_456_1765365932

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1894
• Num Input Tokens Seen: 42758400

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2547	1.0	2121	0.2117	2134176
0.0063	2.0	4242	0.1894	4262848
0.2342	3.0	6363	0.1965	6393376
0.1502	4.0	8484	0.2232	8527168
0.0005	5.0	10605	0.3246	10663616
0.0001	6.0	12726	0.4844	12804064
0.0006	7.0	14847	0.4083	14945088
0.0005	8.0	16968	0.4946	17092000
0.0001	9.0	19089	0.4690	19221568
0.0589	10.0	21210	0.4041	21363392
0.0001	11.0	23331	0.5136	23510432
0.0	12.0	25452	0.5234	25647872
0.0	13.0	27573	0.5372	27792544
0.0	14.0	29694	0.6022	29926112
0.0	15.0	31815	0.7163	32059584
0.0	16.0	33936	0.7540	34199392
0.0	17.0	36057	0.7693	36344896
0.0	18.0	38178	0.7853	38482112
0.0	19.0	40299	0.7943	40624896
0.0	20.0	42420	0.7922	42758400

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1