train_boolq_456_1765377404

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1943
• Num Input Tokens Seen: 42758400

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0993	1.0	2121	0.2367	2134176
0.0802	2.0	4242	0.2049	4262848
0.3084	3.0	6363	0.1943	6393376
0.2871	4.0	8484	0.2014	8527168
0.0976	5.0	10605	0.2174	10663616
0.2931	6.0	12726	0.2334	12804064
0.1306	7.0	14847	0.2484	14945088
0.1506	8.0	16968	0.2774	17092000
0.003	9.0	19089	0.2881	19221568
0.0021	10.0	21210	0.3368	21363392
0.1047	11.0	23331	0.3346	23510432
0.0028	12.0	25452	0.3918	25647872
0.0011	13.0	27573	0.4164	27792544
0.0006	14.0	29694	0.4501	29926112
0.0736	15.0	31815	0.4886	32059584
0.0016	16.0	33936	0.5037	34199392
0.0302	17.0	36057	0.5235	36344896
0.0003	18.0	38178	0.5321	38482112
0.0001	19.0	40299	0.5280	40624896
0.0266	20.0	42420	0.5283	42758400

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1