train_siqa_1754652166

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the siqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.3209
• Num Input Tokens Seen: 29840264

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3295	0.5	3759	0.2344	1495072
0.1035	1.0	7518	0.2057	2984720
0.1792	1.5	11277	0.1993	4477104
0.2668	2.0	15036	0.1963	5970384
0.2112	2.5	18795	0.1914	7462384
0.1166	3.0	22554	0.1923	8954176
0.0182	3.5	26313	0.1974	10445088
0.1845	4.0	30072	0.1851	11937344
0.222	4.5	33831	0.1964	13430048
0.2374	5.0	37590	0.1964	14920992
0.0194	5.5	41349	0.2016	16412032
0.1128	6.0	45108	0.1903	17904680
0.1917	6.5	48867	0.2089	19397416
0.341	7.0	52626	0.2050	20888856
0.0722	7.5	56385	0.2190	22381080
0.3229	8.0	60144	0.2185	23872880
0.1923	8.5	63903	0.2270	25363344
0.1182	9.0	67662	0.2255	26855848
0.1978	9.5	71421	0.2302	28348712
0.09	10.0	75180	0.2308	29840264

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1