train_record_456_1765807514

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 5.3341
• Num Input Tokens Seen: 928892640

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2169	1.0	31242	0.3239	46454656
0.2773	2.0	62484	0.3012	92898208
0.2417	3.0	93726	0.2848	139330944
0.4124	4.0	124968	0.2795	185787424
0.3997	5.0	156210	0.2763	232232736
0.2357	6.0	187452	0.2746	278675168
0.2194	7.0	218694	0.2725	325124320
0.235	8.0	249936	0.2715	371565312
0.2567	9.0	281178	0.2694	418010016
0.1701	10.0	312420	0.2680	464454880
0.186	11.0	343662	0.2702	510906784
0.1366	12.0	374904	0.2738	557340128
0.1861	13.0	406146	0.2771	603790528
0.1506	14.0	437388	0.2804	650253184
0.2098	15.0	468630	0.2858	696691296
0.1228	16.0	499872	0.2906	743122464
0.1218	17.0	531114	0.2921	789557088
0.1628	18.0	562356	0.2962	835994816
0.1751	19.0	593598	0.2975	882444928
0.2896	20.0	624840	0.2984	928892640

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1