train_record_123_1764488339

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3025
• Num Input Tokens Seen: 928969984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3783	1.0	31242	0.4132	46454112
0.1702	2.0	62484	0.3343	92908288
0.2465	3.0	93726	0.3134	139351808
0.2857	4.0	124968	0.3068	185790304
0.2238	5.0	156210	0.3049	232243968
0.3226	6.0	187452	0.3025	278686752
0.1501	7.0	218694	0.3084	325137568
0.1268	8.0	249936	0.3293	371592704
0.153	9.0	281178	0.3367	418033696
0.3234	10.0	312420	0.3395	464483424
0.1716	11.0	343662	0.3537	510926720
0.132	12.0	374904	0.3737	557369088
0.1981	13.0	406146	0.3715	603816992
0.1742	14.0	437388	0.3835	650269248
0.1907	15.0	468630	0.3917	696727936
0.2175	16.0	499872	0.3995	743174112
0.2167	17.0	531114	0.4046	789614720
0.0844	18.0	562356	0.4149	836057280
0.1003	19.0	593598	0.4158	882504192
0.1092	20.0	624840	0.4152	928969984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1