train_record_123_1764257419

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.2694
• Num Input Tokens Seen: 928969984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2609	1.0	31242	0.2989	46454112
0.1509	2.0	62484	0.2694	92908288
0.1567	3.0	93726	0.2741	139351808
0.2178	4.0	124968	0.3017	185790304
0.1202	5.0	156210	0.3430	232243968
0.2103	6.0	187452	0.3672	278686752
0.1165	7.0	218694	0.4037	325137568
0.086	8.0	249936	0.3610	371592704
0.0968	9.0	281178	0.4369	418033696
0.1659	10.0	312420	0.4564	464483424
0.1398	11.0	343662	0.5060	510926720
0.1038	12.0	374904	0.5675	557369088
0.0755	13.0	406146	0.5769	603816992
0.1142	14.0	437388	0.6175	650269248
0.1084	15.0	468630	0.7010	696727936
0.0998	16.0	499872	0.8002	743174112
0.1507	17.0	531114	0.8937	789614720
0.0689	18.0	562356	0.9858	836057280
0.0825	19.0	593598	1.1350	882504192
0.0948	20.0	624840	1.1639	928969984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1