train_record_123_1762842639

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.2596
• Num Input Tokens Seen: 928969984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2409	1.0	31242	0.3039	46454112
0.211	2.0	62484	0.2902	92908288
0.2231	3.0	93726	0.2760	139351808
0.3213	4.0	124968	0.2724	185790304
0.2175	5.0	156210	0.2702	232243968
0.2327	6.0	187452	0.2654	278686752
0.1622	7.0	218694	0.2658	325137568
0.1921	8.0	249936	0.2618	371592704
0.2513	9.0	281178	0.2605	418033696
0.3235	10.0	312420	0.2605	464483424
0.2183	11.0	343662	0.2606	510926720
0.1613	12.0	374904	0.2596	557369088
0.2271	13.0	406146	0.2637	603816992
0.2063	14.0	437388	0.2668	650269248
0.1615	15.0	468630	0.2642	696727936
0.2825	16.0	499872	0.2652	743174112
0.2106	17.0	531114	0.2652	789614720
0.1414	18.0	562356	0.2654	836057280
0.1918	19.0	593598	0.2651	882504192
0.1534	20.0	624840	0.2652	928969984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1