train_record_42_1761754887

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3135
• Num Input Tokens Seen: 929295680

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3681	1.0	31242	0.4127	46467680
0.4585	2.0	62484	0.3429	92922208
0.2527	3.0	93726	0.3168	139391296
0.2222	4.0	124968	0.3135	185843712
0.2186	5.0	156210	0.3138	232305472
0.2817	6.0	187452	0.3141	278763136
0.1354	7.0	218694	0.3186	325229440
0.3922	8.0	249936	0.3250	371696128
0.263	9.0	281178	0.3322	418155040
0.182	10.0	312420	0.3449	464625760
0.1504	11.0	343662	0.3650	511102336
0.1375	12.0	374904	0.3647	557571936
0.2382	13.0	406146	0.3736	604043392
0.1674	14.0	437388	0.3868	650494080
0.2125	15.0	468630	0.3935	696961792
0.1693	16.0	499872	0.4071	743437824
0.179	17.0	531114	0.4132	789906944
0.1036	18.0	562356	0.4161	836372000
0.163	19.0	593598	0.4194	882831072
0.1442	20.0	624840	0.4193	929295680

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1