train_multirc_42_1762157867

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 1.3516
• Num Input Tokens Seen: 235236032

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3043	2.0	10898	0.3264	23532080
0.2302	4.0	21796	0.2811	47052688
0.3924	6.0	32694	0.2795	70559552
0.5919	8.0	43592	0.4028	94090912
0.2932	10.0	54490	0.4974	117581632
0.1446	12.0	65388	0.7075	141126560
0.0681	14.0	76286	0.9348	164667088
0.0119	16.0	87184	1.1813	188204656
0.0001	18.0	98082	1.3180	211741792
0.0	20.0	108980	1.3516	235236032

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1