train_hellaswag_101112_1760638085

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.0788
• Num Input Tokens Seen: 218373904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1243	1.0	8979	0.1606	10920768
0.2253	2.0	17958	0.1173	21838176
0.0933	3.0	26937	0.0993	32764400
0.0239	4.0	35916	0.0916	43678848
0.0874	5.0	44895	0.0878	54599824
0.0372	6.0	53874	0.0856	65511392
0.2015	7.0	62853	0.0806	76440704
0.0822	8.0	71832	0.0804	87358976
0.0112	9.0	80811	0.0811	98270752
0.0286	10.0	89790	0.0788	109182912
0.0253	11.0	98769	0.0805	120100304
0.0921	12.0	107748	0.0824	131010176
0.1129	13.0	116727	0.0835	141940544
0.0102	14.0	125706	0.0824	152867360
0.0214	15.0	134685	0.0841	163792736
0.0069	16.0	143664	0.0843	174711344
0.0488	17.0	152643	0.0848	185629312
0.0249	18.0	161622	0.0852	196536784
0.0236	19.0	170601	0.0849	207451408
0.1324	20.0	179580	0.0852	218373904

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4