train_hellaswag_101112_1760638084

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.6988
• Num Input Tokens Seen: 218373904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.0945	1.0	8979	1.0153	10920768
1.0336	2.0	17958	0.7676	21838176
0.4963	3.0	26937	0.7083	32764400
0.5172	4.0	35916	0.7088	43678848
0.9083	5.0	44895	0.7039	54599824
0.9356	6.0	53874	0.6988	65511392
0.6094	7.0	62853	0.7010	76440704
0.8224	8.0	71832	0.7057	87358976
0.6659	9.0	80811	0.7028	98270752
0.7143	10.0	89790	0.7002	109182912
0.5671	11.0	98769	0.7045	120100304
0.7363	12.0	107748	0.7087	131010176
0.6908	13.0	116727	0.7087	141940544
0.6356	14.0	125706	0.7087	152867360
0.749	15.0	134685	0.7087	163792736
0.695	16.0	143664	0.7087	174711344
0.9739	17.0	152643	0.7087	185629312
0.6252	18.0	161622	0.7087	196536784
0.8867	19.0	170601	0.7087	207451408
0.8269	20.0	179580	0.7087	218373904

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4