metadata
library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
- llama-factory
- prefix-tuning
- generated_from_trainer
model-index:
- name: train_copa_1757340203
results: []
train_copa_1757340203
This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:
- Loss: 1.0366
- Num Input Tokens Seen: 547136
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 123
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
|---|---|---|---|---|
| 0.2307 | 1.0 | 180 | 0.2443 | 27408 |
| 0.2399 | 2.0 | 360 | 0.2335 | 54752 |
| 0.2437 | 3.0 | 540 | 0.2357 | 82176 |
| 0.2275 | 4.0 | 720 | 0.2330 | 109584 |
| 0.2341 | 5.0 | 900 | 0.2357 | 137008 |
| 0.236 | 6.0 | 1080 | 0.2312 | 164336 |
| 0.2103 | 7.0 | 1260 | 0.2379 | 191712 |
| 0.24 | 8.0 | 1440 | 0.2418 | 219072 |
| 0.231 | 9.0 | 1620 | 0.2355 | 246416 |
| 0.2325 | 10.0 | 1800 | 0.2340 | 273712 |
| 0.2153 | 11.0 | 1980 | 0.2357 | 301088 |
| 0.3359 | 12.0 | 2160 | 0.3374 | 328384 |
| 0.2397 | 13.0 | 2340 | 0.2423 | 355760 |
| 0.1748 | 14.0 | 2520 | 0.2951 | 383088 |
| 0.0885 | 15.0 | 2700 | 0.3874 | 410448 |
| 0.1848 | 16.0 | 2880 | 0.6436 | 437776 |
| 0.0092 | 17.0 | 3060 | 0.8915 | 465168 |
| 0.0543 | 18.0 | 3240 | 1.0006 | 492560 |
| 0.1114 | 19.0 | 3420 | 1.0362 | 519840 |
| 0.0002 | 20.0 | 3600 | 1.0366 | 547136 |
Framework versions
- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.8.0+cu128
- Datasets 3.6.0
- Tokenizers 0.21.1