---
library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
- llama-factory
- prefix-tuning
- generated_from_trainer
model-index:
- name: train_boolq_1756729605
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# train_boolq_1756729605

This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on the boolq dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3326
- Num Input Tokens Seen: 18160480

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 123
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Input Tokens Seen |
|:-------------:|:-----:|:-----:|:---------------:|:-----------------:|
| 0.4614        | 0.5   | 2121  | 0.3305          | 910896            |
| 0.2867        | 1.0   | 4242  | 0.3275          | 1820384           |
| 0.3369        | 1.5   | 6363  | 0.3241          | 2725088           |
| 0.2545        | 2.0   | 8484  | 0.3280          | 3635024           |
| 0.2875        | 2.5   | 10605 | 0.3283          | 4551376           |
| 0.3385        | 3.0   | 12726 | 0.3347          | 5454288           |
| 0.3121        | 3.5   | 14847 | 0.3290          | 6365376           |
| 0.3326        | 4.0   | 16968 | 0.3312          | 7267408           |
| 0.3828        | 4.5   | 19089 | 0.3279          | 8180384           |
| 0.3116        | 5.0   | 21210 | 0.3268          | 9086272           |
| 0.4007        | 5.5   | 23331 | 0.3321          | 9998416           |
| 0.3262        | 6.0   | 25452 | 0.3285          | 10904352          |
| 0.2873        | 6.5   | 27573 | 0.3270          | 11814320          |
| 0.2122        | 7.0   | 29694 | 0.3293          | 12718592          |
| 0.3031        | 7.5   | 31815 | 0.3272          | 13626928          |
| 0.3253        | 8.0   | 33936 | 0.3276          | 14536416          |
| 0.2943        | 8.5   | 36057 | 0.3289          | 15443568          |
| 0.3488        | 9.0   | 38178 | 0.3365          | 16348016          |
| 0.3448        | 9.5   | 40299 | 0.3319          | 17255920          |
| 0.3665        | 10.0  | 42420 | 0.3326          | 18160480          |


### Framework versions

- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.8.0+cu128
- Datasets 3.6.0
- Tokenizers 0.21.1