---
language:
- en
- de
- fr
- it
license: apache-2.0
base_model: swiss-ai/Apertus-8B-Instruct-2509
tags:
- quantization
- fp8
- llm-compressor
- text-generation
- vllm
library_name: transformers
---

# Apertus-8B-Instruct-2509-FP8-Dynamic

This is an FP8 dynamically quantized version of [swiss-ai/Apertus-8B-Instruct-2509](https://huggingface.co/swiss-ai/Apertus-8B-Instruct-2509) using [llm-compressor](https://github.com/vllm-project/llm-compressor).

## Quantization Details

- **Quantization Scheme**: FP8_dynamic
- **Method**: Dynamic quantization of weights and activations to FP8 format
- **Targets**: All Linear layers
- **Ignored Layers**: `lm_head` (kept in higher precision for better output quality)
- **Tool**: llm-compressor (Neural Magic)