---
quantized_by: ArtusDev
pipeline_tag: text-generation
base_model: ToastyPigeon/Gemma-3-Starshine-12B
license: gemma
base_model_relation: quantized
language:
- en
tags:
- imatrix
- gemma3_text
---
# Quantization

This repository contains GGUF format model files converted from [`ToastyPigeon/Gemma-3-Starshine-12B`](https://huggingface.co/ToastyPigeon/Gemma-3-Starshine-12B).

The conversion was performed by **ArtusDev** using `llama.cpp`, specifically utilizing the `imatrix` quantization option for potentially improved performance.

# 🌠G3 Starshine 12B🌠
<figure>
  <img src="https://huggingface.co/ToastyPigeon/Gemma-3-Starshine-12B/resolve/main/modelcard_image.jpeg" width="600">
</figure>

*This was Merge A / A1 in the testing set.* 

A creative writing model based on a merge of fine-tunes on Gemma 3 12B IT and Gemma 3 12B PT. 

This is the **Story Focused** merge. This version works better for storytelling and scenarios, as the prose is more novel-like and it has a tendency to impersonate the user character. 

See the [Alternate RP Focused](https://huggingface.co/ToastyPigeon/Gemma-3-Starshine-12B-Alt/) version as well. 

This is a merge of two G3 models, one trained on instruct and one trained on base: 
* [allura-org/Gemma-3-Glitter-12B](https://huggingface.co/allura-org/Gemma-3-Glitter-12B) - Itself a merge of a storywriting and RP train (both also by ToastyPigeon), on instruct
* [ToastyPigeon/Gemma-3-Confetti-12B](https://huggingface.co/ToastyPigeon/Gemma-3-Confetti-12B) - Experimental application of the Glitter data using base instead of instruct, additionally includes some adventure data in the form of SpringDragon.
  
The result is a lovely blend of Glitter's ability to follow instructions and Confetti's free-spirit prose, effectively 'loosening up' much of the hesitancy that was left in Glitter. 

Vision works (as well as any vision works with this model right now) if you pair a GGUF of this with an appropriate mmproj file; I intend to fix the missing vision tower + make this properly multimodal in the near future. 

*Thank you to [jebcarter](https://huggingface.co/jebcarter) for the idea to make this. I love how it turned out!*

## Instruct Format

Uses Gemma2/3 instruct, but has been trained to recognize an optional system role.

*Note: While it won't immediately balk at the system role, results may be better without it.*

```
<start_of_turn>system
{optional system turn with prompt}<end_of_turn>
<start_of_turn>user
{User messages; can also put sysprompt here to use the built-in g3 training}<end_of_turn>
<start_of_turn>model
{model response}<end_of_turn>
```

### Merge Configuration

Yeah, I actually tried several things and surprisingly this one worked best. 

```yaml
models:
  - model: ToastyPigeon/Gemma-3-Confetti-12B
    parameters:
      weight: 0.5
  - model: allura-org/Gemma-3-Glitter-12B
    parameters:
      weight: 0.5
merge_method: linear
tokenizer_source: allura-org/Gemma-3-Glitter-12B
```