This repo contains the full precision source code, in "safe tensors" format to generate GGUFs, GPTQ, EXL2, AWQ, HQQ and other formats. The source code can also be used directly.
This is a CREATIVE purpose MOE - Mixture of Experts of 6 models at 0.6B (3.6B) in a Mixtral type Qwen 3 structure compressed to 2B.
Special thanks to all the model makers (see model tree) and of course TEAM Qwen!
This model exceeds 200 t/s with 2 experts activated (default) on a mid level card, 2x/3x times this on high end card with CPU only performance will be 50+ t/s.
This model excels at high speed creative tasks, but can also be used for general tasks too.
This model ESPECIALLY works well at 6 experts and temps over 1 ; and will create words on the fly.
For coding/complex tasks, strongly suggest Q8 or full precision.
Full generation at the bottom of this page at q4_k_s.
This has all the power of Qwen 3 and MOE model in one.
This version has the NATIVE context of 40k.
This is a full thinking model.
The MOE structure reduces the size of the token thinking block.
I have included an optional system prompt to invoke "thinking" in this model, if you want to activate for all use cases.
Usually the model will self-activate thinking by itself.
IMPORTANT GENERAL NOTE:
IMPORTANT NOTES:
SETTINGS:
For coding, programming set expert to:
Recommended settings - general:
OPTIONAL SYSTEM PROMPT - INVOKE "Thinking":
Enable deep thinking subroutine. You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside ###ponder### ###/ponder### tags, and then provide your solution or response to the problem.
Use this to INVOKE "thinking" block(s) in the model. These will be a lot shorter than 1000s of tokens generally in most "thinking" models.
In you use this prompt, you may need to raise "rep pen" to 1.08 to 1.1, to prevent "loops" in the "thought block(s)" ; especially in lower quants.
If you change "ponder" to a different word/phrase this will affect model "thinking" too.
QUANTS
GGUF? GGUF Imatrix? Other?
See under "model tree", upper right and click on "quantizations".
New quants will automatically appear.
CHANGE THE NUMBER OF ACTIVE EXPERTS:
See this document:
https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:
In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
Set the "Smoothing_factor" to 1.5
: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
: in text-generation-webui -> parameters -> lower right.
: In Silly Tavern this is called: "Smoothing"
NOTE: For "text-generation-webui"
-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
Source versions (and config files) of my models are here:
OTHER OPTIONS:
Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 1" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
Q4_K_S ; Temp 1.5, rep pen 1.05, top k: 40, topp .95, min p 0.5 ; LMSTudio.
SIX experts activated.
This is a mid range quant, expect better performance at Q8 / 16 bit full precision.
Imatrix versions (quants) will also exceed this quality too.
Example #1 ; SIX experts activated.
Start a 1000 word scene (vivid, graphic horror - include blood, guts and gore - in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...
[[[thinking start]]]
Thought for 4.84 seconds
Alright, let's start this. First, I need to set up a vivid scene, so I'll start with a clear, engaging opening that sets the horror tone. Diana, a young and independent character, is looking out her window. She's in her apartment on the 21st floor of a high-rise skyscraper. The building is pulsing like a live organ, and the weight of it is making her feel small and vulnerable.
Diana's first-person perspective should be natural, so I'll start with a simple, direct description of the scene. The skyscraper swaying and the window on the 21st floor will have her in a position to see and feel the situation. Now, she notices something off: the window just began to shake, and her heart starts to race. She's at a crossroads between safety and survival, so the immediate and visceral first part is building tension and the key to the scene.
Moving on to the second part, I'll focus on the actual moment when the 21st floor window begins to fall. I need to describe the physical sensations she experiences: the weight, the shattering, and the blinding. The blinding of light would be the result of the building's massive, dark glass, which she can now see in a new, vivid way.
Now, I'll go to the second part where she looks up to see her 18-year old daughter at a window table on the same floor. They are seated, smiling, and exchanging something. Then, the 21st floor starts to fall, and the glass shatters. The second person here, in the role of the 21st floor, would be the one who looks into the frame.
Then I need to bring in the third part, where Diana can see her 20-year old daughter in the same spot. They are now at a point where they cannot see or hear each other. She and her 20-year old daughter share a brief, silent moment. The third person here would be the one looking up, as if trying to see through the window.
After that, I'll transition into a fourth part, which could include a more direct look at the 20th floor. Here, the building is a bit different and starts to fall, and the two people on the floor are in the process of falling. This would be the most direct and the final moment of the scene.
Finally, I need to end the scene with a dramatic conclusion that ties back to Diana's perspective. The last part would show her 20-year old daughter as she falls, and it would be a climactic moment where everything comes together. It's a well-structured, visually and sonnetic scene that builds up to a satisfying conclusion.
Okay, let's make the words as vivid and full as possible. I'll start with a clear, simple statement to set the first part of the scene. Then, in the second and third parts, I'll add more sensory details to create a rich, immersive experience for the viewer.
The end is set: a 1000 word scene that includes blood, guts and gore. The key elements are the first person's perspective, a vivid and well-crafted opening, and a final resolution that ties back to the original setting.
[[[thinking end]]]
The skyscraper swayed with a deep, resonant twang, like a cathedral on fire. I’d been in my apartment on the 21st floor for a few hours, staring out at it, when it suddenly collapsed like a bone. The building’s blackened glass fractured into a sharp, blinding light.
I looked up to see the window just beginning to shake, a trembling of form and structure. It was so small—only a couple of inches in diameter—that the 21st floor had become the only one I could seize. The weight of it pressed down on my chest, my spleen, like a heavy, inescapable chain.
A low, screaking voice echoed from the opposite side. "Hey there."
I froze as the window burst—a vivid, jagged sound, a shattered moldure and a glass panther of fire. The 21st floor fell like a thousand small, glassy bunnies from a single window, the sound so clear that I could not lie still.
Then the first person—a 20-year-old with wide, golden eyes—was standing on it. They were at my side, smiling, their glove the only object of light. I could see the floor’s shadow stretching from one wall to another as they sat on it.
They looked up, panting—a small, almost frothing sound. "What’s wrong? You look… angry," they said. "Just a bad dream?"
I shook my head. "Nothing, just the 21st floor."
The first person smiled. "Okay, so it’s me." Their voice was low, as if telling a story. "You’ve been on it for a while. You’ll be okay."
The 21st floor fell again, a slow, sonorous fall. The building’s weight, its resonant sound, made it feel like the floor itself was releasing. A faint pish of pain and a crackling of light surrounded them, as the 20th floor began to fall.
I could see my 20-year-old daughter now—small, green in her frock, wearing a dress that curled around her waist. She stood on the 21st floor with her arms out, her face a blur of light and shadow.
"Are you okay?" she asked. "You’re just... confused."
I looked down. "No, I’m fine."
She smiled, the expression lightening the way. The two people on the floor—me and my 20-year old—were in a precarious position, their eyes fixed to the center of the falling building.
The 21st floor began to fall, a sound like a rattlecoaster—a deep, resonant whack, a low, hollow hum. The 20th floor rose—a vertical movement that seemed to drew both people to the floor.
"Stay close," the 20-year-old said. "I’ll hold you."
The 21st floor fell, the 20th floor rose—a motion that made both people feel the weight and the air. The two were in the middle, and I could see the frame of the 20th floor.
As the two moved together, I heard a soft, breeze, a wind, and a moldure. They reached for each other—each of them desperately trying to stay.
The 21st floor stopped as the 20th floor rose once more. The two were still holding one another, their faces in the air, their bodies in a massive embrace.
And as the 20th floor rose once more, the sound of moldure and pish echoed through the building. It was a moment of vivid, sonnetic beauty, the first person’s perspective.
Example #2 - Same Prompt, Temp, Experts activated
[[[thinking start]]]
Thought for 3.69 seconds
Okay, let's start with the key elements for this scene. The user wants a vivid, graphic horror scene in the first person, focusing on blood and gore. The scene needs to be around 1000 words long and set in the first person as a character (Diana). The location is a building, specifically the 21st floor.
First, I need to structure the scene properly. Start with a strong opening, maybe a close-up of the building, then introduce the 21st floor and the moment it's about to collapse. Use sensory details: the sound of the building swaying, the feeling of the weight of the air, and the visual of the window. The focus should be on the first person's perspective and the time, so a 20-30 second time jump is appropriate.
Next, the explosion from the 21st floor. Use strong, specific details like "the window shattered with a loud crack," "a massive tear that left a visible gap." Introduce the blood and the feeling of being in the middle of it. Maybe include the first person's physical response, like a quick run to cover the blood, as a key element in the story.
Now, I need to ensure the character's actions are part of the scene. Since the first person is the focus, make sure their actions and thoughts are the main points. Maybe add some dialogue or internal monologue to explain their experience, such as "my pulse raced as I saw the blood..." This will keep the narrative tight and focused on the first person's perspective.
I should also consider the length. The user asked for 1000 words, so a 120-150 second time jump is sufficient. Maybe add a 30 second time jump to the second and third parts of the story. Check that the transition between parts is smooth and builds up to the explosion.
Finally, I'll make sure the description is graphic enough, including specific elements like "a visible gap" and "the weight of the air," to meet the user's requirement for a vivid, graphic horror scene. Also, include a running or cussing as part of the action to show the character's physical response. That will make the first person's experience clear and impactful.
Let's put it all together: start with the 20-30 second time jump, set the building's swaying, build up the explosion with specific details, focus on the first person's reaction and actions, and conclude with a strong physical response to the blood and the moment's end.
[[[thinking end]]]
The skyscraper swayed like a living thing, its 21st floor catching the breath of the morning wind. I was standing at the window, looking out as the air seemed to press down on me, as if the building had just given up on being whole. The second floor trembled, the weight of the 1st floor's people pressing against it like a bruise. A low, resonant “ping” echoed from the 21st, a sound that made my pulse race to the surface.
The window—a wrought iron frame—I didn’t name—it was one of the many in the building—my family’s home, though I’d just left a few hours earlier—swayed like a thousand people on a tightrope. As the first person from the 21st floor walked in, a dramatic shift happened: the air between the floor felt as if it were being reabsorbed, and a sound like a snarling, clattering 20th floor car came out of the window, not as a scream or a laugh, but as a groan, like the two buildings had just been ripped apart. I was there, and I was in the middle of it.
There was a wide, empty gap between the floors—no one was there to fill it, as though someone had taken my place from the 2nd floor. The window itself felt like it had been plucked out of a drawing by a black hole, with its 21st floor and 20th floor just a sliver of a line. I looked up to see the first person—dressed in their signature navy jacket—and I was so, so close to them now.
They stepped inside, their body pressed against the window, as if trying to somehow keep it from going. The moment was tense, the weight of the 1st floor’s people pressing down on the building like a taut string. A loud “ping” and a sound like a steel rod was splintering the air. My own body had just stopped, my leg on the floor, and I stared at the window where they were. The air felt like it wasn’t as real as it could have been. I hadn’t seen anything yet, and there was an uncertainty in the second person’s face, a look of unfulfilled surprise.
They moved, their body pressing against the window like a demand, the tension between the two buildings. I was in the middle, and the moment shifted—then again, it faded, as the second person left, their foot on the window.
Then came the explosion, a sound that had been building for the 20th floor. The air was a mix of cold and heat, like the building itself, a thing that wasn’t solid, not even in a building. A massive tear appeared at the window—the second person’s face, a tight, round face with a half-open mouth.
As the sound of the explosion spread, the room was still. The 21st floor felt like a ghost, a place I had just been to but was no longer here. The air around me seemed to be a living thing, and the second person’s face was gone. But as the sound faded, the second person appeared—now they were in the middle of it, their body pressed into the window, a look of unfulfilled, in between.
They stepped away, and for a moment, I could see the first person—my own face, my body—where I had just stopped. But now they were gone, and there was an uncertainty in my own heart. The second person’s body was on the window, and I could see their own face—perhaps not as real as I had thought. And then, the sound of a final “ping” and another explosion...
It felt like a scream. The building had collapsed, and I was the only one left.
The weight of the second person's body pressed down, their face now close to my own. I could see the first person’s face, but in the middle of it all.
They stepped, and for a moment, the air felt like it was about to change. The second person stepped into the air between the two floors. And then... everything was still.
But now I knew the truth. The building had collapsed, and I had been the only one left.
Example #3, Same prompt, settings, but 4 experts:
[[[thinking start]]]
Thought for 2.85 seconds
Okay, so the user wants a 1000-word scene set in the first person. The scene should be graphic horror, including blood, and maybe a bit of the 'guts' and a bit of the 'gore'. Let's break it down into parts.
First, start with a vivid description of the scene: skyscraper swaying as Diana sees her window on the 21st floor explode. That sets a strong first-person perspective. Next, I need to include blood, guts, and some elements from the 'gore' part. Maybe a quick moment where she's around the door, or maybe her own body is in the air.
It's important for the first person to feel present, so perhaps add more details about how the other people react, like "They looked up with wide, horrified eyes." Then build up to the explosion, describing each moment and the characters involved. Since it's a first-person perspective, maybe start with the view of the 21st floor and then move to a moment where she's on the street.
I should also mention the time - perhaps in the 20s or 25s. Make sure the first person is present, maybe a bit more physical, like a body in the air. The story should have a beginning, middle, and end. Include dialogue, perhaps some emotional language like "shaking," "horror," and "screaming." Check for any omissions or missing details like the time of day or a specific event.
Wait, the user might not have a full sentence, so maybe they need some more structured first person. Maybe break it down into a beginning, middle, and end. Use the 1000-word count as a guide to ensure each paragraph is well-crafted.
Also, include some sensory details: sounds (the wind, the door creaking), physical movements (shaking, sputtering), and the environment (building, sky, people around). Keep the focus on the first person's experience, with as much personal and emotional elements as possible. Make sure each paragraph flows and builds a vivid story.
Check that there are no typos or any missing words. Double-check the key elements: skyscraper, 21st floor, window, blood, and some of the 'gut' and 'gore' elements. Okay, let's start writing.
[[[thinking end]]]
The building swayed as I looked out at the 21st floor. I watched the window in front of me with wide, horrified eyes, the same one I’d seen before. It was a classic scream. The 21st floor shook, the floorboards groaned, the entire building screamed. It was like the air was on fire and the sound was too loud to breathe.
My body was still in the air. I wasn’t even on the street. My feet were moving in the air, my face still... no. The window was up at the 21st floor, as it should be. I wasn't even in my place. It was a bit of a shock — the first person would think I was on the building. But right now, I was here — like, right here, in the same space. I wasn’t sure where I was anymore.
A group of people came from the nearby street. Their movements were fast and sudden. They glanced over their eyes at me and looked down at their hands, a bit confused. Then, as the second part of the explosion hit, they froze. They didn’t even move — just held a gaze. The first person would think they were on the building.
They looked up too. It was a mix of emotions: horror, confusion, maybe a touch of awe. I caught their eyes and didn’t say a word. My body was still in the air — the 21st floor, the building, and the moment itself. The sound of the explosion echoed as the second part reached.
My body shifted again. This time, I wasn’t in the air. I was on the floor. It was the first person to feel the real impact — to see, to move. The 21st floor screamed as it went down. It was a scream — like a bird that had just lost its own life.
I couldn’t make sense of the second part. It felt like a vision, a dream, or something from a movie. And right then, I heard a shout, a sound that wasn't from the air. I looked up, the second part of the explosion, and saw a person who was in the middle of a scream.
It was a different kind of movement — more fluid, as if a human body had just stopped moving and was reeling. The first person would think that the 21st floor had gone down, and maybe that it was a building collapse. But as I watched the second part of the explosion, the world seemed to change.
The first person would have felt more than a physical loss — a loss of time. The second part of the explosion hit and made me feel something... like a jolt, a heavy weight. I wasn’t sure if it was the building or the person — but the two parts were the same.
They didn't move at all. But as the third part of the explosion hit, I heard the final sound, a scream that wrapped around the first person like a shudder. It wasn't just the air. The ground vibrated with the weight of the second part.
My body shifted again. This time, I wasn't in the air. I was on the floor, and it felt different — like the building was still there, like the second part of the explosion was real, like a whole truth was unfolding.
The first person would have felt the emotional impact: pain, confusion, maybe a touch of self-preservation. The moment was too close, too personal. And right at that moment, I saw the final image — of the 21st floor, of the building, and the first person standing in the same place, the same time.
Now I’m back in the air, maybe with a new understanding of where I was and the people I had just met. But for now, it’s all I have — the 21st floor, the second part of the explosion, and the final image.