unsloth/Kimi-K2-Thinking-GGUF
1T
•
Updated
•
1.01k
•
34
AI & ML interests
Open Source AI 💚
Recent Activity
View all activity
danielhanchen
updated
a
model
about 1 hour ago
Chat template fixes + Q&A
🚀
🔥
1
#2 opened about 7 hours ago
by
danielhanchen
danielhanchen
posted
an
update
about 4 hours ago
Post
133
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs:
unsloth/Kimi-K2-Thinking-GGUF
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
Pls MXFP4
🔥
1
5
#4 opened 5 days ago
by
Kirara702
No thinking tags when it runs?
9
#1 opened about 20 hours ago
by
Disdrix
danielhanchen
updated
a
model
about 14 hours ago
danielhanchen
updated
a
model
about 20 hours ago
danielhanchen
published
a
model
about 21 hours ago
danielhanchen
updated
a
collection
about 22 hours ago
danielhanchen
updated
a
model
1 day ago
danielhanchen
posted
an
update
3 months ago
Post
6155
Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
GGUFs: unsloth/DeepSeek-V3.1-GGUF
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
Guide: https://docs.unsloth.ai/basics/deepseek-v3.1
GGUFs: unsloth/DeepSeek-V3.1-GGUF
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
Guide: https://docs.unsloth.ai/basics/deepseek-v3.1
danielhanchen
posted
an
update
3 months ago
Post
5278
Run OpenAI's new gpt-oss models locally with Unsloth GGUFs! 🔥🦥
20b GGUF: unsloth/gpt-oss-20b-GGUF
120b GGUF: unsloth/gpt-oss-120b-GGUF
Model will run on 14GB RAM for 20b and 66GB for 120b.
20b GGUF: unsloth/gpt-oss-20b-GGUF
120b GGUF: unsloth/gpt-oss-120b-GGUF
Model will run on 14GB RAM for 20b and 66GB for 120b.
- 2 replies
danielhanchen
posted
an
update
4 months ago
Post
3460
It's Qwen3 week! 💜 We uploaded Dynamic 2-bit GGUFs for:
Qwen3-Coder: unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF
Qwen3-2507: unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF
So you can run them both locally!
Guides are in model cards.
Qwen3-Coder: unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF
Qwen3-2507: unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF
So you can run them both locally!
Guides are in model cards.
- 1 reply
danielhanchen
posted
an
update
4 months ago
Post
3697
danielhanchen
posted
an
update
4 months ago
Post
3824
We fixed more issues! Use
* Fixed Nanonets OCR-s unsloth/Nanonets-OCR-s-GGUF
* Fixed THUDM GLM-4 unsloth/GLM-4-32B-0414-GGUF
* DeepSeek Chimera v2 is uploading! unsloth/DeepSeek-TNG-R1T2-Chimera-GGUF
--jinja for all!* Fixed Nanonets OCR-s unsloth/Nanonets-OCR-s-GGUF
* Fixed THUDM GLM-4 unsloth/GLM-4-32B-0414-GGUF
* DeepSeek Chimera v2 is uploading! unsloth/DeepSeek-TNG-R1T2-Chimera-GGUF
danielhanchen
posted
an
update
4 months ago
Post
3062
Gemma 3n finetuning is now 1.5x faster and uses 50% less VRAM in Unsloth!
Click "Use this model" and click "Google Colab"!
unsloth/gemma-3n-E4B-it
unsloth/gemma-3n-E2B-it
https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B)-Conversational.ipynb
Click "Use this model" and click "Google Colab"!
unsloth/gemma-3n-E4B-it
unsloth/gemma-3n-E2B-it
https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B)-Conversational.ipynb
-
-
- 2 replies
danielhanchen
posted
an
update
5 months ago
Post
1228
We updated lots of our GGUFs and uploaded many new ones!
* unsloth/dots.llm1.inst-GGUF
* unsloth/Jan-nano-GGUF
* unsloth/Nanonets-OCR-s-GGUF
* Updated and fixed Q8_0 upload for unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF
* Added Q2_K_XL for unsloth/DeepSeek-R1-0528-GGUF
* Updated and fixed Vision support for unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF
* unsloth/dots.llm1.inst-GGUF
* unsloth/Jan-nano-GGUF
* unsloth/Nanonets-OCR-s-GGUF
* Updated and fixed Q8_0 upload for unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF
* Added Q2_K_XL for unsloth/DeepSeek-R1-0528-GGUF
* Updated and fixed Vision support for unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF
danielhanchen
posted
an
update
5 months ago
Post
2446
Mistral releases Magistral, their new reasoning models! 🔥
GGUFs to run: unsloth/Magistral-Small-2506-GGUF
Magistral-Small-2506 excels at mathematics and coding.
You can run the 24B model locally with just 32GB RAM by using our Dynamic GGUFs.
GGUFs to run: unsloth/Magistral-Small-2506-GGUF
Magistral-Small-2506 excels at mathematics and coding.
You can run the 24B model locally with just 32GB RAM by using our Dynamic GGUFs.
danielhanchen
posted
an
update
5 months ago
Post
3787
New DeepSeek-R1-0528 1.65-bit Dynamic GGUF!
Run the model locally even easier! Will fit on a 192GB Macbook and run at 7 tokens/s.
DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-GGUF
Qwen3-8B DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF
And read our Guide: https://docs.unsloth.ai/basics/deepseek-r1-0528
Run the model locally even easier! Will fit on a 192GB Macbook and run at 7 tokens/s.
DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-GGUF
Qwen3-8B DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF
And read our Guide: https://docs.unsloth.ai/basics/deepseek-r1-0528
danielhanchen
posted
an
update
6 months ago
Post
2287
💜 Qwen3 128K Context Length: We've released Dynamic 2.0 GGUFs + 4-bit safetensors!
Fixed: Now works on any inference engine and fixed issues with the chat template.
Qwen3 GGUFs:
30B-A3B: unsloth/Qwen3-30B-A3B-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-GGUF
32B: unsloth/Qwen3-32B-GGUF
Read our guide on running Qwen3 here: https://docs.unsloth.ai/basics/qwen3-how-to-run-and-finetune
128K Context Length:
30B-A3B: unsloth/Qwen3-30B-A3B-128K-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-128K-GGUF
32B: unsloth/Qwen3-32B-128K-GGUF
All Qwen3 uploads: unsloth/qwen3-680edabfb790c8c34a242f95
Fixed: Now works on any inference engine and fixed issues with the chat template.
Qwen3 GGUFs:
30B-A3B: unsloth/Qwen3-30B-A3B-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-GGUF
32B: unsloth/Qwen3-32B-GGUF
Read our guide on running Qwen3 here: https://docs.unsloth.ai/basics/qwen3-how-to-run-and-finetune
128K Context Length:
30B-A3B: unsloth/Qwen3-30B-A3B-128K-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-128K-GGUF
32B: unsloth/Qwen3-32B-128K-GGUF
All Qwen3 uploads: unsloth/qwen3-680edabfb790c8c34a242f95