Hugging Face's logo

mradermacher
/
Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF

Transformers
GGUF
English
programming
code generation
code
coding
coder
chat
qwen
qwen3
qwencoder
creative
all uses cases
horror
finetune
unsloth
story
short stories
story telling
imatrix
conversational
Model card Files
xet
Community
mradermacher commited on
Commit
5430bc5
·
verified ·
1 Parent(s): f594669

auto-patch README.md

Browse files
Files changed (1) hide show
  1. README.md +92 -0
README.md CHANGED
@@ -1,3 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!-- ### quantize_version: 2 -->
2
  <!-- ### output_tensor_quantised: 1 -->
3
  <!-- ### convert_type: hf -->
@@ -7,3 +35,67 @@
7
  <!-- ### quants_skip: -->
8
  <!-- ### skip_mmproj: -->
9
  weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: apache-2.0
7
+ mradermacher:
8
+ readme_rev: 1
9
+ quantized_by: mradermacher
10
+ tags:
11
+ - programming
12
+ - code generation
13
+ - code
14
+ - coding
15
+ - coder
16
+ - chat
17
+ - code
18
+ - chat
19
+ - qwen
20
+ - qwen3
21
+ - qwencoder
22
+ - creative
23
+ - all uses cases
24
+ - horror
25
+ - finetune
26
+ ---
27
+ ## About
28
+
29
  <!-- ### quantize_version: 2 -->
30
  <!-- ### output_tensor_quantised: 1 -->
31
  <!-- ### convert_type: hf -->
 
35
  <!-- ### quants_skip: -->
36
  <!-- ### skip_mmproj: -->
37
  weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B
38
+
39
+ <!-- provided-files -->
40
+
41
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF).***
42
+
43
+ static quants are available at https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-GGUF
44
+ ## Usage
45
+
46
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
47
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
48
+ more details, including on how to concatenate multi-part files.
49
+
50
+ ## Provided Quants
51
+
52
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
53
+
54
+ | Link | Type | Size/GB | Notes |
55
+ |:-----|:-----|--------:|:------|
56
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own qwuants) |
57
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.3 | for the desperate |
58
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.4 | mostly desperate |
59
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.5 | |
60
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.6 | |
61
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.7 | |
62
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.8 | |
63
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.8 | very low quality |
64
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q2_K.gguf) | i1-Q2_K | 1.9 | IQ3_XXS probably better |
65
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.9 | lower quality |
66
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.1 | |
67
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.2 | IQ3_XS probably better |
68
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ3_S.gguf) | i1-IQ3_S | 2.2 | beats Q3_K* |
69
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ3_M.gguf) | i1-IQ3_M | 2.2 | |
70
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.3 | IQ3_S probably better |
71
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.5 | IQ3_M probably better |
72
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.6 | |
73
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q4_0.gguf) | i1-Q4_0 | 2.7 | fast, low quality |
74
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.7 | prefer IQ4_XS |
75
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.7 | optimal size/speed/quality |
76
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.8 | fast, recommended |
77
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q4_1.gguf) | i1-Q4_1 | 2.9 | |
78
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 3.2 | |
79
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.3 | |
80
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B-i1-GGUF/resolve/main/Qwen3-Short-Story-Instruct-Uncensored-262K-ctx-4B.i1-Q6_K.gguf) | i1-Q6_K | 3.7 | practically like static Q6_K |
81
+
82
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
83
+ types (lower is better):
84
+
85
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
86
+
87
+ And here are Artefact2's thoughts on the matter:
88
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
89
+
90
+ ## FAQ / Model Request
91
+
92
+ See https://huggingface.co/mradermacher/model_requests for some answers to
93
+ questions you might have and/or if you want some other model quantized.
94
+
95
+ ## Thanks
96
+
97
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
98
+ me use its servers and providing upgrades to my workstation to enable
99
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
100
+
101
+ <!-- end -->