UIGEN-T2-7B-GGUF / README.md

Melvin56

Update README.md

5183ac0 verified 7 months ago

preview code

raw

history blame contribute delete

1.27 kB

metadata

base_model:
  - Tesslate/UIGEN-T2-7B
tags:
  - qwen2
  - ui-generation
  - tailwind-css
  - html
license: apache-2.0
language:
  - en
pipeline_tag: text-generation

Melvin56/UIGEN-T2-7B-GGUF

Original Model : Tesslate/UIGEN-T2-7B

Llama.cpp build: 5219 (7d3af70b)

I used imatrix to create all these quants using this Dataset.

	CPU (AVX2)	CPU (ARM NEON)	Metal	cuBLAS	rocBLAS	SYCL	CLBlast	Vulkan	Kompute
K-quants	✅	✅	✅	✅	✅	✅	✅ 🐢5	✅ 🐢5	❌
I-quants	✅ 🐢4	✅ 🐢4	✅ 🐢4	✅	✅	Partial¹	❌	❌	❌

✅: feature works
🚫: feature does not work
❓: unknown, please contribute if you can test it youself
🐢: feature is slow
¹: IQ3_S and IQ1_S, see #5886
²: Only with -ngl 0
³: Inference is 50% slower
⁴: Slower than K-quants of comparable size
⁵: Slower than cuBLAS/rocBLAS on similar cards
⁶: Only q8_0 and iq4_nl