Update README.md
Browse files
README.md
CHANGED
|
@@ -12,31 +12,6 @@ tags:
|
|
| 12 |
|
| 13 |
Quantized to `i1-GGUF` using [SpongeQuant](https://github.com/SpongeEngine/SpongeQuant), the Oobabooga of LLM quantization.
|
| 14 |
|
| 15 |
-
<div style="display: flex; gap: 20px; align-items: center; margin-top:0;">
|
| 16 |
-
<a href="https://github.com/SpongeEngine/SpongeQuant">
|
| 17 |
-
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/github-button.png" width="173">
|
| 18 |
-
</a>
|
| 19 |
-
<a href="https://discord.gg/azNmr2Gdgy">
|
| 20 |
-
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/discord-button.png" width="173">
|
| 21 |
-
</a>
|
| 22 |
-
</div>
|
| 23 |
-
|
| 24 |
-
***
|
| 25 |
-
<figure>
|
| 26 |
-
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/113.png" alt="Titan Centaur launch">
|
| 27 |
-
<figcaption>Titan Centaur launch</figcaption>
|
| 28 |
-
</figure>
|
| 29 |
-
|
| 30 |
-
<figure>
|
| 31 |
-
<audio controls>
|
| 32 |
-
<source src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/003.mp3" type="audio/mp3">
|
| 33 |
-
Your browser does not support the audio element.
|
| 34 |
-
</audio>
|
| 35 |
-
<figcaption>Queen & David Bowie – Under Pressure (UK, 1981)</figcaption>
|
| 36 |
-
</figure>
|
| 37 |
-
|
| 38 |
-
***
|
| 39 |
-
|
| 40 |
### What is a GGUF?
|
| 41 |
GGUF is a file format used for running large language models (LLMs) on different types of computers. It supports both regular processors (CPUs) and graphics cards (GPUs), making it easier to run models across a wide range of hardware. Many LLMs require powerful and expensive GPUs, but GGUF improves compatibility and efficiency by optimizing how models are loaded and executed. If a GPU doesn't have enough memory, GGUF can offload parts of the model to the CPU, allowing it to run even when GPU resources are limited. GGUF is designed to work well with quantized models, which use less memory and run faster, making them ideal for lower-end hardware. However, it can also store full-precision models when needed. Thanks to these optimizations, GGUF allows LLMs to run efficiently on everything from high-end GPUs to laptops and even CPU-only systems.
|
| 42 |
|
|
|
|
| 12 |
|
| 13 |
Quantized to `i1-GGUF` using [SpongeQuant](https://github.com/SpongeEngine/SpongeQuant), the Oobabooga of LLM quantization.
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
### What is a GGUF?
|
| 16 |
GGUF is a file format used for running large language models (LLMs) on different types of computers. It supports both regular processors (CPUs) and graphics cards (GPUs), making it easier to run models across a wide range of hardware. Many LLMs require powerful and expensive GPUs, but GGUF improves compatibility and efficiency by optimizing how models are loaded and executed. If a GPU doesn't have enough memory, GGUF can offload parts of the model to the CPU, allowing it to run even when GPU resources are limited. GGUF is designed to work well with quantized models, which use less memory and run faster, making them ideal for lower-end hardware. However, it can also store full-precision models when needed. Thanks to these optimizations, GGUF allows LLMs to run efficiently on everything from high-end GPUs to laptops and even CPU-only systems.
|
| 17 |
|