Tiny-Purr
Collection
Tiny-Purr, a OSS clone of PurrGPT!
•
5 items
•
Updated
This model is a GGUF conversion of purrgpt-community/Tiny-Purr-350M-merged, created using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details.
Install llama.cpp (works on Mac and Linux):
brew install llama.cpp
You can invoke the model using either the CLI or the server.
llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"
llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048
git clone https://github.com/ggerganov/llama.cpp
LLAMA_CURL=1 flag. Add hardware-specific flags if needed (e.g., LLAMA_CUDA=1 for Nvidia GPUs on Linux):cd llama.cpp
LLAMA_CURL=1 make
./llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048
4-bit
Base model
purrgpt-community/Tiny-Purr-350M-merged