Hey i like the model could you maybe make a NVFP4 version or a version optimised for the dgx spark?
#1
by
Floris111
- opened
above
and W4A16, please :)
NVFP4 would be ideal, but I'd be happy to see that W4A16 too.. The GGUFs just don't work with vLLM.
If a W4A16 is created, please add the script used to run AutoRound so I can see what I have done wrong when trying this myself!