what is a good model for 16GB vram?
what is everyone on 16GB using?
I use q4_k_m text and q5_k_m model on 16gb cause of f32 upcasting, but the original Phr00t does run on 16+32gb also just fine.
Its a little bit up on GPU. Or what workflow u use.
Yesterday i tried around with controlnet, which gave me a black picture but needed alot of vram +6gb and i had to use the q2_k.
since confyui does load partially for me with as good as no speed loss, take whatever fits into your ram without swapping.
--dont-upcast-attention startup flag will keep it in f16/bf16 wont it?
yea, but for my old gfx900 that means also black image.
Bummer. Yes that can happen.
I think the answer is Q3 or Q2
I use q4_k_m text and q5_k_m model on 16gb cause of f32 upcasting, but the original Phr00t does run on 16+32gb also just fine.
Its a little bit up on GPU. Or what workflow u use.
Yesterday i tried around with controlnet, which gave me a black picture but needed alot of vram +6gb and i had to use the q2_k.
since confyui does load partially for me with as good as no speed loss, take whatever fits into your ram without swapping.
I noticed with the Q5_K_S I top at 95% usage, but i do get OOMs every now and then, so the ~2GB difference with the Q4_K_M, might be enough to keep me in the clear of OOMs