SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 3 days agoDo you host your own AI?message-squaremessage-square200linkfedilinkarrow-up1179arrow-down141file-text
arrow-up1138arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 3 days agomessage-square200linkfedilinkfile-text
minus-squaree0qdk@reddthat.comlinkfedilinkEnglisharrow-up3·3 days agoIf you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB. Check ollama show qwen3.5 and see if you get something like this in the result: Model architecture qwen35 parameters 9.7B context length 262144 embedding length 4096 quantization Q4_K_M This is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.
minus-squareSteve@startrek.websitelinkfedilinkEnglisharrow-up2·3 days agoI will check it later. I loaded whichever one cluade suggested lol
If you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB.
Check
ollama show qwen3.5and see if you get something like this in the result:Model architecture qwen35 parameters 9.7B context length 262144 embedding length 4096 quantization Q4_K_MThis is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.
I will check it later. I loaded whichever one cluade suggested lol