Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • robber@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    23 hours ago

    Well compared to the strix, 400GB/s is not that bad, I think with fast system RAM and expert offloading you could squeeze quite something out of it when running stuff in the 100b-a10b regions.

    Your bigger problem is going to be future software support.