Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • 0^2@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    2 days ago

    I went down the same rabbit hole. I have a 6800xt however but have issues getting it to perform outside of llm chats into using tools like pi.dev

    Is it worth getting a v620?

    • mierdabird@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      2
      ·
      2 days ago

      If you are having trouble getting the 6800xt to work with pi.dev I’d be surprised if the V620 would be any different, but I haven’t tried that tool. I can attempt it and get back to ya in a couple days if you’d like.

      I ended up getting it purely as it seemed like the cheapest option for 32GB VRAM that didnt have discontinued driver support. Around Jan/Feb 2026 the MI60’s had recently blown up in price but the V620 still seemed niche/slept on partially because AMD hasn’t released an SR-IOV driver for this. Servethehome forums had a big thread about how these aren’t particularly useful for home server/virtual machines as a result. I think it’s still possible to pass it through to docker containers but I haven’t tried it yet.

      This guy accepted a $350 offer for mine:
      www.ebay.com/itm/157133307609
      Then you’ll need a shroud:
      www.ebay.com/itm/286347509481
      The optional included fan works well, pushes 60CFM but is LOUD. I ended up replacing it with an Arctic P8 Max which is much quieter but only pushes 40CFM, but cools it fine with -100mV undervolt in LACT.