

not for LLMs. I have a 16GB and even what I can fit in there just isn’t really enough to be useful. It can still do things and quickly enough, but I can’t fit models that large enough to be useful.
I also don’t know if your GPU is compatible with ROCM or not.
I actually just wrote about today’s fun experience! https://gotosocial.michaeldileo.org/@mdileo/statuses/01K7YKQ9584YBY1QTYQ8RMW7SS