This is not a SFW instance. NSFW communities will be tagged, and efforts will be made to keep SFW communities clean, but proceed at your own comfort level.
I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.
I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
Just Qwen3.8-27B. That little model is surprisingly smart.
NixOS also gives you the ability to roll back to previous config versions. I have not needed to do that (yet), but it reduces some risk.
“Just” and a 27B model don’t exactly belong in the same sentence if you ask my wallet. 💸
Yeahhh sorry 😕
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.