This is not a SFW instance. NSFW communities will be tagged, and efforts will be made to keep SFW communities clean, but proceed at your own comfort level.
Worth pointing out that you can run models locally on consumer hardware that are as good as frontier models from ~3 months ago (and that gap is getting smaller and smaller) which solves ever single problem listed in this post
The thing I have learned recently attending industry events is that while what you are saying is technically feasible, it’s not as tantalizing as what the AI giants are promising. They are selling an all-in-one solution that was supposed to reduce costs and make skilled work idiot-proof.
To run a local model, they still need to have at least a few people on staff that know what the hell they’re doing to set it up and maintain it. Not exactly the silver bullet that they were expecting.
it’s not as tantalizing as what the AI giants are promising
I lost my job years ago to such con-men selling the idea of “outsource your IT to India, they’ll work for only $2 a day!!! Imagine the savings!!!”
And then their IT infrastructure went to shit. And any of us unemployed IT guys who called them out were labeled as racist for pointing out the people they fired us to outsource to were objectively unqualified.
Business owners need to wise up and realize every sales pitch they hear is probably a conman lying to get their money.
Except for the problem that the idiot managers didn’t do that originally. Also the problem that they don’t seem to be aware of how to do that, probably because they fired their IT team.
Does it link to the internet and do I have to let it access every part of my life?
Idk man in a surveillance state, AI seems like the most obvious way into everyone’s lives that’s ever been deployed by companies who are awful and shamelessly so.
For the case of local models, no not at all. Most local models are under a free license (or at least a non-commericial one) and it all runs locally on your hardware, however what you can run and how well depends on your hardware and some software fiddling. Non-local stuff if 100% a ploy to make everyone dependent on big tech even more than they already are, though
I’ve no interest in trying to run an AI locally on my own equipment but I have set this up for work and it’s internet access is monitored and heavily restricted. It has complete intranet access, so that’s all the documentation it needs a d it can access read only versions of anything saved in one drive. It can visit websites but it doesn’t have the ability to upload anything, it can’t even send emails, it can write the email, but a human needs to press send. It seems to be working pretty well, I haven’t heard a peep from management since, so I’m guessing they’re happy with it.
Qwen 3.8 has a 27b (consumer gpu) model and one that will run in 128gb. deepseek 4.1, glm 5.3 flash, and mimo 2.6 flash also fit in 128-256gb ram, and challenge models for 1-2 months ago.
depends who wants to setup a local model and be responsible for it
it gets to a point where isn’t a youtube to help any further, and you end up making it up as you go along
Worth pointing out that you can run models locally on consumer hardware that are as good as frontier models from ~3 months ago (and that gap is getting smaller and smaller) which solves ever single problem listed in this post
The thing I have learned recently attending industry events is that while what you are saying is technically feasible, it’s not as tantalizing as what the AI giants are promising. They are selling an all-in-one solution that was supposed to reduce costs and make skilled work idiot-proof.
To run a local model, they still need to have at least a few people on staff that know what the hell they’re doing to set it up and maintain it. Not exactly the silver bullet that they were expecting.
I lost my job years ago to such con-men selling the idea of “outsource your IT to India, they’ll work for only $2 a day!!! Imagine the savings!!!”
And then their IT infrastructure went to shit. And any of us unemployed IT guys who called them out were labeled as racist for pointing out the people they fired us to outsource to were objectively unqualified.
Business owners need to wise up and realize every sales pitch they hear is probably a conman lying to get their money.
Except for the problem that the idiot managers didn’t do that originally. Also the problem that they don’t seem to be aware of how to do that, probably because they fired their IT team.
Does it link to the internet and do I have to let it access every part of my life?
Idk man in a surveillance state, AI seems like the most obvious way into everyone’s lives that’s ever been deployed by companies who are awful and shamelessly so.
For the case of local models, no not at all. Most local models are under a free license (or at least a non-commericial one) and it all runs locally on your hardware, however what you can run and how well depends on your hardware and some software fiddling. Non-local stuff if 100% a ploy to make everyone dependent on big tech even more than they already are, though
I’ve no interest in trying to run an AI locally on my own equipment but I have set this up for work and it’s internet access is monitored and heavily restricted. It has complete intranet access, so that’s all the documentation it needs a d it can access read only versions of anything saved in one drive. It can visit websites but it doesn’t have the ability to upload anything, it can’t even send emails, it can write the email, but a human needs to press send. It seems to be working pretty well, I haven’t heard a peep from management since, so I’m guessing they’re happy with it.
What are some local models worth exploring?
Qwen 3.8 has a 27b (consumer gpu) model and one that will run in 128gb. deepseek 4.1, glm 5.3 flash, and mimo 2.6 flash also fit in 128-256gb ram, and challenge models for 1-2 months ago.
128 GB RAM? In this economy ?
depends who wants to setup a local model and be responsible for it it gets to a point where isn’t a youtube to help any further, and you end up making it up as you go along