And what hardware is that, exactly? Speaking as someone with what I’d consider to be a pretty decent homelab, I can’t self host the kind of AI that I use for work, not without literally emptying my savings into the hardware and energy needs. AI is already expected and required in my job, so the only move forward is to give elon money for access to cursor, which my company is already doing. If the company were to self-host, they’d prolly still use one of those shiny new datacenters since there’s real value in offloading hardware ownership to a third party, meaning those never go away and they still dictate the cost of using AI. I have a hard time believing this will happen in most companies simply because they already shelled out tons of money for SAAS software that has been self-hostable for over a decade, so why would you think they’d stop and think “hmm maybe we could cut costs by taking over the hosting and maintenance of the AI ourselves”?
I think we’re at least a decade away from having user-accessible hardware for AI that doesn’t break the bank. I don’t think we have a decade to spare, however.
then you should be looking for a new job… unless you happen to be on a very specific niche, any company so committed to current AI that cannot survive without, will implode in short time
Have fun finding a coding job that doesn’t expect you to use AI. My entire circle of contacts on the field are already there, so right off the bat my main way of getting a job would be crippled.
Yeah the IT department of my company had an all hands meeting today and I was a little taken aback by how much AI push there was, how much devs are expected to use it, and how much it’s costing us. They say it saves time, but there was a notable drop in app quality over the last few months, I can’t say it was caused by devs using AI for sure, but it is the most obvious answer considering there hasn’t been a big staff turnover in the same time period…
Speaking as someone with what I’d consider to be a pretty decent homelab, I can’t self host the kind of AI that I use for work, not without literally emptying my savings into the hardware and energy needs.
I think we’re at least a decade away from having user-accessible hardware for AI that doesn’t break the bank.
So, for coding, which is what Gates was specifically talking about being doable now, maybe we could corral up the hardware. Like, maybe one could cover specific fields.
There are still going to be issues like power and cooling and hardware cost and whether people who make competitive models are even interested in providing it for home use (since it makes it harder for them to make a return on their model). But set that aside.
As I’ve said before on here, I would say pretty confidently that we will not have local models running to do all of the stuff that cloud compute is used for or is being built out to for at least something like four to five years, and that’s if we started immediate, massive buildout of memory fabrication to a much greater degree than we have. You cannot build a new memory factory in less than that timeframe, and we will not have that capacity with existing factories. The majority of fabricated memory now is going to cloud AI use, and cloud AI hardware will have considerably higher capacity utilization than hardware at home. You’d have to have many times over as much memory being produced to have the same compute capacity at home.
I’m not opposed to doing LLMs or parallel compute at home at all. I have a 128GB Framework Desktop and an XT 7900 XTX that I got to do just that. I’m just saying that we are not going to realistically be able to move all of the stuff in the cloud to the home for at least something like half a decade, and very probably more, because humanity does not have the memory available and can’t build enough memory fabrication capacity for it in that timeframe. It doesn’t matter how much value is being provided by some home user of that hardware or what their willingness is to spend on it if we don’t have the memory. Like, even if every person in the world could produce, to pull a number out of the air, a real $1M in value every year via use of a home AI rig, even if all that demand suddenly materialized out of thin air, all that would happen is that prices would rise sufficiently to make the hardware unaffordable even at those extreme levels. The constraint is on the supply end, not the demand end.
Understandably you won’t be running meta level Machine models, but running olama and downloading a open LLM model can get users started for simple things they might have already been doing.
I am running a fully local model for my smart home needs, since Google’s killed of Assistant and is now pushing Gemini down my throat, I figured I would just do it my self.
For anyone interested take a look at NetworkChuck on YouTube. His videos are a little to sensational for me, but he does have some interesting topics covered from time to time.
Understanding is a long time coming. It will probably take businesses realizing they are stealing all their trade secrets when using LLMs for any meaningful information to come out.
Prices will definitely rise to cover operating costs and to recoup all the investment costs.
I honestly hope the whole thing crashes once people realize they can run their own LLM on their own hardware.
And what hardware is that, exactly? Speaking as someone with what I’d consider to be a pretty decent homelab, I can’t self host the kind of AI that I use for work, not without literally emptying my savings into the hardware and energy needs. AI is already expected and required in my job, so the only move forward is to give elon money for access to cursor, which my company is already doing. If the company were to self-host, they’d prolly still use one of those shiny new datacenters since there’s real value in offloading hardware ownership to a third party, meaning those never go away and they still dictate the cost of using AI. I have a hard time believing this will happen in most companies simply because they already shelled out tons of money for SAAS software that has been self-hostable for over a decade, so why would you think they’d stop and think “hmm maybe we could cut costs by taking over the hosting and maintenance of the AI ourselves”?
I think we’re at least a decade away from having user-accessible hardware for AI that doesn’t break the bank. I don’t think we have a decade to spare, however.
then you should be looking for a new job… unless you happen to be on a very specific niche, any company so committed to current AI that cannot survive without, will implode in short time
Have fun finding a coding job that doesn’t expect you to use AI. My entire circle of contacts on the field are already there, so right off the bat my main way of getting a job would be crippled.
Yeah the IT department of my company had an all hands meeting today and I was a little taken aback by how much AI push there was, how much devs are expected to use it, and how much it’s costing us. They say it saves time, but there was a notable drop in app quality over the last few months, I can’t say it was caused by devs using AI for sure, but it is the most obvious answer considering there hasn’t been a big staff turnover in the same time period…
So, for coding, which is what Gates was specifically talking about being doable now, maybe we could corral up the hardware. Like, maybe one could cover specific fields.
There are still going to be issues like power and cooling and hardware cost and whether people who make competitive models are even interested in providing it for home use (since it makes it harder for them to make a return on their model). But set that aside.
As I’ve said before on here, I would say pretty confidently that we will not have local models running to do all of the stuff that cloud compute is used for or is being built out to for at least something like four to five years, and that’s if we started immediate, massive buildout of memory fabrication to a much greater degree than we have. You cannot build a new memory factory in less than that timeframe, and we will not have that capacity with existing factories. The majority of fabricated memory now is going to cloud AI use, and cloud AI hardware will have considerably higher capacity utilization than hardware at home. You’d have to have many times over as much memory being produced to have the same compute capacity at home.
I’m not opposed to doing LLMs or parallel compute at home at all. I have a 128GB Framework Desktop and an XT 7900 XTX that I got to do just that. I’m just saying that we are not going to realistically be able to move all of the stuff in the cloud to the home for at least something like half a decade, and very probably more, because humanity does not have the memory available and can’t build enough memory fabrication capacity for it in that timeframe. It doesn’t matter how much value is being provided by some home user of that hardware or what their willingness is to spend on it if we don’t have the memory. Like, even if every person in the world could produce, to pull a number out of the air, a real $1M in value every year via use of a home AI rig, even if all that demand suddenly materialized out of thin air, all that would happen is that prices would rise sufficiently to make the hardware unaffordable even at those extreme levels. The constraint is on the supply end, not the demand end.
Understandably you won’t be running meta level Machine models, but running olama and downloading a open LLM model can get users started for simple things they might have already been doing.
I am running a fully local model for my smart home needs, since Google’s killed of Assistant and is now pushing Gemini down my throat, I figured I would just do it my self.
For anyone interested take a look at NetworkChuck on YouTube. His videos are a little to sensational for me, but he does have some interesting topics covered from time to time.
https://www.youtube.com/watch?v=QQEgIo4Juxg
Understanding is a long time coming. It will probably take businesses realizing they are stealing all their trade secrets when using LLMs for any meaningful information to come out.