I’m not a tech person, and I very much hate ai, but do others here feel like this is the kind of thing ai might get better at over time? I’m just recalling that a couple years ago images generated of people were laughably bad and now they are at least a little harder to immediately distinguish as AI without closer examination.
I wonder if it will be the same with slop code. Never good, but possibly better than current
LLMs have been getting better at code generation over time. The problem is that AI boosters think that AI is and will get exponentially better over time, but in reality these increases are slowing down and increasing in cost, both in terms of training and inference (actually generating the output). Each advancement is smaller and costs much more than the previous one, creating a natural limit over time.
I’m not sure where the theoretical limitations are when it comes to code generation, but there’s also an economic limit. You can keep throwing resources at the problem, but the curve is exponential. For example, you can increase the size of the model or the context window by throwing more RAM at the problem, but there’s only so much you can cram into a system, assuming that we can even manufacture enough.
You’re right, because that is a matter of technique and process, which our current global structure excels at accelerating.
It’s just a matter of building a good enough list of things to check. The tech-debt of getting to that point shouldn’t really be resting on the public though.
Sadly I can’t give any specific technical details without potentially doxxing myself to my employer, but I’ve reviewed 5 or 6 AI-generated PRs in the last year that have contained some pretty severe bugs — the kind of stuff that would have caused serious service outages and security breaches if allowed to go to production.
Those 5 or 6 were among dozens of PRs, and the amount of code I’m being asked to review is only increasing. Even being diligent, I’m sure it’s only a matter of time before something nasty slips by me, especially now that the process of reviewing code has become so fucking exhausting.
“Just get [insert LLM] to review it for you!”
— a fucking moron, probably
My company unironically. I hate it here.
Make more agents!
Yet another reason to not ship code like this. I keep my slop code to myself, tyvm.
