
i showed this to my friend (who works in cybersecurity) and he corrected me: “i blink -> there’s a new vulnerability. conclusion: don’t blink.”
imagine how pissed the CIA must be that all their zero-day exploits are getting patched.
I doubt it, it’s not like there’s a shortage of zero days these days. I bet way more are discovered than are patched.
There, a bunch of developer jobs appeared!
As others have stated and I will reiterate, this (to me) is all of the technical debt coming home to roost. How often have we intended to go back and refactor something once we have time? I don’t just mean in your current job, I’m talking about throughout your career. I think about code I wrote in 1999 that may still be out there and cringe at what I put together back then. The person that has taken over after me at every job probably just built on top of what works and underlying shit got buried. Of course the business should have gone back and evaluated old code but are they really doing that? There is a vast amount of technical debt out there that was secured solely by the vast amount of other technical debt. There was always too much to dig through but now AI has reduced the resources needed to dig through it all at scale. This is a whole new world and just the beginning of the ride.
The funny thing is that one of the reports on the glasswing project found out a lot (~55%) of the vulnerabilities AI finds are just false positives and not really worth reporting so they just increase the cognitive workload of unpaid maintainers.
Yes, a 55% false positive rate technically means more often than not, it isn’t actually a valid vulnerability and it is wasting your time. But if you have 1000 boxes to go through while looking for a certain item, someone with a 50% false positive rate could identify two boxes for you, and it would be in one of them.
So unless that number is “AI claims there’s a vulnerability on every other line of code you wrote”, which I kinda doubt, and is actually based on made bug/vulnerability reports, then isn’t is extremely helpful even if it’s only right half the time?
Getting what I’ve seen even the actual vulnerabilities are the sort of stuff that is very very very hard to pull off, like it’s identified a .2ms window where if you input something specific, you break something and gain access. Like race conditions that even though they theoretically exist, they’re actually impossible to execute
This was pretty much inevitable - there is so much insecure code out there from practically every era of computing. There’s only one answer, and that is giving developers and security experts the resources (that includes LLMs) and time they need to ensure that all this legacy code is being fixed or replaced with better structured code that is less attackable.
It will be a wild ride for a while, and then things will slow down again when the well of not-yet-discovered exploitable bugs starts to run dry.
when the well of not-yet-discovered exploitable bugs starts to run dry.
i think that roughly 80% of all code in existence is buggy. humans are just really bad at figuring that out. it’s actually amazing that our software infrastructure stood for this long without crashing down.
I’m quite staunchly anti-LLM, but code review is where I think they are most suited. Mind you, only as PART. But their ability to hold large bodies of well-structured code in context is what gives them the edge to find possible bugs and exploits a human has overlooked.
They never should’ve been used for code creation, and in reality they’re being use on both sides of the equation going completely unmonitored or, at best, there’s a burnt out human sitting in between rubber-stamping commits because the entire pitch for this shit is to increase productivity (output) and move at a rate designed to outpace them.
In my experience, AI writes better code (and tests) than most developers and much worse architecture than a senior engineer.
I suspect the architecture is fine, really, but AI is so comfortable juggling 19 balls that it’s architecture never needs simplification and clarity. But as a human reading it, it’s just a mess and humans need much more coherence and encapsulation.
Ever seen them do documentation? I wrote better shit in high school
I feel like AI writes good code but then every once in a while it does the absolute dumbest thing in the world. And you have to be really vigilant so you can’t it doing dumb shit.
Then most developers you know are subpar. Frontier models routinely write far worse code than I. And I am only a moderately decent dev.
I have even seen llm code generated with a frontier model that blew straight through array boundaries without a worry in the world.
They can be useful at work, especially for tech I know little about, and as an additional set of eyes to avoid easy mistakes like off by ones, or doing wrong pointer arithmetic, but they are pretty abysmal at code generation.
Nononono “tech you know little about” means you don’t know enough to see the holes it’s making
I thought the same earlier this year. Opus 4.8 mostly produced garbage code in my eyes. Then Opus 5.5 came and now it can routinely produce about the same quality as me but much faster.
Don’t believe the hype.
The truth is nobody ever has time to test things thoroughly. Proprietary software has to ship early due to unrealistic deadlines. Bad specs, client doesn’t know what the fuck they want. Open source dev doing weekend hobby coding suddenly getting pressured by billion dollar corps for features while trying to balance mental health, family, pay bills etc.
So we take shortcuts. Ship code. Hope for the best. Better compiler warnings, linters and strict coding standards can help but only so much.
LLMs let security researchers search for classes of bug that have been harder to find with other methods. Lots of low hanging fruit for them. Most of the vulnerablities are low impact and over hyped. Once they find them and they are fixed they aren’t going to find the same exact one again. The security stuff will get better. Unfortunately not so much careers.
Nobody has time to test things thoroughly because the fastest code that looks good succeeds because we looked at “the ui is the product” not as a problem to fix but as a customer to be appeased and now we’re paying for it.
Software engineering needs to become a licensed profession like any other with professional engineering standards for efficiency and safety. It’s long over due.
AI Is Breaking the Entire Structure of Software That Powers the World
On both sides even - both by better automating brute force hacking strategies AND by generating weakness-riddled code for network-facing software.
What a great time to be alive!
I actually think the code AI writes is much more hardened than a human’s. But the architecture that it creates is incredibly hard to reason about and feels very brittle and prone to exploitation.
I think you’re confusing AI being better at code hardenning with modern server-side libraries for network-facing software being designed to be idiot-proof and stopping things like SQL injection via parameters - the code is safer because over the last 3 decades the libraries were made idiot-proof, not because the AI is any better than the idiots.
The automated parrot which does things like putting keys and other access credentials plaintext in GitHub only makes code (and just code, it can’t really do design properly) as hardenned as the average found in open repositories, which includes amongst other things simplified examples in Stack Overflow (which probably explains the GitHub plaintext credentials thing)
Most weakness for hacking nowadays are either the traditional social hacking or they’re interactions between multiple systems which haven’t been accounted for (or, even better, which were introduced when one of those systems changed but not the other), the former having nothing to do with AI and all to do with users giving access creditals to anybody who sounds “confidently in charge”, whilst the latter being made much more likely by what you yourself pointed out as a problem of AI.
AI isn’t breaking software by itself. It’s revealing just how much of the world runs on code held together by hope, legacy dependencies, and one exhausted maintainer. The exciting new feature is that attackers can now find those cracks for $200 while the security team is still waiting for budget approval.
That was always going to happen. If one supplier offers bulletproof software but demands a NASA budget and timelines and the other one offers to do it for pocket change and promises, the latter one is chosen.
Yes it is, it is making programmers leave, and it is preventing new programmers from learning the basics of a project.
My wife just told me she that Debian is losing some of it’s most experienced programmers, because code submissions are scanned by stupid LLM that as silly questions about the submissions that MUST be responded to.
Disrupting maintenance absolutely breaks software, and AI is doing that all by itself.
There aren’t even any $200 saved, as Debian is maintained by contributors.And security by obscurity. Keeping source code secret to hide your failures.
Yep, in the age of AI code decompiling, secret source code isn’t going to help you much.
Eh if they can get the binaries off your server somehow, they don’t really need to decompile shit, they can just test their own instance of it
Decompiling is still usefull. It’s easier to find problems in the source code than blindly probing.
A lot of places are using off-the-shelf software that attackers could just buy/download a copy of from the vendor.
this is JAVA owned by ORACLE jesus fucking christ. AI is helping FIND BUGS because well paid well resourced humans make mistakes too.
the luddite lemmings strike again
Refusing to acknowledge that AI helps every sociopathic script kiddie do real damage to the real world doesn’t help either.
IT has been cut to bare bones for a long time, now. Some VP made their cost cutting bonus 10 years ago. Their successor did the same. And their successor. THIS is the result. This is our culture. Make the rich richer now no matter who suffers later.
I work in IT security and I’ve buried many colleagues and friends that died from over-stress. Usually their hearts give out. My best friend died last December. Not that rich C suite executive, though. Now there’s no one left to work late and fix all the problems. They’ve deputized whole divisions to the patching and vulnerability teams, just to stem the bleeding so the companies don’t end up in the news. But as soon as we fixed one, another comes up. I’m not speaking in abstract terms - that was literally my Wednesday. We can’t get ahead of it. A concentrated cyber attack by AI that doesn’t get tied, bored, or blocked will eventually succeed. Our legacy code running Java dependencies on Windows 95 isn’t getting replaced because it’s not in the budget.
I just hope they don’t go for infrastructure.
They will go for infrastructure. There are bad actors and ai capacities enough to attack every system on this planet.
Every system.
I’m sorry for your loss, comrade.
The only ‘AI’ at play is the variant where LLMs infect the Peter Principle: Amplified Incompetence.
It is already broken. Just the cracks come to light.
Waiting for the reveal of the first serious bugs in AMDs and Intel’s management engines. Those tiny OS running on their own core next next to the user’s OS. Or the firmware of the network cards. This will be a fun day for users of older PCs without the chance to receive patches.
It never needed to be like this.
AI may be exposing and exploiting the world’s tech debt. But AI is also the solution to the problem, too.
LLMs now produce code fast enough to make 100% test coverage achievable for every software project.
There are now zero excuses for having under-tested code in production.
Can someone who down downvoted this please explain your perspective for ignorant me?
After having been in the tech industry a couple decades, I have experienced a whole lot of people who get very grumpy when they encounter the truth. People like to lash out when they encounter information that challenges their deeply-held beliefs. Cognitive dissonance is often expressed in funny ways.
What you’re seeing is the kind of large-scale existential crises that only once-in-a-generation technologies can cause. A whole industry of individuals who built their livelihoods on being semi-clever for a living are now being outpaced by semi-competently operated LLMs. Nobody is ever happy to find themselves confronted with adapt-or-die circumstances.
And before anybody objects to my use of the phrase “semi-clever” - we wouldn’t be witnessing a vulnpocalypse if the situation were any different. For better or worse, the incompetence on display by the leadership of the US Federal Government isn’t unique. Most middle-management positions are filled with knuckleheads barely more functional than the current cabinet.
Americans working in the tech sector have been short-changing everybody for so long that “quality craftsmanship” is treated as a four-letter word, uttered in front of your boss only if you’ve already secured a position elsewhere.
All of the epithets I hear leveled at AI Slop are criticisms I saw directed towards incompetent coders on websites like TheDailyWTF over a decade ago. The fact is, LLMs have gotten so good at generating code in the last two years that the entire concept of corporate-managed software services is in the process of dying off - its corpse being cannibalized by millions of LLM-powered startups staffed by single-digits worth of humans.
What we do with these tools is a choice.
AI doesn’t have the skillset and shouldnt be trusted blindly.
Think of it as a beginner dev. It can do shitwork and maybe shows promise but you check every fucking line of code before it goes into production. And reviewing someone else’s code takes time and a very special attention span, so people will try and palm that off to AI as well and it all goes to shit
100 % test coverage and 100 % correctness are very different things, especially if the tests are auto-generated by LLMs at a rate that is impossible to properly review.
Every coder trying to justify their jobs brings that up. And they’re all wrong.
You don’t do 100% correctness already. In terms of correctness, LLMs generating code with 100% coverage will run circles around whatever you do write today. Demonstrably.
You don’t have the winning argument you think you do. You may want to re-read the parable of John Henry.
They’re pointing out the issue is volume.
A person only hitting 60% is gonna get caught in review. A machine doing the same will get lost in a torrent
I understand what they’re pointing out. It’s the wrong thing to fixate on.
A very common cognitive bias in humans is that we tend to over-estimate our own ability and under-estimate things we have limited expeerience with.
The decision to assume I don’t understand what I’m talking about is based on incorrect assumptions about both of us.
If you understand the issue being discussed is volume, then don’t immediately cut to a different track
Thank you for confirming that you’re still making incorrect assumptions.
If you understood what I wrote, you’d realize that I didn’t do the thing you’re accusing me of. You just didn’t understand what I wrote. Your misunderstanding is not my error.

deleted by creator











