The Code Looks Right. That's the Problem.

We ship software with AI in the loop every week. It's part of how a small studio in Greece delivers what used to take a much larger team. So this isn't a piece written by someone standing outside the tools, wagging a finger. It's written by people who use them daily — and who have had to get deliberate about one specific thing.

The code these tools produce almost always looks right. That's the whole problem.

When people worry about AI writing software, they tend to picture obvious nonsense: broken, garbled output a developer would throw away on sight. The reality is more unsettling. AI doesn't write bad code. It writes convincing code — output that compiles, passes a quick read, demos perfectly, and ships. And a meaningful share of it is quietly insecure in ways that only show up when someone goes looking for a way in.

Veracode put this to the test across more than a hundred models. Roughly 45% of AI-generated code samples introduced one of the OWASP Top 10 vulnerabilities — the industry's standard list of the most common, most exploited weaknesses. The uncomfortable part: that number has barely moved across testing cycles from 2025 into 2026, even as the models got dramatically more capable at everything else.

The old mistakes, at new speed

These aren't exotic, futuristic bugs. They're the same mistakes the industry has been making for twenty years — cross-site scripting, SQL injection, secrets hard-coded where they never belonged. What changed is the volume and the velocity. One analysis comparing AI-assisted code against human-written baselines found roughly 2.74 times more vulnerabilities in the AI output. The machine didn't invent new ways to be insecure. It industrialized the old ones.

The velocity trap

Here's the part that's easy to miss, and it matters most: the speed that makes these tools worth using is the same thing that makes them risky.

Teams adopting AI assistants report generating code several times faster — and then finding on the order of ten times more security issues downstream. Review cycles stretch. Post-merge fixes pile up. You feel faster on Monday and quietly pay for it on Thursday.

And the human factor makes it worse. In a controlled study, developers using an AI assistant not only wrote less secure code — they were more confident it was secure. That's the trap in a sentence. The output looks so finished that we stop applying the scrutiny we'd give our own work.

The tool doesn't just generate code. It generates the feeling of being done.

Slopsquatting: a threat that didn't exist three years ago

Then there's a problem almost nobody outside security circles has heard of. It has an unserious name — slopsquatting — and a very serious mechanism.

AI models don't only write weak code. Sometimes they invent things that were never real. Ask one to write a script and it may confidently import a software package that doesn't exist — a plausible-sounding name it essentially made up. Researchers tested sixteen models across roughly three-quarters of a million code samples and found that about one in five referenced a package that wasn't real.

On its own, that's just a failed build. The dangerous part is the pattern: the models don't hallucinate at random. They invent the same fake names, over and over. Attackers noticed. The playbook is cheap and patient — watch which package names AI tools routinely hallucinate, register those exact names on the public software registries, and fill them with malware. Then wait. Every developer who trusts the suggestion and installs the package is now running the attacker's code.

This is not hypothetical. One hallucinated package mimicking a real tool quietly accumulated over thirty thousand downloads in three months — helped along when its bogus installation command got copied into trusted documentation, lending it a legitimacy it never earned.

Sit with what's new here. Traditional supply-chain attacks rely on typos and luck. Slopsquatting relies on neither.

It's a feedback loop in which our trust in the tool is the vulnerability — the more we lean on AI to fill in the gaps, the more reliably it points us at the attacker's door.

So what do you actually do

The wrong lesson — and the easy one — is "don't use AI." That's neither realistic nor honest. Nearly every developer alive already uses these tools, us included. Banning them is a fantasy.

The right lesson is that AI quietly moved the job. For decades, the hard part of building software was writing it. Now the writing is cheap, and the hard part is judging it — reading critically, catching the flaw the demo didn't surface, verifying that the package you're about to install is real, knowing which corners cannot be cut. The scarce skill is no longer typing code. It's the discipline around the machine: scanning what it produces, pinning and checking dependencies, and refusing to ship something just because it looks finished.

That's what "production-grade" has come to mean. It used to imply something about effort — that a human sweated over every line. Now it implies something about judgment — that someone with the experience to be suspicious actually looked. The tools got faster. The standard didn't get easier. If anything, the part that separates real software from convincing software is more human than it has ever been.

The machine will keep writing code that looks right. Deciding whether it actually is — that's still the job. And it always will be.

Sources & further reading

Veracode — security risks in AI-generated code (OWASP Top 10 and the 2.74× figure): https://www.veracode.com/blog/ai-generated-code-security-risks/

FOSSA — slopsquatting and AI package hallucinations: https://fossa.com/blog/slopsquatting-ai-hallucinations-new-software-supply-chain-risk/

Trend Micro — slopsquatting when AI agents hallucinate malicious packages: https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatting-when-ai-agents-hallucinate-malicious-packages

Cloud Security Alliance — research note on the AI-generated supply-chain risk: https://labs.cloudsecurityalliance.org/research/csa-research-note-slopsquatting-ai-supply-chain-20260419-csa/

• AI
• software engineering
• security
• supply chain