Roughly 45% of AI-generated code samples contain a detectable security vulnerability, and that figure has barely shifted across two years of new model releases. That's the headline finding from the largest test of its kind, and it's backed up by four other independent studies using entirely different methods: a controlled human trial, a pull request audit, a secrets-scanning report and enterprise telemetry from some of the largest companies in the world. None of them disagree with each other in any way that matters.
The headline number: Veracode's 45% failure rate
Veracode didn't test one model on one task. The 2025 GenAI Code Security Report ran over 100 LLMs against 80 curated coding tasks spanning four languages, then checked the output against the OWASP Top 10, the industry's standard list of the most common and dangerous web application weaknesses. Across every model, every language and every task, the average security pass rate landed at approximately 55%, which means the code failed 45% of the time.
The detail that should worry anyone shipping Java specifically: it was the worst-performing language in the report, with a failure rate of roughly 70-72%. Python, JavaScript and C# fared better but still clustered in the 38-45% failure range. There was no language where AI-generated code passed reliably.
What makes this finding hard to dismiss as a snapshot of older, weaker models is the trend line. Veracode's report covers two years of model releases, each one marketed as a step change in reasoning and code quality. The security pass rate across that period stayed close to flat. Newer models write more fluent code. They don't write meaningfully more secure code.
The human factor: Stanford's controlled study
A model producing insecure code is one problem. A developer who can't tell is a bigger one, and that's exactly what a Stanford research team set out to measure. In "Do Users Write More Insecure Code with AI Assistants?" (arXiv:2211.03622), Neil Perry, Megha Srivastava, Deepak Kumar and Dan Boneh split participants into two groups: one with access to an AI coding assistant built on OpenAI's Codex, one without. Both groups completed the same set of security-relevant programming tasks, covering things like string encryption, SQL query construction and file path handling.
The group with AI access wrote less secure code than the control group on four of five tasks. That part might be expected. The part that should change how you think about code review is what happened next: participants who used the AI assistant were more likely to believe they had written secure code than the participants who didn't use it at all. The assistant didn't just introduce flaws. It made the people using it more confident, not less, that the flaws weren't there.
That combination, more bugs plus more confidence, is precisely the condition under which vulnerabilities reach production undetected. A developer who suspects their code might be wrong goes looking for a second opinion. A developer who's confident it's fine doesn't.
Beyond security specifically: CodeRabbit's pull request audit
Security flaws don't show up in isolation. They tend to travel with a wider pattern of issues, and CodeRabbit's "State of AI vs Human Code Generation" report (December 2025) measured that pattern directly. The team analysed 470 open-source GitHub pull requests, comparing those with AI co-authorship against human-only submissions.
AI-co-authored pull requests contained roughly 1.7x more issues overall than human-only pull requests. Narrow that down to security-specific issues alone, and the gap widens: up to 2.74x more security issues in the AI-assisted PRs. The same report found AI-generated code was also more likely to contain readability problems and gaps in error handling, which matters because those are usually the first things a rushed reviewer skips over.
Someone still has to check the diff
An AI model can write the code. It can't yet be the second pair of eyes that catches what it missed. vibeharbor.io assigns a named engineer to harden security, stability, performance and deployment on your repo, then hands it back as a reviewed pull request.
See what gets fixed →Where the vulnerabilities end up: secrets in public repositories
One specific, easy-to-measure category of AI-related security failure is hardcoded secrets: API keys, credentials and tokens committed directly into source code. GitGuardian's State of Secrets Sprawl 2026 report, cited by the Cloud Security Alliance's research on AI-generated code, found that AI-assisted commits exposed secrets at a rate of 3.2%, against 1.5% for human-only commits. That's roughly double the leak rate.
At scale, that gap adds up fast. The same report found 28.65 million new hardcoded secrets in public GitHub commits during 2025 alone, a 34% year-over-year increase. Credentials tied specifically to AI service providers rose even faster: up 81% year-over-year, to 1,275,105 detected leaks.
The enterprise picture: a tenfold jump in monthly findings
Smaller studies can be dismissed as unrepresentative of production environments with mature security tooling. Enterprise telemetry closes that gap. Reporting via SecurityWeek on application security data from Fortune 50 companies found that monthly security findings in their codebases rose roughly tenfold between December 2024 and June 2025, climbing from around 1,000 to over 10,000 monthly vulnerabilities. These are organisations with dedicated security teams and established scanning pipelines. The volume of AI-generated code moving through those pipelines outpaced their ability to keep the finding rate flat.
How the five studies compare
| Source | What it measured | Headline finding |
|---|---|---|
| Veracode, 2025 GenAI Code Security Report | 100+ LLMs, 80 tasks, 4 languages, checked against OWASP Top 10 | 45% of code samples introduced a vulnerability |
| Stanford (Perry et al.), arXiv:2211.03622 | Controlled study, developers with vs without an AI assistant | Less secure code on 4/5 tasks, plus false confidence |
| CodeRabbit, State of AI vs Human Code Generation, Dec 2025 | 470 open-source GitHub pull requests | Up to 2.74x more security issues in AI-co-authored PRs |
| GitGuardian, State of Secrets Sprawl 2026 | Secret exposure rate in public GitHub commits | 3.2% leak rate in AI-assisted commits vs 1.5% human-only |
| SecurityWeek (Fortune 50 telemetry) | Monthly security findings, Dec 2024 to June 2025 | Roughly 10x increase, ~1,000 to 10,000+ per month |
Why this isn't improving on its own
Two things are true at the same time, and together they explain the flat trend line. First, per Veracode's data, newer models aren't closing the security gap; pass rates have stayed close to 55% across successive model generations built for better reasoning, not better secure-coding discipline. Second, adoption is accelerating regardless. More code is being written by AI assistants than a year ago, so a flat failure rate applied to a larger volume of output produces exactly the kind of compounding effect SecurityWeek's Fortune 50 data shows: not a worse ratio, just far more total findings.
Layer the Stanford confidence effect on top of that and the picture gets worse. A flat vulnerability rate would be manageable if developers reviewed AI output with proportionally more scrutiny. The research says the opposite happens: people trust the output more, not less, which is also why Snyk's research (reported by Cloudwars) found 75.8% of developers believe AI-generated code is more secure than human-written code, even though more than 56% separately admitted that AI-generated code sometimes or frequently introduced security issues in their own work. Roughly 80% also admitted bypassing security policy at least occasionally when using AI tools. Belief and behaviour are pointed in opposite directions.
Does GitHub Copilot introduce security vulnerabilities?
Copilot-class assistants are covered within the broader category of tools tested across the studies above rather than singled out individually in most of them, but the pattern holds across assistant-generated code generally: Stanford's controlled study using an OpenAI Codex-based assistant found participants wrote less secure code than a control group with no AI access, on four of five tasks.
Is AI-generated code more or less secure than code written by humans?
Less secure, by a consistent margin across independent studies. CodeRabbit's pull request analysis found AI-co-authored PRs contained up to 2.74x more security issues than human-only PRs, and GitGuardian's secrets research found AI-assisted commits leaked credentials at more than double the rate of human-only commits.
Why hasn't AI-generated code security improved as models got better?
Because model improvements have targeted reasoning and fluency, not secure-coding discipline specifically. Veracode's data shows the security pass rate has stayed close to 55% across roughly two years of successive model releases, with no clear upward trend.