Short answer
AI-generated code is not safe by default: Veracode found 45% of samples from over 100 models introduced an OWASP Top 10 flaw. It can be shipped safely when the pipeline assumes that, with security scanning, dependency checks, sandboxed agents and no production access.
Your team already ships AI-written code. Maybe it is a few autocomplete lines a day, maybe agents open whole pull requests. Then someone from security, or a client’s procurement team, asks the obvious question: is that code safe? Nobody has a good answer, because nobody wrote down what “unsafe” would mean.
The short version is no, not by default. The longer version is that “AI code” is not one risk. It is about six, with different odds and different damage, and each one has a control that works. A threat model turns a vague worry into a list you can work through.
What the security studies found
The best-known number comes from Veracode’s 2025 GenAI Code Security Report. They gave more than 100 language models 80 curated coding tasks and scanned the results. In 45% of cases, the generated code introduced a vulnerability from the OWASP Top 10. When a task offered a choice between a secure and an insecure way of doing something, the models picked the insecure one 45% of the time.
The spread by language is wide.
Some weakness classes were far worse than the average. Models failed to defend against cross-site scripting (CWE-80) in 86% of relevant samples, and against log injection (CWE-117) in 88%. Two findings matter more than the headline: larger models did not do significantly better than smaller ones, and while newer models got better at writing code that works, security performance stayed flat.
None of this is new. In 2021, Pearce et al. generated 1,689 programs with GitHub Copilot across 89 security-relevant scenarios and found about 40% vulnerable. Four years of model progress made the code more functional, not more secure.
There is a human factor too. In a Stanford user study, Perry et al. found that participants with an AI assistant wrote significantly less secure code than those without, and were more likely to believe it was secure. The tool lowers quality and raises confidence at the same time.
Six risks in a threat model for AI code
Here are the risks I would put on the board for any team shipping AI-written code, with the evidence and the main control for each.
| Risk | What happens | Main control |
|---|---|---|
| Insecure patterns | Injection, XSS, weak crypto, missing auth checks written in plausible code | Static analysis (SAST) on every change, security criteria in the review |
| Hallucinated dependencies | The code imports a package that does not exist, and an attacker registers it | Dependency allowlist, registry proxy, install only from a lockfile |
| Prompt injection into the agent | Text in an issue, web page or file tells the agent to do something else | Treat all inputs as untrusted, sandbox, no secrets in reach |
| Excessive agency | The agent has permissions it doesn’t need and uses them | Least privilege, no production credentials, approval for irreversible actions |
| Secrets exposure | Keys pasted into prompts, logged, or committed by the agent | Secret scanning in pre-commit and CI, short-lived scoped tokens |
| Review fatigue | Bigger diffs, more of them, skimmed by tired people | Small diffs, automated evidence, an independent reviewer model |
Two of these come straight from the OWASP Top 10 for LLM applications: LLM01 prompt injection and LLM06 excessive agency. The dependency risk is large enough to get its own article. In a USENIX Security 2025 study of 576,000 code samples, 19.7% of recommended packages did not exist.
Review fatigue is the one people underrate. Faros AI’s data on about 10,000 developers shows high-AI teams merging 98% more PRs, each 154% larger, with review time per PR up 91%. Security review is part of that review. If it is done by people, it gets thinner exactly when the volume goes up.
Likelihood and impact: where to start
Not all six deserve the same effort on day one. I place them by how often I’d expect them on a typical product team and how bad a single occurrence is. This is my judgement, not measured data, but it is a reasonable default to argue with.
Risks of AI-written code, by likelihood and impact
My assessment for a typical product team
Data as a table
| Item | Impact of one occurrence | How often it happens |
|---|---|---|
| Insecure patterns | Severe | Common |
| Review fatigue | Severe | Common |
| Hallucinated API calls | Contained | Common |
| Hallucinated packages | Severe | Rare |
| Prompt injection | Severe | Rare |
| Excessive agency | Severe | Rare |
| Secrets in commits | Severe | Rare |
Insecure patterns sit top right: common and serious. That is where scanning and review effort should go first. Hallucinated API calls are common but contained, because the build or the tests usually catch them. Prompt injection and excessive agency are rarer today, but a single case can be catastrophic, so they get hard limits rather than better review.
A public example of the bottom-right corner: in July 2025, a Replit agent deleted a company’s production database during a declared code freeze, covering more than 1,200 executives and over 1,190 companies, according to Fortune. The agent had access it should not have had, and an instruction in the chat was the only thing standing between it and production. Replit’s response was structural: automatic separation of development and production databases, and a planning-only mode.
A pipeline that assumes AI code is unsafe
The useful mindset is the one you would apply to code from an unknown contributor. Not hostile, just unverified. Every change passes the same gates, whoever or whatever wrote it.
Security gates for an AI-written change
- Agent
- Automated check
- Person
Steps as text
| Step | What happens |
|---|---|
| 1. Agent in sandbox | Agent — no prod access, no secrets |
| 2. Dependency check | Automated check — exists, allowlisted, pinned |
| 3. SAST and secrets scan | Automated check — blocks on high severity |
| 4. Reviewer model | Agent — security criteria in the prompt |
| 5. Person | Person — only for risky areas |
| Loop | findings go back: back from step 4 to step 1 |
What each gate does in practice:
- The agent works in a sandbox. It can read and write its working copy and reach the package registry through a proxy. It cannot see production credentials, customer data or your SSH keys. If a prompt injection succeeds, it succeeds inside a box with nothing valuable in it. Guardrails for AI coding agents covers permissions, sandboxes and budgets in detail.
- New dependencies are checked mechanically. Does the package exist? Is it on the allowlist, or older than a set number of days with real maintainers? Is it pinned in the lockfile? A script answers this better than a reviewer.
- Static analysis and secret scanning block the merge. Use the scanners you already have. The point is that they run on every change and a high-severity finding stops it, not that they produce a report someone reads later.
- A reviewer model reads the diff against security criteria. Name the classes you care about in the prompt: input validation, output encoding, authorisation on every endpoint, log sanitisation. Veracode’s worst classes (XSS, log injection) are exactly the ones a targeted second read catches well.
- A person reviews only the risky areas. Authentication, payments, permission models, cryptography, data deletion. Path-based rules route those changes to a human. Everything else ships on the evidence from the gates.
Bad and good answers to the security question
Some answers I hear to “is our AI code safe?” and what I think of them.
“We use the best model.” Weak. Veracode found model size made no significant difference to security, and security did not improve as models got better at other things.
“Developers review everything.” Weak at volume. The Faros numbers show what volume does to review time, and the Perry study shows that people using AI overestimate how secure their code is.
“We don’t let AI write security-critical code.” Partly right, but hard to enforce by policy alone. Enforce it by path: changes under auth/ or billing/ require a human approver, whoever wrote them.
“Every change, human or AI, passes the same automated security gates, and agents run with no production access.” That is the answer I would want to hear. It does not depend on anyone being careful on a Friday afternoon.
The bold version: security review without a person
The obvious next step is to remove the human gate for most changes entirely. I think that is reachable, under stated conditions: the scanners block on high severity, dependencies are allowlisted, the agent never holds production credentials, the reviewer model has security criteria and a track record of catching seeded flaws, and a person still owns the small set of high-risk paths.
Under those assumptions, the residual risk for a routine change is set by the gates, not by how attentive a reviewer happens to be. That is a better position than most teams are in today with human-only review. Getting there is a staged process, which removing people from code review lays out.
So: is AI-generated code safe? Not on its own. A pipeline that treats it as unverified and checks it the same way every time can be. If you want that set up for your team, write to me.
Questions
Is AI-generated code secure?
Not by default. Veracode tested over 100 models on 80 tasks and found that 45% of the generated samples introduced an OWASP Top 10 vulnerability, with Java failing 72% of the time.
Which security flaws are most common in AI-generated code?
In Veracode’s 2025 tests, cross-site scripting and log injection were the worst, with models failing to defend against them in 86% and 88% of relevant samples. Injection flaws and missing input validation are the general pattern.
Are newer AI models more secure at writing code?
Not much. Veracode reported that newer models got better at producing working code while security performance stayed flat, and that larger models did not perform significantly better than smaller ones.
How do you make AI-generated code safe to ship?
Run every change through the same automated gates: sandboxed agents without production access, dependency checks, static analysis and secret scanning that block the merge, and an independent reviewer with security criteria. Keep a person on high-risk areas like authentication and payments.