AI-Built App Security Checklist: What a Developer Checks First

An app built with ChatGPT, Claude, Lovable, Bolt, Replit or v0 can look finished long before it is safe to put in front of customers. The screens work, sign-up works, and the data saves. What a demo rarely shows is who else can read that data, which keys ship to every visitor’s browser, and what happens when someone sends the app a request its interface would never send.

This is the checklist a developer works through first when taking over an AI-built or vibe-coded app, in roughly the order it gets done. Each check covers what the problem looks like in the code, why it matters, and what the official documentation for the tools involved says. For the wider handoff, including getting the code into a repository you own, see You Built It With AI. Here’s How a Developer Takes It Over.

AI-built app review: key facts

Question	Answer
What gets checked first?	Exposed secrets, database access rules, server-side authorization and sign-in, dependencies, input validation, and error handling.
Why start with secrets?	A leaked secret key can give full access to the database or payment account, and removing it from the code does not undo the leak. GitHub’s documentation says the first step is to revoke or rotate it.
Is a key in the browser always a problem?	No. Supabase publishable keys and Stripe publishable keys are designed to be public. Secret keys are not.
What is row-level security?	Per-row access rules enforced by the database itself. Supabase’s documentation says to “Enable RLS on every table in an exposed schema.”
Are AI tools known to invent packages?	Yes. A USENIX Security 2025 study of 16 models found invented package names in at least 5.2% of packages from commercial models and 21.7% from open-source models, on average.
Which standard does this follow?	The checks map to the OWASP Top 10:2025, especially A01 Broken Access Control, A03 Software Supply Chain Failures, A05 Injection, A07 Authentication Failures and A10 Mishandling of Exceptional Conditions.
Does this apply to code from ChatGPT or Claude, not just app builders?	Yes. The checks cover the code and the services it connects to, whichever tool wrote it. App builders add their own setup, such as a Supabase database the browser can query directly.
Can an owner check any of this alone?	Some of it. Searching the browser’s loaded code for secret key prefixes and testing with two separate accounts are good first steps.

Why does an AI-built app need a security review?

Research on AI-written code has repeatedly found programs that work and can still be exploited. In BaxBench, a benchmark of 392 backend tasks presented at ICML 2025, researchers report that “we could successfully execute security exploits on around half of the correct programs generated by each LLM.” The models in that study date from early 2025, and newer ones may do better, but the risk shows up in a specific place. Whether the code came from a ChatGPT or Claude conversation or from an app builder, it gets judged by what its owner can see working, and access rules, secrets and error paths are invisible from there.

People using AI assistants also tend to overrate the result. In a user study published at ACM CCS 2023, Do Users Write More Insecure Code with AI Assistants?, participants with an AI assistant “wrote significantly less secure code than those without access” and “were more likely to believe they wrote secure code.” Confidence in AI-generated code is a poor guide to its safety, which is why the first pass looks at the code itself.

AI-built app security checklist at a glance

#	Check	What it looks like in an AI-built app	OWASP Top 10:2025
1	Exposed secrets and API keys	A secret key in a VITE_ or NEXT_PUBLIC_ variable, a committed .env file, or a key that lives on in Git history	A02 Security Misconfiguration
2	Database rules and row-level security	Tables with RLS off, policies that allow everyone, or Firebase rules left at open access	A01 Broken Access Control
3	Authentication and authorization	Admin pages hidden only by the interface, server functions that never check who is calling, unverified payment webhooks	A01 Broken Access Control, A07 Authentication Failures
4	Dependencies	Packages that don’t exist, aren’t needed, or have known vulnerabilities	A03 Software Supply Chain Failures
5	Input validation	Checks that run only in the browser, SQL built by joining strings, user content rendered as raw HTML	A05 Injection
6	Error handling	Database errors shown to users, empty catch blocks, payments that half complete	A10 Mishandling of Exceptional Conditions

1. Are API keys or other secrets exposed?

Everything the front end of a web app uses is downloaded to the visitor’s browser, where anyone can read it. Build tools make it easy to put a value there by accident. Vite’s environment variable guide states that “Variables prefixed with VITE_ will be exposed in client-side source code after Vite bundling,” and that “VITE_* variables should not contain sensitive information such as API keys.” Next.js, the framework v0 generates, does the same with the NEXT_PUBLIC_ prefix: its documentation says such a value “will be inlined into any JavaScript sent to the browser.”

What it looks like: a .env file with VITE_SUPABASE_SERVICE_ROLE_KEY, NEXT_PUBLIC_STRIPE_SECRET_KEY or an OpenAI key, or a server key pasted directly into a front-end file after ChatGPT, Claude or an app builder was asked to “make the API call work.”

Why it matters: some keys are meant to be public and some are not, and the two kinds grant very different access. Supabase’s API keys guide describes its publishable key as “Safe to expose online,” while the secret key “bypasses Row Level Security, so it must never leave your control.” Stripe’s API keys documentation says “Only publishable keys are safe to expose outside your application’s backend.” A secret key shipped to the browser hands every visitor the same access the app’s own server has. OWASP’s Security Misconfiguration guidance recommends short-lived credentials or platform-provided roles “instead of embedding static keys or secrets in code, configuration files, or pipelines.”

What a developer does:

• Searches the code and the built JavaScript for key prefixes. Stripe secret keys start with sk_live_ (or sk_test_ in a sandbox), and Supabase’s current secret keys start with sb_secret_.
• Searches the full Git history, not just the latest version. GitHub’s secret scanning “scans your entire Git history on all branches of your repository,” and on public repositories it “runs automatically for free.”
• Revokes or rotates anything that leaked before cleaning up. GitHub’s guide to removing sensitive data says “as a first step you need to revoke and/or rotate that secret.”
• Moves calls that need a secret into server code, such as an API route or an Edge Function.

One sign of older generated code: Supabase notes that “If a tool, tutorial, or AI assistant tells you to copy a long key that begins with eyJ, it was written for the legacy keys.” Supabase plans to retire the legacy anon and service_role keys, with a deadline at the end of 2026, so an app still using them needs a planned switch to the new keys.

2. Can one user read another user’s data?

Supabase, which Lovable and Bolt both integrate with, lets the browser query the database directly. That design is only safe when the database enforces its own rules. Supabase’s row-level security documentation is direct about the default: “A table in an exposed schema without RLS is readable and writable by any role with a grant on it.”

What it looks like:

• A table with row-level security turned off.
• A policy that technically exists but allows everyone. Supabase warns that a read policy for the anon role using true “grants every unauthenticated visitor read access to every row the role can already reach through grants.”
• A policy that checks only whether someone is signed in, when it should check whether the row belongs to them. Supabase’s own examples compare the row’s owner to auth.uid(), the signed-in user’s ID. Signed in can mean less than it sounds: Supabase notes that an anonymous user “assumes the authenticated role.”
• A database view over a protected table. The same page explains that “Views bypass RLS by default,” so “A view over a protected table hands out every row its policies were meant to withhold.”
• In a Firebase app, rules left at open access. Firebase’s guide to fixing insecure rules shows allow read, write: if true; and warns that without user authentication and security rules, “anyone who guesses your project ID can steal, modify, or delete the data.”

Why it matters: this is broken access control, ranked first in the OWASP Top 10:2025. OWASP reports that “100% of the applications tested were found to have some form of broken access control.” It has also affected AI-built apps specifically. The U.S. National Vulnerability Database lists CVE-2025-48757, scored 9.3 (Critical) by MITRE, which describes “an insufficient database Row-Level Security policy in Lovable through 2025-04-15” that let unauthenticated attackers read or write database tables of generated sites. Lovable disputes the entry, and the record gives its position: “each individual customer of the Lovable platform accepts a responsibility over protecting the data of their application.” Either way, the app owner is the one whose customers’ data is at risk.

What a developer does: lists every table and view, confirms RLS is on for each, reads every policy, and tests them. A practical test is to create two ordinary accounts and confirm that neither can read, change or delete the other’s records, including by calling the database API directly instead of using the app’s screens. Supabase’s API keys guide also points to the project’s Security Advisor, which “checks for common security problems with the built-in Postgres roles.”

3. Are sign-in, permissions and payments enforced on the server?

An interface can hide what a user shouldn’t do without preventing it. The admin link may be missing for regular users while the admin page and its data still load for anyone who types the address. OWASP’s description of broken access control includes the case where “An application puts all of their access control in their front-end,” and states that “Access control is only effective when implemented in trusted server-side code or serverless APIs, where the attacker cannot modify the access control check or metadata.”

What it looks like:

• Admin features gated by a check in a React component and nothing on the server.
• Server functions that accept a record ID and act on it without checking who owns it. Next.js’s data security guide warns that an exported Server Action “is reachable via a direct POST request, not just through your application’s UI,” and says to “verify authentication and authorization inside each one.”
• A custom login system written from scratch, when the app already uses a hosted authentication service.
• Sign-in, sign-up and password reset messages that reveal whether an email address has an account. OWASP’s Authentication Cheat Sheet says an application “must respond with a generic error message” on login, password reset and password recovery.
• No limit on login attempts. OWASP’s Authentication Failures category includes apps that permit “brute force or other automated, scripted attacks that are not quickly blocked.”
• A payment webhook that trusts whatever arrives. Stripe’s webhook documentation uses the Stripe-Signature header and a signing secret to confirm each event came from Stripe, and notes that endpoints “might occasionally receive the same event more than once.” An unverified endpoint can be sent a fake “payment succeeded” event. One that doesn’t track event IDs can fulfill the same order twice.

Why it matters: these flows can work perfectly on the happy path, which is what clicking through the app tests. The failures appear when someone deliberately calls the server out of order, with another user’s ID, or without paying.

What a developer does: traces every server endpoint and confirms it checks both who the caller is and whether they may act on that specific record. For sign-in, OWASP’s Authentication Failures guidance recommends using “a premade, well-trusted system” for authentication and sessions where possible, so a developer may replace hand-written auth code with the provider the app already has. For payments, they confirm webhook signatures are verified and repeated events are ignored.

4. Are the dependencies real, needed and current?

AI tools add packages as they go, and some of the names they suggest don’t exist. A study presented at USENIX Security 2025, We Have a Package for You!, generated 576,000 code samples with 16 models and reports that “the average percentage of hallucinated packages is at least 5.2% for commercial models and 21.7% for open-source models,” with 205,474 unique invented names. An attacker who registers one of those names on a package registry such as npm can wait for the next app that installs it.

What it looks like: a package.json with packages nobody can explain, two libraries doing the same job, packages with almost no download history, versions several majors behind, or no lockfile in the repository at all.

Why it matters: OWASP ranks this risk third, as A03 Software Supply Chain Failures, and it topped OWASP’s community survey, where exactly 50% of respondents ranked it first. Its example scenarios include the 2025 Shai-Hulud attack, described as “the first successful self-propagating npm worm,” which “reached over 500 package versions before being disrupted by npm.” OWASP’s prevention list includes “removing unused dependencies” and tracking transitive dependencies, the packages your packages install.

What a developer does: confirms each package is the one intended, maintained and actually used; removes the rest; commits the lockfile; and runs npm audit, which per the npm documentation “submits a description of the dependencies configured in your project to your default registry and asks for a report of known vulnerabilities.” Generated code also tends to target older library versions, a pattern covered in AI Code Generation Has a Version Problem.

5. Is input validated on the server?

Form validation in the browser improves the experience, but on its own it doesn’t protect the server. OWASP’s Input Validation Cheat Sheet says validation “must be implemented on the server-side before any data is processed by an application’s functions, as any JavaScript-based input validation performed on the client-side can be circumvented by an attacker who disables JavaScript or uses a web proxy.”

What it looks like:

• A required field or price limit enforced only in the form, while the API route saves whatever it receives.
• SQL assembled by inserting user input into a string. OWASP’s Injection category shows how an attacker can enter ' OR '1'='1 into such a query to return every record in a table.
• User-supplied text rendered with React’s dangerouslySetInnerHTML. React’s documentation warns that if that HTML “isn’t trusted (for example, if it’s based on user data), you risk introducing an XSS vulnerability.”
• File uploads with no checks on type or size.

Why it matters: OWASP ranks injection fifth and notes that “100% of applications” in its data were tested for some form of it. Cross-site scripting accounts for “more than 30k CVEs” and SQL injection for “more than 14k CVEs.”

What a developer does: adds server-side validation for every endpoint that accepts data (for example, with a schema library that also produces clear error messages), replaces string-built queries with parameterized queries or the database client’s query builder, and sanitizes or removes raw HTML rendering.

6. How does the app handle errors?

Clicking through an app mostly exercises the case where everything succeeds. OWASP added a new category for the other case in 2025, A10 Mishandling of Exceptional Conditions, grouping 24 weakness types (CWEs) that cover “improper error handling, logical errors, failing open, and other related scenarios.”

What it looks like:

• Raw database or stack trace messages returned to the user. One of OWASP’s A10 scenarios is a database error that “reveals the full system error to the user,” which an attacker then uses to refine an injection attack.
• Empty catch blocks that swallow errors, so failures leave no trace.
• A multi-step action, such as charging a card and then creating an order, that leaves half its work done when the second step fails. OWASP’s advice is to “roll back every part of the transaction and start again (also known as failing closed).”
• No error monitoring, so the owner learns about failures from customers.

Why it matters: error messages are reconnaissance for attackers, and silent failures turn into lost orders and corrupted records nobody notices. OWASP’s Error Handling Cheat Sheet describes the target: when an unexpected error occurs, “a generic response is returned by the application but the error details are logged server side for investigation, and not returned to the user.”

What a developer does: adds a global error handler, returns generic messages to users, logs details on the server with an alert for repeated failures, and wraps multi-step writes in database transactions.

What order do the fixes go in?

The findings get fixed in order of damage, not in checklist order:

1. Rotate exposed secrets. Until a leaked key is revoked, every other fix can be bypassed with it.
2. Close data access. Turn on and correct row-level security or database rules, and remove open policies and views.
3. Add server-side authorization to every endpoint and server function, and verify payment webhooks.
4. Clean up dependencies, then validation and error handling.

After that comes the work that keeps the app safe as it changes: tests for access rules (Supabase’s RLS guide suggests SQL tests that assert allow and deny for each operation and role), a review step for new changes, and the decision about which parts to keep and which to rebuild. The pillar guide covers whether to fix an AI-built app or start over.

Can you check an AI-built app’s security yourself?

A few checks need no developer and are worth doing before real users sign up:

• Search the code your browser loads. Open the live app, open the browser’s developer tools, and search the loaded sources for sk_live_ and sb_secret_. A match means a secret is exposed and should be rotated right away. Older Supabase keys are long strings beginning with eyJ, and telling a public one from a secret one means decoding the key, a quick job for a developer.
• Try two accounts. Sign up twice with different email addresses and check whether one account can see anything that belongs to the other.
• Open the database dashboard. In Supabase, check whether row-level security is enabled on every table and review what the Security Advisor reports.
• Check the repository. Look for a committed .env file and turn on GitHub secret scanning.

These catch the most damaging problems, not all of them. Authorization in server code, webhooks and dependency risk still need someone who reads the code.

When to bring in a developer

The right time for a first-pass review is before real users, personal data or payments go live, or as soon as one of the checks above turns something up.

Connor on the Web reviews, secures and maintains web apps, including ones started with ChatGPT, Claude, Lovable, Bolt, Replit or v0. Our app development process and pricing are published, and you can get in touch to talk through what you’ve built.