How RapidNative Validates AI-Generated Code Before Preview

The most dangerous AI code failures aren't the loud ones. A syntax error throws immediately, a broken import fails to compile, a null-pointer crash sends a stack trace. Those are easy — the code refuses to run and you know something is wrong.

The dangerous failures are the quiet ones. A migration applies successfully but leaves a table nobody can read. A screen renders but the module graph never re-required the new route. A seed row is inserted twice under different foreign keys, silently corrupting reference data. To the model, every one of these looks like success. To the user, the app just doesn't do what it should — with no error anywhere to grab onto.

AI-generated code validation, when done well, has to catch the quiet failures too. Not just the ones that throw. At RapidNative — the platform that turns natural-language prompts into production-ready React Native and Expo apps — we've built the validation pipeline around a single principle: never trust the text the model writes. Execute it, then inspect the result.

This post walks through the concrete guardrails we run on every AI-generated migration and screen: the in-memory Postgres sandbox that dry-runs SQL before it touches disk, the lint rules that catch the eight silent failures we see most often, the feedback loop that returns lint errors to the model so it can self-correct, and the dev-only tooling that catches preview-pipeline failures no test suite would notice.

Every generated migration runs against a real Postgres engine before it reaches the project — Photo by Markus Spiske on Unsplash

Why "It Compiled" Isn't Validation

If you've built anything on top of an LLM, you know the frustrating class of bug: the code the model wrote looks reasonable, passes type-checking, doesn't throw at runtime, and still produces a broken product. Common examples we see constantly:

• A migration enables Row Level Security (RLS) on a table but forgets to add a policy. Every query now returns zero rows — no error, no warning, just an empty screen.
• A seed script inserts the same primary-key value twice, but the table has no primary key constraint, so Postgres happily stores both rows. The app reads back inconsistent data.
• A new screen file appears in the routes folder, but hot-reload has no accept boundary for a module nothing imports yet, so the app doesn't pick it up. The user taps the link and sees Expo's "Unmatched Route."
• A layout component is wrapped in a new provider — with an outer function named RootLayout, the same name as the template's existing default export. Now the file has a duplicate declaration and infinite recursion. In many cases this even survives type-checking.

Every one of these passes a naive "did it compile" check. A validation pipeline that only runs tsc or eslint — or worse, only trusts the model's own claim that the code is correct — will ship them straight into the preview.

At RapidNative, we treat the model's output the same way you'd treat a pull request from an unknown contributor: assume nothing. Verify by running.

Principle One: Validate by Executing, Not by Parsing

The temptation with SQL is to parse it. Look for CREATE TABLE, look for ALTER, walk the AST, decide what the migration will do. This works until the day it doesn't — a subquery, a DO block, a Postgres-specific USING clause, and your parser silently disagrees with what the database will actually execute.

Our SQL validation layer takes a different route: it runs an in-memory Postgres fork (@tinbase/pg-mem, a Supabase-flavoured extension of pg-mem) and applies every migration to it in order, inside a single session. Then it introspects pg_catalog and information_schema to see what actually got created. Nothing about "what the SQL means" is inferred from the text — it's read from the resulting database state.

That gives us a set of properties that cheaper approaches can't touch:

• Statement-level failures are recorded, not thrown. A migration that fails to apply is stored with its error message and simply reported back to the tool caller. The next tool invocation sees the failure inline instead of silently rendering a broken schema.
• Cache invalidation is content-addressed. The migration cache key includes both the file path and the SQL body of every migration in the run. Edit an old migration in place, and the cache correctly rebuilds — no stale state.
• Nothing hits disk until the sandbox says yes. For any new migration, the SQL is executed on pg-mem before the file is written. If it errors, the engine is discarded, the file is never created, and the tool returns an error the model can react to on its next turn.

The last property is the one that took us the longest to get right. An earlier version wrote the file first and validated afterward, which meant a broken migration could reappear on the next tool call because the file was already on disk. Reversing the order — validate, then write — closed the loop.

Dry-run validation happens on an in-memory Postgres, not against production — Photo by Taylor Vick on Unsplash

Principle Two: Read the Catalogue, Not the SQL

Once a migration has applied to the in-memory database, the lint layer starts asking questions of the catalogue. Every check reads a fact from pg_catalog or information_schema — none match against SQL text. Here are the eight lint rules we run on every AI-generated migration, and the concrete pitfall each one catches.

1. RLS enabled with no policy → ERROR. If a table has RLS on but zero policies, every query returns zero rows and the app looks broken with no error surfacing anywhere. This is the highest-impact silent failure we've ever seen from an LLM, and it's the reason our system prompt requires RLS and at least one policy in the same migration.

2. Missing primary key → ERROR. Without a PK, duplicate rows can be inserted silently. Our seed detector then can't tell whether a duplicate is intentional (the same demo row inserted by two seeders) or a bug. Requiring a PK on every table upstream makes the whole seed pipeline sound.

3. Duplicate seed IDs → ERROR. Even with a PK, the seed file can contain two INSERT statements that would violate uniqueness once combined. Our validator snapshots primary-key values before and after each new migration, and reports any newly-introduced duplicates. The check runs against the full applied set, not just the new migration, so it's order-independent.

4. Foreign keys without a leading index → WARNING. Postgres does not automatically index foreign keys. A missing index doesn't break correctness, but it turns WHERE user_id = $1 from an index scan into a sequential scan. We warn on every FK column that isn't the first column of any index.

5. World-writable policies → WARNING. A policy with using (true) on INSERT/UPDATE/DELETE lets any client mutate any row. Because the Supabase anon key ships inside the mobile app, this effectively opens the table to the internet. We surface the finding with a concrete suggestion (usually auth.uid() = user_id).

6. updated_at without a trigger → WARNING. Convention says an updated_at TIMESTAMP column should be maintained by a BEFORE UPDATE trigger. The model sometimes creates the column and forgets the trigger, leaving the column pinned at its insert value forever.

7. Unreferenced user_id column → WARNING. A user_id column with no policy that references it is almost always an incomplete RLS setup — the model created the column intending scoping but forgot to write the policy that would enforce it.

8. Silent SQL apply failures → ERROR. As mentioned above, a migration that fails to apply is captured with its error message and reported. The cache invalidates so the next attempt gets a fresh slot; the failed timestamp isn't consumed.

Two of these — RLS-with-no-policy and duplicate seed IDs — account for the majority of "silent" failures we see, so they earned the ERROR level and block the migration outright. The rest fire as warnings with concrete remediation text.

Principle Three: Feed Failures Back to the Model

A validation layer that only says "no" is a rejection queue. A validation layer that says "no, and here's exactly what to fix" is a training loop.

Our agent's tool interface returns lint results in a structured shape: ok: true, needsFix: true, lints: [...] for warnings that don't block the write, and ok: false, error: ... for hard failures where nothing was written. Both shapes go back into the model's context on the next turn. The harness that runs the agent (also used for testing) records the exact feedback text the model saw, which is the single most valuable field for debugging why the model made a particular choice.

Two things about this loop matter more than they should:

The feedback has to be specific. "Migration failed" is useless. "Table workouts has RLS enabled but no policies — add a policy in the same migration; for user-scoped data, use auth.uid() = user_id" is a fixable error. Our lint text is written to be actionable, not diagnostic — every warning names the table, the column, the rule violated, and the concrete SQL to add.

Warnings still write the file. If we blocked on every warning, the model would get stuck iterating on one migration forever. Instead, warnings mark the file as needsFix: true and let the model fix things forward in a follow-up migration. This mirrors how a human developer would work — commit the migration, notice the missing index in review, ship a follow-up.

Lint failures return to the model as structured feedback so it can self-correct — Photo by Danielle MacInnes on Unsplash

Principle Four: Constrain the Toolset Before You Constrain the Prompt

Every LLM safety guide recommends system-prompt guardrails ("don't do X"), and we have those. But prompt-level constraints alone are guidance the model can ignore. Tool-level constraints are guarantees. Our agent's tool surface is designed so that a whole class of pitfalls is unrepresentable.

• SQL-first schema. The model can't write a TypeScript schema file. Its only way to modify the database is db_migration_new({ name, sql }). There is no tool that patches types.ts directly — that file is regenerated from the applied schema after every migration. This eliminates schema drift entirely.
• Read-only queries are a separate tool. db_sql runs against the sandbox and cannot mutate. The model can inspect the schema freely without any chance of writing to it.
• Seed rows are not migrations. Every row lives in supabase/seed.sql via db_seed. Migrations that mix DDL and INSERT statements are called out explicitly in the system prompt — the tool naming reinforces this.
• File system access is restricted. The file tools the agent gets don't include delete or overwrite operations. This is a defence against the model deciding to "clean up" a file it doesn't understand.
• No environment or secrets access. The agent can't read .env, can't write to it, can't reference environment variable names. Supabase project keys live in Project Settings, and the runtime injects them — the model never sees them.

Combined, the tool surface makes several dangerous classes of output structurally impossible instead of prohibited-by-request. This matters more than any prompt phrasing.

Principle Five: Watch the Preview, Not Just the Code

Even when the generated code is correct, the pipeline that ships it to the running app can fail. The React Native/Expo preview is a bundler (Metro), a hot-reload channel, an iframe, and a route-context module that lists the app's screens. Any one of those four can break in a way that leaves valid code sitting on disk and a blank screen in the browser.

The four failure modes we've seen most often, and how we catch each:

Brand-new routes require one document reload. Rewriting the route-context file (__expo_ctx.js) doesn't force the running app to re-require a file that didn't exist when the bundle first loaded. HMR emits hot update (0 module(s)) because there's no accept boundary for a module nothing imports yet. We now detect the "new file added to routes" case and issue a single document reload — but only once, not repeatedly.

Repeated mid-stream reloads are destructive. Each restart discards partial render. We saw measurable cases where multiple reloads during a stream produced a blank preview that a single reload wouldn't have. The reload scheduler now debounces aggressively.

Two path-root conventions coexist. The Metro bundler labels files as /app/(app)/index.tsx (Metro-root relative). The workspace labels the same file as mobile/app/(app)/index.tsx (workspace relative). Comparing them naïvely produces false negatives — the same file "doesn't match itself." Every path comparison in the pipeline now does a suffix compare in both directions.

Silent iframe drift. Sometimes the preview is fine, but the iframe navigated to a different route than the one the user is editing. We record iframe URLs and on-screen text at a 500ms cadence so a full timeline shows exactly where the frame diverged.

We built two dev-only debugging surfaces to catch these before they hit users: window.__rnLog, which records the entire pipeline timeline (Metro output, HMR events, VFS writes, iframe state changes, reload decisions) and can be downloaded as JSON; and window.__rnDiag(), a one-shot snapshot of the route context, VFS route files with sizes, and what each iframe is showing. Both live behind dynamic imports so their bundles never load in production.

Common Pitfalls, Explicitly Caught

Bringing it back to the concrete: here are the AI code generation pitfalls that used to reach users, and what now catches each before the preview.

Every row in this table represents a real bug we shipped once and a validator we now run on every generation.

People Also Ask

How do you prevent AI hallucinations in generated code? The most reliable approach isn't to try to detect hallucinations after the fact — it's to make wrong output either impossible or immediately visible. Constrain the tool surface so entire failure modes can't be expressed, validate every mutation by executing it against a sandbox, and feed structured errors back so the model can self-correct. Pure text-based validation is fragile; execution-based validation is grounded in reality.

What are AI code guardrails? AI code guardrails are programmatic constraints that intercept and validate an LLM's outputs independently of the model itself. Good guardrails combine three layers: a constrained tool surface (the model can only take certain actions), pre-execution validation (a sandbox runs the action before it hits production), and structured feedback (validation failures return to the model as fixable errors, not silent rejections).

How does RapidNative validate migrations? Every migration the AI writes is applied to an in-memory Postgres fork before it touches disk. Once applied, a lint layer reads the resulting catalog state — not the SQL text — and checks for eight known silent-failure patterns like RLS-with-no-policy or duplicate seed IDs. Errors block the write; warnings mark the file for follow-up and return to the model as feedback.

Reliable AI Code Isn't a Model Choice — It's a Pipeline Choice

Better models help. Every version of the frontier LLMs writes better SQL, cleaner React Native, and more valid TypeScript than the version before it. But the failures that hurt users most aren't about model capability — they're about the pipeline around the model treating its output as trustworthy.

The RapidNative approach is deliberately paranoid: assume the model will write a subtly wrong migration this turn and catch it. Assume the preview pipeline will drop a route sometimes and observe when it does. Assume the same file path will get formatted two ways in two different modules and compare accordingly. The validation layer isn't a nice-to-have quality gate we bolted on late — it's what makes the "prompt to preview" loop feel reliable in the first place.

If you want to see the pipeline from the outside, start a new project on RapidNative — every migration your prompt generates runs through the sandbox and lint layer described above. Or, for teams evaluating AI app builders more broadly, our how-it-works overview walks through the input side of the same pipeline. Related reading on the LLM side of the same architecture: why we run a 4-step LLM pipeline and how we test AI-generated React Native code at scale.

The bigger lesson isn't specific to mobile apps or to React Native. Any product built on top of an LLM inherits the same constraint: the model will occasionally produce something confidently wrong, and the surrounding system decides whether the user ever sees it. Executing the output, watching the runtime, and feeding failures back are the three moves that make the difference between an AI product that works and an AI product that mostly works.