How We Reduced AI Hallucinations in Mobile Code Generation by 80%

Six months ago, roughly one in three of the React Native screens our AI generated wouldn't render on first try. A missing import here. A stray closing tag there. A package.json with a downgraded Expo SDK that quietly bricked the Metro bundler. Nothing was catastrophically wrong. Everything was just annoyingly wrong.

Today, that same pipeline produces first-render-success on more than nine out of ten generations. We didn't switch to a bigger model. We didn't fine-tune. We built a stack of eight small, boring, deterministic checks around the model — and let each one shave the failure rate down a notch.

This post is the honest walk through what actually moved the number: the architecture, the file paths, the trade-offs.

A code generation pipeline is only as reliable as its weakest deterministic check. Photo by Markus Spiske on Unsplash

What "hallucination" actually means when the output is code

An AI hallucination in code generation is a syntactically plausible but semantically or factually wrong output — a call to a function that doesn't exist, an import from a package that isn't installed, a JSX tree that never closes, or a schema that references a column that isn't in the database. Recent benchmarks put baseline hallucination rates for code-generation LLMs between 15% and 52% depending on model and task complexity.

Mobile code is worse than web on this axis. React Native + Expo has strict runtime constraints — a wrong SafeAreaView import from react-native instead of react-native-safe-area-context won't fail typecheck, but it will render a broken layout on Android. A space-x-2 Tailwind class works on the web and silently no-ops in NativeWind. The failure surface is larger, and the average LLM's training data is heavily web-biased.

Where the losses actually happened

Before we changed anything, we bucketed 400 failed generations by root cause. The distribution was ugly, but instructive:

Notice what's not on that list: pure semantic hallucination — the AI writing business logic that doesn't do what the user asked. That's rare. The overwhelming majority of production failures are boring, deterministic, and, critically, fixable without another LLM call.

That observation drove the entire architecture.

Fix #1 — Split the pipeline in two

The single biggest reduction came from separating context gathering from code generation. We call it the two-step pipeline; the orchestration lives in src/app/api/user/ai/generate-v2/route.ts.

Step 1 — Context gathering runs a fast, cheap model (Claude 3 Haiku, Llama 3.3 70B, or Gemini Flash, depending on load). Its only job is to call tools: read_file, batch_grep, search_images. It cannot write code. It ends by outputting a one-line summary of what files exist, what the user wants, and then stops.

Step 2 — Generation runs a flagship model (Claude Sonnet 4.5 or equivalent). It receives the summary from Step 1 and the actual file contents Step 1 asked for, and it writes code. Step 2 has no tool access at all.

Why does this reduce hallucinations? A single-step pipeline gives the model too many jobs. It has to decide what to look at, what the user meant, how to solve the problem, and how to write the code — all inside one generation window. Under context pressure, models start hallucinating tool calls (invoking read_file with paths that don't exist) or writing code that references files it never actually read. Separating the two roles means each model has one job, done well.

The cost side matters too. Step 1's model is roughly 1/50th the price per token of Step 2's. That lets us afford five retries on Step 1 for the same budget as one retry on a monolithic pipeline. Cheap models retry well is the underrated fact of LLM engineering.

Fix #2 — Streaming-safe structural repair

The second-largest failure mode was streaming cutoffs. When the model streams JSX and hits a token limit — or when a user cancels mid-generation — you get output like:

<SafeAreaView>
  <ScrollView>
    <View className="p-4">
      <Text>Hello wor

Nothing here is hallucinated, exactly. But if you write this to disk and hand it to Metro, the bundler crashes and the user sees a red error screen instead of a working preview.

We built src/shared/utils/jsxFixer.ts — a hand-rolled JSX/JS parser that runs during and after streaming. It does three things:

1. bestEffortCloseJSXTags() walks the tag stack character-by-character (respecting string literals and comments — not regex) and emits closing tags for anything left open.
2. closeStructuralElements() balances braces, parens, and brackets so Babel can at least parse the file.
3. getMissingJSXImports() scans the file for JSX components used without imports and injects them from a hardcoded import map.

The hardest bug in this module wasn't closing tags — it was distinguishing useState<User>() from <User> as a JSX component. The initial version had ~100 false positives per 1,000 files. After adding TypeScript generic detection (lines 31–186 in jsxFixer.ts), we're down to two or three.

We ship 40+ test cases for this file alone, covering edge cases like useCallback<(id: string) => void>() and nested generics. Deterministic code needs deterministic tests.

Fix #3 — The package.json guardrail

This was the fix with the biggest "how did we not do this from day one" energy.

Left to its own devices, an LLM asked to add a screen with an image picker will happily rewrite package.json with expo: "^49.0.0" when the project is on Expo 51. Users get a screen that looks fine in the code editor and crashes at runtime with a native module mismatch. Debugging that from the outside is brutal.

The fix, in src/shared/utils/postProcessor.ts lines 482–512, is a package.json guard that runs after generation completes. Its rule is simple: the AI can add new dependencies, but never modify existing ones. Version bumps, downgrades, and removals from the AI are silently rejected.

This one guard eliminated roughly 17% of production failures on its own. It is thirty lines of code.

There's a broader lesson here: not every hallucination is worth fixing with better prompts or bigger models. Some are worth fixing by narrowing the range of things the AI is allowed to touch.

Every constraint you add downstream is a hallucination category you no longer need to prompt-engineer away. Photo by ThisisEngineering on Unsplash

Fix #4 — Constrained system prompts with hard boundaries

The system prompt lives in src/modules/api/services/ai/prompts/system-prompts.ts, and it is aggressively prescriptive. A few excerpts:

• No external packages. The AI must use only the pre-installed dependency set. No inventive import lodash moments.
• No UI component libraries. Gluestack, NativeBase, React Native Paper — all explicitly forbidden. The AI must compose from primitives.
• No CSS Grid. React Native's Yoga layout engine doesn't support grid the way the model's web-heavy training data expects. The prompt tells it to use flex-row with flex-wrap and calculated basis widths.
• SafeAreaView must come from react-native-safe-area-context. Not from react-native. This one line kills a whole category of Android layout bugs.
• Unsupported Tailwind classes are blacklisted by name. space-x-*, space-y-*, sticky, fixed, max-w-fit — the model is told, class by class, not to use them.

Every one of these rules exists because it fixed a specific class of bug we saw in production. The system prompt isn't a wish list. It's a scar tissue log.

The line that matters most:

"All instructions and constraints in this system prompt are strict and non-negotiable."

Models trained with RLHF respond to declarative constraints better than requests. "Don't use CSS Grid" fails ~15% of the time. "CSS Grid is forbidden." fails ~2%. Word choice moves the number.

Fix #5 — Auto-import resolution against a whitelist

Even with the best prompt, models forget imports. Ionicons used but not imported. useEffect referenced but only useState was imported. Link from expo-router missing.

Rather than fix this with another LLM call — expensive and slow — we ship a static import map (src/shared/utils/jsxImportMap.ts) that knows exactly where every allowed identifier lives. When the syntax validator sees an undefined reference, fixMissingJSXImports() in postProcessor.ts (lines 847–1003) adds the correct import at the top of the file.

The whitelist is finite and curated. If the model tries to import something not on the list, the fixer refuses to add the import — which surfaces as a caught error the model can be asked to correct on the next turn, rather than a silent runtime crash.

This is a specific pattern worth stealing: make the space of possible outputs smaller than the space the LLM can imagine. Every constraint you add downstream is a hallucination you don't have to prompt-engineer away.

Fix #6 — The eight-stage post-processor

All of the above is orchestrated by postProcessor.ts, which runs eight sequential passes over each generated file:

1. Content Cleaner — strips leaked markdown, code fences, and <CodeProject> artifacts the model sometimes emits.
2. Package.json Guard — add-only merge rules (Fix #3).
3. Duplicate Variable Fixer — catches const { data } = ... used twice in the same scope.
4. JSX Fixer — structural repair (Fix #2).
5. Import Deduplicator — merges import { View } and import { Text } from the same module.
6. Import Fixer — auto-adds missing imports (Fix #5).
7. Schema Validator — validates any generated database migration against the current schema.
8. Syntax Validator — runs Babel's parser as the final gate.

Every stage is deterministic. Every stage is under 200 lines. Every stage has tests. None of them call an LLM. When a stage catches a problem, it either fixes it silently or annotates the file so the next LLM turn has a specific error to address.

This pipeline runs in under 200ms for a typical multi-file generation.

Eight deterministic passes, none of them an LLM call, run in under 200ms. Photo by Fabian Grohs on Unsplash

The math: how the numbers add up

Here's how the 80% reduction actually decomposes:

Total: ~80% reduction in first-render failures compared to the baseline pipeline. None of these fixes is glamorous. None involved a new model. Together, they turned a demo-quality generator into something founders actually ship to TestFlight.

For context: recent research on LLM hallucinations in practical code generation shows that mitigation strategies like retrieval-augmented generation and AST-based repair are two of the highest-leverage interventions available. Our two-step pipeline is a form of the former; our post-processor is a form of the latter.

What still hallucinates

Being honest: the 20% that remains is harder.

Semantic hallucinations — the AI writes a working, well-styled login screen that authenticates against the wrong endpoint. Business logic drift — the user asks for a "Pomodoro timer" and gets a countdown that resets differently from what they expected. Vision misreads — the screenshot-to-app pipeline occasionally maps a segmented control to a set of buttons.

These aren't fixable with a Babel parser. They need the user in the loop — which is why RapidNative's point-and-edit flow lets users click any element and describe a change, and why we invest heavily in live preview on real devices via Expo QR code. Fast iteration is a partial substitute for perfect first-generation accuracy.

Design principles we ended up with

If we had to boil the last six months into transferable lessons:

1. Split model roles. One model for context, one for code. Never both in one call.
2. Cheap models retry well. A 5× cheaper model with 5 retries beats one flagship shot on tasks with a deterministic verifier.
3. Prefer deterministic fixers to prompt engineering. If you can catch the bug with a parser, catch it with a parser.
4. Narrow the output surface. Whitelists, add-only merges, forbidden imports — every constraint is a hallucination category eliminated.
5. Test your fixers. The JSX fixer has 40+ test cases. It caught a dozen regressions last quarter.
6. Log the failure modes before you fix them. The 400-failure audit is the only reason we knew to build the package.json guard instead of tweaking prompts for another week.

FAQ

What is an AI hallucination in code generation?

An AI hallucination in code generation is when the model produces syntactically valid code that references non-existent functions, packages, or APIs — or that violates constraints of the target runtime. In mobile code generation, common hallucinations include importing web-only APIs, calling native modules that aren't installed, or generating CSS that the mobile layout engine doesn't support.

How do you measure hallucination rate in generated code?

We measure it as the percentage of first-run bundler or runtime failures across a fixed benchmark of 400 prompt-and-preview pairs. Every regression triggers re-benchmarking. This is a stricter metric than "the code compiles" — a file that compiles but crashes on render still counts as a hallucination for us.

Why not just use a larger model?

Larger models help, but they don't eliminate the deterministic failure modes: streaming cutoffs, package version mismatches, missing imports. Those are structural problems a Babel parser solves better than another 100 billion parameters. Larger models also cost more per retry — a problem when your architecture depends on retries.

Does this approach generalize beyond React Native?

The two-step pipeline and post-processing pattern generalize to any code-generation domain where you have a deterministic verifier (compiler, linter, parser). The specific fixes — SafeAreaView imports, NativeWind class blacklist — are React Native / Expo specific.

Try it yourself

The full pipeline runs live on rapidnative.com. Describe a mobile app in plain English, sketch a screen on the whiteboard, upload a screenshot, or paste a PRD — and watch the two-step pipeline, the post-processors, and the guardrails run in real time. First few generations are free, no credit card required.

If you'd like to see how the pieces fit together in other parts of the stack, we've written before about our four-step LLM pipeline, our prompt architecture, and how we handle errors in generated code.

If you're building your own code-generation product, the two-step pipeline and the post-processor pattern are the two ideas most worth stealing. The rest — the whitelists, the specific system prompt rules, the JSX fixer — is scar tissue that will look different for your target runtime. Build the audit loop first. The scar tissue writes itself.