The complete source code for Lyte is available on GitHub.
In Part 3, we built a Codebase RAG pipeline that fed our LLM the exact context it needed without overflowing its context window.
But even with perfect context, we had a massive problem. The LLM would confidently generate React Navigation v5 syntax in a project that was strictly using Expo Router.
Our prompt was a monolithic 3,000-word template string. It was brittle, un-versioned, and highly susceptible to LLM hallucinations based on out-of-date training data.
We had to stop treating prompts like magic spells, and start treating them like code.
Prompt Engineering as Code
In version 2.0.0 of our prompt architecture, we broke the monolith into modules. We introduced three crucial innovations to anchor the LLM to reality.
1. The Dynamic Version Registry
LLMs hallucinate old APIs because they don't know what you have installed.
Before every generation, we now parse the user's package.json and inject an exact version registry directly into the system prompt:
[Installed Packages]
expo: ~52.0.14
react-native: 0.76.3
expo-router: ~4.0.0The LLM's attention mechanism heavily weights these explicit values. The moment we added this, the hallucinated React Navigation syntax completely disappeared.
2. The "Known Issues" Block
Every time we had a production outage caused by bad AI generation, we didn't just tweak the prompt. We codified the failure into a strict rule block:
- "Do NOT install @react-navigation/native separately — expo-router already includes it."
- "package.json 'main' field MUST be 'expo-router/entry' — NEVER change this."
This is prompt engineering from production failures. Each entry represents hours of debugging distilled into one rigid instruction.
3. Prompt Caching
Because our Agent Loop (from Part 2) calls the LLM 5-7 times sequentially, we were re-sending our massive system prompt on every single turn.
By adding cache_control: { type: 'ephemeral' } to our system prompt, Anthropic caches it for 5 minutes.
- Without caching: 5 writes × $3.00/M tokens.
- With caching: 1 write ($3.75/M) + 4 reads ($0.30/M).
We achieved a 67% savings on system prompt tokens instantly. Treating the system prompt as a global, static, cacheable configuration object fundamentally changed our unit economics.
The Self-Healing Loop
Even with modular prompts, cached context, and a dynamic version registry, LLMs are probabilistic. They will generate code with TypeScript errors roughly 15–30% of the time.
You have three bad options:
- Ship the broken code and let the user see the red squigglies.
- Show the compiler errors to the user and ask them to fix it (defeats the purpose of an AI builder).
- Try to parse the output and fix it with regex (impossible for AST-level errors).
We chose a fourth option: Feed the errors back to the same LLM in the exact same context window.
Taming Probability with Determinism
After the LLM's tool-use loop completes, we do not immediately return the result to the user. Instead, we run the TypeScript compiler on the generated code in the background:
npx tsc --noEmitThe compiler is deterministic. It returns the exact file, line number, column, and error type.
If tsc fails, we catch the stderr output, format it, and append a hidden system message to the LLM's conversation history:
"[TYPE CHECK FAILED] Your code failed to compile with these exact errors: ... Fix the specific lines."
We run another agent loop. The LLM reads the file it messed up, realizes it missed an import, and issues an edit_file tool call to fix the types. It corrects its own mistakes.
Defense in Depth (execFile vs exec)
Running tsc based on AI-generated code inside a sandbox is dangerous.
If we used Node's exec("npx tsc"), we would be opening a raw shell. If the LLM somehow hallucinated a malicious package script that triggered && rm -rf /, the shell would execute both.
Instead, we use execFile('npx', ['tsc', '--noEmit']). This bypasses the shell entirely. The && character is treated strictly as a string argument to the binary. Shell injection is neutralized.
We also wrap this in a strict 60,000ms timeout. If the AI runs npm init without the -y flag, the process will hang indefinitely waiting for user input. The timeout kills the process and returns the error to the LLM so it can learn and try npm init -y instead.
What I Actually Learned
- Prompts are configuration, not prose. They should be versioned, cached, modular, and driven by dynamic state (like package.json parsing) rather than hardcoded assumptions.
- The compiler is the ultimate prompt. Building a deterministic verification layer (TypeScript) that feeds back into a probabilistic generator creates a self-correcting system. This is fundamentally different from traditional software where correctness is asserted via unit tests.
- Shell injection is a feature of
exec. Never use it when dealing with AI-generated file structures. Always useexecFile.
What's Next in Part 5
The backend is now a masterpiece. The AI generates code, runs tsc, heals its own bugs, manages its context window, and stays within its token budget.
But from the frontend, it still just looks like a 4-minute loading spinner.
In Part 5, we will explore how we built a bulletproof real-time UI. We'll look at the ACK protocol, backpressure, WebSockets, and Redis Pub/Sub multiplexing to stream the AI's internal thoughts directly to the user.
