Building Lyte (Part 5): Reliable Real-Time UX and the ACK Protocol

The complete source code for Lyte is available on GitHub.

In Part 4, we built a deterministic self-healing loop that forces our LLM to fix its own TypeScript compiler errors.

Our Agentic Engine was finally bulletproof. But there was a massive disconnect.

When a user clicks "Generate", the job is dropped into an SQS queue. The HTTP request finishes instantly. Meanwhile, a background worker spins up a Sandbox and runs a 5-minute loop of intense file I/O and shell execution.

If the user just stares at a loading spinner for 5 minutes, they will refresh the page. When they refresh the page, they lose their HTTP context.

We needed to stream the AI's internal thoughts and file writes directly to the user in real-time, completely bypassing the HTTP cycle.

Here is how we built the real-time bridge.

The Fire-and-Forget Relay Problem

Our first attempt at a WebSocket relay was naive.

When the Worker generated a file, it emitted an event to our WS Relay: "Write app/page.tsx." The Worker then immediately moved on to the next task. Fire-and-forget.

What went wrong:

1. If the Relay crashed mid-write, the Worker assumed the file was written. It wasn't. The app broke.
2. If the Relay was slow (disk I/O bottlenecks), the Worker kept generating at LLM-speed. The queue built up in memory until the Relay crashed with an Out-of-Memory (OOM) error.
3. If the Worker temporarily disconnected, it had no idea which files had actually been written when it reconnected.

Fire-and-forget is fine for logging. It is fatal for file system operations that downstream systems depend on.

The Five Fixes

We had to harden the WebSocket relay using standard distributed systems patterns.

Fix 1: The ACK Protocol Every event emitted by the Worker now gets a UUID. The Relay must send back an Acknowledgment: { ack: true, eventId, checksum: SHA256(content) }. The Worker awaits this ACK (with a timeout) before treating the write as confirmed.

Fix 2: Idempotent Writes File writes are inherently idempotent. Writing the exact same content twice has no harmful effect. This means that if an ACK times out, we can safely blindly retry the write without needing complex deduplication logic on the Relay.

Fix 3: Staging-then-Promote Expo Metro (the bundler watching our files) would occasionally crash if it read a file while it was only half-written. The Relay now writes to /tmp/.staging/{path} first. Once complete, it calls fs.rename() to move it to /app/{path}. fs.rename is an atomic OS-level operation. Metro never sees a partial file again.

Fix 4: Backpressure Instead of fire-and-forget, the Relay client now maintains a count of pending ACKs. If pending ACKs hit 50, the emitEvent() function blocks. This backpressure propagates all the way up the stack:

• emitEvent() blocks.
• executeTool() blocks in the Worker.
• The for await (const chunk of llmStream) async iterator pauses.
• TCP-level backpressure reaches the LLM API stream itself.

We physically pause the LLM generation if our disk I/O can't keep up.

Fix 5: Reconnection with Replay On disconnect, we use exponential backoff with full jitter to prevent thundering herds. When the Worker reconnects, it queries Postgres: SELECT * FROM actions WHERE relayed_at IS NULL ORDER BY sequence_number. It replays all unacknowledged events in perfect order.

Multiplexing at Scale (Redis Pub/Sub)

With the Worker-to-Relay connection hardened, we still had to push these events to the user's browser.

The problem: The API Gateway holds thousands of active WebSocket connections from users. The Worker doesn't know who the user is, and it isn't connected to them. Furthermore, we run multiple API Gateway instances behind a load balancer.

If Worker A emits an update for Project 123, but the user is connected to API Node B, how does the message reach them?

The Solution: Redis Pub/Sub Multiplexing.

We attached a Redis Pub/Sub adapter to Socket.IO. All API Gateway instances subscribe to the Redis cluster. When the Worker wants to update the user, it publishes a message to Redis:

// The Worker screaming into the void
await redis.publish(
  `project-updates:123`, 
  JSON.stringify({ type: 'progress', message: 'Writing page.tsx' })
);

Redis instantly broadcasts this to all API nodes. The node holding the specific WebSocket connection for project:123 catches it and forwards it to the browser.

We can scale the API nodes horizontally to handle millions of user connections, and scale the Worker nodes based on SQS queue depth. They never talk directly to each other.

The Frontend: Why Zustand?

On the frontend, we needed to receive these events and update the UI instantly.

Initially, we tried using React's useEffect to manage the WebSocket connection. This was a nightmare. React's render lifecycle is completely at odds with a persistent, stateful socket connection. The socket would disconnect on hot-reloads, state updates would batch incorrectly, and the UI would stutter.

We ripped it out and moved to Zustand.

Zustand is an un-opinionated state manager that exists outside the React render tree.

export const useProjectStore = create((set) => ({
  events: [],
  connectWebSocket: (projectId) => {
    const socket = io(API_URL);
    socket.emit('subscribe_project', projectId);
    
    socket.on('project_event', (event) => {
      // Updates state directly. Subscribed React components auto-render.
      set((state) => ({ events: [...state.events, event] }));
    });
  }
}));

By decoupling the WebSocket connection from the React component tree, the UI became bulletproof. You could navigate away from the page, come back, and the Zustand store would still be holding the live socket connection, quietly buffering events.

What I Actually Learned

• Fire-and-forget is an anti-pattern. If a downstream system depends on a state change, you must implement an ACK protocol and backpressure.
• Backpressure can control LLMs. Pausing an async iterator effectively halts TCP traffic, which pauses the LLM API stream. You can physically slow down an AI if your infrastructure needs time to catch up.
• State managers should not be tied to UI lifecycles. Managing WebSockets inside a React component is a mistake. Always manage persistent connections in a store that outlives the component.

What's Next in Part 6 (The Finale)

We have built a distributed, agentic, context-aware, self-healing, real-time AI platform.

But as we prepared for production, a systematic audit revealed 9 critical bottlenecks that would have crushed the system under load. Worse, every time we tweaked a prompt to fix a bug, we had no mathematical way of knowing if we broke something else.

In Part 6, we will look at how we implemented Bulkhead patterns, Priority Queues, and built an LLM-as-a-Judge Eval Framework to guarantee generation quality across thousands of regression tests.

Up Next: Building Lyte (Part 6): Evals and Production Scaling