← Back to Blog
Article - 2026

Where Should the Model Run? Cloud or Local (2026)

📅 November 22, 2026 ⏱️ 5 min read
Every AI feature forces one early decision: does the model run on a server I pay for, or on the user's own machine? I made that choice differently in two projects, and the reasons are a useful rule of thumb.

Cloud: When Quality And Control Matter

In AI Exodus the conversations between twins run on Gemini models behind AWS Lambda functions. That is the right place for it: the reasoning needs a strong model, the work can run in the background, and a server lets me meter usage and protect the key. The cost is latency and a bill that grows with use, which is why the app charges coins only when a match is found and puts a cheap check in front of any expensive call.

Local: When Latency And Privacy Matter

The AI Exodus Companion reacts to what is happening in a game, so a voice that arrives late is worse than silence. Speech synthesis runs locally with a bundled model, and speech recognition for talk-back is a local Whisper model. Nothing about the player's voice has to leave the PC. The cost is size: the installer is over 1 GB and the app uses around 2 GB of memory.

A Simple Rule Of Thumb

  • Run it in the cloud when the task needs the strongest model, can wait a few seconds, and you want to meter or protect it.
  • Run it locally when it must feel instant, handles private input, or has to work without a connection.
  • Mix them when you can: a small local model for the instant part and a cloud model for the heavy thinking.

Whichever you pick, measure it. Both projects had a "thinking" setting that quietly made a model slower or ate the output budget until I looked at real numbers. If you are deciding where an AI feature should live, send me the details and I will tell you what I would do.