Echo is The Product Highway’s voice-agent platform. One of its flows is the shopping assistant on a US e-commerce store. A customer types or speaks a question, and the agent looks up the catalogue, prices and shipping before it answers. In September the replies were slow. When a typed question needed a tool, the reply started after a median of 4.93 s on the server. For a spoken question it was 6.63 s.

The session events showed why. The voice service ran in Mumbai, but the users, the LLM and embeddings, speech-to-text and the store’s API were all in the US. So every step of a US user’s turn went to India and back. Each LLM request spent about 400 ms on the network and each store API call about 225 ms, and a turn makes several of each. Only text-to-speech already ran near the user.

I moved the service, its Postgres and its Redis to Northern California, and a day later to Ohio, the store API’s own region. Then I measured the move on real traffic, not on a benchmark. I read 7,399 sessions through Echo’s own API, rebuilt each turn from its server events and compared the wait before the reply’s first audio, one turn type at a time. As more data came in I wrote three reports. My first headline figure did not survive that data, so I replaced it with the per-type medians and their sample sizes.

Once the service sat next to the store’s API, tool calls were the next cost worth cutting. Customers often ask about the same products and specs. So I added a shared tool cache to Echo (PR #9). A tool opts in with a time-to-live in the flow editor. Results go to Redis with a fixed expiry and a 1 MiB cap, and any Redis fault falls back to a live call. When several callers miss on the same request at once, one fetches and the others wait for its result. Credentials change the cache key but never appear in it. The test suite (240 tests, including real Redis shared across processes) passed, and I turned the cache on for all three of the store’s flows.

The caveats stay with the numbers. Most traffic after the move was internal testing. Only 14 spoken turns passed every check. And this compares two periods of real traffic; it is not a controlled test.