Field report · Live market infrastructure
Fast is a moment.
Reliable is a system.
A connection can be open while the information behind it is already old. Live delivery needs a contract about freshness, fairness and what happens next.
Ingestion and client delivery have different responsibilities.
Do the shared work once.
A market feed has two different shapes. Upstream, events arrive quickly. Downstream, many people want overlapping pieces of the same information. Treating every viewer as an independent polling job repeats the expensive part precisely when demand grows.
The gateway work separated ingestion, aggregation, snapshot storage and client delivery. That separation made it possible to reason about each pressure independently. A burst from a provider is not the same problem as a large watchlist, even when both appear as latency to the person watching.
Shared cache-backed snapshots provide a useful handoff. Connected clients can consume a prepared view rather than repeatedly asking the database to reconstruct it. Persistent aggregates serve another purpose: continuity for charts and historical views beyond a single connection.
A useful rhythm beats an indiscriminate flood.
Not every screen needs every event at the same pace. Interval-aware subscriptions let delivery reflect what the client can use. A rate contract also protects the rest of the system from one unusually broad or enthusiastic subscription.
Symbol limits, update intervals and subscription behavior need to be understandable rules. They are part of the product contract, not obscure configuration that only becomes visible when something breaks. Fairness begins with deciding how work is shared.
The design explored delivery for thousands of concurrent viewers. That is a scale objective, not evidence of unlimited capacity or a measured latency guarantee. Actual behavior still depends on providers, deployment resources, subscription mix and the work performed for each update.
“Connected” describes the transport.
“Fresh” describes the information.
Keep the last useful view. Tell the truth about it.
A connection can remain open while upstream data stops moving. A reconnect can succeed before a fresh snapshot is available. If the interface only distinguishes connected from disconnected, it hides the distinction the reader actually cares about.
Cached and stored fallbacks belong in the normal architecture. They preserve a useful quote or chart when the live path is unavailable. Their age and source need to remain visible so continuity does not masquerade as freshness.
Monitoring then has to follow the full route: ingestion, preparation and delivery. It should help locate where information stopped being useful, rather than simply confirm that a process is running. Reliability is the ability to recover with an explanation, under conditions that refuse to stay constant.
A newly connected client presents another boundary. It needs a starting view as well as future updates. If those are assembled independently, the interface can briefly show a confusing combination of old and new information. Thinking about snapshots and subsequent delivery together makes the continuity problem easier to see.
The right update rate also depends on the surface. A watchlist, a detailed chart and a background summary do not consume information in the same way. Separating those needs keeps “live” from becoming a blanket instruction to move every event through every layer as quickly as possible.
This is why I think of real-time as a promise with terms. The system owes people a useful cadence, fair access and an honest account of what it currently knows.
Related work & references
Selected public entry points and related portfolio pages. A company website or profile is context, not a claim that private source code is available.
Scope, context & further questions
Capacity figures are planning targets. Subscription intervals describe the gateway design; they are not measured latency guarantees.
- Concurrent design target: 1,500. The gateway was shaped around thousands of simultaneous connected users.
- Delivery cadence: 1–3 sec. Subscriptions balanced perceived immediacy with sustainable fan-out.
- Fallback path: 3 layers. Live stream, cache snapshot, and stored aggregates protected continuity.
Questions I am still working through
How should cadence adapt to visibility, volatility, and device state?
Cadence can be treated as a budget: slow background tabs and quiet symbols, reserve faster updates for visible or actively watched data, and apply server-side caps so one client cannot consume disproportionate fan-out capacity.
The adaptive policy still needs measurement against perceived freshness, battery use, and infrastructure cost.
What freshness promise should the interface make during upstream degradation?
Show the timestamp or age of the latest trusted value and explicitly switch the experience from live to delayed or cached. A slightly older value with honest provenance is better than a live-looking stale value.
The acceptable stale window depends on the use case and should be defined as a product contract, not guessed during an incident.