How Eaglecart Delivers Instant Cache Invalidation at Multi-Tenant Scale

Most commerce platforms cache with a timer. A merchant edits a price, and a countdown somewhere decides when the rest of the world finds out. Five minutes. An hour. Occasionally longer. The interface says “Saved” and quietly means “eventually.”

We chose not to build that. On Eaglecart, a change is live the moment it is made — on the storefront, in the point of sale, across every server, immediately. No countdown, no advice to try again shortly, no support ticket asking why an old title is still showing.

That is a deliberate architectural commitment, and it is not free. Expiry-based caching is straightforward precisely because it never needs to know what changed; it simply forgets everything on a schedule. Instant freshness inverts that. Every write must identify exactly what it affected and clear it, everywhere, synchronously.

200× faster save operation

4,666ms → <20ms measured, before and after

constant cost as the platform grows

The cost of a guarantee

Cached entries on Eaglecart are organised into logical groups. Changing a record invalidates its group.

For a long time, invalidating a group meant asking the cache layer to remove every entry sharing a prefix. This is the intuitive approach, it is what most cache client libraries encourage, and at moderate volumes it performs exactly as expected. It carries a property that only becomes visible at scale.

Prefix-based deletion is not a lookup. It is a full traversal with filtering applied afterwards. The prefix determines which entries are returned to the caller — not how many the engine inspects.

The consequence is precise and worth stating plainly: the cost of clearing a group is governed by the total volume of cached data, not by the volume being cleared. Removing ten entries and removing ten thousand cost the same, and both cost more as the cache itself grows.

At modest volumes this is invisible. Approaching a millions cached entries, a single invalidation was taking roughly 760 milliseconds — and a single save performed several of them. Measured end to end, a representative save looked like this:

Data operations	17ms
Cache invalidation	4,649ms
Total	4,666ms

The work was never the bottleneck. Maintaining the guarantee was.

Why this class of problem matters

Any individual component here was correct. Nothing was miswritten, nothing was inefficient in isolation, and nothing would have been flagged in review. The design simply had a cost coupled to aggregate system size rather than to the operation being performed.

That coupling is the failure mode worth naming, because it is silent until it isn’t. Performance degrades gradually, in proportion to growth, and the degradation distributes evenly across everything rather than concentrating anywhere diagnosable. It produces no errors. It appears in no monitoring threshold. It surfaces eventually as a vague report that something feels slow.

In any shared, multi-tenant system it is not enough to know what an operation costs. You have to know what that cost is a function of. An operation whose cost scales with total system size will, given sufficient success, become the constraint on that success.

Generational keys

The resolution was to stop searching for entries and start making them unreachable.

Each cache group owns a counter. The current value of that counter is incorporated into every key written for the group. Invalidating the group means incrementing the counter — a single atomic operation. Every key constructed afterwards carries the new value. The previous generation still exists in memory, but nothing in the system can address it. It is not searched for, enumerated, or individually removed.

Two properties made this the correct choice rather than merely an elegant one.

The counter is shared state, held in the cache layer itself rather than in any application server. Every server therefore derives the same key for the same data. One server populating an entry serves all of them; one server invalidating is observed by all of them, instantly, with no message passing and no local state that can drift out of agreement. Freshness is a property of the platform, not of whichever server happened to serve the request.

It changes the complexity class rather than improving a constant. Tuning the previous approach was available to us and would have produced a meaningfully smaller number. It would not have changed what that number depended on. Generational invalidation removes the dependency on scale entirely — there is no future volume at which this has to be revisited.

The trade is that superseded entries release memory at expiry rather than at the moment of invalidation. Weighed against seconds of latency on every write, that is not a difficult decision.

Results

	Before	After
Save operation	4,666ms	under 20ms
Single invalidation	~760ms	under 1ms
Cost as the platform grows	rises with scale	constant

Over 200× faster on the operation we set out to address. The final row is the more consequential one. Instant invalidation is now a guarantee Eaglecart can hold at any scale, rather than a property that quietly erodes as the platform succeeds.

Two principles we took from it

Establish what your cost is a function of, not merely what it is. A fixed latency is an engineering annoyance. A latency coupled to total system size is a deadline you have not yet been told about.

Measure before forming a theory. The leading hypothesis in this investigation was image processing. Measured, it accounted for under a millisecond of a 4,666-millisecond operation. A small amount of instrumentation resolved in minutes what reasoning alone had not.

Frequently asked questions

Why not use expiry-based caching like most platforms?

Because “your change will appear within the hour” is a materially worse product. Expiry-based caching is simpler to implement because it never needs to know what changed. Instant invalidation requires every write to clear precisely what it affected, across every server, immediately — more demanding to build, but the merchant never waits and never has to wonder whether a change took effect.

Why is prefix-based deletion slow at scale?

Because it is a traversal, not a lookup. The cache engine inspects every entry it holds and filters afterwards. The prefix governs what is returned, not what is examined, so cost scales with total entries stored rather than entries matched.

What is generational cache invalidation?

Each cache group holds a counter whose current value is embedded in every key belonging to that group. Invalidation increments the counter; every key built subsequently uses the new value, rendering the previous generation unreachable without being searched for or deleted individually.

Does retaining superseded entries waste memory?

They occupy memory until expiry. In exchange, invalidation moves from hundreds of milliseconds scaling with system size to a single fixed-cost operation. On write-heavy paths this trade is strongly favourable, and steady-state overhead remains bounded by the expiry policy.