Caching

How the kit caches: HybridCache over Redis or memory, the key and tag conventions, what is cached and for how long, adding a cached read to a slice, tests, Redis sizing and eviction, and what happens while Redis is down.

The api and the worker cache through HybridCache: each process keeps a first level in memory, and Redis is the second level every replica shares. A read goes through the cache; the write that changes the data drops the entries by tag, so the next read goes to the database. The package is packages/caching.

Keys and tags

Every key is built by ColdenCacheKeys, so it carries the area and the organization it belongs to; every entry has tags, by which a write drops it.

An organization's data never sits under a global key, and a key never holds what another organization may not read. RemoveByTagAsync drops the tag's entries in Redis and in the process that made the change; another replica may serve its memory copy until the copy's local lifetime ends, which is why the kit keeps local lifetimes at a minute.

What is cached

An entry that names no lifetime takes Caching:DefaultLocalExpirationMinutes (5) in memory and Caching:DefaultDistributedExpirationMinutes (30) in Redis. Keep a lifetime short when no write can drop the entry (data that changes elsewhere), and longer when every write drops it.

Adding a cached read to a slice

1. Name the key with ColdenCacheKeys and choose the tags: the organization's tag for its data, an entity's tag when one record's change must drop it.

2. Read through the cache, with the state the factory needs passed in (no closure), so the factory is static:

   var summary = await cache.GetOrCreateAsync(
       ColdenCacheKeys.ForTenant("projects", tenantId, "summary"),
       (Db: db, Tenant: tenantId),
       static async (state, token) => await ProjectSummary.ReadAsync(state.Db, state.Tenant, token),
       new HybridCacheEntryOptions { Expiration = TimeSpan.FromMinutes(10), LocalCacheExpiration = TimeSpan.FromMinutes(1) },
       [CacheTags.ForTenant(tenantId), "projects"],
       cancellationToken);
   

3. Drop the tags in every write that changes the data, after it commits: await cache.RemoveByTagAsync("projects", cancellationToken).

4. Cache a value that serializes as JSON (a record), never an entity the context tracks, and nothing that depends on the request (the caller's role, the session's assurance): set those on the copy you return.

5. Prove it with EfCommandCounter: the second read issues no query, and the write's next read shows the change (as HotReadPathTests does).

Tests and local runs

Caching:Provider Memory keeps the second level in the process: the test hosts use it, and so can a host run without Redis. The commands of the api binary run with it too. Each test host has its own cache, so tests never see each other's entries.

Redis sizing and eviction

Redis holds the second level of the cache and the rate limiter's windows; everything in it has a lifetime and can be rebuilt, so the in-cluster Redis keeps nothing on disk (--save "", no append-only file) and evicts the least recently used keys when it is full (redis.maxmemory, redis.maxmemoryPolicy allkeys-lru). The chart's default gives it 200 MB within a 256 MiB container limit, enough for tens of thousands of organizations' entries; raise both together, and keep maxmemory about 80 % of the limit. An external Redis (Azure Cache, ElastiCache, a managed Redis) needs the same policy and no persistence.

Watch the evictions (evicted_keys in INFO stats): a steady count means the cache is too small, and the database takes the reads the cache misses.

While Redis is down

The hosts start and keep serving while Redis is down: the client reconnects on its own (AbortOnConnectFail off), HybridCache serves from each process's memory and reads the database on a miss, and the rate limiter lets requests through. What degrades:

• each replica caches on its own, so an entry dropped on one replica can be served by another until its memory lifetime ends (a minute);
• the database takes more reads;
• the rate limits stop counting until Redis is back.

The redis check reports Degraded on /health/ready, which keeps the pods in service; alert on it.