monitor_cache — JSON bytes storage (no more deepcopy)¶
Motivation¶
Every cache hit in cached_snapshot paid a copy.deepcopy(value) —
_refresh deepcopied the loaded payload before storing, and
_with_cache_meta deepcopied again on every read. AKS monitor payloads
(nodes/pods/jobs) on a cluster of any size are tens of KB to MB; the
SPA polls 6 monitor routes every few seconds, so this was one of the
most expensive paths on the api sidecar event loop's thread pool.
User-facing change¶
None. Same dict shape returned; serialization roundtrip yields a fresh mutable dict (same isolation as deepcopy) at a fraction of the cost.
API / IaC diff¶
api/services/monitor_cache.py_SnapshotEntry.value(dict) →_SnapshotEntry.payload_bytes(bytes)._refreshdoesjson.dumps(payload, default=str)once and stores the encoded bytes._with_cache_metatakesbytes, doesjson.loadsonce, then inserts the cache-meta dict. Behaves identically when the loader returned a non-dict value (wrapped under{"value": …}).default=strkeeps datetime / UUID values from raising during serialization — matches the previous deepcopy's tolerance for arbitrary value types.- All four
entry.value/fallback.value/refreshed.value/loader()call sites updated to pass bytes.
Validation¶
uv run pytest -q api/tests/test_monitor_cache.py api/tests/test_warmup_route.py— 20 passed.uv run ruff check api/services/monitor_cache.py— clean.