Zero 1.11

Cached Results and Sturdier Client Storage

Installation

Copynpm install @rocicorp/zero@1.11

Overview

Zero 1.11 can show cached query results right away, keeps working when local storage fails, and makes Litestream v5 the default for backups. Client cold start is about 7x faster in our benchmarks. There are also a lot of fixes, especially in queries and replication.

Features

• Queries can now start with results that a previous session saved on the device. These come back with the new 'cached' result type, so you can show them right away and still tell them apart from data the server has confirmed. (#6578) (thanks @pawarren!)
• Building the same query again usually gives you the same Query object. useQuery no longer rebuilds and rehashes its query on every render, and queries built inline work as React dependencies. (#6469, #6496) (thanks @charpeni!)
• Zero starts in a new 'initializing' connection state while it loads local data, and slow loading doesn't count against the connect timeout anymore. (#6561)
• If local storage can't be opened, fills up, or hits I/O errors, Zero reports it once through the new onStorageFailure option and keeps running in memory. zero.kvStore tells you which store is in use. (#6585, #6657, #6686) (thanks @pawarren and @Karavil!)
• Clients upgrading from 1.10 keep their local data and pending mutations. Future protocol changes won't reset clients to an empty local database either. (d597c69e8, 28e581483)
• Server mutators can return a JSON value to the client, run again in a new transaction when tx.retryOn() accepts the error, and run at a chosen transaction isolation level. (7756eb9bb, 4fe91c0d8, cb2821f3a) (thanks @pawarren!)
• Litestream v5 backups are now the default. A replica restored from another replica's backup continues that backup incrementally. Periodic snapshots keep backups compact, and the replica's WAL stays bounded. (fb536f686, c1a586810, #6476, #6539, #6717)
• A zero-cache that comes online now waits for replication to catch up before it reports ready. (1c5ddcdb4)
• Inactive client groups are kept for 7 days instead of 48 hours, so users who come back within a week don't need a full resync. (#6484)
• zero-cache now logs slow queries with the reason and suggested indexes, and periodically logs per-query statistics. A new operator password gives access to CPU profiles (/profz) and replica planner statistics (/plannerz) without the admin password. (#6446, #6639, #6673, #6675, #6699)
• A new hydration budget caps the time a view-syncer spends re-hydrating a reconnecting client's inactive queries, and a per-query hydration timeout aborts any query that takes too long to hydrate. Both are off by default. (#6477, #6499)
• New OpenTelemetry metrics track WAL usage. Replication status events now include index-build progress, and a new view-syncer status event reports restore progress. (#6591, #6621, #6714)
• Server mutations can send their logs to your own LogSink. (#6677) (thanks @pawarren!)
• The zero Docker image is down to 807 MB from about 1.09 GB. (#6695)

Performance

Client Cold Start

Apps usually register queries before Zero has finished loading local data, for example when useQuery runs at startup. Zero 1.10 then pushed every loaded row through every registered query. Zero 1.11 waits for the data to load and hydrates each query once (#6472). With 13 queries and about 30,000 local rows, startup is about 7x faster, and about 8x faster on React Native.

Server-Side Incremental Updates

When a transaction replicates, zero-cache has to bring every client group's queries up to date. Zero 1.11 does this with about 2.5x less CPU. We measured 100 client groups on one sync worker, each syncing a query with a related subquery.

React Native

Zero's query engine is about 1.7x faster on React Native. That's the geometric mean of 53 query engine benchmarks on Hermes, and 48 of them got more than 5% faster. Hydrating queries is 1.6x to 4.8x faster, and applying changes 1.2x to 2.2x faster.

Building and Hashing Queries

Components rebuild their queries on every render, and Zero hashes each query to identify it. Building the same query again now usually returns the same object, with its hash already computed (#6469). That makes building and hashing a query about 7x to 17x faster, depending on its size.

Client Writes With Many Joined Queries Got Slower

This is the one benchmark that got slower. With 300 open queries, each joining the same four tables, applying one write in the client's query engine takes about 1.2x longer. Joins now keep an index of their parent rows so they can skip changes no parent can see (#6645, #6708). In this benchmark every change reaches some parent, so keeping the index up to date is pure overhead.

Fixes

Query Results

• Queries using or with a flipped exists could return wrong rows. (#6636)
• Joins could leave a deleted row on clients, keep a stale row in a limit() query, or fail with node does not exist. (#6660)
• Rows behind an exists inside a scalar subquery weren't synced, and changes to them didn't propagate. (913c0665f)
• Catching up a client after a large update could fail with Missing row, and a schema change during rehydration could fail with table has no column named. (#6662, #6616)
• A SQLite plan that nests one multi-index OR inside another could crash zero-cache in a loop. (6ea17bfcd) (thanks @robertoborgone!)
• zero-cache called ZERO_QUERY_URL again for unchanged custom queries whenever it rebuilt its query pipelines. A permission change still re-transforms every query. (#6601)
• In React and SolidJS, two custom queries with the same query but different names or arguments could share one view, and a SolidJS view rebuilt after its query changed kept the old TTL. (#6461, #6497)
• In mutators, a row inserted without its optional columns didn't read those columns back as null. (d4816752c)

Clients and React Native

• SQLite-backed stores (expo-sqlite, op-sqlite) could return corrupted data under concurrent reads, hang after a failed transaction, or keep part of a failed commit. (#6495, #6655, #6683) (thanks @pawarren!)
• Healthy clients logged ChunkNotFoundError during persist. A write that would corrupt the local store now fails instead. (#6670, #6559) (thanks @pawarren!)
• With op-sqlite and a custom location or encryptionKey, dropping the database didn't delete the right file. (#6656) (thanks @pawarren!)
• Clients could go offline after the app resumed from the background or the computer woke from sleep, and React Native connect timeouts left the client stuck in an error state instead of retrying. (#6510, #6485) (thanks @pawarren!)
• Slow local startup could make a healthy server look unreachable. (#6475)
• A mutator that threw while being replayed disconnected the client in a loop. The client now stays connected, and the mutation is still sent to the server. (#6555)
• Client memory grew with every query used in a session. (#6474)
• useSuspenseQuery with an options object and no suspendUntil didn't suspend until the first rows arrived, contrary to the docs (see Breaking Changes). (ffe67b23a) (thanks @charpeni!)
• Routine reconnects, auth errors and server rebalancing were logged as errors. They're now warnings or info, and fatal errors include their cause. (#6581, #6692, #6498, #6682, #6691, 42316d521, 075b4215e, 79adc09fb) (thanks @pawarren!)

zero-cache Reliability

• zero-cache leaked memory and other resources around closed connections, stopped view-syncers, abandoned hydrations, and inspector and profiling requests. (#6500, #6514, #6515, #6516, #6525, #6526, #6527, #6528, #6540, #6541, #6542, #6698)
• View-syncers could crash or return Internal error when another view-syncer advanced the same client group, when a client connection was missing, or when a push arrived on a closing connection. (#6481, #6530, #6600, #6614, #6631, d545bc7e2) (thanks @jdnvn!)
• Servers using zeroNodePg crashed when Postgres closed a pooled connection. (1b220fa07) (thanks @pawarren!)
• Analyzing a query in the Inspector could run a view-syncer out of memory on large tables. (#6596)
• Error messages could include row values, query literals and mutation arguments. (#6413)

Replication, Backfills, and Backups

• Replication could roll back or stop after an interrupted stream, and a replication-manager whose slot was invalidated crash-looped instead of resyncing. (#6562, #6590)
• Backfills could apply rows before replication had caught up to them, didn't resume when a view-syncer moved to another replication-manager, and leaked a Postgres connection that held back vacuum. (#6685, #6684, #6513)
• Replication slots are more robust. Slot creation retries while long transactions block it, slots stay active during long initial syncs, slots dropped outside Zero are handled, and inactive slots are cleaned up. (25a9ea1ac, f3bba7d73, 7a998b566, caf79b161, #6568)
• Replication lag was measured wrong on Aurora. (e6a2590db)
• After initial sync, the replication-manager could serve before a backup covered its replica, so restoring view-syncers fell back to Postgres. It now waits for that backup. (#6465)
• Query pipelines were reset more often than needed, and dropping an upstream index didn't refresh table schemas. (4341cdcbd, 89787f827, bb71cc931)
• View-syncers now fail fast when the replication-manager is unreachable, and a slow handshake is no longer mistaken for a missed heartbeat. (0af1a7109, #6598)
• During a resync, the old replication-manager now keeps serving until the new one takes over. (21f349135)
• Change-database connections leaked after a failed catch-up, and a change-database reset could hang. (#6529, #6632)
• Litestream could run out of memory in small containers. Its upload concurrency now scales with available memory. (240c05730)
• Configuration values that start with a dash, such as some shard IDs, were parsed wrong. (81e6ac570)
• In the zero Docker image, initial sync and backfills could leave a stalled Postgres connection open after copying a table. Other installs keep the 1.10 behavior. (3f139f52a, 22fa3c27e)

Type Checking

• A rejected whereExists(..., {scalar: true}) now explains why, and a misspelled relationship name lists the valid ones. (cc020b368, 12a36dcfe)

Breaking Changes

• Legacy queries: zero-cache now rejects queries sent directly by clients. If you still use legacy queries, set ZERO_ALLOW_LEGACY_QUERIES=true or move to custom queries. (#6399)
• New states: if your code switches over every result type or connection state, handle 'cached' and 'initializing'. Treat 'cached' like 'unknown': the data may be out of date. (#6578, #6561)
• Litestream v5 by default: remove the ZERO_LITESTREAM_VFS_* options; they no longer exist. Each replication-manager backs up to its own folder under the backup URL, so end the URL with / and add a lifecycle policy to your bucket. To keep writing legacy WAL backups with Litestream v0.3, set ZERO_LITESTREAM_BACKUP_USING_V5=false. If you set ZERO_LITESTREAM_EXECUTABLE_V5, use a Litestream build with -fork-to-url. (fb536f686, c1a586810)
• Changed defaults: inactive client groups are kept for 168 hours (was 48), replication lag is reported every 10 seconds (was 30), and ZERO_CHANGE_STREAMER_FLOW_CONTROL_CONSENSUS_TIMEOUT_PROPORTION is 4 (was 2). Startup can take longer while zero-cache waits for a backup and for replication to catch up, so give readiness probes more time. (#6484, 1c5ddcdb4, #6521, #6465)
• Server mutators: return values must be JSON. A custom Database must pass addRetryPredicate on to the transaction it creates, or tx.retryOn() won't work. Using tx after its transaction has ended now throws. (7756eb9bb, 4fe91c0d8)
• ZQLDatabase.run() no longer wraps reads in a transaction, so per-transaction settings such as SET LOCAL don't apply to it. (#6577)
• useSuspenseQuery with an options object and no suspendUntil now suspends until the first rows arrive, as documented. Pass suspendUntil: 'complete' for the old behavior. (ffe67b23a)
• Upgrading from 1.9 or earlier: clients start with a new local database, and mutations they hadn't sent yet are lost. Clients upgrading from 1.10 keep theirs. (d597c69e8, 28e581483)
• Client storage: a client whose local store is corrupt now deletes it and calls onClientStateNotFound. Unsent mutations are lost. (#6558)
• One-time query reload: queries are identified differently in 1.11, so after upgrading, and again after a rollback, zero-cache transforms and hydrates each client's queries once more. Expect a burst of requests to ZERO_QUERY_URL. (#6450, #6462)
• Deprecated: ZERO_CHANGE_STREAMER_MODE. Use ZERO_CHANGE_STREAMER_URI instead. (698286b55)
• Events: there's a new zero/events/status/view-syncer/v1 event type. Event consumers that reject unknown types must accept it. (#6621)