Skip to content

Preserve payload group atomicity across block materialization - #67

Open
marcobambini wants to merge 3 commits into
pg-fixes11092026from
codex/block-payload-group-atomicity
Open

marcobambini wants to merge 3 commits into
pg-fixes11092026from
codex/block-payload-group-atomicity

Conversation

@marcobambini

@marcobambini marcobambini commented Sep 19, 2026

Copy link
Copy Markdown
Member

A payload group containing ordinary and block columns could be partially committed: a block-table write failure left the ordinary values and metadata applied while the block value was missing or stale.

The pre-block flush now writes pending ordinary columns without releasing the PK-group savepoint or advancing the applied-row count. The boundary stays open through block materialization and closes only at the actual PK/table/source-version boundary. A block failure rolls back the whole group, including ordinary columns, metadata, blocks and resurrection changes. The checkpoint stays unchanged, and replay succeeds once the cause is removed.

Validation

  • 120 forced-failure cases: inserts, updates, resurrection, 3–10 blocks, autocommit and caller-owned transactions.
  • Rejection of first (36 cases), middle (58 cases) and last (26 cases) blocks.
  • Exact before/after comparison of data, metadata and blocks in both directions; unchanged checkpoint and preserved caller transaction.
  • Successful retry and duplicate replay for every case.
  • Core and audit suites pass under ASan/UBSan, including SQLite itself; zero outstanding SQLite memory.
  • Independent PostgreSQL 18.6 compatibility suite: 521 checks pass.
  • Negative control: restoring the old flush fails 360 assertions in the new tests.

docs/internal/block-group-atomicity.md documents the boundary and tests.

This is one of three independent follow-ups to #64, based on pg-fixes11092026 at 9d0abb3. Retarget to main after #64 merges. No deployed cloud server was exercised or modified.

Cloud integration reliability

The shared chunked-tenant job is serialized across branches with a queued concurrency group. This prevents a different PR from writing during the negative-cache test's required idle phase; all idle assertions remain intact, and other platforms still run in parallel.

Fresh receivers now wait for actual received rows and expected fixture data rather than requiring the first poll to contain rows. Polling is bounded and SQL/protocol failures abort immediately. A materialized result ensures one network call per attempt. The offline integration_bootstrap test runs as part of make unittest, covering 312 scenarios including delayed/partial delivery, timeout, missing data, protocol errors and malformed responses. Local ordinary and ASan/UBSan runs pass with zero outstanding SQLite memory.

Latest CI verification

Workflow run 35443345054 completed successfully on 5338029: 37 jobs passed, with only the release job skipped. The real-cloud negative-cache test ran and passed on Linux x86_64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant