Store v1: Don't let a sync of an older big write unblock later small writes #129

Open
koolkhel wants to merge 1 commits from koolkhel/vitastor:chaos-seed-133 into master
Contributor

Description

A small write following an unsynced big write waits in WAIT_BIG until that big write is synced, because a big write only gets its journal entry from the sync. ack_sync() unblocked every WAIT_BIG write of the object after any synced big write, including the ones waiting for a newer big write which that sync didn't cover. Their journal entries could then reach the disk before the newer big write's entry, and after a crash or restart the replay put them on top of the older big write, giving a version that was never written.

Sequence on one object (immediate_commit != all): big write v1 acked; sync S starts (takes only v1); big write v2 acked; small write v3 queued in WAIT_BIG behind v2; S completes and mark_sync(v1) releases v3, whose entry is journaled at once; a STABLE/ROLLBACK rewrites that journal sector; crash before v2's entry is written. Replay sees BIG_WRITE v1 + SMALL_WRITE v3 and reports v3 = v3's bytes over v1.

Stop unblocking at the next big write of the object: its own sync unblocks the small writes after it. A big write which is itself in WAIT_BIG (after a delete) is still released.

Tested:

  • test_blockstore_chaos --seed 133 --config 17 fails before the fix ("came back from a power outage holding unexpected content") and passes after it.
  • New test_small_write_waits_for_its_own_big_write in test_blockstore_v1 (no checksums, plain restart instead of a power cut): fails without the fix, passes with it; the whole suite passes.
  • test_blockstore_chaos configs 15, 16, 17, 21, 22, 23: seeds 1-1000 with 10k steps and seeds 1-200 with 50k steps; before the fix only seed 133 of config 17 fails, after it none, and no hangs.

Not changed: write.cpp has a similar release-everything loop for big writes acked under immediate_commit=all; it would only misfire with three writes to one object in flight at once.

Contributor Name

Yury Luneff (MIND Software LLC, https://mindsw.io)

CLA

### Description A small write following an unsynced big write waits in `WAIT_BIG` until that big write is synced, because a big write only gets its journal entry from the sync. `ack_sync()` unblocked every `WAIT_BIG` write of the object after any synced big write, including the ones waiting for a newer big write which that sync didn't cover. Their journal entries could then reach the disk before the newer big write's entry, and after a crash or restart the replay put them on top of the older big write, giving a version that was never written. Sequence on one object (immediate_commit != all): big write v1 acked; sync S starts (takes only v1); big write v2 acked; small write v3 queued in `WAIT_BIG` behind v2; S completes and `mark_sync(v1)` releases v3, whose entry is journaled at once; a STABLE/ROLLBACK rewrites that journal sector; crash before v2's entry is written. Replay sees BIG_WRITE v1 + SMALL_WRITE v3 and reports v3 = v3's bytes over v1. Stop unblocking at the next big write of the object: its own sync unblocks the small writes after it. A big write which is itself in `WAIT_BIG` (after a delete) is still released. Tested: - `test_blockstore_chaos --seed 133 --config 17` fails before the fix ("came back from a power outage holding unexpected content") and passes after it. - New `test_small_write_waits_for_its_own_big_write` in `test_blockstore_v1` (no checksums, plain restart instead of a power cut): fails without the fix, passes with it; the whole suite passes. - `test_blockstore_chaos` configs 15, 16, 17, 21, 22, 23: seeds 1-1000 with 10k steps and seeds 1-200 with 50k steps; before the fix only seed 133 of config 17 fails, after it none, and no hangs. Not changed: `write.cpp` has a similar release-everything loop for big writes acked under immediate_commit=all; it would only misfire with three writes to one object in flight at once. ### Contributor Name Yury Luneff (MIND Software LLC, https://mindsw.io) ### CLA - [x] I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md
koolkhel added 1 commit 2026-11-18 13:04:48 +03:00
A small write following an unsynced big write waits in WAIT_BIG until
that big write is synced, because a big write only gets its journal
entry from the sync. ack_sync() unblocked every WAIT_BIG write of the
object after any synced big write, including the ones waiting for a
newer big write which that sync didn't cover. Their journal entries
could then reach the disk before the newer big write's entry, and after
a crash the replay put them on top of the older big write, giving a
version that was never written.

Stop unblocking at the next big write of the object: its own sync
unblocks the small writes after it.

Found by test_blockstore_chaos --seed 133 --config 17
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u chaos-seed-133:koolkhel-chaos-seed-133
git checkout koolkhel-chaos-seed-133
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vitalif/vitastor#129
No description provided.