Store v1: Fix two flusher hangs with deletions #114

Open

koolkhel wants to merge 3 commits from koolkhel/vitastor:v1-lost-flush into master

Author	SHA1	Message	Date
Yury Luneff	80cd77c09d	Store v1: Wake up the flusher when journal sectors are written (test_blockstore_chaos --seed 5 --config 12 --p-delete 30) The flusher puts back the entries whose journal sector isn't written yet and stops when it finds nothing else to flush. Writing the sector moves dirty_start and makes them flushable, but nothing woke the flusher then: when the writes waiting for journal space or for a deletion were the only thing left, they hung forever, with the queue full of flushable entries and a trim requested. Happens with immediate_commit=all too.	2026-11-02 17:15:39 +03:00
Yury Luneff	c9e29f666c	Store v1: Do not queue a deletion for flushing again while it is being flushed (test_blockstore_chaos --hardened --seed 285 --config 21 --p-delete 30) A write over a deletion asks the flusher to flush the deletion first. If the deletion is being flushed already, that queues it a second time, and the queued entry outlives the deletion. Versions restart from 1 after a deletion, so once the write is renumbered and becomes stable, queueing it is ignored because the stale queued version is higher. The flusher then skips the deletion it can't find, the write is never flushed, and the journal can't be trimmed past it: it fills up and every write fails with EAGAIN until a restart. Let the flush in progress requeue the deletion when it finishes instead, as it already does for versions which come up while an object is being flushed.	2026-11-02 16:31:15 +03:00
Yury Luneff	857ddc298e	Check that stable versions get flushed in test_blockstore_chaos --hardened A store v1 stable version that no flush reaches pins the journal for good, and the store only shows it as writes refused with EAGAIN, which the test takes for journal pressure. With --hardened, check after every drain that the newest stable version of every object is being flushed, or queued under a version that is still in dirty_db and not older than it. The check scans all of dirty_db, so it is off by default.	2026-11-02 16:30:00 +03:00

Author

SHA1

Message

Date

Yury Luneff

80cd77c09d

Store v1: Wake up the flusher when journal sectors are written (test_blockstore_chaos --seed 5 --config 12 --p-delete 30)

The flusher puts back the entries whose journal sector isn't written yet
and stops when it finds nothing else to flush. Writing the sector moves
dirty_start and makes them flushable, but nothing woke the flusher then:
when the writes waiting for journal space or for a deletion were the only
thing left, they hung forever, with the queue full of flushable entries
and a trim requested. Happens with immediate_commit=all too.

2026-11-02 17:15:39 +03:00

Yury Luneff

c9e29f666c

Store v1: Do not queue a deletion for flushing again while it is being flushed (test_blockstore_chaos --hardened --seed 285 --config 21 --p-delete 30)

A write over a deletion asks the flusher to flush the deletion first. If
the deletion is being flushed already, that queues it a second time, and
the queued entry outlives the deletion. Versions restart from 1 after a
deletion, so once the write is renumbered and becomes stable, queueing it
is ignored because the stale queued version is higher. The flusher then
skips the deletion it can't find, the write is never flushed, and the
journal can't be trimmed past it: it fills up and every write fails with
EAGAIN until a restart.

Let the flush in progress requeue the deletion when it finishes instead,
as it already does for versions which come up while an object is being
flushed.

2026-11-02 16:31:15 +03:00

Yury Luneff

857ddc298e

Check that stable versions get flushed in test_blockstore_chaos --hardened

A store v1 stable version that no flush reaches pins the journal for
good, and the store only shows it as writes refused with EAGAIN, which
the test takes for journal pressure. With --hardened, check after every
drain that the newest stable version of every object is being flushed,
or queued under a version that is still in dirty_db and not older than
it. The check scans all of dirty_db, so it is off by default.

2026-11-02 16:30:00 +03:00

Store v1: Fix two flusher hangs with deletions #114

3 Commits