From 81848432e48626844990452c5dbc00be18b668cb Mon Sep 17 00:00:00 2001 From: Changwei Ye Date: Wed, 26 Aug 2026 10:10:16 +0800 Subject: [PATCH] Fix io_uring fsync race condition causing metadata loss In the previous implementation, `unsynced_meta_write_count` was incremented early in `prepare_meta_block_write` before the I/O request was even submitted. Because Vitastor does not use IOSQE_IO_LINK for io_uring batch submissions, the kernel does not guarantee execution order within a batch. Scenario: 1. Client A initiates a metadata write. `unsynced_meta_write_count` is incremented. 2. Client B sends a global SYNC request. The engine sees `unsynced_meta_write_count > 0`, clears it to 0, and prepares an `fsync(meta_fd)` SQE. 3. Both the `writev` and `fsync` SQEs are submitted in the same batch. 4. If the kernel's io_wq schedules the `fsync` before the `writev` (Count Stealing), the `fsync` completes instantly without flushing the intended metadata. 5. Client A's write completes and ACKs the client. 6. Client A issues a SYNC to guarantee its write. However, since the counter was already consumed by Client B's SYNC, Client A's SYNC sees a count of 0 and returns immediately without flushing. 7. A crash at this point results in metadata loss despite Client A receiving a successful SYNC ACK. This commit moves the `unsynced_meta_write_count` increment into the io_uring completion callback of the metadata `writev`. This perfectly mirrors the data area's "count-after-completion" strategy, ensuring that concurrent SYNC operations cannot steal the count before the metadata is safely in the OS page cache. --- src/blockstore/blockstore_write.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/blockstore/blockstore_write.cpp b/src/blockstore/blockstore_write.cpp index dd49d3de..30a74469 100644 --- a/src/blockstore/blockstore_write.cpp +++ b/src/blockstore/blockstore_write.cpp @@ -34,6 +34,7 @@ void blockstore_impl_t::prepare_meta_block_write(uint32_t modified_block) assert(it != modified_blocks.end()); free(it->second.buf); modified_blocks.erase(it); + unsynced_meta_write_count++; heap->complete_block_write(modified_block); ringloop->wakeup(); }; @@ -41,7 +42,6 @@ void blockstore_impl_t::prepare_meta_block_write(uint32_t modified_block) io_uring_prep_writev( sqe, dsk.meta_fd, &data->iov, 1, dsk.meta_offset + ((uint64_t)modified_block+1)*dsk.meta_block_size ); - unsynced_meta_write_count++; pending_modified_blocks.push_back(modified_block); modified_blocks[modified_block] = { .sent = false, .buf = buf }; } -- 2.47.3