Commit Graph

31 Commits (2d9f09dcb686e6dabc27026b0b8571ff72afe1a0)

Author SHA1 Message Date
Vitaliy Filippov 1018764c91 Fix write->delete->write bugs, add & fix some debugging output 3 years ago
Vitaliy Filippov faa5e1436f Attempt journal trim even without new flushes
This is the new guaranteed unblocking method which replaces old trims
in init and rollback, and also fixes a possible stall when just several
writes in the beginning of the journal are flushed without triggering
a subsequent trim.
3 years ago
Vitaliy Filippov 5fbe36198a Fix journal trimming
1) Update journal's used_start in memory only after updating journal superblock.
Doing the opposite is incorrect because part of the journal will be lost if writers
overwrite its old beginning.

2) Sync journal device after updating the superblock.

3) Do not trim in rollback and init because trimming there would also require
updating the superblock. And the only reason to trim in both those places was
to unblock writers. And a guaranteed unblocking method will follow in the next
commit :)
3 years ago
Vitaliy Filippov 0471b09b9c Add license notices to all source code files 3 years ago
Vitaliy Filippov 165c204555 Fix BS_OP_DELETE (the implementation was untested up to this point) 3 years ago
Vitaliy Filippov e6a4b634f8 Fix possible write stall
The stall occurred during fio Q=128 random write tests with low flusher_count (4).
It was caused by flushers being unable to flush the beginning of the journal
because it contained older writes to an object that also had writes in the very end
of the journal, after dirty_start.
3 years ago
Vitaliy Filippov c3737ae3ff Add journal fsync to stabilize/rollback 3 years ago
Vitaliy Filippov c863543bfe Fix possible journal corruption caused by concurrent flushing and writing of the same journal sector 3 years ago
Vitaliy Filippov b27ad550cf Use btree_map instead of sparsepp 3 years ago
Vitaliy Filippov 1733de2db6 Test & fix single-PG primary OSD
- Add support for benchmarking single primary OSD in fio_sec_osd
- Do not wait for the next event in flushers (return resume_0 back)
- Fix flushing of zero-length writes
- Print PG object count when peering
- Print journal free space when starting and when congested
3 years ago
Vitaliy Filippov 2b09710d6f Implement blockstore rollback operation
Rollback operation is required for the primary OSD to kill unstable
object versions in OSD peers so they don't occupy journal space
3 years ago
Vitaliy Filippov cf819eb442 Implement sparse block bitmap to avoid zero-fill 3 years ago
Vitaliy Filippov a7e74670a5 Split blockstore implementation and interface header 4 years ago
Vitaliy Filippov aaea3e1f99 Fix read_fulfill, use vector 4 years ago
Vitaliy Filippov fd13965b90 Force start when journal is full 4 years ago
Vitaliy Filippov aa3b252327 Fsync data before writing metadata 4 years ago
Vitaliy Filippov 76caecf7c7 Inmemory metadata mode 4 years ago
Vitaliy Filippov f4d06ba102 OP_DELETE flushing 4 years ago
Vitaliy Filippov 00eeedae90 Add "fsync disabled" mode 4 years ago
Vitaliy Filippov 40781c67b2 Trim journal on start 4 years ago
Vitaliy Filippov 9fa0d3325f Support inmemory journal 4 years ago
Vitaliy Filippov 95396e2c83 Note read errors 4 years ago
Vitaliy Filippov 35a6ed728d Fix another stall due to bad unstable_writes tracking, do not try to write beyond the end of the journal 4 years ago
Vitaliy Filippov 9568354d78 Fix old_clean_loc treatment (cannot persist sparse_hash_map iterator) 4 years ago
Vitaliy Filippov be3015169f Safe stop procedure 4 years ago
Vitaliy Filippov a4aaa3c7c7 First implementation of journal trimming
In theory it's possible to start testing blockstore at this point!
4 years ago
Vitaliy Filippov c38c8ab029 Prevent two parallel flushes of the same object 4 years ago
Vitaliy Filippov 71194f014a Update dirty_db and clean_db after flushing an entry 4 years ago
Vitaliy Filippov f1e236c6e8 Sync metadata & data after copying from journal 4 years ago
Vitaliy Filippov 1c6b9778a4 Handle all io_uring events using lambdas 4 years ago
Vitaliy Filippov 214da03735 Move flusher into a separate file 4 years ago