Vitaliy Filippov
faa5e1436f
Attempt journal trim even without new flushes
...
This is the new guaranteed unblocking method which replaces old trims
in init and rollback, and also fixes a possible stall when just several
writes in the beginning of the journal are flushed without triggering
a subsequent trim.
2020-10-24 01:28:47 +03:00
Vitaliy Filippov
5fbe36198a
Fix journal trimming
...
1) Update journal's used_start in memory only after updating journal superblock.
Doing the opposite is incorrect because part of the journal will be lost if writers
overwrite its old beginning.
2) Sync journal device after updating the superblock.
3) Do not trim in rollback and init because trimming there would also require
updating the superblock. And the only reason to trim in both those places was
to unblock writers. And a guaranteed unblocking method will follow in the next
commit :)
2020-10-24 01:08:33 +03:00
Vitaliy Filippov
c696a82083
Replace assert with if + error message (may happen on metadata corruption)
2020-10-17 23:30:16 +00:00
Vitaliy Filippov
0471b09b9c
Add license notices to all source code files
2020-09-17 23:07:06 +03:00
Vitaliy Filippov
fb533991b7
"Lock" retried objects from other flushers when accounting for overruns
...
Fixes a rare 100% CPU consuming hang
2020-09-06 02:19:36 +03:00
Vitaliy Filippov
44973e7f27
Fix replicated pool bugs
2020-09-05 21:45:04 +03:00
Vitaliy Filippov
242d9a42a2
Change object format in prints to %lx:%lx v%lu
2020-09-05 17:44:05 +03:00
Vitaliy Filippov
e051db5a73
Check for unsuccessful memory allocations
2020-09-05 01:42:11 +03:00
Vitaliy Filippov
416a80b099
Make blockstore object state a combination of type and workflow
2020-07-04 22:20:32 +03:00
Vitaliy Filippov
592bcd3699
Fix QEMU driver bugs (QEMU and qemu-img now work! hooray!)
2020-06-26 18:25:43 +03:00
Vitaliy Filippov
165c204555
Fix BS_OP_DELETE (the implementation was untested up to this point)
2020-06-02 14:26:01 +03:00
Vitaliy Filippov
e6a4b634f8
Fix possible write stall
...
The stall occurred during fio Q=128 random write tests with low flusher_count (4).
It was caused by flushers being unable to flush the beginning of the journal
because it contained older writes to an object that also had writes in the very end
of the journal, after dirty_start.
2020-06-01 16:18:23 +03:00
Vitaliy Filippov
c22e096943
Output journal offsets in debug trace in hex, add detailed "still waiting" messages
2020-06-01 16:18:19 +03:00
Vitaliy Filippov
afe2e76c87
Implement regular automatic syncs, split osd_t constructor into some methods
2020-04-02 22:16:46 +03:00
Vitaliy Filippov
c3737ae3ff
Add journal fsync to stabilize/rollback
2020-03-09 00:35:58 +03:00
Vitaliy Filippov
c863543bfe
Fix possible journal corruption caused by concurrent flushing and writing of the same journal sector
2020-03-08 01:21:19 +03:00
Vitaliy Filippov
41dddddbf2
Fix some logging
2020-03-07 16:41:53 +03:00
Vitaliy Filippov
2d4e24c9ce
Add journal dumper debugging tool
2020-03-06 02:29:43 +03:00
Vitaliy Filippov
2be4824a7a
Fix a small memory leak and BS_OP_SYNC mishandling, now fio does not hang during primary-osd test
2020-02-28 01:46:39 +03:00
Vitaliy Filippov
1733de2db6
Test & fix single-PG primary OSD
...
- Add support for benchmarking single primary OSD in fio_sec_osd
- Do not wait for the next event in flushers (return resume_0 back)
- Fix flushing of zero-length writes
- Print PG object count when peering
- Print journal free space when starting and when congested
2020-02-26 19:05:29 +03:00
Vitaliy Filippov
2b09710d6f
Implement blockstore rollback operation
...
Rollback operation is required for the primary OSD to kill unstable
object versions in OSD peers so they don't occupy journal space
2020-01-24 20:18:14 +03:00
Vitaliy Filippov
d0ab2a20b2
Make fsync flags separate for data, metadata and journal
2020-01-17 13:41:37 +03:00
Vitaliy Filippov
d5386aa958
Check presence in sync_to_repeat
2020-01-16 02:16:35 +03:00
Vitaliy Filippov
43f6cfeb73
Extract alignments to options
2020-01-16 00:54:25 +03:00
Vitaliy Filippov
36d8c8724f
Fix sparse reads using bitmap, fix journal replay (we could sometimes lose its end)
2020-01-12 23:38:33 +03:00
Vitaliy Filippov
cf819eb442
Implement sparse block bitmap to avoid zero-fill
2020-01-12 02:55:32 +03:00
Vitaliy Filippov
b3f2102f33
Add queue stall tracking
2020-01-10 01:23:46 +03:00
Vitaliy Filippov
bf3eecc159
Extract 512 to constants
2020-01-06 14:11:47 +03:00
Vitaliy Filippov
4677ace4cc
Allow zero-length overwrites
2019-12-21 19:04:36 +03:00
Vitaliy Filippov
19abe6227e
Fix submission ring overflow & ring_data_t reuse conflicts
2019-12-17 11:26:17 +03:00
Vitaliy Filippov
a7e74670a5
Split blockstore implementation and interface header
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
02a0eb49c2
Begin reply code
2019-12-14 20:51:49 +03:00
Vitaliy Filippov
aaea3e1f99
Fix read_fulfill, use vector
2019-12-03 12:09:30 +03:00
Vitaliy Filippov
1f7381e220
Do not use goto resume_0 due to some strange -O3 bug which leads to hangs (it seems it optimizes flush_queue.size() away)
2019-12-03 02:23:29 +03:00
Vitaliy Filippov
fd13965b90
Force start when journal is full
2019-12-03 02:23:29 +03:00
Vitaliy Filippov
2963ad98ec
Start to flush only when there are <sync_threshold> journal entries to flush
2019-12-03 00:20:13 +03:00
Vitaliy Filippov
aa3b252327
Fsync data before writing metadata
2019-12-03 00:20:13 +03:00
Vitaliy Filippov
a165909745
FIXME
2019-12-02 18:16:23 +03:00
Vitaliy Filippov
76caecf7c7
Inmemory metadata mode
2019-12-02 15:42:42 +03:00
Vitaliy Filippov
f4d06ba102
OP_DELETE flushing
2019-12-02 02:41:14 +03:00
Vitaliy Filippov
14b2f49f4d
FIXME
2019-12-01 16:43:21 +03:00
Vitaliy Filippov
00eeedae90
Add "fsync disabled" mode
2019-12-01 16:41:07 +03:00
Vitaliy Filippov
40781c67b2
Trim journal on start
2019-11-29 02:13:32 +03:00
Vitaliy Filippov
45f34fb3b2
Fix linear overwrite, make metadata writes ordered, ignore older entries when recovering journal
2019-11-28 22:36:38 +03:00
Vitaliy Filippov
9fa0d3325f
Support inmemory journal
2019-11-28 18:06:50 +03:00
Vitaliy Filippov
e1ac4dba23
Fix safe stop procedure
2019-11-28 02:27:17 +03:00
Vitaliy Filippov
d56cb290ee
Two FIXMEs
2019-11-28 01:00:22 +03:00
Vitaliy Filippov
95396e2c83
Note read errors
2019-11-28 00:37:20 +03:00
Vitaliy Filippov
35a6ed728d
Fix another stall due to bad unstable_writes tracking, do not try to write beyond the end of the journal
2019-11-28 00:28:08 +03:00
Vitaliy Filippov
2630e2e3b9
Fix metadata partition length, fix journal allocation at the end
2019-11-27 19:39:18 +03:00