Vitaliy Filippov
ec7acc8f3a
Add WRITE_STABLE operation for future replication support
2020-07-05 01:48:02 +03:00
Vitaliy Filippov
416a80b099
Make blockstore object state a combination of type and workflow
2020-07-04 22:20:32 +03:00
Vitaliy Filippov
571be0f380
Make deletions instantly stable
...
"2-phase" (write->stabilize) process is pointless for deletions because it
doesn't protect us from incomplete objects. This happens because it removes
the version information from metadata after stabilization. Deletions require
"3-phase" process with a potentially very long 3rd phase.
So, deletions will be allowed to generate degraded and incomplete objects,
and for it to not affect users' ability to delete something, the cluster
will allow to delete whole inodes while storing a list of them in etcd.
Proper TRIM will be impossible until the implementation of the aforementioned
"3-phase" process, though.
By the way, this change also fixes a possible write stall after rebalancing
which was caused by the lack of "stabilize delete" operations.
2020-06-02 23:45:22 +03:00
Vitaliy Filippov
c22e096943
Output journal offsets in debug trace in hex, add detailed "still waiting" messages
2020-06-01 16:18:19 +03:00
Vitaliy Filippov
eba053febe
Do not start small writes before finishing the last big write to the same object
2020-03-12 02:15:01 +03:00
Vitaliy Filippov
3dd1b22d55
Fix segfault with concurrent OP_SYNCs
2020-03-10 17:00:23 +03:00
Vitaliy Filippov
3f522c66e6
Implement immediate commit mode
2020-03-10 01:59:15 +03:00
Vitaliy Filippov
c3737ae3ff
Add journal fsync to stabilize/rollback
2020-03-09 00:35:58 +03:00
Vitaliy Filippov
1696446545
Rename min/max _used to _flushed
2020-03-07 16:41:58 +03:00
Vitaliy Filippov
2be4824a7a
Fix a small memory leak and BS_OP_SYNC mishandling, now fio does not hang during primary-osd test
2020-02-28 01:46:39 +03:00
Vitaliy Filippov
f4707709c6
Move ack_sync() near sync_state = SYNC_DONE so it does not hang with disable_journal_fsync
2020-01-29 16:55:06 +03:00
Vitaliy Filippov
1e286eed08
Wait for writes to complete before issuing an fsync in blockstore_sync
...
Also fix a dormant bug (OP_SYNC could clear unsynced_*_writes and not be added into syncs_in_progress)
2020-01-29 16:42:40 +03:00
Vitaliy Filippov
47663bd1dc
Add (empty) osd_primary.cpp, rename osd_read to osd_receive, add FIXMEs for fsync
2020-01-28 22:40:50 +03:00
Vitaliy Filippov
d0ab2a20b2
Make fsync flags separate for data, metadata and journal
2020-01-17 13:41:37 +03:00
Vitaliy Filippov
43f6cfeb73
Extract alignments to options
2020-01-16 00:54:25 +03:00
Vitaliy Filippov
a3d3949dce
Do not overwrite same journal sector multiple times
...
It doesn't reduce actual WA, but it reduces tail latency (Q=32, 10% / 50% / 90% / 99% / 99.95%):
- write: 766us/979us/1090us/1303us/1729us vs 1074us/1450us/2212us/3261us/4113us
- sync: 701us/881us/1188us/1762us/2540us vs 269us/955us/1663us/2638us/4146us
2020-01-15 02:53:01 +03:00
Vitaliy Filippov
36d8c8724f
Fix sparse reads using bitmap, fix journal replay (we could sometimes lose its end)
2020-01-12 23:38:33 +03:00
Vitaliy Filippov
b3f2102f33
Add queue stall tracking
2020-01-10 01:23:46 +03:00
Vitaliy Filippov
a7e74670a5
Split blockstore implementation and interface header
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
749ab6e2c6
Rename blockstore_operation to blockstore_op_t
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
a7a0946ba8
WIP OP_DELETE
2019-12-01 17:25:59 +03:00
Vitaliy Filippov
00eeedae90
Add "fsync disabled" mode
2019-12-01 16:41:07 +03:00
Vitaliy Filippov
9ba243b3ee
Add debug prints
2019-11-27 18:07:51 +03:00
Vitaliy Filippov
e2b91968c5
Fix sync confirmations and some pipeline-stall bugs
2019-11-27 18:07:38 +03:00
Vitaliy Filippov
ce5cd13bc8
Use fdatasync (just for testing over an FS)
2019-11-27 02:41:30 +03:00
Vitaliy Filippov
06634054c9
Note to add no-fsync mode in the future
2019-11-27 02:27:38 +03:00
Vitaliy Filippov
2831d40edb
Fix repeated syncs
2019-11-27 02:04:46 +03:00
Vitaliy Filippov
1080cbdf12
Fix used journal sector tracking again
2019-11-27 01:20:09 +03:00
Vitaliy Filippov
b8e53f8c67
Fix op validation, fix journal sector usage tracking
...
Now it runs for a while with fio but then crashes with ENOSPC because
of an apparent bug in the allocator
2019-11-26 20:38:25 +03:00
Vitaliy Filippov
be3015169f
Safe stop procedure
2019-11-25 01:29:07 +03:00
Vitaliy Filippov
50cf3667fa
Track unstable writes
2019-11-25 01:16:34 +03:00
Vitaliy Filippov
7e87290fca
Clear second sector of the journal, init iov for callbacks
2019-11-21 22:06:00 +03:00
Vitaliy Filippov
3bfa2f5f39
Fix io_uring submission, journal sector selection
2019-11-19 18:07:40 +03:00
Vitaliy Filippov
e40a71b2ce
Check result to be equal to iov_len
2019-11-18 02:09:34 +03:00
Vitaliy Filippov
c2de733e35
Copy io_uring_prep_* to my_uring_prep_* so they do not clear user_data
2019-11-17 21:39:30 +03:00
Vitaliy Filippov
eec1c35ea4
Begin to debug ... all of this stuff :)
2019-11-16 02:32:29 +03:00
Vitaliy Filippov
0627dd0f5e
Used journal sector tracking
2019-11-15 02:04:19 +03:00
Vitaliy Filippov
f1e236c6e8
Sync metadata & data after copying from journal
2019-11-14 14:09:07 +03:00
Vitaliy Filippov
1c6b9778a4
Handle all io_uring events using lambdas
2019-11-13 22:46:42 +03:00
Vitaliy Filippov
ae77a228c7
Rename big_write.block to location
2019-11-12 20:58:27 +03:00
Vitaliy Filippov
46e96c5128
Remove duplicate journal buffer submission code
2019-11-11 18:38:57 +03:00
Vitaliy Filippov
8edb9e9d6f
Remove duplicate journal writing code (and fix it at the same time)
2019-11-11 00:28:14 +03:00
Vitaliy Filippov
7aabe11ef9
Switch object states after sync
2019-11-10 13:49:26 +03:00
Vitaliy Filippov
2d0334f9b9
Remove in_progress_ops
2019-11-10 12:52:37 +03:00
Vitaliy Filippov
ae56fe1067
Finish sync algorithm in theory
2019-11-10 12:52:37 +03:00
Vitaliy Filippov
24f5d71595
Move handle_event code into _read.cpp, _write.cpp; move sync code into _sync.cpp
2019-11-10 02:27:27 +03:00