Vitaliy Filippov
0f43f6d3f6
Fix crashes, print some stats
...
Notably:
- fix the `delete op` inside lambda callback crash (it frees the lambda itself
which results in use-after-free with g++)
- fix stop_client() reenterability
- fix a bug in the blockstore layer which resulted in always returning version=0
for zero-length reads
- change error codes for blockstore_stabilize
2020-03-31 17:55:31 +03:00
Vitaliy Filippov
46f9bd2a69
Make blockstore list operation return consistent snapshots
2020-03-14 02:10:25 +03:00
Vitaliy Filippov
6982fe1255
Do not block reads by previous unfinished writes
2020-03-13 21:28:49 +03:00
Vitaliy Filippov
eba053febe
Do not start small writes before finishing the last big write to the same object
2020-03-12 02:15:01 +03:00
Vitaliy Filippov
3f522c66e6
Implement immediate commit mode
2020-03-10 01:59:15 +03:00
Vitaliy Filippov
c863543bfe
Fix possible journal corruption caused by concurrent flushing and writing of the same journal sector
2020-03-08 01:21:19 +03:00
Vitaliy Filippov
1696446545
Rename min/max _used to _flushed
2020-03-07 16:41:58 +03:00
Vitaliy Filippov
41dddddbf2
Fix some logging
2020-03-07 16:41:53 +03:00
Vitaliy Filippov
c71b67f2f7
Move SYNC_STAB_ALL into blockstore implementation
2020-02-23 23:43:57 +03:00
Vitaliy Filippov
ffe073473a
Remove hardcode of the EC(2+1) scheme, now it supports EC(k+1), fix some bugs
2020-02-13 19:13:17 +03:00
Vitaliy Filippov
2b09710d6f
Implement blockstore rollback operation
...
Rollback operation is required for the primary OSD to kill unstable
object versions in OSD peers so they don't occupy journal space
2020-01-24 20:18:14 +03:00
Vitaliy Filippov
43f6cfeb73
Extract alignments to options
2020-01-16 00:54:25 +03:00
Vitaliy Filippov
a3d3949dce
Do not overwrite same journal sector multiple times
...
It doesn't reduce actual WA, but it reduces tail latency (Q=32, 10% / 50% / 90% / 99% / 99.95%):
- write: 766us/979us/1090us/1303us/1729us vs 1074us/1450us/2212us/3261us/4113us
- sync: 701us/881us/1188us/1762us/2540us vs 269us/955us/1663us/2638us/4146us
2020-01-15 02:53:01 +03:00
Vitaliy Filippov
111516381f
Add FIXME
2020-01-14 18:41:56 +03:00
Vitaliy Filippov
cf819eb442
Implement sparse block bitmap to avoid zero-fill
2020-01-12 02:55:32 +03:00
Vitaliy Filippov
4b05bde3a2
Block writes earlier than sync/stabilize would be blocked, too
2020-01-10 20:05:17 +03:00
Vitaliy Filippov
b3f2102f33
Add queue stall tracking
2020-01-10 01:23:46 +03:00
Vitaliy Filippov
ba23824561
Allow to disable zero fill
2020-01-06 21:02:15 +03:00
Vitaliy Filippov
bf3eecc159
Extract 512 to constants
2020-01-06 14:11:47 +03:00
Vitaliy Filippov
4677ace4cc
Allow zero-length overwrites
2019-12-21 19:04:36 +03:00
Vitaliy Filippov
e8f7905e08
Allow to set write/delete version explicitly
2019-12-19 19:17:54 +03:00
Vitaliy Filippov
d3d21e6e0f
Rename OP_ to BS_OP_
2019-12-19 13:56:26 +03:00
Vitaliy Filippov
a7e74670a5
Split blockstore implementation and interface header
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
749ab6e2c6
Rename blockstore_operation to blockstore_op_t
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
a7a0946ba8
WIP OP_DELETE
2019-12-01 17:25:59 +03:00
Vitaliy Filippov
45f34fb3b2
Fix linear overwrite, make metadata writes ordered, ignore older entries when recovering journal
2019-11-28 22:36:38 +03:00
Vitaliy Filippov
b6fff5a77e
Fix metadata area size calculation, print free space, wait for free space
...
FIXME: Now it crashes with -ENOSPC on linear overwrite
2019-11-28 20:23:27 +03:00
Vitaliy Filippov
9fa0d3325f
Support inmemory journal
2019-11-28 18:06:50 +03:00
Vitaliy Filippov
d56cb290ee
Two FIXMEs
2019-11-28 01:00:22 +03:00
Vitaliy Filippov
35a6ed728d
Fix another stall due to bad unstable_writes tracking, do not try to write beyond the end of the journal
2019-11-28 00:28:08 +03:00
Vitaliy Filippov
2630e2e3b9
Fix metadata partition length, fix journal allocation at the end
2019-11-27 19:39:18 +03:00
Vitaliy Filippov
9ba243b3ee
Add debug prints
2019-11-27 18:07:51 +03:00
Vitaliy Filippov
e2b91968c5
Fix sync confirmations and some pipeline-stall bugs
2019-11-27 18:07:38 +03:00
Vitaliy Filippov
74d8ea2f01
Calculate data crc32c
2019-11-27 02:20:38 +03:00
Vitaliy Filippov
1080cbdf12
Fix used journal sector tracking again
2019-11-27 01:20:09 +03:00
Vitaliy Filippov
ff7469ee91
Make allocator a class
2019-11-27 00:50:57 +03:00
Vitaliy Filippov
b8e53f8c67
Fix op validation, fix journal sector usage tracking
...
Now it runs for a while with fio but then crashes with ENOSPC because
of an apparent bug in the allocator
2019-11-26 20:38:25 +03:00
Vitaliy Filippov
299b7288d5
Fix journal loading
2019-11-21 00:52:52 +03:00
Vitaliy Filippov
b6174075de
Fix write iov_len check
2019-11-18 13:37:32 +03:00
Vitaliy Filippov
5b8df6768b
Try to submit a test write operation
2019-11-18 02:36:53 +03:00
Vitaliy Filippov
e40a71b2ce
Check result to be equal to iov_len
2019-11-18 02:09:34 +03:00
Vitaliy Filippov
c2de733e35
Copy io_uring_prep_* to my_uring_prep_* so they do not clear user_data
2019-11-17 21:39:30 +03:00
Vitaliy Filippov
eec1c35ea4
Begin to debug ... all of this stuff :)
2019-11-16 02:32:29 +03:00
Vitaliy Filippov
0627dd0f5e
Used journal sector tracking
2019-11-15 02:04:19 +03:00
Vitaliy Filippov
f1e236c6e8
Sync metadata & data after copying from journal
2019-11-14 14:09:07 +03:00
Vitaliy Filippov
bc549553e4
Rename dirty_entry.size to len, remove some unneeded methods and fields, read metadata correctly
2019-11-14 01:13:07 +03:00
Vitaliy Filippov
1c6b9778a4
Handle all io_uring events using lambdas
2019-11-13 22:46:42 +03:00
Vitaliy Filippov
ae77a228c7
Rename big_write.block to location
2019-11-12 20:58:27 +03:00
Vitaliy Filippov
bb55a7fbf4
Zero-fill new objects and write them to the main storage
2019-11-12 20:49:57 +03:00
Vitaliy Filippov
4afa95b0e3
FSM is a dreadful unreadable thing, reimplement using gotos
2019-11-12 18:16:41 +03:00