Vitaliy Filippov
352caeba14
Fix one more bug with replicated reads
2020-09-06 02:27:44 +03:00
Vitaliy Filippov
fb533991b7
"Lock" retried objects from other flushers when accounting for overruns
...
Fixes a rare 100% CPU consuming hang
2020-09-06 02:19:36 +03:00
Vitaliy Filippov
73e26dbbea
Add up_wait_retry_interval to config and fix it so it actually works
2020-09-05 22:05:21 +03:00
Vitaliy Filippov
44973e7f27
Fix replicated pool bugs
2020-09-05 21:45:04 +03:00
Vitaliy Filippov
242d9a42a2
Change object format in prints to %lx:%lx v%lu
2020-09-05 17:44:05 +03:00
Vitaliy Filippov
68c3e96e46
Add pool setting to fio and qemu drivers
2020-09-05 17:44:00 +03:00
Vitaliy Filippov
cc4714a3a7
Basic fixes for the Monitor
2020-09-05 02:14:43 +03:00
Vitaliy Filippov
e051db5a73
Check for unsuccessful memory allocations
2020-09-05 01:42:11 +03:00
Vitaliy Filippov
4f9b5286a0
Add replicated pool support to OSD logic
...
...in theory :-D now it needs some testing
2020-09-05 01:42:11 +03:00
Vitaliy Filippov
168cc2c803
Add pool support to OSD, part 1
...
This just fixes all the code so it builds and works like before,
but doesn't yet bring the support for replicated pools.
2020-09-04 17:04:17 +03:00
Vitaliy Filippov
4cdad634b5
Add pool support to the cluster client
2020-09-03 00:52:41 +03:00
Vitaliy Filippov
293cb5bd1d
Parse pool configuration in etcd_state_client
2020-09-02 21:54:32 +03:00
Vitaliy Filippov
0918ea08fa
Implement min/max inode filters in LIST operation
2020-09-02 14:42:40 +03:00
Vitaliy Filippov
a8b3cbd6af
Implement per-pool PG calculation, fix some lint warnings
2020-09-01 18:53:54 +03:00
Vitaliy Filippov
fe0d78bf8e
Filter configuration keys with regexps instead of "osd_tree" value checks
2020-09-01 17:07:06 +03:00
Vitaliy Filippov
085c145a18
Document etcd data (to-be state with pools) at least in some form
2020-09-01 16:29:45 +03:00
Vitaliy Filippov
30da4bddbe
Extract scale_pg_count into a separate file
2020-09-01 16:18:58 +03:00
Vitaliy Filippov
14b4a4617e
(re)move placement_tree
2020-09-01 16:18:58 +03:00
Vitaliy Filippov
3932c9b2e2
Add WRITE_STABLE to the secondary OSD for the upcoming replication support
2020-09-01 16:18:58 +03:00
Vitaliy Filippov
2e8c69fc5b
Rename OSD_OP_SECONDARY_* to OSD_OP_SEC_*
2020-08-31 23:57:50 +03:00
Vitaliy Filippov
a86788fe3b
Support optimizing for the case when parity chunks occupy more space than data chunks
...
Mostly as an experiment because the problem solved by this commit
comes from Ceph's EC+compression implementation details and I'm not sure
if my implementation will be the same
2020-08-17 01:44:19 +03:00
Vitaliy Filippov
95ebfad283
Final name is Vitastor
2020-08-03 23:50:59 +03:00
Vitaliy Filippov
6022f28dc9
Add pseudo-random PG generation
2020-07-07 23:13:07 +03:00
Vitaliy Filippov
9d10a4d057
Support arbitrary pg_size in LPOptimizer
2020-07-05 20:28:05 +03:00
Vitaliy Filippov
ec7acc8f3a
Add WRITE_STABLE operation for future replication support
2020-07-05 01:48:02 +03:00
Vitaliy Filippov
416a80b099
Make blockstore object state a combination of type and workflow
2020-07-04 22:20:32 +03:00
Vitaliy Filippov
a7929931eb
Implement PG epochs to prevent the "version split"
...
The "version split" is when:
- A block is written to 1 OSD out of 3, all of them die
- OSDs 2 and 3 come up, the same block is written to both of them
- The remaining OSD comes up. Now all 3 OSDs have the same version of the same object,
but with different data.
2020-07-04 00:55:27 +03:00
Vitaliy Filippov
e680d6c1c3
Rename reconstruct_stripe and calc_rmw_parity to indicate that they are only for XOR N+1
2020-06-30 10:40:43 +03:00
Vitaliy Filippov
9b33f598d3
Fix two more cluster client bugs
...
1) Sync could delete an unfinished write due to the lack of ordering (fixed by introducing syncing_writes)
2) Writes could be postponed indefinitely due to bad resuming of operations after a sync
2020-06-27 02:13:35 +03:00
Vitaliy Filippov
592bcd3699
Fix QEMU driver bugs (QEMU and qemu-img now work! hooray!)
2020-06-26 18:25:43 +03:00
Vitaliy Filippov
5e1e39633d
Implement QEMU block driver
2020-06-25 11:59:43 +03:00
Vitaliy Filippov
41c2655edd
Disconnect sockets when read returns zero
2020-06-24 01:32:19 +03:00
Vitaliy Filippov
d68370304e
Support iovecs in cluster_client_t
2020-06-24 01:31:48 +03:00
Vitaliy Filippov
a22d9f38aa
Only use EPOLLOUT while connecting
2020-06-23 20:18:31 +03:00
Vitaliy Filippov
8736b3ad32
Add destructors, make ringloop optional in cluster_client_t
2020-06-23 20:10:33 +03:00
Vitaliy Filippov
62343c8022
Allow to turn synchronous recvmsg/sendmsg on with a config option
2020-06-23 01:15:07 +03:00
Vitaliy Filippov
9abaf5b735
Use epoll_manager in osd
2020-06-20 01:28:18 +03:00
Vitaliy Filippov
badf68c039
Support iovecs for read operations
2020-06-19 19:47:05 +03:00
Vitaliy Filippov
0f6d193d73
Postpone op callbacks to the end of handle_read(), fix a bug where primary OSD could reply -EPIPE with data to a read operation
2020-06-16 01:36:38 +03:00
Vitaliy Filippov
27ee14a4e6
Fix bugs in cluster_client
2020-06-16 00:08:45 +03:00
Vitaliy Filippov
64afec03ec
In theory, implement syncs and replay for the non-immediate commit mode
2020-06-15 00:04:16 +03:00
Vitaliy Filippov
4dde8b8a42
Oops, fix fio_sec_osd block_order parsing
2020-06-09 00:52:00 +03:00
Vitaliy Filippov
f5ccb154af
Benchmark reads in stub_bench, too
2020-06-08 01:54:44 +03:00
Vitaliy Filippov
73c80e2c39
Move accept_connections() to osd_messenger_t, add a simple uring OSD stub
2020-06-08 01:32:16 +03:00
Vitaliy Filippov
437dc5b630
Implement a FIO engine for testing cluster I/O
2020-06-07 00:30:15 +03:00
Vitaliy Filippov
226f5a2945
Allow to override block_size in fio_sec_osd
2020-06-07 00:10:13 +03:00
Vitaliy Filippov
2187d06eac
Add a parameter to pass the initial config to client
2020-06-07 00:10:12 +03:00
Vitaliy Filippov
c573bc6bb3
(Probably almost) implement cluster client
2020-06-07 00:09:36 +03:00
Vitaliy Filippov
2f6cf605a1
Rename cluster_client to osd_messenger
2020-06-04 12:57:54 +03:00
Vitaliy Filippov
05ea97119f
Fix BS_OP_LIST to account for deleted objects: only list the newest stable entry of each object
...
This allows list responses to be unaffected by journal flushes, which, in turn,
fixes PG peering when a peer OSD is replaying journal and journal contains deletions
2020-06-02 23:52:48 +03:00