Commit Graph

1001 Commits (9c30df83e31833248336c465ef8296b339d21e05)

Author SHA1 Message Date
Vitaliy Filippov 9c30df83e3 Fix a HUGE :) bug in NBD proxy
The bug could result in corrupted data on large writes
2022-04-03 10:42:06 +03:00
Vitaliy Filippov a420c77107 Add rebalance-verify test 2022-04-03 10:42:06 +03:00
Vitaliy Filippov 4100d829c7 Allow to override log file for daemonized NBD proxy 2022-04-03 02:41:04 +03:00
Vitaliy Filippov 79ebda933e Fix a write hang with throttling due to timer reenterability / triggerability 2022-03-28 01:42:06 +03:00
Vitaliy Filippov 65d08e067e Add a script for preparing hybrid (HDD+SSD) OSDs 2022-03-28 01:11:26 +03:00
Vitaliy Filippov d289753df4 Implement snapshot deletion for Cinder driver 2022-03-08 11:41:29 +03:00
Vitaliy Filippov 85298ddae2 Release 0.6.15
- Make peering much faster in medium to large clusters
- Fix a reenterability issue which could rarely lead to peering process hangs
2022-03-06 19:16:34 +03:00
Vitaliy Filippov e23296a327 Rename cli_rm -> cli_rm_data, cli_snap_rm -> cli_rm 2022-02-24 14:34:14 +03:00
Vitaliy Filippov 839ec9e6e0 Shard clean_db by PGs to speedup listings 2022-02-20 00:21:24 +03:00
Vitaliy Filippov 7cbfdff41a Replace some throws with force_stop 2022-02-20 00:21:19 +03:00
Vitaliy Filippov 951272f27f Try to process PG one after another 2022-02-19 19:25:55 +03:00
Vitaliy Filippov a3fb1d4c98 Fix reenterability around set_timer 2022-02-19 18:28:12 +03:00
Vitaliy Filippov 88402e6eb6 Move next_request to run_cb_and_clear 2022-02-19 16:59:03 +03:00
Vitaliy Filippov 390239c51b Don't terminate HTTP requests with timeouts if response is already available in the socket 2022-02-19 13:37:12 +03:00
Vitaliy Filippov b7b2adfa32 Fix http client not continuing requests in case of failure to connect 2022-02-19 13:36:26 +03:00
Vitaliy Filippov 36c276358b Attempt to fix "head-of-line blocking" by LIST operations 2022-02-18 01:31:45 +03:00
Vitaliy Filippov 117d6f0612 Release 0.6.14
- Fix IPv6 address parsing
- Fix "cannot read bytes of undefined" in the monitor on a fresh DB
- Fix possible hangs of read requests on OSD restarts without immediate_commit=all mode
- Fix OSDs skipping misplaced recovery in some cases
- Fix OSDs possibly dying with "map::at" errors when other OSDs are stopped
- Fix division by zero in ls if all pool OSDs are down
2022-02-17 14:43:44 +03:00
Vitaliy Filippov 7d79c58095 Use the larger sockaddr_storage structure 2022-02-12 11:22:56 +03:00
Vitaliy Filippov 46d2bc100f Add some tolerance to stat calculation so it does not fail on a fresh DB 2022-02-11 16:37:16 +03:00
Vitaliy Filippov 732e2804e9 Fix operation dependency counter underflow for reads without immediate_commit=all mode 2022-02-11 10:54:11 +03:00
Vitaliy Filippov abaec2008c Fix OSDs missing misplaced recovery 2022-02-11 01:00:24 +03:00
Vitaliy Filippov 8129d238a4 Different fio versions have different types for xfer_buflen, but Vitastor anyway does not support 128-bit offsets 2022-02-10 01:21:04 +03:00
Vitaliy Filippov 61ebed144a Fix OSDs possibly dying with "map::at" errors when other OSDs are stopped 2022-02-09 10:35:29 +03:00
Vitaliy Filippov 9d3ba113aa Extract bind socket code into a utility function 2022-02-06 00:39:52 +03:00
Vitaliy Filippov 9788045dc9 Fix division by zero in ls if all pool OSDs are down 2022-02-05 17:03:37 +03:00
Vitaliy Filippov d6b0d29af6 4k MEM_ALIGNMENT 2022-02-05 17:03:37 +03:00
Vitaliy Filippov 36f352f06f Release 0.6.13
- Fix client hangs possible on OSD restarts (bug affected versions from 0.5.11)
- Fix "Assertion `sqe != NULL' failed" io_uring-related crashes possible
  on some kernels (0.6.11 increased probability of this bug)
- Fix timeout=0 in NBD proxy
- Fix build under centos 7
2022-02-03 01:50:30 +03:00
Vitaliy Filippov 318cc463c2 Fix warnings 2022-02-03 01:50:30 +03:00
Vitaliy Filippov 145e5cfb86 MCL_ONFAULT is not available under centos 7 2022-02-03 01:42:19 +03:00
Vitaliy Filippov 73ae578981 Add osd_memlock option 2022-02-02 01:40:22 +03:00
Vitaliy Filippov 20ee4ed758 Update some parameter docs 2022-02-01 22:46:13 +03:00
Vitaliy Filippov 63de79d1b2 Change > to | to preserve newlines 2022-02-01 22:45:12 +03:00
Vitaliy Filippov f712967079 And one more sqe starvation fix 2022-02-01 02:50:16 +03:00
Vitaliy Filippov df0cd85352 Fix another part of the "async sqe clear" bug (followup to d9857a5340) 2022-02-01 01:14:56 +03:00
Vitaliy Filippov ebaf4d7a72 Fix compatibility with fio 3.28+ 2022-01-31 23:39:14 +03:00
Vitaliy Filippov d4bc10542c Fix compatibility with liburing >= 2.1 where it only has __pad2[2] 2022-01-31 22:49:40 +03:00
Vitaliy Filippov 140309620a Free recv_buf in nbd_proxy 2022-01-31 20:37:58 +03:00
Vitaliy Filippov 0a610ee943 Destroy the client after completing CLI command 2022-01-31 18:27:04 +03:00
Vitaliy Filippov f3ce166064 Do not print nan% in df when a pool has no available OSDs 2022-01-31 18:23:57 +03:00
Vitaliy Filippov 717d303370 Handle get_sqe failures, don't die with "will fall out of sync" in epoll_manager
Problem is that in recent kernels io_uring may return completions BEFORE
clearing the submission queue. I.e. for example its capacity is 512, there
were 512 requests, one of them completed, so when the request completion is
processed the queue "should have" 1 free slot. But sometimes it doesn't because
io_uring doesn't always clear the submission queue before sending CQE :-/
2022-01-31 02:52:20 +03:00
Vitaliy Filippov d9857a5340 Check for SQEs, not for completions
Should finally fix Assertion `sqe != NULL' failed introduced after journaling
refactor in 0.6.11...
2022-01-31 02:19:10 +03:00
Vitaliy Filippov eb5d9153e8 Fix build under centos 7 2022-01-30 20:29:44 +03:00
Vitaliy Filippov ae6d1ed1d5 Remove completed items 2022-01-30 20:20:06 +03:00
Vitaliy Filippov d123e58ea3 Fix yaml syntax - remove ` in default 2022-01-29 02:08:48 +03:00
Vitaliy Filippov d9869d8116 Add parameter documentation 2022-01-28 02:45:54 +03:00
Vitaliy Filippov 4047ca606f Add missing cancel_op(currently being read op) when stopping a client
Fixes client hangs possible after stopping & restarting an osd.
Hangs happened when a connection was closed in the middle of reading a READ
operation reply from the network. In this case the operation being read was
in read_op and the client didn't free it when closing the connection.

Test case for msgr_read.cpp:
- Partially read reply for a READ operation
- stop_client()
- Check that the READ operation returns EPIPE

The bug was actually introduced in 0.5.11.
2022-01-28 01:53:52 +03:00
Vitaliy Filippov 218e294e9c > 0, of course 2022-01-24 13:36:09 +03:00
Vitaliy Filippov c1929cabe0 Release 0.6.12
etcd connection stability, clang & elbrus support

- Fix build under CLang and Elbrus LCC compilers, making Vitastor compatible
  with Elbrus CPUs :)
- Completely fix the bug where OSDs didn't connect to peers and incorrectly marked
  PGs as incomplete
- Limit I/O depth for deletes the same way as for small writes. Makes OSD crashes
  with "Assertion failed: sqe != NULL" during image deletion go away
- Fix a very old, but rare, journaling bug (credits to
- Fix flushing of unclean journaled objects leading to OSDs sometimes hanging
  after failover in EC setups (bug was introduced in 0.6.7)
- Fix several problems that could prevent smooth operation of a Vitastor cluster
  under the condition of partial etcd failure:
  - OSDs could randomly fail due to too strict error handling
  - New clients and OSDs could be unable to start because of the lack of retries
  - CLI could fail some commands because of the lack of retries
  - Monitor could stop receiving state updates because of the lack of websocket pings
- Fix monitor being unable to rebalance PGs after a downscale of pool pg_size (3->2)
- Exit with failure when trying to nbd map or benchmark a non-existing image
- Use HTTP keep-alive for etcd connections
- Allow to configure etcd request timeouts and retries
- Allow to configure NBD timeout, max devices and partitions, and set default to
  up to 64 devices with up to 3 partitions each
2022-01-24 01:15:25 +03:00
Vitaliy Filippov cc6b24e03a Allow to configure NBD timeout, max devices and partitions
Also set default NBD devices/partitions to 64/3, Linux default is 16/16 which is way too low
2022-01-24 01:15:19 +03:00
Vitaliy Filippov 0757ba630a Do not happily NBD "map" non-existing images, do not try to benchmark them too 2022-01-23 23:03:42 +03:00