antilles
/
vitastor
forked from vitalif/vitastor
1
0
Fork 0
Code Pull Requests Activity
1,563 Commits
44 Branches
57 Tags
3.7 MiB
Commit Graph

1563 Commits (recovery-autotune)

Author SHA1 Message Date
Vitaliy Filippov d4ebbeaf5c WIP Auto-tune recovery speed 2023-12-14 01:11:57 +03:00
Vitaliy Filippov bf0c29a46c Track recovery op latencies + refactor into a structure 2023-12-14 01:11:57 +03:00
Vitaliy Filippov aca2bef15f Add vitastor-disk update-sb command 2023-12-14 01:11:42 +03:00
Vitaliy Filippov 4dd6e89263 Change qemu to qemu-system-x86 in docs 2023-12-14 01:01:00 +03:00
Vitaliy Filippov 9bac99ffb6 Fix incorrect error in CSI when searching for the device in /sys 2023-12-14 01:00:32 +03:00
Vitaliy Filippov 62ed130960 Support building qemu 8.1 from bookworm-backports 2023-12-10 00:34:13 +03:00
Vitaliy Filippov 9c7755b6e8 Use qemu-storage-daemon from QEMU 8.1.2 for CSI 2023-12-08 00:10:12 +03:00
Vitaliy Filippov 691ebd991a Move 2 last log printfs to stderr from stdout in etcd_state_client 2023-12-08 00:01:52 +03:00
Vitaliy Filippov 6d5df908a3 Fix possible out of bounds when checking invalid journal entries 2023-12-08 00:01:07 +03:00
Vitaliy Filippov fa87769ed8 Correct config options in vduse docs 2023-12-06 02:09:04 +03:00
Vitaliy Filippov 2ce8292803 Also log when killing process 2023-12-06 01:06:53 +03:00
Vitaliy Filippov 7f8f7ded52 Check for empty output of vitastor-nbd map (just in case) 2023-12-06 01:01:14 +03:00
Vitaliy Filippov 68553eabbb Log executed CLI commands 2023-12-06 00:48:12 +03:00
Vitaliy Filippov 3147c5c8d5 Remove internal error wrapping 2023-12-06 00:39:42 +03:00
Vitaliy Filippov 576e2ae608 Fix etcd_address check in CSI 2023-12-06 00:28:21 +03:00
Vitaliy Filippov a1c7cc3d8d Release 1.3.1 
Hotfix to 1.3.0 - new "journal space reservation" had a bug which
caused OSDs to crash with EC and without immediate_commit.
 2023-12-04 18:35:09 +03:00
Vitaliy Filippov a5e3dfbc5a Oops, 1.3.0 needs a hotfix 2023-12-04 13:45:54 +03:00
Vitaliy Filippov 7972502eaf Release 1.3.0 
New features:
- RDMA without ODP - much faster and all cards are now supported, not just Mellanox
- VDUSE in CSI - faster, more stable and can even recover after CSI pod restart!
- Reserve journal space for stabilize requests dynamically to prevent stalls under load with EC
- Raise default NBD timeout from 30 to 300 seconds and allow to take it from /etc/vitastor/vitastor.conf
- Remove explicit etcdUrl/etcdPrefix K8S storage class parameter support to prevent
  etcd migration issues for volumes created with these parameters
- Support QEMU 8.1 and pve-qemu 8.1

Bug fixes:
- Fix RDMA connection (and thus memory) leak
- Fix rare crashes under load due to incorrect io_uring queue size tracking
- Fix monitor statistics aggregation in case of empty /osd/stats keys
- Fix crash on unknown long argument to vitastor-disk
- Allow trailing comma in JSONs again
- Fix crash on attempts to dump a long listing of objects "to stabilize" or "to rollback" in a slow op
 2023-12-04 02:36:43 +03:00
Vitaliy Filippov e57b7203b8 Use cmake3 on RHEL 7 2023-12-04 02:36:29 +03:00
Vitaliy Filippov c8a179dcda Note that Proxmox 8.1 is supported 2023-12-04 02:20:33 +03:00
Vitaliy Filippov 845454742d Fix warning with QEMU 8.1 2023-12-04 01:59:07 +03:00
Vitaliy Filippov d65512bd80 Add patches for QEMU 8.1 2023-12-04 01:56:17 +03:00
Vitaliy Filippov 53de2bbd0f Support VDUSE in CSI 
VDUSE has multiple advantages:
- Better performance
- Lack of timeout problems
- And even the ability to recover after restart of the vitastor-csi pod!
 2023-12-04 00:41:24 +03:00
Vitaliy Filippov 628aa59574 Raise default NBD timeout from 30 to 300 seconds and allow to take it from /etc/vitastor/vitastor.conf 2023-12-02 14:11:14 +03:00
Vitaliy Filippov 037cf64a47 Remove explicit etcdUrl/etcdPrefix from volume parameters 2023-12-02 13:26:00 +03:00
Vitaliy Filippov 19e2d9d6fa Fix crash on unknown long argument to vitastor-disk 2023-12-01 00:55:51 +03:00
Vitaliy Filippov bfc7e61909 Add more notes + performance comparison about VDUSE 2023-11-25 02:25:56 +03:00
Vitaliy Filippov 7da4868b37 Fix monitor statistics aggregation in case of empty /osd/stats keys 2023-11-24 01:05:21 +03:00
Vitaliy Filippov b5c020ce0b Use io_uring SQ size for ringloop capacity - otherwise get_sqe could return NULL when space_left() was > 0 under load 
Raise default io_uring size to 1024 for the same effective capacity as previously
 2023-11-20 03:04:06 +03:00
Vitaliy Filippov 6b33ae973d %d -> %lu 2023-11-20 03:02:26 +03:00
Vitaliy Filippov cf36445359 Reserve journal space for stabilize requests dynamically to prevent stalls 2023-11-20 03:01:57 +03:00
Vitaliy Filippov 3fd873d263 Add -fno-omit-frame-pointer by default 2023-11-20 02:59:54 +03:00
Vitaliy Filippov a00e8ae9ed Fix mismatch journal pos format in vitastor-disk 2023-11-19 15:19:54 +03:00
Vitaliy Filippov 75674545dc Limit the number of printed object versions in slow op dump (otherwise it may overflow the fixed buffer) 2023-11-13 01:10:28 +03:00
Vitaliy Filippov 225eb2fe3d Support RDMA without ODP by stupidly copying memory. Disable ODP by default 
ODP is slower than regular RDMA even with memory copy overhead

Example numbers:
- 3950000 random read iops without ODP vs 240000 iops with ODP
- 1447000 random write iops without ODP vs 101000 iops with ODP

Reference: https://tkygtr6.github.io/pub/ISPASS21_slides.pdf
 2023-11-12 15:03:47 +03:00
Vitaliy Filippov 7e82573ed0 Fix RDMA connection leak which was preventing stable functioning of RDMA :) 2023-11-11 23:40:47 +03:00
Vitaliy Filippov 12a6bed2d5 Return the new accidentally rolled back json11 commit ("allow trailing comma") 2023-11-07 15:49:23 +03:00
Vitaliy Filippov 5524dbdab7 Release 1.2.0 
New features:

- Implement CSI volume expansion
- Implement CSI volume snapshots
- CSI driver now requires Kubernetes >= 1.20

Bug fixes:

- Important bug fix for EC: fix EC n+k, k>=2 read recovery in ISA-L version returning
  incorrect data when reading at least the second chunk out of multiple missing chunks
  without reading the first one. All users of EC n+k, k>=2 should upgrade as soon as
  possible, and upgrade should be conducted with downtime: first stop all clients
  (VMs/containers), then all OSDs, then upgrade and restart everything.
- Fix unstable statistics aggregation in monitor (affecting vitastor-cli status and df)
- Make udev not wait for OSDs to start during boot
- Do not report negative numbers of offline PGs in vitastor-cli status when changing PG count
- Report both old and new PG counts in vitastor-cli df when changing it
- Fix OSDs sometimes not starting with "The code only supports journal versions 1 and 2,
  but it is 2 on disk" error after upgrading from pre-1.0 versions and letting OSDs run
  for some time
- Fix monitors sometimes returning old PG count back after OSD configuration changes
- Make monitor PG changes more stable and timeout errors less probable
 2023-11-05 01:48:57 +03:00
Vitaliy Filippov cd3dec06ac Remove spaces from old->new PG count in df 2023-11-05 01:45:45 +03:00
Vitaliy Filippov 371d79e059 Document vitastor-csi features 2023-11-05 01:05:26 +03:00
Vitaliy Filippov 0e888e6c60 Prevent spamming etcd with last_clean_pgs update requests 2023-11-05 00:12:00 +03:00
Vitaliy Filippov 408c21d8f0 Scale last_clean_pgs PG count even if current PGs already contain the new number of PGs 2023-11-04 23:45:59 +03:00
Vitaliy Filippov 43cb9ae212 Prevent multiple parallel recheck_pgs in case of timeouts 2023-11-04 20:59:56 +03:00
Vitaliy Filippov e15b6e7805 Fix "cannot be narrowed" in clang 2023-11-04 18:14:44 +03:00
Vitaliy Filippov 31017d8412 Allow to start with V2 journal with header size from V1, as incorrectly updated by previous versions 2023-11-04 18:13:42 +03:00
Vitaliy Filippov 4819854064 Fix OSDs incorrectly updating journal superblock after upgrade to 1.x from pre-1.x and refusing to start after it 2023-11-04 15:02:24 +03:00
Vitaliy Filippov 1f509cca77 Fix unused capture warnings and void* arithmetic (clang) 2023-11-04 14:55:12 +03:00
Vitaliy Filippov aa8e8e8271 Add version info to --help output 2023-11-04 13:32:12 +03:00
Vitaliy Filippov 4d79e531c5 Do not print "-X offline" in status when changing pool PG count, print it in df instead 2023-11-04 13:12:13 +03:00
Vitaliy Filippov 30dff8893f Fix ISA-L version EC recovery with first missing data chunk not being read 
(Yes, all EC n + k with k >= 2 users should upgrade as soon as possible)
 2023-11-04 01:34:18 +03:00