Commit Graph

940 Commits (098e369a3b9bd0abcd6371f3f7fb356bb8120e14)

Author SHA1 Message Date
Vitaliy Filippov 20d5ed799a Add glob pattern matching for ls 2021-12-08 23:00:34 +03:00
Vitaliy Filippov b262938bca Fix naggy "Failed to get RDMA device list: Unknown error -38" 2021-12-08 02:02:30 +03:00
Vitaliy Filippov 7e54242251 Add patches for Proxmox QEMU 5.1 and 52 2021-12-05 17:45:01 +03:00
Vitaliy Filippov c3c2e68cc1 Now fix resize command :D 2021-12-05 01:38:08 +03:00
Vitaliy Filippov aa1e21dd99 Release 0.6.9
New features:
- Build Vitastor driver as part of QEMU
- Implement renaming images in CLI (vitastor-cli modify --rename)
- Add vitastor-cli alloc-osd and simple-offsets commands and use them in make-osd,
  thus removing the dependency on etcdctl
- Make monitor remove stale deleted inode statistics from etcd automatically
- Implement OSD address selection from a subnet, thus removing the need to specify
  OSD addresses in startup scripts explicitly

Bug fixes:
- Fix client failover in case of etcd shutdown or crash (make client survive etcd failures)
- Stick to the last live etcd in OSD and mon to prevent random failures when one of etcds is down
- Fix incorrect copying of data from journal to the data device which could lead to data corruption
- Prefer local etcd IPs in OSD
- Remove the total PG count restriction in optimize_change which was sometimes leading
  to inability to redistribute PGs over OSDs
- Fix error response parsing on a failed pg state report
- Fix slow linear writes with RDMA by changing default buffer settings
- Fix possible 'TypeError' in openstack nova when using Vitastor cinder driver
- Fix bugs in vitastor-cli create, ls, rm, modify commands

Patch changes:
- Add a patch for libvirt 7.6
- Add patches for QEMU 6.0 and 6.1
- Fix config file path XML location parsing in libvirt patches
- Replace _ with - in QEMU options
- Fix possible 'TypeError' in openstack nova when using Vitastor cinder driver
- Fix possible crashes of QEMU block driver in case of incorrect options
2021-12-03 10:58:54 +03:00
Vitaliy Filippov f4b57d487f Remove +deb10u1 from libvirt version 2021-12-03 10:56:44 +03:00
Vitaliy Filippov 711ecd2f8e Add a Dockerfile to build libvirt 2021-12-03 03:00:10 +03:00
Vitaliy Filippov 9fca01dc62 Add a forgotten return statement 2021-12-03 00:41:49 +03:00
Vitaliy Filippov 0bd3a94efd Use qdict_get_try_int because qdict_get_int may segfault on a missing key 2021-12-03 00:22:17 +03:00
Vitaliy Filippov 9ffdeef93b Install the built liburing version in el8 dockerfile 2021-12-03 00:04:18 +03:00
Vitaliy Filippov 589892d501 Fix rpm dockerfiles 2021-12-02 10:19:47 +03:00
Vitaliy Filippov 5fe3a40416 More fixes for QEMU 2.x :) 2021-12-02 02:25:50 +03:00
Vitaliy Filippov a453db9c8e An attempt to automatically build patched specs inside Docker is mostly broken for now 2021-12-02 01:51:48 +03:00
Vitaliy Filippov e6498a52ca Rename 4.2 el7 spec patch 2021-12-02 01:47:44 +03:00
Vitaliy Filippov 4bc41aed9d Add patches for QEMU 6.0 and for 6.0 RPM spec 2021-12-02 01:47:20 +03:00
Vitaliy Filippov 4da51f9c4c Update QEMU 3.1 patch 2021-12-02 01:29:21 +03:00
Vitaliy Filippov c6cee6f734 Update QEMU 5.0 patch 2021-12-02 01:27:00 +03:00
Vitaliy Filippov 6fc08f5581 Update CentOS 8 QEMU 4.2 spec patch 2021-12-02 01:23:43 +03:00
Vitaliy Filippov 15957b7d13 Update QEMU 4.2 patch and CentOS 7 QEMU 4.2 spec patch 2021-12-02 01:03:19 +03:00
Vitaliy Filippov 09a3987e83 Remove vitastor-qemu from RPM specs 2021-12-01 23:42:20 +03:00
Vitaliy Filippov cd6820c439 Update QEMU 5.1/5.2 patch to include internal vitastor driver 2021-12-01 02:08:02 +03:00
Vitaliy Filippov dcd8f5e76c Remove qemu shenanigans from vitastor build dockerfile 2021-12-01 02:00:14 +03:00
Vitaliy Filippov 5859f913fc Fix client failover in case of etcd shutdown or crash 2021-12-01 00:33:02 +03:00
Vitaliy Filippov cac6a1d8d1 Don't need to download fio in qemu dockerfile anymore 2021-11-30 22:24:48 +03:00
Vitaliy Filippov a0c32e7de9 Update patch for libvirt 7.6 2021-11-30 10:23:51 +03:00
Vitaliy Filippov 8b37610dd0 Rename patch to Nova 23 because it's actually closer 2021-11-30 02:05:24 +03:00
Vitaliy Filippov ae82ca3b08 Pass config path in <config file="" /> element instead of an attribute
libvirt has always had config in the element for Ceph, we don't want to break it
2021-11-30 02:02:07 +03:00
Vitaliy Filippov 92362027a8 Build vitastor driver as part of the QEMU package by default
Old behaviour can be restored with cmake var WITH_QEMU=true
2021-11-29 02:05:26 +03:00
Vitaliy Filippov c4aeeda143 Fix index removal in vitastor-cli rm 2021-11-29 02:00:05 +03:00
Vitaliy Filippov 24f0f8278a Fix modify --readwrite 2021-11-29 01:52:21 +03:00
Vitaliy Filippov 95496d0845 Implement renaming images in CLI (vitastor-cli modify --rename) 2021-11-28 22:38:57 +03:00
Vitaliy Filippov 94b1f09ef2 Create snapshots in the same pool by default 2021-11-28 21:50:42 +03:00
Vitaliy Filippov 32b1312abb Remove stale deleted inode statistics in monitor 2021-11-28 21:02:05 +03:00
Vitaliy Filippov d5c8fde5de Remove kludgy $IP and $ETCD_MON parsing from, suggest to use vitastor.conf 2021-11-28 18:27:05 +03:00
Vitaliy Filippov 7a0b5212fe Exit if unable to restart watches
FIXME: It's probably not OK for the client to exit in this case
2021-11-28 01:43:31 +03:00
Vitaliy Filippov a8f5c71ae8 Use the same etcd address selection algorithm in the monitor 2021-11-28 01:19:42 +03:00
Vitaliy Filippov ce5b6253ab Make OSDs stick to the last successful etcd address
Previously OSDs were selecting a new random etcd from the cluster
on every request so they were failing randomly when part of etcds was down
2021-11-27 23:48:56 +03:00
Vitaliy Filippov 8398ad0117 Fix #36 - Fix old version data sometimes overriding new version data
Reproduction case:
- v3 = (offset 4kb, length 16kb)
- v2 = (offset 24kb, length 16kb)
- v1 = (offset 16kb, length 16kb)
- At the third step it was inserting 16..24kb instead of 20..24kb
2021-11-27 01:17:45 +03:00
Vitaliy Filippov fea451b4db Prefer local etcd in OSD 2021-11-27 00:36:53 +03:00
Vitaliy Filippov 6e12aca53b Remove the total PG count restriction in optimize_change which was leading to unfeasible problems sometimes 2021-11-26 23:05:37 +03:00
Vitaliy Filippov 8b007d531f
Merge pull request #33 from moly7x/fix-TypeError
Fixed TypeError
2021-11-25 14:47:31 +03:00
Vitaliy Filippov 7b7f20fb89
Merge pull request #34 from mirrorll/master
report pg state failed
2021-11-25 10:26:42 +03:00
Vitaliy Filippov 300d507026 Fix capture of out in alloc_osd 2021-11-25 10:20:01 +03:00
harley 6886171289
report pg state failed
after report pg state failed parse response error
2021-11-25 09:34:34 +08:00
Vitaliy Filippov 43f8ea47a0 Ok, something is not allowed somewhere in C99 2021-11-24 11:28:10 +03:00
Vitaliy Filippov 6e0e172e15 Implement OSD address selection from a specified subnet 2021-11-23 21:59:26 +03:00
Vitaliy Filippov 655a2c871d Move into vitastor-client package 2021-11-21 16:33:33 +03:00
Vitaliy Filippov 879fe9b2b4 Add a patch for qemu 6.1 and replace _ with - in qemu options 2021-11-21 16:24:30 +03:00
Vitaliy Filippov 660c3f7b0d Change default RDMA settings to 128x 129K buffers
129K to leave extra space for the header

The problem with 8x 1M buffers is that the following happens with,
for example, 2 OSDs and 4M T1Q1 write:
- Server posts 8 receives
- Client posts 8 sends
- WRs are processed by the RDMA stack, but the OSD doesn't have the time
  to handle them and doesn't refill buffers
- Client posts 1 more send
- RNR retransmission happens and performance drops to zero

Overall it seems that RDMA support should be reworked to use real 'RDMA'
operations i.e. operations writing into remote memory. This has an
additional advantage of avoiding a copy at the receive side of the OSD.
2021-11-21 12:05:52 +03:00
Vitaliy Filippov f0ebfae3b8 Fix vitastor-cli alloc-osd, use vitastor-cli in 2021-11-21 00:01:03 +03:00