Commit Graph

965 Commits (140309620aba6e3697b35bdac1851a3f0090d4d6)

Author SHA1 Message Date
Vitaliy Filippov f93491bc6c Implement journal write batching and slightly refactor journal writes
Slightly reduces WA. For example, in 4K T1Q128 replicated randwrite tests
WA is reduced from ~3.6 to ~3.1, in T1Q64 from ~3.8 to ~3.4.

Only effective without no_same_sector_overwrites.
2021-12-16 00:27:17 +03:00
Vitaliy Filippov 999bed8514 Fix opening regular files as blockstore 2021-12-15 02:08:58 +03:00
Vitaliy Filippov 3f33095fd7 Do not try to initialize client in simple-offsets 2021-12-15 02:07:27 +03:00
Vitaliy Filippov dd74c5ce1b Fix OSDs marking PGs incomplete instead of trying to connect with peers 2021-12-14 01:57:51 +03:00
Vitaliy Filippov c6d104ecd6 Print object version on fatal overwrite 2021-12-14 01:57:04 +03:00
Vitaliy Filippov e544aef7d0 Fix test rw_blocking 2021-12-12 23:24:50 +03:00
Vitaliy Filippov 616c18c786 Fix stub_uring_osd 2021-12-12 23:06:11 +03:00
Vitaliy Filippov fa687d3878 Allow to configure OSD placement in node_placement 2021-12-12 01:25:45 +03:00
Vitaliy Filippov 2c7556e536 Allow to run with 4k sector size. Natural, but it was forbidden 2021-12-11 22:03:16 +00:00
Vitaliy Filippov 2020608a39 Release 0.6.10
- Implement a storage plugin for Proxmox. Now you can use Vitastor with Proxmox!
- Implement `vitastor-cli df` (pool space usage statistics) command
- Add glob pattern support for `vitastor-cli ls`
- Fix several bugs in other CLI commands (resize, create --parent, modify --readonly)
- Use 512 byte logical block size in QEMU driver by default (and thus don't require to set it in QEMU options)
2021-12-10 21:40:12 +03:00
Vitaliy Filippov 139b98d80f Exclude block/vitastor.c from patches and add script to easily re-add it 2021-12-10 21:38:36 +03:00
Vitaliy Filippov f54ff6ad5d Do not crash in simple-offsets when some options are empty, too 2021-12-10 12:27:25 +03:00
Vitaliy Filippov b376ef2ed9 Do not crash on empty matched_addrs 2021-12-10 11:40:59 +03:00
Vitaliy Filippov 5a234588b9 Do not die when invoked via `vita` symlink 2021-12-10 02:45:16 +03:00
Vitaliy Filippov b82c30328f Use vitastor-cli df to show pool stats in Proxmox 2021-12-10 02:42:31 +03:00
Vitaliy Filippov 0ee5e0a7fe Implement vitastor-cli df command 2021-12-10 02:37:02 +03:00
Vitaliy Filippov 0a1640d169 Some important fixes for our new Proxmox driver 2021-12-10 01:17:06 +03:00
Vitaliy Filippov 3482bb0860 Fix readonly/readwrite option parsing 2021-12-10 00:52:59 +03:00
Vitaliy Filippov 526995f486 Do not skip empty iops in listings 2021-12-10 00:52:59 +03:00
Vitaliy Filippov 073b505928 Package Proxmox plugin as pve-storage-vitastor 2021-12-10 00:22:45 +03:00
Vitaliy Filippov a8b21a22d0 Add patch for pve-qemu 6.1 2021-12-09 02:57:43 +03:00
Vitaliy Filippov 0b1ffba62b Add Proxmox storage driver 2021-12-09 02:26:54 +03:00
Vitaliy Filippov 8dfbd7943c Use logical block size = 512 bytes by default 2021-12-08 23:43:40 +03:00
Vitaliy Filippov 39e7f98e54 Allow to change etcd IP in tests 2021-12-08 23:00:48 +03:00
Vitaliy Filippov 3a83a32cb7 Aaand now fix create --parent :D 2021-12-08 23:00:34 +03:00
Vitaliy Filippov 20d5ed799a Add glob pattern matching for ls 2021-12-08 23:00:34 +03:00
Vitaliy Filippov b262938bca Fix naggy "Failed to get RDMA device list: Unknown error -38" 2021-12-08 02:02:30 +03:00
Vitaliy Filippov 7e54242251 Add patches for Proxmox QEMU 5.1 and 52 2021-12-05 17:45:01 +03:00
Vitaliy Filippov c3c2e68cc1 Now fix resize command :D 2021-12-05 01:38:08 +03:00
Vitaliy Filippov aa1e21dd99 Release 0.6.9
New features:
- Build Vitastor driver as part of QEMU
- Implement renaming images in CLI (vitastor-cli modify --rename)
- Add vitastor-cli alloc-osd and simple-offsets commands and use them in make-osd,
  thus removing the dependency on etcdctl
- Make monitor remove stale deleted inode statistics from etcd automatically
- Implement OSD address selection from a subnet, thus removing the need to specify
  OSD addresses in startup scripts explicitly

Bug fixes:
- Fix client failover in case of etcd shutdown or crash (make client survive etcd failures)
- Stick to the last live etcd in OSD and mon to prevent random failures when one of etcds is down
- Fix incorrect copying of data from journal to the data device which could lead to data corruption
- Prefer local etcd IPs in OSD
- Remove the total PG count restriction in optimize_change which was sometimes leading
  to inability to redistribute PGs over OSDs
- Fix error response parsing on a failed pg state report
- Fix slow linear writes with RDMA by changing default buffer settings
- Fix possible 'TypeError' in openstack nova when using Vitastor cinder driver
- Fix bugs in vitastor-cli create, ls, rm, modify commands

Patch changes:
- Add a patch for libvirt 7.6
- Add patches for QEMU 6.0 and 6.1
- Fix config file path XML location parsing in libvirt patches
- Replace _ with - in QEMU options
- Fix possible 'TypeError' in openstack nova when using Vitastor cinder driver
- Fix possible crashes of QEMU block driver in case of incorrect options
2021-12-03 10:58:54 +03:00
Vitaliy Filippov f4b57d487f Remove +deb10u1 from libvirt version 2021-12-03 10:56:44 +03:00
Vitaliy Filippov 711ecd2f8e Add a Dockerfile to build libvirt 2021-12-03 03:00:10 +03:00
Vitaliy Filippov 9fca01dc62 Add a forgotten return statement 2021-12-03 00:41:49 +03:00
Vitaliy Filippov 0bd3a94efd Use qdict_get_try_int because qdict_get_int may segfault on a missing key 2021-12-03 00:22:17 +03:00
Vitaliy Filippov 9ffdeef93b Install the built liburing version in el8 dockerfile 2021-12-03 00:04:18 +03:00
Vitaliy Filippov 589892d501 Fix rpm dockerfiles 2021-12-02 10:19:47 +03:00
Vitaliy Filippov 5fe3a40416 More fixes for QEMU 2.x :) 2021-12-02 02:25:50 +03:00
Vitaliy Filippov a453db9c8e An attempt to automatically build patched specs inside Docker is mostly broken for now 2021-12-02 01:51:48 +03:00
Vitaliy Filippov e6498a52ca Rename 4.2 el7 spec patch 2021-12-02 01:47:44 +03:00
Vitaliy Filippov 4bc41aed9d Add patches for QEMU 6.0 and for 6.0 RPM spec 2021-12-02 01:47:20 +03:00
Vitaliy Filippov 4da51f9c4c Update QEMU 3.1 patch 2021-12-02 01:29:21 +03:00
Vitaliy Filippov c6cee6f734 Update QEMU 5.0 patch 2021-12-02 01:27:00 +03:00
Vitaliy Filippov 6fc08f5581 Update CentOS 8 QEMU 4.2 spec patch 2021-12-02 01:23:43 +03:00
Vitaliy Filippov 15957b7d13 Update QEMU 4.2 patch and CentOS 7 QEMU 4.2 spec patch 2021-12-02 01:03:19 +03:00
Vitaliy Filippov 09a3987e83 Remove vitastor-qemu from RPM specs 2021-12-01 23:42:20 +03:00
Vitaliy Filippov cd6820c439 Update QEMU 5.1/5.2 patch to include internal vitastor driver 2021-12-01 02:08:02 +03:00
Vitaliy Filippov dcd8f5e76c Remove qemu shenanigans from vitastor build dockerfile 2021-12-01 02:00:14 +03:00
Vitaliy Filippov 5859f913fc Fix client failover in case of etcd shutdown or crash 2021-12-01 00:33:02 +03:00
Vitaliy Filippov cac6a1d8d1 Don't need to download fio in qemu dockerfile anymore 2021-11-30 22:24:48 +03:00
Vitaliy Filippov a0c32e7de9 Update patch for libvirt 7.6 2021-11-30 10:23:51 +03:00