Release 0.6.10

- Implement a storage plugin for Proxmox. Now you can use Vitastor with Proxmox!
- Implement `vitastor-cli df` (pool space usage statistics) command
- Add glob pattern support for `vitastor-cli ls`
- Fix several bugs in other CLI commands (resize, create --parent, modify --readonly)
- Use 512 byte logical block size in QEMU driver by default (and thus don't require to set it in QEMU options)

CMakeLists.txt

@@ -2,6 +2,6 @@ cmake_minimum_required(VERSION 2.8)
 
 project(vitastor)
 
-set(VERSION "0.6.9")
+set(VERSION "0.6.10")
 
 add_subdirectory(src)

README-ru.md

@@ -51,13 +51,14 @@ Vitastor на данный момент находится в статусе п
 - Базовая поддержка OpenStack: драйвер Cinder, патчи для Nova и libvirt
 - Слияние снапшотов (vitastor-cli {snap-rm,flatten,merge})
 - Консольный интерфейс для управления образами (vitastor-cli {ls,create,modify})
+- Плагин для Proxmox
 
 ## Планы развития
 
 - Поддержка удаления снапшотов (слияния слоёв)
 - Более корректные скрипты разметки дисков и автоматического запуска OSD
 - Другие инструменты администрирования
-- Плагины для OpenNebula, Proxmox и других облачных систем
+- Плагины для OpenNebula и других облачных систем
 - iSCSI-прокси
 - Более быстрое переключение при отказах
 - Фоновая проверка целостности без контрольных сумм (сверка реплик)
@@ -537,6 +538,75 @@ for i in ./???-*.yaml; do kubectl apply -f $i; done
 
 После этого вы сможете создавать PersistentVolume. Пример смотрите в файле [csi/deploy/example-pvc.yaml](csi/deploy/example-pvc.yaml).
 
+### OpenStack
+
+Чтобы подключить Vitastor к OpenStack:
+
+- Установите пакеты vitastor-client, libvirt и QEMU из DEB или RPM репозитория Vitastor
+- Примените патч `patches/nova-21.diff` или `patches/nova-23.diff` к вашей инсталляции Nova.
+  nova-21.diff подходит для Nova 21-22, nova-23.diff подходит для Nova 23-24.
+- Скопируйте `patches/cinder-vitastor.py` в инсталляцию Cinder как `cinder/volume/drivers/vitastor.py`
+- Создайте тип томов в cinder.conf (см. ниже)
+- Обязательно заблокируйте доступ от виртуальных машин к сети Vitastor (OSD и etcd), т.к. Vitastor (пока) не поддерживает аутентификацию
+- Перезапустите Cinder и Nova
+
+Пример конфигурации Cinder:
+
+```
+[DEFAULT]
+enabled_backends = lvmdriver-1, vitastor-testcluster
+# ...
+
+[vitastor-testcluster]
+volume_driver = cinder.volume.drivers.vitastor.VitastorDriver
+volume_backend_name = vitastor-testcluster
+image_volume_cache_enabled = True
+volume_clear = none
+vitastor_etcd_address = 192.168.7.2:2379
+vitastor_etcd_prefix =
+vitastor_config_path = /etc/vitastor/vitastor.conf
+vitastor_pool_id = 1
+image_upload_use_cinder_backend = True
+```
+
+Чтобы помещать в Vitastor Glance-образы, нужно использовать
+[https://docs.openstack.org/cinder/pike/admin/blockstorage-volume-backed-image.html](образы на основе томов Cinder),
+однако, поддержка этой функции ещё не проверялась.
+
+### Proxmox
+
+Чтобы подключить Vitastor к Proxmox Virtual Environment (поддерживаются версии 6.4 и 7.1):
+
+- Добавьте соответствующий Debian-репозиторий Vitastor в sources.list на хостах Proxmox
+  (buster для 6.4, bullseye для 7.1)
+- Установите пакеты vitastor-client, pve-qemu-kvm, pve-storage-vitastor (* или см. сноску) из репозитория Vitastor
+- Определите тип хранилища в `/etc/pve/storage.cfg` (см. ниже)
+- Обязательно заблокируйте доступ от виртуальных машин к сети Vitastor (OSD и etcd), т.к. Vitastor (пока) не поддерживает аутентификацию
+- Перезапустите демон Proxmox: `systemctl restart pvedaemon`
+
+Пример `/etc/pve/storage.cfg` (единственная обязательная опция - vitastor_pool, все остальные
+перечислены внизу для понимания значений по умолчанию):
+
+```
+vitastor: vitastor
+    # Пул, в который будут помещаться образы дисков
+    vitastor_pool testpool
+    # Путь к файлу конфигурации
+    vitastor_config_path /etc/vitastor/vitastor.conf
+    # Адрес(а) etcd, нужны, только если не указаны в vitastor.conf
+    vitastor_etcd_address 192.168.7.2:2379/v3
+    # Префикс ключей метаданных в etcd
+    vitastor_etcd_prefix /vitastor
+    # Префикс имён образов
+    vitastor_prefix pve/
+    # Монтировать образы через NBD прокси, через ядро (нужно только для контейнеров)
+    vitastor_nbd 0
+```
+
+\* Примечание: вместо установки пакета pve-storage-vitastor вы можете вручную скопировать файл
+[patches/PVE_VitastorPlugin.pm](patches/PVE_VitastorPlugin.pm) на хосты Proxmox как
+`/usr/share/perl5/PVE/Storage/Custom/VitastorPlugin.pm`.
+
 ## Известные проблемы
 
 - Запросы удаления объектов могут в данный момент приводить к "неполным" объектам в EC-пулах,

README.md

@@ -45,6 +45,7 @@ breaking changes in the future. However, the following is implemented:
 - Basic OpenStack support: Cinder driver, Nova and libvirt patches
 - Snapshot merge tool (vitastor-cli {snap-rm,flatten,merge})
 - Image management CLI (vitastor-cli {ls,create,modify})
+- Proxmox storage plugin
 
 ## Roadmap
 
@@ -486,6 +487,73 @@ for i in ./???-*.yaml; do kubectl apply -f $i; done
 
 After that you'll be able to create PersistentVolumes. See example in [csi/deploy/example-pvc.yaml](csi/deploy/example-pvc.yaml).
 
+### OpenStack
+
+To enable Vitastor support in an OpenStack installation:
+
+- Install vitastor-client, patched QEMU and libvirt packages from Vitastor DEB or RPM repository
+- Use `patches/nova-21.diff` or `patches/nova-23.diff` to patch your Nova installation.
+  Patch 21 fits Nova 21-22, patch 23 fits Nova 23-24.
+- Install `patches/cinder-vitastor.py` as `..../cinder/volume/drivers/vitastor.py`
+- Define a volume type in cinder.conf (see below)
+- Block network access from VMs to Vitastor network (to OSDs and etcd), because Vitastor doesn't support authentication (yet)
+- Restart Cinder and Nova
+
+Cinder volume type configuration example:
+
+```
+[DEFAULT]
+enabled_backends = lvmdriver-1, vitastor-testcluster
+# ...
+
+[vitastor-testcluster]
+volume_driver = cinder.volume.drivers.vitastor.VitastorDriver
+volume_backend_name = vitastor-testcluster
+image_volume_cache_enabled = True
+volume_clear = none
+vitastor_etcd_address = 192.168.7.2:2379
+vitastor_etcd_prefix =
+vitastor_config_path = /etc/vitastor/vitastor.conf
+vitastor_pool_id = 1
+image_upload_use_cinder_backend = True
+```
+
+To put Glance images in Vitastor, use [https://docs.openstack.org/cinder/pike/admin/blockstorage-volume-backed-image.html](volume-backed images),
+although the support has not been verified yet.
+
+### Proxmox
+
+To enable Vitastor support in Proxmox Virtual Environment (6.4 and 7.1 are supported):
+
+- Add the corresponding Vitastor Debian repository into sources.list on Proxmox hosts
+  (buster for 6.4, bullseye for 7.1)
+- Install vitastor-client, pve-qemu-kvm, pve-storage-vitastor (* or see note) packages from Vitastor repository
+- Define storage in `/etc/pve/storage.cfg` (see below)
+- Block network access from VMs to Vitastor network (to OSDs and etcd), because Vitastor doesn't support authentication (yet)
+- Restart pvedaemon: `systemctl restart pvedaemon`
+
+`/etc/pve/storage.cfg` example (the only required option is vitastor_pool, all others
+are listed below with their default values):
+
+```
+vitastor: vitastor
+    # pool to put new images into
+    vitastor_pool testpool
+    # path to the configuration file
+    vitastor_config_path /etc/vitastor/vitastor.conf
+    # etcd address(es), required only if missing in the configuration file
+    vitastor_etcd_address 192.168.7.2:2379/v3
+    # prefix for keys in etcd
+    vitastor_etcd_prefix /vitastor
+    # prefix for images
+    vitastor_prefix pve/
+    # use NBD mounter (only required for containers)
+    vitastor_nbd 0
+```
+
+\* Note: you can also manually copy [patches/PVE_VitastorPlugin.pm](patches/PVE_VitastorPlugin.pm) to Proxmox hosts
+as `/usr/share/perl5/PVE/Storage/Custom/VitastorPlugin.pm` instead of installing pve-storage-vitastor.
+
 ## Known Problems
 
 - Object deletion requests may currently lead to 'incomplete' objects in EC pools

csi/Makefile

@@ -1,4 +1,4 @@
-VERSION ?= v0.6.9
+VERSION ?= v0.6.10
 
 all: build push
 

csi/deploy/004-csi-nodeplugin.yaml

@@ -49,7 +49,7 @@ spec:
             capabilities:
               add: ["SYS_ADMIN"]
             allowPrivilegeEscalation: true
-          image: vitalif/vitastor-csi:v0.6.9
+          image: vitalif/vitastor-csi:v0.6.10
           args:
             - "--node=$(NODE_ID)"
             - "--endpoint=$(CSI_ENDPOINT)"

csi/deploy/007-csi-provisioner.yaml

@@ -116,7 +116,7 @@ spec:
             privileged: true
             capabilities:
               add: ["SYS_ADMIN"]
-          image: vitalif/vitastor-csi:v0.6.9
+          image: vitalif/vitastor-csi:v0.6.10
           args:
             - "--node=$(NODE_ID)"
             - "--endpoint=$(CSI_ENDPOINT)"

csi/src/config.go

@@ -5,7 +5,7 @@ package vitastor
 
 const (
     vitastorCSIDriverName    = "csi.vitastor.io"
-    vitastorCSIDriverVersion = "0.6.9"
+    vitastorCSIDriverVersion = "0.6.10"
 )
 
 // Config struct fills the parameters of request or user input

debian/changelog

@@ -1,4 +1,4 @@
-vitastor (0.6.9-1) unstable; urgency=medium
+vitastor (0.6.10-1) unstable; urgency=medium
 
   * RDMA support
   * Bugfixes

debian/control

@@ -47,3 +47,9 @@ Architecture: amd64
 Depends: ${shlibs:Depends}, ${misc:Depends}, vitastor-client (= ${binary:Version}), fio (= ${dep:fio})
 Description: Vitastor, a fast software-defined clustered block storage - fio drivers
  Vitastor fio drivers for benchmarking.
+
+Package: pve-storage-vitastor
+Architecture: amd64
+Depends: ${shlibs:Depends}, ${misc:Depends}, vitastor-client (= ${binary:Version})
+Description: Vitastor Proxmox Virtual Environment storage plugin
+ Vitastor storage plugin for Proxmox Virtual Environment.

debian/pve-storage-vitastor.install

@@ -0,0 +1 @@
+patches/PVE_VitastorPlugin.pm usr/share/perl5/PVE/Storage/Custom/VitastorPlugin.pm

debian/vitastor.Dockerfile

@@ -33,8 +33,8 @@ RUN set -e -x; \
     mkdir -p /root/packages/vitastor-$REL; \
     rm -rf /root/packages/vitastor-$REL/*; \
     cd /root/packages/vitastor-$REL; \
-    cp -r /root/vitastor vitastor-0.6.9; \
-    cd vitastor-0.6.9; \
+    cp -r /root/vitastor vitastor-0.6.10; \
+    cd vitastor-0.6.10; \
     ln -s /root/fio-build/fio-*/ ./fio; \
     FIO=$(head -n1 fio/debian/changelog | perl -pe 's/^.*\((.*?)\).*$/$1/'); \
     ls /usr/include/linux/raw.h || cp ./debian/raw.h /usr/include/linux/raw.h; \
@@ -47,8 +47,8 @@ RUN set -e -x; \
     rm -rf a b; \
     echo "dep:fio=$FIO" > debian/fio_version; \
     cd /root/packages/vitastor-$REL; \
-    tar --sort=name --mtime='2020-01-01' --owner=0 --group=0 --exclude=debian -cJf vitastor_0.6.9.orig.tar.xz vitastor-0.6.9; \
-    cd vitastor-0.6.9; \
+    tar --sort=name --mtime='2020-01-01' --owner=0 --group=0 --exclude=debian -cJf vitastor_0.6.10.orig.tar.xz vitastor-0.6.10; \
+    cd vitastor-0.6.10; \
     V=$(head -n1 debian/changelog | perl -pe 's/^.*\((.*?)\).*$/$1/'); \
     DEBFULLNAME="Vitaliy Filippov <vitalif@yourcmc.ru>" dch -D $REL -v "$V""$REL" "Rebuild for $REL"; \
     DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage --jobs=auto -sa; \

patches/PVE_VitastorPlugin.pm

@@ -0,0 +1,503 @@
+# Install as /usr/share/perl5/PVE/Storage/Custom/VitastorPlugin.pm
+
+# Proxmox Vitastor Driver
+# Copyright (c) Vitaliy Filippov, 2021+
+# License: VNPL-1.1 or GNU AGPLv3.0
+
+package PVE::Storage::Custom::VitastorPlugin;
+
+use strict;
+use warnings;
+
+use JSON;
+
+use PVE::Storage::Plugin;
+use PVE::Tools qw(run_command);
+
+use base qw(PVE::Storage::Plugin);
+
+sub api
+{
+    # Trick it :)
+    return PVE::Storage->APIVER;
+}
+
+sub run_cli
+{
+    my ($scfg, $cmd, %args) = @_;
+    my $retval;
+    my $stderr = '';
+    my $errmsg = $args{errmsg} ? $args{errmsg}.": " : "vitastor-cli error: ";
+    my $json = delete $args{json};
+    $json = 1 if !defined $json;
+    my $binary = delete $args{binary};
+    $binary = '/usr/bin/vitastor-cli' if !defined $binary;
+    if (!exists($args{errfunc}))
+    {
+        $args{errfunc} = sub
+        {
+            my $line = shift;
+            print STDERR $line;
+            *STDERR->flush();
+            $stderr .= $line;
+        };
+    }
+    if (!exists($args{outfunc}))
+    {
+        $retval = '';
+        $args{outfunc} = sub { $retval .= shift };
+        if ($json)
+        {
+            unshift @$cmd, '--json';
+        }
+    }
+    if ($scfg->{vitastor_etcd_address})
+    {
+        unshift @$cmd, '--etcd_address', $scfg->{vitastor_etcd_address};
+    }
+    if ($scfg->{vitastor_config_path})
+    {
+        unshift @$cmd, '--config_path', $scfg->{vitastor_config_path};
+    }
+    unshift @$cmd, $binary;
+    eval { run_command($cmd, %args); };
+    if (my $err = $@)
+    {
+        die "Error invoking vitastor-cli: $err";
+    }
+    if (defined $retval)
+    {
+        # untaint
+        $retval =~ /^(.*)$/s;
+        if ($json)
+        {
+            eval { $retval = JSON::decode_json($1); };
+            if ($@)
+            {
+                die "vitastor-cli returned bad JSON: $@";
+            }
+        }
+        else
+        {
+            $retval = $1;
+        }
+    }
+    return $retval;
+}
+
+# Configuration
+
+sub type
+{
+    return 'vitastor';
+}
+
+sub plugindata
+{
+    return {
+        content => [ { images => 1, rootdir => 1 }, { images => 1 } ],
+    };
+}
+
+sub properties
+{
+    return {
+        vitastor_etcd_address => {
+            description => 'IP address(es) of etcd.',
+            type => 'string',
+            format => 'pve-storage-portal-dns-list',
+        },
+        vitastor_etcd_prefix => {
+            description => 'Prefix for Vitastor etcd metadata',
+            type => 'string',
+        },
+        vitastor_config_path => {
+            description => 'Path to Vitastor configuration file',
+            type => 'string',
+        },
+        vitastor_prefix => {
+            description => 'Image name prefix',
+            type => 'string',
+        },
+        vitastor_pool => {
+            description => 'Default pool to use for images',
+            type => 'string',
+        },
+        vitastor_nbd => {
+            description => 'Use kernel NBD devices (slower)',
+            type => 'boolean',
+        },
+    };
+}
+
+sub options
+{
+    return {
+        nodes => { optional => 1 },
+        disable => { optional => 1 },
+        vitastor_etcd_address => { optional => 1},
+        vitastor_etcd_prefix => { optional => 1 },
+        vitastor_config_path => { optional => 1 },
+        vitastor_prefix => { optional => 1 },
+        vitastor_pool => {},
+        vitastor_nbd => { optional => 1 },
+    };
+}
+
+# Storage implementation
+
+sub parse_volname
+{
+    my ($class, $volname) = @_;
+    if ($volname =~ m/^((base-(\d+)-\S+)\/)?((?:(base)|(vm))-(\d+)-\S+)$/)
+    {
+        # ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format)
+        return ('images', $4, $7, $2, $3, $5, 'raw');
+    }
+    die "unable to parse vitastor volume name '$volname'\n";
+}
+
+sub _qemu_option
+{
+    my ($k, $v) = @_;
+    if (defined $v && $v ne "")
+    {
+        $v =~ s/:/\\:/gso;
+        return ":$k=$v";
+    }
+    return "";
+}
+
+sub path
+{
+    my ($class, $scfg, $volname, $storeid, $snapname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    $name .= '@'.$snapname if $snapname;
+    if ($scfg->{vitastor_nbd})
+    {
+        my $mapped = run_cli($scfg, [ 'ls' ], binary => '/usr/bin/vitastor-nbd');
+        my ($kerneldev) = grep { $mapped->{$_}->{image} eq $prefix.$name } keys %$mapped;
+        die "Image not mapped via NBD" if !$kerneldev;
+        return ($kerneldev, $vmid, $vtype);
+    }
+    my $path = "vitastor";
+    $path .= _qemu_option('config_path', $scfg->{vitastor_config_path});
+    # FIXME This is the only exception: etcd_address -> etcd_host for qemu
+    $path .= _qemu_option('etcd_host', $scfg->{vitastor_etcd_address});
+    $path .= _qemu_option('etcd_prefix', $scfg->{vitastor_etcd_prefix});
+    $path .= _qemu_option('image', $prefix.$name);
+    return ($path, $vmid, $vtype);
+}
+
+sub _find_free_diskname
+{
+    my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_;
+    my $list = _process_list($scfg, $storeid, run_cli($scfg, [ 'ls' ]));
+    $list = [ map { $_->{name} } @$list ];
+    return PVE::Storage::Plugin::get_next_vm_diskname($list, $storeid, $vmid, undef, $scfg);
+}
+
+# Used only in "Create Template" and, in fact, converts a VM into a template
+# As a consequence, this is always invoked with the VM powered off
+# So we just rename vm-xxx to base-xxx and make it a readonly base layer
+sub create_base
+{
+    my ($class, $storeid, $scfg, $volname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+
+    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = $class->parse_volname($volname);
+    die "create_base not possible with base image\n" if $isBase;
+
+    my $info = _process_list($scfg, $storeid, run_cli($scfg, [ 'ls', $prefix.$name ]))->[0];
+    die "image $name does not exist\n" if !$info;
+
+    die "volname '$volname' contains wrong information about parent {$info->{parent}} $basename\n"
+        if $basename && (!$info->{parent} || $info->{parent} ne $basename);
+
+    my $newname = $name;
+    $newname =~ s/^vm-/base-/;
+
+    my $newvolname = $basename ? "$basename/$newname" : "$newname";
+    run_cli($scfg, [ 'modify', '--rename', $prefix.$newname, '--readonly', $prefix.$name ], json => 0);
+
+    return $newvolname;
+}
+
+sub clone_image
+{
+    my ($class, $scfg, $storeid, $volname, $vmid, $snapname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+
+    my $snap = '';
+    $snap = '@'.$snapname if length $snapname;
+
+    my ($vtype, $basename, $basevmid, undef, undef, $isBase) = $class->parse_volname($volname);
+    die "$volname is not a base image and snapname is not provided\n" if !$isBase && !length($snapname);
+
+    my $name = $class->find_free_diskname($storeid, $scfg, $vmid);
+
+    warn "clone $volname: $basename snapname $snap to $name\n";
+
+    my $newvol = "$basename/$name";
+    $newvol = $name if length($snapname);
+
+    run_cli($scfg, [ 'create', '--parent', $prefix.$basename.$snap, $prefix.$name ], json => 0);
+
+    return $newvol;
+}
+
+sub alloc_image
+{
+    # $size is in kb in this method
+    my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    die "illegal name '$name' - should be 'vm-$vmid-*'\n" if $name && $name !~ m/^vm-$vmid-/;
+    $name = $class->find_free_diskname($storeid, $scfg, $vmid) if !$name;
+    run_cli($scfg, [ 'create', '--size', (int(($size+3)/4)*4).'k', '--pool', $scfg->{vitastor_pool}, $prefix.$name ], json => 0);
+    return $name;
+}
+
+sub free_image
+{
+    my ($class, $storeid, $scfg, $volname, $isBase) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid, undef, undef, undef) = $class->parse_volname($volname);
+    $class->deactivate_volume($storeid, $scfg, $volname);
+    my $full_list = run_cli($scfg, [ 'ls', '-l' ]);
+    my $list = _process_list($scfg, $storeid, $full_list);
+    # Remove image and all its snapshots
+    my $rm_names = {
+        map { ($prefix.$_->{name} => 1) }
+        grep { $_->{name} eq $name || substr($_->{name}, 0, length($name)+1) eq ($name.'@') }
+        @$list
+    };
+    my $children = [ grep { $_->{parent_name} && $rm_names->{$_->{parent_name}} } @$full_list ];
+    die "Image has children: ".join(', ', map {
+        substr($_->{name}, 0, length $prefix) eq $prefix
+            ? substr($_->name, length $prefix)
+            : $_->{name}
+    } @$children)."\n" if @$children;
+    my $to_remove = [ grep { $rm_names->{$_->{name}} } @$full_list ];
+    for my $rmi (@$to_remove)
+    {
+        run_cli($scfg, [ 'rm-data', '--pool', $rmi->{pool_id}, '--inode', $rmi->{inode_num} ], json => 0);
+    }
+    for my $rmi (@$to_remove)
+    {
+        run_cli($scfg, [ 'rm', $rmi->{name} ], json => 0);
+    }
+    return undef;
+}
+
+sub _process_list
+{
+    my ($scfg, $storeid, $result) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my $list = [];
+    foreach my $el (@$result)
+    {
+        next if !$el->{name} || length($prefix) && substr($el->{name}, 0, length $prefix) ne $prefix;
+        my $name = substr($el->{name}, length $prefix);
+        next if $name =~ /@/;
+        my ($owner) = $name =~ /^(?:vm|base)-(\d+)-/s;
+        next if !defined $owner;
+        my $parent = !defined $el->{parent_name}
+            ? undef
+            : ($prefix eq '' || substr($el->{parent_name}, 0, length $prefix) eq $prefix
+                ? substr($el->{parent_name}, length $prefix) : '');
+        my $volid = $parent && $parent =~ /^(base-\d+-\S+)$/s
+            ? "$storeid:$1/$name" : "$storeid:$name";
+        push @$list, {
+            format => 'raw',
+            volid => $volid,
+            name => $name,
+            size => $el->{size},
+            parent => $parent,
+            vmid => $owner,
+        };
+    }
+    return $list;
+}
+
+sub list_images
+{
+    my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
+    my $list = _process_list($scfg, $storeid, run_cli($scfg, [ 'ls', '-l' ]));
+    if ($vollist)
+    {
+        my $h = { map { ($_ => 1) } @$vollist };
+        $list = [ grep { $h->{$_->{volid}} } @$list ]
+    }
+    elsif (defined $vmid)
+    {
+        $list = [ grep { $_->{vmid} eq $vmid } @$list ];
+    }
+    return $list;
+}
+
+sub status
+{
+    my ($class, $storeid, $scfg, $cache) = @_;
+    my $stats = [ grep { $_->{name} eq $scfg->{vitastor_pool} } @{ run_cli($scfg, [ 'df' ]) } ]->[0];
+    my $free = $stats ? $stats->{max_available} : 0;
+    my $used = $stats ? $stats->{used_raw}/($stats->{raw_to_usable}||1) : 0;
+    my $total = $free+$used;
+    my $active = $stats ? 1 : 0;
+    return ($total, $free, $used, $active);
+}
+
+sub activate_storage
+{
+    my ($class, $storeid, $scfg, $cache) = @_;
+    return 1;
+}
+
+sub deactivate_storage
+{
+    my ($class, $storeid, $scfg, $cache) = @_;
+    return 1;
+}
+
+sub map_volume
+{
+    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+
+    my ($vtype, $img_name, $vmid) = $class->parse_volname($volname);
+    my $name = $img_name;
+    $name .= '@'.$snapname if $snapname;
+
+    my $mapped = run_cli($scfg, [ 'ls' ], binary => '/usr/bin/vitastor-nbd');
+    my ($kerneldev) = grep { $mapped->{$_}->{image} eq $prefix.$name } keys %$mapped;
+    return $kerneldev if $kerneldev && -b $kerneldev; # already mapped
+
+    $kerneldev = run_cli($scfg, [ 'map', '--image', $prefix.$name ], binary => '/usr/bin/vitastor-nbd', json => 0);
+    return $kerneldev;
+}
+
+sub unmap_volume
+{
+    my ($class, $storeid, $scfg, $volname, $snapname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+
+    return 1 if !$scfg->{vitastor_nbd};
+
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    $name .= '@'.$snapname if $snapname;
+
+    my $mapped = run_cli($scfg, [ 'ls' ], binary => '/usr/bin/vitastor-nbd');
+    my ($kerneldev) = grep { $mapped->{$_}->{image} eq $prefix.$name } keys %$mapped;
+    if ($kerneldev && -b $kerneldev)
+    {
+        run_cli($scfg, [ 'unmap', $kerneldev ], binary => '/usr/bin/vitastor-nbd', json => 0);
+    }
+
+    return 1;
+}
+
+sub activate_volume
+{
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+    $class->map_volume($storeid, $scfg, $volname, $snapname) if $scfg->{vitastor_nbd};
+    return 1;
+}
+
+sub deactivate_volume
+{
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+    $class->unmap_volume($storeid, $scfg, $volname, $snapname);
+    return 1;
+}
+
+sub volume_size_info
+{
+    my ($class, $scfg, $storeid, $volname, $timeout) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    my $info = _process_list($scfg, $storeid, run_cli($scfg, [ 'ls', $prefix.$name ]))->[0];
+    #return wantarray ? ($size, $format, $used, $parent, $st->ctime) : $size;
+    return $info->{size};
+}
+
+sub volume_resize
+{
+    my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    # $size is in bytes in this method
+    run_cli($scfg, [ 'modify', '--resize', (int(($size+4095)/4096)*4).'k', $prefix.$name ], json => 0);
+    return undef;
+}
+
+sub volume_snapshot
+{
+    my ($class, $scfg, $storeid, $volname, $snap) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    run_cli($scfg, [ 'create', '--snapshot', $snap, $prefix.$name ], json => 0);
+    return undef;
+}
+
+sub volume_snapshot_rollback
+{
+    my ($class, $scfg, $storeid, $volname, $snap) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    run_cli($scfg, [ 'rm', $prefix.$name ], json => 0);
+    run_cli($scfg, [ 'create', '--parent', $prefix.$name.'@'.$snap, $prefix.$name ], json => 0);
+    return undef;
+}
+
+sub volume_snapshot_delete
+{
+    my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+    run_cli($scfg, [ 'rm', $prefix.$name.'@'.$snap ], json => 0);
+    return undef;
+}
+
+sub volume_snapshot_needs_fsfreeze
+{
+    return 1;
+}
+
+sub volume_has_feature
+{
+    my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_;
+    my $features = {
+        snapshot => { current => 1, snap => 1 },
+        clone => { base => 1, snap => 1 },
+        template => { current => 1 },
+        copy => { base => 1, current => 1, snap => 1 },
+        sparseinit => { base => 1, current => 1 },
+        rename => { current => 1 },
+    };
+    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = $class->parse_volname($volname);
+    my $key = undef;
+    if ($snapname)
+    {
+        $key = 'snap';
+    }
+    else
+    {
+        $key = $isBase ? 'base' : 'current';
+    }
+    return 1 if $features->{$feature}->{$key};
+    return undef;
+}
+
+sub rename_volume
+{
+    my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
+    my $prefix = defined $scfg->{vitastor_prefix} ? $scfg->{vitastor_prefix} : 'pve/';
+    my (undef, $source_image, $source_vmid, $base_name, $base_vmid, undef, $format) =
+        $class->parse_volname($source_volname);
+    $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format) if !$target_volname;
+    run_cli($scfg, [ 'modify', '--rename', $prefix.$target_volname, $prefix.$source_image ], json => 0);
+    $base_name = $base_name ? "${base_name}/" : '';
+    return "${storeid}:${base_name}${target_volname}";
+}
+
+1;

patches/cinder-vitastor.py

@@ -50,7 +50,7 @@ from cinder.volume import configuration
 from cinder.volume import driver
 from cinder.volume import volume_utils
 
-VERSION = '0.6.9'
+VERSION = '0.6.10'
 
 LOG = logging.getLogger(__name__)
 

patches/nova-21.diff

@@ -0,0 +1,288 @@
+diff --git a/nova/virt/image/model.py b/nova/virt/image/model.py
+index 971f7e9c07..ec3fca72cb 100644
+--- a/nova/virt/image/model.py
++++ b/nova/virt/image/model.py
+@@ -129,3 +129,22 @@ class RBDImage(Image):
+         self.user = user
+         self.password = password
+         self.servers = servers
++
++
++class VitastorImage(Image):
++    """Class for images in a remote Vitastor cluster"""
++
++    def __init__(self, name, etcd_address = None, etcd_prefix = None, config_path = None):
++        """Create a new Vitastor image object
++
++        :param name: name of the image
++        :param etcd_address: etcd URL(s) (optional)
++        :param etcd_prefix: etcd prefix (optional)
++        :param config_path: path to the configuration (optional)
++        """
++        super(VitastorImage, self).__init__(FORMAT_RAW)
++
++        self.name = name
++        self.etcd_address = etcd_address
++        self.etcd_prefix = etcd_prefix
++        self.config_path = config_path
+diff --git a/nova/virt/images.py b/nova/virt/images.py
+index 5358f3766a..ebe3d6effb 100644
+--- a/nova/virt/images.py
++++ b/nova/virt/images.py
+@@ -41,7 +41,7 @@ IMAGE_API = glance.API()
+ 
+ def qemu_img_info(path, format=None):
+     """Return an object containing the parsed output from qemu-img info."""
+-    if not os.path.exists(path) and not path.startswith('rbd:'):
++    if not os.path.exists(path) and not path.startswith('rbd:') and not path.startswith('vitastor:'):
+         raise exception.DiskNotFound(location=path)
+ 
+     info = nova.privsep.qemu.unprivileged_qemu_img_info(path, format=format)
+@@ -50,7 +50,7 @@ def qemu_img_info(path, format=None):
+ 
+ def privileged_qemu_img_info(path, format=None, output_format='json'):
+     """Return an object containing the parsed output from qemu-img info."""
+-    if not os.path.exists(path) and not path.startswith('rbd:'):
++    if not os.path.exists(path) and not path.startswith('rbd:') and not path.startswith('vitastor:'):
+         raise exception.DiskNotFound(location=path)
+ 
+     info = nova.privsep.qemu.privileged_qemu_img_info(path, format=format)
+diff --git a/nova/virt/libvirt/config.py b/nova/virt/libvirt/config.py
+index ea525648b3..d7aa798954 100644
+--- a/nova/virt/libvirt/config.py
++++ b/nova/virt/libvirt/config.py
+@@ -1005,6 +1005,8 @@ class LibvirtConfigGuestDisk(LibvirtConfigGuestDevice):
+         self.driver_iommu = False
+         self.source_path = None
+         self.source_protocol = None
++        self.source_query = None
++        self.source_config = None
+         self.source_name = None
+         self.source_hosts = []
+         self.source_ports = []
+@@ -1133,6 +1135,10 @@ class LibvirtConfigGuestDisk(LibvirtConfigGuestDevice):
+             source = etree.Element("source", protocol=self.source_protocol)
+             if self.source_name is not None:
+                 source.set('name', self.source_name)
++            if self.source_query is not None:
++                source.set('query', self.source_query)
++            if self.source_config is not None:
++                source.append(etree.Element('config', file=self.source_config))
+             hosts_info = zip(self.source_hosts, self.source_ports)
+             for name, port in hosts_info:
+                 host = etree.Element('host', name=name)
+diff --git a/nova/virt/libvirt/driver.py b/nova/virt/libvirt/driver.py
+index fbd033690a..74dc59ce87 100644
+--- a/nova/virt/libvirt/driver.py
++++ b/nova/virt/libvirt/driver.py
+@@ -180,6 +180,7 @@ libvirt_volume_drivers = [
+     'local=nova.virt.libvirt.volume.volume.LibvirtVolumeDriver',
+     'fake=nova.virt.libvirt.volume.volume.LibvirtFakeVolumeDriver',
+     'rbd=nova.virt.libvirt.volume.net.LibvirtNetVolumeDriver',
++    'vitastor=nova.virt.libvirt.volume.vitastor.LibvirtVitastorVolumeDriver',
+     'nfs=nova.virt.libvirt.volume.nfs.LibvirtNFSVolumeDriver',
+     'smbfs=nova.virt.libvirt.volume.smbfs.LibvirtSMBFSVolumeDriver',
+     'fibre_channel='
+@@ -287,10 +288,10 @@ class LibvirtDriver(driver.ComputeDriver):
+         # This prevents the risk of one test setting a capability
+         # which bleeds over into other tests.
+ 
+-        # LVM and RBD require raw images. If we are not configured to
++        # LVM, RBD, Vitastor require raw images. If we are not configured to
+         # force convert images into raw format, then we _require_ raw
+         # images only.
+-        raw_only = ('rbd', 'lvm')
++        raw_only = ('rbd', 'lvm', 'vitastor')
+         requires_raw_image = (CONF.libvirt.images_type in raw_only and
+                               not CONF.force_raw_images)
+         requires_ploop_image = CONF.libvirt.virt_type == 'parallels'
+@@ -703,12 +704,12 @@ class LibvirtDriver(driver.ComputeDriver):
+         # Some imagebackends are only able to import raw disk images,
+         # and will fail if given any other format. See the bug
+         # https://bugs.launchpad.net/nova/+bug/1816686 for more details.
+-        if CONF.libvirt.images_type in ('rbd',):
++        if CONF.libvirt.images_type in ('rbd', 'vitastor'):
+             if not CONF.force_raw_images:
+                 msg = _("'[DEFAULT]/force_raw_images = False' is not "
+-                        "allowed with '[libvirt]/images_type = rbd'. "
++                        "allowed with '[libvirt]/images_type = rbd' or 'vitastor'. "
+                         "Please check the two configs and if you really "
+-                        "do want to use rbd as images_type, set "
++                        "do want to use rbd or vitastor as images_type, set "
+                         "force_raw_images to True.")
+                 raise exception.InvalidConfiguration(msg)
+ 
+@@ -2165,6 +2166,16 @@ class LibvirtDriver(driver.ComputeDriver):
+                     if connection_info['data'].get('auth_enabled'):
+                         username = connection_info['data']['auth_username']
+                         path = f"rbd:{volume_name}:id={username}"
++                elif connection_info['driver_volume_type'] == 'vitastor':
++                    volume_name = connection_info['data']['name']
++                    path = 'vitastor:image='+volume_name.replace(':', '\\:')
++                    for k in [ 'config_path', 'etcd_address', 'etcd_prefix' ]:
++                        if k in connection_info['data']:
++                            kk = k
++                            if kk == 'etcd_address':
++                                # FIXME use etcd_address in qemu driver
++                                kk = 'etcd_host'
++                            path += ":"+kk.replace('_', '-')+"="+connection_info['data'][k].replace(':', '\\:')
+                 else:
+                     path = 'unknown'
+                     raise exception.DiskNotFound(location='unknown')
+@@ -2440,8 +2451,8 @@ class LibvirtDriver(driver.ComputeDriver):
+ 
+         image_format = CONF.libvirt.snapshot_image_format or source_type
+ 
+-        # NOTE(bfilippov): save lvm and rbd as raw
+-        if image_format == 'lvm' or image_format == 'rbd':
++        # NOTE(bfilippov): save lvm and rbd and vitastor as raw
++        if image_format == 'lvm' or image_format == 'rbd' or image_format == 'vitastor':
+             image_format = 'raw'
+ 
+         metadata = self._create_snapshot_metadata(instance.image_meta,
+@@ -2512,7 +2523,7 @@ class LibvirtDriver(driver.ComputeDriver):
+                               expected_state=task_states.IMAGE_UPLOADING)
+ 
+             # TODO(nic): possibly abstract this out to the root_disk
+-            if source_type == 'rbd' and live_snapshot:
++            if (source_type == 'rbd' or source_type == 'vitastor') and live_snapshot:
+                 # Standard snapshot uses qemu-img convert from RBD which is
+                 # not safe to run with live_snapshot.
+                 live_snapshot = False
+@@ -3715,7 +3726,7 @@ class LibvirtDriver(driver.ComputeDriver):
+         # cleanup rescue volume
+         lvm.remove_volumes([lvmdisk for lvmdisk in self._lvm_disks(instance)
+                                 if lvmdisk.endswith('.rescue')])
+-        if CONF.libvirt.images_type == 'rbd':
++        if CONF.libvirt.images_type == 'rbd' or CONF.libvirt.images_type == 'vitastor':
+             filter_fn = lambda disk: (disk.startswith(instance.uuid) and
+                                       disk.endswith('.rescue'))
+             rbd_utils.RBDDriver().cleanup_volumes(filter_fn)
+@@ -3972,6 +3983,8 @@ class LibvirtDriver(driver.ComputeDriver):
+         # TODO(mikal): there is a bug here if images_type has
+         # changed since creation of the instance, but I am pretty
+         # sure that this bug already exists.
++        if CONF.libvirt.images_type == 'vitastor':
++            return 'vitastor'
+         return 'rbd' if CONF.libvirt.images_type == 'rbd' else 'raw'
+ 
+     @staticmethod
+@@ -4370,10 +4383,10 @@ class LibvirtDriver(driver.ComputeDriver):
+                 finally:
+                     # NOTE(mikal): if the config drive was imported into RBD,
+                     # then we no longer need the local copy
+-                    if CONF.libvirt.images_type == 'rbd':
++                    if CONF.libvirt.images_type == 'rbd' or CONF.libvirt.images_type == 'vitastor':
+                         LOG.info('Deleting local config drive %(path)s '
+-                                 'because it was imported into RBD.',
+-                                 {'path': config_disk_local_path},
++                                 'because it was imported into %(type).',
++                                 {'path': config_disk_local_path, 'type': CONF.libvirt.images_type},
+                                  instance=instance)
+                         os.unlink(config_disk_local_path)
+ 
+diff --git a/nova/virt/libvirt/utils.py b/nova/virt/libvirt/utils.py
+index c1dc34daf4..263965912f 100644
+--- a/nova/virt/libvirt/utils.py
++++ b/nova/virt/libvirt/utils.py
+@@ -399,6 +399,10 @@ def find_disk(guest: libvirt_guest.Guest) -> ty.Tuple[str, ty.Optional[str]]:
+             disk_path = disk.source_name
+             if disk_path:
+                 disk_path = 'rbd:' + disk_path
++        elif not disk_path and disk.source_protocol == 'vitastor':
++            disk_path = disk.source_name
++            if disk_path:
++                disk_path = 'vitastor:' + disk_path
+ 
+     if not disk_path:
+         raise RuntimeError(_("Can't retrieve root device path "
+@@ -417,6 +421,8 @@ def get_disk_type_from_path(path: str) -> ty.Optional[str]:
+         return 'lvm'
+     elif path.startswith('rbd:'):
+         return 'rbd'
++    elif path.startswith('vitastor:'):
++        return 'vitastor'
+     elif (os.path.isdir(path) and
+           os.path.exists(os.path.join(path, "DiskDescriptor.xml"))):
+         return 'ploop'
+diff --git a/nova/virt/libvirt/volume/vitastor.py b/nova/virt/libvirt/volume/vitastor.py
+new file mode 100644
+index 0000000000..0256df62c1
+--- /dev/null
++++ b/nova/virt/libvirt/volume/vitastor.py
+@@ -0,0 +1,75 @@
++# Copyright (c) 2021+, Vitaliy Filippov <vitalif@yourcmc.ru>
++#
++#    Licensed under the Apache License, Version 2.0 (the "License"); you may
++#    not use this file except in compliance with the License. You may obtain
++#    a copy of the License at
++#
++#         http://www.apache.org/licenses/LICENSE-2.0
++#
++#    Unless required by applicable law or agreed to in writing, software
++#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
++#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
++#    License for the specific language governing permissions and limitations
++#    under the License.
++
++from os_brick import exception as os_brick_exception
++from os_brick import initiator
++from os_brick.initiator import connector
++from oslo_log import log as logging
++
++import nova.conf
++from nova import utils
++from nova.virt.libvirt.volume import volume as libvirt_volume
++
++
++CONF = nova.conf.CONF
++LOG = logging.getLogger(__name__)
++
++
++class LibvirtVitastorVolumeDriver(libvirt_volume.LibvirtBaseVolumeDriver):
++    """Driver to attach Vitastor volumes to libvirt."""
++    def __init__(self, host):
++        super(LibvirtVitastorVolumeDriver, self).__init__(host, is_block_dev=False)
++
++    def connect_volume(self, connection_info, instance):
++        pass
++
++    def disconnect_volume(self, connection_info, instance):
++        pass
++
++    def get_config(self, connection_info, disk_info):
++        """Returns xml for libvirt."""
++        conf = super(LibvirtVitastorVolumeDriver, self).get_config(connection_info, disk_info)
++        conf.source_type = 'network'
++        conf.source_protocol = 'vitastor'
++        conf.source_name = connection_info['data'].get('name')
++        conf.source_query = connection_info['data'].get('etcd_prefix') or None
++        conf.source_config = connection_info['data'].get('config_path') or None
++        conf.source_hosts = []
++        conf.source_ports = []
++        addresses = connection_info['data'].get('etcd_address', '')
++        if addresses:
++            if not isinstance(addresses, list):
++                addresses = addresses.split(',')
++            for addr in addresses:
++                if addr.startswith('https://'):
++                    raise NotImplementedError('Vitastor block driver does not support SSL for etcd communication yet')
++                if addr.startswith('http://'):
++                    addr = addr[7:]
++                addr = addr.rstrip('/')
++                if addr.endswith('/v3'):
++                    addr = addr[0:-3]
++                p = addr.find('/')
++                if p > 0:
++                    raise NotImplementedError('libvirt does not support custom URL paths for Vitastor etcd yet. Use /etc/vitastor/vitastor.conf')
++                p = addr.find(':')
++                port = '2379'
++                if p > 0:
++                    port = addr[p+1:]
++                    addr = addr[0:p]
++                conf.source_hosts.append(addr)
++                conf.source_ports.append(port)
++        return conf
++
++    def extend_volume(self, connection_info, instance, requested_size):
++        raise NotImplementedError

patches/pve-qemu-5.1-vitastor.patch

@@ -0,0 +1,175 @@
+Index: pve-qemu-kvm-5.1.0/qapi/block-core.json
+===================================================================
+--- pve-qemu-kvm-5.1.0.orig/qapi/block-core.json
++++ pve-qemu-kvm-5.1.0/qapi/block-core.json
+@@ -3041,7 +3041,7 @@
+             'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
+             'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+             { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
+-            'sheepdog', 'pbs',
++            'sheepdog', 'pbs', 'vitastor',
+             'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
+ 
+ ##
+@@ -3889,6 +3889,28 @@
+             '*tag': 'str' } }
+ 
+ ##
++# @BlockdevOptionsVitastor:
++#
++# Driver specific block device options for vitastor
++#
++# @image:       Image name
++# @inode:       Inode number
++# @pool:        Pool ID
++# @size:        Desired image size in bytes
++# @config-path: Path to Vitastor configuration
++# @etcd-host:   etcd connection address(es)
++# @etcd-prefix: etcd key/value prefix
++##
++{ 'struct': 'BlockdevOptionsVitastor',
++  'data': { '*inode': 'uint64',
++            '*pool': 'uint64',
++            '*size': 'uint64',
++            '*image': 'str',
++            '*config-path': 'str',
++            '*etcd-host': 'str',
++            '*etcd-prefix': 'str' } }
++
++##
+ # @ReplicationMode:
+ #
+ # An enumeration of replication modes.
+@@ -4234,6 +4256,7 @@
+       'replication': { 'type': 'BlockdevOptionsReplication',
+                        'if': 'defined(CONFIG_REPLICATION)' },
+       'sheepdog':   'BlockdevOptionsSheepdog',
++      'vitastor':   'BlockdevOptionsVitastor',
+       'ssh':        'BlockdevOptionsSsh',
+       'throttle':   'BlockdevOptionsThrottle',
+       'vdi':        'BlockdevOptionsGenericFormat',
+@@ -4623,6 +4646,17 @@
+             '*cluster-size' :   'size' } }
+ 
+ ##
++# @BlockdevCreateOptionsVitastor:
++#
++# Driver specific image creation options for Vitastor.
++#
++# @size: Size of the virtual disk in bytes
++##
++{ 'struct': 'BlockdevCreateOptionsVitastor',
++  'data': { 'location':         'BlockdevOptionsVitastor',
++            'size':             'size' } }
++
++##
+ # @BlockdevVmdkSubformat:
+ #
+ # Subformat options for VMDK images
+@@ -4884,6 +4918,7 @@
+       'qed':            'BlockdevCreateOptionsQed',
+       'rbd':            'BlockdevCreateOptionsRbd',
+       'sheepdog':       'BlockdevCreateOptionsSheepdog',
++      'vitastor':       'BlockdevCreateOptionsVitastor',
+       'ssh':            'BlockdevCreateOptionsSsh',
+       'vdi':            'BlockdevCreateOptionsVdi',
+       'vhdx':           'BlockdevCreateOptionsVhdx',
+Index: pve-qemu-kvm-5.1.0/configure
+===================================================================
+--- pve-qemu-kvm-5.1.0.orig/configure
++++ pve-qemu-kvm-5.1.0/configure
+@@ -446,6 +446,7 @@ trace_backends="log"
+ trace_file="trace"
+ spice=""
+ rbd=""
++vitastor=""
+ smartcard=""
+ libusb=""
+ usb_redir=""
+@@ -1383,6 +1384,10 @@ for opt do
+   ;;
+   --enable-rbd) rbd="yes"
+   ;;
++  --disable-vitastor) vitastor="no"
++  ;;
++  --enable-vitastor) vitastor="yes"
++  ;;
+   --disable-xfsctl) xfs="no"
+   ;;
+   --enable-xfsctl) xfs="yes"
+@@ -1901,6 +1906,7 @@ disabled with --disable-FEATURE, default
+   vhost-vdpa      vhost-vdpa kernel backend support
+   spice           spice
+   rbd             rados block device (rbd)
++  vitastor        vitastor block device
+   libiscsi        iscsi support
+   libnfs          nfs support
+   smartcard       smartcard support (libcacard)
+@@ -4234,6 +4240,27 @@ EOF
+ fi
+ 
+ ##########################################
++# vitastor probe
++if test "$vitastor" != "no" ; then
++  cat > $TMPC <<EOF
++#include <vitastor_c.h>
++int main(void) {
++  vitastor_c_create_qemu(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
++  return 0;
++}
++EOF
++  vitastor_libs="-lvitastor_client"
++  if compile_prog "" "$vitastor_libs" ; then
++    vitastor=yes
++  else
++    if test "$vitastor" = "yes" ; then
++      feature_not_found "vitastor block device" "Install vitastor-client-dev"
++    fi
++    vitastor=no
++  fi
++fi
++
++##########################################
+ # libssh probe
+ if test "$libssh" != "no" ; then
+   if $pkg_config --exists libssh; then
+@@ -6969,6 +6996,7 @@ echo "Trace output file $trace_file-<pid
+ fi
+ echo "spice support     $spice $(echo_version $spice $spice_protocol_version/$spice_server_version)"
+ echo "rbd support       $rbd"
++echo "vitastor support  $vitastor"
+ echo "xfsctl support    $xfs"
+ echo "smartcard support $smartcard"
+ echo "libusb            $libusb"
+@@ -7644,6 +7672,10 @@ if test "$rbd" = "yes" ; then
+   echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
+   echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
+ fi
++if test "$vitastor" = "yes" ; then
++  echo "CONFIG_VITASTOR=y" >> $config_host_mak
++  echo "VITASTOR_LIBS=$vitastor_libs" >> $config_host_mak
++fi
+ 
+ echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak
+ if test "$coroutine_pool" = "yes" ; then
+Index: pve-qemu-kvm-5.1.0/block/Makefile.objs
+===================================================================
+--- pve-qemu-kvm-5.1.0.orig/block/Makefile.objs
++++ pve-qemu-kvm-5.1.0/block/Makefile.objs
+@@ -32,6 +32,7 @@ block-obj-$(if $(CONFIG_LIBISCSI),y,n) +
+ block-obj-$(CONFIG_LIBNFS) += nfs.o
+ block-obj-$(CONFIG_CURL) += curl.o
+ block-obj-$(CONFIG_RBD) += rbd.o
++block-obj-$(CONFIG_VITASTOR) += vitastor.o
+ block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+ block-obj-$(CONFIG_LIBSSH) += ssh.o
+ block-obj-y += backup-dump.o
+@@ -61,6 +62,8 @@ curl.o-cflags      := $(CURL_CFLAGS)
+ curl.o-libs        := $(CURL_LIBS)
+ rbd.o-cflags       := $(RBD_CFLAGS)
+ rbd.o-libs         := $(RBD_LIBS)
++vitastor.o-cflags  := $(VITASTOR_CFLAGS)
++vitastor.o-libs    := $(VITASTOR_LIBS)
+ gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
+ gluster.o-libs     := $(GLUSTERFS_LIBS)
+ ssh.o-cflags       := $(LIBSSH_CFLAGS)

patches/pve-qemu-5.2-vitastor.patch

@@ -0,0 +1,181 @@
+Index: pve-qemu-kvm-5.2.0/qapi/block-core.json
+===================================================================
+--- pve-qemu-kvm-5.2.0.orig/qapi/block-core.json
++++ pve-qemu-kvm-5.2.0/qapi/block-core.json
+@@ -3076,7 +3076,7 @@
+             'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
+             'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+             { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
+-            'sheepdog', 'pbs',
++            'sheepdog', 'pbs', 'vitastor',
+             'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
+ 
+ ##
+@@ -3924,6 +3924,28 @@
+             '*tag': 'str' } }
+ 
+ ##
++# @BlockdevOptionsVitastor:
++#
++# Driver specific block device options for vitastor
++#
++# @image:       Image name
++# @inode:       Inode number
++# @pool:        Pool ID
++# @size:        Desired image size in bytes
++# @config-path: Path to Vitastor configuration
++# @etcd-host:   etcd connection address(es)
++# @etcd-prefix: etcd key/value prefix
++##
++{ 'struct': 'BlockdevOptionsVitastor',
++  'data': { '*inode': 'uint64',
++            '*pool': 'uint64',
++            '*size': 'uint64',
++            '*image': 'str',
++            '*config-path': 'str',
++            '*etcd-host': 'str',
++            '*etcd-prefix': 'str' } }
++
++##
+ # @ReplicationMode:
+ #
+ # An enumeration of replication modes.
+@@ -4272,6 +4294,7 @@
+       'replication': { 'type': 'BlockdevOptionsReplication',
+                        'if': 'defined(CONFIG_REPLICATION)' },
+       'sheepdog':   'BlockdevOptionsSheepdog',
++      'vitastor':   'BlockdevOptionsVitastor',
+       'ssh':        'BlockdevOptionsSsh',
+       'throttle':   'BlockdevOptionsThrottle',
+       'vdi':        'BlockdevOptionsGenericFormat',
+@@ -4662,6 +4685,17 @@
+             '*cluster-size' :   'size' } }
+ 
+ ##
++# @BlockdevCreateOptionsVitastor:
++#
++# Driver specific image creation options for Vitastor.
++#
++# @size: Size of the virtual disk in bytes
++##
++{ 'struct': 'BlockdevCreateOptionsVitastor',
++  'data': { 'location':         'BlockdevOptionsVitastor',
++            'size':             'size' } }
++
++##
+ # @BlockdevVmdkSubformat:
+ #
+ # Subformat options for VMDK images
+@@ -4923,6 +4957,7 @@
+       'qed':            'BlockdevCreateOptionsQed',
+       'rbd':            'BlockdevCreateOptionsRbd',
+       'sheepdog':       'BlockdevCreateOptionsSheepdog',
++      'vitastor':       'BlockdevCreateOptionsVitastor',
+       'ssh':            'BlockdevCreateOptionsSsh',
+       'vdi':            'BlockdevCreateOptionsVdi',
+       'vhdx':           'BlockdevCreateOptionsVhdx',
+Index: pve-qemu-kvm-5.2.0/block/meson.build
+===================================================================
+--- pve-qemu-kvm-5.2.0.orig/block/meson.build
++++ pve-qemu-kvm-5.2.0/block/meson.build
+@@ -89,6 +89,7 @@ foreach m : [
+   ['CONFIG_LIBNFS', 'nfs', libnfs, 'nfs.c'],
+   ['CONFIG_LIBSSH', 'ssh', libssh, 'ssh.c'],
+   ['CONFIG_RBD', 'rbd', rbd, 'rbd.c'],
++  ['CONFIG_VITASTOR', 'vitastor', vitastor, 'vitastor.c'],
+ ]
+   if config_host.has_key(m[0])
+     if enable_modules
+Index: pve-qemu-kvm-5.2.0/configure
+===================================================================
+--- pve-qemu-kvm-5.2.0.orig/configure
++++ pve-qemu-kvm-5.2.0/configure
+@@ -372,6 +372,7 @@ trace_backends="log"
+ trace_file="trace"
+ spice=""
+ rbd=""
++vitastor=""
+ smartcard=""
+ u2f="auto"
+ libusb=""
+@@ -1264,6 +1265,10 @@ for opt do
+   ;;
+   --enable-rbd) rbd="yes"
+   ;;
++  --disable-vitastor) vitastor="no"
++  ;;
++  --enable-vitastor) vitastor="yes"
++  ;;
+   --disable-xfsctl) xfs="no"
+   ;;
+   --enable-xfsctl) xfs="yes"
+@@ -1807,6 +1812,7 @@ disabled with --disable-FEATURE, default
+   vhost-vdpa      vhost-vdpa kernel backend support
+   spice           spice
+   rbd             rados block device (rbd)
++  vitastor        vitastor block device
+   libiscsi        iscsi support
+   libnfs          nfs support
+   smartcard       smartcard support (libcacard)
+@@ -3700,6 +3706,27 @@ EOF
+ fi
+ 
+ ##########################################
++# vitastor probe
++if test "$vitastor" != "no" ; then
++  cat > $TMPC <<EOF
++#include <vitastor_c.h>
++int main(void) {
++  vitastor_c_create_qemu(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
++  return 0;
++}
++EOF
++  vitastor_libs="-lvitastor_client"
++  if compile_prog "" "$vitastor_libs" ; then
++    vitastor=yes
++  else
++    if test "$vitastor" = "yes" ; then
++      feature_not_found "vitastor block device" "Install vitastor-client-dev"
++    fi
++    vitastor=no
++  fi
++fi
++
++##########################################
+ # libssh probe
+ if test "$libssh" != "no" ; then
+   if $pkg_config --exists libssh; then
+@@ -6437,6 +6464,10 @@ if test "$rbd" = "yes" ; then
+   echo "CONFIG_RBD=y" >> $config_host_mak
+   echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
+ fi
++if test "$vitastor" = "yes" ; then
++  echo "CONFIG_VITASTOR=y" >> $config_host_mak
++  echo "VITASTOR_LIBS=$vitastor_libs" >> $config_host_mak
++fi
+ 
+ echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak
+ if test "$coroutine_pool" = "yes" ; then
+Index: pve-qemu-kvm-5.2.0/meson.build
+===================================================================
+--- pve-qemu-kvm-5.2.0.orig/meson.build
++++ pve-qemu-kvm-5.2.0/meson.build
+@@ -596,6 +596,10 @@ rbd = not_found
+ if 'CONFIG_RBD' in config_host
+   rbd = declare_dependency(link_args: config_host['RBD_LIBS'].split())
+ endif
++vitastor = not_found
++if 'CONFIG_VITASTOR' in config_host
++  vitastor = declare_dependency(link_args: config_host['VITASTOR_LIBS'].split())
++endif
+ glusterfs = not_found
+ if 'CONFIG_GLUSTERFS' in config_host
+   glusterfs = declare_dependency(compile_args: config_host['GLUSTERFS_CFLAGS'].split(),
+@@ -2151,6 +2155,7 @@ endif
+ # TODO: add back protocol and server version
+ summary_info += {'spice support':     config_host.has_key('CONFIG_SPICE')}
+ summary_info += {'rbd support':       config_host.has_key('CONFIG_RBD')}
++summary_info += {'vitastor support':  config_host.has_key('CONFIG_VITASTOR')}
+ summary_info += {'xfsctl support':    config_host.has_key('CONFIG_XFS')}
+ summary_info += {'smartcard support': config_host.has_key('CONFIG_SMARTCARD')}
+ summary_info += {'U2F support':       u2f.found()}

patches/pve-qemu-6.1-vitastor.patch

@@ -0,0 +1,188 @@
+Index: pve-qemu-kvm-6.1.0/qapi/block-core.json
+===================================================================
+--- pve-qemu-kvm-6.1.0.orig/qapi/block-core.json
++++ pve-qemu-kvm-6.1.0/qapi/block-core.json
+@@ -3084,7 +3084,7 @@
+             'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+             { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
+             'pbs',
+-            'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
++            'ssh', 'throttle', 'vdi', 'vhdx', 'vitastor', 'vmdk', 'vpc', 'vvfat' ] }
+ 
+ ##
+ # @BlockdevOptionsFile:
+@@ -4020,6 +4020,28 @@
+             '*server': ['InetSocketAddressBase'] } }
+ 
+ ##
++# @BlockdevOptionsVitastor:
++#
++# Driver specific block device options for vitastor
++#
++# @image:       Image name
++# @inode:       Inode number
++# @pool:        Pool ID
++# @size:        Desired image size in bytes
++# @config-path: Path to Vitastor configuration
++# @etcd-host:   etcd connection address(es)
++# @etcd-prefix: etcd key/value prefix
++##
++{ 'struct': 'BlockdevOptionsVitastor',
++  'data': { '*inode': 'uint64',
++            '*pool': 'uint64',
++            '*size': 'uint64',
++            '*image': 'str',
++            '*config-path': 'str',
++            '*etcd-host': 'str',
++            '*etcd-prefix': 'str' } }
++
++##
+ # @ReplicationMode:
+ #
+ # An enumeration of replication modes.
+@@ -4392,6 +4414,7 @@
+       'throttle':   'BlockdevOptionsThrottle',
+       'vdi':        'BlockdevOptionsGenericFormat',
+       'vhdx':       'BlockdevOptionsGenericFormat',
++      'vitastor':   'BlockdevOptionsVitastor',
+       'vmdk':       'BlockdevOptionsGenericCOWFormat',
+       'vpc':        'BlockdevOptionsGenericFormat',
+       'vvfat':      'BlockdevOptionsVVFAT'
+@@ -4782,6 +4805,17 @@
+             '*encrypt' :        'RbdEncryptionCreateOptions' } }
+ 
+ ##
++# @BlockdevCreateOptionsVitastor:
++#
++# Driver specific image creation options for Vitastor.
++#
++# @size: Size of the virtual disk in bytes
++##
++{ 'struct': 'BlockdevCreateOptionsVitastor',
++  'data': { 'location':         'BlockdevOptionsVitastor',
++            'size':             'size' } }
++
++##
+ # @BlockdevVmdkSubformat:
+ #
+ # Subformat options for VMDK images
+@@ -4977,6 +5011,7 @@
+       'ssh':            'BlockdevCreateOptionsSsh',
+       'vdi':            'BlockdevCreateOptionsVdi',
+       'vhdx':           'BlockdevCreateOptionsVhdx',
++      'vitastor':       'BlockdevCreateOptionsVitastor',
+       'vmdk':           'BlockdevCreateOptionsVmdk',
+       'vpc':            'BlockdevCreateOptionsVpc'
+   } }
+Index: pve-qemu-kvm-6.1.0/block/meson.build
+===================================================================
+--- pve-qemu-kvm-6.1.0.orig/block/meson.build
++++ pve-qemu-kvm-6.1.0/block/meson.build
+@@ -91,6 +91,7 @@ foreach m : [
+   [libnfs, 'nfs', files('nfs.c')],
+   [libssh, 'ssh', files('ssh.c')],
+   [rbd, 'rbd', files('rbd.c')],
++  [vitastor, 'vitastor', files('vitastor.c')],
+ ]
+   if m[0].found()
+     module_ss = ss.source_set()
+Index: pve-qemu-kvm-6.1.0/configure
+===================================================================
+--- pve-qemu-kvm-6.1.0.orig/configure
++++ pve-qemu-kvm-6.1.0/configure
+@@ -375,6 +375,7 @@ trace_file="trace"
+ spice="$default_feature"
+ spice_protocol="auto"
+ rbd="auto"
++vitastor="auto"
+ smartcard="auto"
+ u2f="auto"
+ libusb="auto"
+@@ -1293,6 +1294,10 @@ for opt do
+   ;;
+   --enable-rbd) rbd="enabled"
+   ;;
++  --disable-vitastor) vitastor="disabled"
++  ;;
++  --enable-vitastor) vitastor="enabled"
++  ;;
+   --disable-xfsctl) xfs="no"
+   ;;
+   --enable-xfsctl) xfs="yes"
+@@ -1921,6 +1926,7 @@ disabled with --disable-FEATURE, default
+   spice           spice
+   spice-protocol  spice-protocol
+   rbd             rados block device (rbd)
++  vitastor        vitastor block device
+   libiscsi        iscsi support
+   libnfs          nfs support
+   smartcard       smartcard support (libcacard)
+@@ -5211,7 +5217,7 @@ if test "$skip_meson" = no; then
+         -Dcapstone=$capstone -Dslirp=$slirp -Dfdt=$fdt -Dbrlapi=$brlapi \
+         -Dcurl=$curl -Dglusterfs=$glusterfs -Dbzip2=$bzip2 -Dlibiscsi=$libiscsi \
+         -Dlibnfs=$libnfs -Diconv=$iconv -Dcurses=$curses -Dlibudev=$libudev\
+-        -Drbd=$rbd -Dlzo=$lzo -Dsnappy=$snappy -Dlzfse=$lzfse -Dlibxml2=$libxml2 \
++        -Drbd=$rbd -Dvitastor=$vitastor -Dlzo=$lzo -Dsnappy=$snappy -Dlzfse=$lzfse -Dlibxml2=$libxml2 \
+         -Dlibdaxctl=$libdaxctl -Dlibpmem=$libpmem -Dlinux_io_uring=$linux_io_uring \
+         -Dgnutls=$gnutls -Dnettle=$nettle -Dgcrypt=$gcrypt -Dauth_pam=$auth_pam \
+         -Dzstd=$zstd -Dseccomp=$seccomp -Dvirtfs=$virtfs -Dcap_ng=$cap_ng \
+Index: pve-qemu-kvm-6.1.0/meson.build
+===================================================================
+--- pve-qemu-kvm-6.1.0.orig/meson.build
++++ pve-qemu-kvm-6.1.0/meson.build
+@@ -729,6 +729,26 @@ if not get_option('rbd').auto() or have_
+   endif
+ endif
+ 
++vitastor = not_found
++if not get_option('vitastor').auto() or have_block
++  libvitastor_client = cc.find_library('vitastor_client', has_headers: ['vitastor_c.h'],
++    required: get_option('vitastor'), kwargs: static_kwargs)
++  if libvitastor_client.found()
++    if cc.links('''
++      #include <vitastor_c.h>
++      int main(void) {
++        vitastor_c_create_qemu(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
++        return 0;
++      }''', dependencies: libvitastor_client)
++      vitastor = declare_dependency(dependencies: libvitastor_client)
++    elif get_option('vitastor').enabled()
++      error('could not link libvitastor_client')
++    else
++      warning('could not link libvitastor_client, disabling')
++    endif
++  endif
++endif
++
+ glusterfs = not_found
+ glusterfs_ftruncate_has_stat = false
+ glusterfs_iocb_has_stat = false
+@@ -1268,6 +1288,7 @@ config_host_data.set('CONFIG_LIBNFS', li
+ config_host_data.set('CONFIG_LINUX_IO_URING', linux_io_uring.found())
+ config_host_data.set('CONFIG_LIBPMEM', libpmem.found())
+ config_host_data.set('CONFIG_RBD', rbd.found())
++config_host_data.set('CONFIG_VITASTOR', vitastor.found())
+ config_host_data.set('CONFIG_SDL', sdl.found())
+ config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found())
+ config_host_data.set('CONFIG_SECCOMP', seccomp.found())
+@@ -3087,6 +3108,7 @@ summary_info += {'bpf support': libbpf.f
+ # TODO: add back protocol and server version
+ summary_info += {'spice support':     config_host.has_key('CONFIG_SPICE')}
+ summary_info += {'rbd support':       rbd.found()}
++summary_info += {'vitastor support':  vitastor.found()}
+ summary_info += {'xfsctl support':    config_host.has_key('CONFIG_XFS')}
+ summary_info += {'smartcard support': cacard.found()}
+ summary_info += {'U2F support':       u2f.found()}
+Index: pve-qemu-kvm-6.1.0/meson_options.txt
+===================================================================
+--- pve-qemu-kvm-6.1.0.orig/meson_options.txt
++++ pve-qemu-kvm-6.1.0/meson_options.txt
+@@ -102,6 +102,8 @@ option('lzo', type : 'feature', value :
+        description: 'lzo compression support')
+ option('rbd', type : 'feature', value : 'auto',
+        description: 'Ceph block device driver')
++option('vitastor', type : 'feature', value : 'auto',
++       description: 'Vitastor block device driver')
+ option('gtk', type : 'feature', value : 'auto',
+        description: 'GTK+ user interface')
+ option('sdl', type : 'feature', value : 'auto',

patches/qemu-make-patches.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+# QEMU patches don't include the `block/vitastor.c` file to not duplicate it in sources
+# Run this script to append its creation to all QEMU patches
+
+DIR=$(dirname $0)
+for i in "$DIR"/qemu-*-vitastor.patch "$DIR"/pve-qemu-*-vitastor.patch; do
+    if ! grep -qP '^\+\+\+ .*block/vitastor\.c' $i; then
+        echo 'Index: a/block/vitastor.c' >> $i
+        echo '===================================================================' >> $i
+        echo '--- /dev/null' >> $i
+        echo '+++ a/block/vitastor.c' >> $i
+        echo '@@ -0,0 +1,'$(wc -l "$DIR"/../src/qemu_driver.c)' @@' >> $i
+        cat "$DIR"/../src/qemu_driver.c | sed 's/^/+/' >> $i
+    fi
+done

rpm/build-tarball.sh

@@ -25,4 +25,4 @@ rm fio
 mv fio-copy fio
 FIO=`rpm -qi fio | perl -e 'while(<>) { /^Epoch[\s:]+(\S+)/ && print "$1:"; /^Version[\s:]+(\S+)/ && print $1; /^Release[\s:]+(\S+)/ && print "-$1"; }'`
 perl -i -pe 's/(Requires:\s*fio)([^\n]+)?/$1 = '$FIO'/' $VITASTOR/rpm/vitastor-el$EL.spec
-tar --transform 's#^#vitastor-0.6.9/#' --exclude 'rpm/*.rpm' -czf $VITASTOR/../vitastor-0.6.9$(rpm --eval '%dist').tar.gz *
+tar --transform 's#^#vitastor-0.6.10/#' --exclude 'rpm/*.rpm' -czf $VITASTOR/../vitastor-0.6.10$(rpm --eval '%dist').tar.gz *

rpm/vitastor-el7.Dockerfile

@@ -34,7 +34,7 @@ ADD . /root/vitastor
 RUN set -e; \
     cd /root/vitastor/rpm; \
     sh build-tarball.sh; \
-    cp /root/vitastor-0.6.9.el7.tar.gz ~/rpmbuild/SOURCES; \
+    cp /root/vitastor-0.6.10.el7.tar.gz ~/rpmbuild/SOURCES; \
     cp vitastor-el7.spec ~/rpmbuild/SPECS/vitastor.spec; \
     cd ~/rpmbuild/SPECS/; \
     rpmbuild -ba vitastor.spec; \

rpm/vitastor-el7.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        0.6.9
+Version:        0.6.10
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-0.6.9.el7.tar.gz
+Source0:        vitastor-0.6.10.el7.tar.gz
 
 BuildRequires:  liburing-devel >= 0.6
 BuildRequires:  gperftools-devel

rpm/vitastor-el8.Dockerfile

@@ -33,7 +33,7 @@ ADD . /root/vitastor
 RUN set -e; \
     cd /root/vitastor/rpm; \
     sh build-tarball.sh; \
-    cp /root/vitastor-0.6.9.el8.tar.gz ~/rpmbuild/SOURCES; \
+    cp /root/vitastor-0.6.10.el8.tar.gz ~/rpmbuild/SOURCES; \
     cp vitastor-el8.spec ~/rpmbuild/SPECS/vitastor.spec; \
     cd ~/rpmbuild/SPECS/; \
     rpmbuild -ba vitastor.spec; \

rpm/vitastor-el8.spec

@@ -1,11 +1,11 @@
 Name:           vitastor
-Version:        0.6.9
+Version:        0.6.10
 Release:        1%{?dist}
 Summary:        Vitastor, a fast software-defined clustered block storage
 
 License:        Vitastor Network Public License 1.1
 URL:            https://vitastor.io/
-Source0:        vitastor-0.6.9.el8.tar.gz
+Source0:        vitastor-0.6.10.el8.tar.gz
 
 BuildRequires:  liburing-devel >= 0.6
 BuildRequires:  gperftools-devel

src/CMakeLists.txt

@@ -15,7 +15,7 @@ if("${CMAKE_INSTALL_PREFIX}" MATCHES "^/usr/local/?$")
 	set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
 endif()
 
-add_definitions(-DVERSION="0.6.9")
+add_definitions(-DVERSION="0.6.10")
 add_definitions(-Wall -Wno-sign-compare -Wno-comment -Wno-parentheses -Wno-pointer-arith -fdiagnostics-color=always -I ${CMAKE_SOURCE_DIR}/src)
 if (${WITH_ASAN})
 	add_definitions(-fsanitize=address -fno-omit-frame-pointer)
@@ -153,7 +153,7 @@ target_link_libraries(vitastor-nbd
 
 # vitastor-cli
 add_executable(vitastor-cli
-	cli.cpp cli_alloc_osd.cpp cli_simple_offsets.cpp
+	cli.cpp cli_alloc_osd.cpp cli_simple_offsets.cpp cli_df.cpp
 	cli_ls.cpp cli_create.cpp cli_modify.cpp cli_flatten.cpp cli_merge.cpp cli_rm.cpp cli_snap_rm.cpp
 )
 target_link_libraries(vitastor-cli

src/cli.cpp

@@ -57,6 +57,7 @@ json11::Json::object cli_tool_t::parse_args(int narg, const char *args[])
             const char *opt = args[i]+2;
             cfg[opt] = i == narg-1 || !strcmp(opt, "json") || !strcmp(opt, "wait-list") ||
                 !strcmp(opt, "long") || !strcmp(opt, "del") || !strcmp(opt, "no-color") ||
+                !strcmp(opt, "readonly") || !strcmp(opt, "readwrite") ||
                 !strcmp(opt, "force") || !strcmp(opt, "reverse") ||
                 !strcmp(opt, "writers-stopped") && strcmp("1", args[i+1]) != 0
                 ? "1" : args[++i];
@@ -69,7 +70,7 @@ json11::Json::object cli_tool_t::parse_args(int narg, const char *args[])
     if (!cmd.size())
     {
         std::string exe(exe_name);
-        if (exe.substr(exe.size()-11) == "vitastor-rm")
+        if (exe.size() >= 11 && exe.substr(exe.size()-11) == "vitastor-rm")
         {
             cmd.push_back("rm-data");
         }
@@ -85,8 +86,11 @@ void cli_tool_t::help()
         "(c) Vitaliy Filippov, 2019+ (VNPL-1.1)\n"
         "\n"
         "USAGE:\n"
-        "%s ls [-l] [-p POOL] [--sort FIELD] [-r] [-n N] [<name> ...]\n"
-        "  List images (only specified if <name> passed).\n"
+        "%s df\n"
+        "  Show pool space statistics\n"
+        "\n"
+        "%s ls [-l] [-p POOL] [--sort FIELD] [-r] [-n N] [<glob> ...]\n"
+        "  List images (only matching <glob> patterns if passed).\n"
         "  -p|--pool POOL  Filter images by pool ID or name\n"
         "  -l|--long       Also report allocated size and I/O statistics\n"
         "  --del           Also include delete operation statistics\n"
@@ -103,7 +107,7 @@ void cli_tool_t::help()
         "%s snap-create [-p|--pool <id|name>] <image>@<snapshot>\n"
         "  Create a snapshot of image <name>. May be used live if only a single writer is active.\n"
         "\n"
-        "%s modify <name> [--rename <new-name>] [-s|--size <size>] [--readonly | --readwrite] [-f|--force]\n"
+        "%s modify <name> [--rename <new-name>] [--resize <size>] [--readonly | --readwrite] [-f|--force]\n"
         "  Rename, resize image or change its readonly status. Images with children can't be made read-write.\n"
         "  If the new size is smaller than the old size, extra data will be purged.\n"
         "  You should resize file system in the image, if present, before shrinking it.\n"
@@ -151,7 +155,8 @@ void cli_tool_t::help()
         "  --no-color          Disable colored output\n"
         "  --json              JSON output\n"
         ,
-        exe_name, exe_name, exe_name, exe_name, exe_name, exe_name, exe_name, exe_name, exe_name, exe_name, exe_name
+        exe_name, exe_name, exe_name, exe_name, exe_name, exe_name,
+        exe_name, exe_name, exe_name, exe_name, exe_name, exe_name
     );
     exit(0);
 }
@@ -245,6 +250,11 @@ void cli_tool_t::run(json11::Json cfg)
         fprintf(stderr, "command is missing\n");
         exit(1);
     }
+    else if (cmd[0] == "df")
+    {
+        // Show pool space stats
+        action_cb = start_df(cfg);
+    }
     else if (cmd[0] == "ls")
     {
         // List images

src/cli.h

@@ -50,6 +50,7 @@ public:
     friend struct snap_flattener_t;
     friend struct snap_remover_t;
 
+    std::function<bool(void)> start_df(json11::Json);
     std::function<bool(void)> start_ls(json11::Json);
     std::function<bool(void)> start_create(json11::Json);
     std::function<bool(void)> start_modify(json11::Json);
@@ -61,5 +62,14 @@ public:
     std::function<bool(void)> simple_offsets(json11::Json cfg);
 };
 
-std::string format_size(uint64_t size);
 uint64_t parse_size(std::string size_str);
+
+std::string print_table(json11::Json items, json11::Json header, bool use_esc);
+
+std::string format_size(uint64_t size);
+
+std::string format_lat(uint64_t lat);
+
+std::string format_q(double depth);
+
+bool stupid_glob(const std::string str, const std::string glob);

src/cli_create.cpp

@@ -88,6 +88,31 @@ struct image_creator_t
             goto resume_2;
         else if (state == 3)
             goto resume_3;
+        for (auto & ic: parent->cli->st_cli.inode_config)
+        {
+            if (ic.second.name == image_name)
+            {
+                fprintf(stderr, "Image %s already exists\n", image_name.c_str());
+                exit(1);
+            }
+            if (ic.second.name == new_parent)
+            {
+                new_parent_id = ic.second.num;
+                if (!new_pool_id)
+                {
+                    new_pool_id = INODE_POOL(ic.second.num);
+                }
+                if (!size)
+                {
+                    size = ic.second.size;
+                }
+            }
+        }
+        if (new_parent != "" && !new_parent_id)
+        {
+            fprintf(stderr, "Parent image not found\n");
+            exit(1);
+        }
         if (!new_pool_id)
         {
             fprintf(stderr, "Pool name or ID is missing\n");
@@ -98,14 +123,6 @@ struct image_creator_t
             fprintf(stderr, "Image size is missing\n");
             exit(1);
         }
-        for (auto & ic: parent->cli->st_cli.inode_config)
-        {
-            if (ic.second.name == image_name)
-            {
-                fprintf(stderr, "Image %s already exists\n", image_name.c_str());
-                exit(1);
-            }
-        }
         do
         {
             etcd_txn(json11::Json::object {
@@ -151,6 +168,11 @@ resume_3:
                 exit(1);
             }
         }
+        if (new_parent != "")
+        {
+            fprintf(stderr, "--parent can't be used with snapshots\n");
+            exit(1);
+        }
         do
         {
             // In addition to next_id, get: size, old_id, old_pool_id, new_parent, cfg_mod_rev, idx_mod_rev
@@ -443,6 +465,10 @@ resume_3:
 
 uint64_t parse_size(std::string size_str)
 {
+    if (!size_str.length())
+    {
+        return 0;
+    }
     uint64_t mul = 1;
     char type_char = tolower(size_str[size_str.length()-1]);
     if (type_char == 'k' || type_char == 'm' || type_char == 'g' || type_char == 't')

src/cli_df.cpp

@@ -0,0 +1,229 @@
+// Copyright (c) Vitaliy Filippov, 2019+
+// License: VNPL-1.1 (see README.md for details)
+
+#include "cli.h"
+#include "cluster_client.h"
+#include "base64.h"
+
+// List pools with space statistics
+struct pool_lister_t
+{
+    cli_tool_t *parent;
+
+    int state = 0;
+    json11::Json space_info;
+    std::map<pool_id_t, json11::Json::object> pool_stats;
+
+    bool is_done()
+    {
+        return state == 100;
+    }
+
+    void get_stats()
+    {
+        if (state == 1)
+            goto resume_1;
+        // Space statistics - pool/stats/<pool>
+        parent->waiting++;
+        parent->cli->st_cli.etcd_txn(json11::Json::object {
+            { "success", json11::Json::array {
+                json11::Json::object {
+                    { "request_range", json11::Json::object {
+                        { "key", base64_encode(
+                            parent->cli->st_cli.etcd_prefix+"/pool/stats/"
+                        ) },
+                        { "range_end", base64_encode(
+                            parent->cli->st_cli.etcd_prefix+"/pool/stats0"
+                        ) },
+                    } },
+                },
+                json11::Json::object {
+                    { "request_range", json11::Json::object {
+                        { "key", base64_encode(
+                            parent->cli->st_cli.etcd_prefix+"/osd/stats/"
+                        ) },
+                        { "range_end", base64_encode(
+                            parent->cli->st_cli.etcd_prefix+"/osd/stats0"
+                        ) },
+                    } },
+                },
+            } },
+        }, ETCD_SLOW_TIMEOUT, [this](std::string err, json11::Json res)
+        {
+            parent->waiting--;
+            if (err != "")
+            {
+                fprintf(stderr, "Error reading from etcd: %s\n", err.c_str());
+                exit(1);
+            }
+            space_info = res;
+            parent->ringloop->wakeup();
+        });
+        state = 1;
+resume_1:
+        if (parent->waiting > 0)
+            return;
+        std::map<pool_id_t, uint64_t> osd_free;
+        for (auto & kv_item: space_info["responses"][0]["response_range"]["kvs"].array_items())
+        {
+            auto kv = parent->cli->st_cli.parse_etcd_kv(kv_item);
+            // pool ID
+            pool_id_t pool_id;
+            char null_byte = 0;
+            sscanf(kv.key.substr(parent->cli->st_cli.etcd_prefix.length()).c_str(), "/pool/stats/%u%c", &pool_id, &null_byte);
+            if (!pool_id || pool_id >= POOL_ID_MAX || null_byte != 0)
+            {
+                fprintf(stderr, "Invalid key in etcd: %s\n", kv.key.c_str());
+                continue;
+            }
+            // pool/stats/<N>
+            pool_stats[pool_id] = kv.value.object_items();
+        }
+        for (auto & kv_item: space_info["responses"][1]["response_range"]["kvs"].array_items())
+        {
+            auto kv = parent->cli->st_cli.parse_etcd_kv(kv_item);
+            // osd ID
+            osd_num_t osd_num;
+            char null_byte = 0;
+            sscanf(kv.key.substr(parent->cli->st_cli.etcd_prefix.length()).c_str(), "/osd/stats/%lu%c", &osd_num, &null_byte);
+            if (!osd_num || osd_num >= POOL_ID_MAX || null_byte != 0)
+            {
+                fprintf(stderr, "Invalid key in etcd: %s\n", kv.key.c_str());
+                continue;
+            }
+            // osd/stats/<N>::free
+            osd_free[osd_num] = kv.value["free"].uint64_value();
+        }
+        // Calculate max_avail for each pool
+        for (auto & pp: parent->cli->st_cli.pool_config)
+        {
+            auto & pool_cfg = pp.second;
+            uint64_t pool_avail = UINT64_MAX;
+            std::map<osd_num_t, uint64_t> pg_per_osd;
+            for (auto & pgp: pool_cfg.pg_config)
+            {
+                for (auto pg_osd: pgp.second.target_set)
+                {
+                    if (pg_osd != 0)
+                    {
+                        pg_per_osd[pg_osd]++;
+                    }
+                }
+            }
+            for (auto pg_per_pair: pg_per_osd)
+            {
+                uint64_t pg_free = osd_free[pg_per_pair.first] * pool_cfg.pg_count / pg_per_pair.second;
+                if (pool_avail > pg_free)
+                {
+                    pool_avail = pg_free;
+                }
+            }
+            if (pool_cfg.scheme != POOL_SCHEME_REPLICATED)
+            {
+                pool_avail = pool_avail * (pool_cfg.pg_size - pool_cfg.parity_chunks) / pool_stats[pool_cfg.id]["pg_real_size"].uint64_value();
+            }
+            pool_stats[pool_cfg.id] = json11::Json::object {
+                { "name", pool_cfg.name },
+                { "pg_count", pool_cfg.pg_count },
+                { "scheme", pool_cfg.scheme == POOL_SCHEME_REPLICATED ? "replicated" : "jerasure" },
+                { "scheme_name", pool_cfg.scheme == POOL_SCHEME_REPLICATED
+                    ? std::to_string(pool_cfg.pg_size)+"/"+std::to_string(pool_cfg.pg_minsize)
+                    : "EC "+std::to_string(pool_cfg.pg_size-pool_cfg.parity_chunks)+"+"+std::to_string(pool_cfg.parity_chunks) },
+                { "used_raw", (uint64_t)(pool_stats[pool_cfg.id]["used_raw_tb"].number_value() * (1l<<40)) },
+                { "total_raw", (uint64_t)(pool_stats[pool_cfg.id]["total_raw_tb"].number_value() * (1l<<40)) },
+                { "max_available", pool_avail },
+                { "raw_to_usable", pool_stats[pool_cfg.id]["raw_to_usable"].number_value() },
+                { "space_efficiency", pool_stats[pool_cfg.id]["space_efficiency"].number_value() },
+                { "pg_real_size", pool_stats[pool_cfg.id]["pg_real_size"].uint64_value() },
+                { "failure_domain", pool_cfg.failure_domain },
+            };
+        }
+    }
+
+    json11::Json::array to_list()
+    {
+        json11::Json::array list;
+        for (auto & kv: pool_stats)
+        {
+            list.push_back(kv.second);
+        }
+        return list;
+    }
+
+    void loop()
+    {
+        get_stats();
+        if (parent->waiting > 0)
+            return;
+        if (parent->json_output)
+        {
+            // JSON output
+            printf("%s\n", json11::Json(to_list()).dump().c_str());
+            state = 100;
+            return;
+        }
+        // Table output: name, scheme_name, pg_count, total, used, max_avail, used%, efficiency
+        json11::Json::array cols;
+        cols.push_back(json11::Json::object{
+            { "key", "name" },
+            { "title", "NAME" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "scheme_name" },
+            { "title", "SCHEME" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "pg_count" },
+            { "title", "PGS" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "total_fmt" },
+            { "title", "TOTAL" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "used_fmt" },
+            { "title", "USED" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "max_avail_fmt" },
+            { "title", "AVAILABLE" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "used_pct" },
+            { "title", "USED%" },
+        });
+        cols.push_back(json11::Json::object{
+            { "key", "eff_fmt" },
+            { "title", "EFFICIENCY" },
+        });
+        json11::Json::array list;
+        for (auto & kv: pool_stats)
+        {
+            kv.second["total_fmt"] = format_size(kv.second["total_raw"].uint64_value() / kv.second["raw_to_usable"].number_value());
+            kv.second["used_fmt"] = format_size(kv.second["used_raw"].uint64_value() / kv.second["raw_to_usable"].number_value());
+            kv.second["max_avail_fmt"] = format_size(kv.second["max_available"].uint64_value());
+            kv.second["used_pct"] = format_q(100 - 100*kv.second["max_available"].uint64_value() *
+                kv.second["raw_to_usable"].number_value() / kv.second["total_raw"].uint64_value())+"%";
+            kv.second["eff_fmt"] = format_q(kv.second["space_efficiency"].number_value()*100)+"%";
+        }
+        printf("%s", print_table(to_list(), cols, parent->color).c_str());
+        state = 100;
+    }
+};
+
+std::function<bool(void)> cli_tool_t::start_df(json11::Json cfg)
+{
+    json11::Json::array cmd = cfg["command"].array_items();
+    auto lister = new pool_lister_t();
+    lister->parent = this;
+    return [lister]()
+    {
+        lister->loop();
+        if (lister->is_done())
+        {
+            delete lister;
+            return true;
+        }
+        return false;
+    };
+}

src/cli_ls.cpp

@@ -6,16 +6,6 @@
 #include "cluster_client.h"
 #include "base64.h"
 
-#define MIN(a, b) ((a) < (b) ? (b) : (a))
-
-std::string print_table(json11::Json items, json11::Json header, bool use_esc);
-
-std::string format_size(uint64_t size);
-
-std::string format_lat(uint64_t lat);
-
-std::string format_q(double depth);
-
 // List existing images
 //
 // Again, you can just look into etcd, but this console tool incapsulates it
@@ -213,10 +203,21 @@ resume_1:
         json11::Json::array list;
         for (auto & kv: stats)
         {
-            if (!only_names.size() || only_names.find(kv.second["name"].string_value()) != only_names.end())
+            if (!only_names.size())
             {
                 list.push_back(kv.second);
             }
+            else
+            {
+                for (auto glob: only_names)
+                {
+                    if (stupid_glob(kv.second["name"].string_value(), glob))
+                    {
+                        list.push_back(kv.second);
+                        break;
+                    }
+                }
+            }
         }
         if (sort_field == "name" || sort_field == "pool_name")
         {
@@ -355,6 +356,9 @@ resume_1:
                 kv.second["read_bw"] = format_size(kv.second["read_bps"].uint64_value())+"/s";
                 kv.second["write_bw"] = format_size(kv.second["write_bps"].uint64_value())+"/s";
                 kv.second["delete_bw"] = format_size(kv.second["delete_bps"].uint64_value())+"/s";
+                kv.second["read_iops"] = format_q(kv.second["read_iops"].number_value());
+                kv.second["write_iops"] = format_q(kv.second["write_iops"].number_value());
+                kv.second["delete_iops"] = format_q(kv.second["delete_iops"].number_value());
                 kv.second["read_lat_f"] = format_lat(kv.second["read_lat"].uint64_value());
                 kv.second["write_lat_f"] = format_lat(kv.second["write_lat"].uint64_value());
                 kv.second["delete_lat_f"] = format_lat(kv.second["delete_lat"].uint64_value());
@@ -493,6 +497,62 @@ std::string format_q(double depth)
     return std::string(buf);
 }
 
+struct glob_stack_t
+{
+    int glob_pos;
+    int str_pos;
+};
+
+// Yes I know I could do it by translating the pattern to std::regex O:-)
+bool stupid_glob(const std::string str, const std::string glob)
+{
+    std::vector<glob_stack_t> wildcards;
+    int pos = 0, gp = 0;
+    bool m;
+back:
+    while (true)
+    {
+        if (gp >= glob.length())
+        {
+            if (pos >= str.length())
+                return true;
+            m = false;
+        }
+        else if (glob[gp] == '*')
+        {
+            wildcards.push_back((glob_stack_t){ .glob_pos = ++gp, .str_pos = pos });
+            continue;
+        }
+        else if (glob[gp] == '?')
+            m = pos < str.size();
+        else
+        {
+            if (glob[gp] == '\\' && gp < glob.length()-1)
+                gp++;
+            m = pos < str.size() && str[pos] == glob[gp];
+        }
+        if (!m)
+        {
+            while (wildcards.size() > 0)
+            {
+                // Backtrack
+                pos = (++wildcards[wildcards.size()-1].str_pos);
+                if (pos > str.size())
+                    wildcards.pop_back();
+                else
+                {
+                    gp = wildcards[wildcards.size()-1].glob_pos;
+                    goto back;
+                }
+            }
+            return false;
+        }
+        pos++;
+        gp++;
+    }
+    return true;
+}
+
 std::function<bool(void)> cli_tool_t::start_ls(json11::Json cfg)
 {
     json11::Json::array cmd = cfg["command"].array_items();

src/cli_modify.cpp

@@ -63,6 +63,15 @@ struct image_changer_t
                 break;
             }
         }
+        if ((!set_readwrite || !cfg.readonly) &&
+            (!set_readonly || cfg.readonly) &&
+            (!new_size || cfg.size == new_size) &&
+            (new_name == "" || new_name == image_name))
+        {
+            printf("No change\n");
+            state = 100;
+            return;
+        }
         if (new_size != 0)
         {
             if (cfg.size >= new_size)
@@ -201,11 +210,7 @@ std::function<bool(void)> cli_tool_t::start_modify(json11::Json cfg)
         exit(1);
     }
     changer->new_name = cfg["rename"].string_value();
-    if (changer->new_name == changer->image_name)
-    {
-        changer->new_name = "";
-    }
-    changer->new_size = cfg["size"].uint64_value();
+    changer->new_size = parse_size(cfg["resize"].string_value());
     if (changer->new_size != 0 && (changer->new_size % 4096))
     {
         fprintf(stderr, "Image size should be a multiple of 4096\n");

src/msgr_rdma.cpp

@@ -62,7 +62,13 @@ msgr_rdma_context_t *msgr_rdma_context_t::create(const char *ib_devname, uint8_t
     dev_list = ibv_get_device_list(NULL);
     if (!dev_list)
     {
-        fprintf(stderr, "Failed to get RDMA device list: %s\n", strerror(errno));
+        if (errno == -ENOSYS || errno == ENOSYS)
+        {
+            if (log_level > 0)
+                fprintf(stderr, "No RDMA devices found (RDMA device list returned ENOSYS)\n");
+        }
+        else
+            fprintf(stderr, "Failed to get RDMA device list: %s\n", strerror(errno));
         goto cleanup;
     }
     if (!ib_devname)

src/osd.cpp

@@ -179,6 +179,14 @@ void osd_t::bind_socket()
             fprintf(stderr, "More than 1 address matches requested network(s): %s\n", json11::Json(matched_addrs).dump().c_str());
             force_stop(1);
         }
+        if (!matched_addrs.size())
+        {
+            std::string nets;
+            for (auto v: mask)
+                nets += (nets == "" ? v : ","+v);
+            fprintf(stderr, "Addresses matching osd_network(s) %s not found\n", nets.c_str());
+            force_stop(1);
+        }
         bind_address = matched_addrs[0];
     }
 

src/qemu_driver.c

@@ -308,7 +308,7 @@ static void vitastor_close(BlockDriverState *bs)
 static int vitastor_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz)
 {
     bsz->phys = 4096;
-    bsz->log = 4096;
+    bsz->log = 512;
     return 0;
 }
 #endif

src/vitastor.pc.in

@@ -6,7 +6,7 @@ includedir=${prefix}/@CMAKE_INSTALL_INCLUDEDIR@
 
 Name: Vitastor
 Description: Vitastor client library
-Version: 0.6.9
+Version: 0.6.10
 Libs: -L${libdir} -lvitastor_client
 Cflags: -I${includedir}
 

tests/common.sh

@@ -21,6 +21,7 @@ cd `dirname $0`/..
 trap 'kill -9 $(jobs -p)' EXIT
 
 ETCD=${ETCD:-etcd}
+ETCD_IP=${ETCD_IP:-127.0.0.1}
 ETCD_PORT=${ETCD_PORT:-12379}
 
 if [ "$KEEP_DATA" = "" ]; then
@@ -29,11 +30,11 @@ if [ "$KEEP_DATA" = "" ]; then
 fi
 
 $ETCD -name etcd_test --data-dir ./testdata/etcd \
-    --advertise-client-urls http://127.0.0.1:$ETCD_PORT --listen-client-urls http://127.0.0.1:$ETCD_PORT \
-    --initial-advertise-peer-urls http://127.0.0.1:$((ETCD_PORT+1)) --listen-peer-urls http://127.0.0.1:$((ETCD_PORT+1)) \
+    --advertise-client-urls http://$ETCD_IP:$ETCD_PORT --listen-client-urls http://$ETCD_IP:$ETCD_PORT \
+    --initial-advertise-peer-urls http://$ETCD_IP:$((ETCD_PORT+1)) --listen-peer-urls http://$ETCD_IP:$((ETCD_PORT+1)) \
     --max-txn-ops=100000 --auto-compaction-retention=10 --auto-compaction-mode=revision &>./testdata/etcd.log &
 ETCD_PID=$!
-ETCD_URL=127.0.0.1:$ETCD_PORT/v3
+ETCD_URL=$ETCD_IP:$ETCD_PORT/v3
 ETCDCTL="${ETCD}ctl --endpoints=http://$ETCD_URL"
 
 echo leak:fio >> testdata/lsan-suppress.txt