bump version to 8.1.5-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
backup: factor out & clean up gathering device info into helper
2024-03-12 14:08:48 +01:00 · 2024-03-12 13:55:00 +01:00 · 2024-03-12 13:20:28 +01:00 · 2024-03-12 13:19:30 +01:00 · 2024-03-12 13:11:21 +01:00 · 2024-02-21 20:11:27 +01:00
19 changed files with 366 additions and 150 deletions
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,24 @@
+pve-qemu-kvm (8.1.5-4) bookworm; urgency=medium
+
+  * fix live-import for certain kinds of VMDK images that rely on padding
+
+  * backup: avoid bubbling up first error if it's an ECANCELED one, as those
+    are often a result of cancling the job due to running into an actual
+    issue.
+
+  * backup: factor out & clean up gathering device info into helper
+
+ -- Proxmox Support Team <support@proxmox.com>  Tue, 12 Mar 2024 14:08:40 +0100
+
+pve-qemu-kvm (8.1.5-3) bookworm; urgency=medium
+
+  * backport fix for potential deadlock during QMP stop command if the VM has
+    disks attached through VirtIO-Block and IO-Thread enabled
+
+  * fix #4507: add patch to automatically increase NOFILE soft limit
+
+ -- Proxmox Support Team <support@proxmox.com>  Wed, 21 Feb 2024 20:11:23 +0100
+
 pve-qemu-kvm (8.1.5-2) bookworm; urgency=medium

  * work around for a situation where guest IO might get stuck, if the VM is
--- a/debian/patches/extra/0012-qemu_init-increase-NOFILE-soft-limit-on-POSIX.patch
+++ b/debian/patches/extra/0012-qemu_init-increase-NOFILE-soft-limit-on-POSIX.patch
@@ -0,0 +1,119 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Fiona Ebner <f.ebner@proxmox.com>
+Date: Mon, 18 Dec 2023 11:13:40 +0100
+Subject: [PATCH] qemu_init: increase NOFILE soft limit on POSIX
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+In many configurations, e.g. multiple vNICs with multiple queues or
+with many Ceph OSDs, the default soft limit of 1024 is not enough.
+QEMU is supposed to work fine with file descriptors >= 1024 and does
+not use select() on POSIX. Bump the soft limit to the allowed hard
+limit to avoid issues with the aforementioned configurations.
+
+Of course the limit could be raised from the outside, but the man page
+of systemd.exec states about 'LimitNOFILE=':
+
+> Don't use.
+> [...]
+> Typically applications should increase their soft limit to the hard
+> limit on their own, if they are OK with working with file
+> descriptors above 1023,
+
+If the soft limit is already the same as the hard limit, avoid the
+superfluous setrlimit call. This can avoid a warning with a strict
+seccomp filter blocking setrlimit if NOFILE was already raised before
+executing QEMU.
+
+Buglink: https://bugzilla.proxmox.com/show_bug.cgi?id=4507
+Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+---
+ include/sysemu/os-posix.h |  1 +
+ include/sysemu/os-win32.h |  5 +++++
+ os-posix.c                | 22 ++++++++++++++++++++++
+ softmmu/vl.c              |  2 ++
+ 4 files changed, 30 insertions(+)
+
+diff --git a/include/sysemu/os-posix.h b/include/sysemu/os-posix.h
+index 1030d39904..edc415aff5 100644
+--- a/include/sysemu/os-posix.h
+++ b/include/sysemu/os-posix.h
+@@ -48,6 +48,7 @@ void os_setup_early_signal_handling(void);
+ void os_set_proc_name(const char *s);
+ void os_setup_signal_handling(void);
+ void os_daemonize(void);
+void os_setup_limits(void);
+ void os_setup_post(void);
+ int os_mlock(void);
+ 
+diff --git a/include/sysemu/os-win32.h b/include/sysemu/os-win32.h
+index 91aa0d7ec0..f6e23fe01e 100644
+--- a/include/sysemu/os-win32.h
+++ b/include/sysemu/os-win32.h
+@@ -129,6 +129,11 @@ static inline int os_mlock(void)
+     return -ENOSYS;
+ }
+ 
+void os_setup_limits(void)
+{
+    return;
+}
+
+ #define fsync _commit
+ 
+ #if !defined(lseek)
+diff --git a/os-posix.c b/os-posix.c
+index cfcb96533c..0cc1d991b1 100644
+--- a/os-posix.c
+++ b/os-posix.c
+@@ -24,6 +24,7 @@
+  */
+ 
+ #include "qemu/osdep.h"
+#include <sys/resource.h>
+ #include <sys/wait.h>
+ #include <pwd.h>
+ #include <grp.h>
+@@ -286,6 +287,27 @@ void os_daemonize(void)
+     }
+ }
+ 
+void os_setup_limits(void)
+{
+    struct rlimit nofile;
+
+    if (getrlimit(RLIMIT_NOFILE, &nofile) < 0) {
+        warn_report("unable to query NOFILE limit: %s", strerror(errno));
+        return;
+    }
+
+    if (nofile.rlim_cur == nofile.rlim_max) {
+        return;
+    }
+
+    nofile.rlim_cur = nofile.rlim_max;
+
+    if (setrlimit(RLIMIT_NOFILE, &nofile) < 0) {
+        warn_report("unable to set NOFILE limit: %s", strerror(errno));
+        return;
+    }
+}
+
+ void os_setup_post(void)
+ {
+     int fd = 0;
+diff --git a/softmmu/vl.c b/softmmu/vl.c
+index c9e9ede237..ba6ad8a8df 100644
+--- a/softmmu/vl.c
+++ b/softmmu/vl.c
+@@ -2713,6 +2713,8 @@ void qemu_init(int argc, char **argv)
+     error_init(argv[0]);
+     qemu_init_exec_dir(argv[0]);
+ 
+    os_setup_limits();
+
+     qemu_init_arch_modules();
+ 
+     qemu_init_subsystems();
--- a/debian/patches/extra/0013-virtio-blk-avoid-using-ioeventfd-state-in-irqfd-cond.patch
+++ b/debian/patches/extra/0013-virtio-blk-avoid-using-ioeventfd-state-in-irqfd-cond.patch
@@ -0,0 +1,61 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stefan Hajnoczi <stefanha@redhat.com>
+Date: Mon, 22 Jan 2024 12:26:25 -0500
+Subject: [PATCH] virtio-blk: avoid using ioeventfd state in irqfd conditional
+
+Requests that complete in an IOThread use irqfd to notify the guest
+while requests that complete in the main loop thread use the traditional
+qdev irq code path. The reason for this conditional is that the irq code
+path requires the BQL:
+
+  if (s->ioeventfd_started && !s->ioeventfd_disabled) {
+      virtio_notify_irqfd(vdev, req->vq);
+  } else {
+      virtio_notify(vdev, req->vq);
+  }
+
+There is a corner case where the conditional invokes the irq code path
+instead of the irqfd code path:
+
+  static void virtio_blk_stop_ioeventfd(VirtIODevice *vdev)
+  {
+      ...
+      /*
+       * Set ->ioeventfd_started to false before draining so that host notifiers
+       * are not detached/attached anymore.
+       */
+      s->ioeventfd_started = false;
+
+      /* Wait for virtio_blk_dma_restart_bh() and in flight I/O to complete */
+      blk_drain(s->conf.conf.blk);
+
+During blk_drain() the conditional produces the wrong result because
+ioeventfd_started is false.
+
+Use qemu_in_iothread() instead of checking the ioeventfd state.
+
+Cc: qemu-stable@nongnu.org
+Buglink: https://issues.redhat.com/browse/RHEL-15394
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-ID: <20240122172625.415386-1-stefanha@redhat.com>
+Reviewed-by: Kevin Wolf <kwolf@redhat.com>
+Signed-off-by: Kevin Wolf <kwolf@redhat.com>
+[FE: backport: dataplane -> ioeventfd rework didn't happen yet]
+Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+---
+ hw/block/virtio-blk.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
+index 39e7f23fab..61bd1f6859 100644
+--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
+@@ -64,7 +64,7 @@ static void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status)
+     iov_discard_undo(&req->inhdr_undo);
+     iov_discard_undo(&req->outhdr_undo);
+     virtqueue_push(req->vq, &req->elem, req->in_len);
+-    if (s->dataplane_started && !s->dataplane_disabled) {
+    if (qemu_in_iothread()) {
+         virtio_blk_data_plane_notify(s->dataplane, req->vq);
+     } else {
+         virtio_notify(vdev, req->vq);
--- a/debian/patches/pve/0016-PVE-add-IOChannel-implementation-for-savevm-async.patch
+++ b/debian/patches/pve/0016-PVE-add-IOChannel-implementation-for-savevm-async.patch
@@ -14,6 +14,7 @@ Additionally, allows tracking the current position from the outside
 (intended to be used for progress tracking).

 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 migration/channel-savevm-async.c | 183 +++++++++++++++++++++++++++++++
 migration/channel-savevm-async.h |  51 +++++++++
--- a/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
+++ b/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
@@ -823,7 +823,7 @@ index 8073f5edf5..dc1ececc9c 100644
 DEF("daemonize", 0, QEMU_OPTION_daemonize, \
     "-daemonize      daemonize QEMU after initializing\n", QEMU_ARCH_ALL)
 diff --git a/softmmu/vl.c b/softmmu/vl.c
-index c9e9ede237..3f2681aded 100644
+index ba6ad8a8df..ddeace306e 100644
 --- a/softmmu/vl.c
 +++ b/softmmu/vl.c
@@ -164,6 +164,7 @@ static const char *accelerators;
@@ -847,7 +847,7 @@ index c9e9ede237..3f2681aded 100644
     }
     if (replay_mode != REPLAY_MODE_NONE) {
         replay_vmstate_init();
-@@ -3194,6 +3201,9 @@ void qemu_init(int argc, char **argv)
+@@ -3196,6 +3203,9 @@ void qemu_init(int argc, char **argv)
             case QEMU_OPTION_loadvm:
                 loadvm = optarg;
                 break;
--- a/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
+++ b/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
@@ -28,7 +28,7 @@ index dc1ececc9c..848d2dfdd1 100644
     "-fda/-fdb file  use 'file' as floppy disk 0/1 image\n", QEMU_ARCH_ALL)
 DEF("fdb", HAS_ARG, QEMU_OPTION_fdb, "", QEMU_ARCH_ALL)
 diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 3f2681aded..1a3b9cc4b8 100644
+index ddeace306e..3ee90b3b94 100644
 --- a/softmmu/vl.c
 +++ b/softmmu/vl.c
@@ -2683,6 +2683,7 @@ void qemu_init(int argc, char **argv)
@@ -39,7 +39,7 @@ index 3f2681aded..1a3b9cc4b8 100644
 
     qemu_add_opts(&qemu_drive_opts);
     qemu_add_drive_opts(&qemu_legacy_drive_opts);
-@@ -3306,6 +3307,13 @@ void qemu_init(int argc, char **argv)
+@@ -3308,6 +3309,13 @@ void qemu_init(int argc, char **argv)
                 machine_parse_property_opt(qemu_find_opts("smp-opts"),
                                            "smp", optarg);
                 break;
--- a/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch
+++ b/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch
@@ -72,7 +72,7 @@ index fbb61f18e4..7da3c519ba 100644
 ##
 # @query-machines:
 diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 1a3b9cc4b8..e9b5f62cc3 100644
+index 3ee90b3b94..4b6d0b82fd 100644
 --- a/softmmu/vl.c
 +++ b/softmmu/vl.c
@@ -1597,6 +1597,7 @@ static const QEMUOption *lookup_opt(int argc, char **argv,
@@ -95,7 +95,7 @@ index 1a3b9cc4b8..e9b5f62cc3 100644
     g_slist_free(machines);
     if (local_err) {
         error_append_hint(&local_err, "Use -machine help to list supported machines\n");
-@@ -3248,12 +3254,31 @@ void qemu_init(int argc, char **argv)
+@@ -3250,12 +3256,31 @@ void qemu_init(int argc, char **argv)
             case QEMU_OPTION_machine:
                 {
                     bool help;
--- a/debian/patches/pve/0028-PVE-Backup-add-backup-dump-block-driver.patch
+++ b/debian/patches/pve/0028-PVE-Backup-add-backup-dump-block-driver.patch
@@ -12,20 +12,20 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 [FE: adapt to coroutine changes]
 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 ---
- block/backup-dump.c              | 168 +++++++++++++++++++++++++++++++
+ block/backup-dump.c              | 172 +++++++++++++++++++++++++++++++
 block/backup.c                   |  30 ++----
 block/meson.build                |   1 +
 include/block/block_int-common.h |  35 +++++++
 job.c                            |   3 +-
- 5 files changed, 214 insertions(+), 23 deletions(-)
+ 5 files changed, 218 insertions(+), 23 deletions(-)
 create mode 100644 block/backup-dump.c

 diff --git a/block/backup-dump.c b/block/backup-dump.c
 new file mode 100644
-index 0000000000..232a094426
+index 0000000000..e46abf1070
 --- /dev/null
 +++ b/block/backup-dump.c
-@@ -0,0 +1,168 @@
+@@ -0,0 +1,172 @@
 +/*
 + * BlockDriver to send backup data stream to a callback function
 + *
@@ -37,6 +37,8 @@ index 0000000000..232a094426
 + */
 +
 +#include "qemu/osdep.h"
+
+#include "qapi/qmp/qdict.h"
 +#include "qom/object_interfaces.h"
 +#include "block/block_int.h"
 +
@@ -169,7 +171,7 @@ index 0000000000..232a094426
 +block_init(bdrv_backup_dump_init);
 +
 +
-+BlockDriverState *bdrv_backup_dump_create(
+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
 +    int dump_cb_block_size,
 +    uint64_t byte_size,
 +    BackupDumpFunc *dump_cb,
@@ -177,9 +179,11 @@ index 0000000000..232a094426
 +    Error **errp)
 +{
 +    BDRVBackupDumpState *state;
-+    BlockDriverState *bs = bdrv_new_open_driver(
-+        &bdrv_backup_dump_drive, NULL, BDRV_O_RDWR, errp);
 +
+    QDict *options = qdict_new();
+    qdict_put_str(options, "driver", "backup-dump-drive");
+
+    BlockDriverState *bs = bdrv_co_open(NULL, NULL, options, BDRV_O_RDWR, errp);
 +    if (!bs) {
 +        return NULL;
 +    }
@@ -255,7 +259,7 @@ index 59b71ba9f3..6fde9f7dcd 100644
   'blkdebug.c',
   'blklogwrites.c',
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
-index 74195c3004..0f2e1817ad 100644
+index 74195c3004..0a0339eee4 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -26,6 +26,7 @@
@@ -272,7 +276,7 @@ index 74195c3004..0f2e1817ad 100644
 
 +typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
 +
-+BlockDriverState *bdrv_backup_dump_create(
+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
 +    int dump_cb_block_size,
 +    uint64_t byte_size,
 +    BackupDumpFunc *dump_cb,
--- a/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch
+++ b/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch
@@ -84,63 +84,25 @@ Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
     create jobs in a drained section]
 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 ---
- block/backup-dump.c              |   10 +-
- block/meson.build                |    5 +
- block/monitor/block-hmp-cmds.c   |   39 ++
- blockdev.c                       |    1 +
- hmp-commands-info.hx             |   14 +
- hmp-commands.hx                  |   29 +
- include/block/block_int-common.h |    2 +-
- include/monitor/hmp.h            |    3 +
- meson.build                      |    1 +
- monitor/hmp-cmds.c               |   72 ++
- proxmox-backup-client.c          |  146 ++++
- proxmox-backup-client.h          |   60 ++
- pve-backup.c                     | 1067 ++++++++++++++++++++++++++++++
- qapi/block-core.json             |  229 +++++++
- qapi/common.json                 |   14 +
- qapi/machine.json                |   16 +-
- 16 files changed, 1690 insertions(+), 18 deletions(-)
+ block/meson.build              |    5 +
+ block/monitor/block-hmp-cmds.c |   39 ++
+ blockdev.c                     |    1 +
+ hmp-commands-info.hx           |   14 +
+ hmp-commands.hx                |   29 +
+ include/monitor/hmp.h          |    3 +
+ meson.build                    |    1 +
+ monitor/hmp-cmds.c             |   72 +++
+ proxmox-backup-client.c        |  146 +++++
+ proxmox-backup-client.h        |   60 ++
+ pve-backup.c                   | 1089 ++++++++++++++++++++++++++++++++
+ qapi/block-core.json           |  229 +++++++
+ qapi/common.json               |   14 +
+ qapi/machine.json              |   16 +-
+ 14 files changed, 1704 insertions(+), 14 deletions(-)
 create mode 100644 proxmox-backup-client.c
 create mode 100644 proxmox-backup-client.h
 create mode 100644 pve-backup.c

-diff --git a/block/backup-dump.c b/block/backup-dump.c
-index 232a094426..e46abf1070 100644
--- a/block/backup-dump.c
-+++ b/block/backup-dump.c
-@@ -9,6 +9,8 @@
-  */
- 
- #include "qemu/osdep.h"
-+
-+#include "qapi/qmp/qdict.h"
- #include "qom/object_interfaces.h"
- #include "block/block_int.h"
- 
-@@ -141,7 +143,7 @@ static void bdrv_backup_dump_init(void)
- block_init(bdrv_backup_dump_init);
- 
- 
-BlockDriverState *bdrv_backup_dump_create(
-+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
-     int dump_cb_block_size,
-     uint64_t byte_size,
-     BackupDumpFunc *dump_cb,
-@@ -149,9 +151,11 @@ BlockDriverState *bdrv_backup_dump_create(
-     Error **errp)
- {
-     BDRVBackupDumpState *state;
-    BlockDriverState *bs = bdrv_new_open_driver(
-        &bdrv_backup_dump_drive, NULL, BDRV_O_RDWR, errp);
- 
-+    QDict *options = qdict_new();
-+    qdict_put_str(options, "driver", "backup-dump-drive");
-+
-+    BlockDriverState *bs = bdrv_co_open(NULL, NULL, options, BDRV_O_RDWR, errp);
-     if (!bs) {
-         return NULL;
-     }
 diff --git a/block/meson.build b/block/meson.build
 index 6fde9f7dcd..6d468f89e5 100644
 --- a/block/meson.build
@@ -281,19 +243,6 @@ index e352f86872..0c8b6725fb 100644
 ERST
 
     {
-diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
-index 0f2e1817ad..0a0339eee4 100644
--- a/include/block/block_int-common.h
-+++ b/include/block/block_int-common.h
-@@ -63,7 +63,7 @@
- 
- typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
- 
-BlockDriverState *bdrv_backup_dump_create(
-+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
-     int dump_cb_block_size,
-     uint64_t byte_size,
-     BackupDumpFunc *dump_cb,
 diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
 index 7a7def7530..cba7afe70c 100644
 --- a/include/monitor/hmp.h
@@ -637,10 +586,10 @@ index 0000000000..8cbf645b2c
 +#endif /* PROXMOX_BACKUP_CLIENT_H */
 diff --git a/pve-backup.c b/pve-backup.c
 new file mode 100644
-index 0000000000..d84d807654
+index 0000000000..ae3d137e12
 --- /dev/null
 +++ b/pve-backup.c
-@@ -0,0 +1,1067 @@
+@@ -0,0 +1,1089 @@
 +#include "proxmox-backup-client.h"
 +#include "vma.h"
 +
@@ -950,7 +899,12 @@ index 0000000000..d84d807654
 +
 +    qemu_co_mutex_lock(&backup_state.backup_mutex);
 +
-+    if (ret < 0) {
+    /*
+     * All jobs in the transaction will be canceled when one receives an error.
+     * The first error wins, so only set it for ECANCELED if it was the last
+     * job. This allows more interesting errors from other jobs to win.
+     */
+    if (ret < 0 && (ret != -ECANCELED || !g_list_nth(backup_state.di_list, 1))) {
 +        Error *local_err = NULL;
 +        error_setg(&local_err, "job failed with err %d - %s", ret, strerror(-ret));
 +        pvebackup_propagate_error(local_err);
@@ -1219,6 +1173,66 @@ index 0000000000..d84d807654
 +    aio_co_enter(data->ctx, data->co);
 +}
 +
+/*
+ * Returns a list of device infos, which needs to be freed by the caller. In
+ * case of an error, errp will be set, but the returned value might still be a
+ * list.
+ */
+static GList coroutine_fn *get_device_info(
+    const char *devlist,
+    Error **errp)
+{
+    gchar **devs = NULL;
+    GList *di_list = NULL;
+
+    if (devlist) {
+        devs = g_strsplit_set(devlist, ",;:", -1);
+
+        gchar **d = devs;
+        while (d && *d) {
+            BlockBackend *blk = blk_by_name(*d);
+            if (!blk) {
+                error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+                          "Device '%s' not found", *d);
+                goto err;
+            }
+            BlockDriverState *bs = blk_bs(blk);
+            if (!bdrv_co_is_inserted(bs)) {
+                error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
+                goto err;
+            }
+            PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
+            di->bs = bs;
+            di_list = g_list_append(di_list, di);
+            d++;
+        }
+    } else {
+        BdrvNextIterator it;
+
+        for (BlockDriverState *bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
+            if (!bdrv_co_is_inserted(bs) || bdrv_is_read_only(bs)) {
+                continue;
+            }
+
+            PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
+            di->bs = bs;
+            di_list = g_list_append(di_list, di);
+        }
+    }
+
+    if (!di_list) {
+        error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
+        goto err;
+    }
+
+err:
+    if (devs) {
+        g_strfreev(devs);
+    }
+
+    return di_list;
+}
+
 +UuidInfo coroutine_fn *qmp_backup(
 +    const char *backup_file,
 +    const char *password,
@@ -1244,13 +1258,10 @@ index 0000000000..d84d807654
 +
 +    qemu_co_mutex_lock(&backup_state.backup_mutex);
 +
-+    BlockBackend *blk;
-+    BlockDriverState *bs = NULL;
 +    Error *local_err = NULL;
 +    uuid_t uuid;
 +    VmaWriter *vmaw = NULL;
 +    ProxmoxBackupHandle *pbs = NULL;
-+    gchar **devs = NULL;
 +    GList *di_list = NULL;
 +    GList *l;
 +    UuidInfo *uuid_info;
@@ -1268,48 +1279,12 @@ index 0000000000..d84d807654
 +    /* Todo: try to auto-detect format based on file name */
 +    format = has_format ? format : BACKUP_FORMAT_VMA;
 +
-+    if (devlist) {
-+        devs = g_strsplit_set(devlist, ",;:", -1);
-+
-+        gchar **d = devs;
-+        while (d && *d) {
-+            blk = blk_by_name(*d);
-+            if (blk) {
-+                bs = blk_bs(blk);
-+                if (!bdrv_co_is_inserted(bs)) {
-+                    error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
-+                    goto err;
-+                }
-+                PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
-+                di->bs = bs;
-+                di_list = g_list_append(di_list, di);
-+            } else {
-+                error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-+                          "Device '%s' not found", *d);
-+                goto err;
-+            }
-+            d++;
-+        }
-+
-+    } else {
-+        BdrvNextIterator it;
-+
-+        bs = NULL;
-+        for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-+            if (!bdrv_co_is_inserted(bs) || bdrv_is_read_only(bs)) {
-+                continue;
-+            }
-+
-+            PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
-+            di->bs = bs;
-+            di_list = g_list_append(di_list, di);
-+        }
-+    }
-+
-+    if (!di_list) {
-+        error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
+    di_list = get_device_info(devlist, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
 +        goto err;
 +    }
+    assert(di_list);
 +
 +    size_t total = 0;
 +
@@ -1597,10 +1572,6 @@ index 0000000000..d84d807654
 +    g_list_free(di_list);
 +    backup_state.di_list = NULL;
 +
-+    if (devs) {
-+        g_strfreev(devs);
-+    }
-+
 +    if (vmaw) {
 +        Error *err = NULL;
 +        vma_writer_close(vmaw, &err);
--- a/debian/patches/pve/0033-PVE-redirect-stderr-to-journal-when-daemonized.patch
+++ b/debian/patches/pve/0033-PVE-redirect-stderr-to-journal-when-daemonized.patch
@@ -34,10 +34,10 @@ index cbfc9a43fb..8206270272 100644
 endif
 
 diff --git a/os-posix.c b/os-posix.c
-index cfcb96533c..fb2ad87009 100644
+index 0cc1d991b1..f33d9901cf 100644
 --- a/os-posix.c
 +++ b/os-posix.c
-@@ -28,6 +28,8 @@
+@@ -29,6 +29,8 @@
 #include <pwd.h>
 #include <grp.h>
 #include <libgen.h>
@@ -46,7 +46,7 @@ index cfcb96533c..fb2ad87009 100644
 
 /* Needed early for CONFIG_BSD etc. */
 #include "net/slirp.h"
-@@ -310,9 +312,10 @@ void os_setup_post(void)
+@@ -332,9 +334,10 @@ void os_setup_post(void)
 
         dup2(fd, 0);
         dup2(fd, 1);
--- a/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
+++ b/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
@@ -174,10 +174,10 @@ index 0000000000..887e998b9e
 +                         NULL);
 +}
 diff --git a/pve-backup.c b/pve-backup.c
-index d84d807654..9c8b88d075 100644
+index ae3d137e12..e6b17b797e 100644
 --- a/pve-backup.c
 +++ b/pve-backup.c
-@@ -1060,6 +1060,7 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
+@@ -1082,6 +1082,7 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
     ret->pbs_library_version = g_strdup(proxmox_backup_qemu_version());
     ret->pbs_dirty_bitmap = true;
     ret->pbs_dirty_bitmap_savevm = true;
--- a/debian/patches/pve/0038-block-io-accept-NULL-qiov-in-bdrv_pad_request.patch
+++ b/debian/patches/pve/0038-block-io-accept-NULL-qiov-in-bdrv_pad_request.patch
@@ -8,26 +8,57 @@ results (only copy-on-read matters). In this case they will pass NULL as
 the target QEMUIOVector, which will however trip bdrv_pad_request, since
 it wants to extend its passed vector.

-Simply check for NULL and do nothing, there's no reason to pad the
-target if it will be discarded anyway.
+If there is no qiov, no operation can be done with it, but the bytes
+and offset still need to be updated, so the subsequent aligned read
+will actually be aligned and not run into an assertion failure.

 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
+[FE: do update bytes and offset in any case]
+Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 ---
- block/io.c | 4 ++++
- 1 file changed, 4 insertions(+)
+ block/io.c | 29 ++++++++++++++++-------------
+ 1 file changed, 16 insertions(+), 13 deletions(-)

 diff --git a/block/io.c b/block/io.c
-index 83d1b1dfdc..24a3c84c93 100644
+index 83d1b1dfdc..e927881e40 100644
 --- a/block/io.c
 +++ b/block/io.c
-@@ -1710,6 +1710,10 @@ static int bdrv_pad_request(BlockDriverState *bs,
-     int sliced_niov;
-     size_t sliced_head, sliced_tail;
+@@ -1723,22 +1723,25 @@ static int bdrv_pad_request(BlockDriverState *bs,
+         return 0;
+     }
 
-+    if (!qiov) {
-+        return 0;
-+    }
+-    sliced_iov = qemu_iovec_slice(*qiov, *qiov_offset, *bytes,
+-                                  &sliced_head, &sliced_tail,
+-                                  &sliced_niov);
+-
+-    /* Guaranteed by bdrv_check_request32() */
+-    assert(*bytes <= SIZE_MAX);
+-    ret = bdrv_create_padded_qiov(bs, pad, sliced_iov, sliced_niov,
+-                                  sliced_head, *bytes);
+-    if (ret < 0) {
+-        bdrv_padding_finalize(pad);
+-        return ret;
+    if (qiov && *qiov) {
+        sliced_iov = qemu_iovec_slice(*qiov, *qiov_offset, *bytes,
+                                      &sliced_head, &sliced_tail,
+                                      &sliced_niov);
 +
-     /* Should have been checked by the caller already */
-     ret = bdrv_check_request32(*offset, *bytes, *qiov, *qiov_offset);
-     if (ret < 0) {
+        /* Guaranteed by bdrv_check_request32() */
+        assert(*bytes <= SIZE_MAX);
+        ret = bdrv_create_padded_qiov(bs, pad, sliced_iov, sliced_niov,
+                                      sliced_head, *bytes);
+        if (ret < 0) {
+            bdrv_padding_finalize(pad);
+            return ret;
+        }
+        *qiov = &pad->local_qiov;
+        *qiov_offset = 0;
+     }
+
+     *bytes += pad->head + pad->tail;
+     *offset -= pad->head;
+-    *qiov = &pad->local_qiov;
+-    *qiov_offset = 0;
+     if (padded) {
+         *padded = true;
+     }
--- a/debian/patches/pve/0040-Revert-block-rbd-workaround-for-ceph-issue-53784.patch
+++ b/debian/patches/pve/0040-Revert-block-rbd-workaround-for-ceph-issue-53784.patch
@@ -7,6 +7,7 @@ This reverts commit fc176116cdea816ceb8dd969080b2b95f58edbc0 in
 preparation to revert 0347a8fd4c3faaedf119be04c197804be40a384b.

 Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 block/rbd.c | 42 ++----------------------------------------
 1 file changed, 2 insertions(+), 40 deletions(-)
--- a/debian/patches/pve/0041-Revert-block-rbd-fix-handling-of-holes-in-.bdrv_co_b.patch
+++ b/debian/patches/pve/0041-Revert-block-rbd-fix-handling-of-holes-in-.bdrv_co_b.patch
@@ -8,6 +8,7 @@ This reverts commit 9e302f64bb407a9bb097b626da97228c2654cfee in
 preparation to revert 0347a8fd4c3faaedf119be04c197804be40a384b.

 Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 block/rbd.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
--- a/debian/patches/pve/0042-Revert-block-rbd-implement-bdrv_co_block_status.patch
+++ b/debian/patches/pve/0042-Revert-block-rbd-implement-bdrv_co_block_status.patch
@@ -18,6 +18,7 @@ Upstream bug report:
 https://gitlab.com/qemu-project/qemu/-/issues/1026

 Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 block/rbd.c | 112 ----------------------------------------------------
 1 file changed, 112 deletions(-)
--- a/debian/patches/pve/0043-alloc-track-fix-deadlock-during-drop.patch
+++ b/debian/patches/pve/0043-alloc-track-fix-deadlock-during-drop.patch
@@ -67,6 +67,7 @@ general to not leave unnecessary block nodes lying around.

 Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 block/alloc-track.c | 54 ++++++++++++++-------------------------------
 1 file changed, 16 insertions(+), 38 deletions(-)
--- a/debian/patches/pve/0044-migration-for-snapshots-hold-the-BQL-during-setup-ca.patch
+++ b/debian/patches/pve/0044-migration-for-snapshots-hold-the-BQL-during-setup-ca.patch
@@ -43,6 +43,7 @@ callbacks that need the BQL and only take the lock if it's not already
 held.

 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 include/migration/register.h   |  2 +-
 migration/block-dirty-bitmap.c | 15 ++++++++++++---
--- a/debian/patches/pve/0045-savevm-async-don-t-hold-BQL-during-setup.patch
+++ b/debian/patches/pve/0045-savevm-async-don-t-hold-BQL-during-setup.patch
@@ -8,6 +8,7 @@ callbacks" for why. This is separate, because a version of that one
 will hopefully land upstream.

 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
+Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
 migration/savevm-async.c | 2 --
 1 file changed, 2 deletions(-)
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -9,6 +9,8 @@ extra/0008-target-i386-the-sgx_epc_get_section-stub-is-reachabl.patch
 extra/0009-ui-clipboard-mark-type-as-not-available-when-there-i.patch
 extra/0010-virtio-scsi-Attach-event-vq-notifier-with-no_poll.patch
 extra/0011-virtio-Re-enable-notifications-after-drain.patch
+extra/0012-qemu_init-increase-NOFILE-soft-limit-on-POSIX.patch
+extra/0013-virtio-blk-avoid-using-ioeventfd-state-in-irqfd-cond.patch
 bitmap-mirror/0001-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
 bitmap-mirror/0002-drive-mirror-add-support-for-conditional-and-always-.patch
 bitmap-mirror/0003-mirror-add-check-for-bitmap-mode-without-bitmap.patch
Author	SHA1	Message	Date
Thomas Lamprecht	47bdd04244	bump version to 8.1.5-4 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2024-03-12 14:08:48 +01:00
Thomas Lamprecht	8dd76cc52d	backup: factor out & clean up gathering device info into helper Squash the two original patches [0][1] from Fiona, which got send separate to be easier to review, into the big patch that adds the Proxmox backup integration. [0]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061479.html [1]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061478.html Originally-by: Fiona Ebner <f.ebner@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2024-03-12 13:55:00 +01:00
Fiona Ebner	cd7676f3e6	backup: avoid bubbling up first ECANCELED error With pvebackup_propagate_error(), the first error wins. When one job in the transaction fails, it is expected that later jobs get the ECANCELED error. Those are not interesting and by skipping them a more interesting error, which is likely the actual root cause, can win. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>	2024-03-12 13:20:28 +01:00
Fiona Ebner	862b46e3e0	cleanup: squash backup dump driver change into patch introducing the driver Makes it simpler and shorter. Still results in the same code after applying both patches in question. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>	2024-03-12 13:19:30 +01:00
Fiona Ebner	061e9ceb36	fix patch for accepting NULL qiov when padding All callers of the function pass an address, so dereferencing once before checking for NULL is required. It's also necessary to update bytes and offset nevertheless, so the request will actually be aligned later and not trigger an assertion failure. Seems like this was accidentally broken in `8dca018` ("udpate and rebase to QEMU v6.0.0") and this is effectively a revert to the original version of the patch. The qiov functions changed back then, which might've been the reason Stefan tried to simplify the patch. Should fix live-import for certain kinds of VMDK images. Reported-by: Wolfgang Bumiller <w.bumiller@proxmox.com> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>	2024-03-12 13:11:21 +01:00
Thomas Lamprecht	0d4462207b	bump version to 8.1.5-3 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2024-02-21 20:11:27 +01:00
Fiona Ebner	ed159bc32a	add patch to fix deadlock with VirtIO block and iothread during QMP stop Backported from commit bfa36802d1 ("virtio-blk: avoid using ioeventfd state in irqfd conditional") because the rework/rename dataplane -> ioeventfd didn't happen yet. Reported in the community forum [0] and reproduced doing a backup loop to PBS with suspend mode with fio doing heavy IO in the guest and using an RBD storage (with krbd). [0]: https://forum.proxmox.com/threads/141320 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>	2024-02-21 20:09:22 +01:00
Fiona Ebner	86460aef76	fix #4507 : add patch to automatically increase NOFILE soft limit In many configurations, e.g. multiple vNICs with multiple queues or with many Ceph OSDs, the default soft limit of 1024 is not enough. QEMU is supposed to work fine with file descriptors >= 1024 and does not use select() on POSIX. Bump the soft limit to the allowed hard limit to avoid issues with the aforementioned configurations. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>	2024-02-06 10:33:12 +01:00