vitalif
/
pve-qemu
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

update submodule and patches for 4.2.0 

The long overdue nice rebase+cleanup was done by Dietmar

Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
master
Thomas Lamprecht 2020-03-10 15:12:50 +01:00
parent 75697a753c
commit 6402d96100
62 changed files with 2578 additions and 6149 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
debian/control vendored
View File

@ -18,6 +18,7 @@ Build-Depends: autotools-dev,
libnuma-dev,
libpci-dev,
libpixman-1-dev,
libproxmox-backup-qemu0-dev,
librbd-dev (>= 0.48),
libsdl1.2-dev,
libseccomp-dev,

45
debian/patches/extra/0001-monitor-qmp-resume-monitor-when-clearing-its-queue.patch vendored
View File

@ -1,45 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 2 Oct 2019 10:30:03 +0200
Subject: [PATCH] monitor/qmp: resume monitor when clearing its queue
When a monitor's queue is filled up in handle_qmp_command()
it gets suspended. It's the dispatcher bh's job currently to
resume the monitor, which it does after processing an event
from the queue. However, it is possible for a
CHR_EVENT_CLOSED event to be processed before before the bh
is scheduled, which will clear the queue without resuming
the monitor, thereby preventing the dispatcher from reaching
the resume() call.
Fix this by resuming the monitor when clearing a queue which
was filled up.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
monitor/qmp.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index e1b196217d..fb3e66c62a 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -70,9 +70,19 @@ static void qmp_request_free(QMPRequest *req)
/* Caller must hold mon->qmp.qmp_queue_lock */
static void monitor_qmp_cleanup_req_queue_locked(MonitorQMP *mon)
{
+ bool need_resume = (!qmp_oob_enabled(mon) && mon->qmp_requests->length > 0)
+ || mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX;
while (!g_queue_is_empty(mon->qmp_requests)) {
qmp_request_free(g_queue_pop_head(mon->qmp_requests));
}
+ if (need_resume) {
+ /*
+ * Pairs with the monitor_suspend() in handle_qmp_command() in case the
+ * queue gets cleared from a CH_EVENT_CLOSED event before the dispatch
+ * bh got scheduled.
+ */
+ monitor_resume(&mon->common);
+ }
}
static void monitor_qmp_cleanup_queues(MonitorQMP *mon)

76
debian/patches/extra/0002-virtio-blk-schedule-virtio_notify_config-to-run-on-m.patch vendored
View File

@ -1,76 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Sergio Lopez <slp@redhat.com>
Date: Mon, 16 Sep 2019 13:24:12 +0200
Subject: [PATCH] virtio-blk: schedule virtio_notify_config to run on main
context
virtio_notify_config() needs to acquire the global mutex, which isn't
allowed from an iothread, and may lead to a deadlock like this:
- main thead
* Has acquired: qemu_global_mutex.
* Is trying the acquire: iothread AioContext lock via
AIO_WAIT_WHILE (after aio_poll).
- iothread
* Has acquired: AioContext lock.
* Is trying to acquire: qemu_global_mutex (via
virtio_notify_config->prepare_mmio_access).
If virtio_blk_resize() is called from an iothread, schedule
virtio_notify_config() to be run in the main context BH.
[Removed unnecessary newline as suggested by Kevin Wolf
<kwolf@redhat.com>.
--Stefan]
Signed-off-by: Sergio Lopez <slp@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20190916112411.21636-1-slp@redhat.com
Message-Id: <20190916112411.21636-1-slp@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
hw/block/virtio-blk.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index cbb3729158..0d9adcdaff 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -16,6 +16,7 @@
#include "qemu/iov.h"
#include "qemu/module.h"
#include "qemu/error-report.h"
+#include "qemu/main-loop.h"
#include "trace.h"
#include "hw/block/block.h"
#include "sysemu/blockdev.h"
@@ -1082,11 +1083,24 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
return 0;
}
+static void virtio_resize_cb(void *opaque)
+{
+ VirtIODevice *vdev = opaque;
+
+ assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+ virtio_notify_config(vdev);
+}
+
static void virtio_blk_resize(void *opaque)
{
VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
- virtio_notify_config(vdev);
+ /*
+ * virtio_notify_config() needs to acquire the global mutex,
+ * so it can't be called from an iothread. Instead, schedule
+ * it to be run in the main context BH.
+ */
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), virtio_resize_cb, vdev);
}
static const BlockDevOps virtio_block_ops = {
--
2.20.1

1016
debian/patches/extra/0003-vnc-fix-memory-leak-when-vnc-disconnect.patch vendored
View File

File diff suppressed because it is too large Load Diff

126
debian/patches/extra/0004-util-add-slirp_fmt-helpers.patch vendored
View File

@ -1,126 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Mon, 27 Jan 2020 10:24:09 +0100
Subject: [PATCH 1/2] util: add slirp_fmt() helpers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Various calls to snprintf() in libslirp assume that snprintf() returns
"only" the number of bytes written (excluding terminating NUL).
https://pubs.opengroup.org/onlinepubs/9699919799/functions/snprintf.html#tag_16_159_04
"Upon successful completion, the snprintf() function shall return the
number of bytes that would be written to s had n been sufficiently
large excluding the terminating null byte."
Introduce slirp_fmt() that handles several pathological cases the
way libslirp usually expect:
- treat error as fatal (instead of silently returning -1)
- fmt0() will always \0 end
- return the number of bytes actually written (instead of what would
have been written, which would usually result in OOB later), including
the ending \0 for fmt0()
- warn if truncation happened (instead of ignoring)
Other less common cases can still be handled with strcpy/snprintf() etc.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-Id: <20200127092414.169796-2-marcandre.lureau@redhat.com>
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
---
slirp/src/util.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
slirp/src/util.h | 3 +++
2 files changed, 65 insertions(+)
diff --git a/slirp/src/util.c b/slirp/src/util.c
index e596087..e3b6257 100644
--- a/slirp/src/util.c
+++ b/slirp/src/util.c
@@ -364,3 +364,65 @@ void slirp_pstrcpy(char *buf, int buf_size, const char *str)
}
*q = '\0';
}
+
+static int slirp_vsnprintf(char *str, size_t size,
+ const char *format, va_list args)
+{
+ int rv = vsnprintf(str, size, format, args);
+
+ if (rv < 0) {
+ g_error("vsnprintf() failed: %s", g_strerror(errno));
+ }
+
+ return rv;
+}
+
+/*
+ * A snprintf()-like function that:
+ * - returns the number of bytes written (excluding optional \0-ending)
+ * - dies on error
+ * - warn on truncation
+ */
+int slirp_fmt(char *str, size_t size, const char *format, ...)
+{
+ va_list args;
+ int rv;
+
+ va_start(args, format);
+ rv = slirp_vsnprintf(str, size, format, args);
+ va_end(args);
+
+ if (rv > size) {
+ g_critical("vsnprintf() truncation");
+ }
+
+ return MIN(rv, size);
+}
+
+/*
+ * A snprintf()-like function that:
+ * - always \0-end (unless size == 0)
+ * - returns the number of bytes actually written, including \0 ending
+ * - dies on error
+ * - warn on truncation
+ */
+int slirp_fmt0(char *str, size_t size, const char *format, ...)
+{
+ va_list args;
+ int rv;
+
+ va_start(args, format);
+ rv = slirp_vsnprintf(str, size, format, args);
+ va_end(args);
+
+ if (rv >= size) {
+ g_critical("vsnprintf() truncation");
+ if (size > 0)
+ str[size - 1] = '\0';
+ rv = size;
+ } else {
+ rv += 1; /* include \0 */
+ }
+
+ return rv;
+}
diff --git a/slirp/src/util.h b/slirp/src/util.h
index 3c6223c..0558dfc 100644
--- a/slirp/src/util.h
+++ b/slirp/src/util.h
@@ -177,4 +177,7 @@ static inline int slirp_socket_set_fast_reuse(int fd)
void slirp_pstrcpy(char *buf, int buf_size, const char *str);
+int slirp_fmt(char *str, size_t size, const char *format, ...);
+int slirp_fmt0(char *str, size_t size, const char *format, ...);
+
#endif
--
2.20.1

135
debian/patches/extra/0005-tcp_emu-fix-unsafe-snprintf-usages.patch vendored
View File

@ -1,135 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Mon, 27 Jan 2020 10:24:14 +0100
Subject: [PATCH 2/2] tcp_emu: fix unsafe snprintf() usages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Various calls to snprintf() assume that snprintf() returns "only" the
number of bytes written (excluding terminating NUL).
https://pubs.opengroup.org/onlinepubs/9699919799/functions/snprintf.html#tag_16_159_04
"Upon successful completion, the snprintf() function shall return the
number of bytes that would be written to s had n been sufficiently
large excluding the terminating null byte."
Before patch ce131029, if there isn't enough room in "m_data" for the
"DCC ..." message, we overflow "m_data".
After the patch, if there isn't enough room for the same, we don't
overflow "m_data", but we set "m_len" out-of-bounds. The next time an
access is bounded by "m_len", we'll have a buffer overflow then.
Use slirp_fmt*() to fix potential OOB memory access.
Reported-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-Id: <20200127092414.169796-7-marcandre.lureau@redhat.com>
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
---
slirp/src/tcp_subr.c | 44 +++++++++++++++++++++-----------------------
1 file changed, 21 insertions(+), 23 deletions(-)
diff --git a/slirp/src/tcp_subr.c b/slirp/src/tcp_subr.c
index d6dd133..93b3488 100644
--- a/slirp/src/tcp_subr.c
+++ b/slirp/src/tcp_subr.c
@@ -655,8 +655,7 @@ int tcp_emu(struct socket *so, struct mbuf *m)
NTOHS(n1);
NTOHS(n2);
m_inc(m, snprintf(NULL, 0, "%d,%d\r\n", n1, n2) + 1);
- m->m_len = snprintf(m->m_data, M_ROOM(m), "%d,%d\r\n", n1, n2);
- assert(m->m_len < M_ROOM(m));
+ m->m_len = slirp_fmt(m->m_data, M_ROOM(m), "%d,%d\r\n", n1, n2);
} else {
*eol = '\r';
}
@@ -696,9 +695,9 @@ int tcp_emu(struct socket *so, struct mbuf *m)
n4 = (laddr & 0xff);
m->m_len = bptr - m->m_data; /* Adjust length */
- m->m_len += snprintf(bptr, m->m_size - m->m_len,
- "ORT %d,%d,%d,%d,%d,%d\r\n%s", n1, n2, n3, n4,
- n5, n6, x == 7 ? buff : "");
+ m->m_len += slirp_fmt(bptr, M_FREEROOM(m),
+ "ORT %d,%d,%d,%d,%d,%d\r\n%s",
+ n1, n2, n3, n4, n5, n6, x == 7 ? buff : "");
return 1;
} else if ((bptr = (char *)strstr(m->m_data, "27 Entering")) != NULL) {
/*
@@ -731,11 +730,9 @@ int tcp_emu(struct socket *so, struct mbuf *m)
n4 = (laddr & 0xff);
m->m_len = bptr - m->m_data; /* Adjust length */
- m->m_len +=
- snprintf(bptr, m->m_size - m->m_len,
- "27 Entering Passive Mode (%d,%d,%d,%d,%d,%d)\r\n%s",
- n1, n2, n3, n4, n5, n6, x == 7 ? buff : "");
-
+ m->m_len += slirp_fmt(bptr, M_FREEROOM(m),
+ "27 Entering Passive Mode (%d,%d,%d,%d,%d,%d)\r\n%s",
+ n1, n2, n3, n4, n5, n6, x == 7 ? buff : "");
return 1;
}
@@ -758,8 +755,8 @@ int tcp_emu(struct socket *so, struct mbuf *m)
if (m->m_data[m->m_len - 1] == '\0' && lport != 0 &&
(so = tcp_listen(slirp, INADDR_ANY, 0, so->so_laddr.s_addr,
htons(lport), SS_FACCEPTONCE)) != NULL)
- m->m_len =
- snprintf(m->m_data, m->m_size, "%d", ntohs(so->so_fport)) + 1;
+ m->m_len = slirp_fmt0(m->m_data, M_ROOM(m),
+ "%d", ntohs(so->so_fport));
return 1;
case EMU_IRC:
@@ -778,9 +775,10 @@ int tcp_emu(struct socket *so, struct mbuf *m)
return 1;
}
m->m_len = bptr - m->m_data; /* Adjust length */
- m->m_len += snprintf(bptr, m->m_size, "DCC CHAT chat %lu %u%c\n",
- (unsigned long)ntohl(so->so_faddr.s_addr),
- ntohs(so->so_fport), 1);
+ m->m_len += slirp_fmt(bptr, M_FREEROOM(m),
+ "DCC CHAT chat %lu %u%c\n",
+ (unsigned long)ntohl(so->so_faddr.s_addr),
+ ntohs(so->so_fport), 1);
} else if (sscanf(bptr, "DCC SEND %256s %u %u %u", buff, &laddr, &lport,
&n1) == 4) {
if ((so = tcp_listen(slirp, INADDR_ANY, 0, htonl(laddr),
@@ -788,10 +786,10 @@ int tcp_emu(struct socket *so, struct mbuf *m)
return 1;
}
m->m_len = bptr - m->m_data; /* Adjust length */
- m->m_len +=
- snprintf(bptr, m->m_size, "DCC SEND %s %lu %u %u%c\n", buff,
- (unsigned long)ntohl(so->so_faddr.s_addr),
- ntohs(so->so_fport), n1, 1);
+ m->m_len += slirp_fmt(bptr, M_FREEROOM(m),
+ "DCC SEND %s %lu %u %u%c\n", buff,
+ (unsigned long)ntohl(so->so_faddr.s_addr),
+ ntohs(so->so_fport), n1, 1);
} else if (sscanf(bptr, "DCC MOVE %256s %u %u %u", buff, &laddr, &lport,
&n1) == 4) {
if ((so = tcp_listen(slirp, INADDR_ANY, 0, htonl(laddr),
@@ -799,10 +797,10 @@ int tcp_emu(struct socket *so, struct mbuf *m)
return 1;
}
m->m_len = bptr - m->m_data; /* Adjust length */
- m->m_len +=
- snprintf(bptr, m->m_size, "DCC MOVE %s %lu %u %u%c\n", buff,
- (unsigned long)ntohl(so->so_faddr.s_addr),
- ntohs(so->so_fport), n1, 1);
+ m->m_len += slirp_fmt(bptr, M_FREEROOM(m),
+ "DCC MOVE %s %lu %u %u%c\n", buff,
+ (unsigned long)ntohl(so->so_faddr.s_addr),
+ ntohs(so->so_fport), n1, 1);
}
return 1;
--
2.20.1

10
debian/patches/pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 29 Nov 2017 11:11:46 +0100
Subject: [PATCH] PVE: [Config] block/file: change locking default to off
Date: Tue, 10 Mar 2020 12:54:58 +0100
Subject: [PATCH 01/32] PVE: [Config] block/file: change locking default to off
'auto' only checks whether the system generally supports OFD
locks but not whether the storage the file resides on
@ -14,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 2184aa980c..71aa45ce5d 100644
index 1b805bd938..44b49265ae 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -444,7 +444,7 @@ static QemuOptsList raw_runtime_opts = {
@@ -449,7 +449,7 @@ static QemuOptsList raw_runtime_opts = {
{
.name = "locking",
.type = QEMU_OPT_STRING,
@ -26,7 +26,7 @@ index 2184aa980c..71aa45ce5d 100644
},
{
.name = "pr-manager",
@@ -533,7 +533,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
@@ -538,7 +538,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
s->use_lock = false;
break;
case ON_OFF_AUTO_AUTO:

8
debian/patches/pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:16:49 +0100
Subject: [PATCH] PVE: [Config] Adjust network script path to /etc/kvm/
Date: Tue, 10 Mar 2020 12:54:59 +0100
Subject: [PATCH 02/32] PVE: [Config] Adjust network script path to /etc/kvm/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/include/net/net.h b/include/net/net.h
index acf0451fc4..4a64633577 100644
index e175ba9677..5b9f099d21 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -209,8 +209,9 @@ void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp);
@@ -208,8 +208,9 @@ void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp);
int net_hub_id_for_client(NetClientState *nc, int *id);
NetClientState *net_hub_port_find(int hub_id);

8
debian/patches/pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:30:21 +0100
Subject: [PATCH] PVE: [Config] set the CPU model to kvm64/32 instead of
Date: Tue, 10 Mar 2020 12:55:00 +0100
Subject: [PATCH 03/32] PVE: [Config] set the CPU model to kvm64/32 instead of
qemu64/32
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
@ -10,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 8b3dc5533e..1fea162e02 100644
index cde2a16b94..3e73104bf9 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1725,9 +1725,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
@@ -1940,9 +1940,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
#define CPU_RESOLVING_TYPE TYPE_X86_CPU
#ifdef TARGET_X86_64

6
debian/patches/pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:33:34 +0100
Subject: [PATCH] PVE: [Config] ui/spice: default to pve certificates
Date: Tue, 10 Mar 2020 12:55:01 +0100
Subject: [PATCH 04/32] PVE: [Config] ui/spice: default to pve certificates
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,7 +9,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 2ffc3335f0..c95bbd6c77 100644
index ecc2ec2c55..ca04965ead 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -668,32 +668,35 @@ void qemu_spice_init(void)

8
debian/patches/pve/0005-PVE-Config-smm_available-false.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Tue, 29 Sep 2015 15:37:44 +0200
Subject: [PATCH] PVE: [Config] smm_available = false
Date: Tue, 10 Mar 2020 12:55:02 +0100
Subject: [PATCH 05/32] PVE: [Config] smm_available = false
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
@ -10,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index d011733ff7..e20e189a5f 100644
index ac08e63604..4bd9ab52a0 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2723,7 +2723,7 @@ bool pc_machine_is_smm_enabled(PCMachineState *pcms)
@@ -2040,7 +2040,7 @@ bool pc_machine_is_smm_enabled(PCMachineState *pcms)
if (tcg_enabled() || qtest_enabled()) {
smm_available = true;
} else if (kvm_enabled()) {

7
debian/patches/pve/0006-PVE-Config-glusterfs-no-default-logfile-if-daemonize.patch vendored
View File

@ -1,7 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 24 Oct 2016 09:32:36 +0200
Subject: [PATCH] PVE: [Config] glusterfs: no default logfile if daemonized
Date: Tue, 10 Mar 2020 12:55:03 +0100
Subject: [PATCH 06/32] PVE: [Config] glusterfs: no default logfile if
daemonized
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,7 +10,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/block/gluster.c b/block/gluster.c
index f64dc5b01e..061cab48c0 100644
index 4fa4a77a47..bfb57ba098 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -42,7 +42,7 @@

8
debian/patches/pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Fri, 31 Aug 2018 09:32:55 +0200
Subject: [PATCH] PVE: [Config] rbd: block: rbd: disable
Date: Tue, 10 Mar 2020 12:55:04 +0100
Subject: [PATCH 07/32] PVE: [Config] rbd: block: rbd: disable
rbd_cache_writethrough_until_flush
Either the cache mode asks for a cache or not. There's no
@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+)
diff --git a/block/rbd.c b/block/rbd.c
index 59757b3120..d00c9d2d12 100644
index 027cbcc695..3ac7ff7bd5 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -636,6 +636,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
@@ -637,6 +637,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
rados_conf_set(*cluster, "rbd_cache", "false");
}

12
debian/patches/pve/0008-PVE-Up-qmp-add-get_link_status.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 16:34:41 +0100
Subject: [PATCH] PVE: [Up] qmp: add get_link_status
Date: Tue, 10 Mar 2020 12:55:05 +0100
Subject: [PATCH 08/32] PVE: [Up] qmp: add get_link_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -11,10 +11,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 files changed, 43 insertions(+)
diff --git a/net/net.c b/net/net.c
index 7d4098254f..c6d5e66bfc 100644
index 84aa6d8d00..f548202ec6 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1347,6 +1347,33 @@ void hmp_info_network(Monitor *mon, const QDict *qdict)
@@ -1349,6 +1349,33 @@ void hmp_info_network(Monitor *mon, const QDict *qdict)
}
}
@ -49,7 +49,7 @@ index 7d4098254f..c6d5e66bfc 100644
{
NetClientState *nc;
diff --git a/qapi/net.json b/qapi/net.json
index 728990f4fb..d53c66320b 100644
index 335295be50..7f3ea194c8 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -34,6 +34,21 @@
@ -75,7 +75,7 @@ index 728990f4fb..d53c66320b 100644
# @netdev_add:
#
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 38af54d6b3..d6a4177935 100644
index 9751b11f8f..a449f158e1 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -61,6 +61,7 @@

10
debian/patches/pve/0009-PVE-Up-glusterfs-allow-partial-reads.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 30 Nov 2016 10:27:47 +0100
Subject: [PATCH] PVE: [Up] glusterfs: allow partial reads
Date: Tue, 10 Mar 2020 12:55:06 +0100
Subject: [PATCH 09/32] PVE: [Up] glusterfs: allow partial reads
This should deal with qemu bug #1644754 until upstream
decides which way to go. The general direction seems to be
@ -16,7 +16,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/block/gluster.c b/block/gluster.c
index 061cab48c0..0ed1ec5856 100644
index bfb57ba098..81fff09c6c 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -57,6 +57,7 @@ typedef struct GlusterAIOCB {
@ -59,7 +59,7 @@ index 061cab48c0..0ed1ec5856 100644
ret = glfs_preadv_async(s->fd, qiov->iov, qiov->niov, offset, 0,
gluster_finish_aiocb, &acb);
}
@@ -1279,6 +1285,7 @@ static coroutine_fn int qemu_gluster_co_flush_to_disk(BlockDriverState *bs)
@@ -1280,6 +1286,7 @@ static coroutine_fn int qemu_gluster_co_flush_to_disk(BlockDriverState *bs)
acb.ret = 0;
acb.coroutine = qemu_coroutine_self();
acb.aio_context = bdrv_get_aio_context(bs);
@ -67,7 +67,7 @@ index 061cab48c0..0ed1ec5856 100644
ret = glfs_fsync_async(s->fd, gluster_finish_aiocb, &acb);
if (ret < 0) {
@@ -1325,6 +1332,7 @@ static coroutine_fn int qemu_gluster_co_pdiscard(BlockDriverState *bs,
@@ -1326,6 +1333,7 @@ static coroutine_fn int qemu_gluster_co_pdiscard(BlockDriverState *bs,
acb.ret = 0;
acb.coroutine = qemu_coroutine_self();
acb.aio_context = bdrv_get_aio_context(bs);

9
debian/patches/pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch vendored
View File

@ -1,7 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:18:46 +0100
Subject: [PATCH] PVE: [Up] qemu-img: return success on info without snapshots
Date: Tue, 10 Mar 2020 12:55:07 +0100
Subject: [PATCH 10/32] PVE: [Up] qemu-img: return success on info without
snapshots
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/qemu-img.c b/qemu-img.c
index 79983772de..c64f260876 100644
index 95a24b9762..12211bed76 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2773,7 +2773,8 @@ static int img_info(int argc, char **argv)
@@ -2791,7 +2791,8 @@ static int img_info(int argc, char **argv)
list = collect_image_info_list(image_opts, filename, fmt, chain,
force_share);
if (!list) {

20
debian/patches/pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Fri, 23 Jun 2017 12:01:43 +0200
Subject: [PATCH] PVE: [Up] qemu-img dd: add osize and read from/to
Date: Tue, 10 Mar 2020 12:55:08 +0100
Subject: [PATCH 11/32] PVE: [Up] qemu-img dd: add osize and read from/to
stdin/stdout
Neither convert nor dd were previously able to write to or
@ -53,10 +53,10 @@ index 1c93e6d185..8094abb3ee 100644
DEF("info", img_info,
diff --git a/qemu-img.c b/qemu-img.c
index c64f260876..8129677d7a 100644
index 12211bed76..d2516968c6 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4413,10 +4413,12 @@ out:
@@ -4405,10 +4405,12 @@ out:
#define C_IF 04
#define C_OF 010
#define C_SKIP 020
@ -69,7 +69,7 @@ index c64f260876..8129677d7a 100644
};
struct DdIo {
@@ -4495,6 +4497,20 @@ static int img_dd_skip(const char *arg,
@@ -4487,6 +4489,20 @@ static int img_dd_skip(const char *arg,
return 0;
}
@ -90,7 +90,7 @@ index c64f260876..8129677d7a 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
@@ -4535,6 +4551,7 @@ static int img_dd(int argc, char **argv)
@@ -4527,6 +4543,7 @@ static int img_dd(int argc, char **argv)
{ "if", img_dd_if, C_IF },
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
@ -98,7 +98,7 @@ index c64f260876..8129677d7a 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
@@ -4613,8 +4630,13 @@ static int img_dd(int argc, char **argv)
@@ -4605,8 +4622,13 @@ static int img_dd(int argc, char **argv)
arg = NULL;
}
@ -114,7 +114,7 @@ index c64f260876..8129677d7a 100644
ret = -1;
goto out;
}
@@ -4626,85 +4648,101 @@ static int img_dd(int argc, char **argv)
@@ -4618,85 +4640,101 @@ static int img_dd(int argc, char **argv)
goto out;
}
@ -280,7 +280,7 @@ index c64f260876..8129677d7a 100644
}
if (dd.flags & C_SKIP && (in.offset > INT64_MAX / in.bsz ||
@@ -4722,11 +4760,17 @@ static int img_dd(int argc, char **argv)
@@ -4714,11 +4752,17 @@ static int img_dd(int argc, char **argv)
for (out_pos = 0; in_pos < size; block_count++) {
int in_ret, out_ret;
@ -302,7 +302,7 @@ index c64f260876..8129677d7a 100644
}
if (in_ret < 0) {
error_report("error while reading from input image file: %s",
@@ -4736,9 +4780,13 @@ static int img_dd(int argc, char **argv)
@@ -4728,9 +4772,13 @@ static int img_dd(int argc, char **argv)
}
in_pos += in_ret;

16
debian/patches/pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Fri, 23 Feb 2018 08:43:18 +0100
Subject: [PATCH] PVE: [Up] qemu-img dd: add isize parameter
Date: Tue, 10 Mar 2020 12:55:09 +0100
Subject: [PATCH 12/32] PVE: [Up] qemu-img dd: add isize parameter
for writing small images from stdin to bigger ones
@ -15,10 +15,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 26 insertions(+), 3 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index 8129677d7a..1b7f211368 100644
index d2516968c6..8da1ea3951 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4414,11 +4414,13 @@ out:
@@ -4406,11 +4406,13 @@ out:
#define C_OF 010
#define C_SKIP 020
#define C_OSIZE 040
@ -32,7 +32,7 @@ index 8129677d7a..1b7f211368 100644
};
struct DdIo {
@@ -4511,6 +4513,20 @@ static int img_dd_osize(const char *arg,
@@ -4503,6 +4505,20 @@ static int img_dd_osize(const char *arg,
return 0;
}
@ -53,7 +53,7 @@ index 8129677d7a..1b7f211368 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
@@ -4525,12 +4541,14 @@ static int img_dd(int argc, char **argv)
@@ -4517,12 +4533,14 @@ static int img_dd(int argc, char **argv)
int c, i;
const char *out_fmt = "raw";
const char *fmt = NULL;
@ -69,7 +69,7 @@ index 8129677d7a..1b7f211368 100644
};
struct DdIo in = {
.bsz = 512, /* Block size is by default 512 bytes */
@@ -4552,6 +4570,7 @@ static int img_dd(int argc, char **argv)
@@ -4544,6 +4562,7 @@ static int img_dd(int argc, char **argv)
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
{ "osize", img_dd_osize, C_OSIZE },
@ -77,7 +77,7 @@ index 8129677d7a..1b7f211368 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
@@ -4758,14 +4777,18 @@ static int img_dd(int argc, char **argv)
@@ -4750,14 +4769,18 @@ static int img_dd(int argc, char **argv)
in.buf = g_new(uint8_t, in.bsz);

14
debian/patches/pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Wed, 21 Mar 2018 08:51:34 +0100
Subject: [PATCH] PVE: [Up] qemu-img dd : add -n skip_create
Date: Tue, 10 Mar 2020 12:55:10 +0100
Subject: [PATCH 13/32] PVE: [Up] qemu-img dd : add -n skip_create
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index 1b7f211368..e14d2370c4 100644
index 8da1ea3951..ea3edb4f04 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4543,7 +4543,7 @@ static int img_dd(int argc, char **argv)
@@ -4535,7 +4535,7 @@ static int img_dd(int argc, char **argv)
const char *fmt = NULL;
int64_t size = 0, readsize = 0;
int64_t block_count = 0, out_pos, in_pos;
@ -21,7 +21,7 @@ index 1b7f211368..e14d2370c4 100644
struct DdInfo dd = {
.flags = 0,
.count = 0,
@@ -4581,7 +4581,7 @@ static int img_dd(int argc, char **argv)
@@ -4573,7 +4573,7 @@ static int img_dd(int argc, char **argv)
{ 0, 0, 0, 0 }
};
@ -30,7 +30,7 @@ index 1b7f211368..e14d2370c4 100644
if (c == EOF) {
break;
}
@@ -4601,6 +4601,9 @@ static int img_dd(int argc, char **argv)
@@ -4593,6 +4593,9 @@ static int img_dd(int argc, char **argv)
case 'h':
help();
break;
@ -40,7 +40,7 @@ index 1b7f211368..e14d2370c4 100644
case 'U':
force_share = true;
break;
@@ -4741,13 +4744,15 @@ static int img_dd(int argc, char **argv)
@@ -4733,13 +4736,15 @@ static int img_dd(int argc, char **argv)
size - in.bsz * in.offset, &error_abort);
}

14
debian/patches/pve/0014-PVE-virtio-balloon-improve-query-balloon.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:27:49 +0100
Subject: [PATCH] PVE: virtio-balloon: improve query-balloon
Date: Tue, 10 Mar 2020 12:55:11 +0100
Subject: [PATCH 14/32] PVE: virtio-balloon: improve query-balloon
Actually provide memory information via the query-balloon
command.
@ -14,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 files changed, 81 insertions(+), 4 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 25de154307..7c09716035 100644
index 40b04f5180..76e907e628 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -712,8 +712,37 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
@@ -713,8 +713,37 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
static void virtio_balloon_stat(void *opaque, BalloonInfo *info)
{
VirtIOBalloon *dev = opaque;
@ -58,10 +58,10 @@ index 25de154307..7c09716035 100644
static void virtio_balloon_to_target(void *opaque, ram_addr_t target)
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 5ca3ebe942..1b32c59329 100644
index b2551c16d1..2e725ed818 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -870,7 +870,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
@@ -854,7 +854,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
return;
}
@ -99,7 +99,7 @@ index 5ca3ebe942..1b32c59329 100644
qapi_free_BalloonInfo(info);
}
diff --git a/qapi/misc.json b/qapi/misc.json
index a7fba7230c..2445c950cc 100644
index 33b94e3589..ed65ed27e3 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -408,10 +408,30 @@

32
debian/patches/pve/0015-PVE-qapi-modify-query-machines.patch vendored
View File

@ -1,37 +1,39 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:31:18 +0100
Subject: [PATCH] PVE: qapi: modify query machines
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 12:55:12 +0100
Subject: [PATCH 15/32] PVE: qapi: modify query machines
provide '*is-current' in MachineInfo struct
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
hw/core/machine-qmp-cmds.c | 5 +++++
hw/core/machine-qmp-cmds.c | 6 ++++++
qapi/machine.json | 4 +++-
2 files changed, 8 insertions(+), 1 deletion(-)
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 5bd95b8ab0..fd68f9baf8 100644
index eed5aeb2f7..1953633e82 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -229,6 +229,11 @@ MachineInfoList *qmp_query_machines(Error **errp)
@@ -230,6 +230,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
info->numa_mem_supported = mc->numa_mem_supported;
info->deprecated = !!mc->deprecation_reason;
+
+ if (strcmp(mc->name, MACHINE_GET_CLASS(current_machine)->name) == 0) {
+ info->has_is_current = true;
+ info->is_current = true;
+ }
+
entry = g_malloc0(sizeof(*entry));
entry->value = info;
entry->next = mach_list;
if (mc->default_cpu_type) {
info->default_cpu_type = g_strdup(mc->default_cpu_type);
info->has_default_cpu_type = true;
diff --git a/qapi/machine.json b/qapi/machine.json
index 6db8a7e2ec..7b82c5f7f5 100644
index ca26779f1a..cbdb6f6d66 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -313,6 +313,8 @@
@@ -336,6 +336,8 @@
#
# @is-default: whether the machine is default
#
@ -40,12 +42,12 @@ index 6db8a7e2ec..7b82c5f7f5 100644
# @cpu-max: maximum number of CPUs supported by the machine type
# (since 1.5.0)
#
@@ -329,7 +331,7 @@
@@ -355,7 +357,7 @@
##
{ 'struct': 'MachineInfo',
'data': { 'name': 'str', '*alias': 'str',
- '*is-default': 'bool', 'cpu-max': 'int',
+ '*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
'deprecated': 'bool' } }
'deprecated': 'bool', '*default-cpu-type': 'str' } }

8
debian/patches/pve/0016-PVE-qapi-modify-spice-query.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 14:32:11 +0100
Subject: [PATCH] PVE: qapi: modify spice query
Date: Tue, 10 Mar 2020 12:55:13 +0100
Subject: [PATCH 16/32] PVE: qapi: modify spice query
Provide the last ticket in the SpiceInfo struct optionally.
@ -12,7 +12,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 8 insertions(+)
diff --git a/qapi/ui.json b/qapi/ui.json
index 59e412139a..bcd781a1b9 100644
index e04525d8b4..6127990e23 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -211,11 +211,14 @@
@ -31,7 +31,7 @@ index 59e412139a..bcd781a1b9 100644
'if': 'defined(CONFIG_SPICE)' }
diff --git a/ui/spice-core.c b/ui/spice-core.c
index c95bbd6c77..ccba92a6ed 100644
index ca04965ead..243466c13d 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -539,6 +539,11 @@ SpiceInfo *qmp_query_spice(Error **errp)

67
debian/patches/pve/0017-PVE-internal-snapshot-async.patch vendored
View File

@ -1,9 +1,12 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 16:04:32 +0100
Subject: [PATCH] PVE: internal snapshot async
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 13:49:27 +0100
Subject: [PATCH 17/32] PVE: internal snapshot async
Truncate at 1024 boundary (Fabian Ebner will send a patch for stable)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
Makefile.objs | 1 +
hmp-commands-info.hx | 13 +
@ -14,13 +17,13 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
qapi/migration.json | 34 +++
qapi/misc.json | 32 +++
qemu-options.hx | 13 +
savevm-async.c | 460 +++++++++++++++++++++++++++++++++++
savevm-async.c | 464 +++++++++++++++++++++++++++++++++++
vl.c | 10 +
11 files changed, 658 insertions(+)
11 files changed, 662 insertions(+)
create mode 100644 savevm-async.c
diff --git a/Makefile.objs b/Makefile.objs
index 6a143dcd57..21dd93b58c 100644
index 11ba1a36bd..f97b40f232 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -48,6 +48,7 @@ common-obj-y += bootdevice.o iothread.o
@ -32,7 +35,7 @@ index 6a143dcd57..21dd93b58c 100644
common-obj-y += qdev-monitor.o device-hotplug.o
common-obj-$(CONFIG_WIN32) += os-win32.o
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index c59444c461..444bd8e43d 100644
index 257ee7d7a3..139e673bea 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -608,6 +608,19 @@ STEXI
@ -56,10 +59,10 @@ index c59444c461..444bd8e43d 100644
{
diff --git a/hmp-commands.hx b/hmp-commands.hx
index bfa5681dd2..e075d413c0 100644
index cfcc044ce4..104288322d 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1944,3 +1944,35 @@ ETEXI
@@ -1945,3 +1945,35 @@ ETEXI
STEXI
@end table
ETEXI
@ -130,10 +133,10 @@ index a0e9511440..c6ee8295f0 100644
void hmp_screendump(Monitor *mon, const QDict *qdict);
void hmp_nbd_server_start(Monitor *mon, const QDict *qdict);
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 1b32c59329..39a8020367 100644
index 2e725ed818..90aa34be25 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -2640,6 +2640,63 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
@@ -2607,6 +2607,63 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
hmp_handle_error(mon, &err);
}
@ -198,10 +201,10 @@ index 1b32c59329..39a8020367 100644
{
IOThreadInfoList *info_list = qmp_query_iothreads(NULL);
diff --git a/qapi/migration.json b/qapi/migration.json
index 9cfbaf8c6c..e206355d56 100644
index b7348d0c8b..2792409977 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -219,6 +219,40 @@
@@ -222,6 +222,40 @@
'*compression': 'CompressionStats',
'*socket-address': ['SocketAddress'] } }
@ -243,12 +246,12 @@ index 9cfbaf8c6c..e206355d56 100644
# @query-migrate:
#
diff --git a/qapi/misc.json b/qapi/misc.json
index 2445c950cc..31029e3132 100644
index ed65ed27e3..4c4618a574 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -1384,6 +1384,38 @@
@@ -1368,6 +1368,38 @@
##
{ 'command': 'query-target', 'returns': 'TargetInfo' }
{ 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
+##
+# @savevm-start:
@ -286,10 +289,10 @@ index 2445c950cc..31029e3132 100644
# @AcpiTableOptions:
#
diff --git a/qemu-options.hx b/qemu-options.hx
index 9621e934c0..34994daafd 100644
index 65c9473b73..4cb2681bfc 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3731,6 +3731,19 @@ STEXI
@@ -3818,6 +3818,19 @@ STEXI
Start right away with a saved state (@code{loadvm} in monitor)
ETEXI
@ -311,10 +314,10 @@ index 9621e934c0..34994daafd 100644
"-daemonize daemonize QEMU after initializing\n", QEMU_ARCH_ALL)
diff --git a/savevm-async.c b/savevm-async.c
new file mode 100644
index 0000000000..2149010bb8
index 0000000000..54ceeae26c
--- /dev/null
+++ b/savevm-async.c
@@ -0,0 +1,460 @@
@@ -0,0 +1,464 @@
+#include "qemu/osdep.h"
+#include "migration/migration.h"
+#include "migration/savevm.h"
@ -323,6 +326,7 @@ index 0000000000..2149010bb8
+#include "migration/ram.h"
+#include "migration/qemu-file.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/runstate.h"
+#include "block/block.h"
+#include "sysemu/block-backend.h"
+#include "qapi/error.h"
@ -332,6 +336,8 @@ index 0000000000..2149010bb8
+#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-commands-block.h"
+#include "qemu/cutils.h"
+#include "qemu/main-loop.h"
+#include "qemu/rcu.h"
+
+/* #define DEBUG_SAVEVM_STATE */
+
@ -421,10 +427,11 @@ index 0000000000..2149010bb8
+
+ if (snap_state.target) {
+ /* try to truncate, but ignore errors (will fail on block devices).
+ * note: bdrv_read() need whole blocks, so we round up
+ * note1: bdrv_read() need whole blocks, so we need to round up
+ * note2: PVE requires 1024 (BDRV_SECTOR_SIZE*2) alignment
+ */
+ size_t size = (snap_state.bs_pos + BDRV_SECTOR_SIZE) & BDRV_SECTOR_MASK;
+ blk_truncate(snap_state.target, size, PREALLOC_MODE_OFF, NULL);
+ size_t size = QEMU_ALIGN_UP(snap_state.bs_pos, BDRV_SECTOR_SIZE*2);
+ blk_truncate(snap_state.target, size, false, PREALLOC_MODE_OFF, NULL);
+ blk_op_unblock_all(snap_state.target, snap_state.blocker);
+ error_free(snap_state.blocker);
+ snap_state.blocker = NULL;
@ -455,7 +462,7 @@ index 0000000000..2149010bb8
+ snap_state.state = SAVE_STATE_ERROR;
+}
+
+static int block_state_close(void *opaque)
+static int block_state_close(void *opaque, Error **errp)
+{
+ snap_state.file = NULL;
+ return blk_flush(snap_state.target);
@ -474,7 +481,7 @@ index 0000000000..2149010bb8
+}
+
+static ssize_t block_state_writev_buffer(void *opaque, struct iovec *iov,
+ int iovcnt, int64_t pos)
+ int iovcnt, int64_t pos, Error **errp)
+{
+ QEMUIOVector qiov;
+ BlkRwCo rwco;
@ -709,7 +716,7 @@ index 0000000000..2149010bb8
+}
+
+static ssize_t loadstate_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
+ size_t size)
+ size_t size, Error **errp)
+{
+ BlockBackend *be = opaque;
+ int64_t maxlen = blk_getlength(be);
@ -776,10 +783,10 @@ index 0000000000..2149010bb8
+ return ret;
+}
diff --git a/vl.c b/vl.c
index b426b32134..1c5536e5bb 100644
index 6a65a64bfd..1616f55a38 100644
--- a/vl.c
+++ b/vl.c
@@ -2869,6 +2869,7 @@ int main(int argc, char **argv, char **envp)
@@ -2840,6 +2840,7 @@ int main(int argc, char **argv, char **envp)
int optind;
const char *optarg;
const char *loadvm = NULL;
@ -787,7 +794,7 @@ index b426b32134..1c5536e5bb 100644
MachineClass *machine_class;
const char *cpu_option;
const char *vga_model = NULL;
@@ -3445,6 +3446,9 @@ int main(int argc, char **argv, char **envp)
@@ -3430,6 +3431,9 @@ int main(int argc, char **argv, char **envp)
case QEMU_OPTION_loadvm:
loadvm = optarg;
break;
@ -797,7 +804,7 @@ index b426b32134..1c5536e5bb 100644
case QEMU_OPTION_full_screen:
dpy.has_full_screen = true;
dpy.full_screen = true;
@@ -4444,6 +4448,12 @@ int main(int argc, char **argv, char **envp)
@@ -4442,6 +4446,12 @@ int main(int argc, char **argv, char **envp)
autostart = 0;
exit(1);
}

12
debian/patches/pve/0018-PVE-block-add-the-zeroinit-block-driver-filter.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 17 Mar 2016 11:33:37 +0100
Subject: [PATCH] PVE: block: add the zeroinit block driver filter
Date: Tue, 10 Mar 2020 12:55:15 +0100
Subject: [PATCH 18/32] PVE: block: add the zeroinit block driver filter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -11,7 +11,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
create mode 100644 block/zeroinit.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 35f3bca4d9..6022242c3f 100644
index e394fe0b6c..a10ceabf5b 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -11,6 +11,7 @@ block-obj-$(CONFIG_QED) += qed.o qed-l2-cache.o qed-table.o qed-cluster.o
@ -24,7 +24,7 @@ index 35f3bca4d9..6022242c3f 100644
block-obj-y += blklogwrites.o
diff --git a/block/zeroinit.c b/block/zeroinit.c
new file mode 100644
index 0000000000..e78511d36c
index 0000000000..b74a78ece6
--- /dev/null
+++ b/block/zeroinit.c
@@ -0,0 +1,204 @@
@ -186,9 +186,9 @@ index 0000000000..e78511d36c
+}
+
+static int zeroinit_co_truncate(BlockDriverState *bs, int64_t offset,
+ PreallocMode prealloc, Error **errp)
+ _Bool exact, PreallocMode prealloc, Error **errp)
+{
+ return bdrv_co_truncate(bs->file, offset, prealloc, errp);
+ return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp);
+}
+
+static int zeroinit_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)

63
debian/patches/pve/0019-PVE-backup-modify-job-api.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Wed, 9 Dec 2015 15:04:57 +0100
Subject: [PATCH] PVE: backup: modify job api
Date: Tue, 10 Mar 2020 12:55:16 +0100
Subject: [PATCH 19/32] PVE: backup: modify job api
Introduce a pause_count parameter to start a backup in
paused mode. This way backups of multiple drives can be
@ -10,18 +10,18 @@ having been started at the same point in time.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/backup.c | 2 ++
block/backup.c | 3 +++
block/replication.c | 2 +-
blockdev.c | 4 ++--
blockdev.c | 3 ++-
include/block/block_int.h | 1 +
job.c | 2 +-
5 files changed, 7 insertions(+), 4 deletions(-)
5 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 8761f1f9a7..30008fcc34 100644
index cf62b1a38c..c155081de2 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -550,6 +550,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
@@ -347,6 +347,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BlockCompletionFunc *cb, void *opaque,
@ -29,20 +29,21 @@ index 8761f1f9a7..30008fcc34 100644
JobTxn *txn, Error **errp)
{
int64_t len;
@@ -675,6 +676,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
@@ -468,6 +469,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
&error_abort);
job->len = len;
+ job->common.job.pause_count += pause_count;
+ job->common.job.pause_count += pause_count;
+
return &job->common;
error:
diff --git a/block/replication.c b/block/replication.c
index 23b2993d74..e70a6cf2bd 100644
index 99532ce521..ec8de7b427 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -546,7 +546,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
0, MIRROR_SYNC_MODE_NONE, NULL, false,
0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
- backup_job_completed, bs, NULL, &local_err);
@ -51,32 +52,24 @@ index 23b2993d74..e70a6cf2bd 100644
error_propagate(errp, local_err);
backup_job_cleanup(bs);
diff --git a/blockdev.c b/blockdev.c
index 4d141e9a1f..a7c97b1585 100644
index 8e029e9c01..c7fa663ebf 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3574,7 +3574,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, txn, &local_err);
+ job_flags, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
goto unref;
@@ -3679,7 +3679,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, txn, &local_err);
+ job_flags, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
}
@@ -3583,7 +3583,8 @@ static BlockJob *do_backup_common(BackupCommon *backup,
backup->filter_node_name,
backup->on_source_error,
backup->on_target_error,
- job_flags, NULL, NULL, txn, errp);
+ job_flags, NULL, NULL, 0, txn, errp);
+
return job;
}
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 05ee6b4866..bb2dddca83 100644
index dd033d0b37..b0d5eb9485 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1173,6 +1173,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
@@ -1215,6 +1215,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BlockCompletionFunc *cb, void *opaque,
@ -85,10 +78,10 @@ index 05ee6b4866..bb2dddca83 100644
void hmp_drive_add_node(Monitor *mon, const char *optstr);
diff --git a/job.c b/job.c
index 28dd48f8a5..7a21e83780 100644
index 04409b40aa..7554f735e3 100644
--- a/job.c
+++ b/job.c
@@ -898,7 +898,7 @@ void job_start(Job *job)
@@ -888,7 +888,7 @@ void job_start(Job *job)
job->co = qemu_coroutine_create(job_co_entry, job);
job->pause_count--;
job->busy = true;

14
debian/patches/pve/0025-PVE-Add-dummy-id-command-line-parameter.patch → debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 30 Aug 2018 14:52:56 +0200
Subject: [PATCH] PVE: Add dummy -id command line parameter
Date: Tue, 10 Mar 2020 12:55:17 +0100
Subject: [PATCH 20/32] PVE: Add dummy -id command line parameter
This used to be part of the qemu-side PVE authentication for
VNC. Now this does nothing.
@ -14,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 11 insertions(+)
diff --git a/qemu-options.hx b/qemu-options.hx
index 34994daafd..6db02fe903 100644
index 4cb2681bfc..b84e260fa5 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -802,6 +802,9 @@ STEXI
@@ -826,6 +826,9 @@ STEXI
@table @option
ETEXI
@ -28,10 +28,10 @@ index 34994daafd..6db02fe903 100644
"-fda/-fdb file use 'file' as floppy disk 0/1 image\n", QEMU_ARCH_ALL)
DEF("fdb", HAS_ARG, QEMU_OPTION_fdb, "", QEMU_ARCH_ALL)
diff --git a/vl.c b/vl.c
index 1c5536e5bb..7ffcd271f1 100644
index 1616f55a38..4df15640c5 100644
--- a/vl.c
+++ b/vl.c
@@ -2857,6 +2857,7 @@ static void user_register_global_props(void)
@@ -2828,6 +2828,7 @@ static void user_register_global_props(void)
int main(int argc, char **argv, char **envp)
{
int i;
@ -39,7 +39,7 @@ index 1c5536e5bb..7ffcd271f1 100644
int snapshot, linux_boot;
const char *initrd_filename;
const char *kernel_filename, *kernel_cmdline;
@@ -3570,6 +3571,13 @@ int main(int argc, char **argv, char **envp)
@@ -3560,6 +3561,13 @@ int main(int argc, char **argv, char **envp)
exit(1);
}
break;

1563
debian/patches/pve/0020-PVE-backup-introduce-vma-archive-format.patch vendored
View File

File diff suppressed because it is too large Load Diff

11
debian/patches/pve/0026-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch → debian/patches/pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch vendored
View File

@ -1,7 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 4 Jul 2016 15:02:26 +0200
Subject: [PATCH] PVE: [Config] Revert "target-i386: disable LINT0 after reset"
Date: Tue, 10 Mar 2020 12:55:18 +0100
Subject: [PATCH 21/32] PVE: [Config] Revert "target-i386: disable LINT0 after
reset"
This reverts commit b8eb5512fd8a115f164edbbe897cdf8884920ccb.
@ -11,10 +12,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index e764a2bb03..f66ea72d78 100644
index 375cb6abe9..e7d479c7e9 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -258,6 +258,15 @@ static void apic_reset_common(DeviceState *dev)
@@ -259,6 +259,15 @@ static void apic_reset_common(DeviceState *dev)
info->vapic_base_update(s);
apic_init_reset(dev);
@ -29,4 +30,4 @@ index e764a2bb03..f66ea72d78 100644
+ }
}
/* This function is only used for old state version 1 and 2 */
static const VMStateDescription vmstate_apic_common;

25
debian/patches/pve/0027-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch → debian/patches/pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch vendored
View File

@ -1,7 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Fri, 26 Apr 2019 09:15:28 +0200
Subject: [PATCH] PVE: [Up+Config] file-posix: make locking optiono on create
Date: Tue, 10 Mar 2020 12:55:19 +0100
Subject: [PATCH 22/32] PVE: [Up+Config] file-posix: make locking optiono on
create
Otherwise creating images on nfs/cifs can be problematic.
@ -13,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 43 insertions(+), 21 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 71aa45ce5d..ae1cd3980e 100644
index 44b49265ae..0722b0f529 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -2230,6 +2230,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2250,6 +2250,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
int fd;
uint64_t perm, shared;
int result = 0;
@ -24,7 +25,7 @@ index 71aa45ce5d..ae1cd3980e 100644
/* Validate options and set default values */
assert(options->driver == BLOCKDEV_DRIVER_FILE);
@@ -2263,19 +2264,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2283,19 +2284,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
perm = BLK_PERM_WRITE | BLK_PERM_RESIZE;
shared = BLK_PERM_ALL & ~BLK_PERM_RESIZE;
@ -59,7 +60,7 @@ index 71aa45ce5d..ae1cd3980e 100644
}
/* Clear the file by truncating it to 0 */
@@ -2308,13 +2312,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2328,13 +2332,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
}
out_unlock:
@ -82,7 +83,7 @@ index 71aa45ce5d..ae1cd3980e 100644
}
out_close:
@@ -2335,6 +2341,7 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
@@ -2355,6 +2361,7 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
PreallocMode prealloc;
char *buf = NULL;
Error *local_err = NULL;
@ -90,7 +91,7 @@ index 71aa45ce5d..ae1cd3980e 100644
/* Skip file: protocol prefix */
strstart(filename, "file:", &filename);
@@ -2352,6 +2359,18 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
@@ -2372,6 +2379,18 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
return -EINVAL;
}
@ -109,7 +110,7 @@ index 71aa45ce5d..ae1cd3980e 100644
options = (BlockdevCreateOptions) {
.driver = BLOCKDEV_DRIVER_FILE,
.u.file = {
@@ -2361,6 +2380,8 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
@@ -2381,6 +2400,8 @@ static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
.preallocation = prealloc,
.has_nocow = true,
.nocow = nocow,
@ -118,7 +119,7 @@ index 71aa45ce5d..ae1cd3980e 100644
},
};
return raw_co_create(&options, errp);
@@ -2838,7 +2859,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared,
@@ -2901,7 +2922,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared,
}
/* Copy locks to the new fd */
@ -128,10 +129,10 @@ index 71aa45ce5d..ae1cd3980e 100644
false, errp);
if (ret < 0) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 97cb7ec41c..f432d4f3ec 100644
index 0cf68fea14..783a868eb2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4284,7 +4284,8 @@
@@ -4259,7 +4259,8 @@
'data': { 'filename': 'str',
'size': 'size',
'*preallocation': 'PreallocMode',

187
debian/patches/pve/0022-PVE-vma-add-throttling-options-to-drive-mapping-fifo.patch vendored
View File

@ -1,187 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 15 Feb 2018 11:07:56 +0100
Subject: [PATCH] PVE: vma: add throttling options to drive mapping fifo
protocol
We now need to call initialize the qom module as well.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
vma.c | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 76 insertions(+), 12 deletions(-)
diff --git a/vma.c b/vma.c
index 1b59fd1555..f9f5c308fe 100644
--- a/vma.c
+++ b/vma.c
@@ -18,7 +18,8 @@
#include "qemu-common.h"
#include "qemu/error-report.h"
#include "qemu/main-loop.h"
-#include "qapi/qmp/qstring.h"
+#include "qemu/cutils.h"
+#include "qapi/qmp/qdict.h"
#include "sysemu/block-backend.h"
static void help(void)
@@ -132,9 +133,39 @@ typedef struct RestoreMap {
char *devname;
char *path;
char *format;
+ uint64_t throttling_bps;
+ char *throttling_group;
bool write_zero;
} RestoreMap;
+static bool try_parse_option(char **line, const char *optname, char **out, const char *inbuf) {
+ size_t optlen = strlen(optname);
+ if (strncmp(*line, optname, optlen) != 0 || (*line)[optlen] != '=') {
+ return false;
+ }
+ if (*out) {
+ g_error("read map failed - duplicate value for option '%s'", optname);
+ }
+ char *value = (*line) + optlen + 1; /* including a '=' */
+ char *colon = strchr(value, ':');
+ if (!colon) {
+ g_error("read map failed - option '%s' not terminated ('%s')",
+ optname, inbuf);
+ }
+ *line = colon+1;
+ *out = g_strndup(value, colon - value);
+ return true;
+}
+
+static uint64_t verify_u64(const char *text) {
+ uint64_t value;
+ const char *endptr = NULL;
+ if (qemu_strtou64(text, &endptr, 0, &value) != 0 || !endptr || *endptr) {
+ g_error("read map failed - not a number: %s", text);
+ }
+ return value;
+}
+
static int extract_content(int argc, char **argv)
{
int c, ret = 0;
@@ -208,6 +239,9 @@ static int extract_content(int argc, char **argv)
while (1) {
char inbuf[8192];
char *line = fgets(inbuf, sizeof(inbuf), map);
+ char *format = NULL;
+ char *bps = NULL;
+ char *group = NULL;
if (!line || line[0] == '\0' || !strcmp(line, "done\n")) {
break;
}
@@ -219,15 +253,19 @@ static int extract_content(int argc, char **argv)
}
}
- char *format = NULL;
- if (strncmp(line, "format=", sizeof("format=")-1) == 0) {
- format = line + sizeof("format=")-1;
- char *colon = strchr(format, ':');
- if (!colon) {
- g_error("read map failed - found only a format ('%s')", inbuf);
+ while (1) {
+ if (!try_parse_option(&line, "format", &format, inbuf) &&
+ !try_parse_option(&line, "throttling.bps", &bps, inbuf) &&
+ !try_parse_option(&line, "throttling.group", &group, inbuf))
+ {
+ break;
}
- format = g_strndup(format, colon - format);
- line = colon+1;
+ }
+
+ uint64_t bps_value = 0;
+ if (bps) {
+ bps_value = verify_u64(bps);
+ g_free(bps);
}
const char *path;
@@ -253,6 +291,8 @@ static int extract_content(int argc, char **argv)
map->devname = g_strdup(devname);
map->path = g_strdup(path);
map->format = format;
+ map->throttling_bps = bps_value;
+ map->throttling_group = group;
map->write_zero = write_zero;
g_hash_table_insert(devmap, map->devname, map);
@@ -280,6 +320,8 @@ static int extract_content(int argc, char **argv)
} else if (di) {
char *devfn = NULL;
const char *format = NULL;
+ uint64_t throttling_bps = 0;
+ const char *throttling_group = NULL;
int flags = BDRV_O_RDWR | BDRV_O_NO_FLUSH;
bool write_zero = true;
@@ -291,6 +333,8 @@ static int extract_content(int argc, char **argv)
}
devfn = map->path;
format = map->format;
+ throttling_bps = map->throttling_bps;
+ throttling_group = map->throttling_group;
write_zero = map->write_zero;
} else {
devfn = g_strdup_printf("%s/tmp-disk-%s.raw",
@@ -315,7 +359,7 @@ static int extract_content(int argc, char **argv)
if (format) {
/* explicit format from commandline */
options = qdict_new();
- qdict_put(options, "driver", qstring_from_str(format));
+ qdict_put_str(options, "driver", format);
} else if ((devlen > 4 && strcmp(devfn+devlen-4, ".raw") == 0) ||
strncmp(devfn, "/dev/", 5) == 0)
{
@@ -324,15 +368,34 @@ static int extract_content(int argc, char **argv)
*/
/* explicit raw format */
options = qdict_new();
- qdict_put(options, "driver", qstring_from_str("raw"));
+ qdict_put_str(options, "driver", "raw");
}
-
if (errp || !(blk = blk_new_open(devfn, NULL, options, flags, &errp))) {
g_error("can't open file %s - %s", devfn,
error_get_pretty(errp));
}
+ if (throttling_group) {
+ blk_io_limits_enable(blk, throttling_group);
+ }
+
+ if (throttling_bps) {
+ if (!throttling_group) {
+ blk_io_limits_enable(blk, devfn);
+ }
+
+ ThrottleConfig cfg;
+ throttle_config_init(&cfg);
+ cfg.buckets[THROTTLE_BPS_WRITE].avg = throttling_bps;
+ Error *err = NULL;
+ if (!throttle_is_valid(&cfg, &err)) {
+ error_report_err(err);
+ g_error("failed to apply throttling");
+ }
+ blk_set_io_limits(blk, &cfg);
+ }
+
if (vma_reader_register_bs(vmar, i, blk, write_zero, &errp) < 0) {
g_error("%s", error_get_pretty(errp));
}
@@ -730,6 +793,7 @@ int main(int argc, char **argv)
}
bdrv_init();
+ module_call_init(MODULE_INIT_QOM);
if (argc < 2) {
help();

8
debian/patches/pve/0028-PVE-savevm-async-kick-AIO-wait-on-block-state-write.patch → debian/patches/pve/0023-PVE-savevm-async-kick-AIO-wait-on-block-state-write.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Date: Fri, 21 Jun 2019 10:38:46 +0200
Subject: [PATCH] PVE: savevm-async: kick AIO wait on block state write
Date: Tue, 10 Mar 2020 12:55:20 +0100
Subject: [PATCH 23/32] PVE: savevm-async: kick AIO wait on block state write
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+)
diff --git a/savevm-async.c b/savevm-async.c
index 2149010bb8..0bbbbf51ba 100644
index 54ceeae26c..393d55af2a 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -154,6 +154,7 @@ static void coroutine_fn block_state_write_entry(void *opaque) {
@@ -158,6 +158,7 @@ static void coroutine_fn block_state_write_entry(void *opaque) {
BlkRwCo *rwco = opaque;
rwco->ret = blk_co_pwritev(snap_state.target, rwco->offset, rwco->qiov->size,
rwco->qiov, 0);

93
debian/patches/pve/0023-PVE-vma-add-cache-option-to-device-map.patch vendored
View File

@ -1,93 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 22 Mar 2018 15:32:04 +0100
Subject: [PATCH] PVE: vma: add cache option to device map
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
vma.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/vma.c b/vma.c
index f9f5c308fe..476b7bee00 100644
--- a/vma.c
+++ b/vma.c
@@ -135,6 +135,7 @@ typedef struct RestoreMap {
char *format;
uint64_t throttling_bps;
char *throttling_group;
+ char *cache;
bool write_zero;
} RestoreMap;
@@ -242,6 +243,7 @@ static int extract_content(int argc, char **argv)
char *format = NULL;
char *bps = NULL;
char *group = NULL;
+ char *cache = NULL;
if (!line || line[0] == '\0' || !strcmp(line, "done\n")) {
break;
}
@@ -256,7 +258,8 @@ static int extract_content(int argc, char **argv)
while (1) {
if (!try_parse_option(&line, "format", &format, inbuf) &&
!try_parse_option(&line, "throttling.bps", &bps, inbuf) &&
- !try_parse_option(&line, "throttling.group", &group, inbuf))
+ !try_parse_option(&line, "throttling.group", &group, inbuf) &&
+ !try_parse_option(&line, "cache", &cache, inbuf))
{
break;
}
@@ -293,6 +296,7 @@ static int extract_content(int argc, char **argv)
map->format = format;
map->throttling_bps = bps_value;
map->throttling_group = group;
+ map->cache = cache;
map->write_zero = write_zero;
g_hash_table_insert(devmap, map->devname, map);
@@ -322,6 +326,7 @@ static int extract_content(int argc, char **argv)
const char *format = NULL;
uint64_t throttling_bps = 0;
const char *throttling_group = NULL;
+ const char *cache = NULL;
int flags = BDRV_O_RDWR | BDRV_O_NO_FLUSH;
bool write_zero = true;
@@ -335,6 +340,7 @@ static int extract_content(int argc, char **argv)
format = map->format;
throttling_bps = map->throttling_bps;
throttling_group = map->throttling_group;
+ cache = map->cache;
write_zero = map->write_zero;
} else {
devfn = g_strdup_printf("%s/tmp-disk-%s.raw",
@@ -356,6 +362,7 @@ static int extract_content(int argc, char **argv)
size_t devlen = strlen(devfn);
QDict *options = NULL;
+ bool writethrough;
if (format) {
/* explicit format from commandline */
options = qdict_new();
@@ -370,12 +377,19 @@ static int extract_content(int argc, char **argv)
options = qdict_new();
qdict_put_str(options, "driver", "raw");
}
+ if (cache && bdrv_parse_cache_mode(cache, &flags, &writethrough)) {
+ g_error("invalid cache option: %s\n", cache);
+ }
if (errp || !(blk = blk_new_open(devfn, NULL, options, flags, &errp))) {
g_error("can't open file %s - %s", devfn,
error_get_pretty(errp));
}
+ if (cache) {
+ blk_set_enable_write_cache(blk, !writethrough);
+ }
+
if (throttling_group) {
blk_io_limits_enable(blk, throttling_group);
}

10
debian/patches/pve/0029-PVE-move-snapshot-cleanup-into-bottom-half.patch → debian/patches/pve/0024-PVE-move-snapshot-cleanup-into-bottom-half.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 24 Jun 2019 10:19:50 +0200
Subject: [PATCH] PVE: move snapshot cleanup into bottom half
Date: Tue, 10 Mar 2020 12:55:21 +0100
Subject: [PATCH 24/32] PVE: move snapshot cleanup into bottom half
as per:
(0ceccd858a8d) migration: qemu_savevm_state_cleanup() in cleanup
@ -16,10 +16,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/savevm-async.c b/savevm-async.c
index 0bbbbf51ba..f9355c5036 100644
index 393d55af2a..790e27ae37 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -197,6 +197,8 @@ static void process_savevm_cleanup(void *opaque)
@@ -201,6 +201,8 @@ static void process_savevm_cleanup(void *opaque)
int ret;
qemu_bh_delete(snap_state.cleanup_bh);
snap_state.cleanup_bh = NULL;
@ -28,7 +28,7 @@ index 0bbbbf51ba..f9355c5036 100644
qemu_mutex_unlock_iothread();
qemu_thread_join(&snap_state.thread);
qemu_mutex_lock_iothread();
@@ -273,7 +275,6 @@ static void *process_savevm_thread(void *opaque)
@@ -277,7 +279,6 @@ static void *process_savevm_thread(void *opaque)
save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
break;
}

28
debian/patches/pve/0024-PVE-vma-remove-forced-NO_FLUSH-option.patch vendored
View File

@ -1,28 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Tue, 27 Mar 2018 10:49:03 +0200
Subject: [PATCH] PVE: vma: remove forced NO_FLUSH option
This one's rbd specific and in no way a sane choice for all
types storages. Instead, we want to honor the cache option
passed along.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
vma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/vma.c b/vma.c
index 476b7bee00..3289fd722f 100644
--- a/vma.c
+++ b/vma.c
@@ -327,7 +327,7 @@ static int extract_content(int argc, char **argv)
uint64_t throttling_bps = 0;
const char *throttling_group = NULL;
const char *cache = NULL;
- int flags = BDRV_O_RDWR | BDRV_O_NO_FLUSH;
+ int flags = BDRV_O_RDWR;
bool write_zero = true;
if (readmap) {

8
debian/patches/pve/0030-PVE-monitor-disable-oob-capability.patch → debian/patches/pve/0025-PVE-monitor-disable-oob-capability.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 14 Nov 2019 14:38:02 +0100
Subject: [PATCH] PVE: monitor: disable oob capability
Date: Tue, 10 Mar 2020 12:55:22 +0100
Subject: [PATCH 25/32] PVE: monitor: disable oob capability
A bisect revealed that commit 8258292e18c3
("monitor: Remove "x-oob", offer capability "oob" unconditionally")
@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index fb3e66c62a..ad6703dc52 100644
index b67a8e7d1f..8f44fed944 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -379,8 +379,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty)
@@ -395,8 +395,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty)
MonitorQMP *mon = g_new0(MonitorQMP, 1);
/* Note: we run QMP monitor in I/O thread when @chr supports that */

11
debian/patches/pve/0044-Acquire-aio_context-before-calling-block_job_add_bdr.patch → debian/patches/pve/0026-PVE-Acquire-aio_context-before-calling-block_job_add.patch vendored
View File

@ -1,7 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Mon, 21 Oct 2019 17:32:22 +0200
Subject: [PATCH] Acquire aio_context before calling block_job_add_bdrv
Date: Tue, 10 Mar 2020 12:55:23 +0100
Subject: [PATCH 26/32] PVE: Acquire aio_context before calling
block_job_add_bdrv
Otherwise backups immediately fail with 'permission denied' since
_add_bdrv tries to release a lock we don't own.
@ -12,10 +13,10 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 10 insertions(+)
diff --git a/blockjob.c b/blockjob.c
index 74abb97bfd..75b9180a7f 100644
index c6e20e2fcd..4e6074f18c 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -448,10 +448,20 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,
@@ -436,10 +436,20 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,
notifier_list_add(&job->job.on_ready, &job->ready_notifier);
notifier_list_add(&job->job.on_idle, &job->idle_notifier);
@ -35,4 +36,4 @@ index 74abb97bfd..75b9180a7f 100644
+
bdrv_op_unblock(bs, BLOCK_OP_TYPE_DATAPLANE, job->blocker);
blk_set_allow_aio_context_change(blk, true);
/* Disable request queuing in the BlockBackend to avoid deadlocks on drain:

10
debian/patches/pve/0045-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch → debian/patches/pve/0027-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch vendored
View File

@ -1,8 +1,8 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Date: Fri, 22 Nov 2019 08:56:20 +0100
Subject: [PATCH] PVE: [Compat]: 4.0 used balloon qemu-4-0-config-size false
here
Date: Tue, 10 Mar 2020 12:55:24 +0100
Subject: [PATCH 27/32] PVE: [Compat]: 4.0 used balloon qemu-4-0-config-size
false here
The underlying issue why this change from upstream to us arised in
the first place is that QEMU 4.0 was already released at the point we
@ -26,10 +26,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 32d1ca9abc..0ee68f1473 100644
index 1689ad3bf8..bdcb351ede 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -34,7 +34,8 @@ GlobalProperty hw_compat_4_0[] = {
@@ -39,7 +39,8 @@ GlobalProperty hw_compat_4_0[] = {
{ "virtio-vga", "edid", "false" },
{ "virtio-gpu", "edid", "false" },
{ "virtio-device", "use-started", "false" },

37
debian/patches/pve/0046-PVE-Allow-version-code-in-machine-type.patch → debian/patches/pve/0028-PVE-Allow-version-code-in-machine-type.patch vendored
View File

@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Thu, 14 Nov 2019 17:56:12 +0100
Subject: [PATCH] PVE: Allow version code in machine type
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 12:55:25 +0100
Subject: [PATCH 28/32] PVE: Allow version code in machine type
E.g. pc-i440fx-4.0+pve3 would print 'pve3' as version code while
selecting pc-i440fx-4.0 as machine type.
@ -13,15 +13,15 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
hw/core/machine-qmp-cmds.c | 6 ++++++
include/hw/boards.h | 2 ++
qapi/machine.json | 2 +-
qapi/machine.json | 3 ++-
vl.c | 15 ++++++++++++++-
4 files changed, 23 insertions(+), 2 deletions(-)
4 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index fd68f9baf8..61fcad138d 100644
index 1953633e82..ca8c0dc53d 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -232,6 +232,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
@@ -234,6 +234,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
if (strcmp(mc->name, MACHINE_GET_CLASS(current_machine)->name) == 0) {
info->has_is_current = true;
info->is_current = true;
@ -33,12 +33,12 @@ index fd68f9baf8..61fcad138d 100644
+ }
}
entry = g_malloc0(sizeof(*entry));
if (mc->default_cpu_type) {
diff --git a/include/hw/boards.h b/include/hw/boards.h
index a71d1a53a5..952d2add82 100644
index de45087f34..e24d2134c0 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -178,6 +178,8 @@ struct MachineClass {
@@ -185,6 +185,8 @@ struct MachineClass {
const char *desc;
const char *deprecation_reason;
@ -46,25 +46,26 @@ index a71d1a53a5..952d2add82 100644
+
void (*init)(MachineState *state);
void (*reset)(MachineState *state);
void (*hot_add_cpu)(MachineState *state, const int64_t id, Error **errp);
void (*wakeup)(MachineState *state);
diff --git a/qapi/machine.json b/qapi/machine.json
index 7b82c5f7f5..11fef6714e 100644
index cbdb6f6d66..a2bd4dd304 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -333,7 +333,7 @@
@@ -359,7 +359,8 @@
'data': { 'name': 'str', '*alias': 'str',
'*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
- 'deprecated': 'bool' } }
+ 'deprecated': 'bool', '*pve-version': 'str' } }
- 'deprecated': 'bool', '*default-cpu-type': 'str' } }
+ 'deprecated': 'bool', '*default-cpu-type': 'str',
+ '*pve-version': 'str' } }
##
# @query-machines:
diff --git a/vl.c b/vl.c
index 7ffcd271f1..1305cdaee7 100644
index 4df15640c5..e7f3ce7607 100644
--- a/vl.c
+++ b/vl.c
@@ -2454,6 +2454,8 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
@@ -2475,6 +2475,8 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
{
MachineClass *mc;
GSList *el;
@ -73,7 +74,7 @@ index 7ffcd271f1..1305cdaee7 100644
if (is_help_option(name)) {
printf("Supported machines are:\n");
@@ -2470,12 +2472,23 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
@@ -2491,12 +2493,23 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
exit(0);
}

831
debian/patches/pve/0021-PVE-Deprecated-adding-old-vma-files.patch → debian/patches/pve/0029-PVE-Backup-add-vma-backup-format-code.patch vendored
View File

@ -1,33 +1,26 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 7 Aug 2017 08:51:16 +0200
Subject: [PATCH] PVE: [Deprecated] adding old vma files
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 12:55:26 +0100
Subject: [PATCH 29/32] PVE-Backup: add vma backup format code
TODO: Move to using a libvma block backend
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Makefile | 3 +-
Makefile.objs | 1 +
block/backup.c | 88 ++--
block/replication.c | 1 +
blockdev.c | 208 +++++----
include/block/block_int.h | 4 +
job.c | 3 +-
vma-reader.c | 857 ++++++++++++++++++++++++++++++++++++++
vma-writer.c | 771 ++++++++++++++++++++++++++++++++++
vma.c | 756 +++++++++++++++++++++++++++++++++
vma.h | 150 +++++++
11 files changed, 2737 insertions(+), 105 deletions(-)
Makefile | 3 +-
Makefile.objs | 1 +
vma-reader.c | 857 ++++++++++++++++++++++++++++++++++++++++++++++++++
vma-writer.c | 771 +++++++++++++++++++++++++++++++++++++++++++++
vma.c | 837 ++++++++++++++++++++++++++++++++++++++++++++++++
vma.h | 150 +++++++++
6 files changed, 2618 insertions(+), 1 deletion(-)
create mode 100644 vma-reader.c
create mode 100644 vma-writer.c
create mode 100644 vma.c
create mode 100644 vma.h
diff --git a/Makefile b/Makefile
index 85862fb81a..421e7b486c 100644
index b437a346d7..18d2dba2e4 100644
--- a/Makefile
+++ b/Makefile
@@ -436,7 +436,7 @@ dummy := $(call unnest-vars,, \
@@ -453,7 +453,7 @@ dummy := $(call unnest-vars,, \
include $(SRC_PATH)/tests/Makefile.include
@ -36,7 +29,7 @@ index 85862fb81a..421e7b486c 100644
qemu-version.h: FORCE
$(call quiet-command, \
@@ -544,6 +544,7 @@ qemu-img.o: qemu-img-cmds.h
@@ -567,6 +567,7 @@ qemu-img.o: qemu-img-cmds.h
qemu-img$(EXESUF): qemu-img.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
@ -45,7 +38,7 @@ index 85862fb81a..421e7b486c 100644
qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index 21dd93b58c..87c6033bc1 100644
index f97b40f232..db7fbbe73b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-y += block.o blockjob.o job.o
@ -56,665 +49,6 @@ index 21dd93b58c..87c6033bc1 100644
block-obj-m = block/
diff --git a/block/backup.c b/block/backup.c
index 30008fcc34..fdcfb0529c 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -41,6 +41,7 @@ typedef struct BackupBlockJob {
/* bitmap for sync=incremental */
BdrvDirtyBitmap *sync_bitmap;
MirrorSyncMode sync_mode;
+ BackupDumpFunc *dump_cb;
BlockdevOnError on_source_error;
BlockdevOnError on_target_error;
CoRwlock flush_rwlock;
@@ -129,12 +130,20 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
}
if (buffer_is_zero(*bounce_buffer, nbytes)) {
- ret = blk_co_pwrite_zeroes(job->target, start,
- nbytes, write_flags | BDRV_REQ_MAY_UNMAP);
+ if (job->dump_cb) {
+ ret = job->dump_cb(job->common.job.opaque, job->target, start, nbytes, NULL);
+ } else {
+ ret = blk_co_pwrite_zeroes(job->target, start,
+ nbytes, write_flags | BDRV_REQ_MAY_UNMAP);
+ }
} else {
- ret = blk_co_pwrite(job->target, start,
- nbytes, *bounce_buffer, write_flags |
- (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0));
+ if (job->dump_cb) {
+ ret = job->dump_cb(job->common.job.opaque, job->target, start, nbytes, *bounce_buffer);
+ } else {
+ ret = blk_co_pwrite(job->target, start,
+ nbytes, *bounce_buffer, write_flags |
+ (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0));
+ }
}
if (ret < 0) {
trace_backup_do_cow_write_fail(job, start, ret);
@@ -218,8 +227,11 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
trace_backup_do_cow_process(job, start);
if (job->use_copy_range) {
- ret = backup_cow_with_offload(job, start, dirty_end,
- is_write_notifier);
+ if (job->dump_cb) {
+ ret = - 1;
+ } else {
+ ret = backup_cow_with_offload(job, start, dirty_end, is_write_notifier);
+ }
if (ret < 0) {
job->use_copy_range = false;
}
@@ -304,7 +316,9 @@ static void backup_abort(Job *job)
static void backup_clean(Job *job)
{
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
- assert(s->target);
+ if (!s->target) {
+ return;
+ }
blk_unref(s->target);
s->target = NULL;
@@ -350,9 +364,11 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job,
if (read) {
return block_job_error_action(&job->common, job->on_source_error,
true, error);
- } else {
+ } else if (job->target) {
return block_job_error_action(&job->common, job->on_target_error,
false, error);
+ } else {
+ return BLOCK_ERROR_ACTION_REPORT;
}
}
@@ -549,6 +565,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
int creation_flags,
+ BackupDumpFunc *dump_cb,
BlockCompletionFunc *cb, void *opaque,
int pause_count,
JobTxn *txn, Error **errp)
@@ -560,7 +577,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
HBitmap *copy_bitmap = NULL;
assert(bs);
- assert(target);
+ assert(target || dump_cb);
if (bs == target) {
error_setg(errp, "Source and target cannot be the same");
@@ -573,13 +590,13 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- if (!bdrv_is_inserted(target)) {
+ if (target && !bdrv_is_inserted(target)) {
error_setg(errp, "Device is not inserted: %s",
bdrv_get_device_name(target));
return NULL;
}
- if (compress && target->drv->bdrv_co_pwritev_compressed == NULL) {
+ if (target && compress && target->drv->bdrv_co_pwritev_compressed == NULL) {
error_setg(errp, "Compression is not supported for this drive %s",
bdrv_get_device_name(target));
return NULL;
@@ -589,7 +606,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- if (bdrv_op_is_blocked(target, BLOCK_OP_TYPE_BACKUP_TARGET, errp)) {
+ if (target && bdrv_op_is_blocked(target, BLOCK_OP_TYPE_BACKUP_TARGET, errp)) {
return NULL;
}
@@ -619,7 +636,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- cluster_size = backup_calculate_cluster_size(target, errp);
+ cluster_size = backup_calculate_cluster_size(target ? target : bs, errp);
if (cluster_size < 0) {
goto error;
}
@@ -636,16 +653,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- /* The target must match the source in size, so no resize here either */
- job->target = blk_new(job->common.job.aio_context,
- BLK_PERM_WRITE,
- BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
- BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD);
- ret = blk_insert_bs(job->target, target, errp);
- if (ret < 0) {
- goto error;
+ if (target) {
+ /* The target must match the source in size, so no resize here either */
+ job->target = blk_new(job->common.job.aio_context,
+ BLK_PERM_WRITE,
+ BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
+ BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD);
+ ret = blk_insert_bs(job->target, target, errp);
+ if (ret < 0) {
+ goto error;
+ }
}
+ job->dump_cb = dump_cb;
job->on_source_error = on_source_error;
job->on_target_error = on_target_error;
job->sync_mode = sync_mode;
@@ -658,10 +678,14 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
job->cluster_size = cluster_size;
job->copy_bitmap = copy_bitmap;
copy_bitmap = NULL;
- job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk),
- blk_get_max_transfer(job->target));
- job->copy_range_size = QEMU_ALIGN_DOWN(job->copy_range_size,
- job->cluster_size);
+
+ if (target) {
+ job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk),
+ blk_get_max_transfer(job->target));
+ job->copy_range_size = QEMU_ALIGN_DOWN(job->copy_range_size,
+ job->cluster_size);
+ }
+
/*
* Set use_copy_range, consider the following:
* 1. Compression is not supported for copy_range.
@@ -669,12 +693,16 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
* that in here. If max_transfer is smaller than the job->cluster_size,
* we do not use copy_range (in that case it's zero after aligning down
* above).
+ * 3. If !target, we're using PVE dump_cb callback
*/
- job->use_copy_range = !compress && job->copy_range_size > 0;
+ job->use_copy_range = target && !compress && job->copy_range_size > 0;
+
+ if (target) {
+ /* Required permissions are already taken with target's blk_new() */
+ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
+ &error_abort);
+ }
- /* Required permissions are already taken with target's blk_new() */
- block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
- &error_abort);
job->len = len;
job->common.job.pause_count += pause_count;
diff --git a/block/replication.c b/block/replication.c
index e70a6cf2bd..f060755713 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -546,6 +546,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
0, MIRROR_SYNC_MODE_NONE, NULL, false,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
+ NULL,
backup_job_completed, bs, 0, NULL, &local_err);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/blockdev.c b/blockdev.c
index 7047475a3c..cee7952bbb 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -31,7 +31,6 @@
*/
#include "qemu/osdep.h"
-#include "qemu/uuid.h"
#include "sysemu/block-backend.h"
#include "sysemu/blockdev.h"
#include "hw/block/block.h"
@@ -64,6 +63,7 @@
#include "qemu/cutils.h"
#include "qemu/help_option.h"
#include "qemu/throttle-options.h"
+#include "vma.h"
static QTAILQ_HEAD(, BlockDriverState) monitor_bdrv_states =
QTAILQ_HEAD_INITIALIZER(monitor_bdrv_states);
@@ -3176,15 +3176,14 @@ out:
static struct PVEBackupState {
Error *error;
bool cancel;
- QemuUUID uuid;
+ uuid_t uuid;
char uuid_str[37];
int64_t speed;
time_t start_time;
time_t end_time;
char *backup_file;
- Object *vmaobj;
+ VmaWriter *vmaw;
GList *di_list;
- size_t next_job;
size_t total;
size_t transferred;
size_t zero_bytes;
@@ -3203,6 +3202,71 @@ typedef struct PVEBackupDevInfo {
static void pvebackup_run_next_job(void);
+static int pvebackup_dump_cb(void *opaque, BlockBackend *target,
+ uint64_t start, uint64_t bytes,
+ const void *pbuf)
+{
+ const uint64_t size = bytes;
+ const unsigned char *buf = pbuf;
+ PVEBackupDevInfo *di = opaque;
+
+ if (backup_state.cancel) {
+ return size; // return success
+ }
+
+ uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
+ if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
+ if (!backup_state.error) {
+ error_setg(&backup_state.error,
+ "got unaligned write inside backup dump "
+ "callback (sector %ld)", start);
+ }
+ return -1; // not aligned to cluster size
+ }
+
+ int ret = -1;
+
+ if (backup_state.vmaw) {
+ size_t zero_bytes = 0;
+ uint64_t remaining = size;
+ while (remaining > 0) {
+ ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num,
+ buf, &zero_bytes);
+ ++cluster_num;
+ if (buf) {
+ buf += VMA_CLUSTER_SIZE;
+ }
+ if (ret < 0) {
+ if (!backup_state.error) {
+ vma_writer_error_propagate(backup_state.vmaw, &backup_state.error);
+ }
+ if (di->bs && di->bs->job) {
+ job_cancel(&di->bs->job->job, true);
+ }
+ break;
+ } else {
+ backup_state.zero_bytes += zero_bytes;
+ if (remaining >= VMA_CLUSTER_SIZE) {
+ backup_state.transferred += VMA_CLUSTER_SIZE;
+ remaining -= VMA_CLUSTER_SIZE;
+ } else {
+ backup_state.transferred += remaining;
+ remaining = 0;
+ }
+ }
+ }
+ } else {
+ if (!buf) {
+ backup_state.zero_bytes += size;
+ }
+ backup_state.transferred += size;
+ }
+
+ // Note: always return success, because we want that writes succeed anyways.
+
+ return size;
+}
+
static void pvebackup_cleanup(void)
{
qemu_mutex_lock(&backup_state.backup_mutex);
@@ -3214,9 +3278,11 @@ static void pvebackup_cleanup(void)
backup_state.end_time = time(NULL);
- if (backup_state.vmaobj) {
- object_unparent(backup_state.vmaobj);
- backup_state.vmaobj = NULL;
+ if (backup_state.vmaw) {
+ Error *local_err = NULL;
+ vma_writer_close(backup_state.vmaw, &local_err);
+ error_propagate(&backup_state.error, local_err);
+ backup_state.vmaw = NULL;
}
g_list_free(backup_state.di_list);
@@ -3224,6 +3290,13 @@ static void pvebackup_cleanup(void)
qemu_mutex_unlock(&backup_state.backup_mutex);
}
+static void coroutine_fn backup_close_vma_stream(void *opaque)
+{
+ PVEBackupDevInfo *di = opaque;
+
+ vma_writer_close_stream(backup_state.vmaw, di->dev_id);
+}
+
static void pvebackup_complete_cb(void *opaque, int ret)
{
// This always runs in the main loop
@@ -3240,9 +3313,9 @@ static void pvebackup_complete_cb(void *opaque, int ret)
di->bs = NULL;
di->target = NULL;
- if (backup_state.vmaobj) {
- object_unparent(backup_state.vmaobj);
- backup_state.vmaobj = NULL;
+ if (backup_state.vmaw) {
+ Coroutine *co = qemu_coroutine_create(backup_close_vma_stream, di);
+ qemu_coroutine_enter(co);
}
// remove self from job queue
@@ -3270,14 +3343,9 @@ static void pvebackup_cancel(void *opaque)
error_setg(&backup_state.error, "backup cancelled");
}
- if (backup_state.vmaobj) {
- Error *err;
+ if (backup_state.vmaw) {
/* make sure vma writer does not block anymore */
- if (!object_set_props(backup_state.vmaobj, &err, "blocked", "yes", NULL)) {
- if (err) {
- error_report_err(err);
- }
- }
+ vma_writer_set_error(backup_state.vmaw, "backup cancelled");
}
GList *l = backup_state.di_list;
@@ -3308,18 +3376,14 @@ void qmp_backup_cancel(Error **errp)
Coroutine *co = qemu_coroutine_create(pvebackup_cancel, NULL);
qemu_coroutine_enter(co);
- while (backup_state.vmaobj) {
- /* FIXME: Find something better for this */
+ while (backup_state.vmaw) {
+ /* vma writer use main aio context */
aio_poll(qemu_get_aio_context(), true);
}
}
-void vma_object_add_config_file(Object *obj, const char *name,
- const char *contents, size_t len,
- Error **errp);
static int config_to_vma(const char *file, BackupFormat format,
- Object *vmaobj,
- const char *backup_dir,
+ const char *backup_dir, VmaWriter *vmaw,
Error **errp)
{
char *cdata = NULL;
@@ -3333,7 +3397,12 @@ static int config_to_vma(const char *file, BackupFormat format,
char *basename = g_path_get_basename(file);
if (format == BACKUP_FORMAT_VMA) {
- vma_object_add_config_file(vmaobj, basename, cdata, clen, errp);
+ if (vma_writer_add_config(vmaw, basename, cdata, clen) != 0) {
+ error_setg(errp, "unable to add %s config data to vma archive", file);
+ g_free(cdata);
+ g_free(basename);
+ return 1;
+ }
} else if (format == BACKUP_FORMAT_DIR) {
char config_path[PATH_MAX];
snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, basename);
@@ -3350,28 +3419,30 @@ static int config_to_vma(const char *file, BackupFormat format,
return 0;
}
+bool job_should_pause(Job *job);
static void pvebackup_run_next_job(void)
{
qemu_mutex_lock(&backup_state.backup_mutex);
- GList *next = g_list_nth(backup_state.di_list, backup_state.next_job);
- while (next) {
- PVEBackupDevInfo *di = (PVEBackupDevInfo *)next->data;
- backup_state.next_job++;
+ GList *l = backup_state.di_list;
+ while (l) {
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
if (!di->completed && di->bs && di->bs->job) {
BlockJob *job = di->bs->job;
AioContext *aio_context = blk_get_aio_context(job->blk);
aio_context_acquire(aio_context);
qemu_mutex_unlock(&backup_state.backup_mutex);
- if (backup_state.error || backup_state.cancel) {
- job_cancel_sync(job);
- } else {
- job_resume(job);
+ if (job_should_pause(&job->job)) {
+ if (backup_state.error || backup_state.cancel) {
+ job_cancel_sync(&job->job);
+ } else {
+ job_resume(&job->job);
+ }
}
aio_context_release(aio_context);
return;
}
- next = g_list_next(next);
}
qemu_mutex_unlock(&backup_state.backup_mutex);
@@ -3382,7 +3453,7 @@ static void pvebackup_run_next_job(void)
UuidInfo *qmp_backup(const char *backup_file, bool has_format,
BackupFormat format,
bool has_config_file, const char *config_file,
- bool has_firewall_file, const char *firewall_file,
+ bool has_firewall_file, const char *firewall_file,
bool has_devlist, const char *devlist,
bool has_speed, int64_t speed, Error **errp)
{
@@ -3390,7 +3461,8 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
BlockDriverState *bs = NULL;
const char *backup_dir = NULL;
Error *local_err = NULL;
- QemuUUID uuid;
+ uuid_t uuid;
+ VmaWriter *vmaw = NULL;
gchar **devs = NULL;
GList *di_list = NULL;
GList *l;
@@ -3402,7 +3474,7 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
backup_state.backup_mutex_initialized = true;
}
- if (backup_state.di_list || backup_state.vmaobj) {
+ if (backup_state.di_list) {
error_set(errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
return NULL;
@@ -3477,40 +3549,28 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
total += size;
}
- qemu_uuid_generate(&uuid);
+ uuid_generate(uuid);
if (format == BACKUP_FORMAT_VMA) {
- char uuidstr[UUID_FMT_LEN+1];
- qemu_uuid_unparse(&uuid, uuidstr);
- uuidstr[UUID_FMT_LEN] = 0;
- backup_state.vmaobj =
- object_new_with_props("vma", object_get_objects_root(),
- "vma-backup-obj", &local_err,
- "filename", backup_file,
- "uuid", uuidstr,
- NULL);
- if (!backup_state.vmaobj) {
+ vmaw = vma_writer_create(backup_file, uuid, &local_err);
+ if (!vmaw) {
if (local_err) {
error_propagate(errp, local_err);
}
goto err;
}
+ /* register all devices for vma writer */
l = di_list;
while (l) {
- QDict *options = qdict_new();
-
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
const char *devname = bdrv_get_device_name(di->bs);
- snprintf(di->targetfile, PATH_MAX, "vma-backup-obj/%s.raw", devname);
-
- qdict_put(options, "driver", qstring_from_str("vma-drive"));
- qdict_put(options, "size", qint_from_int(di->size));
- di->target = bdrv_open(di->targetfile, NULL, options, BDRV_O_RDWR, &local_err);
- if (!di->target) {
- error_propagate(errp, local_err);
+ di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
+ if (di->dev_id <= 0) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "register_stream failed");
goto err;
}
}
@@ -3551,14 +3611,14 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
/* add configuration file to archive */
if (has_config_file) {
- if(config_to_vma(config_file, format, backup_state.vmaobj, backup_dir, errp) != 0) {
+ if (config_to_vma(config_file, format, backup_dir, vmaw, errp) != 0) {
goto err;
}
}
/* add firewall file to archive */
if (has_firewall_file) {
- if(config_to_vma(firewall_file, format, backup_state.vmaobj, backup_dir, errp) != 0) {
+ if (config_to_vma(firewall_file, format, backup_dir, vmaw, errp) != 0) {
goto err;
}
}
@@ -3581,12 +3641,13 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
}
backup_state.backup_file = g_strdup(backup_file);
- memcpy(&backup_state.uuid, &uuid, sizeof(uuid));
- qemu_uuid_unparse(&uuid, backup_state.uuid_str);
+ backup_state.vmaw = vmaw;
+
+ uuid_copy(backup_state.uuid, uuid);
+ uuid_unparse_lower(uuid, backup_state.uuid_str);
qemu_mutex_lock(&backup_state.backup_mutex);
backup_state.di_list = di_list;
- backup_state.next_job = 0;
backup_state.total = total;
backup_state.transferred = 0;
@@ -3597,21 +3658,21 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
-
job = backup_job_create(NULL, di->bs, di->target, speed, MIRROR_SYNC_MODE_FULL, NULL,
false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
JOB_DEFAULT,
- pvebackup_complete_cb, di, 2, NULL, &local_err);
- if (di->target) {
- bdrv_unref(di->target);
- di->target = NULL;
- }
+ pvebackup_dump_cb, pvebackup_complete_cb, di,
+ 1, NULL, &local_err);
if (!job || local_err != NULL) {
error_setg(&backup_state.error, "backup_job_create failed");
pvebackup_cancel(NULL);
} else {
job_start(&job->job);
}
+ if (di->target) {
+ bdrv_unref(di->target);
+ di->target = NULL;
+ }
}
qemu_mutex_unlock(&backup_state.backup_mutex);
@@ -3647,9 +3708,10 @@ err:
g_strfreev(devs);
}
- if (backup_state.vmaobj) {
- object_unparent(backup_state.vmaobj);
- backup_state.vmaobj = NULL;
+ if (vmaw) {
+ Error *err = NULL;
+ vma_writer_close(vmaw, &err);
+ unlink(backup_file);
}
if (backup_dir) {
@@ -4110,7 +4172,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, 0, txn, &local_err);
+ job_flags, NULL, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
goto unref;
@@ -4215,7 +4277,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, 0, txn, &local_err);
+ job_flags, NULL, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
}
diff --git a/include/block/block_int.h b/include/block/block_int.h
index bb2dddca83..5a8b2e06c1 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -61,6 +61,9 @@
#define BLOCK_PROBE_BUF_SIZE 512
+typedef int BackupDumpFunc(void *opaque, BlockBackend *be,
+ uint64_t offset, uint64_t bytes, const void *buf);
+
enum BdrvTrackedRequestType {
BDRV_TRACKED_READ,
BDRV_TRACKED_WRITE,
@@ -1172,6 +1175,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
int creation_flags,
+ BackupDumpFunc *dump_cb,
BlockCompletionFunc *cb, void *opaque,
int pause_count,
JobTxn *txn, Error **errp);
diff --git a/job.c b/job.c
index 7a21e83780..9d27999678 100644
--- a/job.c
+++ b/job.c
@@ -248,7 +248,8 @@ static bool job_started(Job *job)
return job->co;
}
-static bool job_should_pause(Job *job)
+bool job_should_pause(Job *job);
+bool job_should_pause(Job *job)
{
return job->pause_count > 0;
}
diff --git a/vma-reader.c b/vma-reader.c
new file mode 100644
index 0000000000..2b1d1cdab3
@ -1580,7 +914,7 @@ index 0000000000..2b1d1cdab3
+
diff --git a/vma-writer.c b/vma-writer.c
new file mode 100644
index 0000000000..fd9567634d
index 0000000000..fe86b18a60
--- /dev/null
+++ b/vma-writer.c
@@ -0,0 +1,771 @@
@ -1785,12 +1119,14 @@ index 0000000000..fd9567634d
+ return n;
+}
+
+static void vma_co_continue_write(void *opaque)
+static void coroutine_fn yield_until_fd_writable(int fd)
+{
+ VmaWriter *vmaw = opaque;
+
+ DPRINTF("vma_co_continue_write\n");
+ qemu_coroutine_enter(vmaw->co_writer);
+ assert(qemu_in_coroutine());
+ AioContext *ctx = qemu_get_current_aio_context();
+ aio_set_fd_handler(ctx, fd, false, NULL, (IOHandler *)qemu_coroutine_enter,
+ NULL, qemu_coroutine_self());
+ qemu_coroutine_yield();
+ aio_set_fd_handler(ctx, fd, false, NULL, NULL, NULL, NULL);
+}
+
+static ssize_t coroutine_fn
@ -1810,14 +1146,12 @@ index 0000000000..fd9567634d
+ vmaw->co_writer = qemu_coroutine_self();
+
+ while (done < bytes) {
+ aio_set_fd_handler(qemu_get_aio_context(), vmaw->fd, false, NULL, vma_co_continue_write, NULL, vmaw);
+ qemu_coroutine_yield();
+ aio_set_fd_handler(qemu_get_aio_context(), vmaw->fd, false, NULL, NULL, NULL, NULL);
+ if (vmaw->status < 0) {
+ DPRINTF("vma_queue_write detected canceled backup\n");
+ done = -1;
+ break;
+ }
+ yield_until_fd_writable(vmaw->fd);
+ ret = write(vmaw->fd, buf + done, bytes - done);
+ if (ret > 0) {
+ done += ret;
@ -2291,9 +1625,7 @@ index 0000000000..fd9567634d
+
+ int i;
+
+ while (vmaw->co_writer) {
+ aio_poll(qemu_get_aio_context(), true);
+ }
+ qemu_co_mutex_lock(&vmaw->flush_lock); // wait for pending writes
+
+ assert(vmaw->co_writer == NULL);
+
@ -2334,6 +1666,8 @@ index 0000000000..fd9567634d
+ error_setg(errp, "%s", vmaw->errmsg);
+ }
+
+ qemu_co_mutex_unlock(&vmaw->flush_lock);
+
+ return vmaw->status;
+}
+
@ -2357,10 +1691,10 @@ index 0000000000..fd9567634d
+}
diff --git a/vma.c b/vma.c
new file mode 100644
index 0000000000..1b59fd1555
index 0000000000..a82752448a
--- /dev/null
+++ b/vma.c
@@ -0,0 +1,756 @@
@@ -0,0 +1,837 @@
+/*
+ * VMA: Virtual Machine Archive
+ *
@ -2379,9 +1713,11 @@ index 0000000000..1b59fd1555
+
+#include "vma.h"
+#include "qemu-common.h"
+#include "qemu/module.h"
+#include "qemu/error-report.h"
+#include "qemu/main-loop.h"
+#include "qapi/qmp/qstring.h"
+#include "qemu/cutils.h"
+#include "qapi/qmp/qdict.h"
+#include "sysemu/block-backend.h"
+
+static void help(void)
@ -2495,9 +1831,40 @@ index 0000000000..1b59fd1555
+ char *devname;
+ char *path;
+ char *format;
+ uint64_t throttling_bps;
+ char *throttling_group;
+ char *cache;
+ bool write_zero;
+} RestoreMap;
+
+static bool try_parse_option(char **line, const char *optname, char **out, const char *inbuf) {
+ size_t optlen = strlen(optname);
+ if (strncmp(*line, optname, optlen) != 0 || (*line)[optlen] != '=') {
+ return false;
+ }
+ if (*out) {
+ g_error("read map failed - duplicate value for option '%s'", optname);
+ }
+ char *value = (*line) + optlen + 1; /* including a '=' */
+ char *colon = strchr(value, ':');
+ if (!colon) {
+ g_error("read map failed - option '%s' not terminated ('%s')",
+ optname, inbuf);
+ }
+ *line = colon+1;
+ *out = g_strndup(value, colon - value);
+ return true;
+}
+
+static uint64_t verify_u64(const char *text) {
+ uint64_t value;
+ const char *endptr = NULL;
+ if (qemu_strtou64(text, &endptr, 0, &value) != 0 || !endptr || *endptr) {
+ g_error("read map failed - not a number: %s", text);
+ }
+ return value;
+}
+
+static int extract_content(int argc, char **argv)
+{
+ int c, ret = 0;
@ -2571,6 +1938,10 @@ index 0000000000..1b59fd1555
+ while (1) {
+ char inbuf[8192];
+ char *line = fgets(inbuf, sizeof(inbuf), map);
+ char *format = NULL;
+ char *bps = NULL;
+ char *group = NULL;
+ char *cache = NULL;
+ if (!line || line[0] == '\0' || !strcmp(line, "done\n")) {
+ break;
+ }
@ -2582,15 +1953,20 @@ index 0000000000..1b59fd1555
+ }
+ }
+
+ char *format = NULL;
+ if (strncmp(line, "format=", sizeof("format=")-1) == 0) {
+ format = line + sizeof("format=")-1;
+ char *colon = strchr(format, ':');
+ if (!colon) {
+ g_error("read map failed - found only a format ('%s')", inbuf);
+ while (1) {
+ if (!try_parse_option(&line, "format", &format, inbuf) &&
+ !try_parse_option(&line, "throttling.bps", &bps, inbuf) &&
+ !try_parse_option(&line, "throttling.group", &group, inbuf) &&
+ !try_parse_option(&line, "cache", &cache, inbuf))
+ {
+ break;
+ }
+ format = g_strndup(format, colon - format);
+ line = colon+1;
+ }
+
+ uint64_t bps_value = 0;
+ if (bps) {
+ bps_value = verify_u64(bps);
+ g_free(bps);
+ }
+
+ const char *path;
@ -2616,6 +1992,9 @@ index 0000000000..1b59fd1555
+ map->devname = g_strdup(devname);
+ map->path = g_strdup(path);
+ map->format = format;
+ map->throttling_bps = bps_value;
+ map->throttling_group = group;
+ map->cache = cache;
+ map->write_zero = write_zero;
+
+ g_hash_table_insert(devmap, map->devname, map);
@ -2643,7 +2022,10 @@ index 0000000000..1b59fd1555
+ } else if (di) {
+ char *devfn = NULL;
+ const char *format = NULL;
+ int flags = BDRV_O_RDWR | BDRV_O_NO_FLUSH;
+ uint64_t throttling_bps = 0;
+ const char *throttling_group = NULL;
+ const char *cache = NULL;
+ int flags = BDRV_O_RDWR;
+ bool write_zero = true;
+
+ if (readmap) {
@ -2654,6 +2036,9 @@ index 0000000000..1b59fd1555
+ }
+ devfn = map->path;
+ format = map->format;
+ throttling_bps = map->throttling_bps;
+ throttling_group = map->throttling_group;
+ cache = map->cache;
+ write_zero = map->write_zero;
+ } else {
+ devfn = g_strdup_printf("%s/tmp-disk-%s.raw",
@ -2675,10 +2060,11 @@ index 0000000000..1b59fd1555
+
+ size_t devlen = strlen(devfn);
+ QDict *options = NULL;
+ bool writethrough;
+ if (format) {
+ /* explicit format from commandline */
+ options = qdict_new();
+ qdict_put(options, "driver", qstring_from_str(format));
+ qdict_put_str(options, "driver", format);
+ } else if ((devlen > 4 && strcmp(devfn+devlen-4, ".raw") == 0) ||
+ strncmp(devfn, "/dev/", 5) == 0)
+ {
@ -2687,15 +2073,41 @@ index 0000000000..1b59fd1555
+ */
+ /* explicit raw format */
+ options = qdict_new();
+ qdict_put(options, "driver", qstring_from_str("raw"));
+ qdict_put_str(options, "driver", "raw");
+ }
+
+ if (cache && bdrv_parse_cache_mode(cache, &flags, &writethrough)) {
+ g_error("invalid cache option: %s\n", cache);
+ }
+
+ if (errp || !(blk = blk_new_open(devfn, NULL, options, flags, &errp))) {
+ g_error("can't open file %s - %s", devfn,
+ error_get_pretty(errp));
+ }
+
+ if (cache) {
+ blk_set_enable_write_cache(blk, !writethrough);
+ }
+
+ if (throttling_group) {
+ blk_io_limits_enable(blk, throttling_group);
+ }
+
+ if (throttling_bps) {
+ if (!throttling_group) {
+ blk_io_limits_enable(blk, devfn);
+ }
+
+ ThrottleConfig cfg;
+ throttle_config_init(&cfg);
+ cfg.buckets[THROTTLE_BPS_WRITE].avg = throttling_bps;
+ Error *err = NULL;
+ if (!throttle_is_valid(&cfg, &err)) {
+ error_report_err(err);
+ g_error("failed to apply throttling");
+ }
+ blk_set_io_limits(blk, &cfg);
+ }
+
+ if (vma_reader_register_bs(vmar, i, blk, write_zero, &errp) < 0) {
+ g_error("%s", error_get_pretty(errp));
+ }
@ -3086,13 +2498,16 @@ index 0000000000..1b59fd1555
+ const char *cmdname;
+ Error *main_loop_err = NULL;
+
+ error_set_progname(argv[0]);
+ error_init(argv[0]);
+ module_call_init(MODULE_INIT_TRACE);
+ qemu_init_exec_dir(argv[0]);
+
+ if (qemu_init_main_loop(&main_loop_err)) {
+ g_error("%s", error_get_pretty(main_loop_err));
+ }
+
+ bdrv_init();
+ module_call_init(MODULE_INIT_QOM);
+
+ if (argc < 2) {
+ help();

322
debian/patches/pve/0030-PVE-Backup-add-backup-dump-block-driver.patch vendored Normal file
View File

@ -0,0 +1,322 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 12:55:27 +0100
Subject: [PATCH 30/32] PVE-Backup: add backup-dump block driver
- add backup-dump block driver block/backup-dump.c
- move BackupBlockJob declaration from block/backup.c to include/block/block_int.h
- block/backup.c - backup-job-create: also consider source cluster size
- block/io.c - bdrv_do_drained_begin_quiesce: check for coroutine
- job.c: make job_should_pause non-static
---
block/Makefile.objs | 1 +
block/backup-dump.c | 169 ++++++++++++++++++++++++++++++++++++++
block/backup.c | 23 ++----
block/io.c | 8 +-
include/block/block_int.h | 30 +++++++
job.c | 3 +-
6 files changed, 214 insertions(+), 20 deletions(-)
create mode 100644 block/backup-dump.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index a10ceabf5b..5cd9e40d8d 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -33,6 +33,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
block-obj-$(CONFIG_GLUSTERFS) += gluster.o
block-obj-$(CONFIG_VXHS) += vxhs.o
block-obj-$(CONFIG_LIBSSH) += ssh.o
+block-obj-y += backup-dump.o
block-obj-y += accounting.o dirty-bitmap.o
block-obj-y += write-threshold.o
block-obj-y += backup.o
diff --git a/block/backup-dump.c b/block/backup-dump.c
new file mode 100644
index 0000000000..3066ab0698
--- /dev/null
+++ b/block/backup-dump.c
@@ -0,0 +1,169 @@
+/*
+ * BlockDriver to send backup data stream to a callback function
+ *
+ * Copyright (C) 2020 Proxmox Server Solutions GmbH
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qom/object_interfaces.h"
+#include "block/block_int.h"
+
+typedef struct {
+ int dump_cb_block_size;
+ uint64_t byte_size;
+ BackupDumpFunc *dump_cb;
+ void *dump_cb_data;
+} BDRVBackupDumpState;
+
+static int qemu_backup_dump_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
+{
+ BDRVBackupDumpState *s = bs->opaque;
+
+ bdi->cluster_size = s->dump_cb_block_size;
+ bdi->unallocated_blocks_are_zero = true;
+ return 0;
+}
+
+static int qemu_backup_dump_check_perm(
+ BlockDriverState *bs,
+ uint64_t perm,
+ uint64_t shared,
+ Error **errp)
+{
+ /* Nothing to do. */
+ return 0;
+}
+
+static void qemu_backup_dump_set_perm(
+ BlockDriverState *bs,
+ uint64_t perm,
+ uint64_t shared)
+{
+ /* Nothing to do. */
+}
+
+static void qemu_backup_dump_abort_perm_update(BlockDriverState *bs)
+{
+ /* Nothing to do. */
+}
+
+static void qemu_backup_dump_refresh_limits(BlockDriverState *bs, Error **errp)
+{
+ bs->bl.request_alignment = BDRV_SECTOR_SIZE; /* No sub-sector I/O */
+}
+
+static void qemu_backup_dump_close(BlockDriverState *bs)
+{
+ /* Nothing to do. */
+}
+
+static int64_t qemu_backup_dump_getlength(BlockDriverState *bs)
+{
+ BDRVBackupDumpState *s = bs->opaque;
+
+ return s->byte_size;
+}
+
+static coroutine_fn int qemu_backup_dump_co_writev(
+ BlockDriverState *bs,
+ int64_t sector_num,
+ int nb_sectors,
+ QEMUIOVector *qiov,
+ int flags)
+{
+ /* flags can be only values we set in supported_write_flags */
+ assert(flags == 0);
+
+ BDRVBackupDumpState *s = bs->opaque;
+ off_t offset = sector_num * BDRV_SECTOR_SIZE;
+
+ uint64_t written = 0;
+
+ for (int i = 0; i < qiov->niov; ++i) {
+ const struct iovec *v = &qiov->iov[i];
+
+ int rc = s->dump_cb(s->dump_cb_data, offset, v->iov_len, v->iov_base);
+ if (rc < 0) {
+ return rc;
+ }
+
+ if (rc != v->iov_len) {
+ return -EIO;
+ }
+
+ written += v->iov_len;
+ offset += v->iov_len;
+ }
+
+ return written;
+}
+
+static void qemu_backup_dump_child_perm(
+ BlockDriverState *bs,
+ BdrvChild *c,
+ const BdrvChildRole *role,
+ BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
+{
+ *nperm = BLK_PERM_ALL;
+ *nshared = BLK_PERM_ALL;
+}
+
+static BlockDriver bdrv_backup_dump_drive = {
+ .format_name = "backup-dump-drive",
+ .protocol_name = "backup-dump",
+ .instance_size = sizeof(BDRVBackupDumpState),
+
+ .bdrv_close = qemu_backup_dump_close,
+ .bdrv_has_zero_init = bdrv_has_zero_init_1,
+ .bdrv_getlength = qemu_backup_dump_getlength,
+ .bdrv_get_info = qemu_backup_dump_get_info,
+
+ .bdrv_co_writev = qemu_backup_dump_co_writev,
+
+ .bdrv_refresh_limits = qemu_backup_dump_refresh_limits,
+ .bdrv_check_perm = qemu_backup_dump_check_perm,
+ .bdrv_set_perm = qemu_backup_dump_set_perm,
+ .bdrv_abort_perm_update = qemu_backup_dump_abort_perm_update,
+ .bdrv_child_perm = qemu_backup_dump_child_perm,
+};
+
+static void bdrv_backup_dump_init(void)
+{
+ bdrv_register(&bdrv_backup_dump_drive);
+}
+
+block_init(bdrv_backup_dump_init);
+
+
+BlockDriverState *bdrv_backup_dump_create(
+ int dump_cb_block_size,
+ uint64_t byte_size,
+ BackupDumpFunc *dump_cb,
+ void *dump_cb_data,
+ Error **errp)
+{
+ BDRVBackupDumpState *state;
+ BlockDriverState *bs = bdrv_new_open_driver(
+ &bdrv_backup_dump_drive, NULL, BDRV_O_RDWR, errp);
+
+ if (!bs) {
+ return NULL;
+ }
+
+ bs->total_sectors = byte_size / BDRV_SECTOR_SIZE;
+ bs->opaque = state = g_new0(BDRVBackupDumpState, 1);
+
+ state->dump_cb_block_size = dump_cb_block_size;
+ state->byte_size = byte_size;
+ state->dump_cb = dump_cb;
+ state->dump_cb_data = dump_cb_data;
+
+ return bs;
+}
diff --git a/block/backup.c b/block/backup.c
index c155081de2..9d23da027f 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -32,24 +32,6 @@
#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
-typedef struct BackupBlockJob {
- BlockJob common;
- BlockDriverState *backup_top;
- BlockDriverState *source_bs;
-
- BdrvDirtyBitmap *sync_bitmap;
-
- MirrorSyncMode sync_mode;
- BitmapSyncMode bitmap_mode;
- BlockdevOnError on_source_error;
- BlockdevOnError on_target_error;
- uint64_t len;
- uint64_t bytes_read;
- int64_t cluster_size;
-
- BlockCopyState *bcs;
-} BackupBlockJob;
-
static const BlockJobDriver backup_job_driver;
static void backup_progress_bytes_callback(int64_t bytes, void *opaque)
@@ -420,6 +402,11 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
+ BlockDriverInfo bdi;
+ if (bdrv_get_info(bs, &bdi) == 0) {
+ cluster_size = MAX(cluster_size, bdi.cluster_size);
+ }
+
/*
* If source is in backing chain of target assume that target is going to be
* used for "image fleecing", i.e. it should represent a kind of snapshot of
diff --git a/block/io.c b/block/io.c
index f75777f5ea..75dea5ae33 100644
--- a/block/io.c
+++ b/block/io.c
@@ -381,7 +381,13 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs,
void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
BdrvChild *parent, bool ignore_bds_parents)
{
- assert(!qemu_in_coroutine());
+ // AFAICT this function is just an optimization, but sadly it doesn't play
+ // nice with the PVE backup code (when we're in a coroutine, even in
+ // pvebackup_co_start), so just call the full-blown drain begin instead
+ if (qemu_in_coroutine()) {
+ bdrv_do_drained_begin(bs, false, parent, ignore_bds_parents, false);
+ return;
+ }
/* Stop things in parent-to-child order */
if (atomic_fetch_inc(&bs->quiesce_counter) == 0) {
diff --git a/include/block/block_int.h b/include/block/block_int.h
index b0d5eb9485..105bffe0f7 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -60,6 +60,36 @@
#define BLOCK_PROBE_BUF_SIZE 512
+typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
+
+BlockDriverState *bdrv_backuo_dump_create(
+ int dump_cb_block_size,
+ uint64_t byte_size,
+ BackupDumpFunc *dump_cb,
+ void *dump_cb_data,
+ Error **errp);
+
+// Needs to be defined here, since it's used in blockdev.c to detect PVE backup
+// jobs with source_bs
+typedef struct BlockCopyState BlockCopyState;
+typedef struct BackupBlockJob {
+ BlockJob common;
+ BlockDriverState *backup_top;
+ BlockDriverState *source_bs;
+
+ BdrvDirtyBitmap *sync_bitmap;
+
+ MirrorSyncMode sync_mode;
+ BitmapSyncMode bitmap_mode;
+ BlockdevOnError on_source_error;
+ BlockdevOnError on_target_error;
+ uint64_t len;
+ uint64_t bytes_read;
+ int64_t cluster_size;
+
+ BlockCopyState *bcs;
+} BackupBlockJob;
+
enum BdrvTrackedRequestType {
BDRV_TRACKED_READ,
BDRV_TRACKED_WRITE,
diff --git a/job.c b/job.c
index 7554f735e3..b03ef989e2 100644
--- a/job.c
+++ b/job.c
@@ -248,7 +248,8 @@ static bool job_started(Job *job)
return job->co;
}
-static bool job_should_pause(Job *job)
+bool job_should_pause(Job *job);
+bool job_should_pause(Job *job)
{
return job->pause_count > 0;
}

1633
debian/patches/pve/0031-PVE-Backup-proxmox-backup-patches-for-qemu.patch vendored Normal file
View File

File diff suppressed because it is too large Load Diff

50
debian/patches/pve/0031-PVE-bug-fix-1071-vma-writer.c-use-correct-AioContext.patch vendored
View File

@ -1,50 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 21 Oct 2019 11:51:57 +0200
Subject: [PATCH] PVE bug fix #1071 - vma-writer.c: use correct AioContext
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
vma-writer.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/vma-writer.c b/vma-writer.c
index fd9567634d..b163fa2d3a 100644
--- a/vma-writer.c
+++ b/vma-writer.c
@@ -199,12 +199,14 @@ int vma_writer_register_stream(VmaWriter *vmaw, const char *devname,
return n;
}
-static void vma_co_continue_write(void *opaque)
+static void coroutine_fn yield_until_fd_writable(int fd)
{
- VmaWriter *vmaw = opaque;
-
- DPRINTF("vma_co_continue_write\n");
- qemu_coroutine_enter(vmaw->co_writer);
+ assert(qemu_in_coroutine());
+ AioContext *ctx = qemu_get_current_aio_context();
+ aio_set_fd_handler(ctx, fd, false, NULL, (IOHandler *)qemu_coroutine_enter,
+ NULL, qemu_coroutine_self());
+ qemu_coroutine_yield();
+ aio_set_fd_handler(ctx, fd, false, NULL, NULL, NULL, NULL);
}
static ssize_t coroutine_fn
@@ -224,14 +226,12 @@ vma_queue_write(VmaWriter *vmaw, const void *buf, size_t bytes)
vmaw->co_writer = qemu_coroutine_self();
while (done < bytes) {
- aio_set_fd_handler(qemu_get_aio_context(), vmaw->fd, false, NULL, vma_co_continue_write, NULL, vmaw);
- qemu_coroutine_yield();
- aio_set_fd_handler(qemu_get_aio_context(), vmaw->fd, false, NULL, NULL, NULL, NULL);
if (vmaw->status < 0) {
DPRINTF("vma_queue_write detected canceled backup\n");
done = -1;
break;
}
+ yield_until_fd_writable(vmaw->fd);
ret = write(vmaw->fd, buf + done, bytes - done);
if (ret > 0) {
done += ret;

248
debian/patches/pve/0032-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch vendored Normal file
View File

@ -0,0 +1,248 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 10 Mar 2020 12:55:29 +0100
Subject: [PATCH 32/32] PVE-Backup: pbs-restore - new command to restore from
proxmox backup server
---
Makefile | 4 +-
pbs-restore.c | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 211 insertions(+), 1 deletion(-)
create mode 100644 pbs-restore.c
diff --git a/Makefile b/Makefile
index 18d2dba2e4..469a1e3666 100644
--- a/Makefile
+++ b/Makefile
@@ -453,7 +453,7 @@ dummy := $(call unnest-vars,, \
include $(SRC_PATH)/tests/Makefile.include
-all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) pbs-restore$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
qemu-version.h: FORCE
$(call quiet-command, \
@@ -568,6 +568,8 @@ qemu-img$(EXESUF): qemu-img.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io
qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+pbs-restore$(EXESUF): pbs-restore.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+pbs-restore$(EXESUF): LIBS += -lproxmox_backup_qemu
qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
diff --git a/pbs-restore.c b/pbs-restore.c
new file mode 100644
index 0000000000..f65de8b890
--- /dev/null
+++ b/pbs-restore.c
@@ -0,0 +1,208 @@
+/*
+ * Qemu image restore helper for Proxmox Backup
+ *
+ * Copyright (C) 2019 Proxmox Server Solutions
+ *
+ * Authors:
+ * Dietmar Maurer (dietmar@proxmox.com)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include <glib.h>
+#include <getopt.h>
+#include <string.h>
+
+#include "qemu-common.h"
+#include "qemu/module.h"
+#include "qemu/error-report.h"
+#include "qemu/main-loop.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "sysemu/block-backend.h"
+
+#include <proxmox-backup-qemu.h>
+
+static void help(void)
+{
+ const char *help_msg =
+ "usage: pbs-restore [--repository <repo>] snapshot archive-name target [command options]\n"
+ ;
+
+ printf("%s", help_msg);
+ exit(1);
+}
+
+typedef struct CallbackData {
+ BlockBackend *target;
+ uint64_t last_offset;
+ bool skip_zero;
+} CallbackData;
+
+static int write_callback(
+ void *callback_data_ptr,
+ uint64_t offset,
+ const unsigned char *data,
+ uint64_t data_len)
+{
+ int res = -1;
+
+ CallbackData *callback_data = (CallbackData *)callback_data_ptr;
+
+ uint64_t last_offset = callback_data->last_offset;
+ if (offset > last_offset) callback_data->last_offset = offset;
+
+ if (data == NULL) {
+ if (callback_data->skip_zero && offset > last_offset) {
+ return 0;
+ }
+ res = blk_pwrite_zeroes(callback_data->target, offset, data_len, 0);
+ } else {
+ res = blk_pwrite(callback_data->target, offset, data, data_len, 0);
+ }
+
+ if (res < 0) {
+ fprintf(stderr, "blk_pwrite failed at offset %ld length %ld (%d) - %s\n", offset, data_len, res, strerror(-res));
+ return res;
+ }
+
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ Error *main_loop_err = NULL;
+ const char *format = "raw";
+ const char *repository = NULL;
+ const char *keyfile = NULL;
+ int verbose = false;
+ bool skip_zero = false;
+
+ error_init(argv[0]);
+
+ for(;;) {
+ static const struct option long_options[] = {
+ {"help", no_argument, 0, 'h'},
+ {"skip-zero", no_argument, 0, 'S'},
+ {"verbose", no_argument, 0, 'v'},
+ {"format", required_argument, 0, 'f'},
+ {"repository", required_argument, 0, 'r'},
+ {"keyfile", required_argument, 0, 'k'},
+ {0, 0, 0, 0}
+ };
+ int c = getopt_long(argc, argv, "hvf:r:k:", long_options, NULL);
+ if (c == -1) {
+ break;
+ }
+ switch(c) {
+ case ':':
+ fprintf(stderr, "missing argument for option '%s'", argv[optind - 1]);
+ return -1;
+ case '?':
+ fprintf(stderr, "unrecognized option '%s'", argv[optind - 1]);
+ return -1;
+ case 'f':
+ format = g_strdup(argv[optind - 1]);
+ break;
+ case 'r':
+ repository = g_strdup(argv[optind - 1]);
+ break;
+ case 'k':
+ keyfile = g_strdup(argv[optind - 1]);
+ break;
+ case 'v':
+ verbose = true;
+ break;
+ case 'S':
+ skip_zero = true;
+ break;
+ case 'h':
+ help();
+ return 0;
+ }
+ }
+
+ if (optind >= argc - 2) {
+ fprintf(stderr, "missing arguments\n");
+ help();
+ return -1;
+ }
+
+ if (repository == NULL) {
+ repository = getenv("PBS_REPOSITORY");
+ }
+
+ if (repository == NULL) {
+ fprintf(stderr, "no repository specified\n");
+ help();
+ return -1;
+ }
+
+ char *snapshot = argv[optind++];
+ char *archive_name = argv[optind++];
+ char *target = argv[optind++];
+
+ const char *password = getenv("PBS_PASSWORD");
+ const char *fingerprint = getenv("PBS_FINGERPRINT");
+ const char *key_password = getenv("PBS_ENCRYPTION_PASSWORD");
+
+ if (qemu_init_main_loop(&main_loop_err)) {
+ g_error("%s", error_get_pretty(main_loop_err));
+ }
+
+ bdrv_init();
+ module_call_init(MODULE_INIT_QOM);
+
+ char *pbs_error = NULL;
+ ProxmoxRestoreHandle *conn = proxmox_restore_connect(
+ repository, snapshot, password, keyfile, key_password, fingerprint, &pbs_error);
+ if (conn == NULL) {
+ fprintf(stderr, "restore failed: %s\n", pbs_error);
+ return -1;
+ }
+
+ QDict *options = qdict_new();
+ qdict_put_str(options, "driver", format);
+
+ if (format) {
+ qdict_put_str(options, "driver", format);
+ }
+
+ Error *local_err = NULL;
+ int flags = BDRV_O_RDWR;
+
+ BlockBackend *blk = blk_new_open(target, NULL, options, flags, &local_err);
+ if (!blk) {
+ fprintf(stderr, "%s\n", error_get_pretty(local_err));
+ return -1;
+ }
+
+ CallbackData *callback_data = calloc(sizeof( CallbackData), 1);
+
+ callback_data->target = blk;
+ callback_data->skip_zero = skip_zero;
+ callback_data->last_offset = 0;
+
+ // blk_set_enable_write_cache(blk, !writethrough);
+
+ int res = proxmox_restore_image(
+ conn,
+ archive_name,
+ write_callback,
+ callback_data,
+ &pbs_error,
+ verbose);
+
+ proxmox_restore_disconnect(conn);
+
+ if (res < 0) {
+ fprintf(stderr, "restore failed: %s\n", pbs_error);
+ return -1;
+ }
+
+ return 0;
+}

511
debian/patches/pve/0032-qmp_backup-run-backup-related-code-inside-coroutines.patch vendored
View File

@ -1,511 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 22 Oct 2019 12:48:17 +0200
Subject: [PATCH] qmp_backup: run backup related code inside coroutines
So that we can safely use coroutines/yield everywhere.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
blockdev.c | 250 ++++++++++++++++++++++++++++++++++++++---------------
1 file changed, 180 insertions(+), 70 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index cee7952bbb..cec0f770e8 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3173,6 +3173,34 @@ out:
/* PVE backup related function */
+typedef struct BlockOnCoroutineWrapper {
+ AioContext *ctx;
+ CoroutineEntry *entry;
+ void *entry_arg;
+ bool finished;
+} BlockOnCoroutineWrapper;
+
+static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
+{
+ BlockOnCoroutineWrapper *wrapper = opaque;
+ wrapper->entry(wrapper->entry_arg);
+ wrapper->finished = true;
+}
+
+static void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
+{
+ AioContext *ctx = qemu_get_current_aio_context();
+ BlockOnCoroutineWrapper wrapper = {
+ .finished = false,
+ .entry = entry,
+ .entry_arg = entry_arg,
+ .ctx = ctx,
+ };
+ Coroutine *wrapper_co = qemu_coroutine_create(block_on_coroutine_wrapper, &wrapper);
+ aio_co_enter(ctx, wrapper_co);
+ AIO_WAIT_WHILE(ctx, !wrapper.finished);
+}
+
static struct PVEBackupState {
Error *error;
bool cancel;
@@ -3200,12 +3228,14 @@ typedef struct PVEBackupDevInfo {
BlockDriverState *target;
} PVEBackupDevInfo;
-static void pvebackup_run_next_job(void);
+static void pvebackup_co_run_next_job(void);
-static int pvebackup_dump_cb(void *opaque, BlockBackend *target,
+static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
uint64_t start, uint64_t bytes,
const void *pbuf)
{
+ assert(qemu_in_coroutine());
+
const uint64_t size = bytes;
const unsigned char *buf = pbuf;
PVEBackupDevInfo *di = opaque;
@@ -3267,8 +3297,10 @@ static int pvebackup_dump_cb(void *opaque, BlockBackend *target,
return size;
}
-static void pvebackup_cleanup(void)
+static void coroutine_fn pvebackup_co_cleanup(void)
{
+ assert(qemu_in_coroutine());
+
qemu_mutex_lock(&backup_state.backup_mutex);
// Avoid race between block jobs and backup-cancel command:
if (!backup_state.vmaw) {
@@ -3290,18 +3322,19 @@ static void pvebackup_cleanup(void)
qemu_mutex_unlock(&backup_state.backup_mutex);
}
-static void coroutine_fn backup_close_vma_stream(void *opaque)
-{
- PVEBackupDevInfo *di = opaque;
+typedef struct PVEBackupCompeteCallbackData {
+ PVEBackupDevInfo *di;
+ int result;
+} PVEBackupCompeteCallbackData;
- vma_writer_close_stream(backup_state.vmaw, di->dev_id);
-}
-
-static void pvebackup_complete_cb(void *opaque, int ret)
+static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
{
- // This always runs in the main loop
+ assert(qemu_in_coroutine());
- PVEBackupDevInfo *di = opaque;
+ PVEBackupCompeteCallbackData *cb_data = opaque;
+
+ PVEBackupDevInfo *di = cb_data->di;
+ int ret = cb_data->result;
di->completed = true;
@@ -3314,8 +3347,7 @@ static void pvebackup_complete_cb(void *opaque, int ret)
di->target = NULL;
if (backup_state.vmaw) {
- Coroutine *co = qemu_coroutine_create(backup_close_vma_stream, di);
- qemu_coroutine_enter(co);
+ vma_writer_close_stream(backup_state.vmaw, di->dev_id);
}
// remove self from job queue
@@ -3325,12 +3357,25 @@ static void pvebackup_complete_cb(void *opaque, int ret)
qemu_mutex_unlock(&backup_state.backup_mutex);
if (!backup_state.cancel) {
- pvebackup_run_next_job();
+ pvebackup_co_run_next_job();
}
}
-static void pvebackup_cancel(void *opaque)
+static void pvebackup_complete_cb(void *opaque, int ret)
{
+ // This always called from the main loop
+ PVEBackupCompeteCallbackData cb_data = {
+ .di = opaque,
+ .result = ret,
+ };
+
+ block_on_coroutine_fn(pvebackup_co_complete_cb, &cb_data);
+}
+
+static void coroutine_fn pvebackup_co_cancel(void *opaque)
+{
+ assert(qemu_in_coroutine());
+
backup_state.cancel = true;
qemu_mutex_lock(&backup_state.backup_mutex);
// Avoid race between block jobs and backup-cancel command:
@@ -3365,21 +3410,16 @@ static void pvebackup_cancel(void *opaque)
}
}
- qemu_mutex_unlock(&backup_state.backup_mutex);
- pvebackup_cleanup();
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ pvebackup_co_cleanup();
}
void qmp_backup_cancel(Error **errp)
{
if (!backup_state.backup_mutex_initialized)
return;
- Coroutine *co = qemu_coroutine_create(pvebackup_cancel, NULL);
- qemu_coroutine_enter(co);
- while (backup_state.vmaw) {
- /* vma writer use main aio context */
- aio_poll(qemu_get_aio_context(), true);
- }
+ block_on_coroutine_fn(pvebackup_co_cancel, NULL);
}
static int config_to_vma(const char *file, BackupFormat format,
@@ -3420,8 +3460,11 @@ static int config_to_vma(const char *file, BackupFormat format,
}
bool job_should_pause(Job *job);
-static void pvebackup_run_next_job(void)
+
+static void coroutine_fn pvebackup_co_run_next_job(void)
{
+ assert(qemu_in_coroutine());
+
qemu_mutex_lock(&backup_state.backup_mutex);
GList *l = backup_state.di_list;
@@ -3447,16 +3490,33 @@ static void pvebackup_run_next_job(void)
qemu_mutex_unlock(&backup_state.backup_mutex);
// no more jobs, run the cleanup
- pvebackup_cleanup();
+ pvebackup_co_cleanup();
}
-UuidInfo *qmp_backup(const char *backup_file, bool has_format,
- BackupFormat format,
- bool has_config_file, const char *config_file,
- bool has_firewall_file, const char *firewall_file,
- bool has_devlist, const char *devlist,
- bool has_speed, int64_t speed, Error **errp)
+typedef struct QmpBackupTask {
+ const char *backup_file;
+ bool has_format;
+ BackupFormat format;
+ bool has_config_file;
+ const char *config_file;
+ bool has_firewall_file;
+ const char *firewall_file;
+ bool has_devlist;
+ const char *devlist;
+ bool has_speed;
+ int64_t speed;
+ Error **errp;
+ UuidInfo *result;
+} QmpBackupTask;
+
+static void coroutine_fn pvebackup_co_start(void *opaque)
{
+ assert(qemu_in_coroutine());
+
+ QmpBackupTask *task = opaque;
+
+ task->result = NULL; // just to be sure
+
BlockBackend *blk;
BlockDriverState *bs = NULL;
const char *backup_dir = NULL;
@@ -3475,16 +3535,16 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
}
if (backup_state.di_list) {
- error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
- return NULL;
+ return;
}
/* Todo: try to auto-detect format based on file name */
- format = has_format ? format : BACKUP_FORMAT_VMA;
+ BackupFormat format = task->has_format ? task->format : BACKUP_FORMAT_VMA;
- if (has_devlist) {
- devs = g_strsplit_set(devlist, ",;:", -1);
+ if (task->has_devlist) {
+ devs = g_strsplit_set(task->devlist, ",;:", -1);
gchar **d = devs;
while (d && *d) {
@@ -3492,18 +3552,18 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
if (blk) {
bs = blk_bs(blk);
if (bdrv_is_read_only(bs)) {
- error_setg(errp, "Node '%s' is read only", *d);
+ error_setg(task->errp, "Node '%s' is read only", *d);
goto err;
}
if (!bdrv_is_inserted(bs)) {
- error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
+ error_setg(task->errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
goto err;
}
PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
di->bs = bs;
di_list = g_list_append(di_list, di);
} else {
- error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+ error_set(task->errp, ERROR_CLASS_DEVICE_NOT_FOUND,
"Device '%s' not found", *d);
goto err;
}
@@ -3526,7 +3586,7 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
}
if (!di_list) {
- error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
goto err;
}
@@ -3536,13 +3596,13 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
+ if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, task->errp)) {
goto err;
}
ssize_t size = bdrv_getlength(di->bs);
if (size < 0) {
- error_setg_errno(errp, -di->size, "bdrv_getlength failed");
+ error_setg_errno(task->errp, -di->size, "bdrv_getlength failed");
goto err;
}
di->size = size;
@@ -3552,10 +3612,10 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
uuid_generate(uuid);
if (format == BACKUP_FORMAT_VMA) {
- vmaw = vma_writer_create(backup_file, uuid, &local_err);
+ vmaw = vma_writer_create(task->backup_file, uuid, &local_err);
if (!vmaw) {
if (local_err) {
- error_propagate(errp, local_err);
+ error_propagate(task->errp, local_err);
}
goto err;
}
@@ -3569,18 +3629,18 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
const char *devname = bdrv_get_device_name(di->bs);
di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
if (di->dev_id <= 0) {
- error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
"register_stream failed");
goto err;
}
}
} else if (format == BACKUP_FORMAT_DIR) {
- if (mkdir(backup_file, 0640) != 0) {
- error_setg_errno(errp, errno, "can't create directory '%s'\n",
- backup_file);
+ if (mkdir(task->backup_file, 0640) != 0) {
+ error_setg_errno(task->errp, errno, "can't create directory '%s'\n",
+ task->backup_file);
goto err;
}
- backup_dir = backup_file;
+ backup_dir = task->backup_file;
l = di_list;
while (l) {
@@ -3594,31 +3654,31 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
bdrv_img_create(di->targetfile, "raw", NULL, NULL, NULL,
di->size, flags, false, &local_err);
if (local_err) {
- error_propagate(errp, local_err);
+ error_propagate(task->errp, local_err);
goto err;
}
di->target = bdrv_open(di->targetfile, NULL, NULL, flags, &local_err);
if (!di->target) {
- error_propagate(errp, local_err);
+ error_propagate(task->errp, local_err);
goto err;
}
}
} else {
- error_set(errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
goto err;
}
/* add configuration file to archive */
- if (has_config_file) {
- if (config_to_vma(config_file, format, backup_dir, vmaw, errp) != 0) {
+ if (task->has_config_file) {
+ if (config_to_vma(task->config_file, format, backup_dir, vmaw, task->errp) != 0) {
goto err;
}
}
/* add firewall file to archive */
- if (has_firewall_file) {
- if (config_to_vma(firewall_file, format, backup_dir, vmaw, errp) != 0) {
+ if (task->has_firewall_file) {
+ if (config_to_vma(task->firewall_file, format, backup_dir, vmaw, task->errp) != 0) {
goto err;
}
}
@@ -3631,7 +3691,7 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
backup_state.error = NULL;
}
- backup_state.speed = (has_speed && speed > 0) ? speed : 0;
+ backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
backup_state.start_time = time(NULL);
backup_state.end_time = 0;
@@ -3639,7 +3699,7 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
if (backup_state.backup_file) {
g_free(backup_state.backup_file);
}
- backup_state.backup_file = g_strdup(backup_file);
+ backup_state.backup_file = g_strdup(task->backup_file);
backup_state.vmaw = vmaw;
@@ -3658,14 +3718,13 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- job = backup_job_create(NULL, di->bs, di->target, speed, MIRROR_SYNC_MODE_FULL, NULL,
+ job = backup_job_create(NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
- JOB_DEFAULT,
- pvebackup_dump_cb, pvebackup_complete_cb, di,
+ JOB_DEFAULT, pvebackup_co_dump_cb, pvebackup_complete_cb, di,
1, NULL, &local_err);
if (!job || local_err != NULL) {
error_setg(&backup_state.error, "backup_job_create failed");
- pvebackup_cancel(NULL);
+ pvebackup_co_cancel(NULL);
} else {
job_start(&job->job);
}
@@ -3678,13 +3737,14 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
qemu_mutex_unlock(&backup_state.backup_mutex);
if (!backup_state.error) {
- pvebackup_run_next_job(); // run one job
+ pvebackup_co_run_next_job(); // run one job
}
uuid_info = g_malloc0(sizeof(*uuid_info));
uuid_info->UUID = g_strdup(backup_state.uuid_str);
- return uuid_info;
+ task->result = uuid_info;
+ return;
err:
@@ -3711,23 +3771,61 @@ err:
if (vmaw) {
Error *err = NULL;
vma_writer_close(vmaw, &err);
- unlink(backup_file);
+ unlink(task->backup_file);
}
if (backup_dir) {
rmdir(backup_dir);
}
- return NULL;
+ task->result = NULL;
+ return;
}
-BackupStatus *qmp_query_backup(Error **errp)
+UuidInfo *qmp_backup(const char *backup_file, bool has_format,
+ BackupFormat format,
+ bool has_config_file, const char *config_file,
+ bool has_firewall_file, const char *firewall_file,
+ bool has_devlist, const char *devlist,
+ bool has_speed, int64_t speed, Error **errp)
+{
+ QmpBackupTask task = {
+ .backup_file = backup_file,
+ .has_format = has_format,
+ .format = format,
+ .has_config_file = has_config_file,
+ .config_file = config_file,
+ .has_firewall_file = has_firewall_file,
+ .firewall_file = firewall_file,
+ .has_devlist = has_devlist,
+ .devlist = devlist,
+ .has_speed = has_speed,
+ .speed = speed,
+ .errp = errp,
+ };
+
+ block_on_coroutine_fn(pvebackup_co_start, &task);
+ return task.result;
+}
+
+
+typedef struct QmpQueryBackupTask {
+ Error **errp;
+ BackupStatus *result;
+} QmpQueryBackupTask;
+
+static void coroutine_fn pvebackup_co_query(void *opaque)
{
+ assert(qemu_in_coroutine());
+
+ QmpQueryBackupTask *task = opaque;
+
BackupStatus *info = g_malloc0(sizeof(*info));
if (!backup_state.start_time) {
/* not started, return {} */
- return info;
+ task->result = info;
+ return;
}
info->has_status = true;
@@ -3763,7 +3861,19 @@ BackupStatus *qmp_query_backup(Error **errp)
info->has_transferred = true;
info->transferred = backup_state.transferred;
- return info;
+ task->result = info;
+}
+
+BackupStatus *qmp_query_backup(Error **errp)
+{
+ QmpQueryBackupTask task = {
+ .errp = errp,
+ .result = NULL,
+ };
+
+ block_on_coroutine_fn(pvebackup_co_query, &task);
+
+ return task.result;
}
void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,

274
debian/patches/pve/0033-qmp_backup-use-a-CoMutex-to-protect-access-to-backup.patch vendored
View File

@ -1,274 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 22 Oct 2019 12:48:18 +0200
Subject: [PATCH] qmp_backup: use a CoMutex to protect access to backup_state
And use job_cancel instead of job_cancel_sync (which hangs sometimes)
Also avoid calling pvebackup_co_cleanup if not necessary. If there are
running jobs, this is called from the block job completion callbacks
anyways.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
blockdev.c | 74 ++++++++++++++++++++++++++++++++++++++----------------
1 file changed, 52 insertions(+), 22 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index cec0f770e8..29196c18d8 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3215,7 +3215,7 @@ static struct PVEBackupState {
size_t total;
size_t transferred;
size_t zero_bytes;
- QemuMutex backup_mutex;
+ CoMutex backup_mutex;
bool backup_mutex_initialized;
} backup_state;
@@ -3240,7 +3240,10 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
const unsigned char *buf = pbuf;
PVEBackupDevInfo *di = opaque;
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
if (backup_state.cancel) {
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
return size; // return success
}
@@ -3251,6 +3254,7 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
"got unaligned write inside backup dump "
"callback (sector %ld)", start);
}
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
return -1; // not aligned to cluster size
}
@@ -3292,6 +3296,8 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
backup_state.transferred += size;
}
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+
// Note: always return success, because we want that writes succeed anyways.
return size;
@@ -3301,10 +3307,9 @@ static void coroutine_fn pvebackup_co_cleanup(void)
{
assert(qemu_in_coroutine());
- qemu_mutex_lock(&backup_state.backup_mutex);
- // Avoid race between block jobs and backup-cancel command:
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
if (!backup_state.vmaw) {
- qemu_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
return;
}
@@ -3319,7 +3324,7 @@ static void coroutine_fn pvebackup_co_cleanup(void)
g_list_free(backup_state.di_list);
backup_state.di_list = NULL;
- qemu_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
}
typedef struct PVEBackupCompeteCallbackData {
@@ -3333,6 +3338,8 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
PVEBackupCompeteCallbackData *cb_data = opaque;
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
PVEBackupDevInfo *di = cb_data->di;
int ret = cb_data->result;
@@ -3351,12 +3358,14 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
}
// remove self from job queue
- qemu_mutex_lock(&backup_state.backup_mutex);
backup_state.di_list = g_list_remove(backup_state.di_list, di);
g_free(di);
- qemu_mutex_unlock(&backup_state.backup_mutex);
- if (!backup_state.cancel) {
+ bool cancel = backup_state.cancel;
+
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+
+ if (!cancel) {
pvebackup_co_run_next_job();
}
}
@@ -3376,11 +3385,13 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
{
assert(qemu_in_coroutine());
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
backup_state.cancel = true;
- qemu_mutex_lock(&backup_state.backup_mutex);
+
// Avoid race between block jobs and backup-cancel command:
if (!backup_state.vmaw) {
- qemu_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
return;
}
@@ -3393,6 +3404,7 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
vma_writer_set_error(backup_state.vmaw, "backup cancelled");
}
+ bool running_jobs = 0;
GList *l = backup_state.di_list;
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
@@ -3403,6 +3415,7 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
AioContext *aio_context = blk_get_aio_context(job->blk);
aio_context_acquire(aio_context);
if (!di->completed) {
+ running_jobs += 1;
job_cancel(&job->job, false);
}
aio_context_release(aio_context);
@@ -3411,7 +3424,8 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
}
qemu_co_mutex_unlock(&backup_state.backup_mutex);
- pvebackup_co_cleanup();
+
+ if (running_jobs == 0) pvebackup_co_cleanup(); // else job will call completion handler
}
void qmp_backup_cancel(Error **errp)
@@ -3465,7 +3479,7 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
{
assert(qemu_in_coroutine());
- qemu_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
GList *l = backup_state.di_list;
while (l) {
@@ -3474,11 +3488,13 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
if (!di->completed && di->bs && di->bs->job) {
BlockJob *job = di->bs->job;
AioContext *aio_context = blk_get_aio_context(job->blk);
+ bool cancel_job = backup_state.error || backup_state.cancel;
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+
aio_context_acquire(aio_context);
- qemu_mutex_unlock(&backup_state.backup_mutex);
if (job_should_pause(&job->job)) {
- if (backup_state.error || backup_state.cancel) {
- job_cancel_sync(&job->job);
+ if (cancel_job) {
+ job_cancel(&job->job, false);
} else {
job_resume(&job->job);
}
@@ -3487,7 +3503,7 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
return;
}
}
- qemu_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
// no more jobs, run the cleanup
pvebackup_co_cleanup();
@@ -3530,11 +3546,14 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
BlockJob *job;
if (!backup_state.backup_mutex_initialized) {
- qemu_mutex_init(&backup_state.backup_mutex);
+ qemu_co_mutex_init(&backup_state.backup_mutex);
backup_state.backup_mutex_initialized = true;
}
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
if (backup_state.di_list) {
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
return;
@@ -3706,7 +3725,6 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
uuid_copy(backup_state.uuid, uuid);
uuid_unparse_lower(uuid, backup_state.uuid_str);
- qemu_mutex_lock(&backup_state.backup_mutex);
backup_state.di_list = di_list;
backup_state.total = total;
@@ -3724,20 +3742,21 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
1, NULL, &local_err);
if (!job || local_err != NULL) {
error_setg(&backup_state.error, "backup_job_create failed");
- pvebackup_co_cancel(NULL);
- } else {
- job_start(&job->job);
+ break;
}
+ job_start(&job->job);
if (di->target) {
bdrv_unref(di->target);
di->target = NULL;
}
}
- qemu_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
if (!backup_state.error) {
pvebackup_co_run_next_job(); // run one job
+ } else {
+ pvebackup_co_cancel(NULL);
}
uuid_info = g_malloc0(sizeof(*uuid_info));
@@ -3778,6 +3797,8 @@ err:
rmdir(backup_dir);
}
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+
task->result = NULL;
return;
}
@@ -3805,6 +3826,7 @@ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
};
block_on_coroutine_fn(pvebackup_co_start, &task);
+
return task.result;
}
@@ -3822,9 +3844,15 @@ static void coroutine_fn pvebackup_co_query(void *opaque)
BackupStatus *info = g_malloc0(sizeof(*info));
+ if (!backup_state.backup_mutex_initialized)
+ return;
+
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
if (!backup_state.start_time) {
/* not started, return {} */
task->result = info;
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
return;
}
@@ -3862,6 +3890,8 @@ static void coroutine_fn pvebackup_co_query(void *opaque)
info->transferred = backup_state.transferred;
task->result = info;
+
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
}
BackupStatus *qmp_query_backup(Error **errp)

35
debian/patches/pve/0034-vma_writer_close-avoid-call-to-aio_poll-acquire-flus.patch vendored
View File

@ -1,35 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Tue, 22 Oct 2019 12:48:19 +0200
Subject: [PATCH] vma_writer_close: avoid call to aio_poll (acquire flush_lock
instead)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
vma-writer.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/vma-writer.c b/vma-writer.c
index b163fa2d3a..fe86b18a60 100644
--- a/vma-writer.c
+++ b/vma-writer.c
@@ -705,9 +705,7 @@ int vma_writer_close(VmaWriter *vmaw, Error **errp)
int i;
- while (vmaw->co_writer) {
- aio_poll(qemu_get_aio_context(), true);
- }
+ qemu_co_mutex_lock(&vmaw->flush_lock); // wait for pending writes
assert(vmaw->co_writer == NULL);
@@ -748,6 +746,8 @@ int vma_writer_close(VmaWriter *vmaw, Error **errp)
error_setg(errp, "%s", vmaw->errmsg);
}
+ qemu_co_mutex_unlock(&vmaw->flush_lock);
+
return vmaw->status;
}

123
debian/patches/pve/0035-backup_job_create-pass-cluster-size-for-dump.patch vendored
View File

@ -1,123 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Thu, 24 Oct 2019 08:06:50 +0200
Subject: [PATCH] backup_job_create: pass cluster size for dump
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
block/backup.c | 8 +++++++-
block/replication.c | 2 +-
blockdev.c | 10 ++++++----
include/block/block_int.h | 4 ++++
4 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index fdcfb0529c..2398a4d602 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -566,6 +566,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BackupDumpFunc *dump_cb,
+ int dump_cb_block_size,
BlockCompletionFunc *cb, void *opaque,
int pause_count,
JobTxn *txn, Error **errp)
@@ -636,7 +637,12 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- cluster_size = backup_calculate_cluster_size(target ? target : bs, errp);
+ if (target) {
+ cluster_size = backup_calculate_cluster_size(target, errp);
+ } else {
+ cluster_size = dump_cb_block_size;
+ }
+
if (cluster_size < 0) {
goto error;
}
diff --git a/block/replication.c b/block/replication.c
index f060755713..b9465c3587 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -546,7 +546,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
0, MIRROR_SYNC_MODE_NONE, NULL, false,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
- NULL,
+ NULL, 0,
backup_job_completed, bs, 0, NULL, &local_err);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/blockdev.c b/blockdev.c
index 29196c18d8..a95beb823e 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3544,6 +3544,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
GList *l;
UuidInfo *uuid_info;
BlockJob *job;
+ int dump_cb_block_size = -1;
if (!backup_state.backup_mutex_initialized) {
qemu_co_mutex_init(&backup_state.backup_mutex);
@@ -3631,6 +3632,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
uuid_generate(uuid);
if (format == BACKUP_FORMAT_VMA) {
+ dump_cb_block_size = VMA_CLUSTER_SIZE;
vmaw = vma_writer_create(task->backup_file, uuid, &local_err);
if (!vmaw) {
if (local_err) {
@@ -3738,8 +3740,8 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
l = g_list_next(l);
job = backup_job_create(NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
- JOB_DEFAULT, pvebackup_co_dump_cb, pvebackup_complete_cb, di,
- 1, NULL, &local_err);
+ JOB_DEFAULT, pvebackup_co_dump_cb, dump_cb_block_size,
+ pvebackup_complete_cb, di, 1, NULL, &local_err);
if (!job || local_err != NULL) {
error_setg(&backup_state.error, "backup_job_create failed");
break;
@@ -4312,7 +4314,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, NULL, 0, txn, &local_err);
+ job_flags, NULL, 0, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
goto unref;
@@ -4417,7 +4419,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn,
job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
backup->sync, bmap, backup->compress,
backup->on_source_error, backup->on_target_error,
- job_flags, NULL, NULL, NULL, 0, txn, &local_err);
+ job_flags, NULL, 0, NULL, NULL, 0, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
}
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 5a8b2e06c1..e7ca823058 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1160,6 +1160,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
* @on_target_error: The action to take upon error writing to the target.
* @creation_flags: Flags that control the behavior of the Job lifetime.
* See @BlockJobCreateFlags
+ * @dump_cb: Callback for PVE backup code. Called for each data block when
+ * target is NULL.
+ * @dump_cb_block_size: The minimum block size expected by dump_cb.
* @cb: Completion function for the job.
* @opaque: Opaque pointer value passed to @cb.
* @txn: Transaction that this job is part of (may be NULL).
@@ -1176,6 +1179,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BackupDumpFunc *dump_cb,
+ int dump_cb_block_size,
BlockCompletionFunc *cb, void *opaque,
int pause_count,
JobTxn *txn, Error **errp);

31
debian/patches/pve/0036-avoid-calling-dump_cb-with-NULL-data-pointer-for-sma.patch vendored
View File

@ -1,31 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Thu, 24 Oct 2019 08:06:51 +0200
Subject: [PATCH] avoid calling dump_cb with NULL data pointer for small/last
cluster
The last block of a backup may be smaller than cluster_size.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
block/backup.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/block/backup.c b/block/backup.c
index 2398a4d602..c682c99f7a 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -131,7 +131,12 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
if (buffer_is_zero(*bounce_buffer, nbytes)) {
if (job->dump_cb) {
- ret = job->dump_cb(job->common.job.opaque, job->target, start, nbytes, NULL);
+ if (nbytes == job->cluster_size) {
+ // Note: pass NULL to indicate that we want to write [0u8; cluster_size]
+ ret = job->dump_cb(job->common.job.opaque, job->target, start, nbytes, NULL);
+ } else {
+ ret = job->dump_cb(job->common.job.opaque, job->target, start, nbytes, *bounce_buffer);
+ }
} else {
ret = blk_co_pwrite_zeroes(job->target, start,
nbytes, write_flags | BDRV_REQ_MAY_UNMAP);

109
debian/patches/pve/0037-rename-config_to_vma-into-pvebackup_co_add_config.patch vendored
View File

@ -1,109 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Thu, 24 Oct 2019 08:06:52 +0200
Subject: [PATCH] rename config_to_vma into pvebackup_co_add_config
- mark it with coroutine_fn
- add an additional parameter 'name'
- return -1 on error (instead of 1)
- code cleanup
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
blockdev.c | 40 ++++++++++++++++++++++++++--------------
1 file changed, 26 insertions(+), 14 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index a95beb823e..530b76c82f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3436,10 +3436,16 @@ void qmp_backup_cancel(Error **errp)
block_on_coroutine_fn(pvebackup_co_cancel, NULL);
}
-static int config_to_vma(const char *file, BackupFormat format,
- const char *backup_dir, VmaWriter *vmaw,
- Error **errp)
+static int coroutine_fn pvebackup_co_add_config(
+ const char *file,
+ const char *name,
+ BackupFormat format,
+ const char *backup_dir,
+ VmaWriter *vmaw,
+ Error **errp)
{
+ int res = 0;
+
char *cdata = NULL;
gsize clen = 0;
GError *err = NULL;
@@ -3449,28 +3455,30 @@ static int config_to_vma(const char *file, BackupFormat format,
}
char *basename = g_path_get_basename(file);
+ if (name == NULL) name = basename;
if (format == BACKUP_FORMAT_VMA) {
- if (vma_writer_add_config(vmaw, basename, cdata, clen) != 0) {
+ if (vma_writer_add_config(vmaw, name, cdata, clen) != 0) {
error_setg(errp, "unable to add %s config data to vma archive", file);
- g_free(cdata);
- g_free(basename);
- return 1;
+ goto err;
}
} else if (format == BACKUP_FORMAT_DIR) {
char config_path[PATH_MAX];
- snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, basename);
+ snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, name);
if (!g_file_set_contents(config_path, cdata, clen, &err)) {
error_setg(errp, "unable to write config file '%s'", config_path);
- g_free(cdata);
- g_free(basename);
- return 1;
+ goto err;
}
}
+ out:
g_free(basename);
g_free(cdata);
- return 0;
+ return res;
+
+ err:
+ res = -1;
+ goto out;
}
bool job_should_pause(Job *job);
@@ -3546,6 +3554,9 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
BlockJob *job;
int dump_cb_block_size = -1;
+ const char *config_name = "qemu-server.conf";
+ const char *firewall_name = "qemu-server.fw";
+
if (!backup_state.backup_mutex_initialized) {
qemu_co_mutex_init(&backup_state.backup_mutex);
backup_state.backup_mutex_initialized = true;
@@ -3690,16 +3701,17 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
goto err;
}
+
/* add configuration file to archive */
if (task->has_config_file) {
- if (config_to_vma(task->config_file, format, backup_dir, vmaw, task->errp) != 0) {
+ if (pvebackup_co_add_config(task->config_file, config_name, format, backup_dir, vmaw, task->errp) != 0) {
goto err;
}
}
/* add firewall file to archive */
if (task->has_firewall_file) {
- if (config_to_vma(task->firewall_file, format, backup_dir, vmaw, task->errp) != 0) {
+ if (pvebackup_co_add_config(task->firewall_file, firewall_name, format, backup_dir, vmaw, task->errp) != 0) {
goto err;
}
}

27
debian/patches/pve/0038-pvebackup_co_dump_cb-do-not-call-job-cancel.patch vendored
View File

@ -1,27 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Wed, 30 Oct 2019 12:15:44 +0100
Subject: [PATCH] pvebackup_co_dump_cb: do not call job->cancel()
The backup loop will automatically abort if we return an error.
---
blockdev.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 530b76c82f..568f71fdb4 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3274,10 +3274,8 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
if (!backup_state.error) {
vma_writer_error_propagate(backup_state.vmaw, &backup_state.error);
}
- if (di->bs && di->bs->job) {
- job_cancel(&di->bs->job->job, true);
- }
- break;
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ return ret;
} else {
backup_state.zero_bytes += zero_bytes;
if (remaining >= VMA_CLUSTER_SIZE) {

44
debian/patches/pve/0039-fix-backup-job-completion.patch vendored
View File

@ -1,44 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Wed, 30 Oct 2019 12:15:45 +0100
Subject: [PATCH] fix backup job completion
With recent changes, pvebackup_co_run_next_job cancels the job async,
so we need to run pvebackup_co_cleanup in the completion handler
instead. We call pvebackup_co_run_next as long as there are
jobs in the list.
---
blockdev.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 568f71fdb4..66f2711185 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3359,12 +3359,14 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
backup_state.di_list = g_list_remove(backup_state.di_list, di);
g_free(di);
- bool cancel = backup_state.cancel;
+ int pending_jobs = g_list_length(backup_state.di_list);
qemu_co_mutex_unlock(&backup_state.backup_mutex);
- if (!cancel) {
+ if (pending_jobs > 0) {
pvebackup_co_run_next_job();
+ } else {
+ pvebackup_co_cleanup();
}
}
@@ -3510,9 +3512,6 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
}
}
qemu_co_mutex_unlock(&backup_state.backup_mutex);
-
- // no more jobs, run the cleanup
- pvebackup_co_cleanup();
}
typedef struct QmpBackupTask {

43
debian/patches/pve/0040-pvebackup_complete_cb-avoid-poll-loop-if-already-ins.patch vendored
View File

@ -1,43 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Wed, 30 Oct 2019 12:15:46 +0100
Subject: [PATCH] pvebackup_complete_cb: avoid poll loop if already inside
coroutine
---
blockdev.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 66f2711185..083ada6c8e 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3189,6 +3189,8 @@ static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
static void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
{
+ assert(!qemu_in_coroutine());
+
AioContext *ctx = qemu_get_current_aio_context();
BlockOnCoroutineWrapper wrapper = {
.finished = false,
@@ -3372,13 +3374,17 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
static void pvebackup_complete_cb(void *opaque, int ret)
{
- // This always called from the main loop
+ // This can be called from the main loop, or from a coroutine
PVEBackupCompeteCallbackData cb_data = {
.di = opaque,
.result = ret,
};
- block_on_coroutine_fn(pvebackup_co_complete_cb, &cb_data);
+ if (qemu_in_coroutine()) {
+ pvebackup_co_complete_cb(&cb_data);
+ } else {
+ block_on_coroutine_fn(pvebackup_co_complete_cb, &cb_data);
+ }
}
static void coroutine_fn pvebackup_co_cancel(void *opaque)

28
debian/patches/pve/0041-PVE-backup-consider-source-cluster-size-as-well.patch vendored
View File

@ -1,28 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 4 Nov 2019 14:18:40 +0100
Subject: [PATCH] PVE: backup: consider source cluster size as well
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
block/backup.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index c682c99f7a..556367f8ba 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -642,11 +642,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- if (target) {
- cluster_size = backup_calculate_cluster_size(target, errp);
- } else {
- cluster_size = dump_cb_block_size;
- }
+ cluster_size = backup_calculate_cluster_size(target ? target : bs, errp);
+ cluster_size = MAX(cluster_size, dump_cb_block_size);
if (cluster_size < 0) {
goto error;

33
debian/patches/pve/0042-PVE-fixup-vma-tool.patch vendored
View File

@ -1,33 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Date: Mon, 26 Aug 2019 18:40:10 +0200
Subject: [PATCH] PVE: fixup: vma tool
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
vma.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/vma.c b/vma.c
index 3289fd722f..a82752448a 100644
--- a/vma.c
+++ b/vma.c
@@ -16,6 +16,7 @@
#include "vma.h"
#include "qemu-common.h"
+#include "qemu/module.h"
#include "qemu/error-report.h"
#include "qemu/main-loop.h"
#include "qemu/cutils.h"
@@ -800,7 +801,9 @@ int main(int argc, char **argv)
const char *cmdname;
Error *main_loop_err = NULL;
- error_set_progname(argv[0]);
+ error_init(argv[0]);
+ module_call_init(MODULE_INIT_TRACE);
+ qemu_init_exec_dir(argv[0]);
if (qemu_init_main_loop(&main_loop_err)) {
g_error("%s", error_get_pretty(main_loop_err));

81
debian/patches/pve/0043-PVE-fixup-blockdev-pvebackup-integration-fix-blockjo.patch vendored
View File

@ -1,81 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Date: Mon, 26 Aug 2019 18:40:44 +0200
Subject: [PATCH] PVE: fixup: blockdev pvebackup integration: fix blockjob
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
blockdev.c | 49 +++++++++++++++++++++++++++----------------------
1 file changed, 27 insertions(+), 22 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 083ada6c8e..46e8a2780a 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3416,15 +3416,17 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
if (!di->completed && di->bs) {
- BlockJob *job = di->bs->job;
- if (job) {
- AioContext *aio_context = blk_get_aio_context(job->blk);
- aio_context_acquire(aio_context);
- if (!di->completed) {
- running_jobs += 1;
- job_cancel(&job->job, false);
+ for (BlockJob *job = block_job_next(NULL); job; job = block_job_next(job)) {
+ if (block_job_has_bdrv(job, di->bs)) {
+ AioContext *aio_context = job->job.aio_context;
+ aio_context_acquire(aio_context);
+
+ if (!di->completed) {
+ running_jobs += 1;
+ job_cancel(&job->job, false);
+ }
+ aio_context_release(aio_context);
}
- aio_context_release(aio_context);
}
}
}
@@ -3499,22 +3501,25 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (!di->completed && di->bs && di->bs->job) {
- BlockJob *job = di->bs->job;
- AioContext *aio_context = blk_get_aio_context(job->blk);
- bool cancel_job = backup_state.error || backup_state.cancel;
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
-
- aio_context_acquire(aio_context);
- if (job_should_pause(&job->job)) {
- if (cancel_job) {
- job_cancel(&job->job, false);
- } else {
- job_resume(&job->job);
+ if (!di->completed && di->bs) {
+ for (BlockJob *job = block_job_next(NULL); job; job = block_job_next(job)) {
+ if (block_job_has_bdrv(job, di->bs)) {
+ AioContext *aio_context = job->job.aio_context;
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ aio_context_acquire(aio_context);
+
+
+ if (job_should_pause(&job->job)) {
+ if (backup_state.error || backup_state.cancel) {
+ job_cancel(&job->job, false);
+ } else {
+ job_resume(&job->job);
+ }
+ }
+ aio_context_release(aio_context);
+ return;
}
}
- aio_context_release(aio_context);
- return;
}
}
qemu_co_mutex_unlock(&backup_state.backup_mutex);

31
debian/patches/pve/0047-PVE-fix-hmp-info-backup-cmd-for-not-initialized-back.patch vendored
View File

@ -1,31 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Tim Marx <t.marx@proxmox.com>
Date: Wed, 22 Jan 2020 16:22:14 +0100
Subject: [PATCH] PVE: fix hmp info backup cmd for not initialized
backup_state.backup_mutex
Signed-off-by: Tim Marx <t.marx@proxmox.com>
---
monitor/hmp-cmds.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index bc9ca346f7..fb820410de 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -201,6 +201,12 @@ void hmp_info_backup(Monitor *mon, const QDict *qdict)
BackupStatus *info;
info = qmp_query_backup(NULL);
+
+ if (!info) {
+ monitor_printf(mon, "Backup status: not initialized\n");
+ return;
+ }
+
if (info->has_status) {
if (info->has_errmsg) {
monitor_printf(mon, "Backup status: %s - %s\n",
--
2.20.1

365
debian/patches/pve/0048-PVE-backup-use-separate-CoRwlock-for-data-accessed-b.patch vendored
View File

@ -1,365 +0,0 @@
From 4c2e4e498716b8a556fccec20fb1348555c2b189 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Fri, 14 Feb 2020 10:21:37 +0100
Subject: [PATCH qemu 1/7] PVE backup: use separate CoRwlock for data accessed
by qmp query_backup
This should minimize contention between query_backup and running backup jobs.
---
blockdev.c | 178 +++++++++++++++++++++++++++++++++--------------------
1 file changed, 112 insertions(+), 66 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 46e8a2780a..4367eaf2a7 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3203,22 +3203,26 @@ static void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
AIO_WAIT_WHILE(ctx, !wrapper.finished);
}
+
static struct PVEBackupState {
- Error *error;
+ struct {
+ CoRwlock rwlock;
+ Error *error;
+ time_t start_time;
+ time_t end_time;
+ char *backup_file;
+ uuid_t uuid;
+ char uuid_str[37];
+ size_t total;
+ size_t transferred;
+ size_t zero_bytes;
+ } stat;
bool cancel;
- uuid_t uuid;
- char uuid_str[37];
int64_t speed;
- time_t start_time;
- time_t end_time;
- char *backup_file;
VmaWriter *vmaw;
GList *di_list;
- size_t total;
- size_t transferred;
- size_t zero_bytes;
CoMutex backup_mutex;
- bool backup_mutex_initialized;
+ bool mutex_initialized;
} backup_state;
typedef struct PVEBackupDevInfo {
@@ -3251,11 +3255,14 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
- if (!backup_state.error) {
- error_setg(&backup_state.error,
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ if (!backup_state.stat.error) {
+ qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
+ error_setg(&backup_state.stat.error,
"got unaligned write inside backup dump "
"callback (sector %ld)", start);
}
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
qemu_co_mutex_unlock(&backup_state.backup_mutex);
return -1; // not aligned to cluster size
}
@@ -3273,27 +3280,35 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
buf += VMA_CLUSTER_SIZE;
}
if (ret < 0) {
- if (!backup_state.error) {
- vma_writer_error_propagate(backup_state.vmaw, &backup_state.error);
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ if (!backup_state.stat.error) {
+ qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
+ vma_writer_error_propagate(backup_state.vmaw, &backup_state.stat.error);
}
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+
qemu_co_mutex_unlock(&backup_state.backup_mutex);
return ret;
} else {
- backup_state.zero_bytes += zero_bytes;
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ backup_state.stat.zero_bytes += zero_bytes;
if (remaining >= VMA_CLUSTER_SIZE) {
- backup_state.transferred += VMA_CLUSTER_SIZE;
+ backup_state.stat.transferred += VMA_CLUSTER_SIZE;
remaining -= VMA_CLUSTER_SIZE;
} else {
- backup_state.transferred += remaining;
+ backup_state.stat.transferred += remaining;
remaining = 0;
}
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
}
}
} else {
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
if (!buf) {
- backup_state.zero_bytes += size;
+ backup_state.stat.zero_bytes += size;
}
- backup_state.transferred += size;
+ backup_state.stat.transferred += size;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
}
qemu_co_mutex_unlock(&backup_state.backup_mutex);
@@ -3313,12 +3328,20 @@ static void coroutine_fn pvebackup_co_cleanup(void)
return;
}
- backup_state.end_time = time(NULL);
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ backup_state.stat.end_time = time(NULL);
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
if (backup_state.vmaw) {
Error *local_err = NULL;
vma_writer_close(backup_state.vmaw, &local_err);
- error_propagate(&backup_state.error, local_err);
+
+ if (local_err != NULL) {
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ error_propagate(&backup_state.stat.error, local_err);
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ }
+
backup_state.vmaw = NULL;
}
@@ -3345,10 +3368,14 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
di->completed = true;
- if (ret < 0 && !backup_state.error) {
- error_setg(&backup_state.error, "job failed with err %d - %s",
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+
+ if (ret < 0 && !backup_state.stat.error) {
+ qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
+ error_setg(&backup_state.stat.error, "job failed with err %d - %s",
ret, strerror(-ret));
}
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
di->bs = NULL;
di->target = NULL;
@@ -3401,9 +3428,12 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
return;
}
- if (!backup_state.error) {
- error_setg(&backup_state.error, "backup cancelled");
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ if (!backup_state.stat.error) {
+ qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
+ error_setg(&backup_state.stat.error, "backup cancelled");
}
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
if (backup_state.vmaw) {
/* make sure vma writer does not block anymore */
@@ -3438,7 +3468,7 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
void qmp_backup_cancel(Error **errp)
{
- if (!backup_state.backup_mutex_initialized)
+ if (!backup_state.mutex_initialized)
return;
block_on_coroutine_fn(pvebackup_co_cancel, NULL);
@@ -3508,9 +3538,13 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
qemu_co_mutex_unlock(&backup_state.backup_mutex);
aio_context_acquire(aio_context);
-
+
if (job_should_pause(&job->job)) {
- if (backup_state.error || backup_state.cancel) {
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ bool error_or_canceled = backup_state.stat.error || backup_state.cancel;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+
+ if (error_or_canceled) {
job_cancel(&job->job, false);
} else {
job_resume(&job->job);
@@ -3565,9 +3599,10 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
const char *config_name = "qemu-server.conf";
const char *firewall_name = "qemu-server.fw";
- if (!backup_state.backup_mutex_initialized) {
+ if (!backup_state.mutex_initialized) {
+ qemu_co_rwlock_init(&backup_state.stat.rwlock);
qemu_co_mutex_init(&backup_state.backup_mutex);
- backup_state.backup_mutex_initialized = true;
+ backup_state.mutex_initialized = true;
}
qemu_co_mutex_lock(&backup_state.backup_mutex);
@@ -3727,31 +3762,36 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
backup_state.cancel = false;
- if (backup_state.error) {
- error_free(backup_state.error);
- backup_state.error = NULL;
- }
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
+ if (backup_state.stat.error) {
+ error_free(backup_state.stat.error);
+ backup_state.stat.error = NULL;
+ }
- backup_state.start_time = time(NULL);
- backup_state.end_time = 0;
+ backup_state.stat.start_time = time(NULL);
+ backup_state.stat.end_time = 0;
- if (backup_state.backup_file) {
- g_free(backup_state.backup_file);
+ if (backup_state.stat.backup_file) {
+ g_free(backup_state.stat.backup_file);
}
- backup_state.backup_file = g_strdup(task->backup_file);
+ backup_state.stat.backup_file = g_strdup(task->backup_file);
- backup_state.vmaw = vmaw;
+ uuid_copy(backup_state.stat.uuid, uuid);
+ uuid_unparse_lower(uuid, backup_state.stat.uuid_str);
+ char *uuid_str = g_strdup(backup_state.stat.uuid_str);
- uuid_copy(backup_state.uuid, uuid);
- uuid_unparse_lower(uuid, backup_state.uuid_str);
+ backup_state.stat.total = total;
+ backup_state.stat.transferred = 0;
+ backup_state.stat.zero_bytes = 0;
- backup_state.di_list = di_list;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+
+ backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
+
+ backup_state.vmaw = vmaw;
- backup_state.total = total;
- backup_state.transferred = 0;
- backup_state.zero_bytes = 0;
+ backup_state.di_list = di_list;
/* start all jobs (paused state) */
l = di_list;
@@ -3763,7 +3803,9 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
JOB_DEFAULT, pvebackup_co_dump_cb, dump_cb_block_size,
pvebackup_complete_cb, di, 1, NULL, &local_err);
if (!job || local_err != NULL) {
- error_setg(&backup_state.error, "backup_job_create failed");
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ error_setg(&backup_state.stat.error, "backup_job_create failed");
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
break;
}
job_start(&job->job);
@@ -3775,14 +3817,18 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
qemu_co_mutex_unlock(&backup_state.backup_mutex);
- if (!backup_state.error) {
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ bool no_errors = !backup_state.stat.error;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+
+ if (no_errors) {
pvebackup_co_run_next_job(); // run one job
} else {
pvebackup_co_cancel(NULL);
}
uuid_info = g_malloc0(sizeof(*uuid_info));
- uuid_info->UUID = g_strdup(backup_state.uuid_str);
+ uuid_info->UUID = uuid_str;
task->result = uuid_info;
return;
@@ -3866,54 +3912,54 @@ static void coroutine_fn pvebackup_co_query(void *opaque)
BackupStatus *info = g_malloc0(sizeof(*info));
- if (!backup_state.backup_mutex_initialized)
+ if (!backup_state.mutex_initialized)
return;
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- if (!backup_state.start_time) {
+ if (!backup_state.stat.start_time) {
/* not started, return {} */
task->result = info;
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
return;
}
info->has_status = true;
info->has_start_time = true;
- info->start_time = backup_state.start_time;
+ info->start_time = backup_state.stat.start_time;
- if (backup_state.backup_file) {
+ if (backup_state.stat.backup_file) {
info->has_backup_file = true;
- info->backup_file = g_strdup(backup_state.backup_file);
+ info->backup_file = g_strdup(backup_state.stat.backup_file);
}
info->has_uuid = true;
- info->uuid = g_strdup(backup_state.uuid_str);
+ info->uuid = g_strdup(backup_state.stat.uuid_str);
- if (backup_state.end_time) {
- if (backup_state.error) {
+ if (backup_state.stat.end_time) {
+ if (backup_state.stat.error) {
info->status = g_strdup("error");
info->has_errmsg = true;
- info->errmsg = g_strdup(error_get_pretty(backup_state.error));
+ info->errmsg = g_strdup(error_get_pretty(backup_state.stat.error));
} else {
info->status = g_strdup("done");
}
info->has_end_time = true;
- info->end_time = backup_state.end_time;
+ info->end_time = backup_state.stat.end_time;
} else {
info->status = g_strdup("active");
}
info->has_total = true;
- info->total = backup_state.total;
+ info->total = backup_state.stat.total;
info->has_zero_bytes = true;
- info->zero_bytes = backup_state.zero_bytes;
+ info->zero_bytes = backup_state.stat.zero_bytes;
info->has_transferred = true;
- info->transferred = backup_state.transferred;
+ info->transferred = backup_state.stat.transferred;
task->result = info;
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
}
BackupStatus *qmp_query_backup(Error **errp)
--
2.20.1

25
debian/patches/pve/0049-PVE-backup-block_on_coroutine_wrapper-call-aio_wait.patch vendored
View File

@ -1,25 +0,0 @@
From ca384caae9a0dbbbbab2174ec09935702ac7a067 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Fri, 14 Feb 2020 11:51:02 +0100
Subject: [PATCH qemu 2/7] PVE backup - block_on_coroutine_wrapper: call
aio_wait_kick() as described in block/aio-wait.h
---
blockdev.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/blockdev.c b/blockdev.c
index 4367eaf2a7..2f84f8832d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3185,6 +3185,7 @@ static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
BlockOnCoroutineWrapper *wrapper = opaque;
wrapper->entry(wrapper->entry_arg);
wrapper->finished = true;
+ aio_wait_kick();
}
static void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
--
2.20.1

95
debian/patches/pve/0050-PVE-backup-move-backup_state.cancel-to-backup_state.patch vendored
View File

@ -1,95 +0,0 @@
From c8856d5cbcf3d427cd02c9556f845486ab5362bc Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Fri, 14 Feb 2020 12:37:27 +0100
Subject: [PATCH qemu 3/7] PVE backup: move backup_state.cancel to
backup_state.stat.cancel
In order to avoid lock contention with qmp_backup_cancel.
---
blockdev.c | 25 +++++++++++++++----------
1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 2f84f8832d..a4b5b98224 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3204,9 +3204,9 @@ static void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
AIO_WAIT_WHILE(ctx, !wrapper.finished);
}
-
static struct PVEBackupState {
struct {
+ // Everithing accessed from qmp command, protected using rwlock
CoRwlock rwlock;
Error *error;
time_t start_time;
@@ -3217,8 +3217,8 @@ static struct PVEBackupState {
size_t total;
size_t transferred;
size_t zero_bytes;
+ bool cancel;
} stat;
- bool cancel;
int64_t speed;
VmaWriter *vmaw;
GList *di_list;
@@ -3247,13 +3247,16 @@ static int coroutine_fn pvebackup_co_dump_cb(void *opaque, BlockBackend *target,
const unsigned char *buf = pbuf;
PVEBackupDevInfo *di = opaque;
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ bool cancel = backup_state.stat.cancel;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- if (backup_state.cancel) {
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ if (cancel) {
return size; // return success
}
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
@@ -3419,9 +3422,11 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
{
assert(qemu_in_coroutine());
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ backup_state.stat.cancel = true;
+ qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- backup_state.cancel = true;
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
// Avoid race between block jobs and backup-cancel command:
if (!backup_state.vmaw) {
@@ -3542,7 +3547,7 @@ static void coroutine_fn pvebackup_co_run_next_job(void)
if (job_should_pause(&job->job)) {
qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool error_or_canceled = backup_state.stat.error || backup_state.cancel;
+ bool error_or_canceled = backup_state.stat.error || backup_state.stat.cancel;
qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
if (error_or_canceled) {
@@ -3761,10 +3766,10 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
}
/* initialize global backup_state now */
- backup_state.cancel = false;
-
qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ backup_state.stat.cancel = false;
+
if (backup_state.stat.error) {
error_free(backup_state.stat.error);
backup_state.stat.error = NULL;
--
2.20.1

49
debian/patches/series vendored
View File

@ -1,8 +1,3 @@
extra/0001-monitor-qmp-resume-monitor-when-clearing-its-queue.patch
extra/0002-virtio-blk-schedule-virtio_notify_config-to-run-on-m.patch
extra/0003-vnc-fix-memory-leak-when-vnc-disconnect.patch
extra/0004-util-add-slirp_fmt-helpers.patch
extra/0005-tcp_emu-fix-unsafe-snprintf-usages.patch
pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
@ -22,34 +17,16 @@ pve/0016-PVE-qapi-modify-spice-query.patch
pve/0017-PVE-internal-snapshot-async.patch
pve/0018-PVE-block-add-the-zeroinit-block-driver-filter.patch
pve/0019-PVE-backup-modify-job-api.patch
pve/0020-PVE-backup-introduce-vma-archive-format.patch
pve/0021-PVE-Deprecated-adding-old-vma-files.patch
pve/0022-PVE-vma-add-throttling-options-to-drive-mapping-fifo.patch
pve/0023-PVE-vma-add-cache-option-to-device-map.patch
pve/0024-PVE-vma-remove-forced-NO_FLUSH-option.patch
pve/0025-PVE-Add-dummy-id-command-line-parameter.patch
pve/0026-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
pve/0027-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
pve/0028-PVE-savevm-async-kick-AIO-wait-on-block-state-write.patch
pve/0029-PVE-move-snapshot-cleanup-into-bottom-half.patch
pve/0030-PVE-monitor-disable-oob-capability.patch
pve/0031-PVE-bug-fix-1071-vma-writer.c-use-correct-AioContext.patch
pve/0032-qmp_backup-run-backup-related-code-inside-coroutines.patch
pve/0033-qmp_backup-use-a-CoMutex-to-protect-access-to-backup.patch
pve/0034-vma_writer_close-avoid-call-to-aio_poll-acquire-flus.patch
pve/0035-backup_job_create-pass-cluster-size-for-dump.patch
pve/0036-avoid-calling-dump_cb-with-NULL-data-pointer-for-sma.patch
pve/0037-rename-config_to_vma-into-pvebackup_co_add_config.patch
pve/0038-pvebackup_co_dump_cb-do-not-call-job-cancel.patch
pve/0039-fix-backup-job-completion.patch
pve/0040-pvebackup_complete_cb-avoid-poll-loop-if-already-ins.patch
pve/0041-PVE-backup-consider-source-cluster-size-as-well.patch
pve/0042-PVE-fixup-vma-tool.patch
pve/0043-PVE-fixup-blockdev-pvebackup-integration-fix-blockjo.patch
pve/0044-Acquire-aio_context-before-calling-block_job_add_bdr.patch
pve/0045-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
pve/0046-PVE-Allow-version-code-in-machine-type.patch
pve/0047-PVE-fix-hmp-info-backup-cmd-for-not-initialized-back.patch
pve/0048-PVE-backup-use-separate-CoRwlock-for-data-accessed-b.patch
pve/0049-PVE-backup-block_on_coroutine_wrapper-call-aio_wait.patch
pve/0050-PVE-backup-move-backup_state.cancel-to-backup_state.patch
pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
pve/0023-PVE-savevm-async-kick-AIO-wait-on-block-state-write.patch
pve/0024-PVE-move-snapshot-cleanup-into-bottom-half.patch
pve/0025-PVE-monitor-disable-oob-capability.patch
pve/0026-PVE-Acquire-aio_context-before-calling-block_job_add.patch
pve/0027-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
pve/0028-PVE-Allow-version-code-in-machine-type.patch
pve/0029-PVE-Backup-add-vma-backup-format-code.patch
pve/0030-PVE-Backup-add-backup-dump-block-driver.patch
pve/0031-PVE-Backup-proxmox-backup-patches-for-qemu.patch
pve/0032-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch

1
debian/rules vendored
View File

@ -120,7 +120,6 @@ install: build
rm $(destdir)/usr/share/kvm/s390-netboot.img
rm $(destdir)/usr/share/kvm/qemu_vga.ndrv
rm $(destdir)/usr/share/kvm/slof.bin
rm $(destdir)/usr/share/kvm/spapr-rtas.bin
rm $(destdir)/usr/share/kvm/u-boot.e500
# remove Aplha files
rm $(destdir)/usr/share/kvm/palcode-clipper

2
qemu

@ -1 +1 @@
Subproject commit 99c5874a9b6c9f70aef285d6eff85d4f46de3c52
Subproject commit b0ca999a43a22b38158a222233d3f5881648bb4f