2020-03-10 17:12:50 +03:00
|
|
|
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
|
|
|
|
|
From: Dietmar Maurer <dietmar@proxmox.com>
|
2020-04-07 17:53:19 +03:00
|
|
|
Date: Mon, 6 Apr 2020 12:16:59 +0200
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
Subject: [PATCH] PVE-Backup: Proxmox backup patches for QEMU
|
|
|
|
|
MIME-Version: 1.0
|
|
|
|
|
Content-Type: text/plain; charset=UTF-8
|
|
|
|
|
Content-Transfer-Encoding: 8bit
|
|
|
|
|
|
|
|
|
|
For PBS, using dirty bitmaps is supported via QEMU's
|
|
|
|
|
MIRROR_SYNC_MODE_BITMAP. When the feature is used, the data-write
|
|
|
|
|
callback is only executed for any changed chunks, the PBS rust code
|
|
|
|
|
will reuse chunks from the previous index for everything it doesn't
|
|
|
|
|
receive if reuse_index is true. On error or cancellation, all dirty
|
|
|
|
|
bitmaps are removed to ensure consistency.
|
|
|
|
|
|
|
|
|
|
By using a JobTxn, we can sync dirty bitmaps only when *all* jobs were
|
|
|
|
|
successful - meaning we don't need to remove them when the backup
|
|
|
|
|
fails, since QEMU's BITMAP_SYNC_MODE_ON_SUCCESS will now handle that
|
|
|
|
|
for us. A sequential transaction is used, so drives will be backed up
|
|
|
|
|
one after the other.
|
|
|
|
|
|
|
|
|
|
The backup and backup-cancel QMP calls are coroutines. This has the
|
|
|
|
|
benefit that calls are asynchronous to the main loop, i.e. long
|
|
|
|
|
running operations like connecting to a PBS server will no longer hang
|
|
|
|
|
the VM.
|
|
|
|
|
|
|
|
|
|
backup_job_create() and job_cancel_sync() cannot be run from a
|
|
|
|
|
coroutine and requires an acuqired AioContext, so the job creation and
|
|
|
|
|
canceling are extracted as bottom halves and called from the
|
|
|
|
|
respective QMP coroutines.
|
|
|
|
|
|
|
|
|
|
To communicate the finishing state, a dedicated property is used for
|
|
|
|
|
query-backup: 'finishing'. A dedicated state is explicitly not used,
|
|
|
|
|
since that would break compatibility with older qemu-server versions.
|
|
|
|
|
|
|
|
|
|
The first call to job_cancel_sync() will cancel and free all jobs in
|
|
|
|
|
the transaction, but it is necessary to pick a job that is:
|
|
|
|
|
1. still referenced. For this, there is a job_ref directly after job
|
|
|
|
|
creation paired with a job_unref in cleanup paths.
|
|
|
|
|
2. not yet finalized. In job_cancel_bh(), the first job that's not
|
|
|
|
|
completed yet is used. This is not necessarily the first job in the
|
|
|
|
|
list, because pvebackup_co_complete_stream() might not yet have
|
|
|
|
|
removed a completed job when job_cancel_bh() runs. Why even bother
|
|
|
|
|
with the bottom half at all and not use job_cancel() in
|
|
|
|
|
qmp_backup_cancel() directly? The reason is that qmp_backup_cancel()
|
|
|
|
|
is a coroutine, so it will hang when reaching AIO_WAIT_WHILE() and
|
|
|
|
|
job_cancel() might end up calling that.
|
|
|
|
|
|
|
|
|
|
Regarding BackupPerf performance settings. For now, only the
|
|
|
|
|
max-workers setting is exposed, because:
|
|
|
|
|
1. use-copy-range would need to be implemented in backup-dump and the
|
|
|
|
|
feature was actually turned off by default in QEMU itself, because it
|
|
|
|
|
didn't provide the expected benefit, see commit 6a30f663d4 ("qapi:
|
|
|
|
|
backup: disable copy_range by default").
|
|
|
|
|
2. max-chunk: enforced to be at least the backup cluster size (4 MiB
|
|
|
|
|
for PBS) and otherwise maximum of source and target cluster size.
|
|
|
|
|
And block-copy has a maximum buffer size of 1 MiB, so setting a larger
|
|
|
|
|
max-chunk doesn't even have an effect. To make the setting sensibly
|
|
|
|
|
usable the check would need to be removed and optionally the
|
|
|
|
|
block-copy max buffer size would need to be bumped. I tried doing just
|
|
|
|
|
that, and tested different source/target combinations with different
|
|
|
|
|
max-chunk settings, but there were no noticable improvements over the
|
|
|
|
|
default "unlimited" (resulting in 1 MiB for block-copy).
|
2020-03-10 17:12:50 +03:00
|
|
|
|
2021-03-03 12:56:02 +03:00
|
|
|
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
[SR: Add dirty-bitmap tracking for incremental backups
|
|
|
|
|
Add query_proxmox_support and query-pbs-bitmap-info QMP calls
|
|
|
|
|
Use a transaction to synchronize job states
|
|
|
|
|
Co-routine and async-related improvements
|
|
|
|
|
Improve finishing backups/cleanups
|
|
|
|
|
Various other improvements]
|
2021-03-03 12:56:02 +03:00
|
|
|
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
[FG: add master key support]
|
|
|
|
|
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
|
2022-01-13 12:34:33 +03:00
|
|
|
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
[WB: add PBS namespace support]
|
|
|
|
|
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
|
2022-12-14 17:16:32 +03:00
|
|
|
[FE: add new force parameter to job_cancel_sync calls
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
adapt for new job lock mechanism replacing AioContext locks
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
adapt to QAPI changes
|
|
|
|
|
improve canceling
|
backup: trim heap after finishing
Reported in the community forum [0]. By default, there can be large
amounts of memory left assigned to the QEMU process after backup.
Likely because of fragmentation, it's necessary to explicitly call
malloc_trim() to tell glibc that it shouldn't keep all that memory
resident for the process.
QEMU itself already does a malloc_trim() in the RCU thread, but that
code path might not be reached (or not for a long time) under usual
operation. The value of 4 MiB for the argument was also copied from
there.
Example with the following configuration:
> agent: 1
> boot: order=scsi0
> cores: 4
> cpu: x86-64-v2-AES
> ide2: none,media=cdrom
> memory: 1024
> name: backup-mem
> net0: virtio=DA:58:18:26:59:9F,bridge=vmbr0,firewall=1
> numa: 0
> ostype: l26
> scsi0: rbd:base-107-disk-0/vm-106-disk-1,size=4302M
> scsihw: virtio-scsi-pci
> smbios1: uuid=b2d4511e-8d01-44f1-afd6-9581b30c24a6
> sockets: 2
> startup: order=2
> virtio0: lvmthin:vm-106-disk-1,iothread=1,size=1G
> virtio1: lvmthin:vm-106-disk-2,iothread=1,size=1G
> virtio2: lvmthin:vm-106-disk-3,iothread=1,size=1G
> vmgenid: 0a1d8751-5e02-449d-977e-c0160e900231
Before the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 370948 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 2114964 kB
After the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 398788 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 424356 kB
[0]: https://forum.proxmox.com/threads/131339/
Co-diagnosed-by: Friedrich Weber <f.weber@proxmox.com>
Co-diagnosed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-14 11:53:19 +03:00
|
|
|
allow passing max-workers setting
|
2023-09-06 11:45:11 +03:00
|
|
|
use malloc_trim after backup
|
|
|
|
|
create jobs in a drained section]
|
2022-12-14 17:16:32 +03:00
|
|
|
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
|
2020-03-10 17:12:50 +03:00
|
|
|
---
|
2023-10-17 15:10:09 +03:00
|
|
|
block/backup-dump.c | 10 +-
|
|
|
|
|
block/meson.build | 5 +
|
|
|
|
|
block/monitor/block-hmp-cmds.c | 39 ++
|
|
|
|
|
blockdev.c | 1 +
|
|
|
|
|
hmp-commands-info.hx | 14 +
|
|
|
|
|
hmp-commands.hx | 29 +
|
|
|
|
|
include/block/block_int-common.h | 2 +-
|
|
|
|
|
include/monitor/hmp.h | 3 +
|
|
|
|
|
meson.build | 1 +
|
|
|
|
|
monitor/hmp-cmds.c | 72 ++
|
|
|
|
|
proxmox-backup-client.c | 146 ++++
|
|
|
|
|
proxmox-backup-client.h | 60 ++
|
|
|
|
|
pve-backup.c | 1067 ++++++++++++++++++++++++++++++
|
|
|
|
|
qapi/block-core.json | 229 +++++++
|
|
|
|
|
qapi/common.json | 14 +
|
|
|
|
|
qapi/machine.json | 16 +-
|
|
|
|
|
16 files changed, 1690 insertions(+), 18 deletions(-)
|
2020-03-10 17:12:50 +03:00
|
|
|
create mode 100644 proxmox-backup-client.c
|
|
|
|
|
create mode 100644 proxmox-backup-client.h
|
|
|
|
|
create mode 100644 pve-backup.c
|
|
|
|
|
|
2023-10-17 15:10:09 +03:00
|
|
|
diff --git a/block/backup-dump.c b/block/backup-dump.c
|
|
|
|
|
index 232a094426..e46abf1070 100644
|
|
|
|
|
--- a/block/backup-dump.c
|
|
|
|
|
+++ b/block/backup-dump.c
|
|
|
|
|
@@ -9,6 +9,8 @@
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include "qemu/osdep.h"
|
|
|
|
|
+
|
|
|
|
|
+#include "qapi/qmp/qdict.h"
|
|
|
|
|
#include "qom/object_interfaces.h"
|
|
|
|
|
#include "block/block_int.h"
|
|
|
|
|
|
|
|
|
|
@@ -141,7 +143,7 @@ static void bdrv_backup_dump_init(void)
|
|
|
|
|
block_init(bdrv_backup_dump_init);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
-BlockDriverState *bdrv_backup_dump_create(
|
|
|
|
|
+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
|
|
|
|
|
int dump_cb_block_size,
|
|
|
|
|
uint64_t byte_size,
|
|
|
|
|
BackupDumpFunc *dump_cb,
|
|
|
|
|
@@ -149,9 +151,11 @@ BlockDriverState *bdrv_backup_dump_create(
|
|
|
|
|
Error **errp)
|
|
|
|
|
{
|
|
|
|
|
BDRVBackupDumpState *state;
|
|
|
|
|
- BlockDriverState *bs = bdrv_new_open_driver(
|
|
|
|
|
- &bdrv_backup_dump_drive, NULL, BDRV_O_RDWR, errp);
|
|
|
|
|
|
|
|
|
|
+ QDict *options = qdict_new();
|
|
|
|
|
+ qdict_put_str(options, "driver", "backup-dump-drive");
|
|
|
|
|
+
|
|
|
|
|
+ BlockDriverState *bs = bdrv_co_open(NULL, NULL, options, BDRV_O_RDWR, errp);
|
|
|
|
|
if (!bs) {
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/block/meson.build b/block/meson.build
|
2023-10-17 15:10:09 +03:00
|
|
|
index 6fde9f7dcd..6d468f89e5 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/block/meson.build
|
|
|
|
|
+++ b/block/meson.build
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -45,6 +45,11 @@ block_ss.add(files(
|
2021-05-27 13:43:32 +03:00
|
|
|
), zstd, zlib, gnutls)
|
2020-04-07 17:53:19 +03:00
|
|
|
|
2021-02-11 19:11:11 +03:00
|
|
|
block_ss.add(files('../vma-writer.c'), libuuid)
|
|
|
|
|
+block_ss.add(files(
|
|
|
|
|
+ '../proxmox-backup-client.c',
|
|
|
|
|
+ '../pve-backup.c',
|
|
|
|
|
+), libproxmox_backup_qemu)
|
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
|
2023-10-17 15:10:09 +03:00
|
|
|
system_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
|
|
|
|
|
system_ss.add(files('block-ram-registrar.c'))
|
2020-04-07 17:53:19 +03:00
|
|
|
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
|
2023-09-06 11:45:12 +03:00
|
|
|
index ca2599de44..6efe28cef5 100644
|
2020-04-07 17:53:19 +03:00
|
|
|
--- a/block/monitor/block-hmp-cmds.c
|
|
|
|
|
+++ b/block/monitor/block-hmp-cmds.c
|
2023-09-06 11:45:12 +03:00
|
|
|
@@ -1029,3 +1029,42 @@ void hmp_change_medium(Monitor *mon, const char *device, const char *target,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
qmp_blockdev_change_medium(device, NULL, target, arg, true, force,
|
|
|
|
|
!!read_only, read_only_mode, errp);
|
2020-04-07 17:53:19 +03:00
|
|
|
}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+void coroutine_fn hmp_backup_cancel(Monitor *mon, const QDict *qdict)
|
2020-04-07 17:53:19 +03:00
|
|
|
+{
|
|
|
|
|
+ Error *error = NULL;
|
|
|
|
|
+
|
|
|
|
|
+ qmp_backup_cancel(&error);
|
|
|
|
|
+
|
|
|
|
|
+ hmp_handle_error(mon, error);
|
|
|
|
|
+}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict)
|
2020-04-07 17:53:19 +03:00
|
|
|
+{
|
|
|
|
|
+ Error *error = NULL;
|
|
|
|
|
+
|
|
|
|
|
+ const char *backup_file = qdict_get_str(qdict, "backupfile");
|
|
|
|
|
+ const char *devlist = qdict_get_try_str(qdict, "devlist");
|
|
|
|
|
+ int64_t speed = qdict_get_try_int(qdict, "speed", 0);
|
|
|
|
|
+
|
|
|
|
|
+ qmp_backup(
|
|
|
|
|
+ backup_file,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, // PBS password
|
|
|
|
|
+ NULL, // PBS keyfile
|
|
|
|
|
+ NULL, // PBS key_password
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ NULL, // PBS master_keyfile
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, // PBS fingerprint
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ NULL, // PBS backup-ns
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, // PBS backup-id
|
2020-06-30 11:28:24 +03:00
|
|
|
+ false, 0, // PBS backup-time
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ false, false, // PBS use-dirty-bitmap
|
|
|
|
|
+ false, false, // PBS compress
|
|
|
|
|
+ false, false, // PBS encrypt
|
2023-09-06 11:45:12 +03:00
|
|
|
+ true, BACKUP_FORMAT_VMA,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, NULL,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ devlist, qdict_haskey(qdict, "speed"), speed,
|
|
|
|
|
+ false, 0, // BackupPerf max-workers
|
|
|
|
|
+ &error);
|
2020-04-07 17:53:19 +03:00
|
|
|
+
|
|
|
|
|
+ hmp_handle_error(mon, error);
|
|
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/blockdev.c b/blockdev.c
|
2023-12-11 16:28:39 +03:00
|
|
|
index cd5f205ad1..7793143d76 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/blockdev.c
|
|
|
|
|
+++ b/blockdev.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -37,6 +37,7 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
#include "block/blockjob.h"
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
#include "block/dirty-bitmap.h"
|
2020-03-10 17:12:50 +03:00
|
|
|
#include "block/qdict.h"
|
|
|
|
|
+#include "block/blockjob_int.h"
|
|
|
|
|
#include "block/throttle-groups.h"
|
|
|
|
|
#include "monitor/monitor.h"
|
|
|
|
|
#include "qemu/error-report.h"
|
|
|
|
|
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
|
2023-10-17 15:10:09 +03:00
|
|
|
index 10fdd822e0..15937793c1 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/hmp-commands-info.hx
|
|
|
|
|
+++ b/hmp-commands-info.hx
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -471,6 +471,20 @@ SRST
|
2021-10-11 14:55:34 +03:00
|
|
|
Show the current VM UUID.
|
2020-04-07 17:53:19 +03:00
|
|
|
ERST
|
|
|
|
|
|
2021-10-11 14:55:34 +03:00
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+ {
|
|
|
|
|
+ .name = "backup",
|
|
|
|
|
+ .args_type = "",
|
|
|
|
|
+ .params = "",
|
|
|
|
|
+ .help = "show backup status",
|
2020-04-07 17:53:19 +03:00
|
|
|
+ .cmd = hmp_info_backup,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ },
|
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
|
+ ``info backup``
|
|
|
|
|
+ Show backup status.
|
|
|
|
|
+ERST
|
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
#if defined(CONFIG_SLIRP)
|
2020-04-07 17:53:19 +03:00
|
|
|
{
|
|
|
|
|
.name = "usernet",
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/hmp-commands.hx b/hmp-commands.hx
|
2023-10-17 15:10:09 +03:00
|
|
|
index e352f86872..0c8b6725fb 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/hmp-commands.hx
|
|
|
|
|
+++ b/hmp-commands.hx
|
2023-09-06 11:45:12 +03:00
|
|
|
@@ -101,6 +101,35 @@ ERST
|
2020-04-07 17:53:19 +03:00
|
|
|
SRST
|
|
|
|
|
``block_stream``
|
|
|
|
|
Copy data from a backing file into a block device.
|
|
|
|
|
+ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ {
|
|
|
|
|
+ .name = "backup",
|
2023-09-06 11:45:12 +03:00
|
|
|
+ .args_type = "backupfile:s,speed:o?,devlist:s?",
|
|
|
|
|
+ .params = "backupfile [speed [devlist]]",
|
|
|
|
|
+ .help = "create a VM backup (VMA format).",
|
2020-03-10 17:12:50 +03:00
|
|
|
+ .cmd = hmp_backup,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ .coroutine = true,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ },
|
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
|
+``backup``
|
|
|
|
|
+ Create a VM backup.
|
|
|
|
|
+ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ {
|
|
|
|
|
+ .name = "backup_cancel",
|
|
|
|
|
+ .args_type = "",
|
|
|
|
|
+ .params = "",
|
|
|
|
|
+ .help = "cancel the current VM backup",
|
2020-04-07 17:53:19 +03:00
|
|
|
+ .cmd = hmp_backup_cancel,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ .coroutine = true,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ },
|
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
|
+``backup_cancel``
|
|
|
|
|
+ Cancel the current VM backup.
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
|
{
|
2023-10-17 15:10:09 +03:00
|
|
|
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
|
|
|
|
|
index 0f2e1817ad..0a0339eee4 100644
|
|
|
|
|
--- a/include/block/block_int-common.h
|
|
|
|
|
+++ b/include/block/block_int-common.h
|
|
|
|
|
@@ -63,7 +63,7 @@
|
|
|
|
|
|
|
|
|
|
typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
|
|
|
|
|
|
|
|
|
|
-BlockDriverState *bdrv_backup_dump_create(
|
|
|
|
|
+BlockDriverState *coroutine_fn bdrv_co_backup_dump_create(
|
|
|
|
|
int dump_cb_block_size,
|
|
|
|
|
uint64_t byte_size,
|
|
|
|
|
BackupDumpFunc *dump_cb,
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
|
2023-10-17 15:10:09 +03:00
|
|
|
index 7a7def7530..cba7afe70c 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/include/monitor/hmp.h
|
|
|
|
|
+++ b/include/monitor/hmp.h
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -32,6 +32,7 @@ void hmp_info_savevm(Monitor *mon, const QDict *qdict);
|
2021-05-27 13:43:32 +03:00
|
|
|
void hmp_info_migrate(Monitor *mon, const QDict *qdict);
|
2020-03-10 17:12:50 +03:00
|
|
|
void hmp_info_migrate_capabilities(Monitor *mon, const QDict *qdict);
|
|
|
|
|
void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict);
|
|
|
|
|
+void hmp_info_backup(Monitor *mon, const QDict *qdict);
|
|
|
|
|
void hmp_info_cpus(Monitor *mon, const QDict *qdict);
|
2020-04-07 17:53:19 +03:00
|
|
|
void hmp_info_vnc(Monitor *mon, const QDict *qdict);
|
|
|
|
|
void hmp_info_spice(Monitor *mon, const QDict *qdict);
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -84,6 +85,8 @@ void hmp_change_vnc(Monitor *mon, const char *device, const char *target,
|
|
|
|
|
void hmp_change_medium(Monitor *mon, const char *device, const char *target,
|
|
|
|
|
const char *arg, const char *read_only, bool force,
|
|
|
|
|
Error **errp);
|
2020-03-10 17:12:50 +03:00
|
|
|
+void hmp_backup(Monitor *mon, const QDict *qdict);
|
|
|
|
|
+void hmp_backup_cancel(Monitor *mon, const QDict *qdict);
|
2020-04-07 17:53:19 +03:00
|
|
|
void hmp_migrate(Monitor *mon, const QDict *qdict);
|
|
|
|
|
void hmp_device_add(Monitor *mon, const QDict *qdict);
|
|
|
|
|
void hmp_device_del(Monitor *mon, const QDict *qdict);
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/meson.build b/meson.build
|
2023-10-17 15:10:09 +03:00
|
|
|
index cd95530d3b..d53976d621 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/meson.build
|
|
|
|
|
+++ b/meson.build
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -1779,6 +1779,7 @@ endif
|
2021-02-11 19:11:11 +03:00
|
|
|
has_gettid = cc.has_function('gettid')
|
|
|
|
|
|
|
|
|
|
libuuid = cc.find_library('uuid', required: true)
|
|
|
|
|
+libproxmox_backup_qemu = cc.find_library('proxmox_backup_qemu', required: true)
|
|
|
|
|
|
2022-02-11 12:24:33 +03:00
|
|
|
# libselinux
|
|
|
|
|
selinux = dependency('libselinux',
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
|
2023-05-24 16:56:53 +03:00
|
|
|
index 91be698308..5b9c231a4c 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/monitor/hmp-cmds.c
|
|
|
|
|
+++ b/monitor/hmp-cmds.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -21,6 +21,7 @@
|
|
|
|
|
#include "qemu/help_option.h"
|
|
|
|
|
#include "monitor/monitor-internal.h"
|
|
|
|
|
#include "qapi/error.h"
|
|
|
|
|
+#include "qapi/qapi-commands-block-core.h"
|
|
|
|
|
#include "qapi/qapi-commands-control.h"
|
|
|
|
|
#include "qapi/qapi-commands-migration.h"
|
|
|
|
|
#include "qapi/qapi-commands-misc.h"
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
@@ -144,6 +145,77 @@ void hmp_sync_profile(Monitor *mon, const QDict *qdict)
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
}
|
2020-03-10 17:12:50 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
+void hmp_info_backup(Monitor *mon, const QDict *qdict)
|
|
|
|
|
+{
|
|
|
|
|
+ BackupStatus *info;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ PBSBitmapInfoList *bitmap_info;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ info = qmp_query_backup(NULL);
|
|
|
|
|
+
|
|
|
|
|
+ if (!info) {
|
|
|
|
|
+ monitor_printf(mon, "Backup status: not initialized\n");
|
|
|
|
|
+ return;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (info->status) {
|
|
|
|
|
+ if (info->errmsg) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ monitor_printf(mon, "Backup status: %s - %s\n",
|
|
|
|
|
+ info->status, info->errmsg);
|
|
|
|
|
+ } else {
|
|
|
|
|
+ monitor_printf(mon, "Backup status: %s\n", info->status);
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (info->backup_file) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ monitor_printf(mon, "Start time: %s", ctime(&info->start_time));
|
|
|
|
|
+ if (info->end_time) {
|
|
|
|
|
+ monitor_printf(mon, "End time: %s", ctime(&info->end_time));
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ monitor_printf(mon, "Backup file: %s\n", info->backup_file);
|
|
|
|
|
+ monitor_printf(mon, "Backup uuid: %s\n", info->uuid);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+ if (!(info->has_total && info->total)) {
|
|
|
|
|
+ // this should not happen normally
|
|
|
|
|
+ monitor_printf(mon, "Total size: %d\n", 0);
|
|
|
|
|
+ } else {
|
|
|
|
|
+ size_t total_or_dirty = info->total;
|
|
|
|
|
+ bitmap_info = qmp_query_pbs_bitmap_info(NULL);
|
|
|
|
|
+
|
|
|
|
|
+ while (bitmap_info) {
|
|
|
|
|
+ monitor_printf(mon, "Drive %s:\n",
|
|
|
|
|
+ bitmap_info->value->drive);
|
|
|
|
|
+ monitor_printf(mon, " bitmap action: %s\n",
|
|
|
|
|
+ PBSBitmapAction_str(bitmap_info->value->action));
|
|
|
|
|
+ monitor_printf(mon, " size: %zd\n",
|
|
|
|
|
+ bitmap_info->value->size);
|
|
|
|
|
+ monitor_printf(mon, " dirty: %zd\n",
|
|
|
|
|
+ bitmap_info->value->dirty);
|
|
|
|
|
+ bitmap_info = bitmap_info->next;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ qapi_free_PBSBitmapInfoList(bitmap_info);
|
|
|
|
|
+
|
|
|
|
|
+ int zero_per = (info->has_zero_bytes && info->zero_bytes) ?
|
|
|
|
|
+ (info->zero_bytes * 100)/info->total : 0;
|
|
|
|
|
+ monitor_printf(mon, "Total size: %zd\n", info->total);
|
|
|
|
|
+ int trans_per = (info->transferred * 100)/total_or_dirty;
|
|
|
|
|
+ monitor_printf(mon, "Transferred bytes: %zd (%d%%)\n",
|
|
|
|
|
+ info->transferred, trans_per);
|
|
|
|
|
+ monitor_printf(mon, "Zero bytes: %zd (%d%%)\n",
|
|
|
|
|
+ info->zero_bytes, zero_per);
|
|
|
|
|
+
|
|
|
|
|
+ if (info->has_reused) {
|
|
|
|
|
+ int reused_per = (info->reused * 100)/total_or_dirty;
|
|
|
|
|
+ monitor_printf(mon, "Reused bytes: %zd (%d%%)\n",
|
|
|
|
|
+ info->reused, reused_per);
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ qapi_free_BackupStatus(info);
|
|
|
|
|
+}
|
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
void hmp_exit_preconfig(Monitor *mon, const QDict *qdict)
|
2020-03-10 17:12:50 +03:00
|
|
|
{
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
Error *err = NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/proxmox-backup-client.c b/proxmox-backup-client.c
|
|
|
|
|
new file mode 100644
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
index 0000000000..0923037dec
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
|
+++ b/proxmox-backup-client.c
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
@@ -0,0 +1,146 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "proxmox-backup-client.h"
|
|
|
|
|
+#include "qemu/main-loop.h"
|
|
|
|
|
+#include "block/aio-wait.h"
|
|
|
|
|
+#include "qapi/error.h"
|
|
|
|
|
+
|
|
|
|
|
+/* Proxmox Backup Server client bindings using coroutines */
|
|
|
|
|
+
|
|
|
|
|
+// This is called from another thread, so we use aio_co_schedule()
|
|
|
|
|
+static void proxmox_backup_schedule_wake(void *data) {
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData *waker = (CoCtxData *)data;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ aio_co_schedule(waker->ctx, waker->co);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_connect(ProxmoxBackupHandle *pbs, Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_connect_async(pbs, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup connect failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_add_config(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ const char *name,
|
|
|
|
|
+ const uint8_t *data,
|
|
|
|
|
+ uint64_t size,
|
|
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_add_config_async(
|
|
|
|
|
+ pbs, name, data, size ,proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup add_config %s failed: %s", name, pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_register_image(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ const char *device_name,
|
|
|
|
|
+ uint64_t size,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ bool incremental,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_register_image_async(
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ pbs, device_name, size, incremental, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup register image failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_finish(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_finish_async(
|
|
|
|
|
+ pbs, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup finish failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_close_image(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_close_image_async(
|
|
|
|
|
+ pbs, dev_id, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup close image failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_write_data(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
|
+ const uint8_t *data,
|
|
|
|
|
+ uint64_t offset,
|
|
|
|
|
+ uint64_t size,
|
|
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_write_data_async(
|
|
|
|
|
+ pbs, dev_id, data, offset, size, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ if (errp) error_setg(errp, "backup write data failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
|
+ }
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+}
|
|
|
|
|
diff --git a/proxmox-backup-client.h b/proxmox-backup-client.h
|
|
|
|
|
new file mode 100644
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
index 0000000000..8cbf645b2c
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
|
+++ b/proxmox-backup-client.h
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
@@ -0,0 +1,60 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#ifndef PROXMOX_BACKUP_CLIENT_H
|
|
|
|
|
+#define PROXMOX_BACKUP_CLIENT_H
|
|
|
|
|
+
|
|
|
|
|
+#include "qemu/osdep.h"
|
|
|
|
|
+#include "qemu/coroutine.h"
|
|
|
|
|
+#include "proxmox-backup-qemu.h"
|
|
|
|
|
+
|
2020-10-29 16:10:36 +03:00
|
|
|
+typedef struct CoCtxData {
|
|
|
|
|
+ Coroutine *co;
|
|
|
|
|
+ AioContext *ctx;
|
|
|
|
|
+ void *data;
|
|
|
|
|
+} CoCtxData;
|
|
|
|
|
+
|
|
|
|
|
+// FIXME: Remove once coroutines are supported for QMP
|
2020-03-10 17:12:50 +03:00
|
|
|
+void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg);
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_connect(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_add_config(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ const char *name,
|
|
|
|
|
+ const uint8_t *data,
|
|
|
|
|
+ uint64_t size,
|
|
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_register_image(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ const char *device_name,
|
|
|
|
|
+ uint64_t size,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ bool incremental,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_finish(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_close_image(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+int coroutine_fn
|
|
|
|
|
+proxmox_backup_co_write_data(
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
|
+ const uint8_t *data,
|
|
|
|
|
+ uint64_t offset,
|
|
|
|
|
+ uint64_t size,
|
|
|
|
|
+ Error **errp);
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+#endif /* PROXMOX_BACKUP_CLIENT_H */
|
|
|
|
|
diff --git a/pve-backup.c b/pve-backup.c
|
|
|
|
|
new file mode 100644
|
2023-10-17 15:10:09 +03:00
|
|
|
index 0000000000..d84d807654
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
|
+++ b/pve-backup.c
|
2023-09-06 11:45:12 +03:00
|
|
|
@@ -0,0 +1,1067 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "proxmox-backup-client.h"
|
|
|
|
|
+#include "vma.h"
|
|
|
|
|
+
|
|
|
|
|
+#include "qemu/osdep.h"
|
|
|
|
|
+#include "qemu/module.h"
|
|
|
|
|
+#include "sysemu/block-backend.h"
|
|
|
|
|
+#include "sysemu/blockdev.h"
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+#include "block/block_int-global-state.h"
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "block/blockjob.h"
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+#include "block/dirty-bitmap.h"
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "qapi/qapi-commands-block.h"
|
|
|
|
|
+#include "qapi/qmp/qerror.h"
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+#include "qemu/cutils.h"
|
|
|
|
|
+
|
backup: trim heap after finishing
Reported in the community forum [0]. By default, there can be large
amounts of memory left assigned to the QEMU process after backup.
Likely because of fragmentation, it's necessary to explicitly call
malloc_trim() to tell glibc that it shouldn't keep all that memory
resident for the process.
QEMU itself already does a malloc_trim() in the RCU thread, but that
code path might not be reached (or not for a long time) under usual
operation. The value of 4 MiB for the argument was also copied from
there.
Example with the following configuration:
> agent: 1
> boot: order=scsi0
> cores: 4
> cpu: x86-64-v2-AES
> ide2: none,media=cdrom
> memory: 1024
> name: backup-mem
> net0: virtio=DA:58:18:26:59:9F,bridge=vmbr0,firewall=1
> numa: 0
> ostype: l26
> scsi0: rbd:base-107-disk-0/vm-106-disk-1,size=4302M
> scsihw: virtio-scsi-pci
> smbios1: uuid=b2d4511e-8d01-44f1-afd6-9581b30c24a6
> sockets: 2
> startup: order=2
> virtio0: lvmthin:vm-106-disk-1,iothread=1,size=1G
> virtio1: lvmthin:vm-106-disk-2,iothread=1,size=1G
> virtio2: lvmthin:vm-106-disk-3,iothread=1,size=1G
> vmgenid: 0a1d8751-5e02-449d-977e-c0160e900231
Before the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 370948 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 2114964 kB
After the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 398788 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 424356 kB
[0]: https://forum.proxmox.com/threads/131339/
Co-diagnosed-by: Friedrich Weber <f.weber@proxmox.com>
Co-diagnosed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-14 11:53:19 +03:00
|
|
|
+#if defined(CONFIG_MALLOC_TRIM)
|
|
|
|
|
+#include <malloc.h>
|
|
|
|
|
+#endif
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+#include <proxmox-backup-qemu.h>
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+/* PVE backup state and related function */
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+/*
|
|
|
|
|
+ * Note: A resume from a qemu_coroutine_yield can happen in a different thread,
|
|
|
|
|
+ * so you may not use normal mutexes within coroutines:
|
|
|
|
|
+ *
|
|
|
|
|
+ * ---bad-example---
|
|
|
|
|
+ * qemu_rec_mutex_lock(lock)
|
|
|
|
|
+ * ...
|
|
|
|
|
+ * qemu_coroutine_yield() // wait for something
|
|
|
|
|
+ * // we are now inside a different thread
|
|
|
|
|
+ * qemu_rec_mutex_unlock(lock) // Crash - wrong thread!!
|
|
|
|
|
+ * ---end-bad-example--
|
|
|
|
|
+ *
|
|
|
|
|
+ * ==> Always use CoMutext inside coroutines.
|
|
|
|
|
+ * ==> Never acquire/release AioContext withing coroutines (because that use QemuRecMutex)
|
|
|
|
|
+ *
|
|
|
|
|
+ */
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+const char *PBS_BITMAP_NAME = "pbs-incremental-dirty-bitmap";
|
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+static struct PVEBackupState {
|
|
|
|
|
+ struct {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ // Everything accessed from qmp_backup_query command is protected using
|
|
|
|
|
+ // this lock. Do NOT hold this lock for long times, as it is sometimes
|
|
|
|
|
+ // acquired from coroutines, and thus any wait time may block the guest.
|
2021-03-03 12:56:02 +03:00
|
|
|
+ QemuMutex lock;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *error;
|
|
|
|
|
+ time_t start_time;
|
|
|
|
|
+ time_t end_time;
|
|
|
|
|
+ char *backup_file;
|
|
|
|
|
+ uuid_t uuid;
|
|
|
|
|
+ char uuid_str[37];
|
|
|
|
|
+ size_t total;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ size_t dirty;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ size_t transferred;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ size_t reused;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ size_t zero_bytes;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ GList *bitmap_list;
|
|
|
|
|
+ bool finishing;
|
|
|
|
|
+ bool starting;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ } stat;
|
|
|
|
|
+ int64_t speed;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ BackupPerf perf;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ VmaWriter *vmaw;
|
|
|
|
|
+ ProxmoxBackupHandle *pbs;
|
|
|
|
|
+ GList *di_list;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ JobTxn *txn;
|
|
|
|
|
+ CoMutex backup_mutex;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ CoMutex dump_callback_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+} backup_state;
|
|
|
|
|
+
|
|
|
|
|
+static void pvebackup_init(void)
|
|
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_init(&backup_state.stat.lock);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_init(&backup_state.backup_mutex);
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_co_mutex_init(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// initialize PVEBackupState at startup
|
|
|
|
|
+opts_init(pvebackup_init);
|
|
|
|
|
+
|
|
|
|
|
+typedef struct PVEBackupDevInfo {
|
|
|
|
|
+ BlockDriverState *bs;
|
|
|
|
|
+ size_t size;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ uint64_t block_size;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ uint8_t dev_id;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ int completed_ret; // INT_MAX if not completed
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char targetfile[PATH_MAX];
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ BdrvDirtyBitmap *bitmap;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ BlockDriverState *target;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ BlockJob *job;
|
2020-03-10 17:12:50 +03:00
|
|
|
+} PVEBackupDevInfo;
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+static void pvebackup_propagate_error(Error *err)
|
|
|
|
|
+{
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ error_propagate(&backup_state.stat.error, err);
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+static bool pvebackup_error_or_canceled(void)
|
|
|
|
|
+{
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ bool error_or_canceled = !!backup_state.stat.error;
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+
|
|
|
|
|
+ return error_or_canceled;
|
|
|
|
|
+}
|
|
|
|
|
+
|
2023-09-08 11:49:07 +03:00
|
|
|
+static void pvebackup_add_transferred_bytes(size_t transferred, size_t zero_bytes, size_t reused)
|
2021-03-03 12:56:02 +03:00
|
|
|
+{
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ backup_state.stat.zero_bytes += zero_bytes;
|
|
|
|
|
+ backup_state.stat.transferred += transferred;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.stat.reused += reused;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// This may get called from multiple coroutines in multiple io-threads
|
|
|
|
|
+// Note1: this may get called after job_cancel()
|
2020-03-10 17:12:50 +03:00
|
|
|
+static int coroutine_fn
|
2021-03-03 12:56:02 +03:00
|
|
|
+pvebackup_co_dump_pbs_cb(
|
2020-03-10 17:12:50 +03:00
|
|
|
+ void *opaque,
|
|
|
|
|
+ uint64_t start,
|
|
|
|
|
+ uint64_t bytes,
|
|
|
|
|
+ const void *pbuf)
|
|
|
|
|
+{
|
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
|
+
|
|
|
|
|
+ const uint64_t size = bytes;
|
|
|
|
|
+ const unsigned char *buf = pbuf;
|
|
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(backup_state.pbs);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ assert(buf);
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ bool is_zero_block = size == di->block_size && buffer_is_zero(buf, size);
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ // avoid deadlock if job is cancelled
|
|
|
|
|
+ if (pvebackup_error_or_canceled()) {
|
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
|
+ return -1;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ uint64_t transferred = 0;
|
|
|
|
|
+ uint64_t reused = 0;
|
|
|
|
|
+ while (transferred < size) {
|
|
|
|
|
+ uint64_t left = size - transferred;
|
|
|
|
|
+ uint64_t to_transfer = left < di->block_size ? left : di->block_size;
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id,
|
|
|
|
|
+ is_zero_block ? NULL : buf + transferred, start + transferred,
|
|
|
|
|
+ to_transfer, &local_err);
|
|
|
|
|
+ transferred += to_transfer;
|
|
|
|
|
+
|
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
|
+ return pbs_res;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ reused += pbs_res == 0 ? to_transfer : 0;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
2023-09-08 11:49:07 +03:00
|
|
|
+ pvebackup_add_transferred_bytes(size, is_zero_block ? size : 0, reused);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ return size;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// This may get called from multiple coroutines in multiple io-threads
|
|
|
|
|
+static int coroutine_fn
|
|
|
|
|
+pvebackup_co_dump_vma_cb(
|
|
|
|
|
+ void *opaque,
|
|
|
|
|
+ uint64_t start,
|
|
|
|
|
+ uint64_t bytes,
|
|
|
|
|
+ const void *pbuf)
|
|
|
|
|
+{
|
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
|
+
|
|
|
|
|
+ const uint64_t size = bytes;
|
|
|
|
|
+ const unsigned char *buf = pbuf;
|
|
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ int ret = -1;
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(backup_state.vmaw);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ assert(buf);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ uint64_t remaining = size;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
|
|
|
|
|
+ if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *local_err = NULL;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ error_setg(&local_err,
|
|
|
|
|
+ "got unaligned write inside backup dump "
|
|
|
|
|
+ "callback (sector %ld)", start);
|
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ return -1; // not aligned to cluster size
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ while (remaining > 0) {
|
|
|
|
|
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
|
|
|
|
|
+ // avoid deadlock if job is cancelled
|
|
|
|
|
+ if (pvebackup_error_or_canceled()) {
|
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
|
+ return -1;
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ size_t zero_bytes = 0;
|
|
|
|
|
+ ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num, buf, &zero_bytes);
|
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ ++cluster_num;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ buf += VMA_CLUSTER_SIZE;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (ret < 0) {
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ vma_writer_error_propagate(backup_state.vmaw, &local_err);
|
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ return ret;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ } else {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (remaining >= VMA_CLUSTER_SIZE) {
|
|
|
|
|
+ assert(ret == VMA_CLUSTER_SIZE);
|
2023-09-08 11:49:07 +03:00
|
|
|
+ pvebackup_add_transferred_bytes(VMA_CLUSTER_SIZE, zero_bytes, 0);
|
2021-03-03 12:56:02 +03:00
|
|
|
+ remaining -= VMA_CLUSTER_SIZE;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ assert(ret == remaining);
|
2023-09-08 11:49:07 +03:00
|
|
|
+ pvebackup_add_transferred_bytes(remaining, zero_bytes, 0);
|
2021-03-03 12:56:02 +03:00
|
|
|
+ remaining = 0;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ return size;
|
|
|
|
|
+}
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+static void coroutine_fn pvebackup_co_cleanup(void)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.stat.finishing = true;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ vma_writer_close(backup_state.vmaw, &local_err);
|
|
|
|
|
+
|
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ backup_state.vmaw = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.pbs) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (!pvebackup_error_or_canceled()) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ proxmox_backup_co_finish(backup_state.pbs, &local_err);
|
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ proxmox_backup_disconnect(backup_state.pbs);
|
|
|
|
|
+ backup_state.pbs = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ g_list_free(backup_state.di_list);
|
|
|
|
|
+ backup_state.di_list = NULL;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ backup_state.stat.end_time = time(NULL);
|
|
|
|
|
+ backup_state.stat.finishing = false;
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
backup: trim heap after finishing
Reported in the community forum [0]. By default, there can be large
amounts of memory left assigned to the QEMU process after backup.
Likely because of fragmentation, it's necessary to explicitly call
malloc_trim() to tell glibc that it shouldn't keep all that memory
resident for the process.
QEMU itself already does a malloc_trim() in the RCU thread, but that
code path might not be reached (or not for a long time) under usual
operation. The value of 4 MiB for the argument was also copied from
there.
Example with the following configuration:
> agent: 1
> boot: order=scsi0
> cores: 4
> cpu: x86-64-v2-AES
> ide2: none,media=cdrom
> memory: 1024
> name: backup-mem
> net0: virtio=DA:58:18:26:59:9F,bridge=vmbr0,firewall=1
> numa: 0
> ostype: l26
> scsi0: rbd:base-107-disk-0/vm-106-disk-1,size=4302M
> scsihw: virtio-scsi-pci
> smbios1: uuid=b2d4511e-8d01-44f1-afd6-9581b30c24a6
> sockets: 2
> startup: order=2
> virtio0: lvmthin:vm-106-disk-1,iothread=1,size=1G
> virtio1: lvmthin:vm-106-disk-2,iothread=1,size=1G
> virtio2: lvmthin:vm-106-disk-3,iothread=1,size=1G
> vmgenid: 0a1d8751-5e02-449d-977e-c0160e900231
Before the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 370948 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 2114964 kB
After the change:
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 398788 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS: 424356 kB
[0]: https://forum.proxmox.com/threads/131339/
Co-diagnosed-by: Friedrich Weber <f.weber@proxmox.com>
Co-diagnosed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-14 11:53:19 +03:00
|
|
|
+
|
|
|
|
|
+#if defined(CONFIG_MALLOC_TRIM)
|
|
|
|
|
+ /*
|
|
|
|
|
+ * Try to reclaim memory for buffers (and, in case of PBS, Rust futures), etc.
|
|
|
|
|
+ * Won't happen by default if there is fragmentation.
|
|
|
|
|
+ */
|
|
|
|
|
+ malloc_trim(4 * 1024 * 1024);
|
|
|
|
|
+#endif
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+static void coroutine_fn pvebackup_co_complete_stream(void *opaque)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ int ret = di->completed_ret;
|
|
|
|
|
+
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ bool starting = backup_state.stat.starting;
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+ if (starting) {
|
|
|
|
|
+ /* in 'starting' state, no tasks have been run yet, meaning we can (and
|
|
|
|
|
+ * must) skip all cleanup, as we don't know what has and hasn't been
|
|
|
|
|
+ * initialized yet. */
|
|
|
|
|
+ return;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
|
|
|
|
|
+
|
|
|
|
|
+ if (ret < 0) {
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ error_setg(&local_err, "job failed with err %d - %s", ret, strerror(-ret));
|
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ di->bs = NULL;
|
|
|
|
|
+
|
|
|
|
|
+ assert(di->target == NULL);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ bool error_or_canceled = pvebackup_error_or_canceled();
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
|
+ vma_writer_close_stream(backup_state.vmaw, di->dev_id);
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.pbs && !error_or_canceled) {
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ proxmox_backup_co_close_image(backup_state.pbs, di->dev_id, &local_err);
|
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (di->job) {
|
|
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
|
+ job_unref_locked(&di->job->job);
|
|
|
|
|
+ di->job = NULL;
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ // remove self from job list
|
2021-03-03 12:56:02 +03:00
|
|
|
+ backup_state.di_list = g_list_remove(backup_state.di_list, di);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ g_free(di);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ /* call cleanup if we're the last job */
|
|
|
|
|
+ if (!g_list_first(backup_state.di_list)) {
|
|
|
|
|
+ pvebackup_co_cleanup();
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
|
2021-03-03 12:56:02 +03:00
|
|
|
+}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+static void pvebackup_complete_cb(void *opaque, int ret)
|
2021-03-03 12:56:02 +03:00
|
|
|
+{
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
|
|
|
|
+ di->completed_ret = ret;
|
|
|
|
|
+
|
|
|
|
|
+ /*
|
|
|
|
|
+ * Schedule stream cleanup in async coroutine. close_image and finish might
|
|
|
|
|
+ * take a while, so we can't block on them here. This way it also doesn't
|
|
|
|
|
+ * matter if we're already running in a coroutine or not.
|
|
|
|
|
+ * Note: di is a pointer to an entry in the global backup_state struct, so
|
|
|
|
|
+ * it stays valid.
|
|
|
|
|
+ */
|
|
|
|
|
+ Coroutine *co = qemu_coroutine_create(pvebackup_co_complete_stream, di);
|
|
|
|
|
+ aio_co_enter(qemu_get_aio_context(), co);
|
|
|
|
|
+}
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+/*
|
|
|
|
|
+ * job_cancel(_sync) does not like to be called from coroutines, so defer to
|
|
|
|
|
+ * main loop processing via a bottom half. Assumes that caller holds
|
|
|
|
|
+ * backup_mutex.
|
|
|
|
|
+ */
|
|
|
|
|
+static void job_cancel_bh(void *opaque) {
|
|
|
|
|
+ CoCtxData *data = (CoCtxData*)opaque;
|
|
|
|
|
+
|
|
|
|
|
+ /*
|
|
|
|
|
+ * Be careful to pick a valid job to cancel:
|
|
|
|
|
+ * 1. job_cancel_sync() does not expect the job to be finalized already.
|
|
|
|
|
+ * 2. job_exit() might run between scheduling and running job_cancel_bh()
|
|
|
|
|
+ * and pvebackup_co_complete_stream() might not have removed the job from
|
|
|
|
|
+ * the list yet (in fact, cannot, because it waits for the backup_mutex).
|
|
|
|
|
+ * Requiring !job_is_completed() ensures that no finalized job is picked.
|
|
|
|
|
+ */
|
|
|
|
|
+ GList *bdi = g_list_first(backup_state.di_list);
|
|
|
|
|
+ while (bdi) {
|
|
|
|
|
+ if (bdi->data) {
|
|
|
|
|
+ BlockJob *bj = ((PVEBackupDevInfo *)bdi->data)->job;
|
|
|
|
|
+ if (bj) {
|
|
|
|
|
+ Job *job = &bj->job;
|
|
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
|
+ if (!job_is_completed_locked(job)) {
|
|
|
|
|
+ job_cancel_sync_locked(job, true);
|
|
|
|
|
+ /*
|
|
|
|
|
+ * It's enough to cancel one job in the transaction, the
|
|
|
|
|
+ * rest will follow automatically.
|
|
|
|
|
+ */
|
|
|
|
|
+ break;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ bdi = g_list_next(bdi);
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ aio_co_enter(data->ctx, data->co);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void coroutine_fn qmp_backup_cancel(Error **errp)
|
|
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ Error *cancel_err = NULL;
|
|
|
|
|
+ error_setg(&cancel_err, "backup canceled");
|
|
|
|
|
+ pvebackup_propagate_error(cancel_err);
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
|
+ /* make sure vma writer does not block anymore */
|
2021-03-03 12:56:02 +03:00
|
|
|
+ vma_writer_set_error(backup_state.vmaw, "backup canceled");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.pbs) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ proxmox_backup_abort(backup_state.pbs, "backup canceled");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ CoCtxData data = {
|
|
|
|
|
+ .ctx = qemu_get_current_aio_context(),
|
|
|
|
|
+ .co = qemu_coroutine_self(),
|
|
|
|
|
+ };
|
|
|
|
|
+ aio_bh_schedule_oneshot(data.ctx, job_cancel_bh, &data);
|
|
|
|
|
+ qemu_coroutine_yield();
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
2020-03-10 17:12:50 +03:00
|
|
|
+static int coroutine_fn pvebackup_co_add_config(
|
|
|
|
|
+ const char *file,
|
|
|
|
|
+ const char *name,
|
|
|
|
|
+ BackupFormat format,
|
|
|
|
|
+ VmaWriter *vmaw,
|
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
|
+ Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ int res = 0;
|
|
|
|
|
+
|
|
|
|
|
+ char *cdata = NULL;
|
|
|
|
|
+ gsize clen = 0;
|
|
|
|
|
+ GError *err = NULL;
|
|
|
|
|
+ if (!g_file_get_contents(file, &cdata, &clen, &err)) {
|
|
|
|
|
+ error_setg(errp, "unable to read file '%s'", file);
|
|
|
|
|
+ return 1;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ char *basename = g_path_get_basename(file);
|
|
|
|
|
+ if (name == NULL) name = basename;
|
|
|
|
|
+
|
|
|
|
|
+ if (format == BACKUP_FORMAT_VMA) {
|
|
|
|
|
+ if (vma_writer_add_config(vmaw, name, cdata, clen) != 0) {
|
|
|
|
|
+ error_setg(errp, "unable to add %s config data to vma archive", file);
|
|
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+ } else if (format == BACKUP_FORMAT_PBS) {
|
|
|
|
|
+ if (proxmox_backup_co_add_config(pbs, name, (unsigned char *)cdata, clen, errp) < 0)
|
|
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ out:
|
|
|
|
|
+ g_free(basename);
|
|
|
|
|
+ g_free(cdata);
|
|
|
|
|
+ return res;
|
|
|
|
|
+
|
|
|
|
|
+ err:
|
|
|
|
|
+ res = -1;
|
|
|
|
|
+ goto out;
|
|
|
|
|
+}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+/*
|
|
|
|
|
+ * backup_job_create can *not* be run from a coroutine (and requires an
|
|
|
|
|
+ * acquired AioContext), so this can't either.
|
|
|
|
|
+ * The caller is responsible that backup_mutex is held nonetheless.
|
|
|
|
|
+ */
|
|
|
|
|
+static void create_backup_jobs_bh(void *opaque) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(!qemu_in_coroutine());
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ CoCtxData *data = (CoCtxData*)opaque;
|
|
|
|
|
+ Error **errp = (Error**)data->data;
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ /* create job transaction to synchronize bitmap commit and cancel all
|
|
|
|
|
+ * jobs in case one errors */
|
|
|
|
|
+ if (backup_state.txn) {
|
|
|
|
|
+ job_txn_unref(backup_state.txn);
|
|
|
|
|
+ }
|
|
|
|
|
+ backup_state.txn = job_txn_new_seq();
|
2021-05-27 13:43:32 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ /* create and start all jobs (paused state) */
|
|
|
|
|
+ GList *l = backup_state.di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
|
|
|
|
+ assert(di->target != NULL);
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ MirrorSyncMode sync_mode = MIRROR_SYNC_MODE_FULL;
|
|
|
|
|
+ BitmapSyncMode bitmap_mode = BITMAP_SYNC_MODE_NEVER;
|
|
|
|
|
+ if (di->bitmap) {
|
|
|
|
|
+ sync_mode = MIRROR_SYNC_MODE_BITMAP;
|
|
|
|
|
+ bitmap_mode = BITMAP_SYNC_MODE_ON_SUCCESS;
|
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+ AioContext *aio_context = bdrv_get_aio_context(di->bs);
|
|
|
|
|
+ aio_context_acquire(aio_context);
|
|
|
|
|
+
|
2023-09-06 11:45:11 +03:00
|
|
|
+ bdrv_drained_begin(di->bs);
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ BlockJob *job = backup_job_create(
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ NULL, di->bs, di->target, backup_state.speed, sync_mode, di->bitmap,
|
|
|
|
|
+ bitmap_mode, false, NULL, &backup_state.perf, BLOCKDEV_ON_ERROR_REPORT,
|
|
|
|
|
+ BLOCKDEV_ON_ERROR_REPORT, JOB_DEFAULT, pvebackup_complete_cb, di, backup_state.txn,
|
|
|
|
|
+ &local_err);
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
2023-09-06 11:45:11 +03:00
|
|
|
+ bdrv_drained_end(di->bs);
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ aio_context_release(aio_context);
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ di->job = job;
|
|
|
|
|
+ if (job) {
|
|
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
|
+ job_ref_locked(&job->job);
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!job || local_err) {
|
|
|
|
|
+ error_setg(errp, "backup_job_create failed: %s",
|
|
|
|
|
+ local_err ? error_get_pretty(local_err) : "null");
|
2021-03-03 12:56:02 +03:00
|
|
|
+ break;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ bdrv_unref(di->target);
|
|
|
|
|
+ di->target = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (*errp) {
|
|
|
|
|
+ /*
|
|
|
|
|
+ * It's enough to cancel one job in the transaction, the rest will
|
|
|
|
|
+ * follow automatically.
|
|
|
|
|
+ */
|
|
|
|
|
+ bool canceled = false;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ l = backup_state.di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
|
|
|
|
+ if (di->target) {
|
|
|
|
|
+ bdrv_unref(di->target);
|
|
|
|
|
+ di->target = NULL;
|
|
|
|
|
+ }
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+ if (di->job) {
|
|
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
|
+ if (!canceled) {
|
|
|
|
|
+ job_cancel_sync_locked(&di->job->job, true);
|
|
|
|
|
+ canceled = true;
|
|
|
|
|
+ }
|
|
|
|
|
+ job_unref_locked(&di->job->job);
|
|
|
|
|
+ di->job = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ /* return */
|
|
|
|
|
+ aio_co_enter(data->ctx, data->co);
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+UuidInfo coroutine_fn *qmp_backup(
|
|
|
|
|
+ const char *backup_file,
|
|
|
|
|
+ const char *password,
|
|
|
|
|
+ const char *keyfile,
|
|
|
|
|
+ const char *key_password,
|
|
|
|
|
+ const char *master_keyfile,
|
|
|
|
|
+ const char *fingerprint,
|
|
|
|
|
+ const char *backup_ns,
|
|
|
|
|
+ const char *backup_id,
|
|
|
|
|
+ bool has_backup_time, int64_t backup_time,
|
|
|
|
|
+ bool has_use_dirty_bitmap, bool use_dirty_bitmap,
|
|
|
|
|
+ bool has_compress, bool compress,
|
|
|
|
|
+ bool has_encrypt, bool encrypt,
|
|
|
|
|
+ bool has_format, BackupFormat format,
|
|
|
|
|
+ const char *config_file,
|
|
|
|
|
+ const char *firewall_file,
|
|
|
|
|
+ const char *devlist,
|
|
|
|
|
+ bool has_speed, int64_t speed,
|
|
|
|
|
+ bool has_max_workers, int64_t max_workers,
|
|
|
|
|
+ Error **errp)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ BlockBackend *blk;
|
|
|
|
|
+ BlockDriverState *bs = NULL;
|
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
|
+ uuid_t uuid;
|
|
|
|
|
+ VmaWriter *vmaw = NULL;
|
|
|
|
|
+ ProxmoxBackupHandle *pbs = NULL;
|
|
|
|
|
+ gchar **devs = NULL;
|
|
|
|
|
+ GList *di_list = NULL;
|
|
|
|
|
+ GList *l;
|
|
|
|
|
+ UuidInfo *uuid_info;
|
|
|
|
|
+
|
|
|
|
|
+ const char *config_name = "qemu-server.conf";
|
|
|
|
|
+ const char *firewall_name = "qemu-server.fw";
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.di_list) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ "previous backup not finished");
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
|
+ return NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ /* Todo: try to auto-detect format based on file name */
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ format = has_format ? format : BACKUP_FORMAT_VMA;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (devlist) {
|
|
|
|
|
+ devs = g_strsplit_set(devlist, ",;:", -1);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ gchar **d = devs;
|
|
|
|
|
+ while (d && *d) {
|
|
|
|
|
+ blk = blk_by_name(*d);
|
|
|
|
|
+ if (blk) {
|
|
|
|
|
+ bs = blk_bs(blk);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!bdrv_co_is_inserted(bs)) {
|
|
|
|
|
+ error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+ PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
|
|
|
|
|
+ di->bs = bs;
|
|
|
|
|
+ di_list = g_list_append(di_list, di);
|
|
|
|
|
+ } else {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ "Device '%s' not found", *d);
|
|
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+ d++;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ } else {
|
|
|
|
|
+ BdrvNextIterator it;
|
|
|
|
|
+
|
|
|
|
|
+ bs = NULL;
|
|
|
|
|
+ for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!bdrv_co_is_inserted(bs) || bdrv_is_read_only(bs)) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ continue;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
|
|
|
|
|
+ di->bs = bs;
|
|
|
|
|
+ di_list = g_list_append(di_list, di);
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (!di_list) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ size_t total = 0;
|
|
|
|
|
+
|
|
|
|
|
+ l = di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ ssize_t size = bdrv_getlength(di->bs);
|
|
|
|
|
+ if (size < 0) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_setg_errno(errp, -size, "bdrv_getlength failed");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+ di->size = size;
|
|
|
|
|
+ total += size;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+ di->completed_ret = INT_MAX;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ uuid_generate(uuid);
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ backup_state.stat.reused = 0;
|
|
|
|
|
+
|
|
|
|
|
+ /* clear previous backup's bitmap_list */
|
|
|
|
|
+ if (backup_state.stat.bitmap_list) {
|
|
|
|
|
+ GList *bl = backup_state.stat.bitmap_list;
|
|
|
|
|
+ while (bl) {
|
|
|
|
|
+ g_free(((PBSBitmapInfo *)bl->data)->drive);
|
|
|
|
|
+ g_free(bl->data);
|
|
|
|
|
+ bl = g_list_next(bl);
|
|
|
|
|
+ }
|
|
|
|
|
+ g_list_free(backup_state.stat.bitmap_list);
|
|
|
|
|
+ backup_state.stat.bitmap_list = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+ if (format == BACKUP_FORMAT_PBS) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!password) {
|
|
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
|
|
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!backup_id) {
|
|
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
|
|
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (!has_backup_time) {
|
|
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
|
|
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ int dump_cb_block_size = PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE; // Hardcoded (4M)
|
|
|
|
|
+ firewall_name = "fw.conf";
|
|
|
|
|
+
|
|
|
|
|
+ char *pbs_err = NULL;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ pbs = proxmox_backup_new_ns(
|
|
|
|
|
+ backup_file,
|
|
|
|
|
+ backup_ns,
|
|
|
|
|
+ backup_id,
|
|
|
|
|
+ backup_time,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ dump_cb_block_size,
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ password,
|
|
|
|
|
+ keyfile,
|
|
|
|
|
+ key_password,
|
|
|
|
|
+ master_keyfile,
|
|
|
|
|
+ has_compress ? compress : true,
|
|
|
|
|
+ has_encrypt ? encrypt : !!keyfile,
|
|
|
|
|
+ fingerprint,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ &pbs_err);
|
|
|
|
|
+
|
|
|
|
|
+ if (!pbs) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ "proxmox_backup_new failed: %s", pbs_err);
|
|
|
|
|
+ proxmox_backup_free_error(pbs_err);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ int connect_result = proxmox_backup_co_connect(pbs, errp);
|
|
|
|
|
+ if (connect_result < 0)
|
|
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ /* register all devices */
|
|
|
|
|
+ l = di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ di->block_size = dump_cb_block_size;
|
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+ const char *devname = bdrv_get_device_name(di->bs);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ PBSBitmapAction action = PBS_BITMAP_ACTION_NOT_USED;
|
|
|
|
|
+ size_t dirty = di->size;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(di->bs, PBS_BITMAP_NAME);
|
|
|
|
|
+ bool expect_only_dirty = false;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (has_use_dirty_bitmap && use_dirty_bitmap) {
|
|
|
|
|
+ if (bitmap == NULL) {
|
|
|
|
|
+ bitmap = bdrv_create_dirty_bitmap(di->bs, dump_cb_block_size, PBS_BITMAP_NAME, errp);
|
|
|
|
|
+ if (!bitmap) {
|
|
|
|
|
+ goto err_mutex;
|
|
|
|
|
+ }
|
|
|
|
|
+ action = PBS_BITMAP_ACTION_NEW;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ expect_only_dirty = proxmox_backup_check_incremental(pbs, devname, di->size) != 0;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (expect_only_dirty) {
|
|
|
|
|
+ /* track clean chunks as reused */
|
|
|
|
|
+ dirty = MIN(bdrv_get_dirty_count(bitmap), di->size);
|
|
|
|
|
+ backup_state.stat.reused += di->size - dirty;
|
|
|
|
|
+ action = PBS_BITMAP_ACTION_USED;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ /* mark entire bitmap as dirty to make full backup */
|
|
|
|
|
+ bdrv_set_dirty_bitmap(bitmap, 0, di->size);
|
|
|
|
|
+ if (action != PBS_BITMAP_ACTION_NEW) {
|
|
|
|
|
+ action = PBS_BITMAP_ACTION_INVALID;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ di->bitmap = bitmap;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ /* after a full backup the old dirty bitmap is invalid anyway */
|
|
|
|
|
+ if (bitmap != NULL) {
|
|
|
|
|
+ bdrv_release_dirty_bitmap(bitmap);
|
|
|
|
|
+ action = PBS_BITMAP_ACTION_NOT_USED_REMOVED;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, expect_only_dirty, errp);
|
|
|
|
|
+ if (dev_id < 0) {
|
|
|
|
|
+ goto err_mutex;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
2023-10-17 15:10:09 +03:00
|
|
|
+ if (!(di->target = bdrv_co_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, errp))) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ di->dev_id = dev_id;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+ PBSBitmapInfo *info = g_malloc(sizeof(*info));
|
|
|
|
|
+ info->drive = g_strdup(devname);
|
|
|
|
|
+ info->action = action;
|
|
|
|
|
+ info->size = di->size;
|
|
|
|
|
+ info->dirty = dirty;
|
|
|
|
|
+ backup_state.stat.bitmap_list = g_list_append(backup_state.stat.bitmap_list, info);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ } else if (format == BACKUP_FORMAT_VMA) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ vmaw = vma_writer_create(backup_file, uuid, &local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ if (!vmaw) {
|
|
|
|
|
+ if (local_err) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_propagate(errp, local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ /* register all devices for vma writer */
|
|
|
|
|
+ l = di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
2023-10-17 15:10:09 +03:00
|
|
|
+ if (!(di->target = bdrv_co_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, errp))) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ const char *devname = bdrv_get_device_name(di->bs);
|
|
|
|
|
+ di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
|
|
|
|
|
+ if (di->dev_id <= 0) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ "register_stream failed");
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ } else {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
|
|
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ /* add configuration file to archive */
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (config_file) {
|
2023-09-06 11:45:12 +03:00
|
|
|
+ if (pvebackup_co_add_config(config_file, config_name, format, vmaw, pbs, errp) != 0) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ /* add firewall file to archive */
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ if (firewall_file) {
|
2023-09-06 11:45:12 +03:00
|
|
|
+ if (pvebackup_co_add_config(firewall_file, firewall_name, format, vmaw, pbs, errp) != 0) {
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ goto err_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ /* initialize global backup_state now */
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ /* note: 'reused' and 'bitmap_list' are initialized earlier */
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (backup_state.stat.error) {
|
|
|
|
|
+ error_free(backup_state.stat.error);
|
|
|
|
|
+ backup_state.stat.error = NULL;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ backup_state.stat.start_time = time(NULL);
|
|
|
|
|
+ backup_state.stat.end_time = 0;
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.stat.backup_file) {
|
|
|
|
|
+ g_free(backup_state.stat.backup_file);
|
|
|
|
|
+ }
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.stat.backup_file = g_strdup(backup_file);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ uuid_copy(backup_state.stat.uuid, uuid);
|
|
|
|
|
+ uuid_unparse_lower(uuid, backup_state.stat.uuid_str);
|
|
|
|
|
+ char *uuid_str = g_strdup(backup_state.stat.uuid_str);
|
|
|
|
|
+
|
|
|
|
|
+ backup_state.stat.total = total;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.stat.dirty = total - backup_state.stat.reused;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ backup_state.stat.transferred = 0;
|
|
|
|
|
+ backup_state.stat.zero_bytes = 0;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.stat.finishing = false;
|
|
|
|
|
+ backup_state.stat.starting = true;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.speed = (has_speed && speed > 0) ? speed : 0;
|
|
|
|
|
+
|
|
|
|
|
+ backup_state.perf = (BackupPerf){ .max_workers = 16 };
|
|
|
|
|
+ if (has_max_workers) {
|
|
|
|
|
+ backup_state.perf.max_workers = max_workers;
|
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ backup_state.vmaw = vmaw;
|
|
|
|
|
+ backup_state.pbs = pbs;
|
|
|
|
|
+
|
|
|
|
|
+ backup_state.di_list = di_list;
|
|
|
|
|
+
|
|
|
|
|
+ uuid_info = g_malloc0(sizeof(*uuid_info));
|
|
|
|
|
+ uuid_info->UUID = uuid_str;
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ /* Run create_backup_jobs_bh outside of coroutine (in BH) but keep
|
|
|
|
|
+ * backup_mutex locked. This is fine, a CoMutex can be held across yield
|
|
|
|
|
+ * points, and we'll release it as soon as the BH reschedules us.
|
|
|
|
|
+ */
|
|
|
|
|
+ CoCtxData waker = {
|
|
|
|
|
+ .co = qemu_coroutine_self(),
|
|
|
|
|
+ .ctx = qemu_get_current_aio_context(),
|
|
|
|
|
+ .data = &local_err,
|
|
|
|
|
+ };
|
|
|
|
|
+ aio_bh_schedule_oneshot(waker.ctx, create_backup_jobs_bh, &waker);
|
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
|
+
|
|
|
|
|
+ if (local_err) {
|
|
|
|
|
+ error_propagate(errp, local_err);
|
|
|
|
|
+ goto err;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
|
+
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+ backup_state.stat.starting = false;
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+
|
|
|
|
|
+ /* start the first job in the transaction */
|
|
|
|
|
+ job_txn_start_seq(backup_state.txn);
|
|
|
|
|
+
|
|
|
|
|
+ return uuid_info;
|
|
|
|
|
+
|
|
|
|
|
+err_mutex:
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+err:
|
|
|
|
|
+
|
|
|
|
|
+ l = di_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
|
|
|
|
+ if (di->target) {
|
2023-05-15 16:39:54 +03:00
|
|
|
+ bdrv_co_unref(di->target);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (di->targetfile[0]) {
|
|
|
|
|
+ unlink(di->targetfile);
|
|
|
|
|
+ }
|
|
|
|
|
+ g_free(di);
|
|
|
|
|
+ }
|
|
|
|
|
+ g_list_free(di_list);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.di_list = NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (devs) {
|
|
|
|
|
+ g_strfreev(devs);
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (vmaw) {
|
|
|
|
|
+ Error *err = NULL;
|
|
|
|
|
+ vma_writer_close(vmaw, &err);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ unlink(backup_file);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ if (pbs) {
|
|
|
|
|
+ proxmox_backup_disconnect(pbs);
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ backup_state.pbs = NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
|
+ return NULL;
|
2021-03-03 12:56:02 +03:00
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+BackupStatus *qmp_query_backup(Error **errp)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
|
+ BackupStatus *info = g_malloc0(sizeof(*info));
|
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+ if (!backup_state.stat.start_time) {
|
|
|
|
|
+ /* not started, return {} */
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+ return info;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ info->has_start_time = true;
|
|
|
|
|
+ info->start_time = backup_state.stat.start_time;
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.stat.backup_file) {
|
|
|
|
|
+ info->backup_file = g_strdup(backup_state.stat.backup_file);
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ info->uuid = g_strdup(backup_state.stat.uuid_str);
|
|
|
|
|
+
|
|
|
|
|
+ if (backup_state.stat.end_time) {
|
|
|
|
|
+ if (backup_state.stat.error) {
|
|
|
|
|
+ info->status = g_strdup("error");
|
|
|
|
|
+ info->errmsg = g_strdup(error_get_pretty(backup_state.stat.error));
|
|
|
|
|
+ } else {
|
|
|
|
|
+ info->status = g_strdup("done");
|
|
|
|
|
+ }
|
|
|
|
|
+ info->has_end_time = true;
|
|
|
|
|
+ info->end_time = backup_state.stat.end_time;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ info->status = g_strdup("active");
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ info->has_total = true;
|
|
|
|
|
+ info->total = backup_state.stat.total;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ info->has_dirty = true;
|
|
|
|
|
+ info->dirty = backup_state.stat.dirty;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ info->has_zero_bytes = true;
|
|
|
|
|
+ info->zero_bytes = backup_state.stat.zero_bytes;
|
|
|
|
|
+ info->has_transferred = true;
|
|
|
|
|
+ info->transferred = backup_state.stat.transferred;
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ info->has_reused = true;
|
|
|
|
|
+ info->reused = backup_state.stat.reused;
|
|
|
|
|
+ info->finishing = backup_state.stat.finishing;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ return info;
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+
|
|
|
|
|
+PBSBitmapInfoList *qmp_query_pbs_bitmap_info(Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ PBSBitmapInfoList *head = NULL, **p_next = &head;
|
|
|
|
|
+
|
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
|
+
|
|
|
|
|
+ GList *l = backup_state.stat.bitmap_list;
|
|
|
|
|
+ while (l) {
|
|
|
|
|
+ PBSBitmapInfo *info = (PBSBitmapInfo *)l->data;
|
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
|
+
|
|
|
|
|
+ /* clone bitmap info to avoid auto free after QMP marshalling */
|
|
|
|
|
+ PBSBitmapInfo *info_ret = g_malloc0(sizeof(*info_ret));
|
|
|
|
|
+ info_ret->drive = g_strdup(info->drive);
|
|
|
|
|
+ info_ret->action = info->action;
|
|
|
|
|
+ info_ret->size = info->size;
|
|
|
|
|
+ info_ret->dirty = info->dirty;
|
|
|
|
|
+
|
|
|
|
|
+ PBSBitmapInfoList *info_list = g_malloc0(sizeof(*info_list));
|
|
|
|
|
+ info_list->value = info_ret;
|
|
|
|
|
+
|
|
|
|
|
+ *p_next = info_list;
|
|
|
|
|
+ p_next = &info_list->next;
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
|
+
|
|
|
|
|
+ return head;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
|
|
|
|
|
+{
|
|
|
|
|
+ ProxmoxSupportStatus *ret = g_malloc0(sizeof(*ret));
|
|
|
|
|
+ ret->pbs_library_version = g_strdup(proxmox_backup_qemu_version());
|
|
|
|
|
+ ret->pbs_dirty_bitmap = true;
|
|
|
|
|
+ ret->pbs_dirty_bitmap_savevm = true;
|
|
|
|
|
+ ret->query_bitmap_info = true;
|
|
|
|
|
+ ret->pbs_masterkey = true;
|
|
|
|
|
+ ret->backup_max_workers = true;
|
|
|
|
|
+ return ret;
|
|
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/qapi/block-core.json b/qapi/block-core.json
|
2024-01-30 17:14:37 +03:00
|
|
|
index bb471c078d..1b8462a51b 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/qapi/block-core.json
|
|
|
|
|
+++ b/qapi/block-core.json
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -839,6 +839,235 @@
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
{ 'command': 'query-block', 'returns': ['BlockInfo'],
|
|
|
|
|
'allow-preconfig': true }
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
|
+##
|
|
|
|
|
+# @BackupStatus:
|
|
|
|
|
+#
|
|
|
|
|
+# Detailed backup status.
|
|
|
|
|
+#
|
|
|
|
|
+# @status: string describing the current backup status.
|
|
|
|
|
+# This can be 'active', 'done', 'error'. If this field is not
|
|
|
|
|
+# returned, no backup process has been initiated
|
|
|
|
|
+#
|
|
|
|
|
+# @errmsg: error message (only returned if status is 'error')
|
|
|
|
|
+#
|
|
|
|
|
+# @total: total amount of bytes involved in the backup process
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @dirty: with incremental mode (PBS) this is the amount of bytes involved
|
|
|
|
|
+# in the backup process which are marked dirty.
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+# @transferred: amount of bytes already backed up.
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @reused: amount of bytes reused due to deduplication.
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+# @zero-bytes: amount of 'zero' bytes detected.
|
|
|
|
|
+#
|
|
|
|
|
+# @start-time: time (epoch) when backup job started.
|
|
|
|
|
+#
|
|
|
|
|
+# @end-time: time (epoch) when backup job finished.
|
|
|
|
|
+#
|
|
|
|
|
+# @backup-file: backup file name
|
|
|
|
|
+#
|
|
|
|
|
+# @uuid: uuid for this backup job
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @finishing: if status='active' and finishing=true, then the backup process is
|
|
|
|
|
+# waiting for the target to finish.
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+##
|
|
|
|
|
+{ 'struct': 'BackupStatus',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ 'data': {'*status': 'str', '*errmsg': 'str', '*total': 'int', '*dirty': 'int',
|
|
|
|
|
+ '*transferred': 'int', '*zero-bytes': 'int', '*reused': 'int',
|
2020-03-10 17:12:50 +03:00
|
|
|
+ '*start-time': 'int', '*end-time': 'int',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ '*backup-file': 'str', '*uuid': 'str', 'finishing': 'bool' } }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @BackupFormat:
|
|
|
|
|
+#
|
|
|
|
|
+# An enumeration of supported backup formats.
|
|
|
|
|
+#
|
|
|
|
|
+# @vma: Proxmox vma backup format
|
2023-09-06 11:45:12 +03:00
|
|
|
+#
|
|
|
|
|
+# @pbs: Proxmox backup server format
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+##
|
|
|
|
|
+{ 'enum': 'BackupFormat',
|
2023-09-06 11:45:12 +03:00
|
|
|
+ 'data': [ 'vma', 'pbs' ] }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @backup:
|
|
|
|
|
+#
|
|
|
|
|
+# Starts a VM backup.
|
|
|
|
|
+#
|
|
|
|
|
+# @backup-file: the backup file name
|
|
|
|
|
+#
|
|
|
|
|
+# @format: format of the backup file
|
|
|
|
|
+#
|
|
|
|
|
+# @config-file: a configuration file to include into
|
2021-02-11 19:11:11 +03:00
|
|
|
+# the backup archive.
|
2020-03-10 17:12:50 +03:00
|
|
|
+#
|
|
|
|
|
+# @speed: the maximum speed, in bytes per second
|
|
|
|
|
+#
|
|
|
|
|
+# @devlist: list of block device names (separated by ',', ';'
|
2021-02-11 19:11:11 +03:00
|
|
|
+# or ':'). By default the backup includes all writable block devices.
|
2020-03-10 17:12:50 +03:00
|
|
|
+#
|
|
|
|
|
+# @password: backup server passsword (required for format 'pbs')
|
|
|
|
|
+#
|
|
|
|
|
+# @keyfile: keyfile used for encryption (optional for format 'pbs')
|
|
|
|
|
+#
|
|
|
|
|
+# @key-password: password for keyfile (optional for format 'pbs')
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @master-keyfile: PEM-formatted master public keyfile (optional for format 'pbs')
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+# @fingerprint: server cert fingerprint (optional for format 'pbs')
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @backup-ns: backup namespace (required for format 'pbs')
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+# @backup-id: backup ID (required for format 'pbs')
|
|
|
|
|
+#
|
|
|
|
|
+# @backup-time: backup timestamp (Unix epoch, required for format 'pbs')
|
|
|
|
|
+#
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+# @use-dirty-bitmap: use dirty bitmap to detect incremental changes since last job (optional for format 'pbs')
|
|
|
|
|
+#
|
|
|
|
|
+# @compress: use compression (optional for format 'pbs', defaults to true)
|
|
|
|
|
+#
|
|
|
|
|
+# @encrypt: use encryption ((optional for format 'pbs', defaults to true if there is a keyfile)
|
|
|
|
|
+#
|
|
|
|
|
+# @max-workers: see @BackupPerf for details. Default 16.
|
|
|
|
|
+#
|
2020-03-10 17:12:50 +03:00
|
|
|
+# Returns: the uuid of the backup job
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'command': 'backup', 'data': { 'backup-file': 'str',
|
|
|
|
|
+ '*password': 'str',
|
|
|
|
|
+ '*keyfile': 'str',
|
|
|
|
|
+ '*key-password': 'str',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ '*master-keyfile': 'str',
|
2020-03-10 17:12:50 +03:00
|
|
|
+ '*fingerprint': 'str',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ '*backup-ns': 'str',
|
2020-03-10 17:12:50 +03:00
|
|
|
+ '*backup-id': 'str',
|
|
|
|
|
+ '*backup-time': 'int',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ '*use-dirty-bitmap': 'bool',
|
|
|
|
|
+ '*compress': 'bool',
|
|
|
|
|
+ '*encrypt': 'bool',
|
2020-03-10 17:12:50 +03:00
|
|
|
+ '*format': 'BackupFormat',
|
|
|
|
|
+ '*config-file': 'str',
|
|
|
|
|
+ '*firewall-file': 'str',
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+ '*devlist': 'str',
|
|
|
|
|
+ '*speed': 'int',
|
|
|
|
|
+ '*max-workers': 'int' },
|
|
|
|
|
+ 'returns': 'UuidInfo', 'coroutine': true }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @query-backup:
|
|
|
|
|
+#
|
|
|
|
|
+# Returns information about current/last backup task.
|
|
|
|
|
+#
|
|
|
|
|
+# Returns: @BackupStatus
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'command': 'query-backup', 'returns': 'BackupStatus' }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @backup-cancel:
|
|
|
|
|
+#
|
|
|
|
|
+# Cancel the current executing backup process.
|
|
|
|
|
+#
|
|
|
|
|
+# Returns: nothing on success
|
|
|
|
|
+#
|
|
|
|
|
+# Notes: This command succeeds even if there is no backup process running.
|
|
|
|
|
+#
|
|
|
|
|
+##
|
squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.
If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.
The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.
No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.
The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.
The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.
The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:56 +03:00
|
|
|
+{ 'command': 'backup-cancel', 'coroutine': true }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @ProxmoxSupportStatus:
|
|
|
|
|
+#
|
|
|
|
|
+# Contains info about supported features added by Proxmox.
|
|
|
|
|
+#
|
|
|
|
|
+# @pbs-dirty-bitmap: True if dirty-bitmap-incremental backups to PBS are
|
|
|
|
|
+# supported.
|
|
|
|
|
+#
|
|
|
|
|
+# @query-bitmap-info: True if the 'query-pbs-bitmap-info' QMP call is supported.
|
|
|
|
|
+#
|
|
|
|
|
+# @pbs-dirty-bitmap-savevm: True if 'dirty-bitmaps' migration capability can
|
|
|
|
|
+# safely be set for savevm-async.
|
|
|
|
|
+#
|
|
|
|
|
+# @pbs-masterkey: True if the QMP backup call supports the 'master_keyfile'
|
|
|
|
|
+# parameter.
|
|
|
|
|
+#
|
|
|
|
|
+# @pbs-library-version: Running version of libproxmox-backup-qemu0 library.
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'struct': 'ProxmoxSupportStatus',
|
|
|
|
|
+ 'data': { 'pbs-dirty-bitmap': 'bool',
|
|
|
|
|
+ 'query-bitmap-info': 'bool',
|
|
|
|
|
+ 'pbs-dirty-bitmap-savevm': 'bool',
|
|
|
|
|
+ 'pbs-masterkey': 'bool',
|
|
|
|
|
+ 'pbs-library-version': 'str',
|
|
|
|
|
+ 'backup-max-workers': 'bool' } }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @query-proxmox-support:
|
|
|
|
|
+#
|
|
|
|
|
+# Returns information about supported features added by Proxmox.
|
|
|
|
|
+#
|
|
|
|
|
+# Returns: @ProxmoxSupportStatus
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'command': 'query-proxmox-support', 'returns': 'ProxmoxSupportStatus' }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @PBSBitmapAction:
|
|
|
|
|
+#
|
|
|
|
|
+# An action taken on a dirty-bitmap when a backup job was started.
|
|
|
|
|
+#
|
|
|
|
|
+# @not-used: Bitmap mode was not enabled.
|
|
|
|
|
+#
|
|
|
|
|
+# @not-used-removed: Bitmap mode was not enabled, but a bitmap from a
|
|
|
|
|
+# previous backup still existed and was removed.
|
|
|
|
|
+#
|
|
|
|
|
+# @new: A new bitmap was attached to the drive for this backup.
|
|
|
|
|
+#
|
|
|
|
|
+# @used: An existing bitmap will be used to only backup changed data.
|
|
|
|
|
+#
|
|
|
|
|
+# @invalid: A bitmap existed, but had to be cleared since it's associated
|
|
|
|
|
+# base snapshot did not match the base given for the current job or
|
|
|
|
|
+# the crypt mode has changed.
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'enum': 'PBSBitmapAction',
|
|
|
|
|
+ 'data': ['not-used', 'not-used-removed', 'new', 'used', 'invalid'] }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @PBSBitmapInfo:
|
|
|
|
|
+#
|
|
|
|
|
+# Contains information about dirty bitmaps used for each drive in a PBS backup.
|
|
|
|
|
+#
|
|
|
|
|
+# @drive: The underlying drive.
|
|
|
|
|
+#
|
|
|
|
|
+# @action: The action that was taken when the backup started.
|
|
|
|
|
+#
|
|
|
|
|
+# @size: The total size of the drive.
|
|
|
|
|
+#
|
|
|
|
|
+# @dirty: How much of the drive is considered dirty and will be backed up,
|
|
|
|
|
+# or 'size' if everything will be.
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'struct': 'PBSBitmapInfo',
|
|
|
|
|
+ 'data': { 'drive': 'str', 'action': 'PBSBitmapAction', 'size': 'int',
|
|
|
|
|
+ 'dirty': 'int' } }
|
|
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @query-pbs-bitmap-info:
|
|
|
|
|
+#
|
|
|
|
|
+# Returns information about dirty bitmaps used on the most recently started
|
|
|
|
|
+# backup. Returns nothing when the last backup was not using PBS or if no
|
|
|
|
|
+# backup occured in this session.
|
|
|
|
|
+#
|
|
|
|
|
+# Returns: @PBSBitmapInfo
|
|
|
|
|
+#
|
|
|
|
|
+##
|
|
|
|
|
+{ 'command': 'query-pbs-bitmap-info', 'returns': ['PBSBitmapInfo'] }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
##
|
|
|
|
|
# @BlockDeviceTimedStats:
|
|
|
|
|
#
|
|
|
|
|
diff --git a/qapi/common.json b/qapi/common.json
|
2023-10-17 15:10:09 +03:00
|
|
|
index 6fed9cde1a..630a2a8f9a 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/qapi/common.json
|
|
|
|
|
+++ b/qapi/common.json
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -207,3 +207,17 @@
|
2022-02-11 12:24:33 +03:00
|
|
|
##
|
|
|
|
|
{ 'struct': 'HumanReadableText',
|
|
|
|
|
'data': { 'human-readable-text': 'str' } }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
|
+##
|
|
|
|
|
+# @UuidInfo:
|
|
|
|
|
+#
|
|
|
|
|
+# Guest UUID information (Universally Unique Identifier).
|
|
|
|
|
+#
|
|
|
|
|
+# @UUID: the UUID of the guest
|
|
|
|
|
+#
|
|
|
|
|
+# Since: 0.14.0
|
|
|
|
|
+#
|
2023-10-17 15:10:09 +03:00
|
|
|
+# Notes: If no UUID was specified for the guest, a null UUID is
|
|
|
|
|
+# returned.
|
2020-03-10 17:12:50 +03:00
|
|
|
+##
|
|
|
|
|
+{ 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/qapi/machine.json b/qapi/machine.json
|
2023-10-17 15:10:09 +03:00
|
|
|
index 7da3c519ba..888457f810 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/qapi/machine.json
|
|
|
|
|
+++ b/qapi/machine.json
|
|
|
|
|
@@ -4,6 +4,8 @@
|
|
|
|
|
# This work is licensed under the terms of the GNU GPL, version 2 or later.
|
|
|
|
|
# See the COPYING file in the top-level directory.
|
|
|
|
|
|
|
|
|
|
+{ 'include': 'common.json' }
|
|
|
|
|
+
|
|
|
|
|
##
|
|
|
|
|
# = Machines
|
|
|
|
|
##
|
2023-10-17 15:10:09 +03:00
|
|
|
@@ -230,20 +232,6 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
##
|
2021-02-11 19:11:11 +03:00
|
|
|
{ 'command': 'query-target', 'returns': 'TargetInfo' }
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
|
-##
|
|
|
|
|
-# @UuidInfo:
|
|
|
|
|
-#
|
|
|
|
|
-# Guest UUID information (Universally Unique Identifier).
|
|
|
|
|
-#
|
|
|
|
|
-# @UUID: the UUID of the guest
|
|
|
|
|
-#
|
2021-05-27 13:43:32 +03:00
|
|
|
-# Since: 0.14
|
2020-03-10 17:12:50 +03:00
|
|
|
-#
|
2023-10-17 15:10:09 +03:00
|
|
|
-# Notes: If no UUID was specified for the guest, a null UUID is
|
|
|
|
|
-# returned.
|
2020-03-10 17:12:50 +03:00
|
|
|
-##
|
|
|
|
|
-{ 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
|
|
|
|
|
-
|
|
|
|
|
##
|
|
|
|
|
# @query-uuid:
|
|
|
|
|
#
|