vitalif
/
pve-qemu
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
pve-qemu/debian/patches/pve/0044-PBS-add-master-key-sup...

99 lines
3.9 KiB
Diff
Raw Normal View History

add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Wed, 10 Feb 2021 11:07:06 +0100
Subject: [PATCH] PBS: add master key support
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
this requires a new enough libproxmox-backup-qemu0, and allows querying
from the PVE side to avoid QMP calls with unsupported parameters.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
update submodule and patches to 6.1.1 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-13 12:34:33 +03:00
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
---
block/monitor/block-hmp-cmds.c | 1 +
pve-backup.c | 3 +++
qapi/block-core.json | 7 +++++++
3 files changed, 11 insertions(+)
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
update submodule and patches to 7.0.0 Only very minor changes needed: * Most patches in extra (or some version of them) are part of 7.0.0. * aio_set_fd_handler got an extra parameter, but can just pass NULL like we did for the related 'poll' parameter. See QEMU commit 826cc32423db2a99d184dbf4f507c737d7e7a4ae for more. * Add include for qemu/memalign.h in vma.c and vma-writer.c. * Add reverts for fixups of already reverted 0347a8fd4c ("block/rbd: implement bdrv_co_block_status") that came in with 7.0.0. Those fixups are not enough, see Proxmox bugzilla #4047. * Two trivial context changes for bitmap-mirror patches. * block_int.h got split up into multiple headers. * Some context changes in configure and meson.build. * Used the oppurtunity to squash fixup of bdrv_backuo_dump_create typo in a later patch into the patch introducing the function (had to move code to new header during rebase). Signed-off-by: Fabian Ebner <f.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-27 14:05:40 +03:00
index ef45552e3b..4c799f00d9 100644
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
update and rebase to QEMU v6.1.0 Very clean rebase, only the +pve version handling needed manual fixing. Drops two applied patches from extra/ and adds one new from upstream (extra/0001*, fixes VNC over unix sockets) as well as 3 of my own for allowing password changes on custom VNC displays again (as seen and reviewed upstream, but not yet applied). Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-11 14:55:34 +03:00
@@ -1039,6 +1039,7 @@ void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict)
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
false, NULL, // PBS password
false, NULL, // PBS keyfile
false, NULL, // PBS key_password
+ false, NULL, // PBS master_keyfile
false, NULL, // PBS fingerprint
false, NULL, // PBS backup-id
false, 0, // PBS backup-time
diff --git a/pve-backup.c b/pve-backup.c
update submodule and patches to 6.2.0 Notable changes: * bdrv_co_p{discard,readv,writev,write_zeroes} function signatures changed, to using int64_t for offsets/bytes and some still had int rather than BrdvRequestFlags for the flags. * job_cancel_sync now has a force parameter. Commit messages in 73895f3838cd7fdaf185cf1dbc47be58844a966f 4cfb3f05627ad82af473e7f7ae113c3884cd04e3 sound like using force=true makes more sense. * Added 3 patches coming in via qemu-stable tag, most important one is to work around a librbd issue. * Added another 3 patches from qemu-devel to fix issue leading to crash when live migrating with iothread. * cluster_size calculation helper changed (see patch pve/0026). * QAPI's if conditionals now use 'CONFIG_FOO' rather than 'defined(CONFIG_FOO)' Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-11 12:24:33 +03:00
index 323014744c..9f6c04a512 100644
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
--- a/pve-backup.c
+++ b/pve-backup.c
udpate and rebase to QEMU v6.0.0 Mostly minor changes, bigger ones summarized: * QEMU's internal backup code now uses a new async system, which allows parallel requests - the default max_workers settings is 64, I chose less, since 64 put enough stress on QEMU that the guest became practically unusable during the backup, and 16 still shows quite a nice measureable performance improvement. Little code changes for us though. * 'malformed' QAPI parameters/functions are now a build error (i.e. using '_' vs '-'), I chose to just whitelist our calls in the name of backwards compatibility. * monitor OOB race fix now uses the upstream variant, cherry-picked from origin/master since it's not in 6.0 by default * last patch fixes a bug with snapshot rollback related to the new yank system Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-27 13:43:32 +03:00
@@ -533,6 +533,7 @@ UuidInfo coroutine_fn *qmp_backup(
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
bool has_password, const char *password,
bool has_keyfile, const char *keyfile,
bool has_key_password, const char *key_password,
+ bool has_master_keyfile, const char *master_keyfile,
bool has_fingerprint, const char *fingerprint,
bool has_backup_id, const char *backup_id,
bool has_backup_time, int64_t backup_time,
udpate and rebase to QEMU v6.0.0 Mostly minor changes, bigger ones summarized: * QEMU's internal backup code now uses a new async system, which allows parallel requests - the default max_workers settings is 64, I chose less, since 64 put enough stress on QEMU that the guest became practically unusable during the backup, and 16 still shows quite a nice measureable performance improvement. Little code changes for us though. * 'malformed' QAPI parameters/functions are now a build error (i.e. using '_' vs '-'), I chose to just whitelist our calls in the name of backwards compatibility. * monitor OOB race fix now uses the upstream variant, cherry-picked from origin/master since it's not in 6.0 by default * last patch fixes a bug with snapshot rollback related to the new yank system Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-27 13:43:32 +03:00
@@ -681,6 +682,7 @@ UuidInfo coroutine_fn *qmp_backup(
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
has_password ? password : NULL,
has_keyfile ? keyfile : NULL,
has_key_password ? key_password : NULL,
+ has_master_keyfile ? master_keyfile : NULL,
has_compress ? compress : true,
has_encrypt ? encrypt : has_keyfile,
has_fingerprint ? fingerprint : NULL,
udpate and rebase to QEMU v6.0.0 Mostly minor changes, bigger ones summarized: * QEMU's internal backup code now uses a new async system, which allows parallel requests - the default max_workers settings is 64, I chose less, since 64 put enough stress on QEMU that the guest became practically unusable during the backup, and 16 still shows quite a nice measureable performance improvement. Little code changes for us though. * 'malformed' QAPI parameters/functions are now a build error (i.e. using '_' vs '-'), I chose to just whitelist our calls in the name of backwards compatibility. * monitor OOB race fix now uses the upstream variant, cherry-picked from origin/master since it's not in 6.0 by default * last patch fixes a bug with snapshot rollback related to the new yank system Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-27 13:43:32 +03:00
@@ -1044,5 +1046,6 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
fix saving and loading dirty bitmaps in snapshots Saving dirty bitmaps from our savevm-async code didn't work, since we use a coroutine which holds the iothread mutex already (upstream savevm is sync, migration uses a thread). Release the mutex before calling the one function that (according to it's documentation) requires the lock to *not* be held: qemu_savevm_state_pending. Additionally, loading dirty bitmaps requires a call to dirty_bitmap_mig_before_vm_start after "loadvm", which the upstream savevm does explicitly afterwards - do that too. This is exposed via the query-proxmox-support property "pbs-dirty-bitmap-savevm". Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-16 19:30:22 +03:00
ret->pbs_dirty_bitmap_savevm = true;
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
ret->pbs_dirty_bitmap_migration = true;
fix saving and loading dirty bitmaps in snapshots Saving dirty bitmaps from our savevm-async code didn't work, since we use a coroutine which holds the iothread mutex already (upstream savevm is sync, migration uses a thread). Release the mutex before calling the one function that (according to it's documentation) requires the lock to *not* be held: qemu_savevm_state_pending. Additionally, loading dirty bitmaps requires a call to dirty_bitmap_mig_before_vm_start after "loadvm", which the upstream savevm does explicitly afterwards - do that too. This is exposed via the query-proxmox-support property "pbs-dirty-bitmap-savevm". Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-16 19:30:22 +03:00
ret->query_bitmap_info = true;
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
+ ret->pbs_masterkey = true;
return ret;
}
diff --git a/qapi/block-core.json b/qapi/block-core.json
update submodule and patches to 7.1.0 Notable changes: * The only big change is the switch to using a custom QIOChannel for savevm-async, because the previously used QEMUFileOps was dropped. Changes to the current implementation: * Switch to vector based methods as required for an IO channel. For short reads the passed-in IO vector is stuffed with zeroes at the end, just to be sure. * For reading: The documentation in include/io/channel.h states that at least one byte should be read, so also error out when whe are at the very end instead of returning 0. * For reading: Fix off-by-one error when request goes beyond end. The wrong code piece was: if ((pos + size) > maxlen) { size = maxlen - pos - 1; } Previously, the last byte would not be read. It's actually possible to get a snapshot .raw file that has content all the way up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any trailing zero bytes (I wrote a script to do it). Luckily, it didn't cause a real issue, because qemu_loadvm_state() is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION) section. The buffer for reading it is simply freed up afterwards and the function will assume that it read the whole section, even if that's not the case. * For writing: Make use of the generated blk_pwritev() wrapper instead of manually wrapping the coroutine to simplify and save a few lines. * Adapt to changed interfaces for blk_{pread,pwrite}: * a9262f551e ("block: Change blk_{pread,pwrite}() param order") * 3b35d4542c ("block: Add a 'flags' param to blk_pread()") * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success") Those changes especially affected the qemu-img dd patches, because the context also changed, but also some of our block drivers used the functions. * Drop qemu-common.h include: it got renamed after essentially everything was moved to other headers. The only remaining user I could find for things dropped from the header between 7.0 and 7.1 was qemu_get_vm_name() in the iscsi-initiatorname patch, but it already includes the header to which the function was moved. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
index ca1966f54b..fc8a125451 100644
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
update submodule and patches to 7.1.0 Notable changes: * The only big change is the switch to using a custom QIOChannel for savevm-async, because the previously used QEMUFileOps was dropped. Changes to the current implementation: * Switch to vector based methods as required for an IO channel. For short reads the passed-in IO vector is stuffed with zeroes at the end, just to be sure. * For reading: The documentation in include/io/channel.h states that at least one byte should be read, so also error out when whe are at the very end instead of returning 0. * For reading: Fix off-by-one error when request goes beyond end. The wrong code piece was: if ((pos + size) > maxlen) { size = maxlen - pos - 1; } Previously, the last byte would not be read. It's actually possible to get a snapshot .raw file that has content all the way up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any trailing zero bytes (I wrote a script to do it). Luckily, it didn't cause a real issue, because qemu_loadvm_state() is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION) section. The buffer for reading it is simply freed up afterwards and the function will assume that it read the whole section, even if that's not the case. * For writing: Make use of the generated blk_pwritev() wrapper instead of manually wrapping the coroutine to simplify and save a few lines. * Adapt to changed interfaces for blk_{pread,pwrite}: * a9262f551e ("block: Change blk_{pread,pwrite}() param order") * 3b35d4542c ("block: Add a 'flags' param to blk_pread()") * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success") Those changes especially affected the qemu-img dd patches, because the context also changed, but also some of our block drivers used the functions. * Drop qemu-common.h include: it got renamed after essentially everything was moved to other headers. The only remaining user I could find for things dropped from the header between 7.0 and 7.1 was qemu_get_vm_name() in the iscsi-initiatorname patch, but it already includes the header to which the function was moved. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
@@ -813,6 +813,8 @@
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
#
# @key-password: password for keyfile (optional for format 'pbs')
#
+# @master-keyfile: PEM-formatted master public keyfile (optional for format 'pbs')
+#
# @fingerprint: server cert fingerprint (optional for format 'pbs')
#
# @backup-id: backup ID (required for format 'pbs')
update submodule and patches to 7.1.0 Notable changes: * The only big change is the switch to using a custom QIOChannel for savevm-async, because the previously used QEMUFileOps was dropped. Changes to the current implementation: * Switch to vector based methods as required for an IO channel. For short reads the passed-in IO vector is stuffed with zeroes at the end, just to be sure. * For reading: The documentation in include/io/channel.h states that at least one byte should be read, so also error out when whe are at the very end instead of returning 0. * For reading: Fix off-by-one error when request goes beyond end. The wrong code piece was: if ((pos + size) > maxlen) { size = maxlen - pos - 1; } Previously, the last byte would not be read. It's actually possible to get a snapshot .raw file that has content all the way up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any trailing zero bytes (I wrote a script to do it). Luckily, it didn't cause a real issue, because qemu_loadvm_state() is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION) section. The buffer for reading it is simply freed up afterwards and the function will assume that it read the whole section, even if that's not the case. * For writing: Make use of the generated blk_pwritev() wrapper instead of manually wrapping the coroutine to simplify and save a few lines. * Adapt to changed interfaces for blk_{pread,pwrite}: * a9262f551e ("block: Change blk_{pread,pwrite}() param order") * 3b35d4542c ("block: Add a 'flags' param to blk_pread()") * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success") Those changes especially affected the qemu-img dd patches, because the context also changed, but also some of our block drivers used the functions. * Drop qemu-common.h include: it got renamed after essentially everything was moved to other headers. The only remaining user I could find for things dropped from the header between 7.0 and 7.1 was qemu_get_vm_name() in the iscsi-initiatorname patch, but it already includes the header to which the function was moved. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
@@ -832,6 +834,7 @@
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
'*password': 'str',
'*keyfile': 'str',
'*key-password': 'str',
+ '*master-keyfile': 'str',
'*fingerprint': 'str',
'*backup-id': 'str',
'*backup-time': 'int',
update submodule and patches to 7.1.0 Notable changes: * The only big change is the switch to using a custom QIOChannel for savevm-async, because the previously used QEMUFileOps was dropped. Changes to the current implementation: * Switch to vector based methods as required for an IO channel. For short reads the passed-in IO vector is stuffed with zeroes at the end, just to be sure. * For reading: The documentation in include/io/channel.h states that at least one byte should be read, so also error out when whe are at the very end instead of returning 0. * For reading: Fix off-by-one error when request goes beyond end. The wrong code piece was: if ((pos + size) > maxlen) { size = maxlen - pos - 1; } Previously, the last byte would not be read. It's actually possible to get a snapshot .raw file that has content all the way up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any trailing zero bytes (I wrote a script to do it). Luckily, it didn't cause a real issue, because qemu_loadvm_state() is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION) section. The buffer for reading it is simply freed up afterwards and the function will assume that it read the whole section, even if that's not the case. * For writing: Make use of the generated blk_pwritev() wrapper instead of manually wrapping the coroutine to simplify and save a few lines. * Adapt to changed interfaces for blk_{pread,pwrite}: * a9262f551e ("block: Change blk_{pread,pwrite}() param order") * 3b35d4542c ("block: Add a 'flags' param to blk_pread()") * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success") Those changes especially affected the qemu-img dd patches, because the context also changed, but also some of our block drivers used the functions. * Drop qemu-common.h include: it got renamed after essentially everything was moved to other headers. The only remaining user I could find for things dropped from the header between 7.0 and 7.1 was qemu_get_vm_name() in the iscsi-initiatorname patch, but it already includes the header to which the function was moved. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
@@ -884,6 +887,9 @@
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
# migration cap if this is false/unset may lead
# to crashes on migration!
#
+# @pbs-masterkey: True if the QMP backup call supports the 'master_keyfile'
+# parameter.
+#
# @pbs-library-version: Running version of libproxmox-backup-qemu0 library.
#
##
update submodule and patches to 7.1.0 Notable changes: * The only big change is the switch to using a custom QIOChannel for savevm-async, because the previously used QEMUFileOps was dropped. Changes to the current implementation: * Switch to vector based methods as required for an IO channel. For short reads the passed-in IO vector is stuffed with zeroes at the end, just to be sure. * For reading: The documentation in include/io/channel.h states that at least one byte should be read, so also error out when whe are at the very end instead of returning 0. * For reading: Fix off-by-one error when request goes beyond end. The wrong code piece was: if ((pos + size) > maxlen) { size = maxlen - pos - 1; } Previously, the last byte would not be read. It's actually possible to get a snapshot .raw file that has content all the way up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any trailing zero bytes (I wrote a script to do it). Luckily, it didn't cause a real issue, because qemu_loadvm_state() is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION) section. The buffer for reading it is simply freed up afterwards and the function will assume that it read the whole section, even if that's not the case. * For writing: Make use of the generated blk_pwritev() wrapper instead of manually wrapping the coroutine to simplify and save a few lines. * Adapt to changed interfaces for blk_{pread,pwrite}: * a9262f551e ("block: Change blk_{pread,pwrite}() param order") * 3b35d4542c ("block: Add a 'flags' param to blk_pread()") * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success") Those changes especially affected the qemu-img dd patches, because the context also changed, but also some of our block drivers used the functions. * Drop qemu-common.h include: it got renamed after essentially everything was moved to other headers. The only remaining user I could find for things dropped from the header between 7.0 and 7.1 was qemu_get_vm_name() in the iscsi-initiatorname patch, but it already includes the header to which the function was moved. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
@@ -892,6 +898,7 @@
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
'query-bitmap-info': 'bool',
fix saving and loading dirty bitmaps in snapshots Saving dirty bitmaps from our savevm-async code didn't work, since we use a coroutine which holds the iothread mutex already (upstream savevm is sync, migration uses a thread). Release the mutex before calling the one function that (according to it's documentation) requires the lock to *not* be held: qemu_savevm_state_pending. Additionally, loading dirty bitmaps requires a call to dirty_bitmap_mig_before_vm_start after "loadvm", which the upstream savevm does explicitly afterwards - do that too. This is exposed via the query-proxmox-support property "pbs-dirty-bitmap-savevm". Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-16 19:30:22 +03:00
'pbs-dirty-bitmap-savevm': 'bool',
add PBS master key support Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-11 19:11:12 +03:00
'pbs-dirty-bitmap-migration': 'bool',
+ 'pbs-masterkey': 'bool',
'pbs-library-version': 'str' } }
##