Compare commits

...

296 Commits

Author SHA1 Message Date
a6885c7d11 Add Vitastor support 2024-02-07 01:11:30 +03:00
Thomas Lamprecht
676adda3c6 bump version to 8.1.5-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-02-02 19:41:31 +01:00
Thomas Lamprecht
4ff04bdfa5 work around stuck guest IO with iothread and VirtIO block/SCSI
This essentially repeats commit 6b7c181 ("add patch to work around
stuck guest IO with iothread and VirtIO block/SCSI") with an added
fix for the SCSI event virtqueue, which requires special handling.
This is to avoid the issue [3] that made the revert 2a49e66 ("Revert
"add patch to work around stuck guest IO with iothread and VirtIO
block/SCSI"") necessary the first time around.

When using iothread, after commits
1665d9326f ("virtio-blk: implement BlockDevOps->drained_begin()")
766aa2de0f ("virtio-scsi: implement BlockDevOps->drained_begin()")
it can happen that polling gets stuck when draining. This would cause
IO in the guest to get completely stuck.

A workaround for users is stopping and resuming the vCPUs because that
would also stop and resume the dataplanes which would kick the host
notifiers.

This can happen with block jobs like backup and drive mirror as well
as with hotplug [2].

Reports in the community forum that might be about this issue[0][1]
and there is also one in the enterprise support channel.

As a workaround in the code, just re-enable notifications and kick the
virt queue after draining. Draining is already costly and rare, so no
need to worry about a performance penalty here.

Take special care to attach the SCSI event virtqueue host notifier
with the _no_poll() variant like in virtio_scsi_dataplane_start().
This avoids the issue from the first attempted fix where the iothread
would suddenly loop with 100% CPU usage whenever some guest IO came in
[3]. This is necessary because of commit 38738f7dbb ("virtio-scsi:
don't waste CPU polling the event virtqueue"). See [4] for the
relevant discussion.

[0]: https://forum.proxmox.com/threads/137286/
[1]: https://forum.proxmox.com/threads/137536/
[2]: https://issues.redhat.com/browse/RHEL-3934
[3]: https://forum.proxmox.com/threads/138140/
[4]: https://lore.kernel.org/qemu-devel/bfc7b20c-2144-46e9-acbc-e726276c5a31@proxmox.com/

Link: https://lore.kernel.org/qemu-devel/20240202153158.788922-1-hreitz@redhat.com/
Originally-by: Fiona Ebner <f.ebner@proxmox.com>
 [ TL: Update to v2 and rebased patch series handling to v8.1.5 ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-02-02 19:35:34 +01:00
Thomas Lamprecht
12b69ed9c5 bump version to 8.1.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-02-02 19:08:16 +01:00
Fiona Ebner
5e8903f875 stable fixes for corner case in i386 emulation and crash with VNC clipboard
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-02-02 19:06:29 +01:00
Fiona Ebner
4b7975e75d update submodule and patches to QEMU 8.1.5
Most notable fixes from a Proxmox VE perspective are:

* "virtio-net: correctly copy vnet header when flushing TX"
  To prevent a stack overflow that could lead to leaking parts of the
  QEMU process's memory.
* "hw/pflash: implement update buffer for block writes"
  To prevent an edge case for half-completed writes. This potentially
  affected EFI disks.
* Fixes to i386 emulation and ARM emulation.

No changes for patches were necessary (all are just automatic context
changes).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-02-02 19:06:29 +01:00
Fiona Ebner
f366bb97ae bump version to 8.1.2-6
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-12-15 14:26:09 +01:00
Fiona Ebner
2a49e667ba Revert "add patch to work around stuck guest IO with iothread and VirtIO block/SCSI"
This reverts commit 6b7c1815e1.

The attempted fix has been reported to cause high CPU usage after
backup [0]. Not difficult to reproduce and it's iothreads getting
stuck in a loop. Downgrading to pve-qemu-kvm=8.1.2-4 helps which was
also verified by Christian, thanks! The issue this was supposed to fix
is much rarer, so revert for now, while upstream is still working on a
proper fix.

[0]: https://forum.proxmox.com/threads/138140/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-12-15 14:16:26 +01:00
Thomas Lamprecht
c6eb05a799 bump version to 8.1.2-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-12-11 16:59:16 +01:00
Fiona Ebner
dfac4f3593 pick fix for potential deadlock with QMP resize and iothread
While the patch gives bdrv_graph_wrlock() as an example where the
issue can manifest, something similar can happen even when that is
disabled. Was able to reproduce the issue with
while true; do qm resize 115 scsi0 +4M; sleep 1; done
while running
fio --name=make-mirror-work --size=100M --direct=1 --rw=randwrite \
 --bs=4k --ioengine=psync --numjobs=5 --runtime=1200 --time_based
in the VM.

Fix picked up from:
https://lists.nongnu.org/archive/html/qemu-devel/2023-12/msg01102.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-12-11 16:56:50 +01:00
Fiona Ebner
6b7c1815e1 add patch to work around stuck guest IO with iothread and VirtIO block/SCSI
When using iothread, after commits
1665d9326f ("virtio-blk: implement BlockDevOps->drained_begin()")
766aa2de0f ("virtio-scsi: implement BlockDevOps->drained_begin()")
it can happen that polling gets stuck when draining. This would cause
IO in the guest to get completely stuck.

A workaround for users is stopping and resuming the vCPUs because that
would also stop and resume the dataplanes which would kick the host
notifiers.

This can happen with block jobs like backup and drive mirror as well
as with hotplug [2].

Reports in the community forum that might be about this issue[0][1]
and there is also one in the enterprise support channel.

As a workaround in the code, just re-enable notifications and kick the
virt queue after draining. Draining is already costly and rare, so no
need to worry about a performance penalty here. This was taken from
the following comment of a QEMU developer [3] (in my debugging,
I had already found re-enabling notification to work around the issue,
but also kicking the queue is more complete).

[0]: https://forum.proxmox.com/threads/137286/
[1]: https://forum.proxmox.com/threads/137536/
[2]: https://issues.redhat.com/browse/RHEL-3934
[3]: https://issues.redhat.com/browse/RHEL-3934?focusedId=23562096&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-23562096

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-12-11 16:56:50 +01:00
Thomas Lamprecht
24d732ac0f bump version to 8.1.2-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-22 14:28:25 +01:00
Fiona Ebner
df2cc786ee add fix for vnc clipboard
This fixes the host->guest direction with noNVC as a client (and
likely others).

Reported-by: Friedrich Weber <f.weber@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Friedrich Weber <f.weber@proxmox.com>
2023-11-22 14:19:45 +01:00
Thomas Lamprecht
38726d3473 bump version to 8.1.2-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-20 10:35:52 +01:00
Fiona Ebner
89b46e17ec fix #5054: backport fix for software reset with SATA
The issue prevented FreeBSD 14 VMs with SATA disk from booting.

The commit it fixes e2a5d9b3d9c3 ("hw/ide/ahci: simplify and document
PxCI handling") is part of stable 8.1.2.

The patch was already applied to the block branch upstream:
https://lists.nongnu.org/archive/html/qemu-devel/2023-11/msg02711.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Friedrich Weber <f.weber@proxmox.com>
2023-11-20 10:35:00 +01:00
Thomas Lamprecht
33b22c3fe0 bump version to 8.1.2-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 11:55:26 +01:00
Fiona Ebner
c38e337f5d revert commit breaking VirtIO network adapters for certain versions of Windows
As reported in the community forum [0] and reproduced locally this
breaks VirtIO network adapters in (at least) the German ISO of Windows
Server 2022. The fix itself was for

> Issue is not fatal but as result acpi-index/"PCI Label ID" property
> is either not shown in device details page or shows incorrect value.

so revert and tolerate that as a stop-gap, rather than have the
devices not working at all.

[0]: https://forum.proxmox.com/threads/92094/post-605684

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-11-17 11:52:52 +01:00
Fiona Ebner
763949965f fix #4710: vma create: don't use O_DIRECT for tmpfs
The implementation of the helper is_path_tmpfs() is similar to the
existing qemu_fd_getfs() function in util/mmap-alloc.c, which
unfortunately only takes an existing fd.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-11-07 16:37:34 +01:00
Thomas Lamprecht
1807330a6f bump version to 8.1.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Thomas Lamprecht
a31ab74058 d/control: add python3-venv as build-dependency
Seems to be required since commit 81e2b198a8 ("configure: create a
python venv unconditionally").

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
b39f726f31 d/control: add versioned Breaks for qemu-server <= 8.0.6
Upstream QEMU commit 4271f40383 ("virtio-net: correctly report maximum
tx_queue_size value") made setting an invalid tx_queue_size for a
non-vDPA/vhost-user net device a hard error. Now, qemu-server before
commit 089aed81 ("cfg2cmd: netdev: fix value for tx_queue_size") did
just that, so the newer QEMU version would break start-up for most VMs
(a default vNIC configuration would be affected).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
a36bda146c add patch to avoid huge snapshot performance regression
Taking a snapshot became prohibitively slow because of the
migration_transferred_bytes() call in migration_rate_exceeded() [0].

This also applied to the async snapshot taking in Proxmox VE, so
work around the issue until it is fixed upstream.

[0]: https://gitlab.com/qemu-project/qemu/-/issues/1821

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
03ff63aa61 add patch to disable graph locking
There are still some issues with graph locking, e.g. deadlocks during
backup canceling [0] and initial attempts to fix it didn't work [1].
Because the AioContext locks still exist, it should still be safe to
disable graph locking.

[0]: https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg00729.html
[1]: https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg06905.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
10e1093325 update submodule and patches to QEMU 8.1.2
Bigger notable changes:

* Commit 1a30b0f5d7 ("block: .bdrv_open is non-coroutine and
  unlocked") broke the PVE backup patches, in particular setting up
  the backup dump block driver, because bdrv_new_open_driver() cannot
  be called from a coroutine. To fix it, bdrv_co_open() is used
  instead, and while it's a much more involved function, the result
  should be essentially the same. The only difference I noticed is
  that the BDRV_O_ALLOW_RDWR flag is also set in the resulting bds
  (block driver state), but that shouldn't hurt.

Smaller notable changes:

* aio_set_fd_handler() dropped its 'is_external' parameter stating
  that all callers now pass false in 60f782b6b7 ("aio: remove
  aio_disable_external() API"). The calls in the PVE patches also
  passed false, so just drop the parameter too.

* global_state_store() does not have a return value anymore, so the
  user in the PVE savevm-async patch was adapted. For context, see
  c33f1829f8 ("migration: never fail in global_state_store()").

* Renames affecting the PVE savevm-async patch:
  migrate_use_block() -> migrate_block() and ram_counters -> mig_stats
  9d4b1e5f22 ("migration: Move migrate_use_block() to options.c")
  aff3f6606d ("migration: Rename ram_counters to mig_stats")

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
0d9c737d61 buildsys: use QEMU's keycodemapdb again
instead of the split-out version that was last updated for QEMU 6.0.
This reverts the relevant part of 6838f03 ("bump version to 2.11.1-1")
which doesn't state a reason why the splitting was done. If something
breaks, we can still re-do it and document the reason this time.

Alternatively, it would be necessary to adapt the paths, because
keycodemapdb lives in subprojects/ rather than ui/ since QEMU commit
c53648abba ("meson: use subproject for keycodemapdb").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
a6ddea7ef7 buildsys: fixup submodule target
It's not enough to initialize the submodules anymore, as some got
replaced by wrap files, see QEMU commit 2019cabfee ("meson:
subprojects: replace submodules with wrap files").

Download the subprojects during initialization of the QEMU submodule,
so building (without the automagical --enable-download) can succeeed
afterwards.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Fiona Ebner
89520c1cd0 d/rules: use disable-download option instead of git-submodules=ignore
See the following QEMU commits for reference:
0c5f3dcbb2 ("configure: add --enable-pypi and --disable-pypi")
ac4ccac740 ("configure: rename --enable-pypi to --enable-download, control subprojects too")
6f3ae23b29 ("configure: remove --with-git-submodules=") removed

The last one removed the option and the closest thing to
git-submodule=ignore is using disable-download. Which will then just
verify that the submodules are present.

Building now will require running either
* Running 'meson subprojects download' in the qemu submodule first.
* Using --enable-download, but then the submodules would be downloaded
  for each build (if not already downloaded in the submodule first)
  and it's just a bit too surprising if downloads happen during build.

The disable-download option will also disable automatic downloading of
missing Python modules from PyPI. Hopefully, it's enough to add them
as Debian build dependencies when required.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-24 15:01:23 +02:00
Thomas Lamprecht
eca4daeeed bump version to 8.0.2-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-04 08:33:39 +02:00
Fiona Ebner
816077299c fix #2874: SATA: avoid unsolicited write to sector 0 during reset
If there is a pending DMA operation during ide_bus_reset(), the fact
that the IDEstate is already reset before the operation is canceled
can be problematic. In particular, ide_dma_cb() might be called and
then use the reset IDEstate which contains the signature after the
reset. When used to construct the IO operation this leads to
ide_get_sector() returning 0 and nsector being 1. This is particularly
bad, because a write command will thus destroy the first sector which
often contains a partition table or similar.

Upstream discussion:
https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg04239.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-26 11:30:22 +02:00
Fiona Ebner
ef3308db71 vma: avoid compiler warning about incompatible pointer type
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-08 11:18:30 +02:00
Filip Schauer
0ff45eb23e backup: Fix spelling error in function name
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
[FE: fixup patch context]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-08 11:13:04 +02:00
Thomas Lamprecht
6c5563e30b bump version to 8.0.2-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-06 17:04:04 +02:00
Fiona Ebner
9e0186f289 backup: drop broken BACKUP_FORMAT_DIR
Since upstream QEMU 8.0, it's no longer possible to call
bdrv_img_create() from a coroutine anymore, meaning a backup with the
directory format would crash the QEMU instance.

The feature is only exposed via the monitor and was intended to be
experimental. There were no user reports about the breakage and it
only was noticed during the rebase for QEMU 8.1, because other parts
of the backup code needed adaptation and I decided to check the
BACKUP_FORMAT_DIR case too.

It should not stay in a broken state of course, but avoid the
maintenance cost and just make it a removed feature for Proxmox VE 8
retroactively.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-06 16:59:12 +02:00
Fiona Ebner
0cffb504e7 backup: create jobs in a drained section
With the drive-backup QMP command, upstream QEMU uses a drained
section for the source drive when creating the backup job. Do the same
here to avoid subtle bugs.

There, the drained section extends until after the job is started, but
this cannot be done here for multi-disk backups (could at most start
the first job). The important thing is that the cbw
(copy-before-write) node is in place and the bcs (block-copy-state)
bitmap is initialized, which both happen during job creation (ensured
by the "block/backup: move bcs bitmap initialization to job creation"
PVE patch).

One such bug is one reported in the community forum [0], where using a
drive with iothread can lead to an overlapping block-copy request and
consequently an assertion failure. The block-copy code relies on the
bcs bitmap to determine if a request for a certain range can be
created. Each time a request is created, it resets the bcs bitmap at
that range to indicate that it's being handled.

The duplicate request can happen as follows:
Thread A attaches the cbw node
Thread B creates a request and resets the bitmap at that range
Thread A clears the bitmap and merges it with the PBS bitmap
The merging can lead to the bitmap being set again at the range of
the previous request, so the block-copy code thinks it's fine to
create a request there.
Thread B creates another requests at an overlapping range before the
other request is finished.

The drained section ensures that nothing else can interfere with the
bcs bitmap between attaching the copy-before-write block node and
initialization of the bitmap.

[0]: https://forum.proxmox.com/threads/133149/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-06 16:59:12 +02:00
Fiona Ebner
f7eed6caa1 regenerate patch stats
Apparently wasn't correct in 0cff91a ("fix #1534: vma: Add extract
filter for disk images").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-06 16:59:12 +02:00
Filip Schauer
0cff91a000 fix #1534: vma: Add extract filter for disk images
Add a filter to the "vma extract" command. A comma seperated list of
disk images that should be extracted can be passed with the "-d" option.

Example to extract an IDE drive and an SCSI drive from vzdump.vma:

vma extract vzdump.vma -d "drive-ide0,drive-scsi0" extractdir

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
2023-08-30 10:40:51 +02:00
Fiona Ebner
6cadf3677d bump version to 8.0.2-5
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-16 11:56:49 +02:00
Fiona Ebner
5f9cb29c3a backup: trim heap after finishing
Reported in the community forum [0]. By default, there can be large
amounts of memory left assigned to the QEMU process after backup.
Likely because of fragmentation, it's necessary to explicitly call
malloc_trim() to tell glibc that it shouldn't keep all that memory
resident for the process.

QEMU itself already does a malloc_trim() in the RCU thread, but that
code path might not be reached (or not for a long time) under usual
operation. The value of 4 MiB for the argument was also copied from
there.

Example with the following configuration:
> agent: 1
> boot: order=scsi0
> cores: 4
> cpu: x86-64-v2-AES
> ide2: none,media=cdrom
> memory: 1024
> name: backup-mem
> net0: virtio=DA:58:18:26:59:9F,bridge=vmbr0,firewall=1
> numa: 0
> ostype: l26
> scsi0: rbd:base-107-disk-0/vm-106-disk-1,size=4302M
> scsihw: virtio-scsi-pci
> smbios1: uuid=b2d4511e-8d01-44f1-afd6-9581b30c24a6
> sockets: 2
> startup: order=2
> virtio0: lvmthin:vm-106-disk-1,iothread=1,size=1G
> virtio1: lvmthin:vm-106-disk-2,iothread=1,size=1G
> virtio2: lvmthin:vm-106-disk-3,iothread=1,size=1G
> vmgenid: 0a1d8751-5e02-449d-977e-c0160e900231

Before the change:

> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS:	  370948 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS:	 2114964 kB

After the change:

> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS:	  398788 kB
> root@pve8a1 ~ # vzdump 106 --storage pbs
> (...)
> INFO: Backup job finished successfully
> root@pve8a1 ~ # grep VmRSS /proc/$(cat /var/run/qemu-server/106.pid)/status
> VmRSS:	  424356 kB

[0]: https://forum.proxmox.com/threads/131339/

Co-diagnosed-by: Friedrich Weber <f.weber@proxmox.com>
Co-diagnosed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-16 11:50:12 +02:00
Fiona Ebner
c36e3f9d17 refresh patch context
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-16 11:50:08 +02:00
Filip Schauer
b8b4ce0480 Add format attributes to function candidates
Add format attributes to functions that take printf-like arguments. This
provides additional compile-time checking that the correct parameters
are passed to the functions.

This fixes compiler warnings generated by the -Wsuggest-attribute=format
flag.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
2023-08-08 09:08:48 +02:00
Fiona Ebner
df47146afe add patch fixing fd leak for vhost
Each pause+resume operation (which is also done as part of taking a VM
snapshot) would increase the number of open file descriptors by the
number of vhost devices (e.g. network devices by default). This could
lead to crashes during backup and surely other issues once the system
limit (default 1024) was reached [0].

[0]: https://forum.proxmox.com/threads/131603/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-03 17:40:13 +02:00
Fabian Grünbichler
d9cbfafeeb bump version to 8.0.2-4
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-07-28 12:59:10 +02:00
Fiona Ebner
5919ec1446 add patch fixing resume for snapshot and hibernate with drive with iothread and a dirty bitmap
Not difficult to run into, just have a drive with iothread, take a PBS
backup and then take a snapshot or hibernate. Resuming will fail with
> qemu: qemu_mutex_unlock_impl: Operation not permitted
because of not acquiring the correct AioContext first.

Migration is not affected, because it runs in coroutine context.

Reported in the community forum:
https://forum.proxmox.com/threads/129899/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-07-28 12:00:50 +02:00
Thomas Lamprecht
409db0cd7b bump version to 8.0.2-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-15 13:59:12 +02:00
Fiona Ebner
ea7662074d fix checks for drive mirror with bitmap
The QAPI change for QEMU 8.0 dropped redundant has_foo parameters, but
in the blockdev_mirror_common() function (which is not part of the
QAPI itself but called from there) the argument pair was has_bitmap
and bitmap_name rather than has_bitmap and bitmap.

Reported-by: Aaron Lauterer <a.lauterer@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-15 13:55:22 +02:00
Fiona Ebner
d847446186 regenerate patches
There's still some context changes not covered by earlier series. No
functional change intended.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-15 13:55:22 +02:00
Thomas Lamprecht
3aaa855e5c bump version to 8.0.2-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 07:58:59 +02:00
Fiona Ebner
99f9ce2cd2 drop deprecated custom drive snapshot QMP commands
They are not required anymore since qemu-server >= 5.0-36.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:35:53 +02:00
Fiona Ebner
a816d2969e drop patch for custom get_link_status QMP command
There doesn't seem to be any Proxmox VE code using this.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:35:40 +02:00
Thomas Lamprecht
0e9a7bfda2 bump version to 8.0.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-06 16:35:20 +02:00
Fiona Ebner
a39364b9d1 update reentrancy patches to version in upstream git
The previous version was picked from the mailing list and still had
an object_dynamic_cast call in a hot path, which is avoided with the
version that landed in git.

Also adds a few more exceptions for devices that need reentrancy.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 16:32:38 +02:00
Fiona Ebner
0f693c2cab update submodule and patches to QEMU 8.0.2
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 16:32:38 +02:00
Thomas Lamprecht
88b1550dfb buildsys: remove edk2 source tree when assembling build-dir
we ship it via pve-edk2-firmware anyway and it only results in bigger
source tar balls and lintian yelling at us due to edk2 not being the
simplest repo to ensure DFSG compat.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-24 10:37:10 +02:00
Thomas Lamprecht
bd3c1fa525 bump version to 8.0.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-23 14:09:12 +02:00
Thomas Lamprecht
de2dde2da9 buildsys: avoid handling noopt locally, rather extend CFLAGS
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-23 14:09:12 +02:00
Thomas Lamprecht
04e0262e2e d/rules: add identation for configure switches for readability
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:23:55 +02:00
Thomas Lamprecht
d3c2ae9683 d/control: drop obsolete build dependencies
drop autotools-dev, texi2html and texinfo build dependencies, they
are not used and have no effect

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:11:33 +02:00
Thomas Lamprecht
d0603efa38 buildsys: auto-generate dbgsym package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
db5d2a4b77 squash related patches
where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.

If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.

The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.

No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.

The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.

The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.

The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
b64c4dec1c PVE backup: don't call no_co_wrapper function from coroutine
Namely, pvebackup_co_prepare() needs to call bdrv_co_open() rather
than bdrv_open(), because it is a coroutine itself.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
53b56ca781 add stable patches for 8.0.0
Changes to other patches are all just metadata/context changes except
for pvebackup_co_prepare() needing to call bdrv_co_unref() rather than
bdrv_unref(), because it is a coroutine itself. This is documented in
d6ee2e324e ("block-coroutine-wrapper: Introduce no_co_wrapper"). The
change is necessary, because one of the stable fixes converts
bdrv_unref and blk_unref into no_co_wrappers (in preparation for a
second patch to fix a hang with the block resize QMP command).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
bf251437e9 update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:

* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.

* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.

* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.

* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.

* Async snapshot-related changes:
  - The pending querying got adapted to the above-mentioned split and
  a patch is added to optimize it/make it more similar to what
  upstream code does.
  - Added initialization of the compression counters (for
    future-proofing).
  - It's necessary the hold the BQL (big QEMU lock = iothread mutex)
  during the setup phase, because block layer functions are used there
  and not doing so leads to racy, hard-to-debug crashes or hangs. It's
  necessary to change some upstream code too for this, a version of
  the patch "migration: for snapshots, hold the BQL during setup
  callbacks" is intended to be upstreamed.
  - Need to take the bdrv graph read lock before flushing.

* hmp_info_balloon was moved to a different file.

* Needed to include a new headers from time to time to still get the
correct functions.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
fb818ea5b9 d/rules: drop virtiofsd switch
virtiofsd is no longer part of QEMU 8.0. It got replaced by a separate
implementation written in Rust, which will be its own package.

See QEMU commit 0aaf44776e ("Merge tag 'pull-virtiofs-20230216b' of
https://gitlab.com/dagrh/qemu into staging").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:09:14 +02:00
Thomas Lamprecht
3c995a426d makefile: convert to use simple parenthesis
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:09:14 +02:00
Thomas Lamprecht
be7ce325c7 d/lintian-overrides: ignore groff line breakage/adjustment warnings
not much we can do here anyway..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:09:14 +02:00
Thomas Lamprecht
19b4b4c50f d/lintian-overrides: sort
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:09:14 +02:00
Thomas Lamprecht
590adba81a d/parse-machines: produce stable json output
Enabling the "canonical" option the keys will be sorted, improving
build reproducibility.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 15:09:14 +02:00
Fiona Ebner
abb04bb627 d/control: define compat level via build-depends and raise to 13
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
6facdf3a08 also exclude hppa-firmware.img ROM from build
We don't use it and with debhelper compat level >= 11, the switch
from detecting files for strip through patters to checking for an ELF
header caused a build failure with the hppa-firmware.img ROM, as some
tools cannot cope with HP PARISC files.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
cb2b3190a4 move cleanup of unused ROMs from d/rules to build-dir generation
this way we save a bit of space and should make build also slightly
faster, otherwise nothing should change.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
2e416ad9d5 d/rules: fix debian-rules-missing-required-target
until we switch fully over to the dh sequencer

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
d80ca49db8 d/rules: cleanup cruft and use dpkg makefile fragements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
d65b507d3f buildsys: update lintian overrides
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 13:29:59 +02:00
Thomas Lamprecht
98fd8612cb add .gitignore file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 12:05:14 +02:00
Thomas Lamprecht
4f56d29218 buildsys: use shorter variable name $@ in $(BUILDIR) target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 12:05:14 +02:00
Thomas Lamprecht
cd148033f3 buildsys: only run lintian for phony dsc target
This allows the sbuild to start much faster (lintian takes ~ minutes
for such big packages), and that without loss as sbuild will run
lintian on both binary and source package anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 12:05:14 +02:00
Thomas Lamprecht
92c6d84f6a d/control: avoid versioned build-dependcies with a -1 revision
no effect besides making it harder to build this for an eventual
backport.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 12:05:14 +02:00
Thomas Lamprecht
b8af8dd4fa debian: normalize packaging files with wrap-and-sort -tkn
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-22 12:05:13 +02:00
Fiona Ebner
6eb3e31968 d/rules: fix comment about when clean target is executed
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:51:16 +02:00
Fiona Ebner
c913853be7 d/rules: move copying config.guess and config.sub to config.status target
It causes problems when done as part of the clean target when building
the dsc with the following error due to the additional files:
dpkg-source: error: aborting due to unexpected upstream changes

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:51:16 +02:00
Fiona Ebner
4fc4b533b5 buildsys: fix lintian overrides
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007002 for more
information.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:51:16 +02:00
Fiona Ebner
023b916380 d/rules: set job flag for make based on DEB_BUILD_OPTIONS
Copied from Debian's QEMU package's d/rules. Otherwise, ninja will end
up using only a single job (in Debian Bookworm/Proxmox VE 8).

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:51:16 +02:00
Fiona Ebner
19a11f24a5 buildsys: expand clean target
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ T: remove all tarballs for a package and any .deb ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:50:56 +02:00
Fiona Ebner
030fa1db4b buildsys: create build directory atomically
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:50:56 +02:00
Fiona Ebner
2d17b4b4d9 buildsys: add sbuild convenience target
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:50:56 +02:00
Fiona Ebner
280d157f1c buildsys: add dsc target
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:50:56 +02:00
Fiona Ebner
f6be0ca51a buildsys: derive upload dist automatically
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-21 15:50:56 +02:00
Thomas Lamprecht
93d558c1ee bump version to 7.2.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-17 15:48:12 +01:00
Fiona Ebner
e752bbe5e2 cherry-pick TCG-related stable fixes for 7.2
When turning off the "KVM hardware virtualization" checkbox in Proxmox
VE, the TCG accelerator is used, so these fixes are relevant then.

The first patch is included to allow cherry-picking the others without
changes.

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-17 15:46:20 +01:00
Thomas Lamprecht
018ef788b3 bump version to 7.2.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-17 12:12:02 +01:00
Fiona Ebner
72fc94c0c6 add patch fixing ACPI CPU hotplug issue with TCG
Required for the debian/edk2-vars-generator.py script in the
pve-edk2-firmware repository when building the edk2-stable202302
release. Without this patch, the QEMU process spawned by the script
would hang indefinietly.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-17 12:06:22 +01:00
Thomas Lamprecht
09186f4b6e bump version to 7.2.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-13 17:42:52 +01:00
Fiona Ebner
ffda59f626 add patches to fix regression with LSI SCSI controller
The patch 0008-memory-prevent-dma-reentracy-issues.patch introduced a
regression for the LSI SCSI controller leading to boot failures [0],
because, in its current form, it relies on reentrancy for a particular
ram_io region.

[0]: https://forum.proxmox.com/threads/123843

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-13 17:36:22 +01:00
Fiona Ebner
3c4f941ac7 add more stable fixes
The patches were selected from the recent "Patch Round-up for stable
7.2.1" [0]. Those that should be relevant for our supported use-cases
(and the upcoming nvme use-case) were picked. Most of the patches
added now have not been submitted to qemu-stable before.

The follow-up for the virtio-rng-pci migration fix will break
migration between versions with the fix and without the fix when a
virtio-pci-rng(-non)-transitional device is used. Luckily Proxmox VE
only uses the virtio-pci-rng device, and this was fixed by
0006-virtio-rng-pci-fix-migration-compat-for-vectors.patch which was
applied before any public version of Proxmox VE's QEMU 7.2 package was
released.

[0]: https://lists.nongnu.org/archive/html/qemu-stable/2023-03/msg00010.html
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2162569

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-13 17:36:19 +01:00
Fiona Ebner
3a94e1a186 fixup patch "ide: avoid potential deadlock when draining during trim"
The patch was incomplete and (re-)introduced an issue with a potential
failing assertion upon cancelation of the DMA request.

There is a patch on qemu-devel now[0], and it's the same as this one
code-wise (except for comments). But the discussion is still ongoing.
While there shouldn't be a real issue with the patch, there might be
better approaches. The plan is to use this as a stop-gap for now and
pick up the proper solution once it's ready.

[0]: https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg03325.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-13 17:36:19 +01:00
Thomas Lamprecht
67cae45f41 bump version to 7.2.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-08 14:32:22 +01:00
Fiona Ebner
58659169de add patch to avoid potential deadlock with trim for IDE/SATA and draining
In particular, the deadlock can occur, together with unlucky timing
between the QEMU threads, when the guest is issuing trim requests
during the start of a backup operation.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ T: resolve trivial merge conflict in series file ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-08 14:22:36 +01:00
Fiona Ebner
10691e04e9 add patch fixing Linux boot failures with megasas SCSI
A regression in 7.2 and easily reproduced.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-03-07 19:50:12 +01:00
Thomas Lamprecht
09723b9298 bump version to 7.2.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-02-21 13:50:08 +01:00
Fiona Ebner
00e2507aac add fix for iscsi double free issue leading to crashes
Reported here[0] and here[1].

[0]: https://gitlab.com/qemu-project/qemu/-/issues/1378
[1]: https://forum.proxmox.com/threads/122776/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-21 13:49:19 +01:00
Fiona Ebner
e7e5f63573 add patch fixing DMA reentrancy issues
that could lead to use-after-frees and stack overflows with a
malicious (or buggy) guest. See [0] for a good summary:

[0]: https://lore.kernel.org/qemu-devel/CAFEAcA_23vc7hE3iaM-JVA6W38LK4hJoWae5KcknhPRD5fPBZA@mail.gmail.com

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-21 10:18:35 +01:00
Fiona Ebner
1688b43738 QMP backup: use correct errno when getting blockdrive length fails
di->size would only be set later. The errno is minus the return value
from the function.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-21 09:19:16 +01:00
Fiona Ebner
eee064d954 savevm-async: keep more free space when entering final stage
In qemu-server, we already allocate 2 * $mem_size + 500 MiB for driver
state (which was 32 MiB long ago according to git history). It seems
likely that the 30 MiB cutoff in the savevm-async implementation was
chosen based on that.

In bug #4476 [0], another issue caused the iteration to not make any
progress and the state file filled up all the way to the 30 MiB +
pending_size cutoff. Since the guest is not stopped immediately after
the check, it can still dirty some RAM and the current cutoff is not
enough for a reproducer VM (was done while bug #4476 still was not
fixed), dirtying memory with
> stress-ng -B 2 --bigheap-growth 64.0M'
After entering the final stage, savevm actually filled up the state
file completely, leading to an I/O error. It's probably the same
scenario as reported in the bug report, the error message was fixed in
commit a020815 ("savevm-async: fix function name in error message")
after the bug report.

If not for the bug, the cutoff will only be reached by a VM that's
dirtying RAM faster than can be written to the storage, so increase
the cutoff to 100 MiB to have a bigger chance to finish successfully,
while still trying to not increase downtime too much for
non-hibernation snapshots.

[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=4476

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-21 08:39:08 +01:00
Fiona Ebner
8051a24b5f fix #4476: savevm-async: avoid looping without progress
when pend_postcopy is large. By definition, pend_postcopy won't
decrease when iterating, so a value larger than the cutoff of 400000
would lead to essentially empty iterations, filling up the state file
until only 30 MiB + pending_size remain and the second half of the
check would trigger.

Avoid this, by not considering pend_postcopy for the cutoff to enter
the final phase.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-21 08:39:08 +01:00
Fiona Ebner
ade9f50160 d/rules: add note explaining why using noopt doesn't currenlty work
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-14 10:04:21 +01:00
Fiona Ebner
0fde60fd10 d/rules: add missing export for CFLAGS
Otherwise, they don't affect the build of QEMU at all.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-02-14 10:04:21 +01:00
Thomas Lamprecht
d82c5eb632 bump version to 7.2.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-27 09:37:53 +01:00
Fiona Ebner
d5f6ef56f0 add patch to fix issue with VirtIO disk using detect-zeroes=unmap
Affects Proxmox VE, when the discard disk setting is used for a
VirtIO disk.

Upstream bug report:
https://gitlab.com/qemu-project/qemu/-/issues/1404

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-27 09:36:41 +01:00
Fabian Grünbichler
658cba46ee d/control: also conflict with "qemu-system-data"
it ships files also shipped by our qemu package, switching from Debian qemu to
ours doesn't work without manual intervention otherwise..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-26 10:55:37 +01:00
Fiona Ebner
a02081501a savevm-async: fix function name in error message
which also makes it distinguishable from the other
"qemu_savevm_state_iterate error" message.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-24 17:08:54 +01:00
Thomas Lamprecht
baf4e3132d bump version to 7.2.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-12 13:13:23 +01:00
Fiona Ebner
48c307550a add regression fix for migration with virtio-rng device
between QEMU less than 7.2 and QEMU 7.2 without the fix (both
directions are affected).

As mentioned in the patch message, this fix itself will break
migration between QEMU 7.2 and QEMU 7.2 with the fix (in both
directions, if a virtio-rng device is attached), but this is fine,
because no pve-qemu-kvm package with QEMU 7.2 has been publicly
released yet.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-12 13:10:19 +01:00
Thomas Lamprecht
89fdfe8975 bump version to 7.2.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-10 15:47:52 +01:00
Fiona Ebner
f64132208a cherry-pick stable fixes for 7.2
Two for virtio-mem and one for vIOMMU. Both features are not yet
exposed in PVE's qemu-server, but planned to be added.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-10 15:42:28 +01:00
Fiona Ebner
271ac0a8a7 add QAPI naming exceptions in patches introducing them
Avoids a patch and is required to compile when not all patches are
applied. No functional change is intended.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-10 15:42:16 +01:00
Fiona Ebner
f4ed54ec37 d/control: drop outdated jemalloc dependencies
Commit 3d785ea ("disable jemalloc") disabled jemalloc support, so
these are not needed anymore.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-21 13:52:16 +01:00
Fiona Ebner
2277182712 d/control: add libslirp-dev as a build dependency
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-21 13:52:16 +01:00
Fiona Ebner
0906461df0 d/rules: enable slirp again
Commit d03e1b3 ("update submodule and patches to 7.2.0") argued that
slirp is not explicitly supported in PVE, but that is not true. In
qemu-server, user networking is supported (via CLI/API) when no bridge
is set on a virtual NIC. So slirp needs to stay to keep such NICs
working.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-21 13:52:16 +01:00
Wolfgang Bumiller
29bee92c59 bump version to 7.2.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-16 13:23:29 +01:00
Fiona Ebner
82640bb859 d/rules: explicitly disable building slirp
Otherwise, it depends on whether libslirp-devel is installed or not.
See the previous commit message for more context.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-16 11:47:25 +01:00
Fiona Ebner
d03e1b3ce3 update submodule and patches to 7.2.0
User-facing breaking change:

The slirp submodule for user networking got removed. It would be
necessary to add the --enable-slirp option to the build and/or install
the appropriate library to continue building it. Since PVE is not
explicitly supporting it, it would require additionally installing the
libslirp0 package on all installations and there is *very* little
mention on the community forum when searching for "slirp" or
"netdev user", the plan is to only enable it again if there is some
real demand for it.

Notable changes:

* The big change for this release is the rework of job locking, using
  a job mutex and introducing _locked() variants of job API functions
  moving away from call-side AioContext locking. See (in the qemu
  submodule) commit 6f592e5aca ("job.c: enable job lock/unlock and
  remove Aiocontext locks") and previous commits for context.

  Changes required for the backup patches:
  * Use WITH_JOB_LOCK_GUARD() and call the _locked() variant of job
    API functions where appropriate (many are only availalbe as
    a _locked() variant).
  * Remove acquiring/releasing AioContext around functions taking the
    job mutex lock internally.

  The patch introducing sequential transaction support for jobs needs
  to temporarily unlock the job mutex to call job_start() when
  starting the next job in the transaction.

* The zeroinit block driver now marks its child as primary.

  The documentation in include/block/block-common.h states:
  > Filter node has exactly one FILTERED|PRIMARY child, and may have
  > other children which must not have these bits

  Without this, an assert will trigger when copying to a zeroinit target
  with qemu-img convert, because bdrv_child_cb_attach() expects any
  non-PRIMARY child to be not FILTERED:
  > qemu-img convert -n -p -f raw -O raw input.raw zeroinit:output.raw
  > qemu-img: ../block.c:1476: bdrv_child_cb_attach: Assertion
  > `!(child->role & BDRV_CHILD_FILTERED)' failed.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-16 11:47:20 +01:00
Thomas Lamprecht
55e33a045e bump version to 7.1.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-22 09:21:10 +01:00
Thomas Lamprecht
8a38e1da9e cherry-pick "block/block-backend: blk_set_enable_write_cache is IO_CODE"
albeit I was short from disarming that GLOBAL_STATE_CODE assert
completely, as its just bogus to assert that on runtime for a lot of
call sites, rather it should be verified on compilation (function
coloring with attributes and maybe a compiler plugin).

But, as this is already solved upstream lets take in that patch.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-22 09:19:00 +01:00
Thomas Lamprecht
3b3d5516ee bump version to 7.1.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-28 10:27:54 +02:00
Thomas Lamprecht
509409fb64 init: daemonize: defuse PID file resolve error to warning
fixes file restore, where we actively unlink the PID file of the
transient VM ourself after opening it - while we use it only for
tracking when the QEMU process itself has finished start up, it's
easier and cleaner to fix this regression now, than to rework that to
something that doesn't depends on the PID file at all.

Applying Fiona's patch as patch-patch tracked under extra, as I
expect that something similar to this gets accepted upstreamed.

Link: https://lists.proxmox.com/pipermail/pve-devel/2022-October/054448.html
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-28 10:22:26 +02:00
Wolfgang Bumiller
bf03cd367f bump version to 7.1.0-2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-18 15:35:09 +02:00
Fiona Ebner
0af826b448 savevm async IO channel: channel writev: fix return value in error case
The documentation in include/io/channel.h states that -1 or
QIO_CHANNEL_ERR_BLOCK should be returned upon error. Simply passing
along the return value from the blk-functions has the potential to
confuse the call sides. Non-blocking mode is not implemented
currently, so -1 it is.

The "return ret" was mistakenly left over from the previous
QEMUFileOps based implementation. Also, use error_setg_errno(), since
the blk(_co)_p{readv,writev} functions return errno codes.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-18 15:32:13 +02:00
Wolfgang Bumiller
ed23707ed7 bump version to 7.1.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-14 14:55:53 +02:00
Fiona Ebner
4e1935c2c9 {alloc track, pbs} block driver: bdrv_co_preadv: adapt return values
to be in-line with what other implementations in QEMU do. Commit
1d39c7098bbfa6862cb96066c4f8f6735ea397c5 mentions the EIO bit and
the function is expected to return 0 upon success (see other
implementations).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 14:52:36 +02:00
Fiona Ebner
a262e9642b savevm async: cleaner initialization of target_close_wait member
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 14:52:34 +02:00
Fiona Ebner
73912aee39 cherry-pick upstream fixes for 7.1.0
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 14:52:32 +02:00
Fiona Ebner
5b15e2ecaf update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
  savevm-async, because the previously used QEMUFileOps was dropped.

  Changes to the current implementation:

  * Switch to vector based methods as required for an IO channel. For
    short reads the passed-in IO vector is stuffed with zeroes at the
    end, just to be sure.

  * For reading: The documentation in include/io/channel.h states that
    at least one byte should be read, so also error out when whe are
    at the very end instead of returning 0.

  * For reading: Fix off-by-one error when request goes beyond end.

    The wrong code piece was:
    if ((pos + size) > maxlen) {
        size = maxlen - pos - 1;
    }

    Previously, the last byte would not be read. It's actually
    possible to get a snapshot .raw file that has content all the way
    up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
    trailing zero bytes (I wrote a script to do it).

    Luckily, it didn't cause a real issue, because qemu_loadvm_state()
    is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
    section. The buffer for reading it is simply freed up afterwards
    and the function will assume that it read the whole section, even
    if that's not the case.

  * For writing: Make use of the generated blk_pwritev() wrapper
    instead of manually wrapping the coroutine to simplify and save a
    few lines.

* Adapt to changed interfaces for blk_{pread,pwrite}:
  * a9262f551e ("block: Change blk_{pread,pwrite}() param order")
  * 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
  * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
  Those changes especially affected the qemu-img dd patches, because
  the context also changed, but also some of our block drivers used
  the functions.

* Drop qemu-common.h include: it got renamed after essentially
  everything was moved to other headers. The only remaining user I
  could find for things dropped from the header between 7.0 and 7.1
  was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
  already includes the header to which the function was moved.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 14:52:29 +02:00
Wolfgang Bumiller
2775b2e378 bump version to 7.0.0-4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-10 11:56:27 +02:00
Wolfgang Bumiller
ed01236593 add patch: PVE Backup: allow passing max-workers performance setting
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-10 11:55:15 +02:00
Fiona Ebner
2b259b70ec d/rules: add revision to package version
This version string can be queried with $BINARY --version as well as
the query-version QMP command.

Useful for qemu-server to be able to report the running QEMU version
exactly. Could also be used to version guard against features as an
alternative to the query-proxmox-support QMP command.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-10 11:26:47 +02:00
Thomas Lamprecht
a186335be5 bump version to 7.0.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-08-30 12:54:12 +02:00
Fiona Ebner
1976ca4607 savevm-async: set SAVE_STATE_DONE when closing state file was successful
Without this change, it's necessary to send a second savevm-end QMP
command after aborting a snaphsot, before a new savevm-start QMP
command can succeed.

In process_savevm_finalize(), no longer set an error in the abort
scenario. If there already is another error, there's no need to
override it. If canceling was done intentionally, qmp_savevm_end()
is responsible for setting the state now.

Reported-by: Mira Limbeck <m.limbeck@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-08-19 09:44:16 +02:00
Fiona Ebner
563c592898 savevm-async: avoid segfault when aborting snapshot
Reported in the community forum[0].

For 6.1.0, there were a few changes to the coroutine-sleep API, but
the adaptations in f376b2b ("update and rebase to QEMU v6.1.0") made
a mistake.

Currently, target_close_wait is NULL when passed to
qemu_co_sleep_ns_wakeable(), which further passes it to
qemu_co_sleep(), but there, it is dereferenced when trying to access
the 'to_wake' member:

> Thread 1 "kvm" received signal SIGSEGV, Segmentation fault.
> qemu_co_sleep (w=0x0) at ../util/qemu-coroutine-sleep.c:57

To fix it, create a proper struct and pass its address instead. Also
call qemu_co_sleep_wake unconditionally, because the NULL check (for
the 'to_wake' member) is done inside the function itself.

This patch is based on what the QEMU commits introducing the changes
to the coroutine-sleep API did to the callers in QEMU:
eaee072085 ("coroutine-sleep: allow qemu_co_sleep_wake that wakes nothing")
29a6ea24eb ("coroutine-sleep: replace QemuCoSleepState pointer with struct in the API")

[0]: https://forum.proxmox.com/threads/112130/

Tested-by: Mira Limbeck <m.limbeck@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-08-19 09:44:14 +02:00
Thomas Lamprecht
1de53d8a45 bump version to 7.0.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-20 09:17:13 +02:00
Fabian Ebner
0e88ec19db add two more stable patches
For the io_uring patch, it's not very clear which configurations can
trigger it, but it should be rather uncommon. See qemu commit
be6a166fde652589761cf70471bcde623e9bd72a for a bit more information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-07-19 17:22:10 +02:00
Wolfgang Bumiller
9ee866b2e9 bump version to 7.0.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 11:08:36 +02:00
Fabian Ebner
14ed554660 cherry-pick upstream fixes for 7.0.0
coming in via qemu-stable (except for the vdmk fix, which was tagged
for-7.0 on the qemu-devel list, but didn't make it into the release).

Also took the chance to switch the gluster fix to the version that
made it into upstream.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 12:29:30 +02:00
Fabian Ebner
eba403aafc d/rules: adapt to changed opensbi riscv filenames in 7.0.0
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 12:29:28 +02:00
Fabian Ebner
b2685aee04 d/rules: drop outdated configure flags
See QEMU commits 9e8be4c546ce8469ca9702715bf8f198d604b685 and
a5730b8bd3675f484ed0eacea052452048eeb35d for more information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 12:29:25 +02:00
Fabian Ebner
dc9827a6a4 update submodule and patches to 7.0.0
Only very minor changes needed:
* Most patches in extra (or some version of them) are part of 7.0.0.
* aio_set_fd_handler got an extra parameter, but can just pass NULL
  like we did for the related 'poll' parameter. See QEMU commit
  826cc32423db2a99d184dbf4f507c737d7e7a4ae for more.
* Add include for qemu/memalign.h in vma.c and vma-writer.c.
* Add reverts for fixups of already reverted 0347a8fd4c ("block/rbd:
  implement bdrv_co_block_status") that came in with 7.0.0. Those
  fixups are not enough, see Proxmox bugzilla #4047.
* Two trivial context changes for bitmap-mirror patches.
* block_int.h got split up into multiple headers.
* Some context changes in configure and meson.build.
* Used the oppurtunity to squash fixup of bdrv_backuo_dump_create typo
  in a later patch into the patch introducing the function (had to
  move code to new header during rebase).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 12:29:21 +02:00
Thomas Lamprecht
4e4b9ab13f bump version to 6.2.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-22 15:54:58 +02:00
Thomas Lamprecht
39e84ba82d vma/alloc-track improvements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-22 15:52:16 +02:00
Thomas Lamprecht
4fd0fa7fb3 re-export patches in normalized form
iow. using:

git format-patch --zero-commit --no-signature --no-numbered --diff-algorithm=myers ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-22 15:49:53 +02:00
Dominik Csapak
539e333eaa add 'namespace' to BlockdevOptionsPbs
so that we can use it for the -blockdev options (used for live-restore)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-06-22 15:10:49 +02:00
Fabian Grünbichler
68569ea2ff bump version to 6.2.0-10
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-09 16:35:57 +02:00
Fabian Grünbichler
41aedfb6db add d/source/include-binaries
to shutup dpkg-source when building a source package

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-09 16:35:01 +02:00
Fabian Ebner
7bd4d8645a fix #4101: acquire job's aio context before calling job_unref
Otherwise, we might run into an abort via bdrv_co_yield_to_drain()
(can at least happen when a disk with iothread is used):
> #0  0x00007fef4f5dece1 __GI_raise (libc.so.6 + 0x3bce1)
> #1  0x00007fef4f5c8537 __GI_abort (libc.so.6 + 0x25537)
> #2  0x00005641bce3c71f error_exit (qemu-system-x86_64 + 0x80371f)
> #3  0x00005641bce3d02b qemu_mutex_unlock_impl (qemu-system-x86_64 + 0x80402b)
> #4  0x00005641bcd51655 bdrv_co_yield_to_drain (qemu-system-x86_64 + 0x718655)
> #5  0x00005641bcd52de8 bdrv_do_drained_begin (qemu-system-x86_64 + 0x719de8)
> #6  0x00005641bcd47e07 blk_drain (qemu-system-x86_64 + 0x70ee07)
> #7  0x00005641bcd498cd blk_unref (qemu-system-x86_64 + 0x7108cd)
> #8  0x00005641bcd31e6f block_job_free (qemu-system-x86_64 + 0x6f8e6f)
> #9  0x00005641bcd32d65 job_unref (qemu-system-x86_64 + 0x6f9d65)
> #10 0x00005641bcd93b3d pvebackup_co_complete_stream (qemu-system-x86_64 + 0x75ab3d)
> #11 0x00005641bce4e353 coroutine_trampoline (qemu-system-x86_64 + 0x815353)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-09 14:57:28 +02:00
Wolfgang Bumiller
ed3b5b8ab8 bump version to 6.2.0-9
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-08 14:04:09 +02:00
Wolfgang Bumiller
7f4326d1dc pbs cleanup fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-08 13:10:51 +02:00
Wolfgang Bumiller
53bff441c5 delete patches which were dropped from the series file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-08 13:07:04 +02:00
Thomas Lamprecht
f9a4b1cea7 bump version to 6.2.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-19 09:25:11 +02:00
Fabian Ebner
dc265df350 add revert to work around performance regression when backing up large RBD disk
resulting in QMP timeouts and very slow backups. The plan is to figure
out (ideally together with upstream) a way to make the implementation
of bdrv_co_block_status for RBD more efficient. But for now, revert
the problematic change as a stop-gap measure.

Upstream bug report:
https://gitlab.com/qemu-project/qemu/-/issues/1026

Forum threads:
https://forum.proxmox.com/threads/109272/
https://forum.proxmox.com/threads/109448/
https://forum.proxmox.com/threads/101334/ (partially)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-05-19 09:23:38 +02:00
Thomas Lamprecht
e0076597c6 bump version to 6.2.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 16:06:00 +02:00
Thomas Lamprecht
ee99c1f813 d/control: bump build-depenceny of proxmox-backup-qemu to 1.3.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 16:05:30 +02:00
Wolfgang Bumiller
58a5492e9c namespace support
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:49:35 +02:00
Thomas Lamprecht
e67b8b5bd9 bump version to 6.2.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-11 10:42:57 +02:00
Thomas Lamprecht
309b5c1694 backport various fixes for gluster, qxl and vnc
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-11 10:40:14 +02:00
Thomas Lamprecht
4ce5937f89 bump version to 6.2.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 10:13:50 +02:00
Thomas Lamprecht
f87d0523df vma: allow partial restore
Introduce a new map line for skipping a certain drive, of the form
skip=drive-scsi0

Since in PVE, most archives are compressed and piped to vma for
restore, it's not easily possible to skip reads.

For the reader, a new skip flag for VmaRestoreState is added and the
target is allowed to be NULL if skip is specified when registering.
If
the skip flag is set, no writes will be made as well as no check for
duplicate clusters. Therefore, the flag is not set for verify.

Originally-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 10:07:37 +02:00
Thomas Lamprecht
2fd4ea2813 patches: update context
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 10:07:01 +02:00
Thomas Lamprecht
2653a5f029 vma: restore: call blk_unref for all opened block devices
Originally-by: Fabian Ebner <f.ebner@proxmox.com>
Link: https://lists.proxmox.com/pipermail/pve-devel/2022-April/052642.html
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 10:05:29 +02:00
Thomas Lamprecht
664ecf59a9 bump version to 6.2.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-22 11:52:34 +02:00
Thomas Lamprecht
4de9440f87 various stable backports
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-22 10:22:39 +02:00
Thomas Lamprecht
9b348f8c6d d/copyright: drop trailing whitespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-22 09:16:23 +02:00
Thomas Lamprecht
799cf8c5a3 d/control: add suggest dependency-hint for libgl1
It pulls in a lot of stuff via the libglx0 -> libglx-mesa0 dependency
chain, so only suggest it for now to avoid installing it in the
installer or via common "PVE on-top Debian" installations, VirGL
integration is experimental after all and we may drop/replace it with
the vulkan based venus one, once available (Debian 12?).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-22 09:14:39 +02:00
Thomas Lamprecht
b02e62dba0 d/control: add libgbm to build dependencies
required for good virgl support

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-22 08:50:36 +02:00
Thomas Lamprecht
fc03e1b6bf bump version to 6.2.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 09:09:43 +02:00
Thomas Lamprecht
c8ba14bed0 cherry-pick fix for passing some acpi slic tables
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 08:07:34 +02:00
Wolfgang Bumiller
daea534caa bump version to 6.2.0-2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-03 12:05:37 +01:00
Fabian Ebner
27199bd753 backup: add patch to initialize bcs bitmap early enough for PBS
This is necessary for multi-disk backups where not all jobs are
immediately started after they are created. QEMU commit
06e0a9c16405c0a4c1eca33cf286cc04c42066a2 did already part of the work,
ensuring that new writes after job creation don't pass through to the
backup, but not yet for the MIRROR_SYNC_MODE_BITMAP case which is used
for PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-03 11:37:17 +01:00
Thomas Lamprecht
e050683663 d/control: mark numactl a recommended package
we do not call in anywhere unconditionally

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 07:51:19 +01:00
Thomas Lamprecht
13184117e4 d/control: drop sdl dependency, we disable it on compile tinme
disabled via d/rules since a while...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 07:51:19 +01:00
Thomas Lamprecht
aa4b14ea10 d/control: libaio1 is added by dh shlibs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 07:51:19 +01:00
Thomas Lamprecht
3aa5b7598d enable zstd support
plan to use that for multifd migration, among other things

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 07:51:19 +01:00
Thomas Lamprecht
13d3e10aa6 compile in virgl support
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 07:51:19 +01:00
Thomas Lamprecht
58c3533a58 bump version to 6.2.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 14:23:41 +01:00
Thomas Lamprecht
fe0542bed9 d/rules: disable libssh by default
was always disabled in our clean builds, this now also avoids
auto-enabling it on "dirty" build hosts

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 14:21:53 +01:00
Fabian Ebner
f6d40bfdf4 add patch for loading a snapshot with qemu-img dd
Will be used when cloning from a qcow2 efidisk.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-15 14:03:07 +01:00
Fabian Ebner
107132becc fix getopt-string when introducing -n option for qemu-img dd
The colon after U is wrong, because it doesn't take an argument.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-15 14:03:07 +01:00
Fabian Ebner
4567474e95 update submodule and patches to 6.2.0
Notable changes:
* bdrv_co_p{discard,readv,writev,write_zeroes} function signatures
  changed, to using int64_t for offsets/bytes and some still had int
  rather than BrdvRequestFlags for the flags.
* job_cancel_sync now has a force parameter. Commit messages in
  73895f3838cd7fdaf185cf1dbc47be58844a966f
  4cfb3f05627ad82af473e7f7ae113c3884cd04e3
  sound like using force=true makes more sense.
* Added 3 patches coming in via qemu-stable tag, most important one is
  to work around a librbd issue.
* Added another 3 patches from qemu-devel to fix issue leading to
  crash when live migrating with iothread.
* cluster_size calculation helper changed (see patch pve/0026).
* QAPI's if conditionals now use 'CONFIG_FOO' rather than
  'defined(CONFIG_FOO)'

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-15 14:03:07 +01:00
Thomas Lamprecht
33a2d3a826 bump version to 6.1.1-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-14 15:53:18 +01:00
Fabian Ebner
2bf61c3eb6 vma: create: register all streams before entering coroutines
Otherwise, the header might already get written by a coroutine and
registering further streams will fail after that.

Also adds a missing g_list_free call for the other GList that's used.

Reported in the community forum:
https://forum.proxmox.com/threads/104744/

Reproducer script (increase beyond 30 if the issue isn't triggered yet):
> #!/usr/bin/perl
>
> my $dir = "./vma-create-bug";
> mkdir $dir;
>
> my $archive_path = "$dir/vzdump-qemu-104-2202_02_02-00_00_00.vma";
> unlink $archive_path;
>
> my $cmd = "vma create $archive_path -v";
> for (my $i = 0; $i < 30; $i++) {
>   system("truncate -s 1M $dir/drive-virtio$i.img");
>   $cmd .= " drive-virtio$i=$dir/drive-virtio$i.img";
> }
> system($cmd);

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-14 15:38:58 +01:00
Thomas Lamprecht
c07b2203b3 bump version to 6.1.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-13 10:57:48 +01:00
Thomas Lamprecht
ddbf7a872d update submodule and patches to 6.1.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-13 10:56:39 +01:00
Thomas Lamprecht
95c7156d1e bump version to 6.1.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-12-01 15:35:49 +01:00
Fabian Ebner
570d4ad51d fix #3738: cherry-pick "block: introduce max_hw_iov for use in scsi-generic"
which fixes the bad commit 18473467d55a20d643b6c9b3a52de42f705b4d35
that was tracked down via bisecting, and has a Cc for qemu-stable as
well.

Issue was easy enough to reproduce with a single virtio-block disk
using a few runs of dd if=/dev/urandom of=file bs=1M count=1000

Commit cc071629539dc1f303175a7e2d4ab854c0a8b20f upstream.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-12-01 15:34:27 +01:00
Dominik Csapak
c5e8e7c998 buildsys: fix build-dependencies on headers for 'vma' and 'pbs_restore'
both of them depend on generated header files, so we have to specify
them as sources. Otherwise, it happens (at least on some machines)
that they will be compiled before the headers are generated, aborting
the build.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-18 08:11:57 +01:00
Fabian Grünbichler
7cf6b60926 fix #3728: handle machine without type
libguestfs starts their helper VMs with `-machine accel=..` without a
machine type, and our pve version suffix handling would segfault in that
case. there might be other scripted use cases that are affected as well.

this regression was introduced with the rebase of our patch set on top
of 6.1.0

Fixes: f376b2b9e2

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-17 17:20:26 +01:00
Thomas Lamprecht
50999525c6 bump version to 6.1.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-16 09:38:18 +01:00
Fabian Grünbichler
edbcc10a69 cherry-pick segfault fix
this was reported multiple times in our forums[1 with backtraces, 2 & 3
with same log messages], fix is taken from upstream master.

1: https://forum.proxmox.com/threads/pve-7-0-14-1-vm-not-running-live-migration-kills-vm-post-ssd-move-pre-ram-move.99704/
2: https://forum.proxmox.com/threads/proxmox-7-0-14-1-crashes-vm-during-migrate-to-other-host.99678
3: https://forum.proxmox.com/threads/cannot-migrate-between-zfs-and-ceph.99685/#post-430152

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-16 09:23:43 +01:00
Thomas Lamprecht
cc707c362e bump version to 6.1.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 17:58:18 +02:00
Stefan Reiter
af64ed13eb add fixup patch for qxl migration logic
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-13 17:58:18 +02:00
Stefan Reiter
f376b2b9e2 update and rebase to QEMU v6.1.0
Very clean rebase, only the +pve version handling needed manual fixing.
Drops two applied patches from extra/ and adds one new from upstream
(extra/0001*, fixes VNC over unix sockets) as well as 3 of my own for
allowing password changes on custom VNC displays again (as seen and
reviewed upstream, but not yet applied).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-11 15:13:26 +02:00
Thomas Lamprecht
89fa943ef9 bump version to 6.0.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 07:30:05 +02:00
Stefan Reiter
26eee146bc add temporary QMP race fix
same as the initial version sent to qemu-devel, it won't be the final
fix we plan to upstream but it should be enough band-aid to
workaround how PVE uses the QMP.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 [ Thomas: add a bit reasoning to commit message body ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 07:28:07 +02:00
Wolfgang Bumiller
277d33454f drop patch force-disabling smm
This drops debian/patches/pve/0005-PVE-Config-smm_available-false.patch
(and renumbers the remaining patches)

From what I could gather, this patch was originally added
due to issues with old kernels. Now we have users which
seem to run into issues *with* the patch.

All this does is toggle an option, and it's available via a
qemu CLI option anyway, so if dropping this patch causes
issues for some people we can just add an option to
qemu-server & UI control smm explicitly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Cc: Alexandre Derumier <aderumier@odiso.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2021-08-24 11:19:05 +02:00
Fabian Grünbichler
611b692181 bump version to 6.0.0-3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-08-03 15:02:40 +02:00
Fabian Ebner
0114d3cd02 io_uring: resubmit when result is -EAGAIN
Linux SCSI can throw spurious -EAGAIN in some corner cases in its
completion path, which will end up being the result in the completed
io_uring request.

Resubmitting such requests should allow block jobs to complete, even
if such spurious errors are encountered.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-29 11:51:57 +02:00
Thomas Lamprecht
bb3e494bdc bump version to 6.0.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 11:04:47 +02:00
Stefan Reiter
403f23a0c3 enable io-uring support in QEMU builds
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-21 09:56:06 +02:00
Thomas Lamprecht
db687e3cac buildsys: change upload dist to bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-08 11:18:10 +02:00
Thomas Lamprecht
8893def37c bump version to 6.0.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-28 11:30:55 +02:00
Stefan Reiter
263fef5b4c update keycodemapdb for 6.0
QEMU 6.0 requires the updated version to build correctly, as the
keymap-gen tool gained some new parameters.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 11:29:44 +02:00
Stefan Reiter
eb96e850ac debian: ignore submodule checks in QEMU build
...we do those manually, and the build dir is not a git repo.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 11:29:44 +02:00
Stefan Reiter
8dca018b68 udpate and rebase to QEMU v6.0.0
Mostly minor changes, bigger ones summarized:
* QEMU's internal backup code now uses a new async system, which allows
  parallel requests - the default max_workers settings is 64, I chose
  less, since 64 put enough stress on QEMU that the guest became
  practically unusable during the backup, and 16 still shows quite a
  nice measureable performance improvement. Little code changes for us
  though.
* 'malformed' QAPI parameters/functions are now a build error (i.e.
  using '_' vs '-'), I chose to just whitelist our calls in the name of
  backwards compatibility.
* monitor OOB race fix now uses the upstream variant, cherry-picked from
  origin/master since it's not in 6.0 by default
* last patch fixes a bug with snapshot rollback related to the new yank
  system

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 11:29:44 +02:00
Thomas Lamprecht
1cbf147251 bump version to 5.2.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-13 19:30:59 +02:00
Thomas Lamprecht
9e4cba3943 d/control: add libjson-perl to build dependencies
we use that to build the available machine/flags list

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-13 19:30:37 +02:00
Thomas Lamprecht
ebcd9ada10 bump version to 5.2.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-15 16:29:52 +02:00
Thomas Lamprecht
0a88214b72 alloc track: use coroutine version of bdrv_pwrite_zeroes
as we're in a coroutine here too

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-06 16:31:53 +02:00
Thomas Lamprecht
76e464784e pbs block driver: run read in the AIO context of the bs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-06 16:31:53 +02:00
Thomas Lamprecht
b36e8acc31 alloc track: acquire BS AIO context during dropping
ran into this when live-restoring a backup configured for IO-threads,
got the good ol':
> qemu: qemu_mutex_unlock_impl: Operation not permitted
error.

Checking out the history of the related bdrv_backup_top_drop(*bs)
method, we can see that it used to do the AIO context acquiring too,
but in the backup path this was problematic and was changed to be
higher up in the call path in a upstream series from Stefan[0].

That said, this is a completely different code path and it is safe to
do so here. We always run from the main threads's AIO context here
and we call it only indirectly once, guarded by checking for
`s->drop_state == DropNone` and set `s->drop_state = DropRequested`
shortly before we schedule the track_drop() in a bh.

[0]: https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg09139.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-06 16:27:48 +02:00
Thomas Lamprecht
aa42ea267e alloc track: keep track_drop() closer to similar block drivers
Reads just nicer with a drain begin *and* end call. Also clearing the
backing link of the alloc track BDS makes it closer to
bdrv_backup_top_drop() with which this driver has a bit in common.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-06 16:27:37 +02:00
Thomas Lamprecht
6d6894394c bump version to 5.2.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-30 18:19:47 +02:00
Stefan Reiter
e79be6c6c4 add upstream fixes for qmp_block_resize
cherry-picked cleanly from 6.0 development tree, fixes an issue with
resizing RBD drives (and reportedly also on krbd or potentially other
storage backends) with iothreads.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-30 18:14:37 +02:00
Thomas Lamprecht
0be2cab670 bump version to 5.2.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-23 15:41:33 +01:00
Stefan Reiter
bb751cab32 Add tentative fix for QMP hang
Not exactly as sent upstream[0] since we're missing a change in our
v5.2.0 branch (irrelevant for us), but functionally works the same.

[0] https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg07590.html
2021-03-22 16:52:40 +01:00
Thomas Lamprecht
154a7c0f8d d/control: drop unused python from dependencies
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-18 14:15:31 +01:00
Stefan Reiter
677d0d169f add alloc-track block driver patch
See added patches for more info, overview:
0044: slightly increase PBS performance by reducing allocations
0045: slightly increase block-stream performance for Ceph
0046: don't crash with block-stream on RBD
0047: add alloc-track driver for live restore

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-16 20:53:18 +01:00
Stefan Reiter
e9b36665c7 fix saving and loading dirty bitmaps in snapshots
Saving dirty bitmaps from our savevm-async code didn't work, since we
use a coroutine which holds the iothread mutex already (upstream savevm
is sync, migration uses a thread). Release the mutex before calling the
one function that (according to it's documentation) requires the lock to
*not* be held: qemu_savevm_state_pending.

Additionally, loading dirty bitmaps requires a call to
dirty_bitmap_mig_before_vm_start after "loadvm", which the upstream
savevm does explicitly afterwards - do that too.

This is exposed via the query-proxmox-support property
"pbs-dirty-bitmap-savevm".

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-16 20:44:06 +01:00
Thomas Lamprecht
970196fc93 bump version to 5.2.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 16:23:16 +01:00
Thomas Lamprecht
6503e6e08d machine list: save as JSON
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 16:21:30 +01:00
Stefan Reiter
40e6b6e5a5 add ACPI compat patch for 5.1 and older machine types
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-05 15:20:14 +01:00
Stefan Reiter
a6ede89808 add static supported machines file
Same rationale as the CPU flags file, avoids calling QEMU binary just to
query machine types.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-05 15:20:14 +01:00
Stefan Reiter
2413972b46 move bitmap-mirror patches to seperate folder
...instead of having them in the middle of the backup related patches.
These might (hopefully) become upstream at some point as well.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:29:05 +01:00
Stefan Reiter
0c893fd820 clean up pve/ patches by squashing patches of patches
No functional change intended.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:29:05 +01:00
Thomas Lamprecht
25b9156448 bump version to 5.2.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-24 19:02:07 +01:00
Stefan Reiter
4194124719 pbs-restore: unref/close target block backend
Use blk_unref to drop the last reference, which will close the block
backend and flush all caches and outstanding writes.

This is especially important for restoring to Ceph, as the userspace
librbd caches will not be flushed if the application exits immediately,
leading to potentially incomplete restores.

Reported-by: Eneko Lacunza <elacunza@binovo.es>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-24 19:02:07 +01:00
Thomas Lamprecht
42a90c4e1c d/patches: backport virtiofsd security fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-24 19:02:07 +01:00
Thomas Lamprecht
448136958e d/rules: build virtiofsd
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-24 19:02:07 +01:00
Thomas Lamprecht
8e231fbe8d bump version to 5.2.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-12 12:10:04 +01:00
Fabian Grünbichler
00fae7cdbe build: drop explicit libproxmox-backup-qemu0 dep
it ships a symbol file now, so it can be auto-generated based on the
build-dep and usage.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-12 12:07:12 +01:00
Stefan Reiter
0b8da68824 add PBS master key support
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-12 10:47:14 +01:00
Stefan Reiter
817b7667e8 Update to QEMU 5.2
Lots of patches touched and some slight changes to the build process
since QEMU switched to meson as their build system. Functionality-wise
very little rebasing required.

New patches introduced:
* pve/0058: to fix VMA backups and clean up some code in general with
  new 5.2 features now available to us (namely coroutine-enabled QMP).
* extra/0002: don't build man pages for guest agent when disabled
* extra/0003: fix live-migration with hugepages
* 0017 and 0018 are adjusted to fix snapshot abort and improve
  snap performance a bit

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-12 10:20:01 +01:00
Fabian Ebner
a16eaaffd3 fix #3084: fall back to open-iscsi initiatorname
Fixes vma restore when the target is an iSCSI storage which expects that
initiatorname. Also avoids the need to always explicitly set the initiatorname
in PVE code, thus fixing moving efidisks from and to such iSCSI storages.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 15:09:15 +01:00
Wolfgang Bumiller
cfa22a5b2a bump version to 5.1.0-8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-07 10:28:46 +01:00
Wolfgang Bumiller
b515d45e6b fix #3225: properly cancel jobs in 'created' state
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-07 10:26:37 +01:00
Stefan Reiter
3d785eabbe disable jemalloc
jemalloc does not play nice with our Rust library (proxmox-backup-qemu),
specifically it never releases memory allocated from Rust to the OS.
This leads to a problem with larger caches (e.g. for the PBS block driver).

It appears to be related to this GitHub issue:
https://github.com/jemalloc/jemalloc/issues/1398

The background_thread solution seems weirdly hacky, so let's disable
jemalloc entirely for now.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-15 14:43:34 +01:00
Thomas Lamprecht
55df4a9eb1 bump version to 5.1.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:09:24 +01:00
Stefan Reiter
cfe02b3b4e update patches with some pbs-state migration cleanups
...and literal cleanup, as in, call save_cleanup after success or error.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-25 11:49:06 +01:00
Stefan Reiter
32ee41155b update patches with squashed in 'include library version'
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

bump build-dependency on libproxmox-backup-qemu0-dev with version query
support

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 11:49:06 +01:00
Thomas Lamprecht
424f6841d5 bump version to 5.1.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-05 18:59:44 +01:00
Thomas Lamprecht
66eae0ae75 fix dirty-bitmap state migration freeze
The idea in general is to migrate all the state, which is small for
us, in a single step once. But, QEMU only calls save state if we
return active true.

Hardcoding is-active to return true, like done initially, makes the
migration freeze, as QEMU thinks this is never done, and only stops
calling us and finishes after a few seconds.

So, add a state with an "active" boolean, set to true when
initializing a migration, and set it to false when the state was
saved.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-05 18:58:15 +01:00
Thomas Lamprecht
2130e925a8 bump version to 5.1.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 18:36:42 +01:00
Thomas Lamprecht
f36fa39113 migration/block-dirty-bitmap: migrate other bitmaps even if one fails
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 18:35:50 +01:00
Thomas Lamprecht
50eb3ea618 bump version to 5.1.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:09:22 +01:00
Thomas Lamprecht
3e2a0699e6 d/control: update versioned dependency for proxmox backup qemu library
to have the proxmox_export_state et al. available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:07:14 +01:00
Thomas Lamprecht
d95ad93eed apply dirty-bitmap state migration + fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:05:43 +01:00
Stefan Reiter
72ae34ecce Several fixes for backup abort and error reporting
Also add my Signed-off-by to some patches where it was missing.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-29 17:57:47 +01:00
Thomas Lamprecht
60a607ad0f bump version to 5.1.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 09:23:06 +02:00
Stefan Reiter
d333327a1b Add transaction patches and fix for blocking finish
With the transaction patches, patch 0026-PVE-Backup-modify-job-api.patch
is no longer necessary, so drop it and rebase all following patches on
top.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-29 09:21:15 +02:00
Thomas Lamprecht
250d87c276 bump version to 5.1.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 20:01:32 +02:00
Thomas Lamprecht
4b7a18845c cherry-pick: "usb: fix setup_len init (CVE-2020-14364)"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 19:38:34 +02:00
Thomas Lamprecht
7895b0d523 work around #3002: revert "qemu-img convert: Don't pre-zero images"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 19:37:45 +02:00
Stefan Reiter
437d68473c Add systemd journal logging patch
Prints QEMU errors that occur *after* the "-daemonize" fork to the
systemd journal, instead of pushing them into /dev/null like before.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-08 17:13:29 +02:00
Thomas Lamprecht
46c0ec19ad bump version to 5.1.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 14:48:58 +02:00
Fabian Grünbichler
a5feeebc51 allow backup of read-only block drives
this is needed for template backups with PBS until we have the backup
equivalent of 'pbs-restore'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 14:11:39 +02:00
Stefan Reiter
60ae3775bf update to QEMU 5.1
No major semantic changes, mostly just deprecations and changed function
signatures. Drop the extra/ patches, as they have been applied upstream.

The added extra/ patch was accepted upstream[0] but has not been picked
up for 5.1. It is required for non-4M aligned backups to work with PBS.

[0] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01671.html

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-20 13:40:36 +02:00
Thomas Lamprecht
878df11e78 bump version to 5.0.0-13
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:19:35 +02:00
Thomas Lamprecht
f00a720d7e PVE: add query-pbs-bitmap-info QMP call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:11:23 +02:00
Thomas Lamprecht
c5f7dc1d72 PVE: add zero block handling to PBS dump callback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 13:56:03 +02:00
Fabian Grünbichler
dde77467ca bump version to 5.0.0-12
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 11:29:21 +02:00
Fabian Grünbichler
2821f02d70 fix PBS write callback with big blocks
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 11:14:36 +02:00
Oguz Bektas
95fd47ecb9 patch for possible DOS in qemu network packet processing
fixes an assertion failure in qemu network packet processing, which can
lead to DOS'ing the qemu process on the host. this affects 'e1000e' and
'vmxnet3' network devices.

patch is cherry-picked from the commit mentioned in the oss-security email.

more info on oss-security [0]

[0]: https://www.openwall.com/lists/oss-security/2020/08/10/1

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-08-11 11:08:39 +02:00
Thomas Lamprecht
41424ed9c8 bump version to 5.0.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-14 12:45:24 +02:00
Stefan Reiter
f257cc05f4 Fix dirty-bitmap PBS backup with multiple drives
"PVE backup: rename incremental to use-dirty-bitmap" merged two
variables (use_dirty_bitmap and incremental) into one, but they served
two different purposes. Rename the original use_dirty_bitmap to
"expect_only_dirty" so the new one doesn't conflict, and rework "PVE:
use proxmox_backup_check_incremental" around that semantic.

In practice, this had the effect that only one disk at a time would
have a bitmap added, as after the first "use_dirty_bitmap" would be set
to one and the rest would behave as if the QMP parameter of the same
name was unset.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-14 10:46:48 +02:00
Wolfgang Bumiller
9886892f10 bump version to 5.0.0-10
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 13:33:44 +02:00
Wolfgang Bumiller
6d46b2ff4c fix backup qmp parameters to pass along encryption info
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 13:31:52 +02:00
Thomas Lamprecht
2e4f5f2a90 bump version to 5.0.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:18:48 +02:00
Thomas Lamprecht
1d606ec161 d/control: update build-dependency of libproxmox-backup-qemu0-dev
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:17:29 +02:00
Thomas Lamprecht
3499c5b45a PBS patches: block driver, adapat encrypt/compress param, add query-proxmox-support QMP cmd
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:15:49 +02:00
Thomas Lamprecht
102ddd7e59 bump version to 5.0.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 23:05:42 +02:00
Thomas Lamprecht
4c17eebee4 fixup: proxmox_backup_check_incremental is negated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 23:05:42 +02:00
Thomas Lamprecht
dfed71b229 bump version to 5.0.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 22:19:12 +02:00
Thomas Lamprecht
3ab149ccdd update/add PBS integration patches
* rename "incremental" param to "use-dirty-bitmap", avoids confusion
  as the backup can be incrementally also with that param set to
  false.
* use new proxmox_backup_check_incremental
* fix setting dirty counter and adapt to new connect API semantic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 22:13:12 +02:00
Thomas Lamprecht
1f8140323f bump version to 5.0.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 19:26:09 +02:00
Thomas Lamprecht
15b9c76e1f pbs: query-backup: set reused field also for dirty-bitmap
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 19:26:09 +02:00
Thomas Lamprecht
d7f4e01a34 debian/patches: squash some followup patches and regroup a bit more together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:33:16 +02:00
Thomas Lamprecht
8efe995b49 fixup: qemu submodule should be at v5.0.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:33:16 +02:00
Thomas Lamprecht
20be7fa0a0 backup: improve QAPI info and remove all dirty-bitmaps on failed drive-job
effectively two commits merged as one:
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044185.html
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044194.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:03:49 +02:00
Thomas Lamprecht
0943af81a6 bump version to 5.0.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-30 12:59:26 +02:00
Thomas Lamprecht
04e436ea6b d/control: build depend on newer libproxmox-backup-qemu0-dev
to ensure we have the new 'incremental' parameter for the
proxmox_backup_register_image and proxmox_backup_register_image_async
functions available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-30 12:53:01 +02:00
Dietmar Maurer
c96a4a38cb add incremental backup patches
and fix typo: s/BPS/PBS/
2020-06-30 10:34:00 +02:00
Stefan Reiter
f0b53ef0b2 fix #2794: Include legacy-igd passthrough fix
See https://bugs.launchpad.net/qemu/+bug/1882784

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 13:24:34 +02:00
Thomas Lamprecht
b570f1c41e Fix backup for not 64k-aligned storages
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Roland Kammerer <roland.kammerer@linbit.com>
2020-06-24 16:26:30 +02:00
Thomas Lamprecht
fff7e250ee pbs-restore: flush verbose log before calling into library
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-02 13:20:56 +02:00
Thomas Lamprecht
5107839915 bump version to 5.0.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 18:48:00 +02:00
Thomas Lamprecht
bce72611f9 pbs-restore: be more verbose if asked to
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 18:48:00 +02:00
Thomas Lamprecht
c6979241f1 small cleanups for pbs-restore
Add trailing newline to two error messages, and drop an extra
unconditional `qdict_put_str(options, "driver", format);`
Besides that it's just formatting.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 15:24:20 +02:00
Thomas Lamprecht
4e74eca7ed install pbs-restore to usr/bin
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 13:54:08 +02:00
Thomas Lamprecht
62b7007c2a bump version to 5.0.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 18:54:06 +02:00
Thomas Lamprecht
f063a8aadb fix vmstate-snapshots with iothread=1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 18:54:06 +02:00
110 changed files with 9553 additions and 10006 deletions

7
.gitignore vendored Normal file
View File

@@ -0,0 +1,7 @@
/*.build
/*.buildinfo
/*.changes
/*.deb
/*.dsc
/*.tar*
/pve-qemu-kvm-*.*/

View File

@@ -1,57 +1,88 @@
include /usr/share/dpkg/pkg-info.mk
include /usr/share/dpkg/architecture.mk
include /usr/share/dpkg/default.mk
PACKAGE = pve-qemu-kvm
SRCDIR := qemu
BUILDDIR ?= ${PACKAGE}-${DEB_VERSION_UPSTREAM}
BUILDDIR ?= $(PACKAGE)-$(DEB_VERSION_UPSTREAM)
ORIG_SRC_TAR=$(PACKAGE)_$(DEB_VERSION_UPSTREAM).orig.tar.gz
GITVERSION := $(shell git rev-parse HEAD)
DEB = ${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}_${DEB_BUILD_ARCH}.deb
DEB_DBG = ${PACKAGE}-dbg_${DEB_VERSION_UPSTREAM_REVISION}_${DEB_BUILD_ARCH}.deb
DSC=$(PACKAGE)_$(DEB_VERSION_UPSTREAM_REVISION).dsc
DEB = $(PACKAGE)_$(DEB_VERSION_UPSTREAM_REVISION)_$(DEB_BUILD_ARCH).deb
DEB_DBG = $(PACKAGE)-dbgsym_$(DEB_VERSION_UPSTREAM_REVISION)_$(DEB_BUILD_ARCH).deb
DEBS = $(DEB) $(DEB_DBG)
all: $(DEBS)
.PHONY: submodule
submodule:
test -f "${SRCDIR}/configure" || git submodule update --init --recursive
ifeq ($(shell test -f "$(SRCDIR)/configure" && echo 1 || echo 0), 0)
git submodule update --init --recursive
cd $(SRCDIR); meson subprojects download
endif
$(BUILDDIR): keycodemapdb | submodule
rm -rf $(BUILDDIR)
cp -a $(SRCDIR) $(BUILDDIR)
cp -a debian $(BUILDDIR)/debian
rm -rf $(BUILDDIR)/ui/keycodemapdb
cp -a keycodemapdb $(BUILDDIR)/ui/
echo "git clone git://git.proxmox.com/git/pve-qemu.git\\ngit checkout $(GITVERSION)" > $(BUILDDIR)/debian/SOURCE
PC_BIOS_FW_PURGE_LIST_IN = \
hppa-firmware.img \
openbios-ppc \
openbios-sparc32 \
openbios-sparc64 \
palcode-clipper \
s390-ccw.img \
s390-netboot.img \
u-boot.e500 \
.*\.dtb \
qemu_vga.ndrv \
slof.bin \
opensbi-riscv.*-generic-fw_dynamic.bin \
BLOB_PURGE_SED_CMDS = $(foreach FILE,$(PC_BIOS_FW_PURGE_LIST_IN),-e "/$(FILE)/d")
BLOB_PURGE_FILTER = $(foreach FILE,$(PC_BIOS_FW_PURGE_LIST_IN),-e "$(FILE)")
$(BUILDDIR): submodule
# check if qemu/ was used for a build
# if so, please run 'make distclean' in the submodule and try again
test ! -f $(SRCDIR)/build/config.status
rm -rf $@.tmp $@
cp -a $(SRCDIR) $@.tmp
cp -a debian $@.tmp/debian
rm -rf $@.tmp/roms/edk2 # packaged separately
find $@.tmp/pc-bios -type f | grep $(BLOB_PURGE_FILTER) | xargs rm -f
sed -i $(BLOB_PURGE_SED_CMDS) $@.tmp/pc-bios/meson.build
echo "git clone git://git.proxmox.com/git/pve-qemu.git\\ngit checkout $(GITVERSION)" > $@.tmp/debian/SOURCE
mv $@.tmp $@
.PHONY: deb kvm
deb kvm: $(DEBS)
$(DEB_DBG): $(DEB)
$(DEB): $(BUILDDIR)
cd $(BUILDDIR); dpkg-buildpackage -b -us -uc -j
cd $(BUILDDIR); dpkg-buildpackage -b -us -uc -j32
lintian $(DEBS)
.PHONY: update
update:
cd $(SRCDIR) && git submodule deinit ui/keycodemapdb || true
rm -rf $(SRCDIR)/ui/keycodemapdb
mkdir $(SRCDIR)/ui/keycodemapdb
cd $(SRCDIR) && git submodule update --init ui/keycodemapdb
rm -rf keycodemapdb
mkdir keycodemapdb
cp -R $(SRCDIR)/ui/keycodemapdb/* keycodemapdb/
git add keycodemapdb
sbuild: $(DSC)
sbuild $(DSC)
$(ORIG_SRC_TAR): $(BUILDDIR)
tar czf $(ORIG_SRC_TAR) --exclude="$(BUILDDIR)/debian" $(BUILDDIR)
.PHONY: dsc
dsc:
rm -rf *.dsc $(BUILDDIR)
$(MAKE) $(DSC)
lintian $(DSC)
$(DSC): $(ORIG_SRC_TAR) $(BUILDDIR)
cd $(BUILDDIR); dpkg-buildpackage -S -us -uc -d
.PHONY: upload
upload: UPLOAD_DIST ?= $(DEB_DISTRIBUTION)
upload: $(DEBS)
tar cf - ${DEBS} | ssh repoman@repo.proxmox.com upload --product pve --dist buster
tar cf - $(DEBS) | ssh repoman@repo.proxmox.com upload --product pve --dist $(UPLOAD_DIST)
.PHONY: distclean clean
distclean: clean
clean:
rm -rf $(BUILDDIR) $(PACKAGE)*.deb *.buildinfo *.changes
rm -rf $(PACKAGE)-[0-9]*/ $(PACKAGE)*.tar* *.deb *.dsc *.build *.buildinfo *.changes
.PHONY: dinstall
dinstall: $(DEBS)

653
debian/changelog vendored
View File

@@ -1,7 +1,656 @@
pve-qemu-kvm (8.1.5-2+vitastor1) bookworm; urgency=medium
* Add Vitastor support
-- Vitaliy Filippov <vitalif@yourcmc.ru> Wed, 07 Feb 2024 01:11:00 +0300
pve-qemu-kvm (8.1.5-2) bookworm; urgency=medium
* work around for a situation where guest IO might get stuck, if the VM is
configure with iothread and VirtIO block/SCSI
-- Proxmox Support Team <support@proxmox.com> Fri, 02 Feb 2024 19:41:27 +0100
pve-qemu-kvm (8.1.5-1) bookworm; urgency=medium
* update to 8.1.5 stable release, including more relevant fixes like:
- virtio-net: correctly copy vnet header when flushing TX
- hw/pflash: implement update buffer for block writes
- Fixes to i386 emulation and ARM emulation.
-- Proxmox Support Team <support@proxmox.com> Fri, 02 Feb 2024 19:08:13 +0100
pve-qemu-kvm (8.1.2-6) bookworm; urgency=medium
* revert attempted fix to avoid rare issue with stuck guest IO when using
iothread, because it caused a much more common issue with iothreads
consuming too much CPU
-- Proxmox Support Team <support@proxmox.com> Fri, 15 Dec 2023 14:22:06 +0100
pve-qemu-kvm (8.1.2-5) bookworm; urgency=medium
* backport workaround for stuck guest IO with iothread and VirtIO block/SCSI
in some rare edge cases
* backport fix for potential deadlock when issuing the "resize" QMP command
for a disk that is using iothread
-- Proxmox Support Team <support@proxmox.com> Mon, 11 Dec 2023 16:58:27 +0100
pve-qemu-kvm (8.1.2-4) bookworm; urgency=medium
* fix vnc clipboard in the host to guest direction
-- Proxmox Support Team <support@proxmox.com> Wed, 22 Nov 2023 14:28:21 +0100
pve-qemu-kvm (8.1.2-3) bookworm; urgency=medium
* fix #5054: backport fix for software reset with SATA, avoiding breakage
with, e.g., some FreeBSD VMs
-- Proxmox Support Team <support@proxmox.com> Mon, 20 Nov 2023 10:24:50 +0100
pve-qemu-kvm (8.1.2-2) bookworm; urgency=medium
* revert "x86: acpi: workaround Windows not handling name references in
Package properly" as that seems to have broken networking (and possibly
other things) one some localized variants of Windows (e.g., the German
versions).
-- Proxmox Support Team <support@proxmox.com> Fri, 17 Nov 2023 11:55:23 +0100
pve-qemu-kvm (8.1.2-1) bookworm; urgency=medium
* update submodule and patches to QEMU 8.1.2
* use QEMU's keycode-map-db again instead of our static copy from QEMU 6.0
* disable graph locking, newly introduced in the 8.1 release, as it has
still various deadlock issuess, e.g., during canceling backup jobs.
-- Proxmox Support Team <support@proxmox.com> Tue, 24 Oct 2023 13:42:45 +0200
pve-qemu-kvm (8.0.2-7) bookworm; urgency=medium
* fix #2874: SATA: avoid unsolicited write to sector 0 during reset
-- Proxmox Support Team <support@proxmox.com> Wed, 04 Oct 2023 08:33:35 +0200
pve-qemu-kvm (8.0.2-6) bookworm; urgency=medium
* fix #1534: vma: add extract-filter for disk images allowing users to pass
a comma separated list of the disks they want to extract from an archive.
* backup: create jobs in a drained section to avoid subtle bugs where
something interferes with the block-copy-state bitmap on initialization
* backup: drop experimental, and since a while also fully broken, directory
backup format (BACKUP_FORMAT_DIR). This format was never exposed via the
Proxmox VE API, but only available via QMP, as its broken since QEMU 8 and
we got zero reports about that, it's safe to assume that there are no
public users, so just remove it completely.
-- Proxmox Support Team <support@proxmox.com> Wed, 06 Sep 2023 17:03:59 +0200
pve-qemu-kvm (8.0.2-5) bookworm; urgency=medium
* improve memory footprint after backup by not keeping as much memory
resident.
* fix file descriptor leak for vhost (used by default by vNICs).
-- Proxmox Support Team <support@proxmox.com> Wed, 16 Aug 2023 11:52:24 +0200
pve-qemu-kvm (8.0.2-4) bookworm; urgency=medium
* fix resume for snapshot and hibernate in combination with iothread and
dirty bitmap
-- Proxmox Support Team <support@proxmox.com> Fri, 28 Jul 2023 12:58:22 +0200
pve-qemu-kvm (8.0.2-3) bookworm; urgency=medium
* fix regression in QEMU 8.0 for drive mirror with bitmap
-- Proxmox Support Team <support@proxmox.com> Thu, 15 Jun 2023 13:57:46 +0200
pve-qemu-kvm (8.0.2-2) bookworm; urgency=medium
* drop custom get_link_status QMP command, was never really used.
* drop custom & deprecated drive snapshot QMP commands, we use a better
alternative since a while.
-- Proxmox Support Team <support@proxmox.com> Fri, 09 Jun 2023 07:57:56 +0200
pve-qemu-kvm (8.0.2-1) bookworm; urgency=medium
* update to QEMU stable release 8.0.2
* update patches for avoiding issues with DMA reentrancy to current,
slightly optimized version.
-- Proxmox Support Team <support@proxmox.com> Tue, 06 Jun 2023 16:34:50 +0200
pve-qemu-kvm (8.0.0-1) bookworm; urgency=medium
* update to QEMU stable release 8.0.0
* re-build for Proxmox VE 8 / Debian 12 Bookworm
* adapt to the local virtiofsd C variant being dropped, it has been
rewritten in Rust and is now hosted in a separate source repository.
-- Proxmox Support Team <support@proxmox.com> Mon, 22 May 2023 13:45:49 +0200
pve-qemu-kvm (7.2.0-8) bullseye; urgency=medium
* backport fix for ACPI CPU hotplug issue with TCG
* cherry-pick TCG-related stable fixes for 7.2 for users that turned off KVM
HW acceleration
-- Proxmox Support Team <support@proxmox.com> Fri, 17 Mar 2023 15:47:08 +0100
pve-qemu-kvm (7.2.0-7) bullseye; urgency=medium
* improve fix for potential deadlock with trim for IDE/SATA and draining
* backport stable fixes:
- hw/nvme: fix missing endian conversions for doorbell buffers
- hw/smbios: fix field corruption in type 4 table
- virtio-rng-pci: fix transitional migration compat for vectors
- hw/timer/hpet: Fix expiration time overflow
- vhost/vdpa: stop all svq on device deletion
- vhost: avoid a potential use of an uninitialized variable in the call to
vhost_svq_poll
- chardev/char-socket: set s->listener = NULL in char_socket_finalize to
fix a potential crash after live-migration
- intel-iommu: fail MAP notifier without caching mode
- intel-iommu: fail DEVIOTLB_UNMAP without dt mode
* fix a regression for when the LSI SCSI controller is used
-- Proxmox Support Team <support@proxmox.com> Mon, 13 Mar 2023 17:42:49 +0100
pve-qemu-kvm (7.2.0-6) bullseye; urgency=medium
* fix 7.2 regression for Linux boot failures with megasas SCSI
* fix 7.0 regression for a potential deadlock with trim for IDE/SATA and
draining
-- Proxmox Support Team <support@proxmox.com> Wed, 08 Mar 2023 14:32:17 +0100
pve-qemu-kvm (7.2.0-5) bullseye; urgency=medium
* fix #4476: savevm-async: avoid looping without progress
* savevm-async: decrease the boundary for free space for (memory) state left
on target from 30 MiB to 100 MiB, improving the heuristic for when to
enter the final "pause and sync" stage.
* QMP backup: use correct error number when getting blockdrive length fails
* backport fix for some DMA reentrancy issues, better protecting against
malicious guests
* backport fix for iSCSI double free issue leading to crashes
-- Proxmox Support Team <support@proxmox.com> Tue, 21 Feb 2023 13:49:43 +0100
pve-qemu-kvm (7.2.0-4) bullseye; urgency=medium
* backport fix for a 7.2 regression when using VirtIO disk with
detect-zeroes=unmap
-- Proxmox Support Team <support@proxmox.com> Fri, 27 Jan 2023 09:37:49 +0100
pve-qemu-kvm (7.2.0-3) bullseye; urgency=medium
* add fix for live-migration with virtio-rng devices, which regressed in
QEMU 7.2.0.
-- Proxmox Support Team <support@proxmox.com> Thu, 12 Jan 2023 13:13:14 +0100
pve-qemu-kvm (7.2.0-2) bullseye; urgency=medium
* enable slirp again for now, as in qemu-server, user networking is
supported (via CLI/API) when no bridge is set on a virtual NIC
* cherry-pick stable fixes for 7.2. Two for virtio-mem and one for vIOMMU.
Both features are not yet exposed in PVE's qemu-server, but there's work
going on to change that.
-- Proxmox Support Team <support@proxmox.com> Tue, 10 Jan 2023 15:47:48 +0100
pve-qemu-kvm (7.2.0-1) bullseye; urgency=medium
* update to QEMU stable release 7.2.0
* drop 'slirp' networking
-- Proxmox Support Team <support@proxmox.com> Fri, 16 Dec 2022 13:18:21 +0100
pve-qemu-kvm (7.1.0-4) bullseye; urgency=medium
* cherry-pick "block/block-backend: blk_set_enable_write_cache is IO_CODE"
-- Proxmox Support Team <support@proxmox.com> Tue, 22 Nov 2022 09:21:06 +0100
pve-qemu-kvm (7.1.0-3) bullseye; urgency=medium
* init: daemonize: defuse PID file resolve error to a warning at max, fixing
some usecases that regressed with 7.1, like tracking start up in our
file-restore VM.
-- Proxmox Support Team <support@proxmox.com> Fri, 28 Oct 2022 10:27:49 +0200
pve-qemu-kvm (7.1.0-2) bullseye; urgency=medium
* fix an issue with error handling in async backup code
-- Proxmox Support Team <support@proxmox.com> Tue, 18 Oct 2022 15:33:44 +0200
pve-qemu-kvm (7.1.0-1) bullseye; urgency=medium
* update to QEMU stable release 7.1.0
* add fix for io_uring_register_ring_fd from upstream
-- Proxmox Support Team <support@proxmox.com> Fri, 14 Oct 2022 14:54:09 +0200
pve-qemu-kvm (7.0.0-4) bullseye; urgency=medium
* add revision to version output
* PVE Backup: allow passing max-workers performance setting
-- Proxmox Support Team <support@proxmox.com> Mon, 10 Oct 2022 11:55:37 +0200
pve-qemu-kvm (7.0.0-3) bullseye; urgency=medium
* savevm-async: avoid segfault when aborting snapshot creation task
* savevm-async: set SAVE_STATE_DONE when closing state file was successful
allowing one to start a new snapshot task after aborting one.
-- Proxmox Support Team <support@proxmox.com> Tue, 30 Aug 2022 12:54:03 +0200
pve-qemu-kvm (7.0.0-2) bullseye; urgency=medium
* backport "io_uring: fix short read slow path"
* backport "e1000: set RX descriptor status in a separate operation"
-- Proxmox Support Team <support@proxmox.com> Wed, 20 Jul 2022 09:17:07 +0200
pve-qemu-kvm (7.0.0-1) bullseye; urgency=medium
* update to QEMU stable release 7.0.0
-- Proxmox Support Team <support@proxmox.com> Thu, 30 Jun 2022 11:07:37 +0200
pve-qemu-kvm (6.2.0-11) bullseye; urgency=medium
* add 'namespace' to BlockdevOptionsPbs for live-restore support
* vma: create: support 64KiB-unaligned input images like to improve backing
up some VM templates
* block: alloc-track: avoid unlikely, but possible premature break
-- Proxmox Support Team <support@proxmox.com> Wed, 22 Jun 2022 15:54:54 +0200
pve-qemu-kvm (6.2.0-10) bullseye; urgency=medium
* fix #4101: fix backup cancellation bug with iothreads
-- Proxmox Support Team <support@proxmox.com> Thu, 9 Jun 2022 16:35:51 +0200
pve-qemu-kvm (6.2.0-9) bullseye; urgency=medium
* fix possible race conditions during cancellation of a PBS backup
-- Proxmox Support Team <support@proxmox.com> Wed, 08 Jun 2022 14:03:22 +0200
pve-qemu-kvm (6.2.0-8) bullseye; urgency=medium
* revert "block/rbd: implement bdrv_co_block_status" to work around
performance regression when backing up large RBD disk
-- Proxmox Support Team <support@proxmox.com> Thu, 19 May 2022 09:24:45 +0200
pve-qemu-kvm (6.2.0-7) bullseye; urgency=medium
* Proxmox Backup Server namespace support
-- Proxmox Support Team <support@proxmox.com> Thu, 12 May 2022 16:05:56 +0200
pve-qemu-kvm (6.2.0-6) bullseye; urgency=medium
* block/gluster: correctly set max_pdiscard which is int64_t to avoid
triggering assertion
* ui/vnc.c: Fixed a deadlock bug
* display/qxl-render: fix race condition in qxl_cursor (CVE-2021-4207) and
integer overflow in cursor_alloc (CVE-2021-4206)
-- Proxmox Support Team <support@proxmox.com> Wed, 11 May 2022 10:42:53 +0200
pve-qemu-kvm (6.2.0-5) bullseye; urgency=medium
* vma: allow partial restore by skipping some disk
-- Proxmox Support Team <support@proxmox.com> Mon, 25 Apr 2022 10:13:46 +0200
pve-qemu-kvm (6.2.0-4) bullseye; urgency=medium
* d/control: add libgbm to build dependencies
* d/control: add suggest dependency-hint for libgl1
* various stable backports:
+ virtio-net: fix map leaking on error during receive
+ memory: Fix incorrect calls of log_global_start/stop
+ acpi: fix OEM ID/OEM Table ID padding
+ vhost-vsock: detach the virqueue element in case of error
+ vhost-user: remove VirtQ notifier restore
+ vhost-user: fix VirtQ notifier cleanup
+ virtio: fix the condition for iommu_platform not supported
-- Proxmox Support Team <support@proxmox.com> Fri, 22 Apr 2022 11:52:30 +0200
pve-qemu-kvm (6.2.0-3) bullseye; urgency=medium
* cherry-pick fix for some manually added ACPI table SLIC entries via the
custom args flag.
-- Proxmox Support Team <support@proxmox.com> Fri, 15 Apr 2022 09:09:37 +0200
pve-qemu-kvm (6.2.0-2) bullseye; urgency=medium
* compile in virgl support
* enable zstd support
* drop sdl dependency (it was disabled at compile time already)
* recommend 'numactl'
* fix an issue with multi-disk backups where chunks would be written
multiple times
-- Proxmox Support Team <support@proxmox.com> Thu, 03 Mar 2022 12:03:44 +0100
pve-qemu-kvm (6.2.0-1) bullseye; urgency=medium
* update to QEMU stable release 6.2.0
-- Proxmox Support Team <support@proxmox.com> Thu, 17 Feb 2022 06:23:14 +0100
pve-qemu-kvm (6.1.1-2) bullseye; urgency=medium
* vma: create: register all streams before entering coroutines to avoid that
an early stream starts to write already before all got registered.
-- Proxmox Support Team <support@proxmox.com> Mon, 14 Feb 2022 15:53:15 +0100
pve-qemu-kvm (6.1.1-1) bullseye; urgency=medium
* update to 6.1.1 stable release
-- Proxmox Support Team <support@proxmox.com> Thu, 13 Jan 2022 10:57:43 +0100
pve-qemu-kvm (6.1.0-3) bullseye; urgency=medium
* fix #3738: cherry-pick "block: introduce max_hw_iov for use in scsi-
generic
-- Proxmox Support Team <support@proxmox.com> Wed, 01 Dec 2021 15:35:43 +0100
pve-qemu-kvm (6.1.0-2) bullseye; urgency=medium
* avoid a possible segmentation fault during block (disk) mirror
-- Proxmox Support Team <support@proxmox.com> Tue, 16 Nov 2021 09:38:10 +0100
pve-qemu-kvm (6.1.0-1) bullseye; urgency=medium
* update to QEMU stable release 6.1.0
-- Proxmox Support Team <support@proxmox.com> Mon, 11 Oct 2021 15:15:19 +0200
pve-qemu-kvm (6.0.0-4) bullseye; urgency=medium
* drop the ancient workaround that force disabled SMM due to observing VM
hangs on old kernel versions.
* monitor/qmp: fix race with clients disconnecting early resulting in other
clients receiving a message with the (now wrong) ID of the former
-- Proxmox Support Team <support@proxmox.com> Mon, 06 Sep 2021 07:30:00 +0200
pve-qemu-kvm (6.0.0-3) bullseye; urgency=medium
* io_uring: resubmit when result is -EAGAIN
-- Proxmox Support Team <support@proxmox.com> Tue, 3 Aug 2021 15:01:31 +0200
pve-qemu-kvm (6.0.0-2) bullseye; urgency=medium
* enable io-uring support in QEMU builds
-- Proxmox Support Team <support@proxmox.com> Wed, 23 Jun 2021 11:03:54 +0200
pve-qemu-kvm (6.0.0-1) bullseye; urgency=medium
* update to QEMU stable release 6.0.0
-- Proxmox Support Team <support@proxmox.com> Fri, 28 May 2021 11:30:50 +0200
pve-qemu-kvm (5.2.0-11) bullseye; urgency=medium
* re-build for Proxmox VE 7 / Debian Bullseye
-- Proxmox Support Team <support@proxmox.com> Thu, 13 May 2021 14:03:00 +0200
pve-qemu-kvm (5.2.0-6) pve; urgency=medium
* improve the alloc-track and Proxmox Backup Server special block driver when
used with IO-threads for the new live-restore feature
-- Proxmox Support Team <support@proxmox.com> Thu, 15 Apr 2021 16:29:48 +0200
pve-qemu-kvm (5.2.0-5) pve; urgency=medium
* cherry-pick fixes for a possible deadlock when resizing a disk while using
IO-threads
-- Proxmox Support Team <support@proxmox.com> Tue, 30 Mar 2021 18:18:18 +0200
pve-qemu-kvm (5.2.0-4) pve; urgency=medium
* monitor/qmp: fix race on CHR_EVENT_CLOSED without OOB
* improve saving and loading dirty bitmaps in live-snapshots
-- Proxmox Support Team <support@proxmox.com> Tue, 23 Mar 2021 15:41:26 +0100
pve-qemu-kvm (5.2.0-3) pve; urgency=medium
* backport "i386/acpi: restore device paths for pre-5.1 vms" patch
* ship list of 'i440fx' and 'q35' machine versions this QEMU version supports
-- Proxmox Support Team <support@proxmox.com> Fri, 05 Mar 2021 16:23:06 +0100
pve-qemu-kvm (5.2.0-2) pve; urgency=medium
* Proxmox Backup restore: ensure all caches are flushed for userspace
attached storages (for example, Ceph when librbd and not KRBD is used)
* ship VirtIOFSd daemon
-- Proxmox Support Team <support@proxmox.com> Wed, 24 Feb 2021 18:25:14 +0100
pve-qemu-kvm (5.2.0-1) pve; urgency=medium
* update to QEMU stable release 5.2.0
* fix #3084: fall back to open-iscsi initiatorname
* fix snapshot abort and improve performance in some edge cases
* add basis for Proxmox Backup Server master key support
-- Proxmox Support Team <support@proxmox.com> Fri, 12 Feb 2021 12:09:59 +0100
pve-qemu-kvm (5.1.0-8) pve; urgency=medium
* disable jemalloc, as it does not play nice with our library written in
rust
* fix #3225: properly cancel jobs in 'created' state when cancelling or
failing backup jobs
-- Proxmox Support Team <support@proxmox.com> Thu, 07 Jan 2021 10:27:33 +0100
pve-qemu-kvm (5.1.0-7) pve; urgency=medium
* allow to query the loaded Proxmox Backup library version
-- Proxmox Support Team <support@proxmox.com> Wed, 25 Nov 2020 14:09:16 +0100
pve-qemu-kvm (5.1.0-6) pve; urgency=medium
* migration/block-dirty-bitmap: avoid telling QEMU that the bitmap migration
is active longer than required
-- Proxmox Support Team <support@proxmox.com> Thu, 05 Nov 2020 18:59:40 +0100
pve-qemu-kvm (5.1.0-5) pve; urgency=medium
* migration/block-dirty-bitmap: migrate other bitmaps even if one fails
-- Proxmox Support Team <support@proxmox.com> Wed, 04 Nov 2020 18:36:32 +0100
pve-qemu-kvm (5.1.0-4) pve; urgency=medium
* several fixes for backup abort edgecases and error reporting
* allow to migrate dirty-bitmap and Proxmox Backup Server state
-- Proxmox Support Team <support@proxmox.com> Thu, 29 Oct 2020 18:09:16 +0100
pve-qemu-kvm (5.1.0-3) pve; urgency=medium
* backup: make more use of coroutines and do not block on finishing
* backup: use transactions to synchronize the disk job states
-- Proxmox Support Team <support@proxmox.com> Tue, 29 Sep 2020 09:22:56 +0200
pve-qemu-kvm (5.1.0-2) pve; urgency=medium
* cherry-pick fix to harden checks for USB devices (CVE-2020-14364)
* work around #3002: revert "qemu-img convert: Don't pre-zero images" as it
correlates with an issue when using LVM as a target storage for disk move
operations.
-- Proxmox Support Team <support@proxmox.com> Mon, 14 Sep 2020 20:01:24 +0200
pve-qemu-kvm (5.1.0-1) pve; urgency=medium
* update to QEMU 5.1.0
-- Proxmox Support Team <support@proxmox.com> Thu, 20 Aug 2020 13:42:10 +0200
pve-qemu-kvm (5.0.0-13) pve; urgency=medium
* improve zero block handling for PBS backups
* allow querying a more detailed dirty-bitmap state per VM disk
-- Proxmox Support Team <support@proxmox.com> Wed, 19 Aug 2020 18:19:26 +0200
pve-qemu-kvm (5.0.0-12) pve; urgency=medium
* patch for possible DOS in qemu network packet processing
* fix PBS write callback with big blocks
-- Proxmox Support Team <support@proxmox.com> Tue, 11 Aug 2020 11:29:13 +0200
pve-qemu-kvm (5.0.0-11) pve; urgency=medium
* improve dirty-bitmap Proxmox Backup Server backup with multiple drives
-- Proxmox Support Team <support@proxmox.com> Tue, 14 Jul 2020 12:44:25 +0200
pve-qemu-kvm (5.0.0-10) pve; urgency=medium
* fix compression and encryption related backup parameters not being passed
on from the HMP command properly
-- Proxmox Support Team <support@proxmox.com> Fri, 10 Jul 2020 13:32:11 +0200
pve-qemu-kvm (5.0.0-9) pve; urgency=medium
* adapt to new compress and encrypt params of the proxmox-backup library
* add block driver for Proxmox Backup Server backed images
* add 'query-proxmox-support' QMP command
-- Proxmox Support Team <support@proxmox.com> Thu, 09 Jul 2020 13:18:45 +0200
pve-qemu-kvm (5.0.0-8) pve; urgency=medium
* backup: rename parameter for dirty-bitmap PBS backup to "use-dirty-bitmap"
* backup: improve checking if a previous backup is available when trying to
reuse it
-- Proxmox Support Team <support@proxmox.com> Mon, 06 Jul 2020 22:58:45 +0200
pve-qemu-kvm (5.0.0-6) pve; urgency=medium
* backup: improve query-backup information and remove all dirty-bitmaps on
failed drive-job
-- Proxmox Support Team <support@proxmox.com> Fri, 03 Jul 2020 17:00:30 +0200
pve-qemu-kvm (5.0.0-5) pve; urgency=medium
* fix backup for not 64k-aligned storages
* fix #2794: Include legacy-igd passthrough fix
* add initial support for incremental backup for running VMs and
Proxmox Backup Server as a target
-- Proxmox Support Team <support@proxmox.com> Tue, 30 Jun 2020 11:12:55 +0200
pve-qemu-kvm (5.0.0-4) pve; urgency=medium
* install missing restore helper binary
-- Proxmox Support Team <support@proxmox.com> Sat, 30 May 2020 15:25:38 +0200
pve-qemu-kvm (5.0.0-3) pve; urgency=medium
* ensure that a data-flush for all drives uses the correct AioContext. Fixes
a potential VM hang happening on some storage types if IOThreads are used.
-- Proxmox Support Team <support@proxmox.com> Wed, 27 May 2020 14:41:31 +0200
pve-qemu-kvm (5.0.0-2) pve; urgency=medium
* fix saving a VM-state (snapshot, suspend-to-disk) in combination with
IOThreads
* fix saving a VM-state (snapshot, suspend-to-disk) with QEMU 5.0
* try to use bigger chunks for saving a VM-state to improve performance on
storage backends like Ceph RBD

1
debian/compat vendored
View File

@@ -1 +0,0 @@
10

40
debian/control vendored
View File

@@ -2,36 +2,43 @@ Source: pve-qemu-kvm
Section: admin
Priority: optional
Maintainer: Proxmox Support Team <support@proxmox.com>
Build-Depends: autotools-dev,
Build-Depends: debhelper-compat (= 13),
check,
debhelper (>= 9),
libacl1-dev,
libaio-dev,
libattr1-dev,
libcap-ng-dev,
libcurl4-gnutls-dev,
libepoxy-dev,
libfdt-dev,
libgbm-dev,
libglusterfs-dev (>= 5.2-2),
libgnutls28-dev,
libiscsi-dev (>= 1.12.0),
libjemalloc-dev,
libjpeg-dev,
libjson-perl,
libnuma-dev,
libpci-dev,
libpixman-1-dev,
libproxmox-backup-qemu0-dev,
libproxmox-backup-qemu0-dev (>= 1.3.0),
librbd-dev (>= 0.48),
libsdl1.2-dev,
libseccomp-dev,
libslirp-dev,
libspice-protocol-dev (>= 0.12.14~),
libspice-server-dev (>= 0.14.0~),
libusb-1.0-0-dev (>= 1.0.17-1),
libsystemd-dev,
liburing-dev,
libusb-1.0-0-dev (>= 1.0.17),
libusbredirparser-dev (>= 0.6-2),
libvirglrenderer-dev,
libzstd-dev,
meson,
python3-minimal,
python3-sphinx,
python3-sphinx-rtd-theme,
python3-venv,
quilt,
texi2html,
texinfo,
uuid-dev,
xfslibs-dev,
Standards-Version: 3.7.2
@@ -40,7 +47,6 @@ Package: pve-qemu-kvm
Architecture: any
Depends: ceph-common (>= 0.48),
iproute2,
libaio1,
libgfapi0 | glusterfs-common (>= 5.6),
libgfchangelog0 | glusterfs-common (>= 5.6),
libgfdb0 | glusterfs-common (>= 5.6),
@@ -49,17 +55,16 @@ Depends: ceph-common (>= 0.48),
libglusterfs-dev | glusterfs-common (>= 5.6),
libglusterfs0 | glusterfs-common (>= 5.6),
libiscsi4 (>= 1.12.0) | libiscsi7,
libjemalloc2,
libjpeg62-turbo,
libsdl1.2debian,
libspice-server1 (>= 0.14.0~),
libusb-1.0-0 (>= 1.0.17-1),
libusbredirparser1 (>= 0.6-2),
vitastor-client (>= 0.9.4),
libuuid1,
numactl,
python,
${misc:Depends},
${shlibs:Depends},
Recommends: numactl,
Suggests: libgl1,
Conflicts: kvm,
pve-kvm,
pve-qemu-kvm-2.6.18,
@@ -67,22 +72,17 @@ Conflicts: kvm,
qemu-kvm,
qemu-system-arm,
qemu-system-common,
qemu-system-data,
qemu-system-x86,
qemu-utils,
Provides: qemu-system-arm, qemu-system-x86, qemu-utils
Provides: qemu-system-arm, qemu-system-x86, qemu-utils,
Replaces: pve-kvm,
pve-qemu-kvm-2.6.18,
qemu-system-arm,
qemu-system-x86,
qemu-utils,
Breaks: qemu-server (<= 8.0.6)
Description: Full virtualization on x86 hardware
Using KVM, one can run multiple virtual PCs, each running unmodified Linux or
Windows images. Each virtual machine has private virtualized hardware: a
network card, disk, graphics adapter, etc.
Package: pve-qemu-kvm-dbg
Architecture: any
Section: debug
Depends: pve-qemu-kvm (= ${binary:Version})
Description: pve qemu debugging symbols
This package contains the debugging symbols for pve-qemu-kvm.

2
debian/copyright vendored
View File

@@ -25,7 +25,7 @@ License:
In particular, the QEMU virtual CPU core library (libqemu.a) is
released under the GNU Lesser General Public License version 2 or later.
On Debian systems, the complete text of the GNU Lesser General Public
On Debian systems, the complete text of the GNU Lesser General Public
License can be found in the file /usr/share/common-licenses/LGPL.
Some hardware device emulation sources and other QEMU functionality are

28
debian/parse-machines.pl vendored Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/perl
use warnings;
use strict;
use JSON;
my $machines = [];
while (<STDIN>) {
if (/^\s*Supported machines are:/) {
next;
}
s/^\s+//;
my @machine = split(/\s+/);
next if $machine[0] !~ m/^pc-(i440fx|q35)-(.+)$/;
push @$machines, {
'id' => $machine[0],
'type' => $1,
'version' => $2,
};
}
die "no QEMU machine types detected from STDIN input" if scalar (@$machines) <= 0;
print to_json($machines, { utf8 => 1, canonical => 1 })
or die "failed to encode detected machines as JSON - $!\n";

View File

@@ -26,19 +26,22 @@ Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebased for 8.1.1]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/mirror.c | 98 ++++++++++++++++++++++++++++++-------
blockdev.c | 39 +++++++++++++--
include/block/block_int.h | 4 +-
qapi/block-core.json | 29 +++++++++--
tests/test-block-iothread.c | 4 +-
5 files changed, 145 insertions(+), 29 deletions(-)
block/mirror.c | 98 +++++++++++++++++++++-----
blockdev.c | 38 +++++++++-
include/block/block_int-global-state.h | 4 +-
qapi/block-core.json | 25 ++++++-
tests/unit/test-block-iothread.c | 4 +-
5 files changed, 142 insertions(+), 27 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index c26fd9260d..3c9cd42c50 100644
index d3cacd1708..1ff42c8af1 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -49,7 +49,7 @@ typedef struct MirrorBlockJob {
@@ -51,7 +51,7 @@ typedef struct MirrorBlockJob {
BlockDriverState *to_replace;
/* Used to block operations on the drive-mirror-replace target */
Error *replace_blocker;
@@ -47,7 +50,7 @@ index c26fd9260d..3c9cd42c50 100644
BlockMirrorBackingMode backing_mode;
/* Whether the target image requires explicit zero-initialization */
bool zero_target;
@@ -64,6 +64,8 @@ typedef struct MirrorBlockJob {
@@ -65,6 +65,8 @@ typedef struct MirrorBlockJob {
size_t buf_size;
int64_t bdev_length;
unsigned long *cow_bitmap;
@@ -56,17 +59,17 @@ index c26fd9260d..3c9cd42c50 100644
BdrvDirtyBitmap *dirty_bitmap;
BdrvDirtyBitmapIter *dbi;
uint8_t *buf;
@@ -676,7 +678,8 @@ static int mirror_exit_common(Job *job)
@@ -705,7 +707,8 @@ static int mirror_exit_common(Job *job)
bdrv_child_refresh_perms(mirror_top_bs, mirror_top_bs->backing,
&error_abort);
if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) {
- BlockDriverState *backing = s->is_none_mode ? src : s->base;
+ BlockDriverState *backing;
+ backing = s->sync_mode == MIRROR_SYNC_MODE_NONE ? src : s->base;
if (backing_bs(target_bs) != backing) {
bdrv_set_backing_hd(target_bs, backing, &local_err);
if (local_err) {
@@ -771,6 +774,16 @@ static void mirror_abort(Job *job)
BlockDriverState *unfiltered_target = bdrv_skip_filters(target_bs);
if (bdrv_cow_bs(unfiltered_target) != backing) {
@@ -809,6 +812,16 @@ static void mirror_abort(Job *job)
assert(ret == 0);
}
@@ -83,7 +86,7 @@ index c26fd9260d..3c9cd42c50 100644
static void coroutine_fn mirror_throttle(MirrorBlockJob *s)
{
int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -949,7 +962,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
@@ -997,7 +1010,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
mirror_free_init(s);
s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -93,23 +96,23 @@ index c26fd9260d..3c9cd42c50 100644
ret = mirror_dirty_init(s);
if (ret < 0 || job_is_cancelled(&s->common.job)) {
goto immediate_exit;
@@ -1181,6 +1195,7 @@ static const BlockJobDriver mirror_job_driver = {
@@ -1251,6 +1265,7 @@ static const BlockJobDriver mirror_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
},
@@ -1196,6 +1211,7 @@ static const BlockJobDriver commit_active_job_driver = {
.cancel = mirror_cancel,
@@ -1267,6 +1282,7 @@ static const BlockJobDriver commit_active_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
},
@@ -1542,7 +1558,10 @@ static BlockJob *mirror_start_job(
.cancel = commit_active_cancel,
@@ -1658,7 +1674,10 @@ static BlockJob *mirror_start_job(
BlockCompletionFunc *cb,
void *opaque,
const BlockJobDriver *driver,
@@ -121,8 +124,8 @@ index c26fd9260d..3c9cd42c50 100644
bool auto_complete, const char *filter_node_name,
bool is_mirror, MirrorCopyMode copy_mode,
Error **errp)
@@ -1555,10 +1574,39 @@ static BlockJob *mirror_start_job(
Error *local_err = NULL;
@@ -1670,10 +1689,39 @@ static BlockJob *mirror_start_job(
uint64_t target_perms, target_shared_perms;
int ret;
- if (granularity == 0) {
@@ -163,7 +166,7 @@ index c26fd9260d..3c9cd42c50 100644
assert(is_power_of_2(granularity));
if (buf_size < 0) {
@@ -1662,7 +1710,9 @@ static BlockJob *mirror_start_job(
@@ -1804,7 +1852,9 @@ static BlockJob *mirror_start_job(
s->replaces = g_strdup(replaces);
s->on_source_error = on_source_error;
s->on_target_error = on_target_error;
@@ -174,7 +177,7 @@ index c26fd9260d..3c9cd42c50 100644
s->backing_mode = backing_mode;
s->zero_target = zero_target;
s->copy_mode = copy_mode;
@@ -1682,6 +1732,18 @@ static BlockJob *mirror_start_job(
@@ -1825,6 +1875,18 @@ static BlockJob *mirror_start_job(
bdrv_disable_dirty_bitmap(s->dirty_bitmap);
}
@@ -193,7 +196,7 @@ index c26fd9260d..3c9cd42c50 100644
ret = block_job_add_bdrv(&s->common, "source", bs, 0,
BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE |
BLK_PERM_CONSISTENT_READ,
@@ -1735,6 +1797,9 @@ fail:
@@ -1902,6 +1964,9 @@ fail:
if (s->dirty_bitmap) {
bdrv_release_dirty_bitmap(s->dirty_bitmap);
}
@@ -203,7 +206,7 @@ index c26fd9260d..3c9cd42c50 100644
job_early_fail(&s->common.job);
}
@@ -1752,29 +1817,23 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
@@ -1919,31 +1984,25 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
@@ -220,6 +223,8 @@ index c26fd9260d..3c9cd42c50 100644
- bool is_none_mode;
BlockDriverState *base;
GLOBAL_STATE_CODE();
- if ((mode == MIRROR_SYNC_MODE_INCREMENTAL) ||
- (mode == MIRROR_SYNC_MODE_BITMAP)) {
- error_setg(errp, "Sync mode '%s' not supported",
@@ -227,7 +232,7 @@ index c26fd9260d..3c9cd42c50 100644
- return;
- }
- is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL;
base = mode == MIRROR_SYNC_MODE_TOP ? bdrv_backing_chain_next(bs) : NULL;
mirror_start_job(job_id, bs, creation_flags, target, replaces,
speed, granularity, buf_size, backing_mode, zero_target,
on_source_error, on_target_error, unmap, NULL, NULL,
@@ -238,7 +243,7 @@ index c26fd9260d..3c9cd42c50 100644
}
BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
@@ -1800,7 +1859,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
@@ -1970,7 +2029,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
job_id, bs, creation_flags, base, NULL, speed, 0, 0,
MIRROR_LEAVE_BACKING_CHAIN, false,
on_error, on_error, true, cb, opaque,
@@ -246,36 +251,35 @@ index c26fd9260d..3c9cd42c50 100644
+ &commit_active_job_driver, MIRROR_SYNC_MODE_FULL,
+ NULL, 0, base, auto_complete,
filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND,
&local_err);
if (local_err) {
errp);
if (!job) {
diff --git a/blockdev.c b/blockdev.c
index f391c3b3c7..bbeff9c439 100644
index c28462a633..a402fa4bf7 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3159,6 +3159,10 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
@@ -2849,6 +2849,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
BlockDriverState *target,
bool has_replaces, const char *replaces,
const char *replaces,
enum MirrorSyncMode sync,
+ bool has_bitmap,
+ const char *bitmap_name,
+ bool has_bitmap_mode,
+ BitmapSyncMode bitmap_mode,
BlockMirrorBackingMode backing_mode,
bool zero_target,
bool has_speed, int64_t speed,
@@ -3177,6 +3181,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
Error **errp)
@@ -2867,6 +2870,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
{
BlockDriverState *unfiltered_bs;
int job_flags = JOB_DEFAULT;
+ BdrvDirtyBitmap *bitmap = NULL;
if (!has_speed) {
speed = 0;
@@ -3231,6 +3236,29 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
GLOBAL_STATE_CODE();
GRAPH_RDLOCK_GUARD_MAINLOOP();
@@ -2921,6 +2925,29 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
+ if (has_bitmap) {
+ if (bitmap_name) {
+ if (granularity) {
+ error_setg(errp, "Granularity and bitmap cannot both be set");
+ return;
@@ -298,53 +302,53 @@ index f391c3b3c7..bbeff9c439 100644
+ }
+ }
+
if (has_replaces) {
BlockDriverState *to_replace_bs;
AioContext *replace_aio_context;
@@ -3268,8 +3296,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
if (!replaces) {
/* We want to mirror from @bs, but keep implicit filters on top */
unfiltered_bs = bdrv_skip_implicit_filters(bs);
@@ -2966,8 +2993,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
* and will allow to check whether the node still exist at mirror completion
*/
mirror_start(job_id, bs, target,
- has_replaces ? replaces : NULL, job_flags,
- replaces, job_flags,
- speed, granularity, buf_size, sync, backing_mode, zero_target,
+ has_replaces ? replaces : NULL, job_flags, speed, granularity,
+ buf_size, sync, bitmap, bitmap_mode, backing_mode, zero_target,
+ replaces, job_flags, speed, granularity, buf_size, sync,
+ bitmap, bitmap_mode, backing_mode, zero_target,
on_source_error, on_target_error, unmap, filter_node_name,
copy_mode, errp);
}
@@ -3410,6 +3438,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
@@ -3115,6 +3142,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
blockdev_mirror_common(arg->has_job_id ? arg->job_id : NULL, bs, target_bs,
arg->has_replaces, arg->replaces, arg->sync,
+ arg->has_bitmap, arg->bitmap,
blockdev_mirror_common(arg->job_id, bs, target_bs,
arg->replaces, arg->sync,
+ arg->bitmap,
+ arg->has_bitmap_mode, arg->bitmap_mode,
backing_mode, zero_target,
arg->has_speed, arg->speed,
arg->has_granularity, arg->granularity,
@@ -3432,6 +3462,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
@@ -3136,6 +3165,8 @@ void qmp_blockdev_mirror(const char *job_id,
const char *device, const char *target,
bool has_replaces, const char *replaces,
const char *replaces,
MirrorSyncMode sync,
+ bool has_bitmap, const char *bitmap,
+ const char *bitmap,
+ bool has_bitmap_mode, BitmapSyncMode bitmap_mode,
bool has_speed, int64_t speed,
bool has_granularity, uint32_t granularity,
bool has_buf_size, int64_t buf_size,
@@ -3482,7 +3514,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
@@ -3184,7 +3215,8 @@ void qmp_blockdev_mirror(const char *job_id,
}
blockdev_mirror_common(has_job_id ? job_id : NULL, bs, target_bs,
- has_replaces, replaces, sync, backing_mode,
+ has_replaces, replaces, sync, has_bitmap,
blockdev_mirror_common(job_id, bs, target_bs,
- replaces, sync, backing_mode,
+ replaces, sync,
+ bitmap, has_bitmap_mode, bitmap_mode, backing_mode,
zero_target, has_speed, speed,
has_granularity, granularity,
has_buf_size, buf_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6d234f1de9..180a5e00fd 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1210,7 +1210,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h
index da5fb31089..32f0f9858a 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -152,7 +152,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
@@ -356,31 +360,26 @@ index 6d234f1de9..180a5e00fd 100644
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 97d1f64636..8bdbccb397 100644
index bca1a0c372..a5cea82139 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2054,10 +2054,19 @@
# (all the disk, only the sectors allocated in the topmost image, or
# only new I/O).
@@ -2145,6 +2145,15 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This argument must
+# be present for bitmap mode and absent otherwise. The bitmap's
+# granularity is used instead of @granularity (since 4.1).
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
+# argument must be present for bitmap mode and absent otherwise.
+# The bitmap's granularity is used instead of @granularity (Since
+# 4.1).
+#
+# @bitmap-mode: Specifies the type of data the bitmap should contain after
+# the operation concludes. Must be present if sync is "bitmap".
+# Must NOT be present otherwise. (Since 4.1)
+# @bitmap-mode: Specifies the type of data the bitmap should contain
+# after the operation concludes. Must be present if sync is
+# "bitmap". Must NOT be present otherwise. (Since 4.1)
+#
# @granularity: granularity of the dirty bitmap, default is 64K
# if the image format doesn't have clusters, 4K if the clusters
# are smaller than that, else the cluster size. Must be a
-# power of 2 between 512 and 64M (since 1.4).
+# power of 2 between 512 and 64M. Must not be specified if
+# @bitmap is present (since 1.4).
#
# @buf-size: maximum amount of data in flight from source to
# target (since 1.4).
@@ -2095,7 +2104,9 @@
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2187,7 +2196,9 @@
{ 'struct': 'DriveMirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*format': 'str', '*node-name': 'str', '*replaces': 'str',
@@ -391,28 +390,23 @@ index 97d1f64636..8bdbccb397 100644
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
@@ -2362,10 +2373,19 @@
# (all the disk, only the sectors allocated in the topmost image, or
# only new I/O).
@@ -2471,6 +2482,15 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This argument must
+# be present for bitmap mode and absent otherwise. The bitmap's
+# granularity is used instead of @granularity (since 4.1).
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
+# argument must be present for bitmap mode and absent otherwise.
+# The bitmap's granularity is used instead of @granularity (since
+# 4.1).
+#
+# @bitmap-mode: Specifies the type of data the bitmap should contain after
+# the operation concludes. Must be present if sync is "bitmap".
+# Must NOT be present otherwise. (Since 4.1)
+# @bitmap-mode: Specifies the type of data the bitmap should contain
+# after the operation concludes. Must be present if sync is
+# "bitmap". Must NOT be present otherwise. (Since 4.1)
+#
# @granularity: granularity of the dirty bitmap, default is 64K
# if the image format doesn't have clusters, 4K if the clusters
# are smaller than that, else the cluster size. Must be a
-# power of 2 between 512 and 64M
+# power of 2 between 512 and 64M . Must not be specified if
+# @bitmap is present.
#
# @buf-size: maximum amount of data in flight from source to
# target
@@ -2414,7 +2434,8 @@
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2521,7 +2541,8 @@
{ 'command': 'blockdev-mirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*replaces': 'str',
@@ -422,11 +416,11 @@ index 97d1f64636..8bdbccb397 100644
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
diff --git a/tests/test-block-iothread.c b/tests/test-block-iothread.c
index 0c861809f0..da87a67a57 100644
--- a/tests/test-block-iothread.c
+++ b/tests/test-block-iothread.c
@@ -611,8 +611,8 @@ static void test_propagate_mirror(void)
diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c
index d727a5fee8..8a34aa2328 100644
--- a/tests/unit/test-block-iothread.c
+++ b/tests/unit/test-block-iothread.c
@@ -757,8 +757,8 @@ static void test_propagate_mirror(void)
/* Start a mirror job */
mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0,
@@ -436,4 +430,4 @@ index 0c861809f0..da87a67a57 100644
+ false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
false, "filter_node", MIRROR_COPY_MODE_BACKGROUND,
&error_abort);
job = job_get("job0");
WITH_JOB_LOCK_GUARD() {

View File

@@ -18,15 +18,16 @@ incremental backup modes; we can use this bitmap to later refresh a
successfully created mirror.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/mirror.c | 24 ++++++++++++++++++------
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 3c9cd42c50..08ac9827f2 100644
index 1ff42c8af1..11b8a8e959 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -653,8 +653,6 @@ static int mirror_exit_common(Job *job)
@@ -682,8 +682,6 @@ static int mirror_exit_common(Job *job)
bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs);
}
@@ -35,9 +36,9 @@ index 3c9cd42c50..08ac9827f2 100644
/* Make sure that the source BDS doesn't go away during bdrv_replace_node,
* before we can call bdrv_drained_end */
bdrv_ref(src);
@@ -752,6 +750,18 @@ static int mirror_exit_common(Job *job)
blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort);
blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort);
@@ -788,6 +786,18 @@ static int mirror_exit_common(Job *job)
block_job_remove_all_bdrv(bjob);
bdrv_replace_node(mirror_top_bs, mirror_top_bs->backing->bs, &error_abort);
+ if (s->sync_bitmap) {
+ if (s->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS ||
@@ -54,7 +55,7 @@ index 3c9cd42c50..08ac9827f2 100644
bs_opaque->job = NULL;
bdrv_drained_end(src);
@@ -1584,10 +1594,6 @@ static BlockJob *mirror_start_job(
@@ -1699,10 +1709,6 @@ static BlockJob *mirror_start_job(
" sync mode",
MirrorSyncMode_str(sync_mode));
return NULL;
@@ -65,7 +66,7 @@ index 3c9cd42c50..08ac9827f2 100644
}
} else if (bitmap) {
error_setg(errp,
@@ -1604,6 +1610,12 @@ static BlockJob *mirror_start_job(
@@ -1719,6 +1725,12 @@ static BlockJob *mirror_start_job(
return NULL;
}
granularity = bdrv_dirty_bitmap_granularity(bitmap);

View File

@@ -10,15 +10,16 @@ as one without the other does not make much sense with the current set
of modes.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
blockdev.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/blockdev.c b/blockdev.c
index bbeff9c439..fa3c2f5548 100644
index a402fa4bf7..01b0ab0549 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3257,6 +3257,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
@@ -2946,6 +2946,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
return;
}
@@ -27,4 +28,4 @@ index bbeff9c439..fa3c2f5548 100644
+ return;
}
if (has_replaces) {
if (!replaces) {

View File

@@ -10,15 +10,16 @@ since sync_bitmap is busy at the point of merging, and we checked access
beforehand.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/mirror.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
block/mirror.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 08ac9827f2..c56b0f87e3 100644
index 11b8a8e959..00f2665ca4 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -756,8 +756,8 @@ static int mirror_exit_common(Job *job)
@@ -792,8 +792,8 @@ static int mirror_exit_common(Job *job)
job->ret == 0 && ret == 0)) {
/* Success; synchronize copy back to sync. */
bdrv_clear_dirty_bitmap(s->sync_bitmap, NULL);
@@ -29,14 +30,17 @@ index 08ac9827f2..c56b0f87e3 100644
}
}
bdrv_release_dirty_bitmap(s->dirty_bitmap);
@@ -1749,8 +1749,8 @@ static BlockJob *mirror_start_job(
@@ -1892,11 +1892,8 @@ static BlockJob *mirror_start_job(
}
if (s->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
- bdrv_merge_dirty_bitmap(s->dirty_bitmap, s->sync_bitmap,
- NULL, &local_err);
- if (local_err) {
- goto fail;
- }
+ bdrv_dirty_bitmap_merge_internal(s->dirty_bitmap, s->sync_bitmap,
+ NULL, true);
if (local_err) {
goto fail;
}
}
ret = block_job_add_bdrv(&s->common, "source", bs, 0,

View File

@@ -20,11 +20,11 @@ intentionally keeping copyright and ownership of original test case to
honor provenance.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
tests/qemu-iotests/384 | 547 +++++++
tests/qemu-iotests/384.out | 2846 ++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 3394 insertions(+)
2 files changed, 3393 insertions(+)
create mode 100755 tests/qemu-iotests/384
create mode 100644 tests/qemu-iotests/384.out
@@ -3433,15 +3433,3 @@ index 0000000000..9b7408b6d6
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "bitmap-mode must be specified if a bitmap is provided"}}
+
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 435dccd5af..939efd9c70 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -270,6 +270,7 @@
253 rw quick
254 rw backing quick
255 rw quick
+384 rw
256 rw auto quick
257 rw
258 rw quick

View File

@@ -11,6 +11,9 @@ mode was never available for drive-mirror, it makes the interface more
uniform w.r.t. backup block jobs.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 8.0]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/mirror.c | 28 +++------------
blockdev.c | 29 +++++++++++++++
@@ -18,11 +21,11 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 files changed, 70 insertions(+), 59 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index c56b0f87e3..dbba6fc80e 100644
index 00f2665ca4..60cf574de5 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1584,31 +1584,13 @@ static BlockJob *mirror_start_job(
Error *local_err = NULL;
@@ -1699,31 +1699,13 @@ static BlockJob *mirror_start_job(
uint64_t target_perms, target_shared_perms;
int ret;
- if (sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
@@ -59,17 +62,17 @@ index c56b0f87e3..dbba6fc80e 100644
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
diff --git a/blockdev.c b/blockdev.c
index fa3c2f5548..206de2b6c2 100644
index 01b0ab0549..cd5f205ad1 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3236,7 +3236,36 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
@@ -2925,7 +2925,36 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
+ if ((sync == MIRROR_SYNC_MODE_BITMAP) ||
+ (sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
+ /* done before desugaring 'incremental' to print the right message */
+ if (!has_bitmap) {
+ if (!bitmap_name) {
+ error_setg(errp, "Must provide a valid bitmap name for "
+ "'%s' sync mode", MirrorSyncMode_str(sync));
+ return;
@@ -90,7 +93,7 @@ index fa3c2f5548..206de2b6c2 100644
+ bitmap_mode = BITMAP_SYNC_MODE_ON_SUCCESS;
+ }
+
if (has_bitmap) {
if (bitmap_name) {
+ if (sync != MIRROR_SYNC_MODE_BITMAP) {
+ error_setg(errp, "Sync mode '%s' not supported with bitmap.",
+ MirrorSyncMode_str(sync));

View File

@@ -0,0 +1,206 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Mon, 23 Aug 2021 11:28:32 +0200
Subject: [PATCH] monitor/qmp: fix race with clients disconnecting early
The following sequence can produce a race condition that results in
responses meant for different clients being sent to the wrong one:
(QMP, no OOB)
1) client A connects
2) client A sends 'qmp_capabilities'
3) 'qmp_dispatch' runs in coroutine, schedules out to
'do_qmp_dispatch_bh' and yields
4) client A disconnects (i.e. aborts, crashes, etc...)
5) client B connects
6) 'do_qmp_dispatch_bh' runs 'qmp_capabilities' and wakes calling coroutine
7) capabilities are now set and 'mon->commands' is set to '&qmp_commands'
8) 'qmp_dispatch' returns to 'monitor_qmp_dispatch'
9) success message is sent to client B *without it ever having sent
'qmp_capabilities' itself*
9a) even if client B ignores it, it will now presumably send it's own
greeting, which will error because caps are already set
The fix proposed here uses an atomic, sequential connection number
stored in the MonitorQMP struct, which is incremented everytime a new
client connects. Since it is not changed on CHR_EVENT_CLOSED, the
behaviour of allowing a client to disconnect only one side of the
connection is retained.
The connection_nr needs to be exposed outside of the monitor subsystem,
since qmp_dispatch lives in qapi code. It needs to be checked twice,
once for actually running the command in the BH (fixes 7), and once for
sending back a response (fixes 9).
This satisfies my local reproducer - using multiple clients constantly
looping to open a connection, send the greeting, then exiting no longer
crashes other, normally behaving clients with unrelated responses.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
include/monitor/monitor.h | 1 +
monitor/monitor-internal.h | 7 +++++++
monitor/monitor.c | 15 +++++++++++++++
monitor/qmp.c | 15 ++++++++++++++-
qapi/qmp-dispatch.c | 21 +++++++++++++++++----
stubs/monitor-core.c | 5 +++++
6 files changed, 59 insertions(+), 5 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 965f5d5450..e04bd059b6 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -16,6 +16,7 @@ extern QemuOptsList qemu_mon_opts;
Monitor *monitor_cur(void);
Monitor *monitor_set_cur(Coroutine *co, Monitor *mon);
bool monitor_cur_is_qmp(void);
+int monitor_get_connection_nr(const Monitor *mon);
void monitor_init_globals(void);
void monitor_init_globals_core(void);
diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
index 252de85681..8db28f9272 100644
--- a/monitor/monitor-internal.h
+++ b/monitor/monitor-internal.h
@@ -151,6 +151,13 @@ typedef struct {
QemuMutex qmp_queue_lock;
/* Input queue that holds all the parsed QMP requests */
GQueue *qmp_requests;
+
+ /*
+ * A sequential number that gets incremented on every new CHR_EVENT_OPENED.
+ * Used to avoid leftover responses in BHs from being sent to the wrong
+ * client. Access with atomics.
+ */
+ int connection_nr;
} MonitorQMP;
/**
diff --git a/monitor/monitor.c b/monitor/monitor.c
index dc352f9e9d..56e1307014 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -117,6 +117,21 @@ bool monitor_cur_is_qmp(void)
return cur_mon && monitor_is_qmp(cur_mon);
}
+/**
+ * If @mon is a QMP monitor, return the connection_nr, otherwise -1.
+ */
+int monitor_get_connection_nr(const Monitor *mon)
+{
+ MonitorQMP *qmp_mon;
+
+ if (!monitor_is_qmp(mon)) {
+ return -1;
+ }
+
+ qmp_mon = container_of(mon, MonitorQMP, common);
+ return qatomic_read(&qmp_mon->connection_nr);
+}
+
/**
* Is @mon is using readline?
* Note: not all HMP monitors use readline, e.g., gdbserver has a
diff --git a/monitor/qmp.c b/monitor/qmp.c
index a239945e8d..589c9524f8 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -165,6 +165,8 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
QDict *rsp;
QDict *error;
+ int conn_nr_before = qatomic_read(&mon->connection_nr);
+
rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
&mon->common);
@@ -180,7 +182,17 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
}
}
- monitor_qmp_respond(mon, rsp);
+ /*
+ * qmp_dispatch might have yielded and waited for a BH, in which case there
+ * is a chance a new client connected in the meantime - if this happened,
+ * the command will not have been executed, but we also need to ensure that
+ * we don't send back a corresponding response on a line that no longer
+ * belongs to this request.
+ */
+ if (conn_nr_before == qatomic_read(&mon->connection_nr)) {
+ monitor_qmp_respond(mon, rsp);
+ }
+
qobject_unref(rsp);
}
@@ -461,6 +473,7 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent event)
switch (event) {
case CHR_EVENT_OPENED:
+ qatomic_inc_fetch(&mon->connection_nr);
mon->commands = &qmp_cap_negotiation_commands;
monitor_qmp_caps_reset(mon);
data = qmp_greeting(mon);
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 176b549473..790bb7d1da 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -117,16 +117,28 @@ typedef struct QmpDispatchBH {
QObject **ret;
Error **errp;
Coroutine *co;
+ int conn_nr;
} QmpDispatchBH;
static void do_qmp_dispatch_bh(void *opaque)
{
QmpDispatchBH *data = opaque;
- assert(monitor_cur() == NULL);
- monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
- data->cmd->fn(data->args, data->ret, data->errp);
- monitor_set_cur(qemu_coroutine_self(), NULL);
+ /*
+ * A QMP monitor tracks it's client with a connection number, if this
+ * changes during the scheduling delay of this BH, we must not execute the
+ * command. Otherwise a badly placed 'qmp_capabilities' might affect the
+ * connection state of a client it was never meant for.
+ */
+ if (data->conn_nr == monitor_get_connection_nr(data->cur_mon)) {
+ assert(monitor_cur() == NULL);
+ monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
+ data->cmd->fn(data->args, data->ret, data->errp);
+ monitor_set_cur(qemu_coroutine_self(), NULL);
+ } else {
+ error_setg(data->errp, "active monitor connection changed");
+ }
+
aio_co_wake(data->co);
}
@@ -253,6 +265,7 @@ QDict *coroutine_mixed_fn qmp_dispatch(const QmpCommandList *cmds, QObject *requ
.ret = &ret,
.errp = &err,
.co = qemu_coroutine_self(),
+ .conn_nr = monitor_get_connection_nr(cur_mon),
};
aio_bh_schedule_oneshot(iohandler_get_aio_context(), do_qmp_dispatch_bh,
&data);
diff --git a/stubs/monitor-core.c b/stubs/monitor-core.c
index afa477aae6..d3ff124bf3 100644
--- a/stubs/monitor-core.c
+++ b/stubs/monitor-core.c
@@ -12,6 +12,11 @@ Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
return NULL;
}
+int monitor_get_connection_nr(const Monitor *mon)
+{
+ return -1;
+}
+
void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
{
}

View File

@@ -0,0 +1,69 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Guenter Roeck <linux@roeck-us.net>
Date: Tue, 28 Feb 2023 09:11:29 -0800
Subject: [PATCH] scsi: megasas: Internal cdbs have 16-byte length
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Host drivers do not necessarily set cdb_len in megasas io commands.
With commits 6d1511cea0 ("scsi: Reject commands if the CDB length
exceeds buf_len") and fe9d8927e2 ("scsi: Add buf_len parameter to
scsi_req_new()"), this results in failures to boot Linux from affected
SCSI drives because cdb_len is set to 0 by the host driver.
Set the cdb length to its actual size to solve the problem.
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
(picked-up from https://lists.nongnu.org/archive/html/qemu-devel/2023-02/msg08653.html)
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/scsi/megasas.c | 14 ++------------
1 file changed, 2 insertions(+), 12 deletions(-)
diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c
index 32c70c9e99..984b6a3145 100644
--- a/hw/scsi/megasas.c
+++ b/hw/scsi/megasas.c
@@ -1781,7 +1781,7 @@ static int megasas_handle_io(MegasasState *s, MegasasCmd *cmd, int frame_cmd)
uint8_t cdb[16];
int len;
struct SCSIDevice *sdev = NULL;
- int target_id, lun_id, cdb_len;
+ int target_id, lun_id;
lba_count = le32_to_cpu(cmd->frame->io.header.data_len);
lba_start_lo = le32_to_cpu(cmd->frame->io.lba_lo);
@@ -1790,7 +1790,6 @@ static int megasas_handle_io(MegasasState *s, MegasasCmd *cmd, int frame_cmd)
target_id = cmd->frame->header.target_id;
lun_id = cmd->frame->header.lun_id;
- cdb_len = cmd->frame->header.cdb_len;
if (target_id < MFI_MAX_LD && lun_id == 0) {
sdev = scsi_device_find(&s->bus, 0, target_id, lun_id);
@@ -1805,15 +1804,6 @@ static int megasas_handle_io(MegasasState *s, MegasasCmd *cmd, int frame_cmd)
return MFI_STAT_DEVICE_NOT_FOUND;
}
- if (cdb_len > 16) {
- trace_megasas_scsi_invalid_cdb_len(
- mfi_frame_desc(frame_cmd), 1, target_id, lun_id, cdb_len);
- megasas_write_sense(cmd, SENSE_CODE(INVALID_OPCODE));
- cmd->frame->header.scsi_status = CHECK_CONDITION;
- s->event_count++;
- return MFI_STAT_SCSI_DONE_WITH_ERROR;
- }
-
cmd->iov_size = lba_count * sdev->blocksize;
if (megasas_map_sgl(s, cmd, &cmd->frame->io.sgl)) {
megasas_write_sense(cmd, SENSE_CODE(TARGET_FAILURE));
@@ -1824,7 +1814,7 @@ static int megasas_handle_io(MegasasState *s, MegasasCmd *cmd, int frame_cmd)
megasas_encode_lba(cdb, lba_start, lba_count, is_write);
cmd->req = scsi_req_new(sdev, cmd->index,
- lun_id, cdb, cdb_len, cmd);
+ lun_id, cdb, sizeof(cdb), cmd);
if (!cmd->req) {
trace_megasas_scsi_req_alloc_failed(
mfi_frame_desc(frame_cmd), target_id, lun_id);

View File

@@ -0,0 +1,100 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Tue, 7 Mar 2023 15:03:02 +0100
Subject: [PATCH] ide: avoid potential deadlock when draining during trim
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The deadlock can happen as follows:
1. ide_issue_trim is called, and increments the in_flight counter.
2. ide_issue_trim_cb calls blk_aio_pdiscard.
3. Somebody else starts draining (e.g. backup to insert the cbw node).
4. ide_issue_trim_cb is called as the completion callback for
blk_aio_pdiscard.
5. ide_issue_trim_cb issues yet another blk_aio_pdiscard request.
6. The request is added to the wait queue via blk_wait_while_drained,
because draining has been started.
7. Nobody ever decrements the in_flight counter and draining can't
finish. This would be done by ide_trim_bh_cb, which is called after
ide_issue_trim_cb has issued its last request, but
ide_issue_trim_cb is not called anymore, because it's the
completion callback of blk_aio_pdiscard, which waits on draining.
Quoting Hanna Czenczek:
> The point of 7e5cdb345f was that we need any in-flight count to
> accompany a set s->bus->dma->aiocb. While blk_aio_pdiscard() is
> happening, we dont necessarily need another count. But we do need
> it while there is no blk_aio_pdiscard().
> ide_issue_trim_cb() returns in two cases (and, recursively through
> its callers, leaves s->bus->dma->aiocb set):
> 1. After calling blk_aio_pdiscard(), which will keep an in-flight
> count,
> 2. After calling replay_bh_schedule_event() (i.e.
> qemu_bh_schedule()), which does not keep an in-flight count.
Thus, even after moving the blk_inc_in_flight to above the
replay_bh_schedule_event call, the invariant "ide_issue_trim_cb
returns with an accompanying in-flight count" is still satisfied.
However, the issue 7e5cdb345f fixed for canceling resurfaces, because
ide_cancel_dma_sync assumes that it just needs to drain once. But now
the in_flight count is not consistently > 0 during the trim operation.
So, change it to drain until !s->bus->dma->aiocb, which means that the
operation finished (s->bus->dma->aiocb is cleared by ide_set_inactive
via the ide_dma_cb when the end of the transfer is reached).
Discussion here:
https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg02506.html
Fixes: 7e5cdb345f ("ide: Increment BB in-flight counter for TRIM BH")
Suggested-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/ide/core.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/hw/ide/core.c b/hw/ide/core.c
index c3508acbb1..289347af58 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -444,7 +444,7 @@ static void ide_trim_bh_cb(void *opaque)
iocb->bh = NULL;
qemu_aio_unref(iocb);
- /* Paired with an increment in ide_issue_trim() */
+ /* Paired with an increment in ide_issue_trim_cb() */
blk_dec_in_flight(blk);
}
@@ -504,6 +504,8 @@ static void ide_issue_trim_cb(void *opaque, int ret)
done:
iocb->aiocb = NULL;
if (iocb->bh) {
+ /* Paired with a decrement in ide_trim_bh_cb() */
+ blk_inc_in_flight(s->blk);
replay_bh_schedule_event(iocb->bh);
}
}
@@ -516,9 +518,6 @@ BlockAIOCB *ide_issue_trim(
IDEDevice *dev = s->unit ? s->bus->slave : s->bus->master;
TrimAIOCB *iocb;
- /* Paired with a decrement in ide_trim_bh_cb() */
- blk_inc_in_flight(s->blk);
-
iocb = blk_aio_get(&trim_aiocb_info, s->blk, cb, cb_opaque);
iocb->s = s;
iocb->bh = qemu_bh_new_guarded(ide_trim_bh_cb, iocb,
@@ -742,8 +741,9 @@ void ide_cancel_dma_sync(IDEState *s)
*/
if (s->bus->dma->aiocb) {
trace_ide_cancel_dma_sync_remaining();
- blk_drain(s->blk);
- assert(s->bus->dma->aiocb == NULL);
+ while (s->bus->dma->aiocb) {
+ blk_drain(s->blk);
+ }
}
}

View File

@@ -0,0 +1,48 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Fri, 28 Jul 2023 10:47:48 +0200
Subject: [PATCH] migration/block-dirty-bitmap: fix loading bitmap when there
is an iothread
The bdrv_create_dirty_bitmap() function (which is also called by
bdrv_dirty_bitmap_create_successor()) uses bdrv_getlength(bs). This is
a wrapper around a coroutine, and thus uses bdrv_poll_co(). Polling
tries to release the AioContext which will trigger an assert() if it
hasn't been acquired before.
The issue does not happen for migration, because there we are in a
coroutine already, so the wrapper will just call bdrv_co_getlength()
directly without polling.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
migration/block-dirty-bitmap.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 032fc5f405..e1ae3b7316 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -805,8 +805,11 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoadState *s)
"destination", bdrv_dirty_bitmap_name(s->bitmap));
return -EINVAL;
} else {
+ AioContext *ctx = bdrv_get_aio_context(s->bs);
+ aio_context_acquire(ctx);
s->bitmap = bdrv_create_dirty_bitmap(s->bs, granularity,
s->bitmap_name, &local_err);
+ aio_context_release(ctx);
if (!s->bitmap) {
error_report_err(local_err);
return -EINVAL;
@@ -833,7 +836,10 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoadState *s)
bdrv_disable_dirty_bitmap(s->bitmap);
if (flags & DIRTY_BITMAP_MIG_START_FLAG_ENABLED) {
+ AioContext *ctx = bdrv_get_aio_context(s->bs);
+ aio_context_acquire(ctx);
bdrv_dirty_bitmap_create_successor(s->bitmap, &local_err);
+ aio_context_release(ctx);
if (local_err) {
error_report_err(local_err);
return -EINVAL;

View File

@@ -0,0 +1,140 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Thu, 28 Sep 2023 10:07:03 +0200
Subject: [PATCH] Revert "Revert "graph-lock: Disable locking for now""
This reverts commit 3cce22defb4b0e47cf135444e30cc673cff5ebad.
There are still some issues with graph locking, e.g. deadlocks during
backup canceling [0]. Because the AioContext locks still exist, it
should be safe to disable locking again.
From the original 80fc5d2600 ("graph-lock: Disable locking for now"):
> We don't currently rely on graph locking yet. It is supposed to replace
> the AioContext lock eventually to enable multiqueue support, but as long
> as we still have the AioContext lock, it is sufficient without the graph
> lock. Once the AioContext lock goes away, the deadlock doesn't exist any
> more either and this commit can be reverted. (Of course, it can also be
> reverted while the AioContext lock still exists if the callers have been
> fixed.)
[0]: https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg00729.html
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/graph-lock.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/block/graph-lock.c b/block/graph-lock.c
index 5e66f01ae8..5c2873262a 100644
--- a/block/graph-lock.c
+++ b/block/graph-lock.c
@@ -30,8 +30,10 @@ BdrvGraphLock graph_lock;
/* Protects the list of aiocontext and orphaned_reader_count */
static QemuMutex aio_context_list_lock;
+#if 0
/* Written and read with atomic operations. */
static int has_writer;
+#endif
/*
* A reader coroutine could move from an AioContext to another.
@@ -88,6 +90,7 @@ void unregister_aiocontext(AioContext *ctx)
g_free(ctx->bdrv_graph);
}
+#if 0
static uint32_t reader_count(void)
{
BdrvGraphRWlock *brdv_graph;
@@ -105,12 +108,19 @@ static uint32_t reader_count(void)
assert((int32_t)rd >= 0);
return rd;
}
+#endif
void bdrv_graph_wrlock(BlockDriverState *bs)
{
+#if 0
AioContext *ctx = NULL;
GLOBAL_STATE_CODE();
+ /*
+ * TODO Some callers hold an AioContext lock when this is called, which
+ * causes deadlocks. Reenable once the AioContext locking is cleaned up (or
+ * AioContext locks are gone).
+ */
assert(!qatomic_read(&has_writer));
/*
@@ -158,11 +168,13 @@ void bdrv_graph_wrlock(BlockDriverState *bs)
if (ctx) {
aio_context_acquire(bdrv_get_aio_context(bs));
}
+#endif
}
void bdrv_graph_wrunlock(void)
{
GLOBAL_STATE_CODE();
+#if 0
QEMU_LOCK_GUARD(&aio_context_list_lock);
assert(qatomic_read(&has_writer));
@@ -174,10 +186,13 @@ void bdrv_graph_wrunlock(void)
/* Wake up all coroutine that are waiting to read the graph */
qemu_co_enter_all(&reader_queue, &aio_context_list_lock);
+#endif
}
void coroutine_fn bdrv_graph_co_rdlock(void)
{
+ /* TODO Reenable when wrlock is reenabled */
+#if 0
BdrvGraphRWlock *bdrv_graph;
bdrv_graph = qemu_get_current_aio_context()->bdrv_graph;
@@ -237,10 +252,12 @@ void coroutine_fn bdrv_graph_co_rdlock(void)
qemu_co_queue_wait(&reader_queue, &aio_context_list_lock);
}
}
+#endif
}
void coroutine_fn bdrv_graph_co_rdunlock(void)
{
+#if 0
BdrvGraphRWlock *bdrv_graph;
bdrv_graph = qemu_get_current_aio_context()->bdrv_graph;
@@ -258,6 +275,7 @@ void coroutine_fn bdrv_graph_co_rdunlock(void)
if (qatomic_read(&has_writer)) {
aio_wait_kick();
}
+#endif
}
void bdrv_graph_rdlock_main_loop(void)
@@ -275,13 +293,19 @@ void bdrv_graph_rdunlock_main_loop(void)
void assert_bdrv_graph_readable(void)
{
/* reader_count() is slow due to aio_context_list_lock lock contention */
+ /* TODO Reenable when wrlock is reenabled */
+#if 0
#ifdef CONFIG_DEBUG_GRAPH_LOCK
assert(qemu_in_main_thread() || reader_count());
#endif
+#endif
}
void assert_bdrv_graph_writable(void)
{
assert(qemu_in_main_thread());
+ /* TODO Reenable when wrlock is reenabled */
+#if 0
assert(qatomic_read(&has_writer));
+#endif
}

View File

@@ -0,0 +1,57 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Thu, 28 Sep 2023 11:19:14 +0200
Subject: [PATCH] migration states: workaround snapshot performance regression
Commit 813cd616 ("migration: Use migration_transferred_bytes() to
calculate rate_limit") introduced a prohibitive performance regression
when taking a snapshot [0]. The reason turns out to be the flushing
done by migration_transferred_bytes()
Just use a _noflush version of the relevant function as a workaround
until upstream fixes the issue. This is inspired by a not-applied
upstream series [1], but doing the very minimum to avoid the
regression.
[0]: https://gitlab.com/qemu-project/qemu/-/issues/1821
[1]: https://lists.nongnu.org/archive/html/qemu-devel/2023-05/msg07708.html
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
migration/migration-stats.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/migration/migration-stats.c b/migration/migration-stats.c
index 095d6d75bb..8073c8ebaa 100644
--- a/migration/migration-stats.c
+++ b/migration/migration-stats.c
@@ -18,6 +18,20 @@
MigrationAtomicStats mig_stats;
+/*
+ * Same as migration_transferred_bytes below, but using the _noflush
+ * variant of qemu_file_transferred() to avoid a performance
+ * regression in migration_rate_exceeded().
+ */
+static uint64_t migration_transferred_bytes_noflush(QEMUFile *f)
+{
+ uint64_t multifd = stat64_get(&mig_stats.multifd_bytes);
+ uint64_t qemu_file = qemu_file_transferred_noflush(f);
+
+ trace_migration_transferred_bytes(qemu_file, multifd);
+ return qemu_file + multifd;
+}
+
bool migration_rate_exceeded(QEMUFile *f)
{
if (qemu_file_get_error(f)) {
@@ -25,7 +39,7 @@ bool migration_rate_exceeded(QEMUFile *f)
}
uint64_t rate_limit_start = stat64_get(&mig_stats.rate_limit_start);
- uint64_t rate_limit_current = migration_transferred_bytes(f);
+ uint64_t rate_limit_current = migration_transferred_bytes_noflush(f);
uint64_t rate_limit_used = rate_limit_current - rate_limit_start;
uint64_t rate_limit_max = stat64_get(&mig_stats.rate_limit_max);

View File

@@ -0,0 +1,45 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Fri, 17 Nov 2023 11:18:06 +0100
Subject: [PATCH] Revert "x86: acpi: workaround Windows not handling name
references in Package properly"
This reverts commit 44d975ef340e2f21f236f9520c53e1b30d2213a4.
As reported in the community forum [0] and reproduced locally this
breaks VirtIO network adapters in (at least) the German ISO of Windows
Server 2022. The fix itself was for
> Issue is not fatal but as result acpi-index/"PCI Label ID" property
> is either not shown in device details page or shows incorrect value.
so revert and tolerate that as a stop-gap, rather than have the
devices not working at all.
[0]: https://forum.proxmox.com/threads/92094/post-605684
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/i386/acpi-build.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index bb12b0ad43..de14d3c3da 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -362,13 +362,9 @@ Aml *aml_pci_device_dsm(void)
{
Aml *params = aml_local(0);
Aml *pkg = aml_package(2);
- aml_append(pkg, aml_int(0));
- aml_append(pkg, aml_int(0));
+ aml_append(pkg, aml_name("BSEL"));
+ aml_append(pkg, aml_name("ASUN"));
aml_append(method, aml_store(pkg, params));
- aml_append(method,
- aml_store(aml_name("BSEL"), aml_index(params, aml_int(0))));
- aml_append(method,
- aml_store(aml_name("ASUN"), aml_index(params, aml_int(1))));
aml_append(method,
aml_return(aml_call5("PDSM", aml_arg(0), aml_arg(1),
aml_arg(2), aml_arg(3), params))

View File

@@ -0,0 +1,34 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Tue, 1 Feb 2022 20:09:41 +0100
Subject: [PATCH] target/i386: the sgx_epc_get_section stub is reachable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The sgx_epc_get_section stub is reachable from cpu_x86_cpuid. It
should not assert, instead it should just return true just like
the "real" sgx_epc_get_section does when SGX is disabled.
Reported-by: Vladimír Beneš <vbenes@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20220201190941.106001-1-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry-picked from commit 219615740425d9683588207b40a365e6741691a6)
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/i386/sgx-stub.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/i386/sgx-stub.c b/hw/i386/sgx-stub.c
index 26833eb233..16b1dfd90b 100644
--- a/hw/i386/sgx-stub.c
+++ b/hw/i386/sgx-stub.c
@@ -34,5 +34,5 @@ void pc_machine_init_sgx_epc(PCMachineState *pcms)
bool sgx_epc_get_section(int section_nr, uint64_t *addr, uint64_t *size)
{
- g_assert_not_reached();
+ return true;
}

View File

@@ -0,0 +1,86 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Wed, 24 Jan 2024 11:57:48 +0100
Subject: [PATCH] ui/clipboard: mark type as not available when there is no
data
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
With VNC, a client can send a non-extended VNC_MSG_CLIENT_CUT_TEXT
message with len=0. In qemu_clipboard_set_data(), the clipboard info
will be updated setting data to NULL (because g_memdup(data, size)
returns NULL when size is 0). If the client does not set the
VNC_ENCODING_CLIPBOARD_EXT feature when setting up the encodings, then
the 'request' callback for the clipboard peer is not initialized.
Later, because data is NULL, qemu_clipboard_request() can be reached
via vdagent_chr_write() and vdagent_clipboard_recv_request() and
there, the clipboard owner's 'request' callback will be attempted to
be called, but that is a NULL pointer.
In particular, this can happen when using the KRDC (22.12.3) VNC
client.
Another scenario leading to the same issue is with two clients (say
noVNC and KRDC):
The noVNC client sets the extension VNC_FEATURE_CLIPBOARD_EXT and
initializes its cbpeer.
The KRDC client does not, but triggers a vnc_client_cut_text() (note
it's not the _ext variant)). There, a new clipboard info with it as
the 'owner' is created and via qemu_clipboard_set_data() is called,
which in turn calls qemu_clipboard_update() with that info.
In qemu_clipboard_update(), the notifier for the noVNC client will be
called, i.e. vnc_clipboard_notify() and also set vs->cbinfo for the
noVNC client. The 'owner' in that clipboard info is the clipboard peer
for the KRDC client, which did not initialize the 'request' function.
That sounds correct to me, it is the owner of that clipboard info.
Then when noVNC sends a VNC_MSG_CLIENT_CUT_TEXT message (it did set
the VNC_FEATURE_CLIPBOARD_EXT feature correctly, so a check for it
passes), that clipboard info is passed to qemu_clipboard_request() and
the original segfault still happens.
Fix the issue by handling updates with size 0 differently. In
particular, mark in the clipboard info that the type is not available.
While at it, switch to g_memdup2(), because g_memdup() is deprecated.
Cc: qemu-stable@nongnu.org
Fixes: CVE-2023-6683
Reported-by: Markus Frank <m.frank@proxmox.com>
Suggested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Markus Frank <m.frank@proxmox.com>
(picked from https://lists.nongnu.org/archive/html/qemu-stable/2024-01/msg00228.html)
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
ui/clipboard.c | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/ui/clipboard.c b/ui/clipboard.c
index 3d14bffaf8..b3f6fa3c9e 100644
--- a/ui/clipboard.c
+++ b/ui/clipboard.c
@@ -163,9 +163,15 @@ void qemu_clipboard_set_data(QemuClipboardPeer *peer,
}
g_free(info->types[type].data);
- info->types[type].data = g_memdup(data, size);
- info->types[type].size = size;
- info->types[type].available = true;
+ if (size) {
+ info->types[type].data = g_memdup2(data, size);
+ info->types[type].size = size;
+ info->types[type].available = true;
+ } else {
+ info->types[type].data = NULL;
+ info->types[type].size = 0;
+ info->types[type].available = false;
+ }
if (update) {
qemu_clipboard_update(info);

View File

@@ -0,0 +1,65 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Hanna Czenczek <hreitz@redhat.com>
Date: Fri, 2 Feb 2024 16:31:56 +0100
Subject: [PATCH] virtio-scsi: Attach event vq notifier with no_poll
As of commit 38738f7dbbda90fbc161757b7f4be35b52205552 ("virtio-scsi:
don't waste CPU polling the event virtqueue"), we only attach an io_read
notifier for the virtio-scsi event virtqueue instead, and no polling
notifiers. During operation, the event virtqueue is typically
non-empty, but none of the buffers are intended to be used immediately.
Instead, they only get used when certain events occur. Therefore, it
makes no sense to continuously poll it when non-empty, because it is
supposed to be and stay non-empty.
We do this by using virtio_queue_aio_attach_host_notifier_no_poll()
instead of virtio_queue_aio_attach_host_notifier() for the event
virtqueue.
Commit 766aa2de0f29b657148e04599320d771c36fd126 ("virtio-scsi: implement
BlockDevOps->drained_begin()") however has virtio_scsi_drained_end() use
virtio_queue_aio_attach_host_notifier() for all virtqueues, including
the event virtqueue. This can lead to it being polled again, undoing
the benefit of commit 38738f7dbbda90fbc161757b7f4be35b52205552.
Fix it by using virtio_queue_aio_attach_host_notifier_no_poll() for the
event virtqueue.
("virtio-scsi: implement BlockDevOps->drained_begin()")
Reported-by: Fiona Ebner <f.ebner@proxmox.com>
Fixes: 766aa2de0f29b657148e04599320d771c36fd126
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
hw/scsi/virtio-scsi.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 45b95ea070..ad24a882fd 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -1148,6 +1148,7 @@ static void virtio_scsi_drained_begin(SCSIBus *bus)
static void virtio_scsi_drained_end(SCSIBus *bus)
{
VirtIOSCSI *s = container_of(bus, VirtIOSCSI, bus);
+ VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
VirtIODevice *vdev = VIRTIO_DEVICE(s);
uint32_t total_queues = VIRTIO_SCSI_VQ_NUM_FIXED +
s->parent_obj.conf.num_queues;
@@ -1165,7 +1166,11 @@ static void virtio_scsi_drained_end(SCSIBus *bus)
for (uint32_t i = 0; i < total_queues; i++) {
VirtQueue *vq = virtio_get_queue(vdev, i);
- virtio_queue_aio_attach_host_notifier(vq, s->ctx);
+ if (vq == vs->event_vq) {
+ virtio_queue_aio_attach_host_notifier_no_poll(vq, s->ctx);
+ } else {
+ virtio_queue_aio_attach_host_notifier(vq, s->ctx);
+ }
}
}

View File

@@ -0,0 +1,125 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Hanna Czenczek <hreitz@redhat.com>
Date: Fri, 2 Feb 2024 16:31:57 +0100
Subject: [PATCH] virtio: Re-enable notifications after drain
During drain, we do not care about virtqueue notifications, which is why
we remove the handlers on it. When removing those handlers, whether vq
notifications are enabled or not depends on whether we were in polling
mode or not; if not, they are enabled (by default); if so, they have
been disabled by the io_poll_start callback.
Because we do not care about those notifications after removing the
handlers, this is fine. However, we have to explicitly ensure they are
enabled when re-attaching the handlers, so we will resume receiving
notifications. We do this in virtio_queue_aio_attach_host_notifier*().
If such a function is called while we are in a polling section,
attaching the notifiers will then invoke the io_poll_start callback,
re-disabling notifications.
Because we will always miss virtqueue updates in the drained section, we
also need to poll the virtqueue once after attaching the notifiers.
Buglink: https://issues.redhat.com/browse/RHEL-3934
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
hw/virtio/virtio.c | 42 ++++++++++++++++++++++++++++++++++++++++++
include/block/aio.h | 7 ++++++-
2 files changed, 48 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 969c25f4cf..02cce83111 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3526,6 +3526,17 @@ static void virtio_queue_host_notifier_aio_poll_end(EventNotifier *n)
void virtio_queue_aio_attach_host_notifier(VirtQueue *vq, AioContext *ctx)
{
+ /*
+ * virtio_queue_aio_detach_host_notifier() can leave notifications disabled.
+ * Re-enable them. (And if detach has not been used before, notifications
+ * being enabled is still the default state while a notifier is attached;
+ * see virtio_queue_host_notifier_aio_poll_end(), which will always leave
+ * notifications enabled once the polling section is left.)
+ */
+ if (!virtio_queue_get_notification(vq)) {
+ virtio_queue_set_notification(vq, 1);
+ }
+
aio_set_event_notifier(ctx, &vq->host_notifier,
virtio_queue_host_notifier_read,
virtio_queue_host_notifier_aio_poll,
@@ -3533,6 +3544,13 @@ void virtio_queue_aio_attach_host_notifier(VirtQueue *vq, AioContext *ctx)
aio_set_event_notifier_poll(ctx, &vq->host_notifier,
virtio_queue_host_notifier_aio_poll_begin,
virtio_queue_host_notifier_aio_poll_end);
+
+ /*
+ * We will have ignored notifications about new requests from the guest
+ * while no notifiers were attached, so "kick" the virt queue to process
+ * those requests now.
+ */
+ event_notifier_set(&vq->host_notifier);
}
/*
@@ -3543,14 +3561,38 @@ void virtio_queue_aio_attach_host_notifier(VirtQueue *vq, AioContext *ctx)
*/
void virtio_queue_aio_attach_host_notifier_no_poll(VirtQueue *vq, AioContext *ctx)
{
+ /* See virtio_queue_aio_attach_host_notifier() */
+ if (!virtio_queue_get_notification(vq)) {
+ virtio_queue_set_notification(vq, 1);
+ }
+
aio_set_event_notifier(ctx, &vq->host_notifier,
virtio_queue_host_notifier_read,
NULL, NULL);
+
+ /*
+ * See virtio_queue_aio_attach_host_notifier().
+ * Note that this may be unnecessary for the type of virtqueues this
+ * function is used for. Still, it will not hurt to have a quick look into
+ * whether we can/should process any of the virtqueue elements.
+ */
+ event_notifier_set(&vq->host_notifier);
}
void virtio_queue_aio_detach_host_notifier(VirtQueue *vq, AioContext *ctx)
{
aio_set_event_notifier(ctx, &vq->host_notifier, NULL, NULL, NULL);
+
+ /*
+ * aio_set_event_notifier_poll() does not guarantee whether io_poll_end()
+ * will run after io_poll_begin(), so by removing the notifier, we do not
+ * know whether virtio_queue_host_notifier_aio_poll_end() has run after a
+ * previous virtio_queue_host_notifier_aio_poll_begin(), i.e. whether
+ * notifications are enabled or disabled. It does not really matter anyway;
+ * we just removed the notifier, so we do not care about notifications until
+ * we potentially re-attach it. The attach_host_notifier functions will
+ * ensure that notifications are enabled again when they are needed.
+ */
}
void virtio_queue_host_notifier_read(EventNotifier *n)
diff --git a/include/block/aio.h b/include/block/aio.h
index 32042e8905..79efadfa48 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -498,9 +498,14 @@ void aio_set_event_notifier(AioContext *ctx,
AioPollFn *io_poll,
EventNotifierHandler *io_poll_ready);
-/* Set polling begin/end callbacks for an event notifier that has already been
+/*
+ * Set polling begin/end callbacks for an event notifier that has already been
* registered with aio_set_event_notifier. Do nothing if the event notifier is
* not registered.
+ *
+ * Note that if the io_poll_end() callback (or the entire notifier) is removed
+ * during polling, it will not be called, so an io_poll_begin() is not
+ * necessarily always followed by an io_poll_end().
*/
void aio_set_event_notifier_poll(AioContext *ctx,
EventNotifier *notifier,

1271
debian/patches/pve-qemu-8.1-vitastor.patch vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -14,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 7e19bbff5f..b527e82a82 100644
index 7f540b03ed..ca551baa42 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -450,7 +450,7 @@ static QemuOptsList raw_runtime_opts = {
@@ -563,7 +563,7 @@ static QemuOptsList raw_runtime_opts = {
{
.name = "locking",
.type = QEMU_OPT_STRING,
@@ -26,7 +26,7 @@ index 7e19bbff5f..b527e82a82 100644
},
{
.name = "pr-manager",
@@ -550,7 +550,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
@@ -663,7 +663,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
s->use_lock = false;
break;
case ON_OFF_AUTO_AUTO:

View File

@@ -5,22 +5,21 @@ Subject: [PATCH] PVE: [Config] Adjust network script path to /etc/kvm/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
include/net/net.h | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
include/net/net.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/net/net.h b/include/net/net.h
index 39085d9444..487e3ea1b4 100644
index 685ec58318..22edf4ee96 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -208,8 +208,9 @@ void netdev_add(QemuOpts *opts, Error **errp);
@@ -260,8 +260,8 @@ void netdev_add(QemuOpts *opts, Error **errp);
int net_hub_id_for_client(NetClientState *nc, int *id);
NetClientState *net_hub_port_find(int hub_id);
-#define DEFAULT_NETWORK_SCRIPT "/etc/qemu-ifup"
-#define DEFAULT_NETWORK_DOWN_SCRIPT "/etc/qemu-ifdown"
+#define DEFAULT_NETWORK_SCRIPT "/etc/kvm/kvm-ifup"
+#define DEFAULT_NETWORK_DOWN_SCRIPT "/etc/kvm/kvm-ifdown"
+
-#define DEFAULT_NETWORK_SCRIPT CONFIG_SYSCONFDIR "/qemu-ifup"
-#define DEFAULT_NETWORK_DOWN_SCRIPT CONFIG_SYSCONFDIR "/qemu-ifdown"
+#define DEFAULT_NETWORK_SCRIPT CONFIG_SYSCONFDIR "/kvm/kvm-ifup"
+#define DEFAULT_NETWORK_DOWN_SCRIPT CONFIG_SYSCONFDIR "/kvm/kvm-ifdown"
#define DEFAULT_BRIDGE_HELPER CONFIG_QEMU_HELPERDIR "/qemu-bridge-helper"
#define DEFAULT_BRIDGE_INTERFACE "br0"

View File

@@ -10,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index e818fc712a..dd9bf7b3da 100644
index 0893b794e9..6d650a58b9 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1954,9 +1954,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
@@ -2243,9 +2243,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
#define CPU_RESOLVING_TYPE TYPE_X86_CPU
#ifdef TARGET_X86_64
@@ -24,4 +24,4 @@ index e818fc712a..dd9bf7b3da 100644
+#define TARGET_DEFAULT_CPU_TYPE X86_CPU_TYPE_NAME("kvm32")
#endif
#define cpu_signal_handler cpu_x86_signal_handler
#define cpu_list x86_cpu_list

View File

@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/ui/spice-core.c b/ui/spice-core.c
index ecc2ec2c55..ca04965ead 100644
index 52a59386d7..b20c25aee0 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -668,32 +668,35 @@ void qemu_spice_init(void)
@@ -691,32 +691,35 @@ static void qemu_spice_init(void)
if (tls_port) {
x509_dir = qemu_opt_get(opts, "x509-dir");

View File

@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/block/gluster.c b/block/gluster.c
index 0aa1f2cda4..dcd1ef7ebc 100644
index ad5fadbe79..d0011085c4 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -42,7 +42,7 @@
@@ -43,7 +43,7 @@
#define GLUSTER_DEBUG_DEFAULT 4
#define GLUSTER_DEBUG_MAX 9
#define GLUSTER_OPT_LOGFILE "logfile"
@@ -21,15 +21,15 @@ index 0aa1f2cda4..dcd1ef7ebc 100644
/*
* Several versions of GlusterFS (3.12? -> 6.0.1) fail when the transfer size
* is greater or equal to 1024 MiB, so we are limiting the transfer size to 512
@@ -424,6 +424,7 @@ static struct glfs *qemu_gluster_glfs_init(BlockdevOptionsGluster *gconf,
@@ -425,6 +425,7 @@ static struct glfs *qemu_gluster_glfs_init(BlockdevOptionsGluster *gconf,
int old_errno;
SocketAddressList *server;
unsigned long long port;
uint64_t port;
+ const char *logfile;
glfs = glfs_find_preopened(gconf->volume);
if (glfs) {
@@ -466,9 +467,15 @@ static struct glfs *qemu_gluster_glfs_init(BlockdevOptionsGluster *gconf,
@@ -467,9 +468,15 @@ static struct glfs *qemu_gluster_glfs_init(BlockdevOptionsGluster *gconf,
}
}

View File

@@ -1,24 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Mon, 6 Apr 2020 12:16:34 +0200
Subject: [PATCH] PVE: [Config] smm_available = false
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
hw/i386/x86.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index b82770024c..bd05b3c79a 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -896,7 +896,7 @@ bool x86_machine_is_smm_enabled(X86MachineState *x86ms)
if (tcg_enabled() || qtest_enabled()) {
smm_available = true;
} else if (kvm_enabled()) {
- smm_available = kvm_has_smm();
+ smm_available = false;
}
if (smm_available) {

View File

@@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+)
diff --git a/block/rbd.c b/block/rbd.c
index e637639a07..5717e7258c 100644
index 978671411e..a4749f3b1b 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -651,6 +651,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
@@ -963,6 +963,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
rados_conf_set(*cluster, "rbd_cache", "false");
}

View File

@@ -16,10 +16,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/block/gluster.c b/block/gluster.c
index dcd1ef7ebc..ac79b4bdb4 100644
index d0011085c4..2df3d6e35d 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -57,6 +57,7 @@ typedef struct GlusterAIOCB {
@@ -58,6 +58,7 @@ typedef struct GlusterAIOCB {
int ret;
Coroutine *coroutine;
AioContext *aio_context;
@@ -27,7 +27,7 @@ index dcd1ef7ebc..ac79b4bdb4 100644
} GlusterAIOCB;
typedef struct BDRVGlusterState {
@@ -763,8 +764,10 @@ static void gluster_finish_aiocb(struct glfs_fd *fd, ssize_t ret,
@@ -753,8 +754,10 @@ static void gluster_finish_aiocb(struct glfs_fd *fd, ssize_t ret,
acb->ret = 0; /* Success */
} else if (ret < 0) {
acb->ret = -errno; /* Read/Write failed */
@@ -39,15 +39,15 @@ index dcd1ef7ebc..ac79b4bdb4 100644
}
aio_co_schedule(acb->aio_context, acb->coroutine);
@@ -1035,6 +1038,7 @@ static coroutine_fn int qemu_gluster_co_pwrite_zeroes(BlockDriverState *bs,
@@ -1021,6 +1024,7 @@ static coroutine_fn int qemu_gluster_co_pwrite_zeroes(BlockDriverState *bs,
acb.ret = 0;
acb.coroutine = qemu_coroutine_self();
acb.aio_context = bdrv_get_aio_context(bs);
+ acb.is_write = true;
ret = glfs_zerofill_async(s->fd, offset, size, gluster_finish_aiocb, &acb);
ret = glfs_zerofill_async(s->fd, offset, bytes, gluster_finish_aiocb, &acb);
if (ret < 0) {
@@ -1216,9 +1220,11 @@ static coroutine_fn int qemu_gluster_co_rw(BlockDriverState *bs,
@@ -1201,9 +1205,11 @@ static coroutine_fn int qemu_gluster_co_rw(BlockDriverState *bs,
acb.aio_context = bdrv_get_aio_context(bs);
if (write) {
@@ -59,7 +59,7 @@ index dcd1ef7ebc..ac79b4bdb4 100644
ret = glfs_preadv_async(s->fd, qiov->iov, qiov->niov, offset, 0,
gluster_finish_aiocb, &acb);
}
@@ -1281,6 +1287,7 @@ static coroutine_fn int qemu_gluster_co_flush_to_disk(BlockDriverState *bs)
@@ -1266,6 +1272,7 @@ static coroutine_fn int qemu_gluster_co_flush_to_disk(BlockDriverState *bs)
acb.ret = 0;
acb.coroutine = qemu_coroutine_self();
acb.aio_context = bdrv_get_aio_context(bs);
@@ -67,11 +67,11 @@ index dcd1ef7ebc..ac79b4bdb4 100644
ret = glfs_fsync_async(s->fd, gluster_finish_aiocb, &acb);
if (ret < 0) {
@@ -1327,6 +1334,7 @@ static coroutine_fn int qemu_gluster_co_pdiscard(BlockDriverState *bs,
@@ -1314,6 +1321,7 @@ static coroutine_fn int qemu_gluster_co_pdiscard(BlockDriverState *bs,
acb.ret = 0;
acb.coroutine = qemu_coroutine_self();
acb.aio_context = bdrv_get_aio_context(bs);
+ acb.is_write = true;
ret = glfs_discard_async(s->fd, offset, size, gluster_finish_aiocb, &acb);
ret = glfs_discard_async(s->fd, offset, bytes, gluster_finish_aiocb, &acb);
if (ret < 0) {

View File

@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/qemu-img.c b/qemu-img.c
index 821cbf610e..667c540a89 100644
index 78433f3746..25d427edd1 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2821,7 +2821,8 @@ static int img_info(int argc, char **argv)
@@ -3062,7 +3062,8 @@ static int img_info(int argc, char **argv)
list = collect_image_info_list(image_opts, filename, fmt, chain,
force_share);
if (!list) {

View File

@@ -1,88 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:37 +0200
Subject: [PATCH] PVE: [Up] qmp: add get_link_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
net/net.c | 27 +++++++++++++++++++++++++++
qapi/net.json | 15 +++++++++++++++
qapi/pragma.json | 1 +
3 files changed, 43 insertions(+)
diff --git a/net/net.c b/net/net.c
index 38778e831d..dabfb482f0 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1331,6 +1331,33 @@ void hmp_info_network(Monitor *mon, const QDict *qdict)
}
}
+int64_t qmp_get_link_status(const char *name, Error **errp)
+{
+ NetClientState *ncs[MAX_QUEUE_NUM];
+ NetClientState *nc;
+ int queues;
+ bool ret;
+
+ queues = qemu_find_net_clients_except(name, ncs,
+ NET_CLIENT_DRIVER__MAX,
+ MAX_QUEUE_NUM);
+
+ if (queues == 0) {
+ error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+ "Device '%s' not found", name);
+ return (int64_t) -1;
+ }
+
+ nc = ncs[0];
+ ret = ncs[0]->link_down;
+
+ if (nc->peer->info->type == NET_CLIENT_DRIVER_NIC) {
+ ret = ncs[0]->peer->link_down;
+ }
+
+ return (int64_t) ret ? 0 : 1;
+}
+
void colo_notify_filters_event(int event, Error **errp)
{
NetClientState *nc;
diff --git a/qapi/net.json b/qapi/net.json
index cebb1b52e3..f6854483b1 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -34,6 +34,21 @@
##
{ 'command': 'set_link', 'data': {'name': 'str', 'up': 'bool'} }
+##
+# @get_link_status:
+#
+# Get the current link state of the nics or nic.
+#
+# @name: name of the nic you get the state of
+#
+# Return: If link is up 1
+# If link is down 0
+# If an error occure an empty string.
+#
+# Notes: this is an Proxmox VE extension and not offical part of Qemu.
+##
+{ 'command': 'get_link_status', 'data': {'name': 'str'} , 'returns': 'int' }
+
##
# @netdev_add:
#
diff --git a/qapi/pragma.json b/qapi/pragma.json
index cffae27666..5a3e3de95f 100644
--- a/qapi/pragma.json
+++ b/qapi/pragma.json
@@ -5,6 +5,7 @@
{ 'pragma': {
# Commands allowed to return a non-dictionary:
'returns-whitelist': [
+ 'get_link_status',
'human-monitor-command',
'qom-get',
'query-migrate-cache-size',

View File

@@ -31,16 +31,17 @@ override the output file's size.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
qemu-img-cmds.hx | 4 +-
qemu-img.c | 192 +++++++++++++++++++++++++++++------------------
2 files changed, 122 insertions(+), 74 deletions(-)
qemu-img.c | 202 ++++++++++++++++++++++++++++++-----------------
2 files changed, 133 insertions(+), 73 deletions(-)
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index c9c54de1df..0f98033658 100644
index 1b1dab5b17..d1616c045a 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -51,9 +51,9 @@ SRST
@@ -58,9 +58,9 @@ SRST
ERST
DEF("dd", img_dd,
@@ -53,10 +54,10 @@ index c9c54de1df..0f98033658 100644
DEF("info", img_info,
diff --git a/qemu-img.c b/qemu-img.c
index 667c540a89..6b7d1fcb51 100644
index 25d427edd1..220e6ec577 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4444,10 +4444,12 @@ out:
@@ -4899,10 +4899,12 @@ static int img_bitmap(int argc, char **argv)
#define C_IF 04
#define C_OF 010
#define C_SKIP 020
@@ -69,7 +70,7 @@ index 667c540a89..6b7d1fcb51 100644
};
struct DdIo {
@@ -4526,6 +4528,20 @@ static int img_dd_skip(const char *arg,
@@ -4978,6 +4980,19 @@ static int img_dd_skip(const char *arg,
return 0;
}
@@ -77,10 +78,9 @@ index 667c540a89..6b7d1fcb51 100644
+ struct DdIo *in, struct DdIo *out,
+ struct DdInfo *dd)
+{
+ dd->osize = cvtnum(arg);
+ dd->osize = cvtnum("size", arg);
+
+ if (dd->osize < 0) {
+ error_report("invalid number: '%s'", arg);
+ return 1;
+ }
+
@@ -90,7 +90,7 @@ index 667c540a89..6b7d1fcb51 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
@@ -4566,6 +4582,7 @@ static int img_dd(int argc, char **argv)
@@ -5018,6 +5033,7 @@ static int img_dd(int argc, char **argv)
{ "if", img_dd_if, C_IF },
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
@@ -98,7 +98,7 @@ index 667c540a89..6b7d1fcb51 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
@@ -4644,8 +4661,13 @@ static int img_dd(int argc, char **argv)
@@ -5093,91 +5109,112 @@ static int img_dd(int argc, char **argv)
arg = NULL;
}
@@ -106,53 +106,30 @@ index 667c540a89..6b7d1fcb51 100644
- error_report("Must specify both input and output files");
+ if (!(dd.flags & C_IF) && (!fmt || strcmp(fmt, "raw") != 0)) {
+ error_report("Input format must be raw when readin from stdin");
+ ret = -1;
+ goto out;
+ }
ret = -1;
goto out;
}
-
- blk1 = img_open(image_opts, in.filename, fmt, 0, false, false,
- force_share);
-
- if (!blk1) {
+ if (!(dd.flags & C_OF) && strcmp(out_fmt, "raw") != 0) {
+ error_report("Output format must be raw when writing to stdout");
ret = -1;
goto out;
}
@@ -4657,85 +4679,101 @@ static int img_dd(int argc, char **argv)
goto out;
}
- blk1 = img_open(image_opts, in.filename, fmt, 0, false, false,
- force_share);
+ if (dd.flags & C_IF) {
+ blk1 = img_open(image_opts, in.filename, fmt, 0, false, false,
+ force_share);
- if (!blk1) {
- ret = -1;
- goto out;
+ if (!blk1) {
+ ret = -1;
+ goto out;
+ }
}
- drv = bdrv_find_format(out_fmt);
- if (!drv) {
- error_report("Unknown file format");
+ if (dd.flags & C_OSIZE) {
+ size = dd.osize;
+ } else if (dd.flags & C_IF) {
+ size = blk_getlength(blk1);
+ if (size < 0) {
+ error_report("Failed to get size for '%s'", in.filename);
+ ret = -1;
+ goto out;
+ }
+ } else if (dd.flags & C_COUNT) {
+ size = dd.count * in.bsz;
+ } else {
+ error_report("Output size must be known when reading from stdin");
ret = -1;
goto out;
}
- ret = -1;
- goto out;
- }
- proto_drv = bdrv_find_protocol(out.filename, true, &local_err);
+ if (dd.flags & C_IF) {
+ blk1 = img_open(image_opts, in.filename, fmt, 0, false, false,
+ force_share);
- if (!proto_drv) {
- error_report_err(local_err);
@@ -170,14 +147,50 @@ index 667c540a89..6b7d1fcb51 100644
- proto_drv->format_name);
- ret = -1;
- goto out;
+ if (!(dd.flags & C_OSIZE) && dd.flags & C_COUNT && dd.count <= INT64_MAX / in.bsz &&
+ dd.count * in.bsz < size) {
+ size = dd.count * in.bsz;
+ if (!blk1) {
+ ret = -1;
+ goto out;
+ }
}
- create_opts = qemu_opts_append(create_opts, drv->create_opts);
- create_opts = qemu_opts_append(create_opts, proto_drv->create_opts);
-
- opts = qemu_opts_create(create_opts, NULL, 0, &error_abort);
- size = blk_getlength(blk1);
- if (size < 0) {
- error_report("Failed to get size for '%s'", in.filename);
+ if (dd.flags & C_OSIZE) {
+ size = dd.osize;
+ } else if (dd.flags & C_IF) {
+ size = blk_getlength(blk1);
+ if (size < 0) {
+ error_report("Failed to get size for '%s'", in.filename);
+ ret = -1;
+ goto out;
+ }
+ } else if (dd.flags & C_COUNT) {
+ size = dd.count * in.bsz;
+ } else {
+ error_report("Output size must be known when reading from stdin");
ret = -1;
goto out;
}
- if (dd.flags & C_COUNT && dd.count <= INT64_MAX / in.bsz &&
+ if (!(dd.flags & C_OSIZE) && dd.flags & C_COUNT && dd.count <= INT64_MAX / in.bsz &&
dd.count * in.bsz < size) {
size = dd.count * in.bsz;
}
- /* Overflow means the specified offset is beyond input image's size */
- if (dd.flags & C_SKIP && (in.offset > INT64_MAX / in.bsz ||
- size < in.bsz * in.offset)) {
- qemu_opt_set_number(opts, BLOCK_OPT_SIZE, 0, &error_abort);
- } else {
- qemu_opt_set_number(opts, BLOCK_OPT_SIZE,
- size - in.bsz * in.offset, &error_abort);
- }
+ if (dd.flags & C_OF) {
+ drv = bdrv_find_format(out_fmt);
+ if (!drv) {
@@ -187,9 +200,11 @@ index 667c540a89..6b7d1fcb51 100644
+ }
+ proto_drv = bdrv_find_protocol(out.filename, true, &local_err);
- size = blk_getlength(blk1);
- if (size < 0) {
- error_report("Failed to get size for '%s'", in.filename);
- ret = bdrv_create(drv, out.filename, opts, &local_err);
- if (ret < 0) {
- error_reportf_err(local_err,
- "%s: error while creating output image: ",
- out.filename);
- ret = -1;
- goto out;
- }
@@ -213,20 +228,18 @@ index 667c540a89..6b7d1fcb51 100644
+ create_opts = qemu_opts_append(create_opts, drv->create_opts);
+ create_opts = qemu_opts_append(create_opts, proto_drv->create_opts);
- if (dd.flags & C_COUNT && dd.count <= INT64_MAX / in.bsz &&
- dd.count * in.bsz < size) {
- size = dd.count * in.bsz;
- }
- /* TODO, we can't honour --image-opts for the target,
- * since it needs to be given in a format compatible
- * with the bdrv_create() call above which does not
- * support image-opts style.
- */
- blk2 = img_open_file(out.filename, NULL, out_fmt, BDRV_O_RDWR,
- false, false, false);
+ opts = qemu_opts_create(create_opts, NULL, 0, &error_abort);
- /* Overflow means the specified offset is beyond input image's size */
- if (dd.flags & C_SKIP && (in.offset > INT64_MAX / in.bsz ||
- size < in.bsz * in.offset)) {
- qemu_opt_set_number(opts, BLOCK_OPT_SIZE, 0, &error_abort);
- } else {
- qemu_opt_set_number(opts, BLOCK_OPT_SIZE,
- size - in.bsz * in.offset, &error_abort);
- }
- if (!blk2) {
- ret = -1;
- goto out;
+ /* Overflow means the specified offset is beyond input image's size */
+ if (dd.flags & C_OSIZE) {
+ qemu_opt_set_number(opts, BLOCK_OPT_SIZE, size, &error_abort);
@@ -237,15 +250,7 @@ index 667c540a89..6b7d1fcb51 100644
+ qemu_opt_set_number(opts, BLOCK_OPT_SIZE,
+ size - in.bsz * in.offset, &error_abort);
+ }
- ret = bdrv_create(drv, out.filename, opts, &local_err);
- if (ret < 0) {
- error_reportf_err(local_err,
- "%s: error while creating output image: ",
- out.filename);
- ret = -1;
- goto out;
- }
+
+ ret = bdrv_create(drv, out.filename, opts, &local_err);
+ if (ret < 0) {
+ error_reportf_err(local_err,
@@ -254,14 +259,7 @@ index 667c540a89..6b7d1fcb51 100644
+ ret = -1;
+ goto out;
+ }
- /* TODO, we can't honour --image-opts for the target,
- * since it needs to be given in a format compatible
- * with the bdrv_create() call above which does not
- * support image-opts style.
- */
- blk2 = img_open_file(out.filename, NULL, out_fmt, BDRV_O_RDWR,
- false, false, false);
+
+ /* TODO, we can't honour --image-opts for the target,
+ * since it needs to be given in a format compatible
+ * with the bdrv_create() call above which does not
@@ -269,10 +267,7 @@ index 667c540a89..6b7d1fcb51 100644
+ */
+ blk2 = img_open_file(out.filename, NULL, out_fmt, BDRV_O_RDWR,
+ false, false, false);
- if (!blk2) {
- ret = -1;
- goto out;
+
+ if (!blk2) {
+ ret = -1;
+ goto out;
@@ -280,41 +275,54 @@ index 667c540a89..6b7d1fcb51 100644
}
if (dd.flags & C_SKIP && (in.offset > INT64_MAX / in.bsz ||
@@ -4753,11 +4791,17 @@ static int img_dd(int argc, char **argv)
@@ -5194,20 +5231,43 @@ static int img_dd(int argc, char **argv)
in.buf = g_new(uint8_t, in.bsz);
for (out_pos = 0; in_pos < size; block_count++) {
int in_ret, out_ret;
for (out_pos = 0; in_pos < size; ) {
+ int in_ret, out_ret;
int bytes = (in_pos + in.bsz > size) ? size - in_pos : in.bsz;
-
- if (in_pos + in.bsz > size) {
- in_ret = blk_pread(blk1, in_pos, in.buf, size - in_pos);
+ size_t in_bsz = in_pos + in.bsz > size ? size - in_pos : in.bsz;
- ret = blk_pread(blk1, in_pos, bytes, in.buf, 0);
- if (ret < 0) {
+ if (blk1) {
+ in_ret = blk_pread(blk1, in_pos, in.buf, in_bsz);
} else {
- in_ret = blk_pread(blk1, in_pos, in.buf, in.bsz);
+ in_ret = read(STDIN_FILENO, in.buf, in_bsz);
+ in_ret = blk_pread(blk1, in_pos, bytes, in.buf, 0);
+ if (in_ret == 0) {
+ in_ret = bytes;
+ }
+ } else {
+ in_ret = read(STDIN_FILENO, in.buf, bytes);
+ if (in_ret == 0) {
+ /* early EOF is considered an error */
+ error_report("Input ended unexpectedly");
+ ret = -1;
+ goto out;
+ }
}
if (in_ret < 0) {
+ }
+ if (in_ret < 0) {
error_report("error while reading from input image file: %s",
@@ -4767,9 +4811,13 @@ static int img_dd(int argc, char **argv)
- strerror(-ret));
+ strerror(-in_ret));
+ ret = -1;
goto out;
}
in_pos += in_ret;
in_pos += bytes;
- out_ret = blk_pwrite(blk2, out_pos, in.buf, in_ret, 0);
- ret = blk_pwrite(blk2, out_pos, bytes, in.buf, 0);
- if (ret < 0) {
+ if (blk2) {
+ out_ret = blk_pwrite(blk2, out_pos, in.buf, in_ret, 0);
+ out_ret = blk_pwrite(blk2, out_pos, in_ret, in.buf, 0);
+ if (out_ret == 0) {
+ out_ret = in_ret;
+ }
+ } else {
+ out_ret = write(STDOUT_FILENO, in.buf, in_ret);
+ }
- if (out_ret < 0) {
+
+ if (out_ret != in_ret) {
error_report("error while writing to output image file: %s",
strerror(-out_ret));
ret = -1;
- strerror(-ret));
+ strerror(-out_ret));
+ ret = -1;
goto out;
}
out_pos += bytes;

View File

@@ -10,15 +10,16 @@ an expected end of input.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
qemu-img.c | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
qemu-img.c | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index 6b7d1fcb51..17393b2f53 100644
index 220e6ec577..58bf9b43d1 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4445,11 +4445,13 @@ out:
@@ -4900,11 +4900,13 @@ static int img_bitmap(int argc, char **argv)
#define C_OF 010
#define C_SKIP 020
#define C_OSIZE 040
@@ -32,7 +33,7 @@ index 6b7d1fcb51..17393b2f53 100644
};
struct DdIo {
@@ -4542,6 +4544,20 @@ static int img_dd_osize(const char *arg,
@@ -4993,6 +4995,19 @@ static int img_dd_osize(const char *arg,
return 0;
}
@@ -40,10 +41,9 @@ index 6b7d1fcb51..17393b2f53 100644
+ struct DdIo *in, struct DdIo *out,
+ struct DdInfo *dd)
+{
+ dd->isize = cvtnum(arg);
+ dd->isize = cvtnum("size", arg);
+
+ if (dd->isize < 0) {
+ error_report("invalid number: '%s'", arg);
+ return 1;
+ }
+
@@ -53,13 +53,13 @@ index 6b7d1fcb51..17393b2f53 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
@@ -4556,12 +4572,14 @@ static int img_dd(int argc, char **argv)
@@ -5007,12 +5022,14 @@ static int img_dd(int argc, char **argv)
int c, i;
const char *out_fmt = "raw";
const char *fmt = NULL;
- int64_t size = 0;
+ int64_t size = 0, readsize = 0;
int64_t block_count = 0, out_pos, in_pos;
int64_t out_pos, in_pos;
bool force_share = false;
struct DdInfo dd = {
.flags = 0,
@@ -69,7 +69,7 @@ index 6b7d1fcb51..17393b2f53 100644
};
struct DdIo in = {
.bsz = 512, /* Block size is by default 512 bytes */
@@ -4583,6 +4601,7 @@ static int img_dd(int argc, char **argv)
@@ -5034,6 +5051,7 @@ static int img_dd(int argc, char **argv)
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
{ "osize", img_dd_osize, C_OSIZE },
@@ -77,20 +77,22 @@ index 6b7d1fcb51..17393b2f53 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
@@ -4789,14 +4808,18 @@ static int img_dd(int argc, char **argv)
@@ -5230,9 +5248,10 @@ static int img_dd(int argc, char **argv)
in.buf = g_new(uint8_t, in.bsz);
- for (out_pos = 0; in_pos < size; block_count++) {
- for (out_pos = 0; in_pos < size; ) {
+ readsize = (dd.isize > 0) ? dd.isize : size;
+ for (out_pos = 0; in_pos < readsize; block_count++) {
+ for (out_pos = 0; in_pos < readsize; ) {
int in_ret, out_ret;
- size_t in_bsz = in_pos + in.bsz > size ? size - in_pos : in.bsz;
+ size_t in_bsz = in_pos + in.bsz > readsize ? readsize - in_pos : in.bsz;
- int bytes = (in_pos + in.bsz > size) ? size - in_pos : in.bsz;
+ int bytes = (in_pos + in.bsz > readsize) ? readsize - in_pos : in.bsz;
if (blk1) {
in_ret = blk_pread(blk1, in_pos, in.buf, in_bsz);
in_ret = blk_pread(blk1, in_pos, bytes, in.buf, 0);
if (in_ret == 0) {
@@ -5241,6 +5260,9 @@ static int img_dd(int argc, char **argv)
} else {
in_ret = read(STDIN_FILENO, in.buf, in_bsz);
in_ret = read(STDIN_FILENO, in.buf, bytes);
if (in_ret == 0) {
+ if (dd.isize == 0) {
+ goto out;

View File

@@ -0,0 +1,121 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Mon, 6 Apr 2020 12:16:42 +0200
Subject: [PATCH] PVE: [Up] qemu-img dd: add -n skip_create
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: fix getopt-string + add documentation]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
docs/tools/qemu-img.rst | 11 ++++++++++-
qemu-img-cmds.hx | 4 ++--
qemu-img.c | 23 ++++++++++++++---------
3 files changed, 26 insertions(+), 12 deletions(-)
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 15aeddc6d8..5e713e231d 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -208,6 +208,10 @@ Parameters to convert subcommand:
Parameters to dd subcommand:
+.. option:: -n
+
+ Skip the creation of the target volume
+
.. program:: qemu-img-dd
.. option:: bs=BLOCK_SIZE
@@ -488,7 +492,7 @@ Command description:
it doesn't need to be specified separately in this case.
-.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] if=INPUT of=OUTPUT
+.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] if=INPUT of=OUTPUT
dd copies from *INPUT* file to *OUTPUT* file converting it from
*FMT* format to *OUTPUT_FMT* format.
@@ -499,6 +503,11 @@ Command description:
The size syntax is similar to :manpage:`dd(1)`'s size syntax.
+ If the ``-n`` option is specified, the target volume creation will be
+ skipped. This is useful for formats such as ``rbd`` if the target
+ volume has already been created with site specific options that cannot
+ be supplied through ``qemu-img``.
+
.. option:: info [--object OBJECTDEF] [--image-opts] [-f FMT] [--output=OFMT] [--backing-chain] [-U] FILENAME
Give information about the disk image *FILENAME*. Use it in
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index d1616c045a..b5b0bb4467 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -58,9 +58,9 @@ SRST
ERST
DEF("dd", img_dd,
- "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [bs=block_size] [count=blocks] [skip=blocks] [osize=output_size] if=input of=output")
+ "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [-n] [bs=block_size] [count=blocks] [skip=blocks] [osize=output_size] if=input of=output")
SRST
-.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] [osize=OUTPUT_SIZE] if=INPUT of=OUTPUT
+.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] [osize=OUTPUT_SIZE] if=INPUT of=OUTPUT
ERST
DEF("info", img_info,
diff --git a/qemu-img.c b/qemu-img.c
index 58bf9b43d1..9d414d639b 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -5024,7 +5024,7 @@ static int img_dd(int argc, char **argv)
const char *fmt = NULL;
int64_t size = 0, readsize = 0;
int64_t out_pos, in_pos;
- bool force_share = false;
+ bool force_share = false, skip_create = false;
struct DdInfo dd = {
.flags = 0,
.count = 0,
@@ -5062,7 +5062,7 @@ static int img_dd(int argc, char **argv)
{ 0, 0, 0, 0 }
};
- while ((c = getopt_long(argc, argv, ":hf:O:U", long_options, NULL))) {
+ while ((c = getopt_long(argc, argv, ":hf:O:Un", long_options, NULL))) {
if (c == EOF) {
break;
}
@@ -5082,6 +5082,9 @@ static int img_dd(int argc, char **argv)
case 'h':
help();
break;
+ case 'n':
+ skip_create = true;
+ break;
case 'U':
force_share = true;
break;
@@ -5212,13 +5215,15 @@ static int img_dd(int argc, char **argv)
size - in.bsz * in.offset, &error_abort);
}
- ret = bdrv_create(drv, out.filename, opts, &local_err);
- if (ret < 0) {
- error_reportf_err(local_err,
- "%s: error while creating output image: ",
- out.filename);
- ret = -1;
- goto out;
+ if (!skip_create) {
+ ret = bdrv_create(drv, out.filename, opts, &local_err);
+ if (ret < 0) {
+ error_reportf_err(local_err,
+ "%s: error while creating output image: ",
+ out.filename);
+ ret = -1;
+ goto out;
+ }
}
/* TODO, we can't honour --image-opts for the target,

View File

@@ -0,0 +1,130 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Mon, 7 Feb 2022 14:21:01 +0100
Subject: [PATCH] qemu-img dd: add -l option for loading a snapshot
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
docs/tools/qemu-img.rst | 6 +++---
qemu-img-cmds.hx | 4 ++--
qemu-img.c | 33 +++++++++++++++++++++++++++++++--
3 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 5e713e231d..9390d5e5cf 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -492,10 +492,10 @@ Command description:
it doesn't need to be specified separately in this case.
-.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] if=INPUT of=OUTPUT
+.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [-l SNAPSHOT_PARAM] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] if=INPUT of=OUTPUT
- dd copies from *INPUT* file to *OUTPUT* file converting it from
- *FMT* format to *OUTPUT_FMT* format.
+ dd copies from *INPUT* file or snapshot *SNAPSHOT_PARAM* to *OUTPUT* file
+ converting it from *FMT* format to *OUTPUT_FMT* format.
The data is by default read and written using blocks of 512 bytes but can be
modified by specifying *BLOCK_SIZE*. If count=\ *BLOCKS* is specified
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index b5b0bb4467..36f97e1f19 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -58,9 +58,9 @@ SRST
ERST
DEF("dd", img_dd,
- "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [-n] [bs=block_size] [count=blocks] [skip=blocks] [osize=output_size] if=input of=output")
+ "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [-n] [-l snapshot_param] [bs=block_size] [count=blocks] [skip=blocks] [osize=output_size] if=input of=output")
SRST
-.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] [osize=OUTPUT_SIZE] if=INPUT of=OUTPUT
+.. option:: dd [--image-opts] [-U] [-f FMT] [-O OUTPUT_FMT] [-n] [-l SNAPSHOT_PARAM] [bs=BLOCK_SIZE] [count=BLOCKS] [skip=BLOCKS] [osize=OUTPUT_SIZE] if=INPUT of=OUTPUT
ERST
DEF("info", img_info,
diff --git a/qemu-img.c b/qemu-img.c
index 9d414d639b..e13a12137b 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -5016,6 +5016,7 @@ static int img_dd(int argc, char **argv)
BlockDriver *drv = NULL, *proto_drv = NULL;
BlockBackend *blk1 = NULL, *blk2 = NULL;
QemuOpts *opts = NULL;
+ QemuOpts *sn_opts = NULL;
QemuOptsList *create_opts = NULL;
Error *local_err = NULL;
bool image_opts = false;
@@ -5025,6 +5026,7 @@ static int img_dd(int argc, char **argv)
int64_t size = 0, readsize = 0;
int64_t out_pos, in_pos;
bool force_share = false, skip_create = false;
+ const char *snapshot_name = NULL;
struct DdInfo dd = {
.flags = 0,
.count = 0,
@@ -5062,7 +5064,7 @@ static int img_dd(int argc, char **argv)
{ 0, 0, 0, 0 }
};
- while ((c = getopt_long(argc, argv, ":hf:O:Un", long_options, NULL))) {
+ while ((c = getopt_long(argc, argv, ":hf:O:l:Un", long_options, NULL))) {
if (c == EOF) {
break;
}
@@ -5085,6 +5087,19 @@ static int img_dd(int argc, char **argv)
case 'n':
skip_create = true;
break;
+ case 'l':
+ if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
+ sn_opts = qemu_opts_parse_noisily(&internal_snapshot_opts,
+ optarg, false);
+ if (!sn_opts) {
+ error_report("Failed in parsing snapshot param '%s'",
+ optarg);
+ goto out;
+ }
+ } else {
+ snapshot_name = optarg;
+ }
+ break;
case 'U':
force_share = true;
break;
@@ -5144,11 +5159,24 @@ static int img_dd(int argc, char **argv)
if (dd.flags & C_IF) {
blk1 = img_open(image_opts, in.filename, fmt, 0, false, false,
force_share);
-
if (!blk1) {
ret = -1;
goto out;
}
+ if (sn_opts) {
+ bdrv_snapshot_load_tmp(blk_bs(blk1),
+ qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
+ qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
+ &local_err);
+ } else if (snapshot_name != NULL) {
+ bdrv_snapshot_load_tmp_by_id_or_name(blk_bs(blk1), snapshot_name,
+ &local_err);
+ }
+ if (local_err) {
+ error_reportf_err(local_err, "Failed to load snapshot: ");
+ ret = -1;
+ goto out;
+ }
}
if (dd.flags & C_OSIZE) {
@@ -5303,6 +5331,7 @@ static int img_dd(int argc, char **argv)
out:
g_free(arg);
qemu_opts_del(opts);
+ qemu_opts_del(sn_opts);
qemu_opts_free(create_opts);
blk_unref(blk1);
blk_unref(blk2);

View File

@@ -1,65 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Mon, 6 Apr 2020 12:16:42 +0200
Subject: [PATCH] PVE: [Up] qemu-img dd : add -n skip_create
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
qemu-img.c | 23 ++++++++++++++---------
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index 17393b2f53..574bb3c73d 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4574,7 +4574,7 @@ static int img_dd(int argc, char **argv)
const char *fmt = NULL;
int64_t size = 0, readsize = 0;
int64_t block_count = 0, out_pos, in_pos;
- bool force_share = false;
+ bool force_share = false, skip_create = false;
struct DdInfo dd = {
.flags = 0,
.count = 0,
@@ -4612,7 +4612,7 @@ static int img_dd(int argc, char **argv)
{ 0, 0, 0, 0 }
};
- while ((c = getopt_long(argc, argv, ":hf:O:U", long_options, NULL))) {
+ while ((c = getopt_long(argc, argv, ":hf:O:U:n", long_options, NULL))) {
if (c == EOF) {
break;
}
@@ -4632,6 +4632,9 @@ static int img_dd(int argc, char **argv)
case 'h':
help();
break;
+ case 'n':
+ skip_create = true;
+ break;
case 'U':
force_share = true;
break;
@@ -4772,13 +4775,15 @@ static int img_dd(int argc, char **argv)
size - in.bsz * in.offset, &error_abort);
}
- ret = bdrv_create(drv, out.filename, opts, &local_err);
- if (ret < 0) {
- error_reportf_err(local_err,
- "%s: error while creating output image: ",
- out.filename);
- ret = -1;
- goto out;
+ if (!skip_create) {
+ ret = bdrv_create(drv, out.filename, opts, &local_err);
+ if (ret < 0) {
+ error_reportf_err(local_err,
+ "%s: error while creating output image: ",
+ out.filename);
+ ret = -1;
+ goto out;
+ }
}
/* TODO, we can't honour --image-opts for the target,

View File

@@ -7,17 +7,62 @@ Actually provide memory information via the query-balloon
command.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: add BalloonInfo to member name exceptions list
rebase for 8.0 - moved to hw/core/machine-hmp-cmds.c]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/core/machine-hmp-cmds.c | 30 +++++++++++++++++++++++++++++-
hw/virtio/virtio-balloon.c | 33 +++++++++++++++++++++++++++++++--
monitor/hmp-cmds.c | 30 +++++++++++++++++++++++++++++-
qapi/misc.json | 22 +++++++++++++++++++++-
3 files changed, 81 insertions(+), 4 deletions(-)
qapi/machine.json | 22 +++++++++++++++++++++-
qapi/pragma.json | 1 +
4 files changed, 82 insertions(+), 4 deletions(-)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..0e32e6201f 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -169,7 +169,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
return;
}
- monitor_printf(mon, "balloon: actual=%" PRId64 "\n", info->actual >> 20);
+ monitor_printf(mon, "balloon: actual=%" PRId64, info->actual >> 20);
+ monitor_printf(mon, " max_mem=%" PRId64, info->max_mem >> 20);
+ if (info->has_total_mem) {
+ monitor_printf(mon, " total_mem=%" PRId64, info->total_mem >> 20);
+ }
+ if (info->has_free_mem) {
+ monitor_printf(mon, " free_mem=%" PRId64, info->free_mem >> 20);
+ }
+
+ if (info->has_mem_swapped_in) {
+ monitor_printf(mon, " mem_swapped_in=%" PRId64, info->mem_swapped_in);
+ }
+ if (info->has_mem_swapped_out) {
+ monitor_printf(mon, " mem_swapped_out=%" PRId64, info->mem_swapped_out);
+ }
+ if (info->has_major_page_faults) {
+ monitor_printf(mon, " major_page_faults=%" PRId64,
+ info->major_page_faults);
+ }
+ if (info->has_minor_page_faults) {
+ monitor_printf(mon, " minor_page_faults=%" PRId64,
+ info->minor_page_faults);
+ }
+ if (info->has_last_update) {
+ monitor_printf(mon, " last_update=%" PRId64,
+ info->last_update);
+ }
+
+ monitor_printf(mon, "\n");
qapi_free_BalloonInfo(info);
}
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index a4729f7fc9..97c1c16ccf 100644
index d004cf29d2..2660ed520b 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -713,8 +713,37 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
@@ -782,8 +782,37 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
static void virtio_balloon_stat(void *opaque, BalloonInfo *info)
{
VirtIOBalloon *dev = opaque;
@@ -57,54 +102,13 @@ index a4729f7fc9..97c1c16ccf 100644
}
static void virtio_balloon_to_target(void *opaque, ram_addr_t target)
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 9b94e67879..0c6f6ff331 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -653,7 +653,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
return;
}
- monitor_printf(mon, "balloon: actual=%" PRId64 "\n", info->actual >> 20);
+ monitor_printf(mon, "balloon: actual=%" PRId64, info->actual >> 20);
+ monitor_printf(mon, " max_mem=%" PRId64, info->max_mem >> 20);
+ if (info->has_total_mem) {
+ monitor_printf(mon, " total_mem=%" PRId64, info->total_mem >> 20);
+ }
+ if (info->has_free_mem) {
+ monitor_printf(mon, " free_mem=%" PRId64, info->free_mem >> 20);
+ }
+
+ if (info->has_mem_swapped_in) {
+ monitor_printf(mon, " mem_swapped_in=%" PRId64, info->mem_swapped_in);
+ }
+ if (info->has_mem_swapped_out) {
+ monitor_printf(mon, " mem_swapped_out=%" PRId64, info->mem_swapped_out);
+ }
+ if (info->has_major_page_faults) {
+ monitor_printf(mon, " major_page_faults=%" PRId64,
+ info->major_page_faults);
+ }
+ if (info->has_minor_page_faults) {
+ monitor_printf(mon, " minor_page_faults=%" PRId64,
+ info->minor_page_faults);
+ }
+ if (info->has_last_update) {
+ monitor_printf(mon, " last_update=%" PRId64,
+ info->last_update);
+ }
+
+ monitor_printf(mon, "\n");
qapi_free_BalloonInfo(info);
}
diff --git a/qapi/misc.json b/qapi/misc.json
index 99b90ac80b..e2a6678eae 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -225,10 +225,30 @@
#
# @actual: the number of bytes the balloon currently contains
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..5c9a4d55f4 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1063,9 +1063,29 @@
# @actual: the logical size of the VM in bytes Formula used:
# logical_vm_size = vm_ram_size - balloon_size
#
+# @last_update: time when stats got updated from guest
+#
@@ -122,8 +126,7 @@ index 99b90ac80b..e2a6678eae 100644
+#
+# @max_mem: amount of memory (in bytes) assigned to the guest
+#
# Since: 0.14.0
#
# Since: 0.14
##
-{ 'struct': 'BalloonInfo', 'data': {'actual': 'int' } }
+{ 'struct': 'BalloonInfo',
@@ -134,3 +137,15 @@ index 99b90ac80b..e2a6678eae 100644
##
# @query-balloon:
diff --git a/qapi/pragma.json b/qapi/pragma.json
index 7f810b0e97..325e684411 100644
--- a/qapi/pragma.json
+++ b/qapi/pragma.json
@@ -35,6 +35,7 @@
'member-name-exceptions': [ # visible in:
'ACPISlotType', # query-acpi-ospm-status
'AcpiTableOptions', # -acpitable
+ 'BalloonInfo', # query-balloon
'BlkdebugEvent', # blockdev-add, -blockdev
'BlkdebugSetStateOptions', # blockdev-add, -blockdev
'BlockDeviceInfo', # query-block

View File

@@ -13,13 +13,13 @@ Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index eed5aeb2f7..1953633e82 100644
index 3860a50c3b..40821e2317 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -230,6 +230,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
@@ -91,6 +91,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
info->numa_mem_supported = mc->numa_mem_supported;
info->deprecated = !!mc->deprecation_reason;
info->acpi = !!object_class_property_find(OBJECT_CLASS(mc), "acpi");
+
+ if (strcmp(mc->name, MACHINE_GET_CLASS(current_machine)->name) == 0) {
+ info->has_is_current = true;
@@ -28,26 +28,26 @@ index eed5aeb2f7..1953633e82 100644
+
if (mc->default_cpu_type) {
info->default_cpu_type = g_strdup(mc->default_cpu_type);
info->has_default_cpu_type = true;
}
diff --git a/qapi/machine.json b/qapi/machine.json
index ff7b5032e3..f6cf28f9fd 100644
index 5c9a4d55f4..fbb61f18e4 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -340,6 +340,8 @@
@@ -139,6 +139,8 @@
#
# @is-default: whether the machine is default
#
+# @is-current: whether this machine is currently used
+#
# @cpu-max: maximum number of CPUs supported by the machine type
# (since 1.5.0)
# (since 1.5)
#
@@ -359,7 +361,7 @@
@@ -163,7 +165,7 @@
##
{ 'struct': 'MachineInfo',
'data': { 'name': 'str', '*alias': 'str',
- '*is-default': 'bool', 'cpu-max': 'int',
+ '*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
'deprecated': 'bool', '*default-cpu-type': 'str' } }
'deprecated': 'bool', '*default-cpu-type': 'str',
'*default-ram-id': 'str', 'acpi': 'bool' } }

View File

@@ -6,40 +6,41 @@ Subject: [PATCH] PVE: qapi: modify spice query
Provide the last ticket in the SpiceInfo struct optionally.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to QAPI change]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
qapi/ui.json | 3 +++
ui/spice-core.c | 5 +++++
2 files changed, 8 insertions(+)
ui/spice-core.c | 4 ++++
2 files changed, 7 insertions(+)
diff --git a/qapi/ui.json b/qapi/ui.json
index e16e98a060..feda6ef090 100644
index 006616aa77..dfd1d3e36b 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -213,11 +213,14 @@
@@ -317,11 +317,14 @@
#
# @channels: a list of @SpiceChannel for each active spice channel
#
+# @ticket: The last ticket set with set_password
+#
# Since: 0.14.0
# Since: 0.14
##
{ 'struct': 'SpiceInfo',
'data': {'enabled': 'bool', 'migrated': 'bool', '*host': 'str', '*port': 'int',
'*tls-port': 'int', '*auth': 'str', '*compiled-version': 'str',
+ '*ticket': 'str',
'mouse-mode': 'SpiceQueryMouseMode', '*channels': ['SpiceChannel']},
'if': 'defined(CONFIG_SPICE)' }
'if': 'CONFIG_SPICE' }
diff --git a/ui/spice-core.c b/ui/spice-core.c
index ca04965ead..243466c13d 100644
index b20c25aee0..26baeb7846 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -539,6 +539,11 @@ SpiceInfo *qmp_query_spice(Error **errp)
@@ -548,6 +548,10 @@ static SpiceInfo *qmp_query_spice_real(Error **errp)
micro = SPICE_SERVER_VERSION & 0xff;
info->compiled_version = g_strdup_printf("%d.%d.%d", major, minor, micro);
+ if (auth_passwd) {
+ info->has_ticket = true;
+ info->ticket = g_strdup(auth_passwd);
+ }
+

View File

@@ -0,0 +1,282 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Thu, 13 Oct 2022 11:33:50 +0200
Subject: [PATCH] PVE: add IOChannel implementation for savevm-async
based on migration/channel-block.c and the implementation that was
present in migration/savevm-async.c before QEMU 7.1.
Passes along read/write requests to the given BlockBackend, while
ensuring that a read request going beyond the end results in a
graceful short read.
Additionally, allows tracking the current position from the outside
(intended to be used for progress tracking).
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
migration/channel-savevm-async.c | 183 +++++++++++++++++++++++++++++++
migration/channel-savevm-async.h | 51 +++++++++
migration/meson.build | 1 +
3 files changed, 235 insertions(+)
create mode 100644 migration/channel-savevm-async.c
create mode 100644 migration/channel-savevm-async.h
diff --git a/migration/channel-savevm-async.c b/migration/channel-savevm-async.c
new file mode 100644
index 0000000000..aab081ce07
--- /dev/null
+++ b/migration/channel-savevm-async.c
@@ -0,0 +1,183 @@
+/*
+ * QIO Channel implementation to be used by savevm-async QMP calls
+ */
+#include "qemu/osdep.h"
+#include "migration/channel-savevm-async.h"
+#include "qapi/error.h"
+#include "sysemu/block-backend.h"
+#include "trace.h"
+
+QIOChannelSavevmAsync *
+qio_channel_savevm_async_new(BlockBackend *be, size_t *bs_pos)
+{
+ QIOChannelSavevmAsync *ioc;
+
+ ioc = QIO_CHANNEL_SAVEVM_ASYNC(object_new(TYPE_QIO_CHANNEL_SAVEVM_ASYNC));
+
+ bdrv_ref(blk_bs(be));
+ ioc->be = be;
+ ioc->bs_pos = bs_pos;
+
+ return ioc;
+}
+
+
+static void
+qio_channel_savevm_async_finalize(Object *obj)
+{
+ QIOChannelSavevmAsync *ioc = QIO_CHANNEL_SAVEVM_ASYNC(obj);
+
+ if (ioc->be) {
+ bdrv_unref(blk_bs(ioc->be));
+ ioc->be = NULL;
+ }
+ ioc->bs_pos = NULL;
+}
+
+
+static ssize_t
+qio_channel_savevm_async_readv(QIOChannel *ioc,
+ const struct iovec *iov,
+ size_t niov,
+ int **fds,
+ size_t *nfds,
+ int flags,
+ Error **errp)
+{
+ QIOChannelSavevmAsync *saioc = QIO_CHANNEL_SAVEVM_ASYNC(ioc);
+ BlockBackend *be = saioc->be;
+ int64_t maxlen = blk_getlength(be);
+ QEMUIOVector qiov;
+ size_t size;
+ int ret;
+
+ qemu_iovec_init_external(&qiov, (struct iovec *)iov, niov);
+
+ if (*saioc->bs_pos >= maxlen) {
+ error_setg(errp, "cannot read beyond maxlen");
+ return -1;
+ }
+
+ if (maxlen - *saioc->bs_pos < qiov.size) {
+ size = maxlen - *saioc->bs_pos;
+ } else {
+ size = qiov.size;
+ }
+
+ // returns 0 on success
+ ret = blk_preadv(be, *saioc->bs_pos, size, &qiov, 0);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "blk_preadv failed");
+ return -1;
+ }
+
+ *saioc->bs_pos += size;
+ return size;
+}
+
+
+static ssize_t
+qio_channel_savevm_async_writev(QIOChannel *ioc,
+ const struct iovec *iov,
+ size_t niov,
+ int *fds,
+ size_t nfds,
+ int flags,
+ Error **errp)
+{
+ QIOChannelSavevmAsync *saioc = QIO_CHANNEL_SAVEVM_ASYNC(ioc);
+ BlockBackend *be = saioc->be;
+ QEMUIOVector qiov;
+ int ret;
+
+ qemu_iovec_init_external(&qiov, (struct iovec *)iov, niov);
+
+ if (qemu_in_coroutine()) {
+ ret = blk_co_pwritev(be, *saioc->bs_pos, qiov.size, &qiov, 0);
+ aio_wait_kick();
+ } else {
+ ret = blk_pwritev(be, *saioc->bs_pos, qiov.size, &qiov, 0);
+ }
+
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "blk(_co)_pwritev failed");
+ return -1;
+ }
+
+ *saioc->bs_pos += qiov.size;
+ return qiov.size;
+}
+
+
+static int
+qio_channel_savevm_async_set_blocking(QIOChannel *ioc,
+ bool enabled,
+ Error **errp)
+{
+ if (!enabled) {
+ error_setg(errp, "Non-blocking mode not supported for savevm-async");
+ return -1;
+ }
+ return 0;
+}
+
+
+static int
+qio_channel_savevm_async_close(QIOChannel *ioc,
+ Error **errp)
+{
+ QIOChannelSavevmAsync *saioc = QIO_CHANNEL_SAVEVM_ASYNC(ioc);
+ int rv = bdrv_flush(blk_bs(saioc->be));
+
+ if (rv < 0) {
+ error_setg_errno(errp, -rv, "Unable to flush VMState");
+ return -1;
+ }
+
+ bdrv_unref(blk_bs(saioc->be));
+ saioc->be = NULL;
+ saioc->bs_pos = NULL;
+
+ return 0;
+}
+
+
+static void
+qio_channel_savevm_async_set_aio_fd_handler(QIOChannel *ioc,
+ AioContext *ctx,
+ IOHandler *io_read,
+ IOHandler *io_write,
+ void *opaque)
+{
+ // if channel-block starts doing something, check if this needs adaptation
+}
+
+
+static void
+qio_channel_savevm_async_class_init(ObjectClass *klass,
+ void *class_data G_GNUC_UNUSED)
+{
+ QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+ ioc_klass->io_writev = qio_channel_savevm_async_writev;
+ ioc_klass->io_readv = qio_channel_savevm_async_readv;
+ ioc_klass->io_set_blocking = qio_channel_savevm_async_set_blocking;
+ ioc_klass->io_close = qio_channel_savevm_async_close;
+ ioc_klass->io_set_aio_fd_handler = qio_channel_savevm_async_set_aio_fd_handler;
+}
+
+static const TypeInfo qio_channel_savevm_async_info = {
+ .parent = TYPE_QIO_CHANNEL,
+ .name = TYPE_QIO_CHANNEL_SAVEVM_ASYNC,
+ .instance_size = sizeof(QIOChannelSavevmAsync),
+ .instance_finalize = qio_channel_savevm_async_finalize,
+ .class_init = qio_channel_savevm_async_class_init,
+};
+
+static void
+qio_channel_savevm_async_register_types(void)
+{
+ type_register_static(&qio_channel_savevm_async_info);
+}
+
+type_init(qio_channel_savevm_async_register_types);
diff --git a/migration/channel-savevm-async.h b/migration/channel-savevm-async.h
new file mode 100644
index 0000000000..17ae2cb261
--- /dev/null
+++ b/migration/channel-savevm-async.h
@@ -0,0 +1,51 @@
+/*
+ * QEMU I/O channels driver for savevm-async.c
+ *
+ * Copyright (c) 2022 Proxmox Server Solutions
+ *
+ * Authors:
+ * Fiona Ebner (f.ebner@proxmox.com)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef QIO_CHANNEL_SAVEVM_ASYNC_H
+#define QIO_CHANNEL_SAVEVM_ASYNC_H
+
+#include "io/channel.h"
+#include "qom/object.h"
+
+#define TYPE_QIO_CHANNEL_SAVEVM_ASYNC "qio-channel-savevm-async"
+OBJECT_DECLARE_SIMPLE_TYPE(QIOChannelSavevmAsync, QIO_CHANNEL_SAVEVM_ASYNC)
+
+
+/**
+ * QIOChannelSavevmAsync:
+ *
+ * The QIOChannelBlock object provides a channel implementation that is able to
+ * perform I/O on any BlockBackend whose BlockDriverState directly contains a
+ * VMState (as opposed to indirectly, like qcow2). It allows tracking the
+ * current position from the outside.
+ */
+struct QIOChannelSavevmAsync {
+ QIOChannel parent;
+ BlockBackend *be;
+ size_t *bs_pos;
+};
+
+
+/**
+ * qio_channel_savevm_async_new:
+ * @be: the block backend
+ * @bs_pos: used to keep track of the IOChannels current position
+ *
+ * Create a new IO channel object that can perform I/O on a BlockBackend object
+ * whose BlockDriverState directly contains a VMState.
+ *
+ * Returns: the new channel object
+ */
+QIOChannelSavevmAsync *
+qio_channel_savevm_async_new(BlockBackend *be, size_t *bs_pos);
+
+#endif /* QIO_CHANNEL_SAVEVM_ASYNC_H */
diff --git a/migration/meson.build b/migration/meson.build
index 1ae28523a1..37ddcb5d60 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -13,6 +13,7 @@ system_ss.add(files(
'block-dirty-bitmap.c',
'channel.c',
'channel-block.c',
+ 'channel-savevm-async.c',
'dirtyrate.c',
'exec.c',
'fd.c',

View File

@@ -0,0 +1,216 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 4 May 2020 11:05:08 +0200
Subject: [PATCH] PVE: add optional buffer size to QEMUFile
So we can use a 4M buffer for savevm-async which should
increase performance storing the state onto ceph.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
[increase max IOV count in QEMUFile to actually write more data]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to removal of QEMUFileOps]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
migration/qemu-file.c | 49 +++++++++++++++++++++++++++-------------
migration/qemu-file.h | 2 ++
migration/savevm-async.c | 5 ++--
3 files changed, 38 insertions(+), 18 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 19c33c9985..e9ffff0f0a 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -33,8 +33,8 @@
#include "options.h"
#include "qapi/error.h"
-#define IO_BUF_SIZE 32768
-#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
+#define DEFAULT_IO_BUF_SIZE 32768
+#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 256)
struct QEMUFile {
const QEMUFileHooks *hooks;
@@ -46,7 +46,8 @@ struct QEMUFile {
int buf_index;
int buf_size; /* 0 when writing */
- uint8_t buf[IO_BUF_SIZE];
+ size_t buf_allocated_size;
+ uint8_t *buf;
DECLARE_BITMAP(may_free, MAX_IOV_SIZE);
struct iovec iov[MAX_IOV_SIZE];
@@ -100,7 +101,9 @@ int qemu_file_shutdown(QEMUFile *f)
return 0;
}
-static QEMUFile *qemu_file_new_impl(QIOChannel *ioc, bool is_writable)
+static QEMUFile *qemu_file_new_impl(QIOChannel *ioc,
+ bool is_writable,
+ size_t buffer_size)
{
QEMUFile *f;
@@ -109,6 +112,8 @@ static QEMUFile *qemu_file_new_impl(QIOChannel *ioc, bool is_writable)
object_ref(ioc);
f->ioc = ioc;
f->is_writable = is_writable;
+ f->buf_allocated_size = buffer_size;
+ f->buf = malloc(buffer_size);
return f;
}
@@ -119,17 +124,27 @@ static QEMUFile *qemu_file_new_impl(QIOChannel *ioc, bool is_writable)
*/
QEMUFile *qemu_file_get_return_path(QEMUFile *f)
{
- return qemu_file_new_impl(f->ioc, !f->is_writable);
+ return qemu_file_new_impl(f->ioc, !f->is_writable, DEFAULT_IO_BUF_SIZE);
}
QEMUFile *qemu_file_new_output(QIOChannel *ioc)
{
- return qemu_file_new_impl(ioc, true);
+ return qemu_file_new_impl(ioc, true, DEFAULT_IO_BUF_SIZE);
+}
+
+QEMUFile *qemu_file_new_output_sized(QIOChannel *ioc, size_t buffer_size)
+{
+ return qemu_file_new_impl(ioc, true, buffer_size);
}
QEMUFile *qemu_file_new_input(QIOChannel *ioc)
{
- return qemu_file_new_impl(ioc, false);
+ return qemu_file_new_impl(ioc, false, DEFAULT_IO_BUF_SIZE);
+}
+
+QEMUFile *qemu_file_new_input_sized(QIOChannel *ioc, size_t buffer_size)
+{
+ return qemu_file_new_impl(ioc, false, buffer_size);
}
void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks)
@@ -375,7 +390,7 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f)
do {
len = qio_channel_read(f->ioc,
(char *)f->buf + pending,
- IO_BUF_SIZE - pending,
+ f->buf_allocated_size - pending,
&local_error);
if (len == QIO_CHANNEL_ERR_BLOCK) {
if (qemu_in_coroutine()) {
@@ -425,6 +440,8 @@ int qemu_fclose(QEMUFile *f)
}
g_clear_pointer(&f->ioc, object_unref);
+ free(f->buf);
+
/* If any error was spotted before closing, we should report it
* instead of the close() return value.
*/
@@ -479,7 +496,7 @@ static void add_buf_to_iovec(QEMUFile *f, size_t len)
{
if (!add_to_iovec(f, f->buf + f->buf_index, len, false)) {
f->buf_index += len;
- if (f->buf_index == IO_BUF_SIZE) {
+ if (f->buf_index == f->buf_allocated_size) {
qemu_fflush(f);
}
}
@@ -504,7 +521,7 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, size_t size)
}
while (size > 0) {
- l = IO_BUF_SIZE - f->buf_index;
+ l = f->buf_allocated_size - f->buf_index;
if (l > size) {
l = size;
}
@@ -549,8 +566,8 @@ size_t coroutine_mixed_fn qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t si
size_t index;
assert(!qemu_file_is_writable(f));
- assert(offset < IO_BUF_SIZE);
- assert(size <= IO_BUF_SIZE - offset);
+ assert(offset < f->buf_allocated_size);
+ assert(size <= f->buf_allocated_size - offset);
/* The 1st byte to read from */
index = f->buf_index + offset;
@@ -600,7 +617,7 @@ size_t coroutine_mixed_fn qemu_get_buffer(QEMUFile *f, uint8_t *buf, size_t size
size_t res;
uint8_t *src;
- res = qemu_peek_buffer(f, &src, MIN(pending, IO_BUF_SIZE), 0);
+ res = qemu_peek_buffer(f, &src, MIN(pending, f->buf_allocated_size), 0);
if (res == 0) {
return done;
}
@@ -634,7 +651,7 @@ size_t coroutine_mixed_fn qemu_get_buffer(QEMUFile *f, uint8_t *buf, size_t size
*/
size_t coroutine_mixed_fn qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size)
{
- if (size < IO_BUF_SIZE) {
+ if (size < f->buf_allocated_size) {
size_t res;
uint8_t *src = NULL;
@@ -659,7 +676,7 @@ int coroutine_mixed_fn qemu_peek_byte(QEMUFile *f, int offset)
int index = f->buf_index + offset;
assert(!qemu_file_is_writable(f));
- assert(offset < IO_BUF_SIZE);
+ assert(offset < f->buf_allocated_size);
if (index >= f->buf_size) {
qemu_fill_buffer(f);
@@ -777,7 +794,7 @@ static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len,
ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
const uint8_t *p, size_t size)
{
- ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t);
+ ssize_t blen = f->buf_allocated_size - f->buf_index - sizeof(int32_t);
if (blen < compressBound(size)) {
return -1;
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 47015f5201..1312b7c903 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -63,7 +63,9 @@ typedef struct QEMUFileHooks {
} QEMUFileHooks;
QEMUFile *qemu_file_new_input(QIOChannel *ioc);
+QEMUFile *qemu_file_new_input_sized(QIOChannel *ioc, size_t buffer_size);
QEMUFile *qemu_file_new_output(QIOChannel *ioc);
+QEMUFile *qemu_file_new_output_sized(QIOChannel *ioc, size_t buffer_size);
void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
int qemu_fclose(QEMUFile *f);
diff --git a/migration/savevm-async.c b/migration/savevm-async.c
index e9fc18fb10..80624fada8 100644
--- a/migration/savevm-async.c
+++ b/migration/savevm-async.c
@@ -378,7 +378,7 @@ void qmp_savevm_start(const char *statefile, Error **errp)
QIOChannel *ioc = QIO_CHANNEL(qio_channel_savevm_async_new(snap_state.target,
&snap_state.bs_pos));
- snap_state.file = qemu_file_new_output(ioc);
+ snap_state.file = qemu_file_new_output_sized(ioc, 4 * 1024 * 1024);
if (!snap_state.file) {
error_set(errp, ERROR_CLASS_GENERIC_ERROR, "failed to open '%s'", statefile);
@@ -496,7 +496,8 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
blk_op_block_all(be, blocker);
/* restore the VM state */
- f = qemu_file_new_input(QIO_CHANNEL(qio_channel_savevm_async_new(be, &bs_pos)));
+ f = qemu_file_new_input_sized(QIO_CHANNEL(qio_channel_savevm_async_new(be, &bs_pos)),
+ 4 * 1024 * 1024);
if (!f) {
error_setg(errp, "Could not open VM state file");
goto the_end;

View File

@@ -4,30 +4,32 @@ Date: Mon, 6 Apr 2020 12:16:47 +0200
Subject: [PATCH] PVE: block: add the zeroinit block driver filter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to changed function signatures]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/Makefile.objs | 1 +
block/zeroinit.c | 197 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 198 insertions(+)
block/meson.build | 1 +
block/zeroinit.c | 200 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 201 insertions(+)
create mode 100644 block/zeroinit.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 3635b6b4c1..1282445672 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -11,6 +11,7 @@ block-obj-$(CONFIG_QED) += qed.o qed-l2-cache.o qed-table.o qed-cluster.o
block-obj-$(CONFIG_QED) += qed-check.o
block-obj-y += vhdx.o vhdx-endian.o vhdx-log.o
block-obj-y += quorum.o
+block-obj-y += zeroinit.o
block-obj-y += blkdebug.o blkverify.o blkreplay.o
block-obj-$(CONFIG_PARALLELS) += parallels.o
block-obj-y += blklogwrites.o
diff --git a/block/meson.build b/block/meson.build
index 529fc172c6..1833c71ce9 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -40,6 +40,7 @@ block_ss.add(files(
'throttle-groups.c',
'throttle.c',
'write-threshold.c',
+ 'zeroinit.c',
), zstd, zlib, gnutls)
system_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
diff --git a/block/zeroinit.c b/block/zeroinit.c
new file mode 100644
index 0000000000..ff38388d94
index 0000000000..1257342724
--- /dev/null
+++ b/block/zeroinit.c
@@ -0,0 +1,197 @@
@@ -0,0 +1,200 @@
+/*
+ * Filter to fake a zero-initialized block device.
+ *
@@ -41,6 +43,7 @@ index 0000000000..ff38388d94
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "block/block_int.h"
+#include "block/block-io.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "qemu/cutils.h"
@@ -107,7 +110,9 @@ index 0000000000..ff38388d94
+
+ /* Open the raw file */
+ bs->file = bdrv_open_child(qemu_opt_get(opts, "x-next"), options, "next",
+ bs, &child_file, false, &local_err);
+ bs, &child_of_bds,
+ BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+ false, &local_err);
+ if (local_err) {
+ ret = -EINVAL;
+ error_propagate(errp, local_err);
@@ -132,28 +137,28 @@ index 0000000000..ff38388d94
+ (void)s;
+}
+
+static int64_t zeroinit_getlength(BlockDriverState *bs)
+static coroutine_fn int64_t zeroinit_co_getlength(BlockDriverState *bs)
+{
+ return bdrv_getlength(bs->file->bs);
+ return bdrv_co_getlength(bs->file->bs);
+}
+
+static int coroutine_fn zeroinit_co_preadv(BlockDriverState *bs,
+ uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags)
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
+}
+
+static int coroutine_fn zeroinit_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset,
+ int count, BdrvRequestFlags flags)
+ int64_t bytes, BdrvRequestFlags flags)
+{
+ BDRVZeroinitState *s = bs->opaque;
+ if (offset >= s->extents)
+ return 0;
+ return bdrv_pwrite_zeroes(bs->file, offset, count, flags);
+ return bdrv_pwrite_zeroes(bs->file, offset, bytes, flags);
+}
+
+static int coroutine_fn zeroinit_co_pwritev(BlockDriverState *bs,
+ uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags)
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ BDRVZeroinitState *s = bs->opaque;
+ int64_t extents = offset + bytes;
@@ -174,20 +179,22 @@ index 0000000000..ff38388d94
+}
+
+static int coroutine_fn zeroinit_co_pdiscard(BlockDriverState *bs,
+ int64_t offset, int count)
+ int64_t offset, int64_t bytes)
+{
+ return bdrv_co_pdiscard(bs->file, offset, count);
+ return bdrv_co_pdiscard(bs->file, offset, bytes);
+}
+
+static int zeroinit_co_truncate(BlockDriverState *bs, int64_t offset,
+ _Bool exact, PreallocMode prealloc, Error **errp)
+ _Bool exact, PreallocMode prealloc,
+ BdrvRequestFlags req_flags, Error **errp)
+{
+ return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp);
+ return bdrv_co_truncate(bs->file, offset, exact, prealloc, req_flags, errp);
+}
+
+static int zeroinit_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
+static coroutine_fn int zeroinit_co_get_info(BlockDriverState *bs,
+ BlockDriverInfo *bdi)
+{
+ return bdrv_get_info(bs->file->bs, bdi);
+ return bdrv_co_get_info(bs->file->bs, bdi);
+}
+
+static BlockDriver bdrv_zeroinit = {
@@ -198,8 +205,8 @@ index 0000000000..ff38388d94
+ .bdrv_parse_filename = zeroinit_parse_filename,
+ .bdrv_file_open = zeroinit_open,
+ .bdrv_close = zeroinit_close,
+ .bdrv_getlength = zeroinit_getlength,
+ .bdrv_child_perm = bdrv_filter_default_perms,
+ .bdrv_co_getlength = zeroinit_co_getlength,
+ .bdrv_child_perm = bdrv_default_perms,
+ .bdrv_co_flush_to_disk = zeroinit_co_flush,
+
+ .bdrv_co_pwrite_zeroes = zeroinit_co_pwrite_zeroes,
@@ -211,12 +218,10 @@ index 0000000000..ff38388d94
+
+ .bdrv_has_zero_init = zeroinit_has_zero_init,
+
+ .bdrv_co_block_status = bdrv_co_block_status_from_file,
+
+ .bdrv_co_pdiscard = zeroinit_co_pdiscard,
+
+ .bdrv_co_truncate = zeroinit_co_truncate,
+ .bdrv_get_info = zeroinit_get_info,
+ .bdrv_co_get_info = zeroinit_co_get_info,
+};
+
+static void bdrv_zeroinit_init(void)

View File

@@ -14,12 +14,12 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 11 insertions(+)
diff --git a/qemu-options.hx b/qemu-options.hx
index 55eef64ddf..e11b4f8ff5 100644
index dc1ececc9c..848d2dfdd1 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -904,6 +904,9 @@ DEFHEADING()
@@ -1197,6 +1197,9 @@ legacy PC, they are not recommended for modern configurations.
DEFHEADING(Block device options:)
ERST
+DEF("id", HAS_ARG, QEMU_OPTION_id,
+ "-id n set the VMID", QEMU_ARCH_ALL)
@@ -28,20 +28,20 @@ index 55eef64ddf..e11b4f8ff5 100644
"-fda/-fdb file use 'file' as floppy disk 0/1 image\n", QEMU_ARCH_ALL)
DEF("fdb", HAS_ARG, QEMU_OPTION_fdb, "", QEMU_ARCH_ALL)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 4b45eb0c37..9de81875fd 100644
index 3f2681aded..1a3b9cc4b8 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2815,6 +2815,7 @@ static void create_default_memdev(MachineState *ms, const char *path)
void qemu_init(int argc, char **argv, char **envp)
{
int i;
@@ -2683,6 +2683,7 @@ void qemu_init(int argc, char **argv)
MachineClass *machine_class;
bool userconfig = true;
FILE *vmstate_dump_file = NULL;
+ long vm_id;
int snapshot, linux_boot;
const char *initrd_filename;
const char *kernel_filename, *kernel_cmdline;
@@ -3518,6 +3519,13 @@ void qemu_init(int argc, char **argv, char **envp)
exit(1);
}
qemu_add_opts(&qemu_drive_opts);
qemu_add_drive_opts(&qemu_legacy_drive_opts);
@@ -3306,6 +3307,13 @@ void qemu_init(int argc, char **argv)
machine_parse_property_opt(qemu_find_opts("smp-opts"),
"smp", optarg);
break;
+ case QEMU_OPTION_id:
+ vm_id = strtol(optarg, (char **)&optarg, 10);
@@ -51,5 +51,5 @@ index 4b45eb0c37..9de81875fd 100644
+ }
+ break;
case QEMU_OPTION_vnc:
vnc_parse(optarg, &error_fatal);
vnc_parse(optarg);
break;

View File

@@ -11,10 +11,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index 9ec0f2deb2..a00d45251f 100644
index 4a34f03047..59b917e50c 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -259,6 +259,15 @@ static void apic_reset_common(DeviceState *dev)
@@ -252,6 +252,15 @@ static void apic_reset_common(DeviceState *dev)
info->vapic_base_update(s);
apic_init_reset(dev);

View File

@@ -8,15 +8,15 @@ Otherwise creating images on nfs/cifs can be problematic.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/file-posix.c | 61 +++++++++++++++++++++++++++++---------------
block/file-posix.c | 59 ++++++++++++++++++++++++++++++--------------
qapi/block-core.json | 3 ++-
2 files changed, 43 insertions(+), 21 deletions(-)
2 files changed, 42 insertions(+), 20 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index b527e82a82..36ebd0967e 100644
index ca551baa42..8b3b83e9d4 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -2309,6 +2309,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2873,6 +2873,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
int fd;
uint64_t perm, shared;
int result = 0;
@@ -24,7 +24,7 @@ index b527e82a82..36ebd0967e 100644
/* Validate options and set default values */
assert(options->driver == BLOCKDEV_DRIVER_FILE);
@@ -2342,19 +2343,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2913,19 +2914,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
perm = BLK_PERM_WRITE | BLK_PERM_RESIZE;
shared = BLK_PERM_ALL & ~BLK_PERM_RESIZE;
@@ -59,7 +59,7 @@ index b527e82a82..36ebd0967e 100644
}
/* Clear the file by truncating it to 0 */
@@ -2387,13 +2391,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
@@ -2979,13 +2983,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
}
out_unlock:
@@ -82,7 +82,7 @@ index b527e82a82..36ebd0967e 100644
}
out_close:
@@ -2416,6 +2422,7 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
@@ -3009,6 +3015,7 @@ raw_co_create_opts(BlockDriver *drv, const char *filename,
PreallocMode prealloc;
char *buf = NULL;
Error *local_err = NULL;
@@ -90,7 +90,7 @@ index b527e82a82..36ebd0967e 100644
/* Skip file: protocol prefix */
strstart(filename, "file:", &filename);
@@ -2433,6 +2440,18 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
@@ -3031,6 +3038,18 @@ raw_co_create_opts(BlockDriver *drv, const char *filename,
return -EINVAL;
}
@@ -109,34 +109,25 @@ index b527e82a82..36ebd0967e 100644
options = (BlockdevCreateOptions) {
.driver = BLOCKDEV_DRIVER_FILE,
.u.file = {
@@ -2442,6 +2461,8 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
.preallocation = prealloc,
.has_nocow = true,
@@ -3042,6 +3061,8 @@ raw_co_create_opts(BlockDriver *drv, const char *filename,
.nocow = nocow,
.has_extent_size_hint = has_extent_size_hint,
.extent_size_hint = extent_size_hint,
+ .has_locking = true,
+ .locking = locking,
},
};
return raw_co_create(&options, errp);
@@ -2983,7 +3004,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared,
}
/* Copy locks to the new fd */
- if (s->perm_change_fd) {
+ if (s->use_lock && s->perm_change_fd) {
ret = raw_apply_lock_bytes(NULL, s->perm_change_fd, perm, ~shared,
false, errp);
if (ret < 0) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 943df1926a..4c55464f86 100644
index a5cea82139..bb471c078d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4183,7 +4183,8 @@
'data': { 'filename': 'str',
'size': 'size',
'*preallocation': 'PreallocMode',
- '*nocow': 'bool' } }
+ '*nocow': 'bool',
@@ -4880,7 +4880,8 @@
'size': 'size',
'*preallocation': 'PreallocMode',
'*nocow': 'bool',
- '*extent-size-hint': 'size'} }
+ '*extent-size-hint': 'size',
+ '*locking': 'OnOffAuto' } }
##

View File

@@ -1,22 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:51 +0200
Subject: [PATCH] PVE: savevm-async: kick AIO wait on block state write
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
savevm-async.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/savevm-async.c b/savevm-async.c
index 54ceeae26c..393d55af2a 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -158,6 +158,7 @@ static void coroutine_fn block_state_write_entry(void *opaque) {
BlkRwCo *rwco = opaque;
rwco->ret = blk_co_pwritev(snap_state.target, rwco->offset, rwco->qiov->size,
rwco->qiov, 0);
+ aio_wait_kick();
}
static ssize_t block_state_writev_buffer(void *opaque, struct iovec *iov,

View File

@@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index f89e7daf27..ed5e39fcf7 100644
index 589c9524f8..2505dd658a 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -406,8 +406,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
@@ -536,8 +536,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
qemu_chr_fe_set_echo(&mon->common.chr, true);
/* Note: we run QMP monitor in I/O thread when @chr supports that */

View File

@@ -1,38 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:52 +0200
Subject: [PATCH] PVE: move snapshot cleanup into bottom half
as per:
(0ceccd858a8d) migration: qemu_savevm_state_cleanup() in cleanup
may affect held locks and therefore change assumptions made
by that function!
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
savevm-async.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/savevm-async.c b/savevm-async.c
index 393d55af2a..790e27ae37 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -201,6 +201,8 @@ static void process_savevm_cleanup(void *opaque)
int ret;
qemu_bh_delete(snap_state.cleanup_bh);
snap_state.cleanup_bh = NULL;
+ qemu_savevm_state_cleanup();
+
qemu_mutex_unlock_iothread();
qemu_thread_join(&snap_state.thread);
qemu_mutex_lock_iothread();
@@ -277,7 +279,6 @@ static void *process_savevm_thread(void *opaque)
save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
break;
}
- qemu_savevm_state_cleanup();
DPRINTF("save complete\n");
break;
}

View File

@@ -26,10 +26,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index c1a444cb75..9f56ecc4e8 100644
index f0d35c6401..1427983543 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -56,7 +56,8 @@ GlobalProperty hw_compat_4_0[] = {
@@ -148,7 +148,8 @@ GlobalProperty hw_compat_4_0[] = {
{ "virtio-vga", "edid", "false" },
{ "virtio-gpu-device", "edid", "false" },
{ "virtio-device", "use-started", "false" },

View File

@@ -0,0 +1,129 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:55 +0200
Subject: [PATCH] PVE: Allow version code in machine type
E.g. pc-i440fx-4.0+pve3 would print 'pve3' as version code while
selecting pc-i440fx-4.0 as machine type.
Version is made available as 'pve-version' in query-machines (same as,
and only if 'is-current').
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to QAPI changes]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
hw/core/machine-qmp-cmds.c | 5 +++++
include/hw/boards.h | 2 ++
qapi/machine.json | 4 +++-
softmmu/vl.c | 25 +++++++++++++++++++++++++
4 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 40821e2317..ee93ddd69a 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -95,6 +95,11 @@ MachineInfoList *qmp_query_machines(Error **errp)
if (strcmp(mc->name, MACHINE_GET_CLASS(current_machine)->name) == 0) {
info->has_is_current = true;
info->is_current = true;
+
+ // PVE version string only exists for current machine
+ if (mc->pve_version) {
+ info->pve_version = g_strdup(mc->pve_version);
+ }
}
if (mc->default_cpu_type) {
diff --git a/include/hw/boards.h b/include/hw/boards.h
index ed83360198..f8b88cd86a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -235,6 +235,8 @@ struct MachineClass {
const char *desc;
const char *deprecation_reason;
+ const char *pve_version;
+
void (*init)(MachineState *state);
void (*reset)(MachineState *state, ShutdownCause reason);
void (*wakeup)(MachineState *state);
diff --git a/qapi/machine.json b/qapi/machine.json
index fbb61f18e4..7da3c519ba 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -161,6 +161,8 @@
#
# @acpi: machine type supports ACPI (since 8.0)
#
+# @pve-version: custom PVE version suffix specified as 'machine+pveN'
+#
# Since: 1.2
##
{ 'struct': 'MachineInfo',
@@ -168,7 +170,7 @@
'*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
'deprecated': 'bool', '*default-cpu-type': 'str',
- '*default-ram-id': 'str', 'acpi': 'bool' } }
+ '*default-ram-id': 'str', 'acpi': 'bool', '*pve-version': 'str' } }
##
# @query-machines:
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 1a3b9cc4b8..e9b5f62cc3 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1597,6 +1597,7 @@ static const QEMUOption *lookup_opt(int argc, char **argv,
static MachineClass *select_machine(QDict *qdict, Error **errp)
{
const char *optarg = qdict_get_try_str(qdict, "type");
+ const char *pvever = qdict_get_try_str(qdict, "pvever");
GSList *machines = object_class_get_list(TYPE_MACHINE, false);
MachineClass *machine_class;
Error *local_err = NULL;
@@ -1614,6 +1615,11 @@ static MachineClass *select_machine(QDict *qdict, Error **errp)
}
}
+ if (machine_class) {
+ machine_class->pve_version = g_strdup(pvever);
+ qdict_del(qdict, "pvever");
+ }
+
g_slist_free(machines);
if (local_err) {
error_append_hint(&local_err, "Use -machine help to list supported machines\n");
@@ -3248,12 +3254,31 @@ void qemu_init(int argc, char **argv)
case QEMU_OPTION_machine:
{
bool help;
+ size_t pvever_index, name_len;
+ const gchar *name;
+ gchar *name_clean, *pvever;
keyval_parse_into(machine_opts_dict, optarg, "type", &help, &error_fatal);
if (help) {
machine_help_func(machine_opts_dict);
exit(EXIT_SUCCESS);
}
+
+ // PVE version is specified with '+' as seperator, e.g. pc-i440fx+pvever
+ name = qdict_get_try_str(machine_opts_dict, "type");
+ if (name != NULL) {
+ name_len = strlen(name);
+ pvever_index = strcspn(name, "+");
+ if (pvever_index < name_len) {
+ name_clean = g_strndup(name, pvever_index);
+ pvever = g_strndup(name + pvever_index + 1, name_len - pvever_index - 1);
+ qdict_put_str(machine_opts_dict, "pvever", pvever);
+ qdict_put_str(machine_opts_dict, "type", name_clean);
+ g_free(name_clean);
+ g_free(pvever);
+ }
+ }
+
break;
}
case QEMU_OPTION_accel:

View File

@@ -1,101 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:55 +0200
Subject: [PATCH] PVE: Allow version code in machine type
E.g. pc-i440fx-4.0+pve3 would print 'pve3' as version code while
selecting pc-i440fx-4.0 as machine type.
Version is made available as 'pve-version' in query-machines (same as,
and only if 'is-current').
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
hw/core/machine-qmp-cmds.c | 6 ++++++
include/hw/boards.h | 2 ++
qapi/machine.json | 3 ++-
softmmu/vl.c | 15 ++++++++++++++-
4 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 1953633e82..ca8c0dc53d 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -234,6 +234,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
if (strcmp(mc->name, MACHINE_GET_CLASS(current_machine)->name) == 0) {
info->has_is_current = true;
info->is_current = true;
+
+ // PVE version string only exists for current machine
+ if (mc->pve_version) {
+ info->has_pve_version = true;
+ info->pve_version = g_strdup(mc->pve_version);
+ }
}
if (mc->default_cpu_type) {
diff --git a/include/hw/boards.h b/include/hw/boards.h
index fd4d62b501..dd395e9232 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -170,6 +170,8 @@ struct MachineClass {
const char *desc;
const char *deprecation_reason;
+ const char *pve_version;
+
void (*init)(MachineState *state);
void (*reset)(MachineState *state);
void (*wakeup)(MachineState *state);
diff --git a/qapi/machine.json b/qapi/machine.json
index f6cf28f9fd..a7f9c79a59 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -363,7 +363,8 @@
'data': { 'name': 'str', '*alias': 'str',
'*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
- 'deprecated': 'bool', '*default-cpu-type': 'str' } }
+ 'deprecated': 'bool', '*default-cpu-type': 'str',
+ '*pve-version': 'str' } }
##
# @query-machines:
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 9de81875fd..8340c4ca53 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2300,6 +2300,8 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
{
MachineClass *mc;
GSList *el;
+ size_t pvever_index = 0;
+ gchar *name_clean;
if (is_help_option(name)) {
printf("Supported machines are:\n");
@@ -2316,12 +2318,23 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
exit(0);
}
- mc = find_machine(name, machines);
+ // PVE version is specified with '+' as seperator, e.g. pc-i440fx+pvever
+ pvever_index = strcspn(name, "+");
+
+ name_clean = g_strndup(name, pvever_index);
+ mc = find_machine(name_clean, machines);
+ g_free(name_clean);
+
if (!mc) {
error_report("unsupported machine type");
error_printf("Use -machine help to list supported machines\n");
exit(1);
}
+
+ if (pvever_index < strlen(name)) {
+ mc->pve_version = &name[pvever_index+1];
+ }
+
return mc;
}

View File

@@ -0,0 +1,59 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Wed, 2 Mar 2022 08:35:05 +0100
Subject: [PATCH] block/backup: move bcs bitmap initialization to job creation
For backing up the state of multiple disks from the same time, a job
for each disk has to be created. It's convenient if the jobs don't
have to be started at the same time and if operation of the VM can be
resumed after job creation. This would lead to a window between job
creation and running the job, where writes can happen. But no writes
should happen between setting up the copy-before-write filter and
setting up the block copy state bitmap, because then new writes would
just pass through.
Commit 06e0a9c16405c0a4c1eca33cf286cc04c42066a2 moved initalization of
the bitmap to setting up the copy-before-write filter when sync_mode
is not MIRROR_SYNC_MODE_BITMAP. Ensure that the bitmap is initialized
upon job creation for the remaining case too, by moving the
backup_init_bcs_bitmap call to backup_job_create.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/backup.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index db3791f4d1..39410dcf8d 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -237,8 +237,8 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
true);
} else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
/*
- * We can't hog the coroutine to initialize this thoroughly.
- * Set a flag and resume work when we are able to yield safely.
+ * Initialization is costly here. Simply set a flag and let the
+ * backup_run coroutine resume work once it can yield safely.
*/
block_copy_set_skip_unallocated(job->bcs, true);
}
@@ -252,8 +252,6 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
int ret;
- backup_init_bcs_bitmap(s);
-
if (s->sync_mode == MIRROR_SYNC_MODE_TOP) {
int64_t offset = 0;
int64_t count;
@@ -495,6 +493,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
&error_abort);
+ backup_init_bcs_bitmap(job);
+
return &job->common;
error:

View File

@@ -3,58 +3,70 @@ From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:57 +0200
Subject: [PATCH] PVE-Backup: add vma backup format code
Notes about partial restoring: skipping a certain drive is done via a
map line of the form skip=drive-scsi0. Since in PVE, most archives are
compressed and piped to vma for restore, it's not easily possible to
skip reads.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: improvements during create
allow partial restore]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
Makefile | 3 +-
Makefile.objs | 1 +
vma-reader.c | 857 ++++++++++++++++++++++++++++++++++++++++++++++++++
vma-writer.c | 771 +++++++++++++++++++++++++++++++++++++++++++++
vma.c | 837 ++++++++++++++++++++++++++++++++++++++++++++++++
vma.h | 150 +++++++++
6 files changed, 2618 insertions(+), 1 deletion(-)
block/meson.build | 2 +
meson.build | 5 +
vma-reader.c | 867 ++++++++++++++++++++++++++++++++++++++++++++
vma-writer.c | 818 +++++++++++++++++++++++++++++++++++++++++
vma.c | 900 ++++++++++++++++++++++++++++++++++++++++++++++
vma.h | 150 ++++++++
6 files changed, 2742 insertions(+)
create mode 100644 vma-reader.c
create mode 100644 vma-writer.c
create mode 100644 vma.c
create mode 100644 vma.h
diff --git a/Makefile b/Makefile
index 8a9113e666..74c2039005 100644
--- a/Makefile
+++ b/Makefile
@@ -479,7 +479,7 @@ dummy := $(call unnest-vars,, \
diff --git a/block/meson.build b/block/meson.build
index 1833c71ce9..59b71ba9f3 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -43,6 +43,8 @@ block_ss.add(files(
'zeroinit.c',
), zstd, zlib, gnutls)
include $(SRC_PATH)/tests/Makefile.include
+block_ss.add(files('../vma-writer.c'), libuuid)
+
system_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
system_ss.add(files('block-ram-registrar.c'))
-all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
diff --git a/meson.build b/meson.build
index a9c4f28247..cd95530d3b 100644
--- a/meson.build
+++ b/meson.build
@@ -1778,6 +1778,8 @@ endif
qemu-version.h: FORCE
$(call quiet-command, \
@@ -608,6 +608,7 @@ qemu-img$(EXESUF): qemu-img.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io
qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) $(storage-daemon-obj-y) $(COMMON_LDADDS)
+vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
has_gettid = cc.has_function('gettid')
qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index d0b4dde836..05031a3da7 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-y += block.o blockjob.o job.o
block-obj-y += block/ scsi/
block-obj-y += qemu-io-cmds.o
block-obj-$(CONFIG_REPLICATION) += replication.o
+block-obj-y += vma-writer.o
block-obj-m = block/
+libuuid = cc.find_library('uuid', required: true)
+
# libselinux
selinux = dependency('libselinux',
required: get_option('selinux'),
@@ -3908,6 +3910,9 @@ if have_tools
dependencies: [blockdev, qemuutil, gnutls, selinux],
install: true)
+ vma = executable('vma', files('vma.c', 'vma-reader.c') + genh,
+ dependencies: [authz, block, crypto, io, qom], install: true)
+
subdir('storage-daemon')
subdir('contrib/rdmacm-mux')
subdir('contrib/elf2dmp')
diff --git a/vma-reader.c b/vma-reader.c
new file mode 100644
index 0000000000..2b1d1cdab3
index 0000000000..81a891c6b1
--- /dev/null
+++ b/vma-reader.c
@@ -0,0 +1,857 @@
@@ -0,0 +1,867 @@
+/*
+ * VMA: Virtual Machine Archive
+ *
@@ -72,7 +84,6 @@ index 0000000000..2b1d1cdab3
+#include <glib.h>
+#include <uuid/uuid.h>
+
+#include "qemu-common.h"
+#include "qemu/timer.h"
+#include "qemu/ratelimit.h"
+#include "vma.h"
@@ -86,6 +97,7 @@ index 0000000000..2b1d1cdab3
+ bool write_zeroes;
+ unsigned long *bitmap;
+ int bitmap_size;
+ bool skip;
+} VmaRestoreState;
+
+struct VmaReader {
@@ -249,6 +261,9 @@ index 0000000000..2b1d1cdab3
+ if (vmar->rstate[i].bitmap) {
+ g_free(vmar->rstate[i].bitmap);
+ }
+ if (vmar->rstate[i].target) {
+ blk_unref(vmar->rstate[i].target);
+ }
+ }
+
+ if (vmar->md5csum) {
@@ -480,13 +495,14 @@ index 0000000000..2b1d1cdab3
+}
+
+static void allocate_rstate(VmaReader *vmar, guint8 dev_id,
+ BlockBackend *target, bool write_zeroes)
+ BlockBackend *target, bool write_zeroes, bool skip)
+{
+ assert(vmar);
+ assert(dev_id);
+
+ vmar->rstate[dev_id].target = target;
+ vmar->rstate[dev_id].write_zeroes = write_zeroes;
+ vmar->rstate[dev_id].skip = skip;
+
+ int64_t size = vmar->devinfo[dev_id].size;
+
@@ -501,28 +517,30 @@ index 0000000000..2b1d1cdab3
+}
+
+int vma_reader_register_bs(VmaReader *vmar, guint8 dev_id, BlockBackend *target,
+ bool write_zeroes, Error **errp)
+ bool write_zeroes, bool skip, Error **errp)
+{
+ assert(vmar);
+ assert(target != NULL);
+ assert(target != NULL || skip);
+ assert(dev_id);
+ assert(vmar->rstate[dev_id].target == NULL);
+ assert(vmar->rstate[dev_id].target == NULL && !vmar->rstate[dev_id].skip);
+
+ int64_t size = blk_getlength(target);
+ int64_t size_diff = size - vmar->devinfo[dev_id].size;
+ if (target != NULL) {
+ int64_t size = blk_getlength(target);
+ int64_t size_diff = size - vmar->devinfo[dev_id].size;
+
+ /* storage types can have different size restrictions, so it
+ * is not always possible to create an image with exact size.
+ * So we tolerate a size difference up to 4MB.
+ */
+ if ((size_diff < 0) || (size_diff > 4*1024*1024)) {
+ error_setg(errp, "vma_reader_register_bs for stream %s failed - "
+ "unexpected size %zd != %zd", vmar->devinfo[dev_id].devname,
+ size, vmar->devinfo[dev_id].size);
+ return -1;
+ /* storage types can have different size restrictions, so it
+ * is not always possible to create an image with exact size.
+ * So we tolerate a size difference up to 4MB.
+ */
+ if ((size_diff < 0) || (size_diff > 4*1024*1024)) {
+ error_setg(errp, "vma_reader_register_bs for stream %s failed - "
+ "unexpected size %zd != %zd", vmar->devinfo[dev_id].devname,
+ size, vmar->devinfo[dev_id].size);
+ return -1;
+ }
+ }
+
+ allocate_rstate(vmar, dev_id, target, write_zeroes);
+ allocate_rstate(vmar, dev_id, target, write_zeroes, skip);
+
+ return 0;
+}
@@ -580,7 +598,7 @@ index 0000000000..2b1d1cdab3
+ }
+ }
+ } else {
+ int res = blk_pwrite(target, sector_num * BDRV_SECTOR_SIZE, buf, nb_sectors * BDRV_SECTOR_SIZE, 0);
+ int res = blk_pwrite(target, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE, buf, 0);
+ if (res < 0) {
+ error_setg(errp, "blk_pwrite to %s failed (%d)",
+ bdrv_get_device_name(blk_bs(target)), res);
@@ -615,19 +633,23 @@ index 0000000000..2b1d1cdab3
+ VmaRestoreState *rstate = &vmar->rstate[dev_id];
+ BlockBackend *target = NULL;
+
+ bool skip = rstate->skip;
+
+ if (dev_id != vmar->vmstate_stream) {
+ target = rstate->target;
+ if (!verify && !target) {
+ if (!verify && !target && !skip) {
+ error_setg(errp, "got wrong dev id %d", dev_id);
+ return -1;
+ }
+
+ if (vma_reader_get_bitmap(rstate, cluster_num)) {
+ error_setg(errp, "found duplicated cluster %zd for stream %s",
+ cluster_num, vmar->devinfo[dev_id].devname);
+ return -1;
+ if (!skip) {
+ if (vma_reader_get_bitmap(rstate, cluster_num)) {
+ error_setg(errp, "found duplicated cluster %zd for stream %s",
+ cluster_num, vmar->devinfo[dev_id].devname);
+ return -1;
+ }
+ vma_reader_set_bitmap(rstate, cluster_num, 1);
+ }
+ vma_reader_set_bitmap(rstate, cluster_num, 1);
+
+ max_sector = vmar->devinfo[dev_id].size/BDRV_SECTOR_SIZE;
+ } else {
@@ -673,7 +695,7 @@ index 0000000000..2b1d1cdab3
+ return -1;
+ }
+
+ if (!verify) {
+ if (!verify && !skip) {
+ int nb_sectors = end_sector - sector_num;
+ if (restore_write_data(vmar, dev_id, target, vmstate_fd,
+ buf + start, sector_num, nb_sectors,
@@ -709,7 +731,7 @@ index 0000000000..2b1d1cdab3
+ return -1;
+ }
+
+ if (!verify) {
+ if (!verify && !skip) {
+ int nb_sectors = end_sector - sector_num;
+ if (restore_write_data(vmar, dev_id, target, vmstate_fd,
+ buf + start, sector_num,
@@ -734,7 +756,7 @@ index 0000000000..2b1d1cdab3
+ vmar->partial_zero_cluster_data += zero_size;
+ }
+
+ if (rstate->write_zeroes && !verify) {
+ if (rstate->write_zeroes && !verify && !skip) {
+ if (restore_write_data(vmar, dev_id, target, vmstate_fd,
+ zero_vma_block, sector_num,
+ nb_sectors, errp) < 0) {
@@ -905,7 +927,7 @@ index 0000000000..2b1d1cdab3
+
+ for (dev_id = 1; dev_id < 255; dev_id++) {
+ if (vma_reader_get_device_info(vmar, dev_id)) {
+ allocate_rstate(vmar, dev_id, NULL, false);
+ allocate_rstate(vmar, dev_id, NULL, false, false);
+ }
+ }
+
@@ -914,10 +936,10 @@ index 0000000000..2b1d1cdab3
+
diff --git a/vma-writer.c b/vma-writer.c
new file mode 100644
index 0000000000..fe86b18a60
index 0000000000..126b296647
--- /dev/null
+++ b/vma-writer.c
@@ -0,0 +1,771 @@
@@ -0,0 +1,818 @@
+/*
+ * VMA: Virtual Machine Archive
+ *
@@ -933,6 +955,8 @@ index 0000000000..fe86b18a60
+
+#include "qemu/osdep.h"
+#include <glib.h>
+#include <linux/magic.h>
+#include <sys/vfs.h>
+#include <uuid/uuid.h>
+
+#include "vma.h"
@@ -941,6 +965,8 @@ index 0000000000..fe86b18a60
+#include "qemu/main-loop.h"
+#include "qemu/coroutine.h"
+#include "qemu/cutils.h"
+#include "qemu/error-report.h"
+#include "qemu/memalign.h"
+
+#define DEBUG_VMA 0
+
@@ -1123,10 +1149,10 @@ index 0000000000..fe86b18a60
+{
+ assert(qemu_in_coroutine());
+ AioContext *ctx = qemu_get_current_aio_context();
+ aio_set_fd_handler(ctx, fd, false, NULL, (IOHandler *)qemu_coroutine_enter,
+ aio_set_fd_handler(ctx, fd, NULL, (IOHandler *)qemu_coroutine_enter, NULL,
+ NULL, qemu_coroutine_self());
+ qemu_coroutine_yield();
+ aio_set_fd_handler(ctx, fd, false, NULL, NULL, NULL, NULL);
+ aio_set_fd_handler(ctx, fd, NULL, NULL, NULL, NULL, NULL);
+}
+
+static ssize_t coroutine_fn
@@ -1175,6 +1201,23 @@ index 0000000000..fe86b18a60
+ return (done == bytes) ? bytes : -1;
+}
+
+static bool is_path_tmpfs(const char *path) {
+ struct statfs fs;
+ int ret;
+
+ do {
+ ret = statfs(path, &fs);
+ } while (ret != 0 && errno == EINTR);
+
+ if (ret != 0) {
+ warn_report("statfs call for %s failed, assuming not tmpfs - %s\n",
+ path, strerror(errno));
+ return false;
+ }
+
+ return fs.f_type == TMPFS_MAGIC;
+}
+
+VmaWriter *vma_writer_create(const char *filename, uuid_t uuid, Error **errp)
+{
+ const char *p;
@@ -1213,23 +1256,30 @@ index 0000000000..fe86b18a60
+
+ if ((stat(filename, &st) == 0) && S_ISFIFO(st.st_mode)) {
+ oflags = O_NONBLOCK|O_WRONLY;
+ vmaw->fd = qemu_open(filename, oflags, 0644);
+ vmaw->fd = qemu_open(filename, oflags, errp);
+ } else if (strstart(filename, "/dev/fdset/", &tmp_id_str)) {
+ oflags = O_NONBLOCK|O_WRONLY;
+ vmaw->fd = qemu_open(filename, oflags, 0644);
+ vmaw->fd = qemu_open(filename, oflags, errp);
+ } else if (strstart(filename, "/dev/fdname/", &tmp_id_str)) {
+ vmaw->fd = monitor_get_fd(cur_mon, tmp_id_str, errp);
+ vmaw->fd = monitor_get_fd(monitor_cur(), tmp_id_str, errp);
+ if (vmaw->fd < 0) {
+ goto err;
+ }
+ /* try to use O_NONBLOCK */
+ fcntl(vmaw->fd, F_SETFL, fcntl(vmaw->fd, F_GETFL)|O_NONBLOCK);
+ } else {
+ oflags = O_NONBLOCK|O_DIRECT|O_WRONLY|O_CREAT|O_EXCL;
+ vmaw->fd = qemu_open(filename, oflags, 0644);
+ } else {
+ gchar *dirname = g_path_get_dirname(filename);
+ oflags = O_NONBLOCK|O_WRONLY|O_EXCL;
+ if (!is_path_tmpfs(dirname)) {
+ oflags |= O_DIRECT;
+ }
+ g_free(dirname);
+ vmaw->fd = qemu_create(filename, oflags, 0644, errp);
+ }
+
+ if (vmaw->fd < 0) {
+ error_free(*errp);
+ *errp = NULL;
+ error_setg(errp, "can't open file %s - %s\n", filename,
+ g_strerror(errno));
+ goto err;
@@ -1553,17 +1603,33 @@ index 0000000000..fe86b18a60
+
+ DPRINTF("VMA WRITE %d %zd\n", dev_id, cluster_num);
+
+ uint64_t dev_size = vmaw->stream_info[dev_id].size;
+ uint16_t mask = 0;
+
+ if (buf) {
+ int i;
+ int bit = 1;
+ uint64_t byte_offset = cluster_num * VMA_CLUSTER_SIZE;
+ for (i = 0; i < 16; i++) {
+ const unsigned char *vmablock = buf + (i*VMA_BLOCK_SIZE);
+ if (!buffer_is_zero(vmablock, VMA_BLOCK_SIZE)) {
+
+ // Note: If the source is not 64k-aligned, we might reach 4k blocks
+ // after the end of the device. Always mark these as zero in the
+ // mask, so the restore handles them correctly.
+ if (byte_offset < dev_size &&
+ !buffer_is_zero(vmablock, VMA_BLOCK_SIZE))
+ {
+ mask |= bit;
+ memcpy(vmaw->outbuf + vmaw->outbuf_pos, vmablock,
+ VMA_BLOCK_SIZE);
+
+ // prevent memory leakage on unaligned last block
+ if (byte_offset + VMA_BLOCK_SIZE > dev_size) {
+ uint64_t real_data_in_block = dev_size - byte_offset;
+ memset(vmaw->outbuf + vmaw->outbuf_pos + real_data_in_block,
+ 0, VMA_BLOCK_SIZE - real_data_in_block);
+ }
+
+ vmaw->outbuf_pos += VMA_BLOCK_SIZE;
+ } else {
+ DPRINTF("VMA WRITE %zd ZERO BLOCK %d\n", cluster_num, i);
@@ -1571,6 +1637,7 @@ index 0000000000..fe86b18a60
+ *zero_bytes += VMA_BLOCK_SIZE;
+ }
+
+ byte_offset += VMA_BLOCK_SIZE;
+ bit = bit << 1;
+ }
+ } else {
@@ -1596,8 +1663,8 @@ index 0000000000..fe86b18a60
+
+ if (dev_id != vmaw->vmstate_stream) {
+ uint64_t last = (cluster_num + 1) * VMA_CLUSTER_SIZE;
+ if (last > vmaw->stream_info[dev_id].size) {
+ uint64_t diff = last - vmaw->stream_info[dev_id].size;
+ if (last > dev_size) {
+ uint64_t diff = last - dev_size;
+ if (diff >= VMA_CLUSTER_SIZE) {
+ vma_writer_set_error(vmaw, "vma_writer_write: "
+ "read after last cluster");
@@ -1687,14 +1754,16 @@ index 0000000000..fe86b18a60
+ g_checksum_free(vmaw->md5csum);
+ }
+
+ qemu_vfree(vmaw->headerbuf);
+ qemu_vfree(vmaw->outbuf);
+ g_free(vmaw);
+}
diff --git a/vma.c b/vma.c
new file mode 100644
index 0000000000..a82752448a
index 0000000000..347f6283ca
--- /dev/null
+++ b/vma.c
@@ -0,0 +1,837 @@
@@ -0,0 +1,900 @@
+/*
+ * VMA: Virtual Machine Archive
+ *
@@ -1712,11 +1781,11 @@ index 0000000000..a82752448a
+#include <glib.h>
+
+#include "vma.h"
+#include "qemu-common.h"
+#include "qemu/module.h"
+#include "qemu/error-report.h"
+#include "qemu/main-loop.h"
+#include "qemu/cutils.h"
+#include "qemu/memalign.h"
+#include "qapi/qmp/qdict.h"
+#include "sysemu/block-backend.h"
+
@@ -1728,7 +1797,7 @@ index 0000000000..a82752448a
+ "vma list <filename>\n"
+ "vma config <filename> [-c config]\n"
+ "vma create <filename> [-c config] pathname ...\n"
+ "vma extract <filename> [-r <fifo>] <targetdir>\n"
+ "vma extract <filename> [-d <drive-list>] [-r <fifo>] <targetdir>\n"
+ "vma verify <filename> [-v]\n"
+ ;
+
@@ -1835,6 +1904,7 @@ index 0000000000..a82752448a
+ char *throttling_group;
+ char *cache;
+ bool write_zero;
+ bool skip;
+} RestoreMap;
+
+static bool try_parse_option(char **line, const char *optname, char **out, const char *inbuf) {
@@ -1872,9 +1942,10 @@ index 0000000000..a82752448a
+ const char *filename;
+ const char *dirname;
+ const char *readmap = NULL;
+ gchar **drive_list = NULL;
+
+ for (;;) {
+ c = getopt(argc, argv, "hvr:");
+ c = getopt(argc, argv, "hvd:r:");
+ if (c == -1) {
+ break;
+ }
@@ -1883,6 +1954,9 @@ index 0000000000..a82752448a
+ case 'h':
+ help();
+ break;
+ case 'd':
+ drive_list = g_strsplit(optarg, ",", 254);
+ break;
+ case 'r':
+ readmap = optarg;
+ break;
@@ -1942,47 +2016,61 @@ index 0000000000..a82752448a
+ char *bps = NULL;
+ char *group = NULL;
+ char *cache = NULL;
+ char *devname = NULL;
+ bool skip = false;
+ uint64_t bps_value = 0;
+ const char *path = NULL;
+ bool write_zero = true;
+
+ if (!line || line[0] == '\0' || !strcmp(line, "done\n")) {
+ break;
+ }
+ int len = strlen(line);
+ if (line[len - 1] == '\n') {
+ line[len - 1] = '\0';
+ if (len == 1) {
+ len = len - 1;
+ if (len == 0) {
+ break;
+ }
+ }
+
+ while (1) {
+ if (!try_parse_option(&line, "format", &format, inbuf) &&
+ !try_parse_option(&line, "throttling.bps", &bps, inbuf) &&
+ !try_parse_option(&line, "throttling.group", &group, inbuf) &&
+ !try_parse_option(&line, "cache", &cache, inbuf))
+ {
+ break;
+ if (strncmp(line, "skip", 4) == 0) {
+ if (len < 6 || line[4] != '=') {
+ g_error("read map failed - option 'skip' has no value ('%s')",
+ inbuf);
+ } else {
+ devname = line + 5;
+ skip = true;
+ }
+ }
+
+ uint64_t bps_value = 0;
+ if (bps) {
+ bps_value = verify_u64(bps);
+ g_free(bps);
+ }
+
+ const char *path;
+ bool write_zero;
+ if (line[0] == '0' && line[1] == ':') {
+ path = line + 2;
+ write_zero = false;
+ } else if (line[0] == '1' && line[1] == ':') {
+ path = line + 2;
+ write_zero = true;
+ } else {
+ g_error("read map failed - parse error ('%s')", inbuf);
+ while (1) {
+ if (!try_parse_option(&line, "format", &format, inbuf) &&
+ !try_parse_option(&line, "throttling.bps", &bps, inbuf) &&
+ !try_parse_option(&line, "throttling.group", &group, inbuf) &&
+ !try_parse_option(&line, "cache", &cache, inbuf))
+ {
+ break;
+ }
+ }
+
+ if (bps) {
+ bps_value = verify_u64(bps);
+ g_free(bps);
+ }
+
+ if (line[0] == '0' && line[1] == ':') {
+ path = line + 2;
+ write_zero = false;
+ } else if (line[0] == '1' && line[1] == ':') {
+ path = line + 2;
+ write_zero = true;
+ } else {
+ g_error("read map failed - parse error ('%s')", inbuf);
+ }
+
+ path = extract_devname(path, &devname, -1);
+ }
+
+ char *devname = NULL;
+ path = extract_devname(path, &devname, -1);
+ if (!devname) {
+ g_error("read map failed - no dev name specified ('%s')",
+ inbuf);
@@ -1996,6 +2084,7 @@ index 0000000000..a82752448a
+ map->throttling_group = group;
+ map->cache = cache;
+ map->write_zero = write_zero;
+ map->skip = skip;
+
+ g_hash_table_insert(devmap, map->devname, map);
+
@@ -2004,14 +2093,12 @@ index 0000000000..a82752448a
+
+ int i;
+ int vmstate_fd = -1;
+ guint8 vmstate_stream = 0;
+
+ BlockBackend *blk = NULL;
+ bool drive_rename_bitmap[255];
+ memset(drive_rename_bitmap, 0, sizeof(drive_rename_bitmap));
+
+ for (i = 1; i < 255; i++) {
+ VmaDeviceInfo *di = vma_reader_get_device_info(vmar, i);
+ if (di && (strcmp(di->devname, "vmstate") == 0)) {
+ vmstate_stream = i;
+ char *statefn = g_strdup_printf("%s/vmstate.bin", dirname);
+ vmstate_fd = open(statefn, O_WRONLY|O_CREAT|O_EXCL, 0644);
+ if (vmstate_fd < 0) {
@@ -2027,8 +2114,25 @@ index 0000000000..a82752448a
+ const char *cache = NULL;
+ int flags = BDRV_O_RDWR;
+ bool write_zero = true;
+ bool skip = false;
+
+ if (readmap) {
+ BlockBackend *blk = NULL;
+
+ if (drive_list) {
+ skip = true;
+ int j;
+ for (j = 0; drive_list[j]; j++) {
+ if (strcmp(drive_list[j], di->devname) == 0) {
+ skip = false;
+ drive_rename_bitmap[i] = true;
+ break;
+ }
+ }
+ } else {
+ drive_rename_bitmap[i] = true;
+ }
+
+ if (!skip && readmap) {
+ RestoreMap *map;
+ map = (RestoreMap *)g_hash_table_lookup(devmap, di->devname);
+ if (map == NULL) {
@@ -2040,7 +2144,8 @@ index 0000000000..a82752448a
+ throttling_group = map->throttling_group;
+ cache = map->cache;
+ write_zero = map->write_zero;
+ } else {
+ skip = map->skip;
+ } else if (!skip) {
+ devfn = g_strdup_printf("%s/tmp-disk-%s.raw",
+ dirname, di->devname);
+ printf("DEVINFO %s %zd\n", devfn, di->size);
@@ -2058,57 +2163,60 @@ index 0000000000..a82752448a
+ write_zero = false;
+ }
+
+ size_t devlen = strlen(devfn);
+ QDict *options = NULL;
+ bool writethrough;
+ if (format) {
+ /* explicit format from commandline */
+ options = qdict_new();
+ qdict_put_str(options, "driver", format);
+ } else if ((devlen > 4 && strcmp(devfn+devlen-4, ".raw") == 0) ||
+ strncmp(devfn, "/dev/", 5) == 0)
+ {
+ /* This part is now deprecated for PVE as well (just as qemu
+ * deprecated not specifying an explicit raw format, too.
+ */
+ /* explicit raw format */
+ options = qdict_new();
+ qdict_put_str(options, "driver", "raw");
+ }
+ if (cache && bdrv_parse_cache_mode(cache, &flags, &writethrough)) {
+ g_error("invalid cache option: %s\n", cache);
+ }
+
+ if (errp || !(blk = blk_new_open(devfn, NULL, options, flags, &errp))) {
+ g_error("can't open file %s - %s", devfn,
+ error_get_pretty(errp));
+ }
+
+ if (cache) {
+ blk_set_enable_write_cache(blk, !writethrough);
+ }
+
+ if (throttling_group) {
+ blk_io_limits_enable(blk, throttling_group);
+ }
+
+ if (throttling_bps) {
+ if (!throttling_group) {
+ blk_io_limits_enable(blk, devfn);
+ if (!skip) {
+ size_t devlen = strlen(devfn);
+ QDict *options = NULL;
+ bool writethrough;
+ if (format) {
+ /* explicit format from commandline */
+ options = qdict_new();
+ qdict_put_str(options, "driver", format);
+ } else if ((devlen > 4 && strcmp(devfn+devlen-4, ".raw") == 0) ||
+ strncmp(devfn, "/dev/", 5) == 0)
+ {
+ /* This part is now deprecated for PVE as well (just as qemu
+ * deprecated not specifying an explicit raw format, too.
+ */
+ /* explicit raw format */
+ options = qdict_new();
+ qdict_put_str(options, "driver", "raw");
+ }
+
+ ThrottleConfig cfg;
+ throttle_config_init(&cfg);
+ cfg.buckets[THROTTLE_BPS_WRITE].avg = throttling_bps;
+ Error *err = NULL;
+ if (!throttle_is_valid(&cfg, &err)) {
+ error_report_err(err);
+ g_error("failed to apply throttling");
+ if (cache && bdrv_parse_cache_mode(cache, &flags, &writethrough)) {
+ g_error("invalid cache option: %s\n", cache);
+ }
+
+ if (errp || !(blk = blk_new_open(devfn, NULL, options, flags, &errp))) {
+ g_error("can't open file %s - %s", devfn,
+ error_get_pretty(errp));
+ }
+
+ if (cache) {
+ blk_set_enable_write_cache(blk, !writethrough);
+ }
+
+ if (throttling_group) {
+ blk_io_limits_enable(blk, throttling_group);
+ }
+
+ if (throttling_bps) {
+ if (!throttling_group) {
+ blk_io_limits_enable(blk, devfn);
+ }
+
+ ThrottleConfig cfg;
+ throttle_config_init(&cfg);
+ cfg.buckets[THROTTLE_BPS_WRITE].avg = throttling_bps;
+ Error *err = NULL;
+ if (!throttle_is_valid(&cfg, &err)) {
+ error_report_err(err);
+ g_error("failed to apply throttling");
+ }
+ blk_set_io_limits(blk, &cfg);
+ }
+ blk_set_io_limits(blk, &cfg);
+ }
+
+ if (vma_reader_register_bs(vmar, i, blk, write_zero, &errp) < 0) {
+ if (vma_reader_register_bs(vmar, i, blk, write_zero, skip, &errp) < 0) {
+ g_error("%s", error_get_pretty(errp));
+ }
+
@@ -2118,6 +2226,10 @@ index 0000000000..a82752448a
+ }
+ }
+
+ if (drive_list) {
+ g_strfreev(drive_list);
+ }
+
+ if (vma_reader_restore(vmar, vmstate_fd, verbose, &errp) < 0) {
+ g_error("restore failed - %s", error_get_pretty(errp));
+ }
@@ -2125,7 +2237,7 @@ index 0000000000..a82752448a
+ if (!readmap) {
+ for (i = 1; i < 255; i++) {
+ VmaDeviceInfo *di = vma_reader_get_device_info(vmar, i);
+ if (di && (i != vmstate_stream)) {
+ if (di && drive_rename_bitmap[i]) {
+ char *tmpfn = g_strdup_printf("%s/tmp-disk-%s.raw",
+ dirname, di->devname);
+ char *fn = g_strdup_printf("%s/disk-%s.raw",
@@ -2140,8 +2252,6 @@ index 0000000000..a82752448a
+
+ vma_reader_destroy(vmar);
+
+ blk_unref(blk);
+
+ bdrv_close_all();
+
+ return ret;
@@ -2226,7 +2336,7 @@ index 0000000000..a82752448a
+ struct iovec iov;
+ QEMUIOVector qiov;
+
+ int64_t start, end;
+ int64_t start, end, readlen;
+ int ret = 0;
+
+ unsigned char *buf = blk_blockalign(job->target, VMA_CLUSTER_SIZE);
@@ -2240,16 +2350,24 @@ index 0000000000..a82752448a
+ iov.iov_len = VMA_CLUSTER_SIZE;
+ qemu_iovec_init_external(&qiov, &iov, 1);
+
+ if (start + 1 == end) {
+ memset(buf, 0, VMA_CLUSTER_SIZE);
+ readlen = job->len - start * VMA_CLUSTER_SIZE;
+ assert(readlen > 0 && readlen <= VMA_CLUSTER_SIZE);
+ } else {
+ readlen = VMA_CLUSTER_SIZE;
+ }
+
+ ret = blk_co_preadv(job->target, start * VMA_CLUSTER_SIZE,
+ VMA_CLUSTER_SIZE, &qiov, 0);
+ readlen, &qiov, 0);
+ if (ret < 0) {
+ vma_writer_set_error(job->vmaw, "read error", -1);
+ vma_writer_set_error(job->vmaw, "read error");
+ goto out;
+ }
+
+ size_t zb = 0;
+ if (vma_writer_write(job->vmaw, job->dev_id, start, buf, &zb) < 0) {
+ vma_writer_set_error(job->vmaw, "backup_dump_cb vma_writer_write failed", -1);
+ vma_writer_set_error(job->vmaw, "backup_dump_cb vma_writer_write failed");
+ goto out;
+ }
+ }
@@ -2262,6 +2380,7 @@ index 0000000000..a82752448a
+ g_warning("vma_writer_close failed %s", error_get_pretty(err));
+ }
+ }
+ qemu_vfree(buf);
+}
+
+static int create_archive(int argc, char **argv)
@@ -2269,6 +2388,7 @@ index 0000000000..a82752448a
+ int i, c;
+ int verbose = 0;
+ const char *archivename;
+ GList *backup_coroutines = NULL;
+ GList *config_files = NULL;
+
+ for (;;) {
@@ -2357,7 +2477,9 @@ index 0000000000..a82752448a
+ job->dev_id = dev_id;
+
+ Coroutine *co = qemu_coroutine_create(backup_run, job);
+ qemu_coroutine_enter(co);
+ // Don't enter coroutine yet, because it might write the header before
+ // all streams can be registered.
+ backup_coroutines = g_list_append(backup_coroutines, co);
+ }
+
+ VmaStatus vmastat;
@@ -2365,6 +2487,13 @@ index 0000000000..a82752448a
+ int last_percent = -1;
+
+ if (devcount) {
+ GList *entry = backup_coroutines;
+ while (entry && entry->data) {
+ Coroutine *co = entry->data;
+ qemu_coroutine_enter(co);
+ entry = g_list_next(entry);
+ }
+
+ while (1) {
+ main_loop_wait(false);
+ vma_writer_get_status(vmaw, &vmastat);
@@ -2429,6 +2558,9 @@ index 0000000000..a82752448a
+ g_error("creating vma archive failed");
+ }
+
+ g_list_free(backup_coroutines);
+ g_list_free(config_files);
+ vma_writer_destroy(vmaw);
+ return 0;
+}
+
@@ -2534,7 +2666,7 @@ index 0000000000..a82752448a
+}
diff --git a/vma.h b/vma.h
new file mode 100644
index 0000000000..c895c97f6d
index 0000000000..86d2873aa5
--- /dev/null
+++ b/vma.h
@@ -0,0 +1,150 @@
@@ -2672,7 +2804,7 @@ index 0000000000..c895c97f6d
+int coroutine_fn vma_writer_flush_output(VmaWriter *vmaw);
+
+int vma_writer_get_status(VmaWriter *vmaw, VmaStatus *status);
+void vma_writer_set_error(VmaWriter *vmaw, const char *fmt, ...);
+void vma_writer_set_error(VmaWriter *vmaw, const char *fmt, ...) G_GNUC_PRINTF(2, 3);
+
+
+VmaReader *vma_reader_create(const char *filename, Error **errp);
@@ -2682,7 +2814,7 @@ index 0000000000..c895c97f6d
+VmaDeviceInfo *vma_reader_get_device_info(VmaReader *vmar, guint8 dev_id);
+int vma_reader_register_bs(VmaReader *vmar, guint8 dev_id,
+ BlockBackend *target, bool write_zeroes,
+ Error **errp);
+ bool skip, Error **errp);
+int vma_reader_restore(VmaReader *vmar, int vmstate_fd, bool verbose,
+ Error **errp);
+int vma_reader_verify(VmaReader *vmar, bool verbose, Error **errp);

View File

@@ -1,92 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:56 +0200
Subject: [PATCH] PVE-Backup: modify job api
Introduce a pause_count parameter to start a backup in
paused mode. This way backups of multiple drives can be
started up sequentially via the completion callback while
having been started at the same point in time.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/backup.c | 3 +++
block/replication.c | 2 +-
blockdev.c | 3 ++-
include/block/block_int.h | 1 +
job.c | 2 +-
5 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index a7a7dcaf4c..ecd93e91e0 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -338,6 +338,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BlockCompletionFunc *cb, void *opaque,
+ int pause_count,
JobTxn *txn, Error **errp)
{
int64_t len;
@@ -459,6 +460,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
&error_abort);
+ job->common.job.pause_count += pause_count;
+
return &job->common;
error:
diff --git a/block/replication.c b/block/replication.c
index da013c2041..17246a822c 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -554,7 +554,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
- backup_job_completed, bs, NULL, &local_err);
+ backup_job_completed, bs, 0, NULL, &local_err);
if (local_err) {
error_propagate(errp, local_err);
backup_job_cleanup(bs);
diff --git a/blockdev.c b/blockdev.c
index 5faddaa705..65c358e4ef 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3114,7 +3114,8 @@ static BlockJob *do_backup_common(BackupCommon *backup,
backup->filter_node_name,
backup->on_source_error,
backup->on_target_error,
- job_flags, NULL, NULL, txn, errp);
+ job_flags, NULL, NULL, 0, txn, errp);
+
return job;
}
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 4c3587ea19..336f71e69d 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1219,6 +1219,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockdevOnError on_target_error,
int creation_flags,
BlockCompletionFunc *cb, void *opaque,
+ int pause_count,
JobTxn *txn, Error **errp);
BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
diff --git a/job.c b/job.c
index 53be57a3a0..e82253e041 100644
--- a/job.c
+++ b/job.c
@@ -918,7 +918,7 @@ void job_start(Job *job)
job->co = qemu_coroutine_create(job_co_entry, job);
job->pause_count--;
job->busy = true;
- job->paused = false;
+ job->paused = job->pause_count > 0;
job_state_transition(job, JOB_STATUS_RUNNING);
aio_co_enter(job->aio_context, job->co);
}

View File

@@ -7,33 +7,25 @@ Subject: [PATCH] PVE-Backup: add backup-dump block driver
- move BackupBlockJob declaration from block/backup.c to include/block/block_int.h
- block/backup.c - backup-job-create: also consider source cluster size
- job.c: make job_should_pause non-static
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to coroutine changes]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/Makefile.objs | 1 +
block/backup-dump.c | 169 ++++++++++++++++++++++++++++++++++++++
block/backup.c | 23 ++----
include/block/block_int.h | 30 +++++++
job.c | 3 +-
5 files changed, 207 insertions(+), 19 deletions(-)
block/backup-dump.c | 168 +++++++++++++++++++++++++++++++
block/backup.c | 30 ++----
block/meson.build | 1 +
include/block/block_int-common.h | 35 +++++++
job.c | 3 +-
5 files changed, 214 insertions(+), 23 deletions(-)
create mode 100644 block/backup-dump.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 1282445672..8af7073c83 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -34,6 +34,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
block-obj-$(CONFIG_GLUSTERFS) += gluster.o
block-obj-$(CONFIG_VXHS) += vxhs.o
block-obj-$(CONFIG_LIBSSH) += ssh.o
+block-obj-y += backup-dump.o
block-obj-y += accounting.o dirty-bitmap.o
block-obj-y += write-threshold.o
block-obj-y += backup.o
diff --git a/block/backup-dump.c b/block/backup-dump.c
new file mode 100644
index 0000000000..3066ab0698
index 0000000000..232a094426
--- /dev/null
+++ b/block/backup-dump.c
@@ -0,0 +1,169 @@
@@ -0,0 +1,168 @@
+/*
+ * BlockDriver to send backup data stream to a callback function
+ *
@@ -45,7 +37,6 @@ index 0000000000..3066ab0698
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qom/object_interfaces.h"
+#include "block/block_int.h"
+
@@ -56,12 +47,12 @@ index 0000000000..3066ab0698
+ void *dump_cb_data;
+} BDRVBackupDumpState;
+
+static int qemu_backup_dump_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
+static coroutine_fn int qemu_backup_dump_co_get_info(BlockDriverState *bs,
+ BlockDriverInfo *bdi)
+{
+ BDRVBackupDumpState *s = bs->opaque;
+
+ bdi->cluster_size = s->dump_cb_block_size;
+ bdi->unallocated_blocks_are_zero = true;
+ return 0;
+}
+
@@ -98,7 +89,7 @@ index 0000000000..3066ab0698
+ /* Nothing to do. */
+}
+
+static int64_t qemu_backup_dump_getlength(BlockDriverState *bs)
+static coroutine_fn int64_t qemu_backup_dump_co_getlength(BlockDriverState *bs)
+{
+ BDRVBackupDumpState *s = bs->opaque;
+
@@ -142,7 +133,7 @@ index 0000000000..3066ab0698
+static void qemu_backup_dump_child_perm(
+ BlockDriverState *bs,
+ BdrvChild *c,
+ const BdrvChildRole *role,
+ BdrvChildRole role,
+ BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
@@ -158,8 +149,8 @@ index 0000000000..3066ab0698
+
+ .bdrv_close = qemu_backup_dump_close,
+ .bdrv_has_zero_init = bdrv_has_zero_init_1,
+ .bdrv_getlength = qemu_backup_dump_getlength,
+ .bdrv_get_info = qemu_backup_dump_get_info,
+ .bdrv_co_getlength = qemu_backup_dump_co_getlength,
+ .bdrv_co_get_info = qemu_backup_dump_co_get_info,
+
+ .bdrv_co_writev = qemu_backup_dump_co_writev,
+
@@ -204,17 +195,18 @@ index 0000000000..3066ab0698
+ return bs;
+}
diff --git a/block/backup.c b/block/backup.c
index ecd93e91e0..cf8f5ad25d 100644
index 39410dcf8d..af87fa6aa9 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -32,24 +32,6 @@
@@ -29,28 +29,6 @@
#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
#include "block/copy-before-write.h"
-typedef struct BackupBlockJob {
- BlockJob common;
- BlockDriverState *backup_top;
- BlockDriverState *cbw;
- BlockDriverState *source_bs;
- BlockDriverState *target_bs;
-
- BdrvDirtyBitmap *sync_bitmap;
-
@@ -223,38 +215,64 @@ index ecd93e91e0..cf8f5ad25d 100644
- BlockdevOnError on_source_error;
- BlockdevOnError on_target_error;
- uint64_t len;
- uint64_t bytes_read;
- int64_t cluster_size;
- BackupPerf perf;
-
- BlockCopyState *bcs;
-
- bool wait;
- BlockCopyCallState *bg_bcs_call;
-} BackupBlockJob;
-
static const BlockJobDriver backup_job_driver;
static void backup_progress_bytes_callback(int64_t bytes, void *opaque)
@@ -411,6 +393,11 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
@@ -457,6 +435,14 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
}
cluster_size = block_copy_cluster_size(bcs);
+ if (cluster_size < 0) {
+ goto error;
+ }
+
+ BlockDriverInfo bdi;
+ if (bdrv_get_info(bs, &bdi) == 0) {
+ cluster_size = MAX(cluster_size, bdi.cluster_size);
+ }
+
/*
* If source is in backing chain of target assume that target is going to be
* used for "image fleecing", i.e. it should represent a kind of snapshot of
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 336f71e69d..62e5579723 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -60,6 +60,36 @@
if (perf->max_chunk && perf->max_chunk < cluster_size) {
error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
diff --git a/block/meson.build b/block/meson.build
index 59b71ba9f3..6fde9f7dcd 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -4,6 +4,7 @@ block_ss.add(files(
'aio_task.c',
'amend.c',
'backup.c',
+ 'backup-dump.c',
'copy-before-write.c',
'blkdebug.c',
'blklogwrites.c',
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index 74195c3004..0f2e1817ad 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -26,6 +26,7 @@
#include "block/aio.h"
#include "block/block-common.h"
+#include "block/block-copy.h"
#include "block/block-global-state.h"
#include "block/snapshot.h"
#include "qemu/iov.h"
@@ -60,6 +61,40 @@
#define BLOCK_PROBE_BUF_SIZE 512
+typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
+
+BlockDriverState *bdrv_backuo_dump_create(
+BlockDriverState *bdrv_backup_dump_create(
+ int dump_cb_block_size,
+ uint64_t byte_size,
+ BackupDumpFunc *dump_cb,
@@ -266,8 +284,9 @@ index 336f71e69d..62e5579723 100644
+typedef struct BlockCopyState BlockCopyState;
+typedef struct BackupBlockJob {
+ BlockJob common;
+ BlockDriverState *backup_top;
+ BlockDriverState *cbw;
+ BlockDriverState *source_bs;
+ BlockDriverState *target_bs;
+
+ BdrvDirtyBitmap *sync_bitmap;
+
@@ -276,26 +295,29 @@ index 336f71e69d..62e5579723 100644
+ BlockdevOnError on_source_error;
+ BlockdevOnError on_target_error;
+ uint64_t len;
+ uint64_t bytes_read;
+ int64_t cluster_size;
+ BackupPerf perf;
+
+ BlockCopyState *bcs;
+
+ bool wait;
+ BlockCopyCallState *bg_bcs_call;
+} BackupBlockJob;
+
enum BdrvTrackedRequestType {
BDRV_TRACKED_READ,
BDRV_TRACKED_WRITE,
diff --git a/job.c b/job.c
index e82253e041..bcbbb0be02 100644
index 72d57f0934..93e22d180b 100644
--- a/job.c
+++ b/job.c
@@ -269,7 +269,8 @@ static bool job_started(Job *job)
return job->co;
@@ -330,7 +330,8 @@ static bool job_started_locked(Job *job)
}
-static bool job_should_pause(Job *job)
+bool job_should_pause(Job *job);
+bool job_should_pause(Job *job)
/* Called with job_mutex held. */
-static bool job_should_pause_locked(Job *job)
+bool job_should_pause_locked(Job *job);
+bool job_should_pause_locked(Job *job)
{
return job->pause_count > 0;
}

View File

@@ -0,0 +1,101 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Thu, 20 Aug 2020 14:31:59 +0200
Subject: [PATCH] PVE: Add sequential job transaction support
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
include/qemu/job.h | 12 ++++++++++++
job.c | 34 ++++++++++++++++++++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/include/qemu/job.h b/include/qemu/job.h
index e502787dd8..963cf2bef5 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -381,6 +381,18 @@ void job_unlock(void);
*/
JobTxn *job_txn_new(void);
+/**
+ * Create a new transaction and set it to sequential mode, i.e. run all jobs
+ * one after the other instead of at the same time.
+ */
+JobTxn *job_txn_new_seq(void);
+
+/**
+ * Helper method to start the first job in a sequential transaction to kick it
+ * off. Other jobs will be run after this one completes.
+ */
+void job_txn_start_seq(JobTxn *txn);
+
/**
* Release a reference that was previously acquired with job_txn_add_job or
* job_txn_new. If it's the last reference to the object, it will be freed.
diff --git a/job.c b/job.c
index 93e22d180b..2b31f1e14f 100644
--- a/job.c
+++ b/job.c
@@ -93,6 +93,8 @@ struct JobTxn {
/* Reference count */
int refcnt;
+
+ bool sequential;
};
void job_lock(void)
@@ -118,6 +120,25 @@ JobTxn *job_txn_new(void)
return txn;
}
+JobTxn *job_txn_new_seq(void)
+{
+ JobTxn *txn = job_txn_new();
+ txn->sequential = true;
+ return txn;
+}
+
+void job_txn_start_seq(JobTxn *txn)
+{
+ assert(txn->sequential);
+ assert(!txn->aborting);
+
+ Job *first = QLIST_FIRST(&txn->jobs);
+ assert(first);
+ assert(first->status == JOB_STATUS_CREATED);
+
+ job_start(first);
+}
+
/* Called with job_mutex held. */
static void job_txn_ref_locked(JobTxn *txn)
{
@@ -1057,6 +1078,12 @@ static void job_completed_txn_success_locked(Job *job)
*/
QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
if (!job_is_completed_locked(other_job)) {
+ if (txn->sequential) {
+ job_unlock();
+ /* Needs to be called without holding the job lock */
+ job_start(other_job);
+ job_lock();
+ }
return;
}
assert(other_job->ret == 0);
@@ -1268,6 +1295,13 @@ int job_finish_sync_locked(Job *job,
return -EBUSY;
}
+ /* in a sequential transaction jobs with status CREATED can appear at time
+ * of cancelling, these have not begun work so job_enter won't do anything,
+ * let's ensure they are marked as ABORTING if required */
+ if (job->status == JOB_STATUS_CREATED && job->txn->sequential) {
+ job_update_rc_locked(job);
+ }
+
job_unlock();
AIO_WAIT_WHILE_UNLOCKED(job->aio_context,
(job_enter(job), !job_is_completed(job)));

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,39 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:17:00 +0200
Subject: [PATCH] PVE-Backup: aquire aio_context before calling
backup_job_create
And do not set target in same aoi_context as source, because
this is already done in bdrv_backup_top_append ...
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
pve-backup.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
index 9ae89fb679..38dd33e28b 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -757,17 +757,15 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- // make sure target runs in same aoi_context as source
AioContext *aio_context = bdrv_get_aio_context(di->bs);
aio_context_acquire(aio_context);
- GSList *ignore = NULL;
- bdrv_set_aio_context_ignore(di->target, aio_context, &ignore);
- g_slist_free(ignore);
- aio_context_release(aio_context);
job = backup_job_create(NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
BITMAP_SYNC_MODE_NEVER, false, NULL, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
JOB_DEFAULT, pvebackup_complete_cb, di, 1, NULL, &local_err);
+
+ aio_context_release(aio_context);
+
if (!job || local_err != NULL) {
qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
error_setg(&backup_state.stat.error, "backup_job_create failed");

View File

@@ -4,40 +4,36 @@ Date: Mon, 6 Apr 2020 12:17:01 +0200
Subject: [PATCH] PVE-Backup: pbs-restore - new command to restore from proxmox
backup server
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[WB: add namespace support]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
Makefile | 4 +-
pbs-restore.c | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 211 insertions(+), 1 deletion(-)
meson.build | 4 +
pbs-restore.c | 236 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 240 insertions(+)
create mode 100644 pbs-restore.c
diff --git a/Makefile b/Makefile
index dbd9542ae4..7c1fb58e18 100644
--- a/Makefile
+++ b/Makefile
@@ -479,7 +479,7 @@ dummy := $(call unnest-vars,, \
include $(SRC_PATH)/tests/Makefile.include
-all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) pbs-restore$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
qemu-version.h: FORCE
$(call quiet-command, \
@@ -610,6 +610,8 @@ qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-o
qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) $(storage-daemon-obj-y) $(COMMON_LDADDS)
qemu-storage-daemon$(EXESUF): LIBS += -lproxmox_backup_qemu
vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+pbs-restore$(EXESUF): pbs-restore.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+pbs-restore$(EXESUF): LIBS += -lproxmox_backup_qemu
qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
diff --git a/meson.build b/meson.build
index d53976d621..c3330310d9 100644
--- a/meson.build
+++ b/meson.build
@@ -3914,6 +3914,10 @@ if have_tools
vma = executable('vma', files('vma.c', 'vma-reader.c') + genh,
dependencies: [authz, block, crypto, io, qom], install: true)
+ pbs_restore = executable('pbs-restore', files('pbs-restore.c') + genh,
+ dependencies: [authz, block, crypto, io, qom,
+ libproxmox_backup_qemu], install: true)
+
subdir('storage-daemon')
subdir('contrib/rdmacm-mux')
subdir('contrib/elf2dmp')
diff --git a/pbs-restore.c b/pbs-restore.c
new file mode 100644
index 0000000000..f65de8b890
index 0000000000..f03d9bab8d
--- /dev/null
+++ b/pbs-restore.c
@@ -0,0 +1,208 @@
@@ -0,0 +1,236 @@
+/*
+ * Qemu image restore helper for Proxmox Backup
+ *
@@ -56,7 +52,6 @@ index 0000000000..f65de8b890
+#include <getopt.h>
+#include <string.h>
+
+#include "qemu-common.h"
+#include "qemu/module.h"
+#include "qemu/error-report.h"
+#include "qemu/main-loop.h"
@@ -70,7 +65,7 @@ index 0000000000..f65de8b890
+static void help(void)
+{
+ const char *help_msg =
+ "usage: pbs-restore [--repository <repo>] snapshot archive-name target [command options]\n"
+ "usage: pbs-restore [--repository <repo>] [--ns namespace] snapshot archive-name target [command options]\n"
+ ;
+
+ printf("%s", help_msg);
@@ -102,7 +97,7 @@ index 0000000000..f65de8b890
+ }
+ res = blk_pwrite_zeroes(callback_data->target, offset, data_len, 0);
+ } else {
+ res = blk_pwrite(callback_data->target, offset, data, data_len, 0);
+ res = blk_pwrite(callback_data->target, offset, data_len, data, 0);
+ }
+
+ if (res < 0) {
@@ -118,19 +113,21 @@ index 0000000000..f65de8b890
+ Error *main_loop_err = NULL;
+ const char *format = "raw";
+ const char *repository = NULL;
+ const char *backup_ns = NULL;
+ const char *keyfile = NULL;
+ int verbose = false;
+ bool skip_zero = false;
+
+ error_init(argv[0]);
+
+ for(;;) {
+ for (;;) {
+ static const struct option long_options[] = {
+ {"help", no_argument, 0, 'h'},
+ {"skip-zero", no_argument, 0, 'S'},
+ {"verbose", no_argument, 0, 'v'},
+ {"format", required_argument, 0, 'f'},
+ {"repository", required_argument, 0, 'r'},
+ {"ns", required_argument, 0, 'n'},
+ {"keyfile", required_argument, 0, 'k'},
+ {0, 0, 0, 0}
+ };
@@ -138,31 +135,34 @@ index 0000000000..f65de8b890
+ if (c == -1) {
+ break;
+ }
+ switch(c) {
+ case ':':
+ fprintf(stderr, "missing argument for option '%s'", argv[optind - 1]);
+ return -1;
+ case '?':
+ fprintf(stderr, "unrecognized option '%s'", argv[optind - 1]);
+ return -1;
+ case 'f':
+ format = g_strdup(argv[optind - 1]);
+ break;
+ case 'r':
+ repository = g_strdup(argv[optind - 1]);
+ break;
+ case 'k':
+ keyfile = g_strdup(argv[optind - 1]);
+ break;
+ case 'v':
+ verbose = true;
+ break;
+ case 'S':
+ skip_zero = true;
+ break;
+ case 'h':
+ help();
+ return 0;
+ switch (c) {
+ case ':':
+ fprintf(stderr, "missing argument for option '%s'\n", argv[optind - 1]);
+ return -1;
+ case '?':
+ fprintf(stderr, "unrecognized option '%s'\n", argv[optind - 1]);
+ return -1;
+ case 'f':
+ format = g_strdup(argv[optind - 1]);
+ break;
+ case 'r':
+ repository = g_strdup(argv[optind - 1]);
+ break;
+ case 'n':
+ backup_ns = g_strdup(argv[optind - 1]);
+ break;
+ case 'k':
+ keyfile = g_strdup(argv[optind - 1]);
+ break;
+ case 'v':
+ verbose = true;
+ break;
+ case 'S':
+ skip_zero = true;
+ break;
+ case 'h':
+ help();
+ return 0;
+ }
+ }
+
@@ -197,31 +197,50 @@ index 0000000000..f65de8b890
+ bdrv_init();
+ module_call_init(MODULE_INIT_QOM);
+
+ if (verbose) {
+ fprintf(stderr, "connecting to repository '%s'\n", repository);
+ }
+ char *pbs_error = NULL;
+ ProxmoxRestoreHandle *conn = proxmox_restore_connect(
+ repository, snapshot, password, keyfile, key_password, fingerprint, &pbs_error);
+ ProxmoxRestoreHandle *conn = proxmox_restore_new_ns(
+ repository,
+ snapshot,
+ backup_ns,
+ password,
+ keyfile,
+ key_password,
+ fingerprint,
+ &pbs_error
+ );
+ if (conn == NULL) {
+ fprintf(stderr, "restore failed: %s\n", pbs_error);
+ return -1;
+ }
+
+ int res = proxmox_restore_connect(conn, &pbs_error);
+ if (res < 0 || pbs_error) {
+ fprintf(stderr, "restore failed (connection error): %s\n", pbs_error);
+ return -1;
+ }
+
+ QDict *options = qdict_new();
+ qdict_put_str(options, "driver", format);
+
+ if (format) {
+ qdict_put_str(options, "driver", format);
+ }
+
+
+ if (verbose) {
+ fprintf(stderr, "open block backend for target '%s'\n", target);
+ }
+ Error *local_err = NULL;
+ int flags = BDRV_O_RDWR;
+
+ BlockBackend *blk = blk_new_open(target, NULL, options, flags, &local_err);
+ if (!blk) {
+ fprintf(stderr, "%s\n", error_get_pretty(local_err));
+ return -1;
+ }
+
+ CallbackData *callback_data = calloc(sizeof( CallbackData), 1);
+ CallbackData *callback_data = calloc(sizeof(CallbackData), 1);
+
+ callback_data->target = blk;
+ callback_data->skip_zero = skip_zero;
@@ -229,7 +248,11 @@ index 0000000000..f65de8b890
+
+ // blk_set_enable_write_cache(blk, !writethrough);
+
+ int res = proxmox_restore_image(
+ if (verbose) {
+ fprintf(stderr, "starting to restore snapshot '%s'\n", snapshot);
+ fflush(stderr); // ensure we do not get printed after the progress log
+ }
+ res = proxmox_restore_image(
+ conn,
+ archive_name,
+ write_callback,
@@ -238,6 +261,7 @@ index 0000000000..f65de8b890
+ verbose);
+
+ proxmox_restore_disconnect(conn);
+ blk_unref(blk);
+
+ if (res < 0) {
+ fprintf(stderr, "restore failed: %s\n", pbs_error);

View File

@@ -0,0 +1,454 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Wed, 8 Jul 2020 09:50:54 +0200
Subject: [PATCH] PVE: Add PBS block driver to map backup archives into VMs
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
[error cleanups, file_open implementation]
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[WB: add namespace support]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
[FE: adapt to changed function signatures
make pbs_co_preadv return values consistent with QEMU
getlength is now a coroutine function]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/meson.build | 3 +
block/pbs.c | 305 +++++++++++++++++++++++++++++++++++++++++++
configure | 9 ++
meson.build | 2 +-
qapi/block-core.json | 13 ++
qapi/pragma.json | 1 +
6 files changed, 332 insertions(+), 1 deletion(-)
create mode 100644 block/pbs.c
diff --git a/block/meson.build b/block/meson.build
index 6d468f89e5..becc99ac4e 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -50,6 +50,9 @@ block_ss.add(files(
'../pve-backup.c',
), libproxmox_backup_qemu)
+block_ss.add(when: 'CONFIG_PBS_BDRV', if_true: files('pbs.c'))
+block_ss.add(when: 'CONFIG_PBS_BDRV', if_true: libproxmox_backup_qemu)
+
system_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
system_ss.add(files('block-ram-registrar.c'))
diff --git a/block/pbs.c b/block/pbs.c
new file mode 100644
index 0000000000..a2211e0f3b
--- /dev/null
+++ b/block/pbs.c
@@ -0,0 +1,305 @@
+/*
+ * Proxmox Backup Server read-only block driver
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "qemu/module.h"
+#include "qemu/option.h"
+#include "qemu/cutils.h"
+#include "block/block_int.h"
+#include "block/block-io.h"
+
+#include <proxmox-backup-qemu.h>
+
+#define PBS_OPT_REPOSITORY "repository"
+#define PBS_OPT_NAMESPACE "namespace"
+#define PBS_OPT_SNAPSHOT "snapshot"
+#define PBS_OPT_ARCHIVE "archive"
+#define PBS_OPT_KEYFILE "keyfile"
+#define PBS_OPT_PASSWORD "password"
+#define PBS_OPT_FINGERPRINT "fingerprint"
+#define PBS_OPT_ENCRYPTION_PASSWORD "key_password"
+
+typedef struct {
+ ProxmoxRestoreHandle *conn;
+ char aid;
+ int64_t length;
+
+ char *repository;
+ char *namespace;
+ char *snapshot;
+ char *archive;
+} BDRVPBSState;
+
+static QemuOptsList runtime_opts = {
+ .name = "pbs",
+ .head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head),
+ .desc = {
+ {
+ .name = PBS_OPT_REPOSITORY,
+ .type = QEMU_OPT_STRING,
+ .help = "The server address and repository to connect to.",
+ },
+ {
+ .name = PBS_OPT_NAMESPACE,
+ .type = QEMU_OPT_STRING,
+ .help = "Optional: The snapshot's namespace.",
+ },
+ {
+ .name = PBS_OPT_SNAPSHOT,
+ .type = QEMU_OPT_STRING,
+ .help = "The snapshot to read.",
+ },
+ {
+ .name = PBS_OPT_ARCHIVE,
+ .type = QEMU_OPT_STRING,
+ .help = "Which archive within the snapshot should be accessed.",
+ },
+ {
+ .name = PBS_OPT_PASSWORD,
+ .type = QEMU_OPT_STRING,
+ .help = "Server password. Can be passed as env var 'PBS_PASSWORD'.",
+ },
+ {
+ .name = PBS_OPT_FINGERPRINT,
+ .type = QEMU_OPT_STRING,
+ .help = "Server fingerprint. Can be passed as env var 'PBS_FINGERPRINT'.",
+ },
+ {
+ .name = PBS_OPT_ENCRYPTION_PASSWORD,
+ .type = QEMU_OPT_STRING,
+ .help = "Optional: Key password. Can be passed as env var 'PBS_ENCRYPTION_PASSWORD'.",
+ },
+ {
+ .name = PBS_OPT_KEYFILE,
+ .type = QEMU_OPT_STRING,
+ .help = "Optional: The path to the keyfile to use.",
+ },
+ { /* end of list */ }
+ },
+};
+
+
+// filename format:
+// pbs:repository=<repo>,namespace=<ns>,snapshot=<snap>,password=<pw>,key_password=<kpw>,fingerprint=<fp>,archive=<archive>
+static void pbs_parse_filename(const char *filename, QDict *options,
+ Error **errp)
+{
+
+ if (!strstart(filename, "pbs:", &filename)) {
+ if (errp) error_setg(errp, "pbs_parse_filename failed - missing 'pbs:' prefix");
+ }
+
+
+ QemuOpts *opts = qemu_opts_parse_noisily(&runtime_opts, filename, false);
+ if (!opts) {
+ if (errp) error_setg(errp, "pbs_parse_filename failed");
+ return;
+ }
+
+ qemu_opts_to_qdict(opts, options);
+
+ qemu_opts_del(opts);
+}
+
+static int pbs_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
+{
+ QemuOpts *opts;
+ BDRVPBSState *s = bs->opaque;
+ char *pbs_error = NULL;
+
+ opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
+ qemu_opts_absorb_qdict(opts, options, &error_abort);
+
+ s->repository = g_strdup(qemu_opt_get(opts, PBS_OPT_REPOSITORY));
+ s->snapshot = g_strdup(qemu_opt_get(opts, PBS_OPT_SNAPSHOT));
+ s->archive = g_strdup(qemu_opt_get(opts, PBS_OPT_ARCHIVE));
+ const char *keyfile = qemu_opt_get(opts, PBS_OPT_KEYFILE);
+ const char *password = qemu_opt_get(opts, PBS_OPT_PASSWORD);
+ const char *namespace = qemu_opt_get(opts, PBS_OPT_NAMESPACE);
+ const char *fingerprint = qemu_opt_get(opts, PBS_OPT_FINGERPRINT);
+ const char *key_password = qemu_opt_get(opts, PBS_OPT_ENCRYPTION_PASSWORD);
+
+ if (!password) {
+ password = getenv("PBS_PASSWORD");
+ }
+ if (!fingerprint) {
+ fingerprint = getenv("PBS_FINGERPRINT");
+ }
+ if (!key_password) {
+ key_password = getenv("PBS_ENCRYPTION_PASSWORD");
+ }
+ if (namespace) {
+ s->namespace = g_strdup(namespace);
+ }
+
+ /* connect to PBS server in read mode */
+ s->conn = proxmox_restore_new_ns(s->repository, s->snapshot, s->namespace, password,
+ keyfile, key_password, fingerprint, &pbs_error);
+
+ /* invalidates qemu_opt_get char pointers from above */
+ qemu_opts_del(opts);
+
+ if (!s->conn) {
+ if (pbs_error && errp) error_setg(errp, "PBS restore_new failed: %s", pbs_error);
+ if (pbs_error) proxmox_backup_free_error(pbs_error);
+ return -ENOMEM;
+ }
+
+ int ret = proxmox_restore_connect(s->conn, &pbs_error);
+ if (ret < 0) {
+ if (pbs_error && errp) error_setg(errp, "PBS connect failed: %s", pbs_error);
+ if (pbs_error) proxmox_backup_free_error(pbs_error);
+ return -ECONNREFUSED;
+ }
+
+ /* acquire handle and length */
+ s->aid = proxmox_restore_open_image(s->conn, s->archive, &pbs_error);
+ if (s->aid < 0) {
+ if (pbs_error && errp) error_setg(errp, "PBS open_image failed: %s", pbs_error);
+ if (pbs_error) proxmox_backup_free_error(pbs_error);
+ return -ENODEV;
+ }
+ s->length = proxmox_restore_get_image_length(s->conn, s->aid, &pbs_error);
+ if (s->length < 0) {
+ if (pbs_error && errp) error_setg(errp, "PBS get_image_length failed: %s", pbs_error);
+ if (pbs_error) proxmox_backup_free_error(pbs_error);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int pbs_file_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
+{
+ return pbs_open(bs, options, flags, errp);
+}
+
+static void pbs_close(BlockDriverState *bs) {
+ BDRVPBSState *s = bs->opaque;
+ g_free(s->repository);
+ g_free(s->namespace);
+ g_free(s->snapshot);
+ g_free(s->archive);
+ proxmox_restore_disconnect(s->conn);
+}
+
+static coroutine_fn int64_t pbs_co_getlength(BlockDriverState *bs)
+{
+ BDRVPBSState *s = bs->opaque;
+ return s->length;
+}
+
+typedef struct ReadCallbackData {
+ Coroutine *co;
+ AioContext *ctx;
+} ReadCallbackData;
+
+static void read_callback(void *callback_data)
+{
+ ReadCallbackData *rcb = callback_data;
+ aio_co_schedule(rcb->ctx, rcb->co);
+}
+
+static coroutine_fn int pbs_co_preadv(BlockDriverState *bs,
+ int64_t offset, int64_t bytes,
+ QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ BDRVPBSState *s = bs->opaque;
+ int ret;
+ char *pbs_error = NULL;
+ uint8_t *buf;
+ bool inline_buf = true;
+
+ /* for single-buffer IO vectors we can fast-path the write directly to it */
+ if (qiov->niov == 1 && qiov->iov->iov_len >= bytes) {
+ buf = qiov->iov->iov_base;
+ } else {
+ inline_buf = false;
+ buf = g_malloc(bytes);
+ }
+
+ if (offset < 0 || bytes < 0) {
+ fprintf(stderr, "unexpected negative 'offset' or 'bytes' value!\n");
+ return -EIO;
+ }
+
+ ReadCallbackData rcb = {
+ .co = qemu_coroutine_self(),
+ .ctx = bdrv_get_aio_context(bs),
+ };
+
+ proxmox_restore_read_image_at_async(s->conn, s->aid, buf, (uint64_t)offset, (uint64_t)bytes,
+ read_callback, (void *) &rcb, &ret, &pbs_error);
+
+ qemu_coroutine_yield();
+
+ if (ret < 0) {
+ fprintf(stderr, "error during PBS read: %s\n", pbs_error ? pbs_error : "unknown error");
+ if (pbs_error) proxmox_backup_free_error(pbs_error);
+ return -EIO;
+ }
+
+ if (!inline_buf) {
+ qemu_iovec_from_buf(qiov, 0, buf, bytes);
+ g_free(buf);
+ }
+
+ return 0;
+}
+
+static coroutine_fn int pbs_co_pwritev(BlockDriverState *bs,
+ int64_t offset, int64_t bytes,
+ QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ fprintf(stderr, "pbs-bdrv: cannot write to backup file, make sure "
+ "any attached disk devices are set to read-only!\n");
+ return -EPERM;
+}
+
+static void pbs_refresh_filename(BlockDriverState *bs)
+{
+ BDRVPBSState *s = bs->opaque;
+ if (s->namespace) {
+ snprintf(bs->exact_filename, sizeof(bs->exact_filename), "%s/%s:%s(%s)",
+ s->repository, s->namespace, s->snapshot, s->archive);
+ } else {
+ snprintf(bs->exact_filename, sizeof(bs->exact_filename), "%s/%s(%s)",
+ s->repository, s->snapshot, s->archive);
+ }
+}
+
+static const char *const pbs_strong_runtime_opts[] = {
+ NULL
+};
+
+static BlockDriver bdrv_pbs_co = {
+ .format_name = "pbs",
+ .protocol_name = "pbs",
+ .instance_size = sizeof(BDRVPBSState),
+
+ .bdrv_parse_filename = pbs_parse_filename,
+
+ .bdrv_file_open = pbs_file_open,
+ .bdrv_open = pbs_open,
+ .bdrv_close = pbs_close,
+ .bdrv_co_getlength = pbs_co_getlength,
+
+ .bdrv_co_preadv = pbs_co_preadv,
+ .bdrv_co_pwritev = pbs_co_pwritev,
+
+ .bdrv_refresh_filename = pbs_refresh_filename,
+ .strong_runtime_opts = pbs_strong_runtime_opts,
+};
+
+static void bdrv_pbs_init(void)
+{
+ bdrv_register(&bdrv_pbs_co);
+}
+
+block_init(bdrv_pbs_init);
diff --git a/configure b/configure
index 133f4e3235..f5a830c1f3 100755
--- a/configure
+++ b/configure
@@ -256,6 +256,7 @@ qemu_suffix="qemu"
softmmu="yes"
linux_user=""
bsd_user=""
+pbs_bdrv="yes"
plugins="$default_feature"
ninja=""
python=
@@ -809,6 +810,10 @@ for opt do
;;
--enable-download) download="enabled"; git_submodules_action=update;
;;
+ --disable-pbs-bdrv) pbs_bdrv="no"
+ ;;
+ --enable-pbs-bdrv) pbs_bdrv="yes"
+ ;;
--enable-plugins) if test "$mingw32" = "yes"; then
error_exit "TCG plugins not currently supported on Windows platforms"
else
@@ -959,6 +964,7 @@ cat << EOF
bsd-user all BSD usermode emulation targets
pie Position Independent Executables
debug-tcg TCG debugging (default is disabled)
+ pbs-bdrv Proxmox backup server read-only block driver support
NOTE: The object files are built at the place where configure is launched
EOF
@@ -1744,6 +1750,9 @@ if test "$solaris" = "yes" ; then
fi
echo "SRC_PATH=$source_path" >> $config_host_mak
echo "TARGET_DIRS=$target_list" >> $config_host_mak
+if test "$pbs_bdrv" = "yes" ; then
+ echo "CONFIG_PBS_BDRV=y" >> $config_host_mak
+fi
# XXX: suppress that
if [ "$bsd" = "yes" ] ; then
diff --git a/meson.build b/meson.build
index c3330310d9..cbfc9a43fb 100644
--- a/meson.build
+++ b/meson.build
@@ -4319,7 +4319,7 @@ summary_info += {'bzip2 support': libbzip2}
summary_info += {'lzfse support': liblzfse}
summary_info += {'zstd support': zstd}
summary_info += {'NUMA host support': numa}
-summary_info += {'capstone': capstone}
+summary_info += {'PBS bdrv support': config_host.has_key('CONFIG_PBS_BDRV')}
summary_info += {'libpmem support': libpmem}
summary_info += {'libdaxctl support': libdaxctl}
summary_info += {'libudev': libudev}
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1b8462a51b..d67a6d448a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3396,6 +3396,7 @@
'parallels', 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum',
'raw', 'rbd',
{ 'name': 'replication', 'if': 'CONFIG_REPLICATION' },
+ 'pbs',
'ssh', 'throttle', 'vdi', 'vhdx',
{ 'name': 'virtio-blk-vfio-pci', 'if': 'CONFIG_BLKIO' },
{ 'name': 'virtio-blk-vhost-user', 'if': 'CONFIG_BLKIO' },
@@ -3482,6 +3483,17 @@
{ 'struct': 'BlockdevOptionsNull',
'data': { '*size': 'int', '*latency-ns': 'uint64', '*read-zeroes': 'bool' } }
+##
+# @BlockdevOptionsPbs:
+#
+# Driver specific block device options for the PBS backend.
+#
+##
+{ 'struct': 'BlockdevOptionsPbs',
+ 'data': { 'repository': 'str', 'snapshot': 'str', 'archive': 'str',
+ '*keyfile': 'str', '*password': 'str', '*fingerprint': 'str',
+ '*key_password': 'str', '*namespace': 'str' } }
+
##
# @BlockdevOptionsNVMe:
#
@@ -4890,6 +4902,7 @@
'nfs': 'BlockdevOptionsNfs',
'null-aio': 'BlockdevOptionsNull',
'null-co': 'BlockdevOptionsNull',
+ 'pbs': 'BlockdevOptionsPbs',
'nvme': 'BlockdevOptionsNVMe',
'nvme-io_uring': { 'type': 'BlockdevOptionsNvmeIoUring',
'if': 'CONFIG_BLKIO' },
diff --git a/qapi/pragma.json b/qapi/pragma.json
index 325e684411..b6079f6a0e 100644
--- a/qapi/pragma.json
+++ b/qapi/pragma.json
@@ -45,6 +45,7 @@
'BlockInfo', # query-block
'BlockdevAioOptions', # blockdev-add, -blockdev
'BlockdevDriver', # blockdev-add, query-blockstats, ...
+ 'BlockdevOptionsPbs', # for PBS backwards compat
'BlockdevVmdkAdapterType', # blockdev-create (to match VMDK spec)
'BlockdevVmdkSubformat', # blockdev-create (to match VMDK spec)
'ColoCompareProperties', # object_add, -object

View File

@@ -1,884 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:17:02 +0200
Subject: [PATCH] PVE-Backup: avoid coroutines to fix AIO freeze, cleanups
We observed various AIO pool loop freezes, so we decided to avoid
coroutines and restrict ourselfes using similar code as upstream
(see blockdev.c: do_backup_common).
* avoid coroutine for job related code (causes hangs with iothreads)
- this imply to use normal QemuRecMutex instead of CoMutex
* split pvebackup_co_dump_cb into:
- pvebackup_co_dump_pbs_cb and
- pvebackup_co_dump_pbs_cb
* new helper functions
- pvebackup_propagate_error
- pvebackup_error_or_canceled
- pvebackup_add_transfered_bytes
* avoid cancel flag (not needed)
* simplify backup_cancel logic
There is progress on upstream to support running qmp commands inside
coroutines, see:
https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg04852.html
We should consider using that when it is available in upstream qemu.
---
pve-backup.c | 611 +++++++++++++++++++++++++--------------------------
1 file changed, 299 insertions(+), 312 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
index 38dd33e28b..169f0c68d0 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -11,11 +11,10 @@
/* PVE backup state and related function */
-
static struct PVEBackupState {
struct {
- // Everithing accessed from qmp command, protected using rwlock
- CoRwlock rwlock;
+ // Everithing accessed from qmp_backup_query command is protected using lock
+ QemuRecMutex lock;
Error *error;
time_t start_time;
time_t end_time;
@@ -25,19 +24,18 @@ static struct PVEBackupState {
size_t total;
size_t transferred;
size_t zero_bytes;
- bool cancel;
} stat;
int64_t speed;
VmaWriter *vmaw;
ProxmoxBackupHandle *pbs;
GList *di_list;
- CoMutex backup_mutex;
+ QemuRecMutex backup_mutex;
} backup_state;
static void pvebackup_init(void)
{
- qemu_co_rwlock_init(&backup_state.stat.rwlock);
- qemu_co_mutex_init(&backup_state.backup_mutex);
+ qemu_rec_mutex_init(&backup_state.stat.lock);
+ qemu_rec_mutex_init(&backup_state.backup_mutex);
}
// initialize PVEBackupState at startup
@@ -52,10 +50,54 @@ typedef struct PVEBackupDevInfo {
BlockDriverState *target;
} PVEBackupDevInfo;
-static void pvebackup_co_run_next_job(void);
+static void pvebackup_run_next_job(void);
+
+static BlockJob *
+lookup_active_block_job(PVEBackupDevInfo *di)
+{
+ if (!di->completed && di->bs) {
+ for (BlockJob *job = block_job_next(NULL); job; job = block_job_next(job)) {
+ if (job->job.driver->job_type != JOB_TYPE_BACKUP) {
+ continue;
+ }
+
+ BackupBlockJob *bjob = container_of(job, BackupBlockJob, common);
+ if (bjob && bjob->source_bs == di->bs) {
+ return job;
+ }
+ }
+ }
+ return NULL;
+}
+
+static void pvebackup_propagate_error(Error *err)
+{
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
+ error_propagate(&backup_state.stat.error, err);
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
+}
+
+static bool pvebackup_error_or_canceled(void)
+{
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
+ bool error_or_canceled = !!backup_state.stat.error;
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
+
+ return error_or_canceled;
+}
+static void pvebackup_add_transfered_bytes(size_t transferred, size_t zero_bytes)
+{
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
+ backup_state.stat.zero_bytes += zero_bytes;
+ backup_state.stat.transferred += transferred;
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
+}
+
+// This may get called from multiple coroutines in multiple io-threads
+// Note1: this may get called after job_cancel()
static int coroutine_fn
-pvebackup_co_dump_cb(
+pvebackup_co_dump_pbs_cb(
void *opaque,
uint64_t start,
uint64_t bytes,
@@ -67,137 +109,129 @@ pvebackup_co_dump_cb(
const unsigned char *buf = pbuf;
PVEBackupDevInfo *di = opaque;
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool cancel = backup_state.stat.cancel;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ assert(backup_state.pbs);
+
+ Error *local_err = NULL;
+ int pbs_res = -1;
+
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
- if (cancel) {
- return size; // return success
+ // avoid deadlock if job is cancelled
+ if (pvebackup_error_or_canceled()) {
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ return -1;
}
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, buf, start, size, &local_err);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- int ret = -1;
+ if (pbs_res < 0) {
+ pvebackup_propagate_error(local_err);
+ return pbs_res;
+ } else {
+ pvebackup_add_transfered_bytes(size, !buf ? size : 0);
+ }
- if (backup_state.vmaw) {
- size_t zero_bytes = 0;
- uint64_t remaining = size;
-
- uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
- if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- if (!backup_state.stat.error) {
- qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
- error_setg(&backup_state.stat.error,
- "got unaligned write inside backup dump "
- "callback (sector %ld)", start);
- }
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return -1; // not aligned to cluster size
- }
+ return size;
+}
- while (remaining > 0) {
- ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num,
- buf, &zero_bytes);
- ++cluster_num;
- if (buf) {
- buf += VMA_CLUSTER_SIZE;
- }
- if (ret < 0) {
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- if (!backup_state.stat.error) {
- qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
- vma_writer_error_propagate(backup_state.vmaw, &backup_state.stat.error);
- }
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+// This may get called from multiple coroutines in multiple io-threads
+static int coroutine_fn
+pvebackup_co_dump_vma_cb(
+ void *opaque,
+ uint64_t start,
+ uint64_t bytes,
+ const void *pbuf)
+{
+ assert(qemu_in_coroutine());
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return ret;
- } else {
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- backup_state.stat.zero_bytes += zero_bytes;
- if (remaining >= VMA_CLUSTER_SIZE) {
- backup_state.stat.transferred += VMA_CLUSTER_SIZE;
- remaining -= VMA_CLUSTER_SIZE;
- } else {
- backup_state.stat.transferred += remaining;
- remaining = 0;
- }
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- }
- }
- } else if (backup_state.pbs) {
- Error *local_err = NULL;
- int pbs_res = -1;
+ const uint64_t size = bytes;
+ const unsigned char *buf = pbuf;
+ PVEBackupDevInfo *di = opaque;
- pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, buf, start, size, &local_err);
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ int ret = -1;
- if (pbs_res < 0) {
- error_propagate(&backup_state.stat.error, local_err);
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return pbs_res;
- } else {
- if (!buf) {
- backup_state.stat.zero_bytes += size;
- }
- backup_state.stat.transferred += size;
+ assert(backup_state.vmaw);
+
+ uint64_t remaining = size;
+
+ uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
+ if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
+ Error *local_err = NULL;
+ error_setg(&local_err,
+ "got unaligned write inside backup dump "
+ "callback (sector %ld)", start);
+ pvebackup_propagate_error(local_err);
+ return -1; // not aligned to cluster size
+ }
+
+ while (remaining > 0) {
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ // avoid deadlock if job is cancelled
+ if (pvebackup_error_or_canceled()) {
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ return -1;
}
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ size_t zero_bytes = 0;
+ ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num, buf, &zero_bytes);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- } else {
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- if (!buf) {
- backup_state.stat.zero_bytes += size;
+ ++cluster_num;
+ if (buf) {
+ buf += VMA_CLUSTER_SIZE;
+ }
+ if (ret < 0) {
+ Error *local_err = NULL;
+ vma_writer_error_propagate(backup_state.vmaw, &local_err);
+ pvebackup_propagate_error(local_err);
+ return ret;
+ } else {
+ if (remaining >= VMA_CLUSTER_SIZE) {
+ assert(ret == VMA_CLUSTER_SIZE);
+ pvebackup_add_transfered_bytes(VMA_CLUSTER_SIZE, zero_bytes);
+ remaining -= VMA_CLUSTER_SIZE;
+ } else {
+ assert(ret == remaining);
+ pvebackup_add_transfered_bytes(remaining, zero_bytes);
+ remaining = 0;
+ }
}
- backup_state.stat.transferred += size;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
}
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
-
return size;
}
-static void coroutine_fn pvebackup_co_cleanup(void)
+static void coroutine_fn pvebackup_co_cleanup(void *unused)
{
assert(qemu_in_coroutine());
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
backup_state.stat.end_time = time(NULL);
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
if (backup_state.vmaw) {
Error *local_err = NULL;
vma_writer_close(backup_state.vmaw, &local_err);
if (local_err != NULL) {
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- error_propagate(&backup_state.stat.error, local_err);
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- }
+ pvebackup_propagate_error(local_err);
+ }
backup_state.vmaw = NULL;
}
if (backup_state.pbs) {
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool error_or_canceled = backup_state.stat.error || backup_state.stat.cancel;
- if (!error_or_canceled) {
+ if (!pvebackup_error_or_canceled()) {
Error *local_err = NULL;
proxmox_backup_co_finish(backup_state.pbs, &local_err);
if (local_err != NULL) {
- qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
- error_propagate(&backup_state.stat.error, local_err);
- }
+ pvebackup_propagate_error(local_err);
+ }
}
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
proxmox_backup_disconnect(backup_state.pbs);
backup_state.pbs = NULL;
@@ -205,43 +239,14 @@ static void coroutine_fn pvebackup_co_cleanup(void)
g_list_free(backup_state.di_list);
backup_state.di_list = NULL;
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
}
-typedef struct PVEBackupCompeteCallbackData {
- PVEBackupDevInfo *di;
- int result;
-} PVEBackupCompeteCallbackData;
-
-static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
+static void coroutine_fn pvebackup_complete_stream(void *opaque)
{
- assert(qemu_in_coroutine());
-
- PVEBackupCompeteCallbackData *cb_data = opaque;
-
- qemu_co_mutex_lock(&backup_state.backup_mutex);
-
- PVEBackupDevInfo *di = cb_data->di;
- int ret = cb_data->result;
-
- di->completed = true;
-
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool error_or_canceled = backup_state.stat.error || backup_state.stat.cancel;
-
- if (ret < 0 && !backup_state.stat.error) {
- qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
- error_setg(&backup_state.stat.error, "job failed with err %d - %s",
- ret, strerror(-ret));
- }
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
-
- di->bs = NULL;
+ PVEBackupDevInfo *di = opaque;
- if (di->target) {
- bdrv_unref(di->target);
- di->target = NULL;
- }
+ bool error_or_canceled = pvebackup_error_or_canceled();
if (backup_state.vmaw) {
vma_writer_close_stream(backup_state.vmaw, di->dev_id);
@@ -251,108 +256,96 @@ static void coroutine_fn pvebackup_co_complete_cb(void *opaque)
Error *local_err = NULL;
proxmox_backup_co_close_image(backup_state.pbs, di->dev_id, &local_err);
if (local_err != NULL) {
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- error_propagate(&backup_state.stat.error, local_err);
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ pvebackup_propagate_error(local_err);
}
}
+}
- // remove self from job queue
- backup_state.di_list = g_list_remove(backup_state.di_list, di);
- g_free(di);
+static void pvebackup_complete_cb(void *opaque, int ret)
+{
+ assert(!qemu_in_coroutine());
+
+ PVEBackupDevInfo *di = opaque;
- int pending_jobs = g_list_length(backup_state.di_list);
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ di->completed = true;
- if (pending_jobs > 0) {
- pvebackup_co_run_next_job();
- } else {
- pvebackup_co_cleanup();
+ if (ret < 0) {
+ Error *local_err = NULL;
+ error_setg(&local_err, "job failed with err %d - %s", ret, strerror(-ret));
+ pvebackup_propagate_error(local_err);
}
-}
-static void pvebackup_complete_cb(void *opaque, int ret)
-{
- // This can be called from the main loop, or from a coroutine
- PVEBackupCompeteCallbackData cb_data = {
- .di = opaque,
- .result = ret,
- };
+ di->bs = NULL;
- if (qemu_in_coroutine()) {
- pvebackup_co_complete_cb(&cb_data);
- } else {
- block_on_coroutine_fn(pvebackup_co_complete_cb, &cb_data);
- }
-}
+ assert(di->target == NULL);
-static void coroutine_fn pvebackup_co_cancel(void *opaque)
-{
- assert(qemu_in_coroutine());
+ block_on_coroutine_fn(pvebackup_complete_stream, di);
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- backup_state.stat.cancel = true;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ // remove self from job queue
+ backup_state.di_list = g_list_remove(backup_state.di_list, di);
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ g_free(di);
- // Avoid race between block jobs and backup-cancel command:
- if (!(backup_state.vmaw || backup_state.pbs)) {
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return;
- }
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- if (!backup_state.stat.error) {
- qemu_co_rwlock_upgrade(&backup_state.stat.rwlock);
- error_setg(&backup_state.stat.error, "backup cancelled");
- }
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ pvebackup_run_next_job();
+}
+
+static void pvebackup_cancel(void)
+{
+ Error *cancel_err = NULL;
+ error_setg(&cancel_err, "backup canceled");
+ pvebackup_propagate_error(cancel_err);
+
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
if (backup_state.vmaw) {
/* make sure vma writer does not block anymore */
- vma_writer_set_error(backup_state.vmaw, "backup cancelled");
+ vma_writer_set_error(backup_state.vmaw, "backup canceled");
}
if (backup_state.pbs) {
- proxmox_backup_abort(backup_state.pbs, "backup cancelled");
+ proxmox_backup_abort(backup_state.pbs, "backup canceled");
}
- bool running_jobs = 0;
- GList *l = backup_state.di_list;
- while (l) {
- PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
- l = g_list_next(l);
- if (!di->completed && di->bs) {
- for (BlockJob *job = block_job_next(NULL); job; job = block_job_next(job)) {
- if (job->job.driver->job_type != JOB_TYPE_BACKUP) {
- continue;
- }
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- BackupBlockJob *bjob = container_of(job, BackupBlockJob, common);
- if (bjob && bjob->source_bs == di->bs) {
- AioContext *aio_context = job->job.aio_context;
- aio_context_acquire(aio_context);
+ for(;;) {
- if (!di->completed) {
- running_jobs += 1;
- job_cancel(&job->job, false);
- }
- aio_context_release(aio_context);
- }
+ BlockJob *next_job = NULL;
+
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
+
+ GList *l = backup_state.di_list;
+ while (l) {
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
+
+ BlockJob *job = lookup_active_block_job(di);
+ if (job != NULL) {
+ next_job = job;
+ break;
}
}
- }
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- if (running_jobs == 0) pvebackup_co_cleanup(); // else job will call completion handler
+ if (next_job) {
+ AioContext *aio_context = next_job->job.aio_context;
+ aio_context_acquire(aio_context);
+ job_cancel_sync(&next_job->job);
+ aio_context_release(aio_context);
+ } else {
+ break;
+ }
+ }
}
void qmp_backup_cancel(Error **errp)
{
- block_on_coroutine_fn(pvebackup_co_cancel, NULL);
+ pvebackup_cancel();
}
static int coroutine_fn pvebackup_co_add_config(
@@ -406,46 +399,97 @@ static int coroutine_fn pvebackup_co_add_config(
bool job_should_pause(Job *job);
-static void coroutine_fn pvebackup_co_run_next_job(void)
+static void pvebackup_run_next_job(void)
{
- assert(qemu_in_coroutine());
+ assert(!qemu_in_coroutine());
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
GList *l = backup_state.di_list;
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (!di->completed && di->bs) {
- for (BlockJob *job = block_job_next(NULL); job; job = block_job_next(job)) {
- if (job->job.driver->job_type != JOB_TYPE_BACKUP) {
- continue;
- }
- BackupBlockJob *bjob = container_of(job, BackupBlockJob, common);
- if (bjob && bjob->source_bs == di->bs) {
- AioContext *aio_context = job->job.aio_context;
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
- aio_context_acquire(aio_context);
-
- if (job_should_pause(&job->job)) {
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool error_or_canceled = backup_state.stat.error || backup_state.stat.cancel;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
-
- if (error_or_canceled) {
- job_cancel(&job->job, false);
- } else {
- job_resume(&job->job);
- }
- }
- aio_context_release(aio_context);
- return;
+ BlockJob *job = lookup_active_block_job(di);
+
+ if (job) {
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+
+ AioContext *aio_context = job->job.aio_context;
+ aio_context_acquire(aio_context);
+
+ if (job_should_pause(&job->job)) {
+ bool error_or_canceled = pvebackup_error_or_canceled();
+ if (error_or_canceled) {
+ job_cancel_sync(&job->job);
+ } else {
+ job_resume(&job->job);
}
}
+ aio_context_release(aio_context);
+ return;
}
}
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+
+ block_on_coroutine_fn(pvebackup_co_cleanup, NULL); // no more jobs, run cleanup
+}
+
+static bool create_backup_jobs(void) {
+
+ assert(!qemu_in_coroutine());
+
+ Error *local_err = NULL;
+
+ /* create and start all jobs (paused state) */
+ GList *l = backup_state.di_list;
+ while (l) {
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
+
+ assert(di->target != NULL);
+
+ AioContext *aio_context = bdrv_get_aio_context(di->bs);
+ aio_context_acquire(aio_context);
+
+ BlockJob *job = backup_job_create(
+ NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
+ BITMAP_SYNC_MODE_NEVER, false, NULL, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
+ JOB_DEFAULT, pvebackup_complete_cb, di, 1, NULL, &local_err);
+
+ aio_context_release(aio_context);
+
+ if (!job || local_err != NULL) {
+ Error *create_job_err = NULL;
+ error_setg(&create_job_err, "backup_job_create failed: %s",
+ local_err ? error_get_pretty(local_err) : "null");
+
+ pvebackup_propagate_error(create_job_err);
+ break;
+ }
+ job_start(&job->job);
+
+ bdrv_unref(di->target);
+ di->target = NULL;
+ }
+
+ bool errors = pvebackup_error_or_canceled();
+
+ if (errors) {
+ l = backup_state.di_list;
+ while (l) {
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
+
+ if (di->target) {
+ bdrv_unref(di->target);
+ di->target = NULL;
+ }
+ }
+ }
+
+ return errors;
}
typedef struct QmpBackupTask {
@@ -476,7 +520,7 @@ typedef struct QmpBackupTask {
UuidInfo *result;
} QmpBackupTask;
-static void coroutine_fn pvebackup_co_start(void *opaque)
+static void coroutine_fn pvebackup_co_prepare(void *opaque)
{
assert(qemu_in_coroutine());
@@ -495,15 +539,14 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
GList *di_list = NULL;
GList *l;
UuidInfo *uuid_info;
- BlockJob *job;
const char *config_name = "qemu-server.conf";
const char *firewall_name = "qemu-server.fw";
- qemu_co_mutex_lock(&backup_state.backup_mutex);
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
if (backup_state.di_list) {
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
return;
@@ -631,7 +674,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
if (dev_id < 0)
goto err;
- if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_cb, di, task->errp))) {
+ if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, task->errp))) {
goto err;
}
@@ -652,7 +695,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_cb, di, task->errp))) {
+ if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, task->errp))) {
goto err;
}
@@ -717,9 +760,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
}
/* initialize global backup_state now */
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
-
- backup_state.stat.cancel = false;
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
if (backup_state.stat.error) {
error_free(backup_state.stat.error);
@@ -742,7 +783,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
backup_state.stat.transferred = 0;
backup_state.stat.zero_bytes = 0;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
@@ -751,45 +792,7 @@ static void coroutine_fn pvebackup_co_start(void *opaque)
backup_state.di_list = di_list;
- /* start all jobs (paused state) */
- l = di_list;
- while (l) {
- PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
- l = g_list_next(l);
-
- AioContext *aio_context = bdrv_get_aio_context(di->bs);
- aio_context_acquire(aio_context);
-
- job = backup_job_create(NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
- BITMAP_SYNC_MODE_NEVER, false, NULL, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
- JOB_DEFAULT, pvebackup_complete_cb, di, 1, NULL, &local_err);
-
- aio_context_release(aio_context);
-
- if (!job || local_err != NULL) {
- qemu_co_rwlock_wrlock(&backup_state.stat.rwlock);
- error_setg(&backup_state.stat.error, "backup_job_create failed");
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- break;
- }
- job_start(&job->job);
- if (di->target) {
- bdrv_unref(di->target);
- di->target = NULL;
- }
- }
-
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
-
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
- bool no_errors = !backup_state.stat.error;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
-
- if (no_errors) {
- pvebackup_co_run_next_job(); // run one job
- } else {
- pvebackup_co_cancel(NULL);
- }
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
uuid_info = g_malloc0(sizeof(*uuid_info));
uuid_info->UUID = uuid_str;
@@ -833,7 +836,7 @@ err:
rmdir(backup_dir);
}
- qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
task->result = NULL;
return;
@@ -878,32 +881,28 @@ UuidInfo *qmp_backup(
.errp = errp,
};
- block_on_coroutine_fn(pvebackup_co_start, &task);
+ block_on_coroutine_fn(pvebackup_co_prepare, &task);
+
+ if (*errp == NULL) {
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ create_backup_jobs();
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ pvebackup_run_next_job();
+ }
return task.result;
}
-
-typedef struct QmpQueryBackupTask {
- Error **errp;
- BackupStatus *result;
-} QmpQueryBackupTask;
-
-static void coroutine_fn pvebackup_co_query(void *opaque)
+BackupStatus *qmp_query_backup(Error **errp)
{
- assert(qemu_in_coroutine());
-
- QmpQueryBackupTask *task = opaque;
-
BackupStatus *info = g_malloc0(sizeof(*info));
- qemu_co_rwlock_rdlock(&backup_state.stat.rwlock);
+ qemu_rec_mutex_lock(&backup_state.stat.lock);
if (!backup_state.stat.start_time) {
/* not started, return {} */
- task->result = info;
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
- return;
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ return info;
}
info->has_status = true;
@@ -939,19 +938,7 @@ static void coroutine_fn pvebackup_co_query(void *opaque)
info->has_transferred = true;
info->transferred = backup_state.stat.transferred;
- task->result = info;
+ qemu_rec_mutex_unlock(&backup_state.stat.lock);
- qemu_co_rwlock_unlock(&backup_state.stat.rwlock);
-}
-
-BackupStatus *qmp_query_backup(Error **errp)
-{
- QmpQueryBackupTask task = {
- .errp = errp,
- .result = NULL,
- };
-
- block_on_coroutine_fn(pvebackup_co_query, &task);
-
- return task.result;
+ return info;
}

View File

@@ -0,0 +1,61 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Tue, 12 Jan 2021 14:12:20 +0100
Subject: [PATCH] PVE: redirect stderr to journal when daemonized
QEMU uses the logging for error messages usually, so LOG_ERR is most
fitting.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
meson.build | 2 ++
os-posix.c | 7 +++++--
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index cbfc9a43fb..8206270272 100644
--- a/meson.build
+++ b/meson.build
@@ -1779,6 +1779,7 @@ endif
has_gettid = cc.has_function('gettid')
libuuid = cc.find_library('uuid', required: true)
+libsystemd = cc.find_library('systemd', required: true)
libproxmox_backup_qemu = cc.find_library('proxmox_backup_qemu', required: true)
# libselinux
@@ -3406,6 +3407,7 @@ if have_block
# os-posix.c contains POSIX-specific functions used by qemu-storage-daemon,
# os-win32.c does not
blockdev_ss.add(when: 'CONFIG_POSIX', if_true: files('os-posix.c'))
+ blockdev_ss.add(when: 'CONFIG_POSIX', if_true: libsystemd)
system_ss.add(when: 'CONFIG_WIN32', if_true: [files('os-win32.c')])
endif
diff --git a/os-posix.c b/os-posix.c
index cfcb96533c..fb2ad87009 100644
--- a/os-posix.c
+++ b/os-posix.c
@@ -28,6 +28,8 @@
#include <pwd.h>
#include <grp.h>
#include <libgen.h>
+#include <systemd/sd-journal.h>
+#include <syslog.h>
/* Needed early for CONFIG_BSD etc. */
#include "net/slirp.h"
@@ -310,9 +312,10 @@ void os_setup_post(void)
dup2(fd, 0);
dup2(fd, 1);
- /* In case -D is given do not redirect stderr to /dev/null */
+ /* In case -D is given do not redirect stderr to journal */
if (!qemu_log_enabled()) {
- dup2(fd, 2);
+ int journal_fd = sd_journal_stream_fd("QEMU", LOG_ERR, 0);
+ dup2(journal_fd, 2);
}
close(fd);

View File

@@ -0,0 +1,211 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Thu, 22 Oct 2020 17:34:18 +0200
Subject: [PATCH] PVE: Migrate dirty bitmap state via savevm
QEMU provides 'savevm' registrations as a mechanism for arbitrary state
to be migrated along with a VM. Use this to send a serialized version of
dirty bitmap state data from proxmox-backup-qemu, and restore it on the
target node.
Also add a flag to query-proxmox-support so qemu-server can determine if
safe migration is possible and makes sense.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: split up state_pending for 8.0]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
include/migration/misc.h | 3 ++
migration/meson.build | 2 +
migration/migration.c | 1 +
migration/pbs-state.c | 104 +++++++++++++++++++++++++++++++++++++++
pve-backup.c | 1 +
qapi/block-core.json | 6 +++
6 files changed, 117 insertions(+)
create mode 100644 migration/pbs-state.c
diff --git a/include/migration/misc.h b/include/migration/misc.h
index 7dcc0b5c2c..4c940b2475 100644
--- a/include/migration/misc.h
+++ b/include/migration/misc.h
@@ -77,4 +77,7 @@ bool migration_in_bg_snapshot(void);
/* migration/block-dirty-bitmap.c */
void dirty_bitmap_mig_init(void);
+/* migration/pbs-state.c */
+void pbs_state_mig_init(void);
+
#endif
diff --git a/migration/meson.build b/migration/meson.build
index 07f6057acc..343994d891 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -7,7 +7,9 @@ migration_files = files(
'vmstate.c',
'qemu-file.c',
'yank_functions.c',
+ 'pbs-state.c',
)
+system_ss.add(libproxmox_backup_qemu)
system_ss.add(files(
'block-dirty-bitmap.c',
diff --git a/migration/migration.c b/migration/migration.c
index 7a4c8beb5d..0a955a2a18 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -162,6 +162,7 @@ void migration_object_init(void)
blk_mig_init();
ram_mig_init();
dirty_bitmap_mig_init();
+ pbs_state_mig_init();
}
void migration_cancel(const Error *error)
diff --git a/migration/pbs-state.c b/migration/pbs-state.c
new file mode 100644
index 0000000000..887e998b9e
--- /dev/null
+++ b/migration/pbs-state.c
@@ -0,0 +1,104 @@
+/*
+ * PBS (dirty-bitmap) state migration
+ */
+
+#include "qemu/osdep.h"
+#include "migration/misc.h"
+#include "qemu-file.h"
+#include "migration/vmstate.h"
+#include "migration/register.h"
+#include "proxmox-backup-qemu.h"
+
+typedef struct PBSState {
+ bool active;
+} PBSState;
+
+/* state is accessed via this static variable directly, 'opaque' is NULL */
+static PBSState pbs_state;
+
+static void pbs_state_pending(void *opaque, uint64_t *must_precopy,
+ uint64_t *can_postcopy)
+{
+ /* we send everything in save_setup, so nothing is ever pending */
+}
+
+/* receive PBS state via f and deserialize, called on target */
+static int pbs_state_load(QEMUFile *f, void *opaque, int version_id)
+{
+ /* safe cast, we cannot migrate to target with less bits than source */
+ size_t buf_size = (size_t)qemu_get_be64(f);
+
+ uint8_t *buf = (uint8_t *)malloc(buf_size);
+ size_t read = qemu_get_buffer(f, buf, buf_size);
+
+ if (read < buf_size) {
+ fprintf(stderr, "error receiving PBS state: not enough data\n");
+ return -EIO;
+ }
+
+ proxmox_import_state(buf, buf_size);
+
+ free(buf);
+ return 0;
+}
+
+/* serialize PBS state and send to target via f, called on source */
+static int pbs_state_save_setup(QEMUFile *f, void *opaque)
+{
+ size_t buf_size;
+ uint8_t *buf = proxmox_export_state(&buf_size);
+
+ /* LV encoding */
+ qemu_put_be64(f, buf_size);
+ qemu_put_buffer(f, buf, buf_size);
+
+ proxmox_free_state_buf(buf);
+ pbs_state.active = false;
+ return 0;
+}
+
+static bool pbs_state_is_active(void *opaque)
+{
+ /* we need to return active exactly once, else .save_setup is never called,
+ * but if we'd just return true the migration doesn't make progress since
+ * it'd be waiting for us */
+ return pbs_state.active;
+}
+
+static bool pbs_state_is_active_iterate(void *opaque)
+{
+ /* we don't iterate, everything is sent in save_setup */
+ return pbs_state_is_active(opaque);
+}
+
+static bool pbs_state_has_postcopy(void *opaque)
+{
+ /* PBS state can't change during a migration (since that's blocking any
+ * potential backups), so we can copy everything before the VM is stopped */
+ return false;
+}
+
+static void pbs_state_save_cleanup(void *opaque)
+{
+ /* reset active after migration succeeds or fails */
+ pbs_state.active = false;
+}
+
+static SaveVMHandlers savevm_pbs_state_handlers = {
+ .save_setup = pbs_state_save_setup,
+ .has_postcopy = pbs_state_has_postcopy,
+ .state_pending_exact = pbs_state_pending,
+ .state_pending_estimate = pbs_state_pending,
+ .is_active_iterate = pbs_state_is_active_iterate,
+ .load_state = pbs_state_load,
+ .is_active = pbs_state_is_active,
+ .save_cleanup = pbs_state_save_cleanup,
+};
+
+void pbs_state_mig_init(void)
+{
+ pbs_state.active = true;
+ register_savevm_live("pbs-state", 0, 1,
+ &savevm_pbs_state_handlers,
+ NULL);
+}
diff --git a/pve-backup.c b/pve-backup.c
index d84d807654..9c8b88d075 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -1060,6 +1060,7 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
ret->pbs_library_version = g_strdup(proxmox_backup_qemu_version());
ret->pbs_dirty_bitmap = true;
ret->pbs_dirty_bitmap_savevm = true;
+ ret->pbs_dirty_bitmap_migration = true;
ret->query_bitmap_info = true;
ret->pbs_masterkey = true;
ret->backup_max_workers = true;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index d67a6d448a..09de550c95 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -991,6 +991,11 @@
# @pbs-dirty-bitmap-savevm: True if 'dirty-bitmaps' migration capability can
# safely be set for savevm-async.
#
+# @pbs-dirty-bitmap-migration: True if safe migration of dirty-bitmaps including
+# PBS state is supported. Enabling 'dirty-bitmaps'
+# migration cap if this is false/unset may lead
+# to crashes on migration!
+#
# @pbs-masterkey: True if the QMP backup call supports the 'master_keyfile'
# parameter.
#
@@ -1001,6 +1006,7 @@
'data': { 'pbs-dirty-bitmap': 'bool',
'query-bitmap-info': 'bool',
'pbs-dirty-bitmap-savevm': 'bool',
+ 'pbs-dirty-bitmap-migration': 'bool',
'pbs-masterkey': 'bool',
'pbs-library-version': 'str',
'backup-max-workers': 'bool' } }

View File

@@ -0,0 +1,33 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Tue, 3 Nov 2020 14:57:32 +0100
Subject: [PATCH] migration/block-dirty-bitmap: migrate other bitmaps even if
one fails
If the checks in bdrv_dirty_bitmap_check fail, that only means that this
one specific bitmap cannot be migrated. That is not an error condition
for any other bitmaps on the same block device.
Fixes dirty-bitmap migration with sync=bitmap, as the bitmaps used for
that are obviously marked as "busy", which would cause none at all to be
transferred.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
migration/block-dirty-bitmap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index e1ae3b7316..285dd1d148 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -540,7 +540,7 @@ static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs,
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, &local_err)) {
error_report_err(local_err);
- return -1;
+ continue;
}
if (bitmap_aliases) {

View File

@@ -0,0 +1,69 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Tue, 17 Nov 2020 10:51:05 +0100
Subject: [PATCH] PVE: fall back to open-iscsi initiatorname
When no explicit option is given, try reading the initiator name from
/etc/iscsi/initiatorname.iscsi and only use the generic fallback, i.e.
iqn.2008-11.org.linux-kvmXXX, as a third alternative.
This avoids the need to add an explicit option for vma and to explicitly set it
for each call to qemu that deals with iSCSI disks, while still allowing to set
the option if a different name is needed.
According to RFC 3720, an initiator name is at most 223 bytes long, so the
4 KiB buffer is big enough, even if many whitespaces are used.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/iscsi.c | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/block/iscsi.c b/block/iscsi.c
index 34f97ab646..398782963d 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -1391,12 +1391,42 @@ static char *get_initiator_name(QemuOpts *opts)
const char *name;
char *iscsi_name;
UuidInfo *uuid_info;
+ FILE *name_fh;
name = qemu_opt_get(opts, "initiator-name");
if (name) {
return g_strdup(name);
}
+ name_fh = fopen("/etc/iscsi/initiatorname.iscsi", "r");
+ if (name_fh) {
+ const char *key = "InitiatorName";
+ char buffer[4096];
+ char *line;
+
+ while ((line = fgets(buffer, sizeof(buffer), name_fh))) {
+ line = g_strstrip(line);
+ if (!strncmp(line, key, strlen(key))) {
+ line = strchr(line, '=');
+ if (!line || strlen(line) == 1) {
+ continue;
+ }
+ line++;
+ g_strstrip(line);
+ if (!strlen(line)) {
+ continue;
+ }
+ name = line;
+ break;
+ }
+ }
+ fclose(name_fh);
+
+ if (name) {
+ return g_strdup(name);
+ }
+ }
+
uuid_info = qmp_query_uuid(NULL);
if (strcmp(uuid_info->UUID, UUID_NONE) == 0) {
name = qemu_get_vm_name();

View File

@@ -0,0 +1,25 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Tue, 2 Mar 2021 16:34:28 +0100
Subject: [PATCH] PVE: block/stream: increase chunk size
Ceph favors bigger chunks, so increase to 4M.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/stream.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/stream.c b/block/stream.c
index e522bbdec5..afed72db55 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -27,7 +27,7 @@ enum {
* large enough to process multiple clusters in a single call, so
* that populating contiguous regions of the image is efficient.
*/
- STREAM_CHUNK = 512 * 1024, /* in bytes */
+ STREAM_CHUNK = 4 * 1024 * 1024, /* in bytes */
};
typedef struct StreamBlockJob {

View File

@@ -0,0 +1,33 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Tue, 2 Mar 2021 16:11:54 +0100
Subject: [PATCH] block/io: accept NULL qiov in bdrv_pad_request
Some operations, e.g. block-stream, perform reads while discarding the
results (only copy-on-read matters). In this case they will pass NULL as
the target QEMUIOVector, which will however trip bdrv_pad_request, since
it wants to extend its passed vector.
Simply check for NULL and do nothing, there's no reason to pad the
target if it will be discarded anyway.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
block/io.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/block/io.c b/block/io.c
index 83d1b1dfdc..24a3c84c93 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1710,6 +1710,10 @@ static int bdrv_pad_request(BlockDriverState *bs,
int sliced_niov;
size_t sliced_head, sliced_tail;
+ if (!qiov) {
+ return 0;
+ }
+
/* Should have been checked by the caller already */
ret = bdrv_check_request32(*offset, *bytes, *qiov, *qiov_offset);
if (ret < 0) {

View File

@@ -0,0 +1,406 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Mon, 7 Dec 2020 15:21:03 +0100
Subject: [PATCH] block: add alloc-track driver
Add a new filter node 'alloc-track', which seperates reads and writes to
different children, thus allowing to put a backing image behind any
blockdev (regardless of driver support). Since we can't detect any
pre-allocated blocks, we can only track new writes, hence the write
target ('file') for this node must always be empty.
Intended use case is for live restoring, i.e. add a backup image as a
block device into a VM, then put an alloc-track on the restore target
and set the backup as backing. With this, one can use a regular
'block-stream' to restore the image, while the VM can already run in the
background. Copy-on-read will help make progress as the VM reads as
well.
This only worked if the target supports backing images, so up until now
only for qcow2, with alloc-track any driver for the target can be used.
If 'auto-remove' is set, alloc-track will automatically detach itself
once the backing image is removed. It will be replaced by 'file'.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: adapt to changed function signatures
make error return value consistent with QEMU
avoid premature break during read]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/alloc-track.c | 352 ++++++++++++++++++++++++++++++++++++++++++++
block/meson.build | 1 +
2 files changed, 353 insertions(+)
create mode 100644 block/alloc-track.c
diff --git a/block/alloc-track.c b/block/alloc-track.c
new file mode 100644
index 0000000000..b75d7c6460
--- /dev/null
+++ b/block/alloc-track.c
@@ -0,0 +1,352 @@
+/*
+ * Node to allow backing images to be applied to any node. Assumes a blank
+ * image to begin with, only new writes are tracked as allocated, thus this
+ * must never be put on a node that already contains data.
+ *
+ * Copyright (c) 2020 Proxmox Server Solutions GmbH
+ * Copyright (c) 2020 Stefan Reiter <s.reiter@proxmox.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "block/block_int.h"
+#include "block/dirty-bitmap.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "qemu/cutils.h"
+#include "qemu/option.h"
+#include "qemu/module.h"
+#include "sysemu/block-backend.h"
+
+#define TRACK_OPT_AUTO_REMOVE "auto-remove"
+
+typedef enum DropState {
+ DropNone,
+ DropRequested,
+ DropInProgress,
+} DropState;
+
+typedef struct {
+ BdrvDirtyBitmap *bitmap;
+ DropState drop_state;
+ bool auto_remove;
+} BDRVAllocTrackState;
+
+static QemuOptsList runtime_opts = {
+ .name = "alloc-track",
+ .head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head),
+ .desc = {
+ {
+ .name = TRACK_OPT_AUTO_REMOVE,
+ .type = QEMU_OPT_BOOL,
+ .help = "automatically replace this node with 'file' when 'backing'"
+ "is detached",
+ },
+ { /* end of list */ }
+ },
+};
+
+static void track_refresh_limits(BlockDriverState *bs, Error **errp)
+{
+ BlockDriverInfo bdi;
+
+ if (!bs->file) {
+ return;
+ }
+
+ /* always use alignment from underlying write device so RMW cycle for
+ * bdrv_pwritev reads data from our backing via track_co_preadv (no partial
+ * cluster allocation in 'file') */
+ bdrv_get_info(bs->file->bs, &bdi);
+ bs->bl.request_alignment = MAX(bs->file->bs->bl.request_alignment,
+ MAX(bdi.cluster_size, BDRV_SECTOR_SIZE));
+}
+
+static int track_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+ QemuOpts *opts;
+ Error *local_err = NULL;
+ int ret = 0;
+
+ opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
+ qemu_opts_absorb_qdict(opts, options, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ ret = -EINVAL;
+ goto fail;
+ }
+
+ s->auto_remove = qemu_opt_get_bool(opts, TRACK_OPT_AUTO_REMOVE, false);
+
+ /* open the target (write) node, backing will be attached by block layer */
+ bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+ BDRV_CHILD_DATA | BDRV_CHILD_METADATA, false,
+ &local_err);
+ if (local_err) {
+ ret = -EINVAL;
+ error_propagate(errp, local_err);
+ goto fail;
+ }
+
+ track_refresh_limits(bs, errp);
+ uint64_t gran = bs->bl.request_alignment;
+ s->bitmap = bdrv_create_dirty_bitmap(bs->file->bs, gran, NULL, &local_err);
+ if (local_err) {
+ ret = -EIO;
+ error_propagate(errp, local_err);
+ goto fail;
+ }
+
+ s->drop_state = DropNone;
+
+fail:
+ if (ret < 0) {
+ bdrv_unref_child(bs, bs->file);
+ if (s->bitmap) {
+ bdrv_release_dirty_bitmap(s->bitmap);
+ }
+ }
+ qemu_opts_del(opts);
+ return ret;
+}
+
+static void track_close(BlockDriverState *bs)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+ if (s->bitmap) {
+ bdrv_release_dirty_bitmap(s->bitmap);
+ }
+}
+
+static coroutine_fn int64_t track_co_getlength(BlockDriverState *bs)
+{
+ return bdrv_co_getlength(bs->file->bs);
+}
+
+static int coroutine_fn track_co_preadv(BlockDriverState *bs,
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+ QEMUIOVector local_qiov;
+ int ret;
+
+ /* 'cur_offset' is relative to 'offset', 'local_offset' to image start */
+ uint64_t cur_offset, local_offset;
+ int64_t local_bytes;
+ bool alloc;
+
+ if (offset < 0 || bytes < 0) {
+ fprintf(stderr, "unexpected negative 'offset' or 'bytes' value!\n");
+ return -EIO;
+ }
+
+ /* a read request can span multiple granularity-sized chunks, and can thus
+ * contain blocks with different allocation status - we could just iterate
+ * granularity-wise, but for better performance use bdrv_dirty_bitmap_next_X
+ * to find the next flip and consider everything up to that in one go */
+ for (cur_offset = 0; cur_offset < bytes; cur_offset += local_bytes) {
+ local_offset = offset + cur_offset;
+ alloc = bdrv_dirty_bitmap_get(s->bitmap, local_offset);
+ if (alloc) {
+ local_bytes = bdrv_dirty_bitmap_next_zero(s->bitmap, local_offset,
+ bytes - cur_offset);
+ } else {
+ local_bytes = bdrv_dirty_bitmap_next_dirty(s->bitmap, local_offset,
+ bytes - cur_offset);
+ }
+
+ /* _bitmap_next_X return is -1 if no end found within limit, otherwise
+ * offset of next flip (to start of image) */
+ local_bytes = local_bytes < 0 ?
+ bytes - cur_offset :
+ local_bytes - local_offset;
+
+ qemu_iovec_init_slice(&local_qiov, qiov, cur_offset, local_bytes);
+
+ if (alloc) {
+ ret = bdrv_co_preadv(bs->file, local_offset, local_bytes,
+ &local_qiov, flags);
+ } else if (bs->backing) {
+ ret = bdrv_co_preadv(bs->backing, local_offset, local_bytes,
+ &local_qiov, flags);
+ } else {
+ qemu_iovec_memset(&local_qiov, cur_offset, 0, local_bytes);
+ ret = 0;
+ }
+
+ if (ret != 0) {
+ break;
+ }
+ }
+
+ return ret;
+}
+
+static int coroutine_fn track_co_pwritev(BlockDriverState *bs,
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+ return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags);
+}
+
+static int coroutine_fn track_co_pwrite_zeroes(BlockDriverState *bs,
+ int64_t offset, int64_t bytes, BdrvRequestFlags flags)
+{
+ return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags);
+}
+
+static int coroutine_fn track_co_pdiscard(BlockDriverState *bs,
+ int64_t offset, int64_t bytes)
+{
+ return bdrv_co_pdiscard(bs->file, offset, bytes);
+}
+
+static coroutine_fn int track_co_flush(BlockDriverState *bs)
+{
+ return bdrv_co_flush(bs->file->bs);
+}
+
+static int coroutine_fn track_co_block_status(BlockDriverState *bs,
+ bool want_zero,
+ int64_t offset,
+ int64_t bytes,
+ int64_t *pnum,
+ int64_t *map,
+ BlockDriverState **file)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+
+ bool alloc = bdrv_dirty_bitmap_get(s->bitmap, offset);
+ int64_t next_flipped;
+ if (alloc) {
+ next_flipped = bdrv_dirty_bitmap_next_zero(s->bitmap, offset, bytes);
+ } else {
+ next_flipped = bdrv_dirty_bitmap_next_dirty(s->bitmap, offset, bytes);
+ }
+
+ /* in case not the entire region has the same state, we need to set pnum to
+ * indicate for how many bytes our result is valid */
+ *pnum = next_flipped == -1 ? bytes : next_flipped - offset;
+ *map = offset;
+
+ if (alloc) {
+ *file = bs->file->bs;
+ return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID;
+ } else if (bs->backing) {
+ *file = bs->backing->bs;
+ }
+ return 0;
+}
+
+static void track_child_perm(BlockDriverState *bs, BdrvChild *c,
+ BdrvChildRole role, BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+
+ *nshared = BLK_PERM_ALL;
+
+ /* in case we're currently dropping ourselves, claim to not use any
+ * permissions at all - which is fine, since from this point on we will
+ * never issue a read or write anymore */
+ if (s->drop_state == DropInProgress) {
+ *nperm = 0;
+ return;
+ }
+
+ if (role & BDRV_CHILD_DATA) {
+ *nperm = perm & DEFAULT_PERM_PASSTHROUGH;
+ } else {
+ /* 'backing' is also a child of our BDS, but we don't expect it to be
+ * writeable, so we only forward 'consistent read' */
+ *nperm = perm & BLK_PERM_CONSISTENT_READ;
+ }
+}
+
+static void track_drop(void *opaque)
+{
+ BlockDriverState *bs = (BlockDriverState*)opaque;
+ BlockDriverState *file = bs->file->bs;
+ BDRVAllocTrackState *s = bs->opaque;
+
+ assert(file);
+
+ /* we rely on the fact that we're not used anywhere else, so let's wait
+ * until we're only used once - in the drive connected to the guest (and one
+ * ref is held by bdrv_ref in track_change_backing_file) */
+ if (bs->refcnt > 2) {
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), track_drop, opaque);
+ return;
+ }
+ AioContext *aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+
+ bdrv_drained_begin(bs);
+
+ /* now that we're drained, we can safely set 'DropInProgress' */
+ s->drop_state = DropInProgress;
+ bdrv_child_refresh_perms(bs, bs->file, &error_abort);
+
+ bdrv_replace_node(bs, file, &error_abort);
+ bdrv_set_backing_hd(bs, NULL, &error_abort);
+ bdrv_drained_end(bs);
+ bdrv_unref(bs);
+ aio_context_release(aio_context);
+}
+
+static int track_change_backing_file(BlockDriverState *bs,
+ const char *backing_file,
+ const char *backing_fmt)
+{
+ BDRVAllocTrackState *s = bs->opaque;
+ if (s->auto_remove && s->drop_state == DropNone &&
+ backing_file == NULL && backing_fmt == NULL)
+ {
+ /* backing file has been disconnected, there's no longer any use for
+ * this node, so let's remove ourselves from the block graph - we need
+ * to schedule this for later however, since when this function is
+ * called, the blockjob modifying us is probably not done yet and has a
+ * blocker on 'bs' */
+ s->drop_state = DropRequested;
+ bdrv_ref(bs);
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), track_drop, (void*)bs);
+ }
+
+ return 0;
+}
+
+static BlockDriver bdrv_alloc_track = {
+ .format_name = "alloc-track",
+ .instance_size = sizeof(BDRVAllocTrackState),
+
+ .bdrv_file_open = track_open,
+ .bdrv_close = track_close,
+ .bdrv_co_getlength = track_co_getlength,
+ .bdrv_child_perm = track_child_perm,
+ .bdrv_refresh_limits = track_refresh_limits,
+
+ .bdrv_co_pwrite_zeroes = track_co_pwrite_zeroes,
+ .bdrv_co_pwritev = track_co_pwritev,
+ .bdrv_co_preadv = track_co_preadv,
+ .bdrv_co_pdiscard = track_co_pdiscard,
+
+ .bdrv_co_flush = track_co_flush,
+ .bdrv_co_flush_to_disk = track_co_flush,
+
+ .supports_backing = true,
+
+ .bdrv_co_block_status = track_co_block_status,
+ .bdrv_change_backing_file = track_change_backing_file,
+};
+
+static void bdrv_alloc_track_init(void)
+{
+ bdrv_register(&bdrv_alloc_track);
+}
+
+block_init(bdrv_alloc_track_init);
diff --git a/block/meson.build b/block/meson.build
index becc99ac4e..0a69836593 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -2,6 +2,7 @@ block_ss.add(genh)
block_ss.add(files(
'accounting.c',
'aio_task.c',
+ 'alloc-track.c',
'amend.c',
'backup.c',
'backup-dump.c',

View File

@@ -1,88 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
Date: Wed, 8 Apr 2020 15:29:03 +0200
Subject: [PATCH] PVE: savevm-async: set up migration state
code mostly adapted from upstream savevm.c
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
savevm-async.c | 30 ++++++++++++++++++++++++++++--
1 file changed, 28 insertions(+), 2 deletions(-)
diff --git a/savevm-async.c b/savevm-async.c
index 790e27ae37..a38b15d652 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -225,6 +225,7 @@ static void *process_savevm_thread(void *opaque)
{
int ret;
int64_t maxlen;
+ MigrationState *ms = migrate_get_current();
rcu_register_thread();
@@ -234,8 +235,7 @@ static void *process_savevm_thread(void *opaque)
if (ret < 0) {
save_snapshot_error("qemu_savevm_state_setup failed");
- rcu_unregister_thread();
- return NULL;
+ goto out;
}
while (snap_state.state == SAVE_STATE_ACTIVE) {
@@ -287,6 +287,12 @@ static void *process_savevm_thread(void *opaque)
qemu_bh_schedule(snap_state.cleanup_bh);
qemu_mutex_unlock_iothread();
+out:
+ /* set migration state accordingly and clear soon-to-be stale file */
+ migrate_set_state(&ms->state, MIGRATION_STATUS_SETUP,
+ ret ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
+ ms->to_dst_file = NULL;
+
rcu_unregister_thread();
return NULL;
}
@@ -294,6 +300,7 @@ static void *process_savevm_thread(void *opaque)
void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
{
Error *local_err = NULL;
+ MigrationState *ms = migrate_get_current();
int bdrv_oflags = BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_NO_FLUSH;
@@ -303,6 +310,17 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
return;
}
+ if (migration_is_running(ms->state)) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, QERR_MIGRATION_ACTIVE);
+ return;
+ }
+
+ if (migrate_use_block()) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "Block migration and snapshots are incompatible");
+ return;
+ }
+
/* initialize snapshot info */
snap_state.saved_vm_running = runstate_is_running();
snap_state.bs_pos = 0;
@@ -341,6 +359,14 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
goto restart;
}
+ /*
+ * qemu_savevm_* paths use migration code and expect a migration state.
+ * State is cleared in process_savevm_thread, but has to be initialized
+ * here (blocking main thread, from QMP) to avoid race conditions.
+ */
+ migrate_init(ms);
+ memset(&ram_counters, 0, sizeof(ram_counters));
+ ms->to_dst_file = snap_state.file;
error_setg(&snap_state.blocker, "block device is in use by savevm");
blk_op_block_all(snap_state.target, snap_state.blocker);

View File

@@ -0,0 +1,80 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Thu, 23 Jun 2022 14:00:05 +0200
Subject: [PATCH] Revert "block/rbd: workaround for ceph issue #53784"
This reverts commit fc176116cdea816ceb8dd969080b2b95f58edbc0 in
preparation to revert 0347a8fd4c3faaedf119be04c197804be40a384b.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
block/rbd.c | 42 ++----------------------------------------
1 file changed, 2 insertions(+), 40 deletions(-)
diff --git a/block/rbd.c b/block/rbd.c
index a4749f3b1b..53e0396b51 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -1511,7 +1511,6 @@ static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs,
int status, r;
RBDDiffIterateReq req = { .offs = offset };
uint64_t features, flags;
- uint64_t head = 0;
assert(offset + bytes <= s->image_size);
@@ -1539,43 +1538,7 @@ static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs,
return status;
}
-#if LIBRBD_VERSION_CODE < LIBRBD_VERSION(1, 17, 0)
- /*
- * librbd had a bug until early 2022 that affected all versions of ceph that
- * supported fast-diff. This bug results in reporting of incorrect offsets
- * if the offset parameter to rbd_diff_iterate2 is not object aligned.
- * Work around this bug by rounding down the offset to object boundaries.
- * This is OK because we call rbd_diff_iterate2 with whole_object = true.
- * However, this workaround only works for non cloned images with default
- * striping.
- *
- * See: https://tracker.ceph.com/issues/53784
- */
-
- /* check if RBD image has non-default striping enabled */
- if (features & RBD_FEATURE_STRIPINGV2) {
- return status;
- }
-
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
- /*
- * check if RBD image is a clone (= has a parent).
- *
- * rbd_get_parent_info is deprecated from Nautilus onwards, but the
- * replacement rbd_get_parent is not present in Luminous and Mimic.
- */
- if (rbd_get_parent_info(s->image, NULL, 0, NULL, 0, NULL, 0) != -ENOENT) {
- return status;
- }
-#pragma GCC diagnostic pop
-
- head = req.offs & (s->object_size - 1);
- req.offs -= head;
- bytes += head;
-#endif
-
- r = rbd_diff_iterate2(s->image, NULL, req.offs, bytes, true, true,
+ r = rbd_diff_iterate2(s->image, NULL, offset, bytes, true, true,
qemu_rbd_diff_iterate_cb, &req);
if (r < 0 && r != QEMU_RBD_EXIT_DIFF_ITERATE2) {
return status;
@@ -1594,8 +1557,7 @@ static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs,
status = BDRV_BLOCK_ZERO | BDRV_BLOCK_OFFSET_VALID;
}
- assert(req.bytes > head);
- *pnum = req.bytes - head;
+ *pnum = req.bytes;
return status;
}

View File

@@ -1,211 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Fri, 17 Apr 2020 08:57:47 +0200
Subject: [PATCH] PVE Backup: avoid use QemuRecMutex inside coroutines
---
pve-backup.c | 59 +++++++++++++++++++++++++++++++++-------------------
1 file changed, 38 insertions(+), 21 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
index 169f0c68d0..dddf430399 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -11,6 +11,23 @@
/* PVE backup state and related function */
+/*
+ * Note: A resume from a qemu_coroutine_yield can happen in a different thread,
+ * so you may not use normal mutexes within coroutines:
+ *
+ * ---bad-example---
+ * qemu_rec_mutex_lock(lock)
+ * ...
+ * qemu_coroutine_yield() // wait for something
+ * // we are now inside a different thread
+ * qemu_rec_mutex_unlock(lock) // Crash - wrong thread!!
+ * ---end-bad-example--
+ *
+ * ==> Always use CoMutext inside coroutines.
+ * ==> Never acquire/release AioContext withing coroutines (because that use QemuRecMutex)
+ *
+ */
+
static struct PVEBackupState {
struct {
// Everithing accessed from qmp_backup_query command is protected using lock
@@ -30,12 +47,14 @@ static struct PVEBackupState {
ProxmoxBackupHandle *pbs;
GList *di_list;
QemuRecMutex backup_mutex;
+ CoMutex dump_callback_mutex;
} backup_state;
static void pvebackup_init(void)
{
qemu_rec_mutex_init(&backup_state.stat.lock);
qemu_rec_mutex_init(&backup_state.backup_mutex);
+ qemu_co_mutex_init(&backup_state.dump_callback_mutex);
}
// initialize PVEBackupState at startup
@@ -114,16 +133,16 @@ pvebackup_co_dump_pbs_cb(
Error *local_err = NULL;
int pbs_res = -1;
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
// avoid deadlock if job is cancelled
if (pvebackup_error_or_canceled()) {
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
return -1;
}
pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, buf, start, size, &local_err);
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
if (pbs_res < 0) {
pvebackup_propagate_error(local_err);
@@ -149,7 +168,6 @@ pvebackup_co_dump_vma_cb(
const unsigned char *buf = pbuf;
PVEBackupDevInfo *di = opaque;
-
int ret = -1;
assert(backup_state.vmaw);
@@ -167,16 +185,16 @@ pvebackup_co_dump_vma_cb(
}
while (remaining > 0) {
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
// avoid deadlock if job is cancelled
if (pvebackup_error_or_canceled()) {
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
return -1;
}
size_t zero_bytes = 0;
ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num, buf, &zero_bytes);
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
++cluster_num;
if (buf) {
@@ -203,12 +221,11 @@ pvebackup_co_dump_vma_cb(
return size;
}
+// assumes the caller holds backup_mutex
static void coroutine_fn pvebackup_co_cleanup(void *unused)
{
assert(qemu_in_coroutine());
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
-
qemu_rec_mutex_lock(&backup_state.stat.lock);
backup_state.stat.end_time = time(NULL);
qemu_rec_mutex_unlock(&backup_state.stat.lock);
@@ -239,9 +256,9 @@ static void coroutine_fn pvebackup_co_cleanup(void *unused)
g_list_free(backup_state.di_list);
backup_state.di_list = NULL;
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
}
+// assumes the caller holds backup_mutex
static void coroutine_fn pvebackup_complete_stream(void *opaque)
{
PVEBackupDevInfo *di = opaque;
@@ -295,6 +312,8 @@ static void pvebackup_complete_cb(void *opaque, int ret)
static void pvebackup_cancel(void)
{
+ assert(!qemu_in_coroutine());
+
Error *cancel_err = NULL;
error_setg(&cancel_err, "backup canceled");
pvebackup_propagate_error(cancel_err);
@@ -348,6 +367,7 @@ void qmp_backup_cancel(Error **errp)
pvebackup_cancel();
}
+// assumes the caller holds backup_mutex
static int coroutine_fn pvebackup_co_add_config(
const char *file,
const char *name,
@@ -431,9 +451,9 @@ static void pvebackup_run_next_job(void)
}
}
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
-
block_on_coroutine_fn(pvebackup_co_cleanup, NULL); // no more jobs, run cleanup
+
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
}
static bool create_backup_jobs(void) {
@@ -520,6 +540,7 @@ typedef struct QmpBackupTask {
UuidInfo *result;
} QmpBackupTask;
+// assumes the caller holds backup_mutex
static void coroutine_fn pvebackup_co_prepare(void *opaque)
{
assert(qemu_in_coroutine());
@@ -543,11 +564,8 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
const char *config_name = "qemu-server.conf";
const char *firewall_name = "qemu-server.fw";
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
-
if (backup_state.di_list) {
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
return;
}
@@ -792,8 +810,6 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
backup_state.di_list = di_list;
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
-
uuid_info = g_malloc0(sizeof(*uuid_info));
uuid_info->UUID = uuid_str;
@@ -836,8 +852,6 @@ err:
rmdir(backup_dir);
}
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
-
task->result = NULL;
return;
}
@@ -881,13 +895,16 @@ UuidInfo *qmp_backup(
.errp = errp,
};
+ qemu_rec_mutex_lock(&backup_state.backup_mutex);
+
block_on_coroutine_fn(pvebackup_co_prepare, &task);
if (*errp == NULL) {
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
create_backup_jobs();
qemu_rec_mutex_unlock(&backup_state.backup_mutex);
pvebackup_run_next_job();
+ } else {
+ qemu_rec_mutex_unlock(&backup_state.backup_mutex);
}
return task.result;

View File

@@ -0,0 +1,35 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Thu, 23 Jun 2022 14:00:07 +0200
Subject: [PATCH] Revert "block/rbd: fix handling of holes in
.bdrv_co_block_status"
This reverts commit 9e302f64bb407a9bb097b626da97228c2654cfee in
preparation to revert 0347a8fd4c3faaedf119be04c197804be40a384b.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
block/rbd.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/block/rbd.c b/block/rbd.c
index 53e0396b51..0913a0af39 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -1470,11 +1470,11 @@ static int qemu_rbd_diff_iterate_cb(uint64_t offs, size_t len,
RBDDiffIterateReq *req = opaque;
assert(req->offs + req->bytes <= offs);
-
- /* treat a hole like an unallocated area and bail out */
- if (!exists) {
- return 0;
- }
+ /*
+ * we do not diff against a snapshot so we should never receive a callback
+ * for a hole.
+ */
+ assert(exists);
if (!req->exists && offs > req->offs) {
/*

View File

@@ -1,227 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Fri, 17 Apr 2020 08:57:48 +0200
Subject: [PATCH] PVE Backup: use QemuMutex instead of QemuRecMutex
We acquire/release all mutexes outside coroutines now, so we can now
correctly use a normal mutex.
---
pve-backup.c | 58 ++++++++++++++++++++++++++--------------------------
1 file changed, 29 insertions(+), 29 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
index dddf430399..bb917ee972 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -31,7 +31,7 @@
static struct PVEBackupState {
struct {
// Everithing accessed from qmp_backup_query command is protected using lock
- QemuRecMutex lock;
+ QemuMutex lock;
Error *error;
time_t start_time;
time_t end_time;
@@ -46,14 +46,14 @@ static struct PVEBackupState {
VmaWriter *vmaw;
ProxmoxBackupHandle *pbs;
GList *di_list;
- QemuRecMutex backup_mutex;
+ QemuMutex backup_mutex;
CoMutex dump_callback_mutex;
} backup_state;
static void pvebackup_init(void)
{
- qemu_rec_mutex_init(&backup_state.stat.lock);
- qemu_rec_mutex_init(&backup_state.backup_mutex);
+ qemu_mutex_init(&backup_state.stat.lock);
+ qemu_mutex_init(&backup_state.backup_mutex);
qemu_co_mutex_init(&backup_state.dump_callback_mutex);
}
@@ -91,26 +91,26 @@ lookup_active_block_job(PVEBackupDevInfo *di)
static void pvebackup_propagate_error(Error *err)
{
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
error_propagate(&backup_state.stat.error, err);
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
}
static bool pvebackup_error_or_canceled(void)
{
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
bool error_or_canceled = !!backup_state.stat.error;
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
return error_or_canceled;
}
static void pvebackup_add_transfered_bytes(size_t transferred, size_t zero_bytes)
{
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
backup_state.stat.zero_bytes += zero_bytes;
backup_state.stat.transferred += transferred;
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
}
// This may get called from multiple coroutines in multiple io-threads
@@ -226,9 +226,9 @@ static void coroutine_fn pvebackup_co_cleanup(void *unused)
{
assert(qemu_in_coroutine());
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
backup_state.stat.end_time = time(NULL);
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
if (backup_state.vmaw) {
Error *local_err = NULL;
@@ -284,7 +284,7 @@ static void pvebackup_complete_cb(void *opaque, int ret)
PVEBackupDevInfo *di = opaque;
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_mutex_lock(&backup_state.backup_mutex);
di->completed = true;
@@ -305,7 +305,7 @@ static void pvebackup_complete_cb(void *opaque, int ret)
g_free(di);
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
pvebackup_run_next_job();
}
@@ -318,7 +318,7 @@ static void pvebackup_cancel(void)
error_setg(&cancel_err, "backup canceled");
pvebackup_propagate_error(cancel_err);
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_mutex_lock(&backup_state.backup_mutex);
if (backup_state.vmaw) {
/* make sure vma writer does not block anymore */
@@ -329,13 +329,13 @@ static void pvebackup_cancel(void)
proxmox_backup_abort(backup_state.pbs, "backup canceled");
}
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
for(;;) {
BlockJob *next_job = NULL;
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_mutex_lock(&backup_state.backup_mutex);
GList *l = backup_state.di_list;
while (l) {
@@ -349,7 +349,7 @@ static void pvebackup_cancel(void)
}
}
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
if (next_job) {
AioContext *aio_context = next_job->job.aio_context;
@@ -423,7 +423,7 @@ static void pvebackup_run_next_job(void)
{
assert(!qemu_in_coroutine());
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_mutex_lock(&backup_state.backup_mutex);
GList *l = backup_state.di_list;
while (l) {
@@ -433,7 +433,7 @@ static void pvebackup_run_next_job(void)
BlockJob *job = lookup_active_block_job(di);
if (job) {
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
AioContext *aio_context = job->job.aio_context;
aio_context_acquire(aio_context);
@@ -453,7 +453,7 @@ static void pvebackup_run_next_job(void)
block_on_coroutine_fn(pvebackup_co_cleanup, NULL); // no more jobs, run cleanup
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
}
static bool create_backup_jobs(void) {
@@ -778,7 +778,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
}
/* initialize global backup_state now */
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
if (backup_state.stat.error) {
error_free(backup_state.stat.error);
@@ -801,7 +801,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
backup_state.stat.transferred = 0;
backup_state.stat.zero_bytes = 0;
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
@@ -895,16 +895,16 @@ UuidInfo *qmp_backup(
.errp = errp,
};
- qemu_rec_mutex_lock(&backup_state.backup_mutex);
+ qemu_mutex_lock(&backup_state.backup_mutex);
block_on_coroutine_fn(pvebackup_co_prepare, &task);
if (*errp == NULL) {
create_backup_jobs();
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
pvebackup_run_next_job();
} else {
- qemu_rec_mutex_unlock(&backup_state.backup_mutex);
+ qemu_mutex_unlock(&backup_state.backup_mutex);
}
return task.result;
@@ -914,11 +914,11 @@ BackupStatus *qmp_query_backup(Error **errp)
{
BackupStatus *info = g_malloc0(sizeof(*info));
- qemu_rec_mutex_lock(&backup_state.stat.lock);
+ qemu_mutex_lock(&backup_state.stat.lock);
if (!backup_state.stat.start_time) {
/* not started, return {} */
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
return info;
}
@@ -955,7 +955,7 @@ BackupStatus *qmp_query_backup(Error **errp)
info->has_transferred = true;
info->transferred = backup_state.stat.transferred;
- qemu_rec_mutex_unlock(&backup_state.stat.lock);
+ qemu_mutex_unlock(&backup_state.stat.lock);
return info;
}

View File

@@ -0,0 +1,161 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fabian Ebner <f.ebner@proxmox.com>
Date: Tue, 17 May 2022 09:46:02 +0200
Subject: [PATCH] Revert "block/rbd: implement bdrv_co_block_status"
During backup, bdrv_co_block_status is called for each block copy
chunk. When RBD is used, the current implementation with
rbd_diff_iterate2() using whole_object=true takes about linearly more
time, depending on the image size. Since there are linearly more
chunks, the slowdown is quadratic, becoming unacceptable for large
images (starting somewhere between 500-1000 GiB in my testing).
This reverts commit 0347a8fd4c3faaedf119be04c197804be40a384b as a
stop-gap measure, until it's clear how to make the implemenation
more efficient.
Upstream bug report:
https://gitlab.com/qemu-project/qemu/-/issues/1026
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
block/rbd.c | 112 ----------------------------------------------------
1 file changed, 112 deletions(-)
diff --git a/block/rbd.c b/block/rbd.c
index 0913a0af39..1dab254517 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -108,12 +108,6 @@ typedef struct RBDTask {
int64_t ret;
} RBDTask;
-typedef struct RBDDiffIterateReq {
- uint64_t offs;
- uint64_t bytes;
- bool exists;
-} RBDDiffIterateReq;
-
static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
BlockdevOptionsRbd *opts, bool cache,
const char *keypairs, const char *secretid,
@@ -1456,111 +1450,6 @@ static ImageInfoSpecific *qemu_rbd_get_specific_info(BlockDriverState *bs,
return spec_info;
}
-/*
- * rbd_diff_iterate2 allows to interrupt the exection by returning a negative
- * value in the callback routine. Choose a value that does not conflict with
- * an existing exitcode and return it if we want to prematurely stop the
- * execution because we detected a change in the allocation status.
- */
-#define QEMU_RBD_EXIT_DIFF_ITERATE2 -9000
-
-static int qemu_rbd_diff_iterate_cb(uint64_t offs, size_t len,
- int exists, void *opaque)
-{
- RBDDiffIterateReq *req = opaque;
-
- assert(req->offs + req->bytes <= offs);
- /*
- * we do not diff against a snapshot so we should never receive a callback
- * for a hole.
- */
- assert(exists);
-
- if (!req->exists && offs > req->offs) {
- /*
- * we started in an unallocated area and hit the first allocated
- * block. req->bytes must be set to the length of the unallocated area
- * before the allocated area. stop further processing.
- */
- req->bytes = offs - req->offs;
- return QEMU_RBD_EXIT_DIFF_ITERATE2;
- }
-
- if (req->exists && offs > req->offs + req->bytes) {
- /*
- * we started in an allocated area and jumped over an unallocated area,
- * req->bytes contains the length of the allocated area before the
- * unallocated area. stop further processing.
- */
- return QEMU_RBD_EXIT_DIFF_ITERATE2;
- }
-
- req->bytes += len;
- req->exists = true;
-
- return 0;
-}
-
-static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs,
- bool want_zero, int64_t offset,
- int64_t bytes, int64_t *pnum,
- int64_t *map,
- BlockDriverState **file)
-{
- BDRVRBDState *s = bs->opaque;
- int status, r;
- RBDDiffIterateReq req = { .offs = offset };
- uint64_t features, flags;
-
- assert(offset + bytes <= s->image_size);
-
- /* default to all sectors allocated */
- status = BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID;
- *map = offset;
- *file = bs;
- *pnum = bytes;
-
- /* check if RBD image supports fast-diff */
- r = rbd_get_features(s->image, &features);
- if (r < 0) {
- return status;
- }
- if (!(features & RBD_FEATURE_FAST_DIFF)) {
- return status;
- }
-
- /* check if RBD fast-diff result is valid */
- r = rbd_get_flags(s->image, &flags);
- if (r < 0) {
- return status;
- }
- if (flags & RBD_FLAG_FAST_DIFF_INVALID) {
- return status;
- }
-
- r = rbd_diff_iterate2(s->image, NULL, offset, bytes, true, true,
- qemu_rbd_diff_iterate_cb, &req);
- if (r < 0 && r != QEMU_RBD_EXIT_DIFF_ITERATE2) {
- return status;
- }
- assert(req.bytes <= bytes);
- if (!req.exists) {
- if (r == 0) {
- /*
- * rbd_diff_iterate2 does not invoke callbacks for unallocated
- * areas. This here catches the case where no callback was
- * invoked at all (req.bytes == 0).
- */
- assert(req.bytes == 0);
- req.bytes = bytes;
- }
- status = BDRV_BLOCK_ZERO | BDRV_BLOCK_OFFSET_VALID;
- }
-
- *pnum = req.bytes;
- return status;
-}
-
static int64_t coroutine_fn qemu_rbd_co_getlength(BlockDriverState *bs)
{
BDRVRBDState *s = bs->opaque;
@@ -1796,7 +1685,6 @@ static BlockDriver bdrv_rbd = {
#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES
.bdrv_co_pwrite_zeroes = qemu_rbd_co_pwrite_zeroes,
#endif
- .bdrv_co_block_status = qemu_rbd_co_block_status,
.bdrv_snapshot_create = qemu_rbd_snap_create,
.bdrv_snapshot_delete = qemu_rbd_snap_remove,

View File

@@ -0,0 +1,153 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Thu, 6 Apr 2023 14:59:31 +0200
Subject: [PATCH] alloc-track: fix deadlock during drop
by replacing the block node directly after changing the backing file
instead of rescheduling it.
With changes in QEMU 8.0, calling bdrv_get_info (and bdrv_unref)
during drop can lead to a deadlock when using iothread (only triggered
with multiple disks, except during debugging where it also triggered
with one disk sometimes):
1. job_unref_locked acquires the AioContext and calls job->driver->free
2. track_drop gets scheduled
3. bdrv_graph_wrlock is called and polls which leads to track_drop being
called
4. track_drop acquires the AioContext recursively
5. bdrv_get_info is a wrapped coroutine (since 8.0) and thus polls for
bdrv_co_get_info. This releases the AioContext, but only once! The
documentation for the AIO_WAIT_WHILE macro states that the
AioContext lock needs to be acquired exactly once, but there does
not seem to be a way for track_drop to know if it acquired the lock
recursively or not (without adding further hacks).
6. Because the AioContext is still held by the main thread once, it can't
be acquired before entering bdrv_co_get_info in co_schedule_bh_cb
which happens in the iothread
When doing the operation in change_backing_file, the AioContext has
already been acquired by the caller, so the issue with the recursive
lock goes away.
The comment explaining why delaying the replace is necessary is
> we need to schedule this for later however, since when this function
> is called, the blockjob modifying us is probably not done yet and
> has a blocker on 'bs'
However, there is no check for blockers in bdrv_replace_node. It would
need to be done by us, the caller, with check_to_replace_node.
Furthermore, the mirror job also does its call to bdrv_replace_node
while there is an active blocker (inserted by mirror itself) and they
use a specialized version to check for blockers instead of
check_to_replace_node there. Alloc-track could also do something
similar to check for other blockers, but it should be fine to rely on
Proxmox VE that no other operation with the blockdev is going on.
Mirror also drains the target before replacing the node, but the
target can have other users. In case of alloc-track the file child
should not be accessible by anybody else and so there can't be an
in-flight operation for the file child when alloc-track is drained.
The rescheduling based on refcounting is a hack and it doesn't seem to
be necessary anymore. It's not clear what the original issue from the
comment was. Testing with older builds with track_drop done directly
without rescheduling also didn't lead to any noticable issue for me.
One issue it might have been is the one fixed by b1e1af394d
("block/stream: Drain subtree around graph change"), where
block-stream had a use-after-free if the base node changed at an
inconvenient time (which alloc-track's auto-drop does).
It's also not possible to just not auto-replace the alloc-track. Not
replacing it at all leads to other operations like block resize
hanging, and there is no good way to replace it manually via QMP
(there is x-blockdev-change, but it is experimental and doesn't
implement the required operation yet). Also, it's just cleaner in
general to not leave unnecessary block nodes lying around.
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/alloc-track.c | 54 ++++++++++++++-------------------------------
1 file changed, 16 insertions(+), 38 deletions(-)
diff --git a/block/alloc-track.c b/block/alloc-track.c
index b75d7c6460..76da140a68 100644
--- a/block/alloc-track.c
+++ b/block/alloc-track.c
@@ -25,7 +25,6 @@
typedef enum DropState {
DropNone,
- DropRequested,
DropInProgress,
} DropState;
@@ -268,37 +267,6 @@ static void track_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
-static void track_drop(void *opaque)
-{
- BlockDriverState *bs = (BlockDriverState*)opaque;
- BlockDriverState *file = bs->file->bs;
- BDRVAllocTrackState *s = bs->opaque;
-
- assert(file);
-
- /* we rely on the fact that we're not used anywhere else, so let's wait
- * until we're only used once - in the drive connected to the guest (and one
- * ref is held by bdrv_ref in track_change_backing_file) */
- if (bs->refcnt > 2) {
- aio_bh_schedule_oneshot(qemu_get_aio_context(), track_drop, opaque);
- return;
- }
- AioContext *aio_context = bdrv_get_aio_context(bs);
- aio_context_acquire(aio_context);
-
- bdrv_drained_begin(bs);
-
- /* now that we're drained, we can safely set 'DropInProgress' */
- s->drop_state = DropInProgress;
- bdrv_child_refresh_perms(bs, bs->file, &error_abort);
-
- bdrv_replace_node(bs, file, &error_abort);
- bdrv_set_backing_hd(bs, NULL, &error_abort);
- bdrv_drained_end(bs);
- bdrv_unref(bs);
- aio_context_release(aio_context);
-}
-
static int track_change_backing_file(BlockDriverState *bs,
const char *backing_file,
const char *backing_fmt)
@@ -308,13 +276,23 @@ static int track_change_backing_file(BlockDriverState *bs,
backing_file == NULL && backing_fmt == NULL)
{
/* backing file has been disconnected, there's no longer any use for
- * this node, so let's remove ourselves from the block graph - we need
- * to schedule this for later however, since when this function is
- * called, the blockjob modifying us is probably not done yet and has a
- * blocker on 'bs' */
- s->drop_state = DropRequested;
+ * this node, so let's remove ourselves from the block graph */
+ BlockDriverState *file = bs->file->bs;
+
+ /* Just to be sure, because bdrv_replace_node unrefs it */
bdrv_ref(bs);
- aio_bh_schedule_oneshot(qemu_get_aio_context(), track_drop, (void*)bs);
+ bdrv_drained_begin(bs);
+
+ /* now that we're drained, we can safely set 'DropInProgress' */
+ s->drop_state = DropInProgress;
+
+ bdrv_child_refresh_perms(bs, bs->file, &error_abort);
+
+ bdrv_replace_node(bs, file, &error_abort);
+ bdrv_set_backing_hd(bs, NULL, &error_abort);
+
+ bdrv_drained_end(bs);
+ bdrv_unref(bs);
}
return 0;

View File

@@ -1,111 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Thu, 30 Apr 2020 15:55:37 +0200
Subject: [PATCH] move savevm-async back into a coroutine
Move qemu_savevm_state_{header,setup} into the main loop and
the rest of the iteration into a coroutine. The former need
to lock the iothread (and we can't unlock it in the
coroutine), and the latter can't deal with being in a
separate thread, so a coroutine it must be.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
savevm-async.c | 28 +++++++++-------------------
1 file changed, 9 insertions(+), 19 deletions(-)
diff --git a/savevm-async.c b/savevm-async.c
index a38b15d652..af865b9a0a 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -51,7 +51,7 @@ static struct SnapshotState {
QEMUFile *file;
int64_t total_time;
QEMUBH *cleanup_bh;
- QemuThread thread;
+ Coroutine *co;
} snap_state;
SaveVMInfo *qmp_query_savevm(Error **errp)
@@ -201,11 +201,9 @@ static void process_savevm_cleanup(void *opaque)
int ret;
qemu_bh_delete(snap_state.cleanup_bh);
snap_state.cleanup_bh = NULL;
+ snap_state.co = NULL;
qemu_savevm_state_cleanup();
- qemu_mutex_unlock_iothread();
- qemu_thread_join(&snap_state.thread);
- qemu_mutex_lock_iothread();
ret = save_snapshot_cleanup();
if (ret < 0) {
save_snapshot_error("save_snapshot_cleanup error %d", ret);
@@ -221,18 +219,13 @@ static void process_savevm_cleanup(void *opaque)
}
}
-static void *process_savevm_thread(void *opaque)
+static void process_savevm_coro(void *opaque)
{
int ret;
int64_t maxlen;
MigrationState *ms = migrate_get_current();
- rcu_register_thread();
-
- qemu_savevm_state_header(snap_state.file);
- qemu_savevm_state_setup(snap_state.file);
ret = qemu_file_get_error(snap_state.file);
-
if (ret < 0) {
save_snapshot_error("qemu_savevm_state_setup failed");
goto out;
@@ -247,16 +240,13 @@ static void *process_savevm_thread(void *opaque)
maxlen = blk_getlength(snap_state.target) - 30*1024*1024;
if (pending_size > 400000 && snap_state.bs_pos + pending_size < maxlen) {
- qemu_mutex_lock_iothread();
ret = qemu_savevm_state_iterate(snap_state.file, false);
if (ret < 0) {
save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
break;
}
- qemu_mutex_unlock_iothread();
DPRINTF("savevm inerate pending size %lu ret %d\n", pending_size, ret);
} else {
- qemu_mutex_lock_iothread();
qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL);
ret = global_state_store();
if (ret) {
@@ -285,16 +275,12 @@ static void *process_savevm_thread(void *opaque)
}
qemu_bh_schedule(snap_state.cleanup_bh);
- qemu_mutex_unlock_iothread();
out:
/* set migration state accordingly and clear soon-to-be stale file */
migrate_set_state(&ms->state, MIGRATION_STATUS_SETUP,
ret ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
ms->to_dst_file = NULL;
-
- rcu_unregister_thread();
- return NULL;
}
void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
@@ -373,8 +359,12 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
snap_state.state = SAVE_STATE_ACTIVE;
snap_state.cleanup_bh = qemu_bh_new(process_savevm_cleanup, &snap_state);
- qemu_thread_create(&snap_state.thread, "savevm-async", process_savevm_thread,
- NULL, QEMU_THREAD_JOINABLE);
+ snap_state.co = qemu_coroutine_create(&process_savevm_coro, NULL);
+ qemu_mutex_unlock_iothread();
+ qemu_savevm_state_header(snap_state.file);
+ qemu_savevm_state_setup(snap_state.file);
+ qemu_mutex_lock_iothread();
+ aio_co_schedule(iohandler_get_aio_context(), snap_state.co);
return;

View File

@@ -1,183 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 4 May 2020 11:05:08 +0200
Subject: [PATCH] add optional buffer size to QEMUFile
So we can use a 4M buffer for savevm-async which should
increase performance storing the state onto ceph.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
---
migration/qemu-file.c | 36 ++++++++++++++++++++++++------------
migration/qemu-file.h | 1 +
savevm-async.c | 4 ++--
3 files changed, 27 insertions(+), 14 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 1c3a358a14..7362e51c71 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -30,7 +30,7 @@
#include "trace.h"
#include "qapi/error.h"
-#define IO_BUF_SIZE 32768
+#define DEFAULT_IO_BUF_SIZE 32768
#define MAX_IOV_SIZE MIN(IOV_MAX, 64)
struct QEMUFile {
@@ -45,7 +45,8 @@ struct QEMUFile {
when reading */
int buf_index;
int buf_size; /* 0 when writing */
- uint8_t buf[IO_BUF_SIZE];
+ size_t buf_allocated_size;
+ uint8_t *buf;
DECLARE_BITMAP(may_free, MAX_IOV_SIZE);
struct iovec iov[MAX_IOV_SIZE];
@@ -101,7 +102,7 @@ bool qemu_file_mode_is_not_valid(const char *mode)
return false;
}
-QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops)
+QEMUFile *qemu_fopen_ops_sized(void *opaque, const QEMUFileOps *ops, size_t buffer_size)
{
QEMUFile *f;
@@ -109,9 +110,17 @@ QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops)
f->opaque = opaque;
f->ops = ops;
+ f->buf_allocated_size = buffer_size;
+ f->buf = malloc(buffer_size);
+
return f;
}
+QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops)
+{
+ return qemu_fopen_ops_sized(opaque, ops, DEFAULT_IO_BUF_SIZE);
+}
+
void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks)
{
@@ -346,7 +355,7 @@ static ssize_t qemu_fill_buffer(QEMUFile *f)
}
len = f->ops->get_buffer(f->opaque, f->buf + pending, f->pos,
- IO_BUF_SIZE - pending, &local_error);
+ f->buf_allocated_size - pending, &local_error);
if (len > 0) {
f->buf_size += len;
f->pos += len;
@@ -386,6 +395,9 @@ int qemu_fclose(QEMUFile *f)
ret = ret2;
}
}
+
+ free(f->buf);
+
/* If any error was spotted before closing, we should report it
* instead of the close() return value.
*/
@@ -435,7 +447,7 @@ static void add_buf_to_iovec(QEMUFile *f, size_t len)
{
if (!add_to_iovec(f, f->buf + f->buf_index, len, false)) {
f->buf_index += len;
- if (f->buf_index == IO_BUF_SIZE) {
+ if (f->buf_index == f->buf_allocated_size) {
qemu_fflush(f);
}
}
@@ -461,7 +473,7 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, size_t size)
}
while (size > 0) {
- l = IO_BUF_SIZE - f->buf_index;
+ l = f->buf_allocated_size - f->buf_index;
if (l > size) {
l = size;
}
@@ -508,8 +520,8 @@ size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t offset)
size_t index;
assert(!qemu_file_is_writable(f));
- assert(offset < IO_BUF_SIZE);
- assert(size <= IO_BUF_SIZE - offset);
+ assert(offset < f->buf_allocated_size);
+ assert(size <= f->buf_allocated_size - offset);
/* The 1st byte to read from */
index = f->buf_index + offset;
@@ -559,7 +571,7 @@ size_t qemu_get_buffer(QEMUFile *f, uint8_t *buf, size_t size)
size_t res;
uint8_t *src;
- res = qemu_peek_buffer(f, &src, MIN(pending, IO_BUF_SIZE), 0);
+ res = qemu_peek_buffer(f, &src, MIN(pending, f->buf_allocated_size), 0);
if (res == 0) {
return done;
}
@@ -593,7 +605,7 @@ size_t qemu_get_buffer(QEMUFile *f, uint8_t *buf, size_t size)
*/
size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size)
{
- if (size < IO_BUF_SIZE) {
+ if (size < f->buf_allocated_size) {
size_t res;
uint8_t *src;
@@ -618,7 +630,7 @@ int qemu_peek_byte(QEMUFile *f, int offset)
int index = f->buf_index + offset;
assert(!qemu_file_is_writable(f));
- assert(offset < IO_BUF_SIZE);
+ assert(offset < f->buf_allocated_size);
if (index >= f->buf_size) {
qemu_fill_buffer(f);
@@ -770,7 +782,7 @@ static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len,
ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
const uint8_t *p, size_t size)
{
- ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t);
+ ssize_t blen = f->buf_allocated_size - f->buf_index - sizeof(int32_t);
if (blen < compressBound(size)) {
return -1;
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index a9b6d6ccb7..8752d27c74 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -120,6 +120,7 @@ typedef struct QEMUFileHooks {
} QEMUFileHooks;
QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
+QEMUFile *qemu_fopen_ops_sized(void *opaque, const QEMUFileOps *ops, size_t buffer_size);
void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
int qemu_get_fd(QEMUFile *f);
int qemu_fclose(QEMUFile *f);
diff --git a/savevm-async.c b/savevm-async.c
index af865b9a0a..c3fe741c38 100644
--- a/savevm-async.c
+++ b/savevm-async.c
@@ -338,7 +338,7 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
goto restart;
}
- snap_state.file = qemu_fopen_ops(&snap_state, &block_file_ops);
+ snap_state.file = qemu_fopen_ops_sized(&snap_state, &block_file_ops, 4 * 1024 * 1024);
if (!snap_state.file) {
error_set(errp, ERROR_CLASS_GENERIC_ERROR, "failed to open '%s'", statefile);
@@ -454,7 +454,7 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
blk_op_block_all(be, blocker);
/* restore the VM state */
- f = qemu_fopen_ops(be, &loadstate_file_ops);
+ f = qemu_fopen_ops_sized(be, &loadstate_file_ops, 4 * 1024 * 1024);
if (!f) {
error_setg(errp, "Could not open VM state file");
goto the_end;

View File

@@ -0,0 +1,190 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Fri, 5 May 2023 13:39:53 +0200
Subject: [PATCH] migration: for snapshots, hold the BQL during setup callbacks
In spirit, this is a partial revert of commit 9b09503752 ("migration:
run setup callbacks out of big lock"), but only for the snapshot case.
For snapshots, the bdrv_writev_vmstate() function is used during setup
(in QIOChannelBlock backing the QEMUFile), but not holding the BQL
while calling it could lead to an assertion failure. To understand
how, first note the following:
1. Generated coroutine wrappers for block layer functions spawn the
coroutine and use AIO_WAIT_WHILE()/aio_poll() to wait for it.
2. If the host OS switches threads at an inconvenient time, it can
happen that a bottom half scheduled for the main thread's AioContext
is executed as part of a vCPU thread's aio_poll().
An example leading to the assertion failure is as follows:
main thread:
1. A snapshot-save QMP command gets issued.
2. snapshot_save_job_bh() is scheduled.
vCPU thread:
3. aio_poll() for the main thread's AioContext is called (e.g. when
the guest writes to a pflash device, as part of blk_pwrite which is a
generated coroutine wrapper).
4. snapshot_save_job_bh() is executed as part of aio_poll().
3. qemu_savevm_state() is called.
4. qemu_mutex_unlock_iothread() is called. Now
qemu_get_current_aio_context() returns 0x0.
5. bdrv_writev_vmstate() is executed during the usual savevm setup.
But this function is a generated coroutine wrapper, so it uses
AIO_WAIT_WHILE. There, the assertion
assert(qemu_get_current_aio_context() == qemu_get_aio_context());
will fail.
To fix it, ensure that the BQL is held during setup. To avoid changing
the behavior for migration too, introduce conditionals for the setup
callbacks that need the BQL and only take the lock if it's not already
held.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
include/migration/register.h | 2 +-
migration/block-dirty-bitmap.c | 15 ++++++++++++---
migration/block.c | 15 ++++++++++++---
migration/ram.c | 16 +++++++++++++---
migration/savevm.c | 2 --
5 files changed, 38 insertions(+), 12 deletions(-)
diff --git a/include/migration/register.h b/include/migration/register.h
index 90914f32f5..c728fd9120 100644
--- a/include/migration/register.h
+++ b/include/migration/register.h
@@ -43,9 +43,9 @@ typedef struct SaveVMHandlers {
* by other locks.
*/
int (*save_live_iterate)(QEMUFile *f, void *opaque);
+ int (*save_setup)(QEMUFile *f, void *opaque);
/* This runs outside the iothread lock! */
- int (*save_setup)(QEMUFile *f, void *opaque);
/* Note for save_live_pending:
* must_precopy:
* - must be migrated in precopy or in stopped state
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 285dd1d148..f7ee5a74d9 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -1219,10 +1219,17 @@ static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque)
{
DBMSaveState *s = &((DBMState *)opaque)->save;
SaveBitmapState *dbms = NULL;
+ bool release_lock = false;
- qemu_mutex_lock_iothread();
+ /* For snapshots, the BQL is held during setup. */
+ if (!qemu_mutex_iothread_locked()) {
+ qemu_mutex_lock_iothread();
+ release_lock = true;
+ }
if (init_dirty_bitmap_migration(s) < 0) {
- qemu_mutex_unlock_iothread();
+ if (release_lock) {
+ qemu_mutex_unlock_iothread();
+ }
return -1;
}
@@ -1230,7 +1237,9 @@ static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque)
send_bitmap_start(f, s, dbms);
}
qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
- qemu_mutex_unlock_iothread();
+ if (release_lock) {
+ qemu_mutex_unlock_iothread();
+ }
return 0;
}
diff --git a/migration/block.c b/migration/block.c
index 86c2256a2b..8423e0c9f9 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -725,21 +725,30 @@ static void block_migration_cleanup(void *opaque)
static int block_save_setup(QEMUFile *f, void *opaque)
{
int ret;
+ bool release_lock = false;
trace_migration_block_save("setup", block_mig_state.submitted,
block_mig_state.transferred);
- qemu_mutex_lock_iothread();
+ /* For snapshots, the BQL is held during setup. */
+ if (!qemu_mutex_iothread_locked()) {
+ qemu_mutex_lock_iothread();
+ release_lock = true;
+ }
ret = init_blk_migration(f);
if (ret < 0) {
- qemu_mutex_unlock_iothread();
+ if (release_lock) {
+ qemu_mutex_unlock_iothread();
+ }
return ret;
}
/* start track dirty blocks */
ret = set_dirty_tracking();
- qemu_mutex_unlock_iothread();
+ if (release_lock) {
+ qemu_mutex_unlock_iothread();
+ }
if (ret) {
return ret;
diff --git a/migration/ram.c b/migration/ram.c
index 6e1514f69f..6a1aec7031 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2896,8 +2896,16 @@ static void migration_bitmap_clear_discarded_pages(RAMState *rs)
static void ram_init_bitmaps(RAMState *rs)
{
- /* For memory_global_dirty_log_start below. */
- qemu_mutex_lock_iothread();
+ bool release_lock = false;
+
+ /*
+ * For memory_global_dirty_log_start below.
+ * For snapshots, the BQL is held during setup.
+ */
+ if (!qemu_mutex_iothread_locked()) {
+ qemu_mutex_lock_iothread();
+ release_lock = true;
+ }
qemu_mutex_lock_ramlist();
WITH_RCU_READ_LOCK_GUARD() {
@@ -2909,7 +2917,9 @@ static void ram_init_bitmaps(RAMState *rs)
}
}
qemu_mutex_unlock_ramlist();
- qemu_mutex_unlock_iothread();
+ if (release_lock) {
+ qemu_mutex_unlock_iothread();
+ }
/*
* After an eventual first bitmap sync, fixup the initial bitmap
diff --git a/migration/savevm.c b/migration/savevm.c
index d60c4f487a..3c015722f7 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1625,10 +1625,8 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp)
reset_vfio_bytes_transferred();
ms->to_dst_file = f;
- qemu_mutex_unlock_iothread();
qemu_savevm_state_header(f);
qemu_savevm_state_setup(f);
- qemu_mutex_lock_iothread();
while (qemu_file_get_error(f) == 0) {
if (qemu_savevm_state_iterate(f, false) > 0) {

View File

@@ -0,0 +1,29 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Fiona Ebner <f.ebner@proxmox.com>
Date: Fri, 5 May 2023 15:30:16 +0200
Subject: [PATCH] savevm-async: don't hold BQL during setup
See commit "migration: for snapshots, hold the BQL during setup
callbacks" for why. This is separate, because a version of that one
will hopefully land upstream.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
migration/savevm-async.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/migration/savevm-async.c b/migration/savevm-async.c
index 80624fada8..b1d85a4b41 100644
--- a/migration/savevm-async.c
+++ b/migration/savevm-async.c
@@ -401,10 +401,8 @@ void qmp_savevm_start(const char *statefile, Error **errp)
snap_state.state = SAVE_STATE_ACTIVE;
snap_state.finalize_bh = qemu_bh_new(process_savevm_finalize, &snap_state);
snap_state.co = qemu_coroutine_create(&process_savevm_co, NULL);
- qemu_mutex_unlock_iothread();
qemu_savevm_state_header(snap_state.file);
qemu_savevm_state_setup(snap_state.file);
- qemu_mutex_lock_iothread();
/* Async processing from here on out happens in iohandler context, so let
* the target bdrv have its home there.

99
debian/patches/series vendored
View File

@@ -1,44 +1,63 @@
extra/0001-monitor-qmp-fix-race-with-clients-disconnecting-earl.patch
extra/0002-scsi-megasas-Internal-cdbs-have-16-byte-length.patch
extra/0003-ide-avoid-potential-deadlock-when-draining-during-tr.patch
extra/0004-migration-block-dirty-bitmap-fix-loading-bitmap-when.patch
extra/0005-Revert-Revert-graph-lock-Disable-locking-for-now.patch
extra/0006-migration-states-workaround-snapshot-performance-reg.patch
extra/0007-Revert-x86-acpi-workaround-Windows-not-handling-name.patch
extra/0008-target-i386-the-sgx_epc_get_section-stub-is-reachabl.patch
extra/0009-ui-clipboard-mark-type-as-not-available-when-there-i.patch
extra/0010-virtio-scsi-Attach-event-vq-notifier-with-no_poll.patch
extra/0011-virtio-Re-enable-notifications-after-drain.patch
bitmap-mirror/0001-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
bitmap-mirror/0002-drive-mirror-add-support-for-conditional-and-always-.patch
bitmap-mirror/0003-mirror-add-check-for-bitmap-mode-without-bitmap.patch
bitmap-mirror/0004-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch
bitmap-mirror/0005-iotests-add-test-for-bitmap-mirror.patch
bitmap-mirror/0006-mirror-move-some-checks-to-qmp.patch
pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch
pve/0005-PVE-Config-smm_available-false.patch
pve/0006-PVE-Config-glusterfs-no-default-logfile-if-daemonize.patch
pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch
pve/0008-PVE-Up-qmp-add-get_link_status.patch
pve/0009-PVE-Up-glusterfs-allow-partial-reads.patch
pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch
pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch
pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch
pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch
pve/0014-PVE-virtio-balloon-improve-query-balloon.patch
pve/0015-PVE-qapi-modify-query-machines.patch
pve/0016-PVE-qapi-modify-spice-query.patch
pve/0017-PVE-internal-snapshot-async.patch
pve/0018-PVE-block-add-the-zeroinit-block-driver-filter.patch
pve/0019-PVE-Add-dummy-id-command-line-parameter.patch
pve/0020-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
pve/0021-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
pve/0022-PVE-savevm-async-kick-AIO-wait-on-block-state-write.patch
pve/0023-PVE-move-snapshot-cleanup-into-bottom-half.patch
pve/0024-PVE-monitor-disable-oob-capability.patch
pve/0025-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
pve/0026-PVE-Allow-version-code-in-machine-type.patch
pve/0027-PVE-Backup-modify-job-api.patch
pve/0028-PVE-Backup-add-vma-backup-format-code.patch
pve/0029-PVE-Backup-add-backup-dump-block-driver.patch
pve/0030-PVE-Backup-proxmox-backup-patches-for-qemu.patch
pve/0031-PVE-Backup-aquire-aio_context-before-calling-backup_.patch
pve/0032-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch
pve/0033-PVE-Backup-avoid-coroutines-to-fix-AIO-freeze-cleanu.patch
pve/0034-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
pve/0035-drive-mirror-add-support-for-conditional-and-always-.patch
pve/0036-mirror-add-check-for-bitmap-mode-without-bitmap.patch
pve/0037-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch
pve/0038-iotests-add-test-for-bitmap-mirror.patch
pve/0039-mirror-move-some-checks-to-qmp.patch
pve/0040-PVE-savevm-async-set-up-migration-state.patch
pve/0041-PVE-Backup-avoid-use-QemuRecMutex-inside-coroutines.patch
pve/0042-PVE-Backup-use-QemuMutex-instead-of-QemuRecMutex.patch
pve/0043-move-savevm-async-back-into-a-coroutine.patch
pve/0044-add-optional-buffer-size-to-QEMUFile.patch
pve/0005-PVE-Config-glusterfs-no-default-logfile-if-daemonize.patch
pve/0006-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch
pve/0007-PVE-Up-glusterfs-allow-partial-reads.patch
pve/0008-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch
pve/0009-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch
pve/0010-PVE-Up-qemu-img-dd-add-isize-parameter.patch
pve/0011-PVE-Up-qemu-img-dd-add-n-skip_create.patch
pve/0012-qemu-img-dd-add-l-option-for-loading-a-snapshot.patch
pve/0013-PVE-virtio-balloon-improve-query-balloon.patch
pve/0014-PVE-qapi-modify-query-machines.patch
pve/0015-PVE-qapi-modify-spice-query.patch
pve/0016-PVE-add-IOChannel-implementation-for-savevm-async.patch
pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch
pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
pve/0023-PVE-monitor-disable-oob-capability.patch
pve/0024-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
pve/0025-PVE-Allow-version-code-in-machine-type.patch
pve/0026-block-backup-move-bcs-bitmap-initialization-to-job-c.patch
pve/0027-PVE-Backup-add-vma-backup-format-code.patch
pve/0028-PVE-Backup-add-backup-dump-block-driver.patch
pve/0029-PVE-Add-sequential-job-transaction-support.patch
pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch
pve/0031-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch
pve/0032-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch
pve/0033-PVE-redirect-stderr-to-journal-when-daemonized.patch
pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
pve/0035-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
pve/0036-PVE-fall-back-to-open-iscsi-initiatorname.patch
pve/0037-PVE-block-stream-increase-chunk-size.patch
pve/0038-block-io-accept-NULL-qiov-in-bdrv_pad_request.patch
pve/0039-block-add-alloc-track-driver.patch
pve/0040-Revert-block-rbd-workaround-for-ceph-issue-53784.patch
pve/0041-Revert-block-rbd-fix-handling-of-holes-in-.bdrv_co_b.patch
pve/0042-Revert-block-rbd-implement-bdrv_co_block_status.patch
pve/0043-alloc-track-fix-deadlock-during-drop.patch
pve/0044-migration-for-snapshots-hold-the-BQL-during-setup-ca.patch
pve/0045-savevm-async-don-t-hold-BQL-during-setup.patch
pve-qemu-8.1-vitastor.patch

View File

@@ -1,8 +1,6 @@
# install the userspace utilities
vma usr/bin/
debian/kvm-ifup etc/kvm/
debian/kvm-ifdown etc/kvm/
#install ovmf uefi rom
debian/OVMF_CODE-pure-efi.fd usr/share/kvm/
debian/OVMF_VARS-pure-efi.fd usr/share/kvm/
debian/kvm-ifdown etc/kvm/
# install the userspace utilities
debian/kvm-ifup etc/kvm/

View File

@@ -1,16 +1,13 @@
usr/bin/qemu-system-x86_64 usr/bin/kvm
# qemu-system-i386 and qemu-system-x86_64 provides the same hardware emulation
usr/bin/qemu-system-x86_64 usr/bin/qemu-system-i386
# also use aarch64 for 32 bit arm
usr/bin/qemu-system-aarch64 usr/bin/qemu-system-arm
usr/bin/qemu-system-x86_64 usr/bin/kvm
# qemu-system-i386 and qemu-system-x86_64 provides the same hardware emulation
usr/bin/qemu-system-x86_64 usr/bin/qemu-system-i386
# upstream provides a qemu man page,
# we symlink to kvm for backward compatibility
# and to qemu-system-{i386,x86_64} to fullfill our 'Provides: qemu-system-x86'
usr/share/man/man1/qemu.1 usr/share/man/man1/kvm.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-aarch64.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-arm.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-i386.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-x86_64.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-arm.1
usr/share/man/man1/qemu.1 usr/share/man/man1/qemu-system-aarch64.1

View File

@@ -1,4 +1,7 @@
pve-qemu-kvm: arch-dependent-file-in-usr-share usr/share/kvm/hppa-firmware.img
pve-qemu-kvm: binary-from-other-architecture usr/share/kvm/hppa-firmware.img
pve-qemu-kvm: unstripped-binary-or-object usr/share/kvm/hppa-firmware.img
pve-qemu-kvm: statically-linked-binary usr/share/kvm/hppa-firmware.img
pve-qemu-kvm: arch-dependent-file-in-usr-share [usr/share/kvm/hppa-firmware.img]
pve-qemu-kvm: binary-from-other-architecture [usr/share/kvm/hppa-firmware.img]
pve-qemu-kvm: embedded-javascript-library please use * [usr/share/doc/pve-qemu-kvm/kvm/_static/*]
pve-qemu-kvm: groff-message *: warning [*]: can't break line [usr/share/man/*]
pve-qemu-kvm: groff-message *: warning [*]: cannot adjust line [usr/share/man/*]
pve-qemu-kvm: statically-linked-binary [usr/share/kvm/hppa-firmware.img]
pve-qemu-kvm: unstripped-binary-or-object [usr/share/kvm/hppa-firmware.img]

144
debian/rules vendored
View File

@@ -1,80 +1,89 @@
#!/usr/bin/make -f
# -*- makefile -*-
# Sample debian/rules that uses debhelper.
# This file was originally written by Joey Hess and Craig Small.
# As a special exception, when this file is copied by dh-make into a
# dh-make output file, you may use that output file without restriction.
# This special exception was added by Craig Small in version 0.37 of dh-make.
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
include /usr/share/dpkg/pkg-info.mk
include /usr/share/dpkg/default.mk
# These are used for cross-compiling and for saving the configure script
# from having to guess our platform (since we know it already)
DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
ARCH ?= $(shell dpkg-architecture -qDEB_HOST_GNU_CPU)
HOST_CPU ?= $(DEB_HOST_GNU_CPU)
PACKAGE=pve-qemu-kvm
destdir := $(CURDIR)/debian/$(PACKAGE)
flagfile := $(destdir)/usr/share/kvm/recognized-CPUID-flags-x86_64
machinefile := $(destdir)/usr/share/kvm/machine-versions-x86_64.json
CFLAGS = -Wall
# default QEMU out-of-tree build directory is ./build
BUILDDIR=build
ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
CFLAGS += -O0
else
CFLAGS += -O2
endif
# FIXME: pass to configure as --extra-cflags=CFLAGS ?! also LDFLAGS?
CFLAGS += -Wall
config.status: configure
export CFLAGS
# DEB_BUILD_OPTIONS=parallel=N
MAKEFLAGS += $(subst parallel=,-j,$(filter parallel=%,${DEB_BUILD_OPTIONS}))
${BUILDDIR}/config.status: configure
dh_testdir
# Add here commands to configure the package.
ifneq "$(wildcard /usr/share/misc/config.sub)" ""
cp -f /usr/share/misc/config.sub config.sub
endif
ifneq "$(wildcard /usr/share/misc/config.guess)" ""
cp -f /usr/share/misc/config.guess config.guess
endif
# guest-agent is only required for guest systems
./configure \
--docdir=/usr/share/doc/pve-qemu-kvm \
--localstatedir=/var \
--prefix=/usr \
--sysconfdir=/etc \
--target-list=$(ARCH)-softmmu,aarch64-softmmu \
--with-confsuffix="/kvm" \
--with-pkgversion="${DEB_SOURCE}_${DEB_VERSION_UPSTREAM}" \
--audio-drv-list="alsa" \
--datadir=/usr/share \
--disable-capstone \
--disable-gtk \
--disable-guest-agent \
--disable-guest-agent-msi \
--disable-libnfs \
--disable-libxml2 \
--disable-sdl \
--disable-smartcard \
--disable-strip \
--disable-xen \
--enable-curl \
--enable-docs \
--enable-glusterfs \
--enable-gnutls \
--enable-jemalloc \
--enable-libiscsi \
--enable-libusb \
--enable-linux-aio \
--enable-numa \
--enable-rbd \
--enable-seccomp \
--enable-spice \
--enable-usb-redir \
--enable-virtfs \
--enable-xfsctl
--disable-download \
--docdir=/usr/share/doc/pve-qemu-kvm \
--localstatedir=/var \
--prefix=/usr \
--sysconfdir=/etc \
--target-list=$(HOST_CPU)-softmmu,aarch64-softmmu \
--with-suffix="kvm" \
--with-pkgversion="${DEB_SOURCE}_${DEB_VERSION_UPSTREAM_REVISION}" \
--audio-drv-list="alsa" \
--datadir=/usr/share \
--libexecdir=/usr/lib/kvm \
--disable-capstone \
--disable-gtk \
--disable-guest-agent \
--disable-guest-agent-msi \
--disable-libnfs \
--disable-libssh \
--disable-sdl \
--disable-smartcard \
--disable-strip \
--disable-xen \
--enable-curl \
--enable-docs \
--enable-glusterfs \
--enable-gnutls \
--enable-libiscsi \
--enable-libusb \
--enable-linux-aio \
--enable-linux-io-uring \
--enable-numa \
--enable-opengl \
--enable-rbd \
--enable-seccomp \
--enable-slirp \
--enable-spice \
--enable-usb-redir \
--enable-virglrenderer \
--enable-virtfs \
--enable-zstd
build: build-arch build-indep
build-arch: build-stamp
build-indep: build-stamp
build: build-stamp
build-stamp: config.status
build-stamp: ${BUILDDIR}/config.status
dh_testdir
# Add here commands to compile the package.
@@ -89,15 +98,8 @@ clean:
dh_testroot
rm -f build-stamp
# Add here commands to clean up after the build process.
# Add here commands to clean up before the build process.
-$(MAKE) distclean
ifneq "$(wildcard /usr/share/misc/config.sub)" ""
cp -f /usr/share/misc/config.sub config.sub
endif
ifneq "$(wildcard /usr/share/misc/config.guess)" ""
cp -f /usr/share/misc/config.guess config.guess
endif
dh_clean
@@ -111,25 +113,15 @@ install: build
# Add here commands to install the package into debian/pve-kvm.
$(MAKE) DESTDIR=$(destdir) install
# we do not need openbios files (sparc/ppc)
rm -rf $(destdir)/usr/share/kvm/openbios-*
# remove ppc files
rm $(destdir)/usr/share/kvm/*.dtb
rm $(destdir)/usr/share/kvm/s390-ccw.img
rm $(destdir)/usr/share/kvm/s390-netboot.img
rm $(destdir)/usr/share/kvm/qemu_vga.ndrv
rm $(destdir)/usr/share/kvm/slof.bin
rm $(destdir)/usr/share/kvm/u-boot.e500
# remove Aplha files
rm $(destdir)/usr/share/kvm/palcode-clipper
# Remove things we don't package at all, would be a "kvm-dev" package
rm -Rf $(destdir)/usr/include/linux/
rm -Rf $(destdir)/usr/include
rm -Rf $(destdir)/usr/lib*
rm -f $(destdir)/usr/lib/kvm/qemu-bridge-helper
rm -f $(destdir)/usr/lib/kvm/virtfs-proxy-helper
# CPU flags are static for QEMU version, allows avoiding more costly checks
$(destdir)/usr/bin/qemu-system-x86_64 -cpu help | ./debian/parse-cpu-flags.pl > $(flagfile)
$(destdir)/usr/bin/qemu-system-x86_64 -machine help | ./debian/parse-machines.pl > $(machinefile)
# Build architecture-independent files here.
binary-indep: build install
@@ -156,7 +148,7 @@ binary-arch: build install
# dh_installinfo
dh_installman
dh_link
dh_strip --dbg-package=pve-qemu-kvm-dbg
dh_strip --dbgsym-migration='pve-qemu-kvm-dbg (<<8.0.0-1~)'
dh_compress
dh_fixperms
# dh_perl

3
debian/source/include-binaries vendored Normal file
View File

@@ -0,0 +1,3 @@
debian/OVMF_CODE-pure-efi.fd
debian/OVMF_VARS-pure-efi.fd
debian/Logo.bmp

1
debian/source/lintian-overrides vendored Normal file
View File

@@ -0,0 +1 @@
source-is-missing [roms/SLOF/*.oco]

View File

@@ -1,27 +0,0 @@
Copyright (c) Individual contributors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of PyCA Cryptography nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -1,339 +0,0 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

View File

@@ -1,114 +0,0 @@
Key code / scan code / key symbol mapping database
==================================================
This module provides a database that maps between different
key code / scan code / key symbol sets:
- Linux evdev
- OS-X
- AT Set 1
- AT Set 2
- AT Set 3
- XT
- Linux XT KBD driver
- USB HID
- Win32
- XWin XT
- XKBD XT
- Xorg Evdev
- Xorg KBD
- Xorg OS-X
- XOrg Cygwin
- RFB
Licensing
---------
The contents of this package are dual licensed under the terms of:
- GNU General Public License (version 2 or later)
- 3-clause BSD License
The output files generated by keymap-gen may be distributed & used under
the terms of either of the above licenses.
Data formats
------------
The following output formats are possible
- Code map
An array mapping between key code sets values
Indexes in the array are values from the source code set.
Entries in the array are values from the target code set
- Code table
An array listing all values in a key code set
Indexes in the array are simply a numeric counter
Entries in the array are values from the key code set
The size of the array matches the total number of entries in
the keycode database.
- Name map
An array mapping between key code sets values and names
Indexes in the array are values from the source code set
Entries in the array are names from the target code set
- Name table
An array listing all names in a key code set
Indexes in the array are simply a numeric counter
Entries in the array are values from the key code set
The size of the array matches the total number of entries in
the keycode database.
Output languages
----------------
The tool is capable of generating data tables for the following
programming languages / environments
- Standard C
- GLib2 (standard C, but with GLib2 data types)
- Python
- Perl
Usage
-----
Map values from AT Set 1 to USB HID, generating tables for the
C programming language
$ keymap-gen --lang stdc code-map data/keymaps.csv atset1 usb
Generate a tables of names for Linux key codes, OS-X key codes,
in python - equivalent array indexes map between the two sets.
A variable name override is used
$ keymap-gen --varname linux_keycodes --lang stdc \
code-table data/keymaps.csv linux
$ keymap-gen --varname osx_keycodes --lang stdc \
code-table data/keymaps.csv os-x
Generate a mapping from XOrg XWin values to Win32 names
$ keymap-gen --lang perl name-map data/keymaps.csv xorgxwin win32
Generate a table of names for Linux key codes in Perl
$ keymap-gen --lang perl name-table data/keymaps.csv linux

View File

@@ -1,89 +0,0 @@
This directory contains the raw data for mapping between different
keyboard codes. Naming if often based on the US keyboard layout, but
does not indicate the symbol actually generated by the key.
The columns currently in this data set are:
Linux
-----
Name and value of the hardware independent keycodes used by the linux
kernel and exposed through the input subsystem.
References: linux/input.h
macOS
-----
Low level key codes as exposed by Mac OS X/macOS.
References: Carbon/HIToolbox/Events.h
PC scan code sets
-----------------
Scan codes for the three orignal PC keyboard generations:
Set 1: XT
Set 2: AT
Set 3: PS/2
The sets include codes for modern keys as well and not just the keys
present on those original keyboards.
References: linux/drivers/input/keyboard/atkbd.c
USB HID
-------
Codes as specified by the HID profile in USB.
References: linux/drivers/hid/usbhid/usbkbd.c
Windows Virtual-key codes
-------------------------
The low level, hardware independent "VKEYs" exposed by Windows.
References: mingw32/winuser.h
XWin XT
-------
X11 keycodes generated by the XWin server. Based on the XT scan code
set.
References: xorg-server/hw/xwin/{winkeybd.c,winkeynames.h}
Xfree86 KBD XT
--------------
X11 keycodes generated by the Xfree86 keyboard drivers. Based on the XT
scan code set.
References: xf86-input-keyboard/src/at_scancode.c
X11 keysyms
-----------
Corresponding X11 keysym value(s) for a US keyboard layout.
WARNING: These columns represent symbols, not physical keys, and should
be used with extreme care.
References: http://cgit.freedesktop.org/xorg/proto/x11proto/plain/keysymdef.h
HTML KeyboardEvent.code
-----------------------
Key codes seen in the KeyboardEvent.code attribute as part of the
UI Events specification.
References: https://www.w3.org/TR/uievents-code/
XKEYBOARD key names
-------------------
Hardware independent key names as used in the XKEYBOARD extension.
References: /usr/share/X11/xkb/keycodes/

View File

@@ -1,539 +0,0 @@
"Linux Name","Linux Keycode","OS-X Name","OS-X Keycode","AT set1 keycode","AT set2 keycode","AT set3 keycode","USB Keycodes","Win32 Name","Win32 Keycode","Xwin XT","Xfree86 KBD XT","X11 keysym name","X11 keysym","HTML code","XKB key name","QEMU QKeyCode","Sun KBD","Apple ADB"
KEY_RESERVED,0,,0xff,,,,,,,,,,,,,unmapped,,0xff
KEY_ESC,1,Escape,0x35,0x01,0x76,0x08,41,VK_ESCAPE,0x1b,1,1,XK_Escape,0xff1b,Escape,ESC,esc,0x1d,0x35
KEY_1,2,ANSI_1,0x12,0x02,0x16,0x16,30,VK_1,0x31,2,2,XK_1,0x0031,Digit1,AE01,1,0x1e,0x12
KEY_1,2,ANSI_1,0x12,0x02,0x16,0x16,30,VK_1,0x31,2,2,XK_exclam,0x0021,Digit1,AE01,1,0x1e,0x12
KEY_2,3,ANSI_2,0x13,0x03,0x1e,0x1e,31,VK_2,0x32,3,3,XK_2,0x0032,Digit2,AE02,2,0x1f,0x13
KEY_2,3,ANSI_2,0x13,0x03,0x1e,0x1e,31,VK_2,0x32,3,3,XK_at,0x0040,Digit2,AE02,2,0x1f,0x13
KEY_3,4,ANSI_3,0x14,0x04,0x26,0x26,32,VK_3,0x33,4,4,XK_3,0x0033,Digit3,AE03,3,0x20,0x14
KEY_3,4,ANSI_3,0x14,0x04,0x26,0x26,32,VK_3,0x33,4,4,XK_numbersign,0x0023,Digit3,AE03,3,0x20,0x14
KEY_4,5,ANSI_4,0x15,0x05,0x25,0x25,33,VK_4,0x34,5,5,XK_4,0x0034,Digit4,AE04,4,0x21,0x15
KEY_4,5,ANSI_4,0x15,0x05,0x25,0x25,33,VK_4,0x34,5,5,XK_dollar,0x0024,Digit4,AE04,4,0x21,0x15
KEY_5,6,ANSI_5,0x17,0x06,0x2e,0x2e,34,VK_5,0x35,6,6,XK_5,0x0035,Digit5,AE05,5,0x22,0x17
KEY_5,6,ANSI_5,0x17,0x06,0x2e,0x2e,34,VK_5,0x35,6,6,XK_percent,0x0025,Digit5,AE05,5,0x22,0x17
KEY_6,7,ANSI_6,0x16,0x07,0x36,0x36,35,VK_6,0x36,7,7,XK_6,0x0036,Digit6,AE06,6,0x23,0x16
KEY_6,7,ANSI_6,0x16,0x07,0x36,0x36,35,VK_6,0x36,7,7,XK_asciicircum,0x005e,Digit6,AE06,6,0x23,0x16
KEY_7,8,ANSI_7,0x1a,0x08,0x3d,0x3d,36,VK_7,0x37,8,8,XK_7,0x0037,Digit7,AE07,7,0x24,0x1a
KEY_7,8,ANSI_7,0x1a,0x08,0x3d,0x3d,36,VK_7,0x37,8,8,XK_ampersand,0x0026,Digit7,AE07,7,0x24,0x1a
KEY_8,9,ANSI_8,0x1c,0x09,0x3e,0x3e,37,VK_8,0x38,9,9,XK_8,0x0038,Digit8,AE08,8,0x25,0x1c
KEY_8,9,ANSI_8,0x1c,0x09,0x3e,0x3e,37,VK_8,0x38,9,9,XK_asterisk,0x002a,Digit8,AE08,8,0x25,0x1c
KEY_9,10,ANSI_9,0x19,0x0a,0x46,0x46,38,VK_9,0x39,10,10,XK_9,0x0039,Digit9,AE09,9,0x26,0x19
KEY_9,10,ANSI_9,0x19,0x0a,0x46,0x46,38,VK_9,0x39,10,10,XK_parenleft,0x0028,Digit9,AE09,9,0x26,0x19
KEY_0,11,ANSI_0,0x1d,0x0b,0x45,0x45,39,VK_0,0x30,11,11,XK_0,0x0030,Digit0,AE10,0,0x27,0x1d
KEY_0,11,ANSI_0,0x1d,0x0b,0x45,0x45,39,VK_0,0x30,11,11,XK_parenright,0x0029,Digit0,AE10,0,0x27,0x1d
KEY_MINUS,12,ANSI_Minus,0x1b,0x0c,0x4e,0x4e,45,VK_OEM_MINUS,0xbd,12,12,XK_minus,0x002d,Minus,AE11,minus,0x28,0x1b
KEY_MINUS,12,ANSI_Minus,0x1b,0x0c,0x4e,0x4e,45,VK_OEM_MINUS,0xbd,12,12,XK_underscore,0x005f,Minus,AE11,minus,0x28,0x1b
KEY_EQUAL,13,ANSI_Equal,0x18,0x0d,0x55,0x55,46,VK_OEM_PLUS,0xbb,13,13,XK_equal,0x003d,Equal,AE12,equal,0x29,0x18
KEY_EQUAL,13,ANSI_Equal,0x18,0x0d,0x55,0x55,46,VK_OEM_PLUS,0xbb,13,13,XK_plus,0x002b,Equal,AE12,equal,0x29,0x18
KEY_BACKSPACE,14,Delete,0x33,0x0e,0x66,0x66,42,VK_BACK,0x08,14,14,XK_BackSpace,0xff08,Backspace,BKSP,backspace,0x2b,0x33
KEY_TAB,15,Tab,0x30,0x0f,0x0d,0x0d,43,VK_TAB,0x09,15,15,XK_Tab,0xff09,Tab,TAB,tab,0x35,0x30
KEY_Q,16,ANSI_Q,0xc,0x10,0x15,0x15,20,VK_Q,0x51,16,16,XK_Q,0x0051,KeyQ,AD01,q,0x36,0xc
KEY_Q,16,ANSI_Q,0xc,0x10,0x15,0x15,20,VK_Q,0x51,16,16,XK_q,0x0071,KeyQ,AD01,q,0x36,0xc
KEY_W,17,ANSI_W,0xd,0x11,0x1d,0x1d,26,VK_W,0x57,17,17,XK_W,0x0057,KeyW,AD02,w,0x37,0xd
KEY_W,17,ANSI_W,0xd,0x11,0x1d,0x1d,26,VK_W,0x57,17,17,XK_w,0x0077,KeyW,AD02,w,0x37,0xd
KEY_E,18,ANSI_E,0xe,0x12,0x24,0x24,8,VK_E,0x45,18,18,XK_E,0x0045,KeyE,AD03,e,0x38,0xe
KEY_E,18,ANSI_E,0xe,0x12,0x24,0x24,8,VK_E,0x45,18,18,XK_e,0x0065,KeyE,AD03,e,0x38,0xe
KEY_R,19,ANSI_R,0xf,0x13,0x2d,0x2d,21,VK_R,0x52,19,19,XK_R,0x0052,KeyR,AD04,r,0x39,0xf
KEY_R,19,ANSI_R,0xf,0x13,0x2d,0x2d,21,VK_R,0x52,19,19,XK_r,0x0072,KeyR,AD04,r,0x39,0xf
KEY_T,20,ANSI_T,0x11,0x14,0x2c,0x2c,23,VK_T,0x54,20,20,XK_T,0x0054,KeyT,AD05,t,0x3a,0x11
KEY_T,20,ANSI_T,0x11,0x14,0x2c,0x2c,23,VK_T,0x54,20,20,XK_t,0x0074,KeyT,AD05,t,0x3a,0x11
KEY_Y,21,ANSI_Y,0x10,0x15,0x35,0x35,28,VK_Y,0x59,21,21,XK_Y,0x0059,KeyY,AD06,y,0x3b,0x10
KEY_Y,21,ANSI_Y,0x10,0x15,0x35,0x35,28,VK_Y,0x59,21,21,XK_y,0x0079,KeyY,AD06,y,0x3b,0x10
KEY_U,22,ANSI_U,0x20,0x16,0x3c,0x3c,24,VK_U,0x55,22,22,XK_U,0x0055,KeyU,AD07,u,0x3c,0x20
KEY_U,22,ANSI_U,0x20,0x16,0x3c,0x3c,24,VK_U,0x55,22,22,XK_u,0x0075,KeyU,AD07,u,0x3c,0x20
KEY_I,23,ANSI_I,0x22,0x17,0x43,0x43,12,VK_I,0x49,23,23,XK_I,0x0049,KeyI,AD08,i,0x3d,0x22
KEY_I,23,ANSI_I,0x22,0x17,0x43,0x43,12,VK_I,0x49,23,23,XK_i,0x0069,KeyI,AD08,i,0x3d,0x22
KEY_O,24,ANSI_O,0x1f,0x18,0x44,0x44,18,VK_O,0x4f,24,24,XK_O,0x004f,KeyO,AD09,o,0x3e,0x1f
KEY_O,24,ANSI_O,0x1f,0x18,0x44,0x44,18,VK_O,0x4f,24,24,XK_o,0x006f,KeyO,AD09,o,0x3e,0x1f
KEY_P,25,ANSI_P,0x23,0x19,0x4d,0x4d,19,VK_P,0x50,25,25,XK_P,0x0050,KeyP,AD10,p,0x3f,0x23
KEY_P,25,ANSI_P,0x23,0x19,0x4d,0x4d,19,VK_P,0x50,25,25,XK_p,0x0070,KeyP,AD10,p,0x3f,0x23
KEY_LEFTBRACE,26,ANSI_LeftBracket,0x21,0x1a,0x54,0x54,47,VK_OEM_4,0xdb,26,26,XK_bracketleft,0x005b,BracketLeft,AD11,bracket_left,0x40,0x21
KEY_LEFTBRACE,26,ANSI_LeftBracket,0x21,0x1a,0x54,0x54,47,VK_OEM_4,0xdb,26,26,XK_braceleft,0x007b,BracketLeft,AD11,bracket_left,0x40,0x21
KEY_RIGHTBRACE,27,ANSI_RightBracket,0x1e,0x1b,0x5b,0x5b,48,VK_OEM_6,0xdd,27,27,XK_bracketright,0x005d,BracketRight,AD12,bracket_right,0x41,0x1e
KEY_RIGHTBRACE,27,ANSI_RightBracket,0x1e,0x1b,0x5b,0x5b,48,VK_OEM_6,0xdd,27,27,XK_braceright,0x007d,BracketRight,AD12,bracket_right,0x41,0x1e
KEY_ENTER,28,Return,0x24,0x1c,0x5a,0x5a,40,VK_RETURN,0x0d,28,28,XK_Return,0xff0d,Enter,RTRN,ret,0x59,0x24
KEY_LEFTCTRL,29,Control,0x3b,0x1d,0x14,0x11,224,VK_LCONTROL,0xa2,29,29,XK_Control_L,0xffe3,ControlLeft,LCTL,ctrl,0x4c,0x36
KEY_LEFTCTRL,29,Control,0x3b,0x1d,0x14,0x11,224,VK_CONTROL,0x11,29,29,XK_Control_L,0xffe3,ControlLeft,LCTL,ctrl,0x4c,0x36
KEY_A,30,ANSI_A,0x0,0x1e,0x1c,0x1c,4,VK_A,0x41,30,30,XK_A,0x0041,KeyA,AC01,a,0x4d,0x0
KEY_A,30,ANSI_A,0x0,0x1e,0x1c,0x1c,4,VK_A,0x41,30,30,XK_a,0x0061,KeyA,AC01,a,0x4d,0x0
KEY_S,31,ANSI_S,0x1,0x1f,0x1b,0x1b,22,VK_S,0x53,31,31,XK_S,0x0053,KeyS,AC02,s,0x4e,0x1
KEY_S,31,ANSI_S,0x1,0x1f,0x1b,0x1b,22,VK_S,0x53,31,31,XK_s,0x0073,KeyS,AC02,s,0x4e,0x1
KEY_D,32,ANSI_D,0x2,0x20,0x23,0x23,7,VK_D,0x44,32,32,XK_D,0x0044,KeyD,AC03,d,0x4f,0x2
KEY_D,32,ANSI_D,0x2,0x20,0x23,0x23,7,VK_D,0x44,32,32,XK_d,0x0064,KeyD,AC03,d,0x4f,0x2
KEY_F,33,ANSI_F,0x3,0x21,0x2b,0x2b,9,VK_F,0x46,33,33,XK_F,0x0046,KeyF,AC04,f,0x50,0x3
KEY_F,33,ANSI_F,0x3,0x21,0x2b,0x2b,9,VK_F,0x46,33,33,XK_f,0x0066,KeyF,AC04,f,0x50,0x3
KEY_G,34,ANSI_G,0x5,0x22,0x34,0x34,10,VK_G,0x47,34,34,XK_G,0x0047,KeyG,AC05,g,0x51,0x5
KEY_G,34,ANSI_G,0x5,0x22,0x34,0x34,10,VK_G,0x47,34,34,XK_g,0x0067,KeyG,AC05,g,0x51,0x5
KEY_H,35,ANSI_H,0x4,0x23,0x33,0x33,11,VK_H,0x48,35,35,XK_H,0x0048,KeyH,AC06,h,0x52,0x4
KEY_H,35,ANSI_H,0x4,0x23,0x33,0x33,11,VK_H,0x48,35,35,XK_h,0x0068,KeyH,AC06,h,0x52,0x4
KEY_J,36,ANSI_J,0x26,0x24,0x3b,0x3b,13,VK_J,0x4a,36,36,XK_J,0x004a,KeyJ,AC07,j,0x53,0x26
KEY_J,36,ANSI_J,0x26,0x24,0x3b,0x3b,13,VK_J,0x4a,36,36,XK_j,0x006a,KeyJ,AC07,j,0x53,0x26
KEY_K,37,ANSI_K,0x28,0x25,0x42,0x42,14,VK_K,0x4b,37,37,XK_K,0x004b,KeyK,AC08,k,0x54,0x28
KEY_K,37,ANSI_K,0x28,0x25,0x42,0x42,14,VK_K,0x4b,37,37,XK_k,0x006b,KeyK,AC08,k,0x54,0x28
KEY_L,38,ANSI_L,0x25,0x26,0x4b,0x4b,15,VK_L,0x4c,38,38,XK_L,0x004c,KeyL,AC09,l,0x55,0x25
KEY_L,38,ANSI_L,0x25,0x26,0x4b,0x4b,15,VK_L,0x4c,38,38,XK_l,0x006c,KeyL,AC09,l,0x55,0x25
KEY_SEMICOLON,39,ANSI_Semicolon,0x29,0x27,0x4c,0x4c,51,VK_OEM_1,0xba,39,39,XK_semicolon,0x003b,Semicolon,AC10,semicolon,0x56,0x29
KEY_SEMICOLON,39,ANSI_Semicolon,0x29,0x27,0x4c,0x4c,51,VK_OEM_1,0xba,39,39,XK_colon,0x003a,Semicolon,AC10,semicolon,0x56,0x29
KEY_APOSTROPHE,40,ANSI_Quote,0x27,0x28,0x52,0x52,52,VK_OEM_7,0xde,40,40,XK_apostrophe,0x0027,Quote,AC11,apostrophe,0x57,0x27
KEY_APOSTROPHE,40,ANSI_Quote,0x27,0x28,0x52,0x52,52,VK_OEM_7,0xde,40,40,XK_quotedbl,0x0022,Quote,AC11,apostrophe,0x57,0x27
KEY_GRAVE,41,ANSI_Grave,0x32,0x29,0x0e,0x0e,53,VK_OEM_3,0xc0,41,41,XK_grave,0x0060,Backquote,TLDE,grave_accent,0x2a,0x32
KEY_GRAVE,41,ANSI_Grave,0x32,0x29,0x0e,0x0e,53,VK_OEM_3,0xc0,41,41,XK_grave,0x0060,Backquote,AB00,grave_accent,0x2a,0x32
KEY_GRAVE,41,ANSI_Grave,0x32,0x29,0x0e,0x0e,53,VK_OEM_3,0xc0,41,41,XK_asciitilde,0x007e,Backquote,TLDE,grave_accent,0x2a,0x32
KEY_GRAVE,41,ANSI_Grave,0x32,0x29,0x0e,0x0e,53,VK_OEM_3,0xc0,41,41,XK_asciitilde,0x007e,Backquote,AB00,grave_accent,0x2a,0x32
KEY_SHIFT,42,Shift,0x38,0x2a,0x12,0x12,225,VK_SHIFT,0x10,42,42,XK_Shift_L,0xffe1,ShiftLeft,LFSH,shift,0x63,0x38
KEY_LEFTSHIFT,42,Shift,0x38,0x2a,0x12,0x12,225,VK_LSHIFT,0xa0,42,42,XK_Shift_L,0xffe1,ShiftLeft,LFSH,shift,0x63,0x38
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,49,VK_OEM_5,0xdc,43,43,XK_backslash,0x005c,Backslash,BKSL,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,49,VK_OEM_5,0xdc,43,43,XK_backslash,0x005c,Backslash,AC12,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,49,VK_OEM_5,0xdc,43,43,XK_bar,0x007c,Backslash,BKSL,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,49,VK_OEM_5,0xdc,43,43,XK_bar,0x007c,Backslash,AC12,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,50,VK_OEM_5,0xdc,43,43,XK_backslash,0x005c,Backslash,BKSL,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,50,VK_OEM_5,0xdc,43,43,XK_backslash,0x005c,Backslash,AC12,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,50,VK_OEM_5,0xdc,43,43,XK_bar,0x007c,Backslash,BKSL,backslash,0x58,0x2a
KEY_BACKSLASH,43,ANSI_Backslash,0x2a,0x2b,0x5d,0x5c,50,VK_OEM_5,0xdc,43,43,XK_bar,0x007c,Backslash,AC12,backslash,0x58,0x2a
KEY_Z,44,ANSI_Z,0x6,0x2c,0x1a,0x1a,29,VK_Z,0x5a,44,44,XK_Z,0x005a,KeyZ,AB01,z,0x64,0x6
KEY_Z,44,ANSI_Z,0x6,0x2c,0x1a,0x1a,29,VK_Z,0x5a,44,44,XK_z,0x007a,KeyZ,AB01,z,0x64,0x6
KEY_X,45,ANSI_X,0x7,0x2d,0x22,0x22,27,VK_X,0x58,45,45,XK_X,0x0058,KeyX,AB02,x,0x65,0x7
KEY_X,45,ANSI_X,0x7,0x2d,0x22,0x22,27,VK_X,0x58,45,45,XK_x,0x0078,KeyX,AB02,x,0x65,0x7
KEY_C,46,ANSI_C,0x8,0x2e,0x21,0x21,6,VK_C,0x43,46,46,XK_C,0x0043,KeyC,AB03,c,0x66,0x8
KEY_C,46,ANSI_C,0x8,0x2e,0x21,0x21,6,VK_C,0x43,46,46,XK_c,0x0063,KeyC,AB03,c,0x66,0x8
KEY_V,47,ANSI_V,0x9,0x2f,0x2a,0x2a,25,VK_V,0x56,47,47,XK_V,0x0056,KeyV,AB04,v,0x67,0x9
KEY_V,47,ANSI_V,0x9,0x2f,0x2a,0x2a,25,VK_V,0x56,47,47,XK_v,0x0076,KeyV,AB04,v,0x67,0x9
KEY_B,48,ANSI_B,0xb,0x30,0x32,0x32,5,VK_B,0x42,48,48,XK_B,0x0042,KeyB,AB05,b,0x68,0xb
KEY_B,48,ANSI_B,0xb,0x30,0x32,0x32,5,VK_B,0x42,48,48,XK_b,0x0062,KeyB,AB05,b,0x68,0xb
KEY_N,49,ANSI_N,0x2d,0x31,0x31,0x31,17,VK_N,0x4e,49,49,XK_N,0x004e,KeyN,AB06,n,0x69,0x2d
KEY_N,49,ANSI_N,0x2d,0x31,0x31,0x31,17,VK_N,0x4e,49,49,XK_n,0x006e,KeyN,AB06,n,0x69,0x2d
KEY_M,50,ANSI_M,0x2e,0x32,0x3a,0x3a,16,VK_M,0x4d,50,50,XK_M,0x004d,KeyM,AB07,m,0x6a,0x2e
KEY_M,50,ANSI_M,0x2e,0x32,0x3a,0x3a,16,VK_M,0x4d,50,50,XK_m,0x006d,KeyM,AB07,m,0x6a,0x2e
KEY_COMMA,51,ANSI_Comma,0x2b,0x33,0x41,0x41,54,VK_OEM_COMMA,0xbc,51,51,XK_comma,0x002c,Comma,AB08,comma,0x6b,0x2b
KEY_COMMA,51,ANSI_Comma,0x2b,0x33,0x41,0x41,54,VK_OEM_COMMA,0xbc,51,51,XK_less,0x003c,Comma,AB08,comma,0x6b,0x2b
KEY_DOT,52,ANSI_Period,0x2f,0x34,0x49,0x49,55,VK_OEM_PERIOD,0xbe,52,52,XK_period,0x002e,Period,AB09,dot,0x6c,0x2f
KEY_DOT,52,ANSI_Period,0x2f,0x34,0x49,0x49,55,VK_OEM_PERIOD,0xbe,52,52,XK_greater,0x003e,Period,AB09,dot,0x6c,0x2f
KEY_SLASH,53,ANSI_Slash,0x2c,0x35,0x4a,0x4a,56,VK_OEM_2,0xbf,53,53,XK_slash,0x002f,Slash,AB10,slash,0x6d,0x2c
KEY_SLASH,53,ANSI_Slash,0x2c,0x35,0x4a,0x4a,56,VK_OEM_2,0xbf,53,53,XK_question,0x003f,Slash,AB10,slash,0x6d,0x2c
KEY_RIGHTSHIFT,54,RightShift,0x3c,0x36,0x59,0x59,229,VK_RSHIFT,0xa1,54,54,XK_Shift_R,0xffe2,ShiftRight,RTSH,shift_r,0x6e,0x7b
KEY_KPASTERISK,55,ANSI_KeypadMultiply,0x43,0x37,0x7c,0x7e,85,VK_MULTIPLY,0x6a,55,55,XK_multiply,0x00d7,NumpadMultiply,KPMU,asterisk,0x2f,0x43
KEY_KPASTERISK,55,ANSI_KeypadMultiply,0x43,0x37,0x7c,0x7e,85,VK_MULTIPLY,0x6a,55,55,XK_multiply,0x00d7,NumpadMultiply,KPMU,kp_multiply,0x2f,0x43
KEY_LEFTALT,56,Option,0x3a,0x38,0x11,0x19,226,VK_LMENU,0xa4,56,56,XK_Alt_L,0xffe9,AltLeft,LALT,alt,0x13,0x3a
KEY_LEFTALT,56,Option,0x3a,0x38,0x11,0x19,226,VK_MENU,0x12,56,56,XK_Alt_L,0xffe9,AltLeft,LALT,alt,0x13,0x3a
KEY_SPACE,57,Space,0x31,0x39,0x29,0x29,44,VK_SPACE,0x20,57,57,XK_space,0x0020,Space,SPCE,spc,0x79,0x31
KEY_CAPSLOCK,58,CapsLock,0x39,0x3a,0x58,0x14,57,VK_CAPITAL,0x14,58,58,XK_Caps_Lock,0xffe5,CapsLock,CAPS,caps_lock,0x77,0x39
KEY_F1,59,F1,0x7a,0x3b,0x05,0x07,58,VK_F1,0x70,59,59,XK_F1,0xffbe,F1,FK01,f1,0x05,0x7a
KEY_F2,60,F2,0x78,0x3c,0x06,0x0f,59,VK_F2,0x71,60,60,XK_F2,0xffbf,F2,FK02,f2,0x06,0x78
KEY_F3,61,F3,0x63,0x3d,0x04,0x17,60,VK_F3,0x72,61,61,XK_F3,0xffc0,F3,FK03,f3,0x08,0x63
KEY_F4,62,F4,0x76,0x3e,0x0c,0x1f,61,VK_F4,0x73,62,62,XK_F4,0xffc1,F4,FK04,f4,0x0a,0x76
KEY_F5,63,F5,0x60,0x3f,0x03,0x27,62,VK_F5,0x74,63,63,XK_F5,0xffc2,F5,FK05,f5,0x0c,0x60
KEY_F6,64,F6,0x61,0x40,0x0b,0x2f,63,VK_F6,0x75,64,64,XK_F6,0xffc3,F6,FK06,f6,0x0e,0x61
KEY_F7,65,F7,0x62,0x41,0x83,0x37,64,VK_F7,0x76,65,65,XK_F7,0xffc4,F7,FK07,f7,0x10,0x62
KEY_F8,66,F8,0x64,0x42,0x0a,0x3f,65,VK_F8,0x77,66,66,XK_F8,0xffc5,F8,FK08,f8,0x11,0x64
KEY_F9,67,F9,0x65,0x43,0x01,0x47,66,VK_F9,0x78,67,67,XK_F9,0xffc6,F9,FK09,f9,0x12,0x65
KEY_F10,68,F10,0x6d,0x44,0x09,0x4f,67,VK_F10,0x79,68,68,XK_F10,0xffc7,F10,FK10,f10,0x07,0x6d
KEY_NUMLOCK,69,ANSI_KeypadClear,0x47,0x45,0x77,0x76,83,VK_NUMLOCK,0x90,69,69,XK_Num_Lock,0xff7f,NumLock,NMLK,num_lock,0x62,0x47
KEY_SCROLLLOCK,70,,,0x46,0x7e,0x5f,71,VK_SCROLL,0x91,70,70,XK_Scroll_Lock,0xff14,ScrollLock,SCLK,scroll_lock,0x17,0x6b
KEY_KP7,71,ANSI_Keypad7,0x59,0x47,0x6c,0x6c,95,VK_NUMPAD7,0x67,71,71,XK_KP_7,0xffb7,Numpad7,KP7,kp_7,0x44,0x59
KEY_KP8,72,ANSI_Keypad8,0x5b,0x48,0x75,0x75,96,VK_NUMPAD8,0x68,72,72,XK_KP_8,0xffb8,Numpad8,KP8,kp_8,0x45,0x5b
KEY_KP9,73,ANSI_Keypad9,0x5c,0x49,0x7d,0x7d,97,VK_NUMPAD9,0x69,73,73,XK_KP_9,0xffb9,Numpad9,KP9,kp_9,0x46,0x5c
KEY_KPMINUS,74,ANSI_KeypadMinus,0x4e,0x4a,0x7b,0x4e,86,VK_SUBTRACT,0x6d,74,74,XK_KP_Subtract,0xffad,NumpadSubtract,KPSU,kp_subtract,0x47,0x4e
KEY_KP4,75,ANSI_Keypad4,0x56,0x4b,0x6b,0x6b,92,VK_NUMPAD4,0x64,75,75,XK_KP_4,0xffb4,Numpad4,KP4,kp_4,0x5b,0x56
KEY_KP5,76,ANSI_Keypad5,0x57,0x4c,0x73,0x73,93,VK_NUMPAD5,0x65,76,76,XK_KP_5,0xffb5,Numpad5,KP5,kp_5,0x5c,0x57
KEY_KP6,77,ANSI_Keypad6,0x58,0x4d,0x74,0x74,94,VK_NUMPAD6,0x66,77,77,XK_KP_6,0xffb6,Numpad6,KP6,kp_6,0x5d,0x58
KEY_KPPLUS,78,ANSI_KeypadPlus,0x45,0x4e,0x79,0x7c,87,VK_ADD,0x6b,78,78,XK_KP_Add,0xffab,NumpadAdd,KPAD,kp_add,0x7d,0x45
KEY_KP1,79,ANSI_Keypad1,0x53,0x4f,0x69,0x69,89,VK_NUMPAD1,0x61,79,79,XK_KP_1,0xffb1,Numpad1,KP1,kp_1,0x70,0x53
KEY_KP2,80,ANSI_Keypad2,0x54,0x50,0x72,0x72,90,VK_NUMPAD2,0x62,80,80,XK_KP_2,0xffb2,Numpad2,KP2,kp_2,0x71,0x54
KEY_KP3,81,ANSI_Keypad3,0x55,0x51,0x7a,0x7a,91,VK_NUMPAD3,0x63,81,81,XK_KP_3,0xffb3,Numpad3,KP3,kp_3,0x72,0x55
KEY_KP0,82,ANSI_Keypad0,0x52,0x52,0x70,0x70,98,VK_NUMPAD0,0x60,82,82,XK_KP_0,0xffb0,Numpad0,KP0,kp_0,0x5e,0x52
KEY_KPDOT,83,ANSI_KeypadDecimal,0x41,0x53,0x71,0x71,99,VK_DECIMAL,0x6e,83,83,XK_KP_Decimal,0xffae,NumpadDecimal,KPDL,kp_decimal,0x32,0x41
KEY_KPDOT,83,ANSI_KeypadDecimal,0x41,0x53,0x71,0x71,99,VK_DECIMAL,0x6e,83,83,XK_KP_Decimal,0xffae,NumpadDecimal,KPDC,kp_decimal,0x32,0x41
,84,,,0x54,,,,,,,,,,,,,,
KEY_ZENKAKUHANKAKU,85,,,0x76,0x5f,,148,,,,,,,Lang5,HZTG,,,
KEY_102ND,86,,,0x56,0x61,0x13,100,VK_OEM_102,0xe1,86,86,,,IntlBackslash,LSGT,less,0x7c,
KEY_F11,87,F11,0x67,0x57,0x78,0x56,68,VK_F11,0x7a,87,87,XK_F11,0xffc8,F11,FK11,f11,0x09,0x67
KEY_F12,88,F12,0x6f,0x58,0x07,0x5e,69,VK_F12,0x7b,88,88,XK_F12,0xffc9,F12,FK12,f12,0x0b,0x6f
KEY_RO,89,,,0x73,0x51,,135,,,,,,,IntlRo,AB11,ro,,
KEY_KATAKANA,90,JIS_Kana,0x68,0x78,0x63,,146,VK_KANA,0x15,,,,,Katakana,KATA,,,
KEY_KATAKANA,90,JIS_Kana,0x68,0x78,0x63,,146,VK_KANA,0x15,,,,,Lang3,KATA,,,
KEY_HIRAGANA,91,,,0x77,0x62,0x87,147,,,,,,,Hiragana,HIRA,hiragana,,
KEY_HIRAGANA,91,,,0x77,0x62,0x87,147,,,,,,,Lang4,HIRA,hiragana,,
KEY_HENKAN,92,,,0x79,0x64,0x86,138,,,,,,,Convert,HENK,henkan,,
KEY_KATAKANAHIRAGANA,93,,,0x70,0x13,0x87,136,,,0xc8,0xc8,,,KanaMode,HKTG,katakanahiragana,,
KEY_MUHENKAN,94,,,0x7b,0x67,0x85,139,,,,,,,NonConvert,NFER,muhenkan,,
KEY_MUHENKAN,94,,,0x7b,0x67,0x85,139,,,,,,,NonConvert,MUHE,muhenkan,,
KEY_KPJPCOMMA,95,JIS_KeypadComma,0x5f,0x5c,0x27,,140,,,,,XK_KP_Separator,0xffac,,KPSP,,,
KEY_KPJPCOMMA,95,JIS_KeypadComma,0x5f,0x5c,0x27,,140,,,,,XK_KP_Separator,0xffac,,JPCM,,,
KEY_KPENTER,96,ANSI_KeypadEnter,0x4c,0xe01c,0xe05a,0x79,88,,,0x64,0x64,XK_KP_Enter,0xff8d,NumpadEnter,KPEN,kp_enter,0x5a,0x4c
KEY_RIGHTCTRL,97,RightControl,0x3e,0xe01d,0xe014,0x58,228,VK_RCONTROL,0xa3,0x65,0x65,XK_Control_R,0xffe4,ControlRight,RCTL,ctrl_r,0x4c,0x7d
KEY_KPSLASH,98,ANSI_KeypadDivide,0x4b,0xe035,0xe04a,0x4a,84,VK_DIVIDE,0x6f,0x68,0x68,XK_KP_Divide,0xffaf,NumpadDivide,KPDV,kp_divide,0x2e,0x4b
KEY_SYSRQ,99,,,0x54,0x7f,0x57,70,VK_SNAPSHOT,0x2c,0x67,0x67,XK_Sys_Req,0xff15,PrintScreen,PRSC,print,0x16,0x69
KEY_SYSRQ,99,,,0x54,0x7f,0x57,70,VK_SNAPSHOT,0x2c,0x67,0x67,XK_Sys_Req,0xff15,PrintScreen,SYRQ,sysrq,0x16,0x69
KEY_RIGHTALT,100,RightOption,0x3d,0xe038,0xe011,0x39,230,VK_RMENU,0xa5,0x69,0x69,XK_Alt_R,0xffea,AltRight,ALGR,alt_r,0x0d,0x7c
KEY_RIGHTALT,100,RightOption,0x3d,0xe038,0xe011,0x39,230,VK_RMENU,0xa5,0x69,0x69,XK_Alt_R,0xffea,AltRight,RALT,alt_r,0x0d,0x7c
KEY_LINEFEED,101,,,0x5b,,,,,,,,,,,LNFD,lf,0x6f,
KEY_HOME,102,Home,0x73,0xe047,0xe06c,0x6e,74,VK_HOME,0x24,0x59,0x59,XK_Home,0xff50,Home,HOME,home,0x34,0x73
KEY_UP,103,UpArrow,0x7e,0xe048,0xe075,0x63,82,VK_UP,0x26,0x5a,0x5a,XK_Up,0xff52,ArrowUp,UP,up,0x14,0x3e
KEY_PAGEUP,104,PageUp,0x74,0xe049,0xe07d,0x6f,75,VK_PRIOR,0x21,0x5b,0x5b,XK_Page_Up,0xff55,PageUp,PGUP,pgup,0x60,0x74
KEY_LEFT,105,LeftArrow,0x7b,0xe04b,0xe06b,0x61,80,VK_LEFT,0x25,0x5c,0x5c,XK_Left,0xff51,ArrowLeft,LEFT,left,0x18,0x3b
KEY_RIGHT,106,RightArrow,0x7c,0xe04d,0xe074,0x6a,79,VK_RIGHT,0x27,0x5e,0x5e,XK_Right,0xff53,ArrowRight,RGHT,right,0x1c,0x3c
KEY_END,107,End,0x77,0xe04f,0xe069,0x65,77,VK_END,0x23,0x5f,0x5f,XK_End,0xff57,End,END,end,0x4a,0x77
KEY_DOWN,108,DownArrow,0x7d,0xe050,0xe072,0x60,81,VK_DOWN,0x28,0x60,0x60,XK_Down,0xff54,ArrowDown,DOWN,down,0x1b,0x3d
KEY_PAGEDOWN,109,PageDown,0x79,0xe051,0xe07a,0x6d,78,VK_NEXT,0x22,0x61,0x61,XK_Page_Down,0xff56,PageDown,PGDN,pgdn,0x7b,0x79
KEY_INSERT,110,,,0xe052,0xe070,0x67,73,VK_INSERT,0x2d,0x62,0x62,XK_Insert,0xff63,Insert,INS,insert,0x2c,0x72
KEY_DELETE,111,ForwardDelete,0x75,0xe053,0xe071,0x64,76,VK_DELETE,0x2e,0x63,0x63,XK_Delete,0xffff,Delete,DEL,delete,0x42,0x75
KEY_DELETE,111,ForwardDelete,0x75,0xe053,0xe071,0x64,76,VK_DELETE,0x2e,0x63,0x63,XK_Delete,0xffff,Delete,DELE,,0x42,0x75
KEY_MACRO,112,,,0xe06f,0xe06f,0x8e,,,,,,,,,I120,,,
KEY_MUTE,113,Mute,0x4a,0xe020,0xe023,0x9c,127,VK_VOLUME_MUTE,0xad,,,,,AudioVolumeMute,MUTE,audiomute,,
KEY_MUTE,113,Mute,0x4a,0xe020,0xe023,0x9c,239,VK_VOLUME_MUTE,0xad,,,,,AudioVolumeMute,MUTE,audiomute,,
KEY_VOLUMEDOWN,114,VolumeDown,0x49,0xe02e,0xe021,0x9d,129,VK_VOLUME_DOWN,0xae,,,,,AudioVolumeDown,VOL-,volumedown,,
KEY_VOLUMEDOWN,114,VolumeDown,0x49,0xe02e,0xe021,0x9d,238,VK_VOLUME_DOWN,0xae,,,,,AudioVolumeDown,VOL-,volumedown,,
KEY_VOLUMEUP,115,VolumeUp,0x48,0xe030,0xe032,0x95,128,VK_VOLUME_UP,0xaf,,,,,AudioVolumeUp,VOL+,volumeup,,
KEY_VOLUMEUP,115,VolumeUp,0x48,0xe030,0xe032,0x95,237,VK_VOLUME_UP,0xaf,,,,,AudioVolumeUp,VOL+,volumeup,,
KEY_POWER,116,,,0xe05e,0xe037,,102,,,,,,,Power,POWR,power,,0x7f7f
KEY_KPEQUAL,117,ANSI_KeypadEquals,0x51,0x59,0x0f,,103,,,0x76,0x76,XK_KP_Equal,0xffbd,NumpadEqual,KPEQ,kp_equals,0x2d,0x51
KEY_KPPLUSMINUS,118,,,0xe04e,0xe079,,,,,,,,,,I126,,,
KEY_PAUSE,119,,,0xe046,0xe077,0x62,72,VK_PAUSE,0x013,0x66,0x66,XK_Pause,0xff13,Pause,PAUS,pause,0x15,0x71
KEY_SCALE,120,,,0xe00b,,,,,,,,,,,I128,,,
KEY_KPCOMMA,121,,,0x7e,0x6d,,133,VK_SEPARATOR??,0x6c,,,,,NumpadComma,KPCO,kp_comma,,
KEY_KPCOMMA,121,,,0x7e,0x6d,,133,VK_SEPARATOR??,0x6c,,,,,NumpadComma,I129,,,
KEY_HANGEUL,122,,,,,,144,VK_HANGEUL,0x15,,,,,,HNGL,,,
KEY_HANJA,123,,,0xe00d,,,145,VK_HANJA,0x19,,,,,,HJCV,,,
KEY_YEN,124,JIS_Yen,0x5d,0x7d,0x6a,0x5d,137,,,0x7d,0x7d,,,IntlYen,AE13,yen,,
KEY_LEFTMETA,125,Command,0x37,0xe05b,0xe01f,0x8b,227,VK_LWIN,0x5b,0x6b,0x6b,XK_Meta_L,0xffe7,MetaLeft,LMTA,meta_l,0x78,0x37
KEY_LEFTMETA,125,Command,0x37,0xe05b,0xe01f,0x8b,227,VK_LWIN,0x5b,0x6b,0x6b,XK_Meta_L,0xffe7,MetaLeft,LWIN,meta_l,0x78,0x37
KEY_RIGHTMETA,126,RightCommand,0x36,0xe05c,0xe027,0x8c,231,VK_RWIN,0x5c,0x6c,0x6c,XK_Meta_R,0xffe8,MetaRight,RMTA,meta_r,0x7a,0x37
KEY_RIGHTMETA,126,RightCommand,0x36,0xe05c,0xe027,0x8c,231,VK_RWIN,0x5c,0x6c,0x6c,XK_Meta_R,0xffe8,MetaRight,RWIN,meta_r,0x7a,0x37
KEY_COMPOSE,127,,0x6e,0xe05d,0xe02f,0x8d,101,VK_APPS,0x5d,0x6d,0x6d,,,ContextMenu,MENU,compose,0x43,
KEY_COMPOSE,127,,0x6e,0xe05d,0xe02f,0x8d,101,VK_APPS,0x5d,0x6d,0x6d,,,ContextMenu,COMP,compose,0x43,
KEY_STOP,128,,,0xe068,0xe028,0x0a,120,VK_BROWSER_STOP,0xa9,,,,,BrowserStop,STOP,stop,0x01,
KEY_STOP,128,,,0xe068,0xe028,0x0a,243,VK_BROWSER_STOP,0xa9,,,,,BrowserStop,STOP,stop,0x01,
KEY_AGAIN,129,,,0xe005,,0x0b,121,,,,,,,Again,AGAI,again,0x03,
KEY_PROPS,130,,,0xe006,,0x0c,,,,,,,,Props,PROP,props,0x19,
KEY_UNDO,131,,,0xe007,,0x10,122,,,,,,,Undo,UNDO,undo,0x1a,
KEY_FRONT,132,,,0xe00c,,,119,,,,,,,,FRNT,front,0x31,
KEY_COPY,133,,,0xe078,,0x18,124,,,,,,,Copy,COPY,copy,0x33,
KEY_OPEN,134,,,0x64,,0x20,116,,,,,,,Open,OPEN,open,0x48,
KEY_PASTE,135,,,0x65,,0x28,125,,,,,,,Paste,PAST,paste,0x49,
KEY_FIND,136,,,0xe041,,0x30,126,,,,,,,Find,FIND,find,0x5f,
KEY_FIND,136,,,0xe041,,0x30,244,,,,,,,Find,FIND,find,0x5f,
KEY_CUT,137,,,0xe03c,,0x38,123,,,,,,,Cut,CUT,cut,0x61,
KEY_HELP,138,Help,0x72,0xe075,,0x09,117,VK_HELP,0x2f,,,XK_Help,0xff6a,Help,HELP,help,0x76,
KEY_MENU,139,,,0xe01e,,0x91,118,,,,,,,,I147,menu,,
KEY_CALC,140,,,0xe021,0xe02b,0xa3,251,,,,,,,LaunchApp2,I148,calculator,,
KEY_SETUP,141,,,0x66,,,,,,,,,,,I149,,,
KEY_SLEEP,142,,,0xe05f,0xe03f,,248,VK_SLEEP,0x5f,,,,,Sleep,I150,sleep,,
KEY_WAKEUP,143,,,0xe063,0xe05e,,,,,,,,,WakeUp,I151,wake,,
KEY_FILE,144,,,0x67,,,,,,,,,,,I152,,,
KEY_SENDFILE,145,,,0x68,,,,,,,,,,,I153,,,
KEY_DELETEFILE,146,,,0x69,,,,,,,,,,,I154,,,
KEY_XFER,147,,,0xe013,,0xa2,,,,,,,,,XFER,,,
KEY_XFER,147,,,0xe013,,0xa2,,,,,,,,,I155,,,
KEY_PROG1,148,,,0xe01f,,0xa0,,,,,,,,,I156,,,
KEY_PROG2,149,,,0xe017,,0xa1,,,,,,,,,I157,,,
KEY_WWW,150,,,0xe002,,,240,,,,,,,,I158,,,
KEY_MSDOS,151,,,0x6a,,,,,,,,,,,I159,,,
KEY_SCREENLOCK,152,,,0xe012,,0x96,249,,,,,,,,I160,,,
KEY_DIRECTION,153,,,0x6b,,,,,,,,,,,I161,,,
KEY_CYCLEWINDOWS,154,,,0xe026,,0x9b,,,,,,,,,I162,,,
KEY_MAIL,155,,,0xe06c,0xe048,,,,,,,,,LaunchMail,I163,mail,,
KEY_BOOKMARKS,156,,,0xe066,0xe018,,,,,,,,,BrowserFavorites,I164,ac_bookmarks,,
KEY_COMPUTER,157,,,0xe06b,0xe040,,,,,,,,,LaunchApp1,I165,computer,,
KEY_BACK,158,,,0xe06a,0xe038,,241,VK_BROWSER_BACK,0xa6,,,,,BrowserBack,I166,ac_back,,
KEY_FORWARD,159,,,0xe069,0xe030,,242,VK_BROWSER_FORWARD,0xa7,,,,,BrowserForward,I167,ac_forward,,
KEY_CLOSECD,160,,,0xe023,,0x9a,,,,,,,,,I168,,,
KEY_EJECTCD,161,,,0x6c,,,236,,,,,,,,I169,,,
KEY_EJECTCLOSECD,162,,,0xe07d,,,,,,,,,,Eject,I170,,,
KEY_NEXTSONG,163,,,0xe019,0xe04d,0x93,235,VK_MEDIA_NEXT_TRACK,0xb0,,,,,MediaTrackNext,I171,audionext,,
KEY_PLAYPAUSE,164,,,0xe022,0xe034,,232,VK_MEDIA_PLAY_PAUSE,0xb3,,,,,MediaPlayPause,I172,audioplay,,
KEY_PREVIOUSSONG,165,,,0xe010,0xe015,0x94,234,VK_MEDIA_PREV_TRACK,0xb1,,,,,MediaTrackPrevious,I173,audioprev,,
KEY_STOPCD,166,,,0xe024,0xe03b,0x98,233,VK_MEDIA_STOP,0xb2,,,,,MediaStop,I174,audiostop,,
KEY_RECORD,167,,,0xe031,,0x9e,,,,,,,,,I175,,,
KEY_REWIND,168,,,0xe018,,0x9f,,,,,,,,,I176,,,
KEY_PHONE,169,,,0x63,,,,,,,,,,,I177,,,
KEY_ISO,170,ISO_Section,0xa,,,,,,,,,,,,I178,,,
KEY_CONFIG,171,,,0xe001,,,,,,,,,,,I179,,,
KEY_HOMEPAGE,172,,,0xe032,0xe03a,0x97,,VK_BROWSER_HOME,0xac,,,,,BrowserHome,I180,ac_home,,
KEY_REFRESH,173,,,0xe067,0xe020,,250,VK_BROWSER_REFRESH,0xa8,,,,,BrowserRefresh,I181,ac_refresh,,
KEY_EXIT,174,,,0x71,,,,,,,,,,,I182,,,
KEY_MOVE,175,,,0x72,,,,,,,,,,,I183,,,
KEY_EDIT,176,,,0xe008,,,247,,,,,,,,I184,,,
KEY_SCROLLUP,177,,,0x75,,,245,,,,,,,,I185,,,
KEY_SCROLLDOWN,178,,,0xe00f,,,246,,,,,,,,I186,,,
KEY_KPLEFTPAREN,179,,,0xe076,,,182,,,,,,,NumpadParenLeft,I187,,,
KEY_KPRIGHTPAREN,180,,,0xe07b,,,183,,,,,,,NumpadParenRight,I188,,,
KEY_NEW,181,,,0xe009,,,,,,,,,,,I189,,,
KEY_REDO,182,,,0xe00a,,,,,,,,,,,I190,,,
KEY_F13,183,F13,0x69,0x5d,0x2f,0x7f,104,VK_F13,0x7c,0x6e,0x6e,,,F13,FK13,,,0x69
KEY_F14,184,F14,0x6b,0x5e,0x37,0x80,105,VK_F14,0x7d,0x6f,0x6f,,,F14,FK14,,,0x6b
KEY_F15,185,F15,0x71,0x5f,0x3f,0x81,106,VK_F15,0x7e,0x70,0x70,,,F15,FK15,,,0x71
KEY_F16,186,F16,0x6a,0x55,,0x82,107,VK_F16,0x7f,0x71,0x71,,,F16,FK16,,,
KEY_F17,187,F17,0x40,0xe003,,0x83,108,VK_F17,0x80,0x72,0x72,,,F17,FK17,,,
KEY_F18,188,F18,0x4f,0xe077,,,109,VK_F18,0x81,,,,,F18,FK18,,,
KEY_F19,189,F19,0x50,0xe004,,,110,VK_F19,0x82,,,,,F19,FK19,,,
KEY_F20,190,F20,0x5a,0x5a,,,111,VK_F20,0x83,,,,,F20,FK20,,,
KEY_F21,191,,,0x74,,,112,VK_F21,0x84,,,,,F21,FK21,,,
KEY_F22,192,,,0xe079,,,113,VK_F22,0x85,,,,,F22,FK22,,,
KEY_F23,193,,,0x6d,,,114,VK_F23,0x86,,,,,F23,FK23,,,
KEY_F24,194,,,0x6f,,,115,VK_F24,0x87,,,,,F24,FK24,,,
,195,,,0xe015,,,,,,,,,,,,,,
,196,,,0xe016,,,,,,,,,,,,,,
,197,,,0xe01a,,,,,,,,,,,,,,
,198,,,0xe01b,,,,,,,,,,,,,,
,199,,,0xe027,,,,,,,,,,,,,,
KEY_PLAYCD,200,,,0xe028,,,,,,,,,,,I208,,,
KEY_PAUSECD,201,,,0xe029,,,,,,,,,,,I209,,,
KEY_PROG3,202,,,0xe02b,,,,,,,,,,,I210,,,
KEY_PROG4,203,,,0xe02c,,,,,,,,,,,I211,,,
KEY_DASHBOARD,204,,,0xe02d,,,,,,,,,,,I212,,,
KEY_SUSPEND,205,,,0xe025,,,,,,,,,,Suspend,I213,,,
KEY_CLOSE,206,,,0xe02f,,,,,,,,,,,I214,,,
KEY_PLAY,207,,,0xe033,,,,VK_PLAY,0xfa,,,,,,I215,,,
KEY_FASTFORWARD,208,,,0xe034,,,,,,,,,,,I216,,,
KEY_BASSBOOST,209,,,0xe036,,,,,,,,,,,I217,,,
KEY_PRINT,210,,,0xe039,,,,VK_PRINT,0x2a,,,,,,I218,,,
KEY_HP,211,,,0xe03a,,,,,,,,,,,I219,,,
KEY_CAMERA,212,,,0xe03b,,,,,,,,,,,I220,,,
KEY_SOUND,213,,,0xe03d,,,,,,,,,,,I221,,,
KEY_QUESTION,214,,,0xe03e,,,,,,,,,,,I222,,,
KEY_EMAIL,215,,,0xe03f,,,,VK_LAUNCH_MAIL,0xb4,,,,,,I223,,,
KEY_CHAT,216,,,0xe040,,,,,,,,,,,I224,,,
KEY_SEARCH,217,,,0xe065,0xe010,,,VK_BROWSER_SEARCH,0xaa,,,,,BrowserSearch,I225,,,
KEY_CONNECT,218,,,0xe042,,,,,,,,,,,I226,,,
KEY_FINANCE,219,,,0xe043,,,,,,,,,,,I227,,,
KEY_SPORT,220,,,0xe044,,,,,,,,,,,I228,,,
KEY_SHOP,221,,,0xe045,,,,,,,,,,,I229,,,
KEY_ALTERASE,222,,,0xe014,,,,,,,,,,,I230,,,
KEY_CANCEL,223,,,0xe04a,,,,,,,,,,,I231,,,
KEY_BRIGHTNESSDOWN,224,,,0xe04c,,,,,,,,,,,I232,,,
KEY_BRIGHTNESSUP,225,,,0xe054,,,,,,,,,,,I233,,,
KEY_MEDIA,226,,,0xe06d,0xe050,,,,,,,,,MediaSelect,I234,mediaselect,,
KEY_SWITCHVIDEOMODE,227,,,0xe056,,,,,,,,,,,I235,,,
KEY_KBDILLUMTOGGLE,228,,,0xe057,,,,,,,,,,,I236,,,
KEY_KBDILLUMDOWN,229,,,0xe058,,,,,,,,,,,I237,,,
KEY_KBDILLUMUP,230,,,0xe059,,,,,,,,,,,I238,,,
KEY_SEND,231,,,0xe05a,,,,,,,,,,,I239,,,
KEY_REPLY,232,,,0xe064,,,,,,,,,,,I240,,,
KEY_FORWARDMAIL,233,,,0xe00e,,,,,,,,,,,I241,,,
KEY_SAVE,234,,,0xe055,,,,,,,,,,,I242,,,
KEY_DOCUMENTS,235,,,0xe070,,,,,,,,,,,I243,,,
KEY_BATTERY,236,,,0xe071,,,,,,,,,,,I244,,,
KEY_BLUETOOTH,237,,,0xe072,,,,,,,,,,,I245,,,
KEY_WLAN,238,,,0xe073,,,,,,,,,,,I246,,,
KEY_UWB,239,,,0xe074,,,,,,,,,,,I247,,,
KEY_UNKNOWN,240,,,,,,,,,,,,,,I248,,,
KEY_VIDEO_NEXT,241,,,,,,,,,,,,,,I249,,,
KEY_VIDEO_PREV,242,,,,,,,,,,,,,,I250,,,
KEY_BRIGHTNESS_CYCLE,243,,,,,,,,,,,,,,I251,,,
KEY_BRIGHTNESS_ZERO,244,,,,,,,,,,,,,,I252,,,
KEY_DISPLAY_OFF,245,,,,,,,,,,,,,,I253,,,
KEY_WIMAX,246,,,,,,,,,,,,,,,,,
,247,,,,,,,,,,,,,,,,,
,248,,,,,,,,,,,,,,,,,
,249,,,,,,,,,,,,,,,,,
,250,,,,,,,,,,,,,,,,,
,251,,,,,,,,,,,,,,,,,
,252,,,,,,,,,,,,,,,,,
,253,,,,,,,,,,,,,,,,,
,254,,,,,,,,,,,,,,,,,
,255,,,,0xe012,,,,,,,,,,,,,
BTN_MISC,0x100,,,,,,,,,,,,,,,,,
BTN_0,0x100,,,,,,,VK_LBUTTON,0x01,,,,,,,,,
BTN_1,0x101,,,,,,,VK_RBUTTON,0x02,,,,,,,,,
BTN_2,0x102,,,,,,,VK_MBUTTON,0x04,,,,,,,,,
BTN_3,0x103,,,,,,,VK_XBUTTON1,0x05,,,,,,,,,
BTN_4,0x104,,,,,,,VK_XBUTTON2,0x06,,,,,,,,,
BTN_5,0x105,,,,,,,,,,,,,,,,,
BTN_6,0x106,,,,,,,,,,,,,,,,,
BTN_7,0x107,,,,,,,,,,,,,,,,,
BTN_8,0x108,,,,,,,,,,,,,,,,,
BTN_9,0x109,,,,,,,,,,,,,,,,,
BTN_MOUSE,0x110,,,,,,,,,,,,,,,,,
BTN_LEFT,0x110,,,,,,,,,,,,,,,,,
BTN_RIGHT,0x111,,,,,,,,,,,,,,,,,
BTN_MIDDLE,0x112,,,,,,,,,,,,,,,,,
BTN_SIDE,0x113,,,,,,,,,,,,,,,,,
BTN_EXTRA,0x114,,,,,,,,,,,,,,,,,
BTN_FORWARD,0x115,,,,,,,,,,,,,,,,,
BTN_BACK,0x116,,,,,,,,,,,,,,,,,
BTN_TASK,0x117,,,,,,,,,,,,,,,,,
BTN_JOYSTICK,0x120,,,,,,,,,,,,,,,,,
BTN_TRIGGER,0x120,,,,,,,,,,,,,,,,,
BTN_THUMB,0x121,,,,,,,,,,,,,,,,,
BTN_THUMB2,0x122,,,,,,,,,,,,,,,,,
BTN_TOP,0x123,,,,,,,,,,,,,,,,,
BTN_TOP2,0x124,,,,,,,,,,,,,,,,,
BTN_PINKIE,0x125,,,,,,,,,,,,,,,,,
BTN_BASE,0x126,,,,,,,,,,,,,,,,,
BTN_BASE2,0x127,,,,,,,,,,,,,,,,,
BTN_BASE3,0x128,,,,,,,,,,,,,,,,,
BTN_BASE4,0x129,,,,,,,,,,,,,,,,,
BTN_BASE5,0x12a,,,,,,,,,,,,,,,,,
BTN_BASE6,0x12b,,,,,,,,,,,,,,,,,
BTN_DEAD,0x12f,,,,,,,,,,,,,,,,,
BTN_GAMEPAD,0x130,,,,,,,,,,,,,,,,,
BTN_A,0x130,,,,,,,,,,,,,,,,,
BTN_B,0x131,,,,,,,,,,,,,,,,,
BTN_C,0x132,,,,,,,,,,,,,,,,,
BTN_X,0x133,,,,,,,,,,,,,,,,,
BTN_Y,0x134,,,,,,,,,,,,,,,,,
BTN_Z,0x135,,,,,,,,,,,,,,,,,
BTN_TL,0x136,,,,,,,,,,,,,,,,,
BTN_TR,0x137,,,,,,,,,,,,,,,,,
BTN_TL2,0x138,,,,,,,,,,,,,,,,,
BTN_TR2,0x139,,,,,,,,,,,,,,,,,
BTN_SELECT,0x13a,,,,,,,,,,,,,,,,,
BTN_START,0x13b,,,,,,,,,,,,,,,,,
BTN_MODE,0x13c,,,,,,,,,,,,,,,,,
BTN_THUMBL,0x13d,,,,,,,,,,,,,,,,,
BTN_THUMBR,0x13e,,,,,,,,,,,,,,,,,
BTN_DIGI,0x140,,,,,,,,,,,,,,,,,
BTN_TOOL_PEN,0x140,,,,,,,,,,,,,,,,,
BTN_TOOL_RUBBER,0x141,,,,,,,,,,,,,,,,,
BTN_TOOL_BRUSH,0x142,,,,,,,,,,,,,,,,,
BTN_TOOL_PENCIL,0x143,,,,,,,,,,,,,,,,,
BTN_TOOL_AIRBRUSH,0x144,,,,,,,,,,,,,,,,,
BTN_TOOL_FINGER,0x145,,,,,,,,,,,,,,,,,
BTN_TOOL_MOUSE,0x146,,,,,,,,,,,,,,,,,
BTN_TOOL_LENS,0x147,,,,,,,,,,,,,,,,,
BTN_TOUCH,0x14a,,,,,,,,,,,,,,,,,
BTN_STYLUS,0x14b,,,,,,,,,,,,,,,,,
BTN_STYLUS2,0x14c,,,,,,,,,,,,,,,,,
BTN_TOOL_DOUBLETAP,0x14d,,,,,,,,,,,,,,,,,
BTN_TOOL_TRIPLETAP,0x14e,,,,,,,,,,,,,,,,,
BTN_TOOL_QUADTAP,0x14f,,,,,,,,,,,,,,,,,
BTN_WHEEL,0x150,,,,,,,,,,,,,,,,,
BTN_GEAR_DOWN,0x150,,,,,,,,,,,,,,,,,
BTN_GEAR_UP,0x151,,,,,,,,,,,,,,,,,
KEY_OK,0x160,,,,,,,,,,,,,,,,,
KEY_SELECT,0x161,,,,,,,VK_SELECT,0x29,,,XK_Select,0xff60,Select,SELE,,,
KEY_GOTO,0x162,,,,,,,,,,,,,,,,,
KEY_CLEAR,0x163,,,,,,,,,,,,,NumpadClear,CLR,,,
KEY_POWER2,0x164,,,,,,,,,,,,,,,,,
KEY_OPTION,0x165,,,,,,,,,,,,,,,,,
KEY_INFO,0x166,,,,,,,,,,,,,,,,,
KEY_TIME,0x167,,,,,,,,,,,,,,,,,
KEY_VENDOR,0x168,,,,,,,,,,,,,,,,,
KEY_ARCHIVE,0x169,,,,,,,,,,,,,,,,,
KEY_PROGRAM,0x16a,,,,,,,,,,,,,,,,,
KEY_CHANNEL,0x16b,,,,,,,,,,,,,,,,,
KEY_FAVORITES,0x16c,,,,,,,VK_BROWSER_FAVOURITES,0xab,,,,,,,,,
KEY_EPG,0x16d,,,,,,,,,,,,,,,,,
KEY_PVR,0x16e,,,,,,,,,,,,,,,,,
KEY_MHP,0x16f,,,,,,,,,,,,,,,,,
KEY_LANGUAGE,0x170,,,,,,,,,,,,,,,,,
KEY_TITLE,0x171,,,,,,,,,,,,,,,,,
KEY_SUBTITLE,0x172,,,,,,,,,,,,,,,,,
KEY_ANGLE,0x173,,,,,,,,,,,,,,,,,
KEY_ZOOM,0x174,,,,,,,VK_ZOOM,0xfb,,,,,,,,,
KEY_MODE,0x175,,,,,,,,,,,,,,,,,
KEY_KEYBOARD,0x176,,,,,,,,,,,,,,,,,
KEY_SCREEN,0x177,,,,,,,,,,,,,,,,,
KEY_PC,0x178,,,,,,,,,,,,,,,,,
KEY_TV,0x179,,,,,,,,,,,,,,,,,
KEY_TV2,0x17a,,,,,,,,,,,,,,,,,
KEY_VCR,0x17b,,,,,,,,,,,,,,,,,
KEY_VCR2,0x17c,,,,,,,,,,,,,,,,,
KEY_SAT,0x17d,,,,,,,,,,,,,,,,,
KEY_SAT2,0x17e,,,,,,,,,,,,,,,,,
KEY_CD,0x17f,,,,,,,,,,,,,,,,,
KEY_TAPE,0x180,,,,,,,,,,,,,,,,,
KEY_RADIO,0x181,,,,,,,,,,,,,,,,,
KEY_TUNER,0x182,,,,,,,,,,,,,,,,,
KEY_PLAYER,0x183,,,,,,,,,,,,,,,,,
KEY_TEXT,0x184,,,,,,,,,,,,,,,,,
KEY_DVD,0x185,,,,,,,,,,,,,,,,,
KEY_AUX,0x186,,,,,,,,,,,,,,,,,
KEY_MP3,0x187,,,,,,,,,,,,,,,,,
KEY_AUDIO,0x188,,,,,,,,,,,,,,,,,
KEY_VIDEO,0x189,,,,,,,,,,,,,,,,,
KEY_DIRECTORY,0x18a,,,,,,,,,,,,,,,,,
KEY_LIST,0x18b,,,,,,,,,,,,,,,,,
KEY_MEMO,0x18c,,,,,,,,,,,,,,,,,
KEY_CALENDAR,0x18d,,,,,,,,,,,,,,,,,
KEY_RED,0x18e,,,,,,,,,,,,,,,,,
KEY_GREEN,0x18f,,,,,,,,,,,,,,,,,
KEY_YELLOW,0x190,,,,,,,,,,,,,,,,,
KEY_BLUE,0x191,,,,,,,,,,,,,,,,,
KEY_CHANNELUP,0x192,,,,,,,,,,,,,,,,,
KEY_CHANNELDOWN,0x193,,,,,,,,,,,,,,,,,
KEY_FIRST,0x194,,,,,,,,,,,,,,,,,
KEY_LAST,0x195,,,,,,,,,,,,,,,,,
KEY_AB,0x196,,,,,,,,,,,,,,,,,
KEY_NEXT,0x197,,,,,,,,,,,,,,,,,
KEY_RESTART,0x198,,,,,,,,,,,,,,,,,
KEY_SLOW,0x199,,,,,,,,,,,,,,,,,
KEY_SHUFFLE,0x19a,,,,,,,,,,,,,,,,,
KEY_BREAK,0x19b,,,,,,,,,,,,,,BREA,,,
KEY_BREAK,0x19b,,,,,,,,,,,,,,BRK,,,
KEY_PREVIOUS,0x19c,,,,,,,,,,,,,,,,,
KEY_DIGITS,0x19d,,,,,,,,,,,,,,,,,
KEY_TEEN,0x19e,,,,,,,,,,,,,,,,,
KEY_TWEN,0x19f,,,,,,,,,,,,,,,,,
KEY_VIDEOPHONE,0x1a0,,,,,,,,,,,,,,,,,
KEY_GAMES,0x1a1,,,,,,,,,,,,,,,,,
KEY_ZOOMIN,0x1a2,,,,,,,,,,,,,,,,,
KEY_ZOOMOUT,0x1a3,,,,,,,,,,,,,,,,,
KEY_ZOOMRESET,0x1a4,,,,,,,,,,,,,,,,,
KEY_WORDPROCESSOR,0x1a5,,,,,,,,,,,,,,,,,
KEY_EDITOR,0x1a6,,,,,,,,,,,,,,,,,
KEY_SPREADSHEET,0x1a7,,,,,,,,,,,,,,,,,
KEY_GRAPHICSEDITOR,0x1a8,,,,,,,,,,,,,,,,,
KEY_PRESENTATION,0x1a9,,,,,,,,,,,,,,,,,
KEY_DATABASE,0x1aa,,,,,,,,,,,,,,,,,
KEY_NEWS,0x1ab,,,,,,,,,,,,,,,,,
KEY_VOICEMAIL,0x1ac,,,,,,,,,,,,,,,,,
KEY_ADDRESSBOOK,0x1ad,,,,,,,,,,,,,,,,,
KEY_MESSENGER,0x1ae,,,,,,,,,,,,,,,,,
KEY_DISPLAYTOGGLE,0x1af,,,,,,,,,,,,,,,,,
KEY_SPELLCHECK,0x1b0,,,,,,,,,,,,,,,,,
KEY_LOGOFF,0x1b1,,,,,,,,,,,,,,,,,
KEY_DOLLAR,0x1b2,,,,,,,,,,,,,,,,,
KEY_EURO,0x1b3,,,,,,,,,,,,,,,,,
KEY_FRAMEBACK,0x1b4,,,,,,,,,,,,,,,,,
KEY_FRAMEFORWARD,0x1b5,,,,,,,,,,,,,,,,,
KEY_CONTEXT_MENU,0x1b6,,,,,,,,,,,,,,,,,
KEY_MEDIA_REPEAT,0x1b7,,,,,,,,,,,,,,,,,
KEY_DEL_EOL,0x1c0,,,,,,,,,,,,,,,,,
KEY_DEL_EOS,0x1c1,,,,,,,,,,,,,,,,,
KEY_INS_LINE,0x1c2,,,,,,,,,,,,,,,,,
KEY_DEL_LINE,0x1c3,,,,,,,,,,,,,,,,,
KEY_FN,0x1d0,Function,0x3f,,,,,,,,,,,Fn,,,,
KEY_FN_ESC,0x1d1,,,,,,,,,,,,,,,,,
KEY_FN_F1,0x1d2,,,,,,,,,,,,,,,,,
KEY_FN_F2,0x1d3,,,,,,,,,,,,,,,,,
KEY_FN_F3,0x1d4,,,,,,,,,,,,,,,,,
KEY_FN_F4,0x1d5,,,,,,,,,,,,,,,,,
KEY_FN_F5,0x1d6,,,,,,,,,,,,,,,,,
KEY_FN_F6,0x1d7,,,,,,,,,,,,,,,,,
KEY_FN_F7,0x1d8,,,,,,,,,,,,,,,,,
KEY_FN_F8,0x1d9,,,,,,,,,,,,,,,,,
KEY_FN_F9,0x1da,,,,,,,,,,,,,,,,,
KEY_FN_F10,0x1db,,,,,,,,,,,,,,,,,
KEY_FN_F11,0x1dc,,,,,,,,,,,,,,,,,
KEY_FN_F12,0x1dd,,,,,,,,,,,,,,,,,
KEY_FN_1,0x1de,,,,,,,,,,,,,,,,,
KEY_FN_2,0x1df,,,,,,,,,,,,,,,,,
KEY_FN_D,0x1e0,,,,,,,,,,,,,,,,,
KEY_FN_E,0x1e1,,,,,,,,,,,,,,,,,
KEY_FN_F,0x1e2,,,,,,,,,,,,,,,,,
KEY_FN_S,0x1e3,,,,,,,,,,,,,,,,,
KEY_FN_B,0x1e4,,,,,,,,,,,,,,,,,
KEY_BRL_DOT1,0x1f1,,,,,,,,,,,,,,,,,
KEY_BRL_DOT2,0x1f2,,,,,,,,,,,,,,,,,
KEY_BRL_DOT3,0x1f3,,,,,,,,,,,,,,,,,
KEY_BRL_DOT4,0x1f4,,,,,,,,,,,,,,,,,
KEY_BRL_DOT5,0x1f5,,,,,,,,,,,,,,,,,
KEY_BRL_DOT6,0x1f6,,,,,,,,,,,,,,,,,
KEY_BRL_DOT7,0x1f7,,,,,,,,,,,,,,,,,
KEY_BRL_DOT8,0x1f8,,,,,,,,,,,,,,,,,
KEY_BRL_DOT9,0x1f9,,,,,,,,,,,,,,,,,
KEY_BRL_DOT10,0x1fa,,,,,,,,,,,,,,,,,
KEY_NUMERIC_0,0x200,,,,,,,,,,,,,,,,,
KEY_NUMERIC_1,0x201,,,,,,,,,,,,,,,,,
KEY_NUMERIC_2,0x202,,,,,,,,,,,,,,,,,
KEY_NUMERIC_3,0x203,,,,,,,,,,,,,,,,,
KEY_NUMERIC_4,0x204,,,,,,,,,,,,,,,,,
KEY_NUMERIC_5,0x205,,,,,,,,,,,,,,,,,
KEY_NUMERIC_6,0x206,,,,,,,,,,,,,,,,,
KEY_NUMERIC_7,0x207,,,,,,,,,,,,,,,,,
KEY_NUMERIC_8,0x208,,,,,,,,,,,,,,,,,
KEY_NUMERIC_9,0x209,,,,,,,,,,,,,,,,,
KEY_NUMERIC_STAR,0x20a,,,,,,,,,,,,,NumpadStar,,,,
KEY_NUMERIC_POUND,0x20b,,,,,,,,,,,,,NumpadHash,,,,
KEY_RFKILL,0x20c,,,,,,,,,,,,,,,,,
1 Linux Name Linux Keycode OS-X Name OS-X Keycode AT set1 keycode AT set2 keycode AT set3 keycode USB Keycodes Win32 Name Win32 Keycode Xwin XT Xfree86 KBD XT X11 keysym name X11 keysym HTML code XKB key name QEMU QKeyCode Sun KBD Apple ADB
2 KEY_RESERVED 0 0xff unmapped 0xff
3 KEY_ESC 1 Escape 0x35 0x01 0x76 0x08 41 VK_ESCAPE 0x1b 1 1 XK_Escape 0xff1b Escape ESC esc 0x1d 0x35
4 KEY_1 2 ANSI_1 0x12 0x02 0x16 0x16 30 VK_1 0x31 2 2 XK_1 0x0031 Digit1 AE01 1 0x1e 0x12
5 KEY_1 2 ANSI_1 0x12 0x02 0x16 0x16 30 VK_1 0x31 2 2 XK_exclam 0x0021 Digit1 AE01 1 0x1e 0x12
6 KEY_2 3 ANSI_2 0x13 0x03 0x1e 0x1e 31 VK_2 0x32 3 3 XK_2 0x0032 Digit2 AE02 2 0x1f 0x13
7 KEY_2 3 ANSI_2 0x13 0x03 0x1e 0x1e 31 VK_2 0x32 3 3 XK_at 0x0040 Digit2 AE02 2 0x1f 0x13
8 KEY_3 4 ANSI_3 0x14 0x04 0x26 0x26 32 VK_3 0x33 4 4 XK_3 0x0033 Digit3 AE03 3 0x20 0x14
9 KEY_3 4 ANSI_3 0x14 0x04 0x26 0x26 32 VK_3 0x33 4 4 XK_numbersign 0x0023 Digit3 AE03 3 0x20 0x14
10 KEY_4 5 ANSI_4 0x15 0x05 0x25 0x25 33 VK_4 0x34 5 5 XK_4 0x0034 Digit4 AE04 4 0x21 0x15
11 KEY_4 5 ANSI_4 0x15 0x05 0x25 0x25 33 VK_4 0x34 5 5 XK_dollar 0x0024 Digit4 AE04 4 0x21 0x15
12 KEY_5 6 ANSI_5 0x17 0x06 0x2e 0x2e 34 VK_5 0x35 6 6 XK_5 0x0035 Digit5 AE05 5 0x22 0x17
13 KEY_5 6 ANSI_5 0x17 0x06 0x2e 0x2e 34 VK_5 0x35 6 6 XK_percent 0x0025 Digit5 AE05 5 0x22 0x17
14 KEY_6 7 ANSI_6 0x16 0x07 0x36 0x36 35 VK_6 0x36 7 7 XK_6 0x0036 Digit6 AE06 6 0x23 0x16
15 KEY_6 7 ANSI_6 0x16 0x07 0x36 0x36 35 VK_6 0x36 7 7 XK_asciicircum 0x005e Digit6 AE06 6 0x23 0x16
16 KEY_7 8 ANSI_7 0x1a 0x08 0x3d 0x3d 36 VK_7 0x37 8 8 XK_7 0x0037 Digit7 AE07 7 0x24 0x1a
17 KEY_7 8 ANSI_7 0x1a 0x08 0x3d 0x3d 36 VK_7 0x37 8 8 XK_ampersand 0x0026 Digit7 AE07 7 0x24 0x1a
18 KEY_8 9 ANSI_8 0x1c 0x09 0x3e 0x3e 37 VK_8 0x38 9 9 XK_8 0x0038 Digit8 AE08 8 0x25 0x1c
19 KEY_8 9 ANSI_8 0x1c 0x09 0x3e 0x3e 37 VK_8 0x38 9 9 XK_asterisk 0x002a Digit8 AE08 8 0x25 0x1c
20 KEY_9 10 ANSI_9 0x19 0x0a 0x46 0x46 38 VK_9 0x39 10 10 XK_9 0x0039 Digit9 AE09 9 0x26 0x19
21 KEY_9 10 ANSI_9 0x19 0x0a 0x46 0x46 38 VK_9 0x39 10 10 XK_parenleft 0x0028 Digit9 AE09 9 0x26 0x19
22 KEY_0 11 ANSI_0 0x1d 0x0b 0x45 0x45 39 VK_0 0x30 11 11 XK_0 0x0030 Digit0 AE10 0 0x27 0x1d
23 KEY_0 11 ANSI_0 0x1d 0x0b 0x45 0x45 39 VK_0 0x30 11 11 XK_parenright 0x0029 Digit0 AE10 0 0x27 0x1d
24 KEY_MINUS 12 ANSI_Minus 0x1b 0x0c 0x4e 0x4e 45 VK_OEM_MINUS 0xbd 12 12 XK_minus 0x002d Minus AE11 minus 0x28 0x1b
25 KEY_MINUS 12 ANSI_Minus 0x1b 0x0c 0x4e 0x4e 45 VK_OEM_MINUS 0xbd 12 12 XK_underscore 0x005f Minus AE11 minus 0x28 0x1b
26 KEY_EQUAL 13 ANSI_Equal 0x18 0x0d 0x55 0x55 46 VK_OEM_PLUS 0xbb 13 13 XK_equal 0x003d Equal AE12 equal 0x29 0x18
27 KEY_EQUAL 13 ANSI_Equal 0x18 0x0d 0x55 0x55 46 VK_OEM_PLUS 0xbb 13 13 XK_plus 0x002b Equal AE12 equal 0x29 0x18
28 KEY_BACKSPACE 14 Delete 0x33 0x0e 0x66 0x66 42 VK_BACK 0x08 14 14 XK_BackSpace 0xff08 Backspace BKSP backspace 0x2b 0x33
29 KEY_TAB 15 Tab 0x30 0x0f 0x0d 0x0d 43 VK_TAB 0x09 15 15 XK_Tab 0xff09 Tab TAB tab 0x35 0x30
30 KEY_Q 16 ANSI_Q 0xc 0x10 0x15 0x15 20 VK_Q 0x51 16 16 XK_Q 0x0051 KeyQ AD01 q 0x36 0xc
31 KEY_Q 16 ANSI_Q 0xc 0x10 0x15 0x15 20 VK_Q 0x51 16 16 XK_q 0x0071 KeyQ AD01 q 0x36 0xc
32 KEY_W 17 ANSI_W 0xd 0x11 0x1d 0x1d 26 VK_W 0x57 17 17 XK_W 0x0057 KeyW AD02 w 0x37 0xd
33 KEY_W 17 ANSI_W 0xd 0x11 0x1d 0x1d 26 VK_W 0x57 17 17 XK_w 0x0077 KeyW AD02 w 0x37 0xd
34 KEY_E 18 ANSI_E 0xe 0x12 0x24 0x24 8 VK_E 0x45 18 18 XK_E 0x0045 KeyE AD03 e 0x38 0xe
35 KEY_E 18 ANSI_E 0xe 0x12 0x24 0x24 8 VK_E 0x45 18 18 XK_e 0x0065 KeyE AD03 e 0x38 0xe
36 KEY_R 19 ANSI_R 0xf 0x13 0x2d 0x2d 21 VK_R 0x52 19 19 XK_R 0x0052 KeyR AD04 r 0x39 0xf
37 KEY_R 19 ANSI_R 0xf 0x13 0x2d 0x2d 21 VK_R 0x52 19 19 XK_r 0x0072 KeyR AD04 r 0x39 0xf
38 KEY_T 20 ANSI_T 0x11 0x14 0x2c 0x2c 23 VK_T 0x54 20 20 XK_T 0x0054 KeyT AD05 t 0x3a 0x11
39 KEY_T 20 ANSI_T 0x11 0x14 0x2c 0x2c 23 VK_T 0x54 20 20 XK_t 0x0074 KeyT AD05 t 0x3a 0x11
40 KEY_Y 21 ANSI_Y 0x10 0x15 0x35 0x35 28 VK_Y 0x59 21 21 XK_Y 0x0059 KeyY AD06 y 0x3b 0x10
41 KEY_Y 21 ANSI_Y 0x10 0x15 0x35 0x35 28 VK_Y 0x59 21 21 XK_y 0x0079 KeyY AD06 y 0x3b 0x10
42 KEY_U 22 ANSI_U 0x20 0x16 0x3c 0x3c 24 VK_U 0x55 22 22 XK_U 0x0055 KeyU AD07 u 0x3c 0x20
43 KEY_U 22 ANSI_U 0x20 0x16 0x3c 0x3c 24 VK_U 0x55 22 22 XK_u 0x0075 KeyU AD07 u 0x3c 0x20
44 KEY_I 23 ANSI_I 0x22 0x17 0x43 0x43 12 VK_I 0x49 23 23 XK_I 0x0049 KeyI AD08 i 0x3d 0x22
45 KEY_I 23 ANSI_I 0x22 0x17 0x43 0x43 12 VK_I 0x49 23 23 XK_i 0x0069 KeyI AD08 i 0x3d 0x22
46 KEY_O 24 ANSI_O 0x1f 0x18 0x44 0x44 18 VK_O 0x4f 24 24 XK_O 0x004f KeyO AD09 o 0x3e 0x1f
47 KEY_O 24 ANSI_O 0x1f 0x18 0x44 0x44 18 VK_O 0x4f 24 24 XK_o 0x006f KeyO AD09 o 0x3e 0x1f
48 KEY_P 25 ANSI_P 0x23 0x19 0x4d 0x4d 19 VK_P 0x50 25 25 XK_P 0x0050 KeyP AD10 p 0x3f 0x23
49 KEY_P 25 ANSI_P 0x23 0x19 0x4d 0x4d 19 VK_P 0x50 25 25 XK_p 0x0070 KeyP AD10 p 0x3f 0x23
50 KEY_LEFTBRACE 26 ANSI_LeftBracket 0x21 0x1a 0x54 0x54 47 VK_OEM_4 0xdb 26 26 XK_bracketleft 0x005b BracketLeft AD11 bracket_left 0x40 0x21
51 KEY_LEFTBRACE 26 ANSI_LeftBracket 0x21 0x1a 0x54 0x54 47 VK_OEM_4 0xdb 26 26 XK_braceleft 0x007b BracketLeft AD11 bracket_left 0x40 0x21
52 KEY_RIGHTBRACE 27 ANSI_RightBracket 0x1e 0x1b 0x5b 0x5b 48 VK_OEM_6 0xdd 27 27 XK_bracketright 0x005d BracketRight AD12 bracket_right 0x41 0x1e
53 KEY_RIGHTBRACE 27 ANSI_RightBracket 0x1e 0x1b 0x5b 0x5b 48 VK_OEM_6 0xdd 27 27 XK_braceright 0x007d BracketRight AD12 bracket_right 0x41 0x1e
54 KEY_ENTER 28 Return 0x24 0x1c 0x5a 0x5a 40 VK_RETURN 0x0d 28 28 XK_Return 0xff0d Enter RTRN ret 0x59 0x24
55 KEY_LEFTCTRL 29 Control 0x3b 0x1d 0x14 0x11 224 VK_LCONTROL 0xa2 29 29 XK_Control_L 0xffe3 ControlLeft LCTL ctrl 0x4c 0x36
56 KEY_LEFTCTRL 29 Control 0x3b 0x1d 0x14 0x11 224 VK_CONTROL 0x11 29 29 XK_Control_L 0xffe3 ControlLeft LCTL ctrl 0x4c 0x36
57 KEY_A 30 ANSI_A 0x0 0x1e 0x1c 0x1c 4 VK_A 0x41 30 30 XK_A 0x0041 KeyA AC01 a 0x4d 0x0
58 KEY_A 30 ANSI_A 0x0 0x1e 0x1c 0x1c 4 VK_A 0x41 30 30 XK_a 0x0061 KeyA AC01 a 0x4d 0x0
59 KEY_S 31 ANSI_S 0x1 0x1f 0x1b 0x1b 22 VK_S 0x53 31 31 XK_S 0x0053 KeyS AC02 s 0x4e 0x1
60 KEY_S 31 ANSI_S 0x1 0x1f 0x1b 0x1b 22 VK_S 0x53 31 31 XK_s 0x0073 KeyS AC02 s 0x4e 0x1
61 KEY_D 32 ANSI_D 0x2 0x20 0x23 0x23 7 VK_D 0x44 32 32 XK_D 0x0044 KeyD AC03 d 0x4f 0x2
62 KEY_D 32 ANSI_D 0x2 0x20 0x23 0x23 7 VK_D 0x44 32 32 XK_d 0x0064 KeyD AC03 d 0x4f 0x2
63 KEY_F 33 ANSI_F 0x3 0x21 0x2b 0x2b 9 VK_F 0x46 33 33 XK_F 0x0046 KeyF AC04 f 0x50 0x3
64 KEY_F 33 ANSI_F 0x3 0x21 0x2b 0x2b 9 VK_F 0x46 33 33 XK_f 0x0066 KeyF AC04 f 0x50 0x3
65 KEY_G 34 ANSI_G 0x5 0x22 0x34 0x34 10 VK_G 0x47 34 34 XK_G 0x0047 KeyG AC05 g 0x51 0x5
66 KEY_G 34 ANSI_G 0x5 0x22 0x34 0x34 10 VK_G 0x47 34 34 XK_g 0x0067 KeyG AC05 g 0x51 0x5
67 KEY_H 35 ANSI_H 0x4 0x23 0x33 0x33 11 VK_H 0x48 35 35 XK_H 0x0048 KeyH AC06 h 0x52 0x4
68 KEY_H 35 ANSI_H 0x4 0x23 0x33 0x33 11 VK_H 0x48 35 35 XK_h 0x0068 KeyH AC06 h 0x52 0x4
69 KEY_J 36 ANSI_J 0x26 0x24 0x3b 0x3b 13 VK_J 0x4a 36 36 XK_J 0x004a KeyJ AC07 j 0x53 0x26
70 KEY_J 36 ANSI_J 0x26 0x24 0x3b 0x3b 13 VK_J 0x4a 36 36 XK_j 0x006a KeyJ AC07 j 0x53 0x26
71 KEY_K 37 ANSI_K 0x28 0x25 0x42 0x42 14 VK_K 0x4b 37 37 XK_K 0x004b KeyK AC08 k 0x54 0x28
72 KEY_K 37 ANSI_K 0x28 0x25 0x42 0x42 14 VK_K 0x4b 37 37 XK_k 0x006b KeyK AC08 k 0x54 0x28
73 KEY_L 38 ANSI_L 0x25 0x26 0x4b 0x4b 15 VK_L 0x4c 38 38 XK_L 0x004c KeyL AC09 l 0x55 0x25
74 KEY_L 38 ANSI_L 0x25 0x26 0x4b 0x4b 15 VK_L 0x4c 38 38 XK_l 0x006c KeyL AC09 l 0x55 0x25
75 KEY_SEMICOLON 39 ANSI_Semicolon 0x29 0x27 0x4c 0x4c 51 VK_OEM_1 0xba 39 39 XK_semicolon 0x003b Semicolon AC10 semicolon 0x56 0x29
76 KEY_SEMICOLON 39 ANSI_Semicolon 0x29 0x27 0x4c 0x4c 51 VK_OEM_1 0xba 39 39 XK_colon 0x003a Semicolon AC10 semicolon 0x56 0x29
77 KEY_APOSTROPHE 40 ANSI_Quote 0x27 0x28 0x52 0x52 52 VK_OEM_7 0xde 40 40 XK_apostrophe 0x0027 Quote AC11 apostrophe 0x57 0x27
78 KEY_APOSTROPHE 40 ANSI_Quote 0x27 0x28 0x52 0x52 52 VK_OEM_7 0xde 40 40 XK_quotedbl 0x0022 Quote AC11 apostrophe 0x57 0x27
79 KEY_GRAVE 41 ANSI_Grave 0x32 0x29 0x0e 0x0e 53 VK_OEM_3 0xc0 41 41 XK_grave 0x0060 Backquote TLDE grave_accent 0x2a 0x32
80 KEY_GRAVE 41 ANSI_Grave 0x32 0x29 0x0e 0x0e 53 VK_OEM_3 0xc0 41 41 XK_grave 0x0060 Backquote AB00 grave_accent 0x2a 0x32
81 KEY_GRAVE 41 ANSI_Grave 0x32 0x29 0x0e 0x0e 53 VK_OEM_3 0xc0 41 41 XK_asciitilde 0x007e Backquote TLDE grave_accent 0x2a 0x32
82 KEY_GRAVE 41 ANSI_Grave 0x32 0x29 0x0e 0x0e 53 VK_OEM_3 0xc0 41 41 XK_asciitilde 0x007e Backquote AB00 grave_accent 0x2a 0x32
83 KEY_SHIFT 42 Shift 0x38 0x2a 0x12 0x12 225 VK_SHIFT 0x10 42 42 XK_Shift_L 0xffe1 ShiftLeft LFSH shift 0x63 0x38
84 KEY_LEFTSHIFT 42 Shift 0x38 0x2a 0x12 0x12 225 VK_LSHIFT 0xa0 42 42 XK_Shift_L 0xffe1 ShiftLeft LFSH shift 0x63 0x38
85 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 49 VK_OEM_5 0xdc 43 43 XK_backslash 0x005c Backslash BKSL backslash 0x58 0x2a
86 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 49 VK_OEM_5 0xdc 43 43 XK_backslash 0x005c Backslash AC12 backslash 0x58 0x2a
87 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 49 VK_OEM_5 0xdc 43 43 XK_bar 0x007c Backslash BKSL backslash 0x58 0x2a
88 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 49 VK_OEM_5 0xdc 43 43 XK_bar 0x007c Backslash AC12 backslash 0x58 0x2a
89 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 50 VK_OEM_5 0xdc 43 43 XK_backslash 0x005c Backslash BKSL backslash 0x58 0x2a
90 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 50 VK_OEM_5 0xdc 43 43 XK_backslash 0x005c Backslash AC12 backslash 0x58 0x2a
91 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 50 VK_OEM_5 0xdc 43 43 XK_bar 0x007c Backslash BKSL backslash 0x58 0x2a
92 KEY_BACKSLASH 43 ANSI_Backslash 0x2a 0x2b 0x5d 0x5c 50 VK_OEM_5 0xdc 43 43 XK_bar 0x007c Backslash AC12 backslash 0x58 0x2a
93 KEY_Z 44 ANSI_Z 0x6 0x2c 0x1a 0x1a 29 VK_Z 0x5a 44 44 XK_Z 0x005a KeyZ AB01 z 0x64 0x6
94 KEY_Z 44 ANSI_Z 0x6 0x2c 0x1a 0x1a 29 VK_Z 0x5a 44 44 XK_z 0x007a KeyZ AB01 z 0x64 0x6
95 KEY_X 45 ANSI_X 0x7 0x2d 0x22 0x22 27 VK_X 0x58 45 45 XK_X 0x0058 KeyX AB02 x 0x65 0x7
96 KEY_X 45 ANSI_X 0x7 0x2d 0x22 0x22 27 VK_X 0x58 45 45 XK_x 0x0078 KeyX AB02 x 0x65 0x7
97 KEY_C 46 ANSI_C 0x8 0x2e 0x21 0x21 6 VK_C 0x43 46 46 XK_C 0x0043 KeyC AB03 c 0x66 0x8
98 KEY_C 46 ANSI_C 0x8 0x2e 0x21 0x21 6 VK_C 0x43 46 46 XK_c 0x0063 KeyC AB03 c 0x66 0x8
99 KEY_V 47 ANSI_V 0x9 0x2f 0x2a 0x2a 25 VK_V 0x56 47 47 XK_V 0x0056 KeyV AB04 v 0x67 0x9
100 KEY_V 47 ANSI_V 0x9 0x2f 0x2a 0x2a 25 VK_V 0x56 47 47 XK_v 0x0076 KeyV AB04 v 0x67 0x9
101 KEY_B 48 ANSI_B 0xb 0x30 0x32 0x32 5 VK_B 0x42 48 48 XK_B 0x0042 KeyB AB05 b 0x68 0xb
102 KEY_B 48 ANSI_B 0xb 0x30 0x32 0x32 5 VK_B 0x42 48 48 XK_b 0x0062 KeyB AB05 b 0x68 0xb
103 KEY_N 49 ANSI_N 0x2d 0x31 0x31 0x31 17 VK_N 0x4e 49 49 XK_N 0x004e KeyN AB06 n 0x69 0x2d
104 KEY_N 49 ANSI_N 0x2d 0x31 0x31 0x31 17 VK_N 0x4e 49 49 XK_n 0x006e KeyN AB06 n 0x69 0x2d
105 KEY_M 50 ANSI_M 0x2e 0x32 0x3a 0x3a 16 VK_M 0x4d 50 50 XK_M 0x004d KeyM AB07 m 0x6a 0x2e
106 KEY_M 50 ANSI_M 0x2e 0x32 0x3a 0x3a 16 VK_M 0x4d 50 50 XK_m 0x006d KeyM AB07 m 0x6a 0x2e
107 KEY_COMMA 51 ANSI_Comma 0x2b 0x33 0x41 0x41 54 VK_OEM_COMMA 0xbc 51 51 XK_comma 0x002c Comma AB08 comma 0x6b 0x2b
108 KEY_COMMA 51 ANSI_Comma 0x2b 0x33 0x41 0x41 54 VK_OEM_COMMA 0xbc 51 51 XK_less 0x003c Comma AB08 comma 0x6b 0x2b
109 KEY_DOT 52 ANSI_Period 0x2f 0x34 0x49 0x49 55 VK_OEM_PERIOD 0xbe 52 52 XK_period 0x002e Period AB09 dot 0x6c 0x2f
110 KEY_DOT 52 ANSI_Period 0x2f 0x34 0x49 0x49 55 VK_OEM_PERIOD 0xbe 52 52 XK_greater 0x003e Period AB09 dot 0x6c 0x2f
111 KEY_SLASH 53 ANSI_Slash 0x2c 0x35 0x4a 0x4a 56 VK_OEM_2 0xbf 53 53 XK_slash 0x002f Slash AB10 slash 0x6d 0x2c
112 KEY_SLASH 53 ANSI_Slash 0x2c 0x35 0x4a 0x4a 56 VK_OEM_2 0xbf 53 53 XK_question 0x003f Slash AB10 slash 0x6d 0x2c
113 KEY_RIGHTSHIFT 54 RightShift 0x3c 0x36 0x59 0x59 229 VK_RSHIFT 0xa1 54 54 XK_Shift_R 0xffe2 ShiftRight RTSH shift_r 0x6e 0x7b
114 KEY_KPASTERISK 55 ANSI_KeypadMultiply 0x43 0x37 0x7c 0x7e 85 VK_MULTIPLY 0x6a 55 55 XK_multiply 0x00d7 NumpadMultiply KPMU asterisk 0x2f 0x43
115 KEY_KPASTERISK 55 ANSI_KeypadMultiply 0x43 0x37 0x7c 0x7e 85 VK_MULTIPLY 0x6a 55 55 XK_multiply 0x00d7 NumpadMultiply KPMU kp_multiply 0x2f 0x43
116 KEY_LEFTALT 56 Option 0x3a 0x38 0x11 0x19 226 VK_LMENU 0xa4 56 56 XK_Alt_L 0xffe9 AltLeft LALT alt 0x13 0x3a
117 KEY_LEFTALT 56 Option 0x3a 0x38 0x11 0x19 226 VK_MENU 0x12 56 56 XK_Alt_L 0xffe9 AltLeft LALT alt 0x13 0x3a
118 KEY_SPACE 57 Space 0x31 0x39 0x29 0x29 44 VK_SPACE 0x20 57 57 XK_space 0x0020 Space SPCE spc 0x79 0x31
119 KEY_CAPSLOCK 58 CapsLock 0x39 0x3a 0x58 0x14 57 VK_CAPITAL 0x14 58 58 XK_Caps_Lock 0xffe5 CapsLock CAPS caps_lock 0x77 0x39
120 KEY_F1 59 F1 0x7a 0x3b 0x05 0x07 58 VK_F1 0x70 59 59 XK_F1 0xffbe F1 FK01 f1 0x05 0x7a
121 KEY_F2 60 F2 0x78 0x3c 0x06 0x0f 59 VK_F2 0x71 60 60 XK_F2 0xffbf F2 FK02 f2 0x06 0x78
122 KEY_F3 61 F3 0x63 0x3d 0x04 0x17 60 VK_F3 0x72 61 61 XK_F3 0xffc0 F3 FK03 f3 0x08 0x63
123 KEY_F4 62 F4 0x76 0x3e 0x0c 0x1f 61 VK_F4 0x73 62 62 XK_F4 0xffc1 F4 FK04 f4 0x0a 0x76
124 KEY_F5 63 F5 0x60 0x3f 0x03 0x27 62 VK_F5 0x74 63 63 XK_F5 0xffc2 F5 FK05 f5 0x0c 0x60
125 KEY_F6 64 F6 0x61 0x40 0x0b 0x2f 63 VK_F6 0x75 64 64 XK_F6 0xffc3 F6 FK06 f6 0x0e 0x61
126 KEY_F7 65 F7 0x62 0x41 0x83 0x37 64 VK_F7 0x76 65 65 XK_F7 0xffc4 F7 FK07 f7 0x10 0x62
127 KEY_F8 66 F8 0x64 0x42 0x0a 0x3f 65 VK_F8 0x77 66 66 XK_F8 0xffc5 F8 FK08 f8 0x11 0x64
128 KEY_F9 67 F9 0x65 0x43 0x01 0x47 66 VK_F9 0x78 67 67 XK_F9 0xffc6 F9 FK09 f9 0x12 0x65
129 KEY_F10 68 F10 0x6d 0x44 0x09 0x4f 67 VK_F10 0x79 68 68 XK_F10 0xffc7 F10 FK10 f10 0x07 0x6d
130 KEY_NUMLOCK 69 ANSI_KeypadClear 0x47 0x45 0x77 0x76 83 VK_NUMLOCK 0x90 69 69 XK_Num_Lock 0xff7f NumLock NMLK num_lock 0x62 0x47
131 KEY_SCROLLLOCK 70 0x46 0x7e 0x5f 71 VK_SCROLL 0x91 70 70 XK_Scroll_Lock 0xff14 ScrollLock SCLK scroll_lock 0x17 0x6b
132 KEY_KP7 71 ANSI_Keypad7 0x59 0x47 0x6c 0x6c 95 VK_NUMPAD7 0x67 71 71 XK_KP_7 0xffb7 Numpad7 KP7 kp_7 0x44 0x59
133 KEY_KP8 72 ANSI_Keypad8 0x5b 0x48 0x75 0x75 96 VK_NUMPAD8 0x68 72 72 XK_KP_8 0xffb8 Numpad8 KP8 kp_8 0x45 0x5b
134 KEY_KP9 73 ANSI_Keypad9 0x5c 0x49 0x7d 0x7d 97 VK_NUMPAD9 0x69 73 73 XK_KP_9 0xffb9 Numpad9 KP9 kp_9 0x46 0x5c
135 KEY_KPMINUS 74 ANSI_KeypadMinus 0x4e 0x4a 0x7b 0x4e 86 VK_SUBTRACT 0x6d 74 74 XK_KP_Subtract 0xffad NumpadSubtract KPSU kp_subtract 0x47 0x4e
136 KEY_KP4 75 ANSI_Keypad4 0x56 0x4b 0x6b 0x6b 92 VK_NUMPAD4 0x64 75 75 XK_KP_4 0xffb4 Numpad4 KP4 kp_4 0x5b 0x56
137 KEY_KP5 76 ANSI_Keypad5 0x57 0x4c 0x73 0x73 93 VK_NUMPAD5 0x65 76 76 XK_KP_5 0xffb5 Numpad5 KP5 kp_5 0x5c 0x57
138 KEY_KP6 77 ANSI_Keypad6 0x58 0x4d 0x74 0x74 94 VK_NUMPAD6 0x66 77 77 XK_KP_6 0xffb6 Numpad6 KP6 kp_6 0x5d 0x58
139 KEY_KPPLUS 78 ANSI_KeypadPlus 0x45 0x4e 0x79 0x7c 87 VK_ADD 0x6b 78 78 XK_KP_Add 0xffab NumpadAdd KPAD kp_add 0x7d 0x45
140 KEY_KP1 79 ANSI_Keypad1 0x53 0x4f 0x69 0x69 89 VK_NUMPAD1 0x61 79 79 XK_KP_1 0xffb1 Numpad1 KP1 kp_1 0x70 0x53
141 KEY_KP2 80 ANSI_Keypad2 0x54 0x50 0x72 0x72 90 VK_NUMPAD2 0x62 80 80 XK_KP_2 0xffb2 Numpad2 KP2 kp_2 0x71 0x54
142 KEY_KP3 81 ANSI_Keypad3 0x55 0x51 0x7a 0x7a 91 VK_NUMPAD3 0x63 81 81 XK_KP_3 0xffb3 Numpad3 KP3 kp_3 0x72 0x55
143 KEY_KP0 82 ANSI_Keypad0 0x52 0x52 0x70 0x70 98 VK_NUMPAD0 0x60 82 82 XK_KP_0 0xffb0 Numpad0 KP0 kp_0 0x5e 0x52
144 KEY_KPDOT 83 ANSI_KeypadDecimal 0x41 0x53 0x71 0x71 99 VK_DECIMAL 0x6e 83 83 XK_KP_Decimal 0xffae NumpadDecimal KPDL kp_decimal 0x32 0x41
145 KEY_KPDOT 83 ANSI_KeypadDecimal 0x41 0x53 0x71 0x71 99 VK_DECIMAL 0x6e 83 83 XK_KP_Decimal 0xffae NumpadDecimal KPDC kp_decimal 0x32 0x41
146 84 0x54
147 KEY_ZENKAKUHANKAKU 85 0x76 0x5f 148 Lang5 HZTG
148 KEY_102ND 86 0x56 0x61 0x13 100 VK_OEM_102 0xe1 86 86 IntlBackslash LSGT less 0x7c
149 KEY_F11 87 F11 0x67 0x57 0x78 0x56 68 VK_F11 0x7a 87 87 XK_F11 0xffc8 F11 FK11 f11 0x09 0x67
150 KEY_F12 88 F12 0x6f 0x58 0x07 0x5e 69 VK_F12 0x7b 88 88 XK_F12 0xffc9 F12 FK12 f12 0x0b 0x6f
151 KEY_RO 89 0x73 0x51 135 IntlRo AB11 ro
152 KEY_KATAKANA 90 JIS_Kana 0x68 0x78 0x63 146 VK_KANA 0x15 Katakana KATA
153 KEY_KATAKANA 90 JIS_Kana 0x68 0x78 0x63 146 VK_KANA 0x15 Lang3 KATA
154 KEY_HIRAGANA 91 0x77 0x62 0x87 147 Hiragana HIRA hiragana
155 KEY_HIRAGANA 91 0x77 0x62 0x87 147 Lang4 HIRA hiragana
156 KEY_HENKAN 92 0x79 0x64 0x86 138 Convert HENK henkan
157 KEY_KATAKANAHIRAGANA 93 0x70 0x13 0x87 136 0xc8 0xc8 KanaMode HKTG katakanahiragana
158 KEY_MUHENKAN 94 0x7b 0x67 0x85 139 NonConvert NFER muhenkan
159 KEY_MUHENKAN 94 0x7b 0x67 0x85 139 NonConvert MUHE muhenkan
160 KEY_KPJPCOMMA 95 JIS_KeypadComma 0x5f 0x5c 0x27 140 XK_KP_Separator 0xffac KPSP
161 KEY_KPJPCOMMA 95 JIS_KeypadComma 0x5f 0x5c 0x27 140 XK_KP_Separator 0xffac JPCM
162 KEY_KPENTER 96 ANSI_KeypadEnter 0x4c 0xe01c 0xe05a 0x79 88 0x64 0x64 XK_KP_Enter 0xff8d NumpadEnter KPEN kp_enter 0x5a 0x4c
163 KEY_RIGHTCTRL 97 RightControl 0x3e 0xe01d 0xe014 0x58 228 VK_RCONTROL 0xa3 0x65 0x65 XK_Control_R 0xffe4 ControlRight RCTL ctrl_r 0x4c 0x7d
164 KEY_KPSLASH 98 ANSI_KeypadDivide 0x4b 0xe035 0xe04a 0x4a 84 VK_DIVIDE 0x6f 0x68 0x68 XK_KP_Divide 0xffaf NumpadDivide KPDV kp_divide 0x2e 0x4b
165 KEY_SYSRQ 99 0x54 0x7f 0x57 70 VK_SNAPSHOT 0x2c 0x67 0x67 XK_Sys_Req 0xff15 PrintScreen PRSC print 0x16 0x69
166 KEY_SYSRQ 99 0x54 0x7f 0x57 70 VK_SNAPSHOT 0x2c 0x67 0x67 XK_Sys_Req 0xff15 PrintScreen SYRQ sysrq 0x16 0x69
167 KEY_RIGHTALT 100 RightOption 0x3d 0xe038 0xe011 0x39 230 VK_RMENU 0xa5 0x69 0x69 XK_Alt_R 0xffea AltRight ALGR alt_r 0x0d 0x7c
168 KEY_RIGHTALT 100 RightOption 0x3d 0xe038 0xe011 0x39 230 VK_RMENU 0xa5 0x69 0x69 XK_Alt_R 0xffea AltRight RALT alt_r 0x0d 0x7c
169 KEY_LINEFEED 101 0x5b LNFD lf 0x6f
170 KEY_HOME 102 Home 0x73 0xe047 0xe06c 0x6e 74 VK_HOME 0x24 0x59 0x59 XK_Home 0xff50 Home HOME home 0x34 0x73
171 KEY_UP 103 UpArrow 0x7e 0xe048 0xe075 0x63 82 VK_UP 0x26 0x5a 0x5a XK_Up 0xff52 ArrowUp UP up 0x14 0x3e
172 KEY_PAGEUP 104 PageUp 0x74 0xe049 0xe07d 0x6f 75 VK_PRIOR 0x21 0x5b 0x5b XK_Page_Up 0xff55 PageUp PGUP pgup 0x60 0x74
173 KEY_LEFT 105 LeftArrow 0x7b 0xe04b 0xe06b 0x61 80 VK_LEFT 0x25 0x5c 0x5c XK_Left 0xff51 ArrowLeft LEFT left 0x18 0x3b
174 KEY_RIGHT 106 RightArrow 0x7c 0xe04d 0xe074 0x6a 79 VK_RIGHT 0x27 0x5e 0x5e XK_Right 0xff53 ArrowRight RGHT right 0x1c 0x3c
175 KEY_END 107 End 0x77 0xe04f 0xe069 0x65 77 VK_END 0x23 0x5f 0x5f XK_End 0xff57 End END end 0x4a 0x77
176 KEY_DOWN 108 DownArrow 0x7d 0xe050 0xe072 0x60 81 VK_DOWN 0x28 0x60 0x60 XK_Down 0xff54 ArrowDown DOWN down 0x1b 0x3d
177 KEY_PAGEDOWN 109 PageDown 0x79 0xe051 0xe07a 0x6d 78 VK_NEXT 0x22 0x61 0x61 XK_Page_Down 0xff56 PageDown PGDN pgdn 0x7b 0x79
178 KEY_INSERT 110 0xe052 0xe070 0x67 73 VK_INSERT 0x2d 0x62 0x62 XK_Insert 0xff63 Insert INS insert 0x2c 0x72
179 KEY_DELETE 111 ForwardDelete 0x75 0xe053 0xe071 0x64 76 VK_DELETE 0x2e 0x63 0x63 XK_Delete 0xffff Delete DEL delete 0x42 0x75
180 KEY_DELETE 111 ForwardDelete 0x75 0xe053 0xe071 0x64 76 VK_DELETE 0x2e 0x63 0x63 XK_Delete 0xffff Delete DELE 0x42 0x75
181 KEY_MACRO 112 0xe06f 0xe06f 0x8e I120
182 KEY_MUTE 113 Mute 0x4a 0xe020 0xe023 0x9c 127 VK_VOLUME_MUTE 0xad AudioVolumeMute MUTE audiomute
183 KEY_MUTE 113 Mute 0x4a 0xe020 0xe023 0x9c 239 VK_VOLUME_MUTE 0xad AudioVolumeMute MUTE audiomute
184 KEY_VOLUMEDOWN 114 VolumeDown 0x49 0xe02e 0xe021 0x9d 129 VK_VOLUME_DOWN 0xae AudioVolumeDown VOL- volumedown
185 KEY_VOLUMEDOWN 114 VolumeDown 0x49 0xe02e 0xe021 0x9d 238 VK_VOLUME_DOWN 0xae AudioVolumeDown VOL- volumedown
186 KEY_VOLUMEUP 115 VolumeUp 0x48 0xe030 0xe032 0x95 128 VK_VOLUME_UP 0xaf AudioVolumeUp VOL+ volumeup
187 KEY_VOLUMEUP 115 VolumeUp 0x48 0xe030 0xe032 0x95 237 VK_VOLUME_UP 0xaf AudioVolumeUp VOL+ volumeup
188 KEY_POWER 116 0xe05e 0xe037 102 Power POWR power 0x7f7f
189 KEY_KPEQUAL 117 ANSI_KeypadEquals 0x51 0x59 0x0f 103 0x76 0x76 XK_KP_Equal 0xffbd NumpadEqual KPEQ kp_equals 0x2d 0x51
190 KEY_KPPLUSMINUS 118 0xe04e 0xe079 I126
191 KEY_PAUSE 119 0xe046 0xe077 0x62 72 VK_PAUSE 0x013 0x66 0x66 XK_Pause 0xff13 Pause PAUS pause 0x15 0x71
192 KEY_SCALE 120 0xe00b I128
193 KEY_KPCOMMA 121 0x7e 0x6d 133 VK_SEPARATOR?? 0x6c NumpadComma KPCO kp_comma
194 KEY_KPCOMMA 121 0x7e 0x6d 133 VK_SEPARATOR?? 0x6c NumpadComma I129
195 KEY_HANGEUL 122 144 VK_HANGEUL 0x15 HNGL
196 KEY_HANJA 123 0xe00d 145 VK_HANJA 0x19 HJCV
197 KEY_YEN 124 JIS_Yen 0x5d 0x7d 0x6a 0x5d 137 0x7d 0x7d IntlYen AE13 yen
198 KEY_LEFTMETA 125 Command 0x37 0xe05b 0xe01f 0x8b 227 VK_LWIN 0x5b 0x6b 0x6b XK_Meta_L 0xffe7 MetaLeft LMTA meta_l 0x78 0x37
199 KEY_LEFTMETA 125 Command 0x37 0xe05b 0xe01f 0x8b 227 VK_LWIN 0x5b 0x6b 0x6b XK_Meta_L 0xffe7 MetaLeft LWIN meta_l 0x78 0x37
200 KEY_RIGHTMETA 126 RightCommand 0x36 0xe05c 0xe027 0x8c 231 VK_RWIN 0x5c 0x6c 0x6c XK_Meta_R 0xffe8 MetaRight RMTA meta_r 0x7a 0x37
201 KEY_RIGHTMETA 126 RightCommand 0x36 0xe05c 0xe027 0x8c 231 VK_RWIN 0x5c 0x6c 0x6c XK_Meta_R 0xffe8 MetaRight RWIN meta_r 0x7a 0x37
202 KEY_COMPOSE 127 0x6e 0xe05d 0xe02f 0x8d 101 VK_APPS 0x5d 0x6d 0x6d ContextMenu MENU compose 0x43
203 KEY_COMPOSE 127 0x6e 0xe05d 0xe02f 0x8d 101 VK_APPS 0x5d 0x6d 0x6d ContextMenu COMP compose 0x43
204 KEY_STOP 128 0xe068 0xe028 0x0a 120 VK_BROWSER_STOP 0xa9 BrowserStop STOP stop 0x01
205 KEY_STOP 128 0xe068 0xe028 0x0a 243 VK_BROWSER_STOP 0xa9 BrowserStop STOP stop 0x01
206 KEY_AGAIN 129 0xe005 0x0b 121 Again AGAI again 0x03
207 KEY_PROPS 130 0xe006 0x0c Props PROP props 0x19
208 KEY_UNDO 131 0xe007 0x10 122 Undo UNDO undo 0x1a
209 KEY_FRONT 132 0xe00c 119 FRNT front 0x31
210 KEY_COPY 133 0xe078 0x18 124 Copy COPY copy 0x33
211 KEY_OPEN 134 0x64 0x20 116 Open OPEN open 0x48
212 KEY_PASTE 135 0x65 0x28 125 Paste PAST paste 0x49
213 KEY_FIND 136 0xe041 0x30 126 Find FIND find 0x5f
214 KEY_FIND 136 0xe041 0x30 244 Find FIND find 0x5f
215 KEY_CUT 137 0xe03c 0x38 123 Cut CUT cut 0x61
216 KEY_HELP 138 Help 0x72 0xe075 0x09 117 VK_HELP 0x2f XK_Help 0xff6a Help HELP help 0x76
217 KEY_MENU 139 0xe01e 0x91 118 I147 menu
218 KEY_CALC 140 0xe021 0xe02b 0xa3 251 LaunchApp2 I148 calculator
219 KEY_SETUP 141 0x66 I149
220 KEY_SLEEP 142 0xe05f 0xe03f 248 VK_SLEEP 0x5f Sleep I150 sleep
221 KEY_WAKEUP 143 0xe063 0xe05e WakeUp I151 wake
222 KEY_FILE 144 0x67 I152
223 KEY_SENDFILE 145 0x68 I153
224 KEY_DELETEFILE 146 0x69 I154
225 KEY_XFER 147 0xe013 0xa2 XFER
226 KEY_XFER 147 0xe013 0xa2 I155
227 KEY_PROG1 148 0xe01f 0xa0 I156
228 KEY_PROG2 149 0xe017 0xa1 I157
229 KEY_WWW 150 0xe002 240 I158
230 KEY_MSDOS 151 0x6a I159
231 KEY_SCREENLOCK 152 0xe012 0x96 249 I160
232 KEY_DIRECTION 153 0x6b I161
233 KEY_CYCLEWINDOWS 154 0xe026 0x9b I162
234 KEY_MAIL 155 0xe06c 0xe048 LaunchMail I163 mail
235 KEY_BOOKMARKS 156 0xe066 0xe018 BrowserFavorites I164 ac_bookmarks
236 KEY_COMPUTER 157 0xe06b 0xe040 LaunchApp1 I165 computer
237 KEY_BACK 158 0xe06a 0xe038 241 VK_BROWSER_BACK 0xa6 BrowserBack I166 ac_back
238 KEY_FORWARD 159 0xe069 0xe030 242 VK_BROWSER_FORWARD 0xa7 BrowserForward I167 ac_forward
239 KEY_CLOSECD 160 0xe023 0x9a I168
240 KEY_EJECTCD 161 0x6c 236 I169
241 KEY_EJECTCLOSECD 162 0xe07d Eject I170
242 KEY_NEXTSONG 163 0xe019 0xe04d 0x93 235 VK_MEDIA_NEXT_TRACK 0xb0 MediaTrackNext I171 audionext
243 KEY_PLAYPAUSE 164 0xe022 0xe034 232 VK_MEDIA_PLAY_PAUSE 0xb3 MediaPlayPause I172 audioplay
244 KEY_PREVIOUSSONG 165 0xe010 0xe015 0x94 234 VK_MEDIA_PREV_TRACK 0xb1 MediaTrackPrevious I173 audioprev
245 KEY_STOPCD 166 0xe024 0xe03b 0x98 233 VK_MEDIA_STOP 0xb2 MediaStop I174 audiostop
246 KEY_RECORD 167 0xe031 0x9e I175
247 KEY_REWIND 168 0xe018 0x9f I176
248 KEY_PHONE 169 0x63 I177
249 KEY_ISO 170 ISO_Section 0xa I178
250 KEY_CONFIG 171 0xe001 I179
251 KEY_HOMEPAGE 172 0xe032 0xe03a 0x97 VK_BROWSER_HOME 0xac BrowserHome I180 ac_home
252 KEY_REFRESH 173 0xe067 0xe020 250 VK_BROWSER_REFRESH 0xa8 BrowserRefresh I181 ac_refresh
253 KEY_EXIT 174 0x71 I182
254 KEY_MOVE 175 0x72 I183
255 KEY_EDIT 176 0xe008 247 I184
256 KEY_SCROLLUP 177 0x75 245 I185
257 KEY_SCROLLDOWN 178 0xe00f 246 I186
258 KEY_KPLEFTPAREN 179 0xe076 182 NumpadParenLeft I187
259 KEY_KPRIGHTPAREN 180 0xe07b 183 NumpadParenRight I188
260 KEY_NEW 181 0xe009 I189
261 KEY_REDO 182 0xe00a I190
262 KEY_F13 183 F13 0x69 0x5d 0x2f 0x7f 104 VK_F13 0x7c 0x6e 0x6e F13 FK13 0x69
263 KEY_F14 184 F14 0x6b 0x5e 0x37 0x80 105 VK_F14 0x7d 0x6f 0x6f F14 FK14 0x6b
264 KEY_F15 185 F15 0x71 0x5f 0x3f 0x81 106 VK_F15 0x7e 0x70 0x70 F15 FK15 0x71
265 KEY_F16 186 F16 0x6a 0x55 0x82 107 VK_F16 0x7f 0x71 0x71 F16 FK16
266 KEY_F17 187 F17 0x40 0xe003 0x83 108 VK_F17 0x80 0x72 0x72 F17 FK17
267 KEY_F18 188 F18 0x4f 0xe077 109 VK_F18 0x81 F18 FK18
268 KEY_F19 189 F19 0x50 0xe004 110 VK_F19 0x82 F19 FK19
269 KEY_F20 190 F20 0x5a 0x5a 111 VK_F20 0x83 F20 FK20
270 KEY_F21 191 0x74 112 VK_F21 0x84 F21 FK21
271 KEY_F22 192 0xe079 113 VK_F22 0x85 F22 FK22
272 KEY_F23 193 0x6d 114 VK_F23 0x86 F23 FK23
273 KEY_F24 194 0x6f 115 VK_F24 0x87 F24 FK24
274 195 0xe015
275 196 0xe016
276 197 0xe01a
277 198 0xe01b
278 199 0xe027
279 KEY_PLAYCD 200 0xe028 I208
280 KEY_PAUSECD 201 0xe029 I209
281 KEY_PROG3 202 0xe02b I210
282 KEY_PROG4 203 0xe02c I211
283 KEY_DASHBOARD 204 0xe02d I212
284 KEY_SUSPEND 205 0xe025 Suspend I213
285 KEY_CLOSE 206 0xe02f I214
286 KEY_PLAY 207 0xe033 VK_PLAY 0xfa I215
287 KEY_FASTFORWARD 208 0xe034 I216
288 KEY_BASSBOOST 209 0xe036 I217
289 KEY_PRINT 210 0xe039 VK_PRINT 0x2a I218
290 KEY_HP 211 0xe03a I219
291 KEY_CAMERA 212 0xe03b I220
292 KEY_SOUND 213 0xe03d I221
293 KEY_QUESTION 214 0xe03e I222
294 KEY_EMAIL 215 0xe03f VK_LAUNCH_MAIL 0xb4 I223
295 KEY_CHAT 216 0xe040 I224
296 KEY_SEARCH 217 0xe065 0xe010 VK_BROWSER_SEARCH 0xaa BrowserSearch I225
297 KEY_CONNECT 218 0xe042 I226
298 KEY_FINANCE 219 0xe043 I227
299 KEY_SPORT 220 0xe044 I228
300 KEY_SHOP 221 0xe045 I229
301 KEY_ALTERASE 222 0xe014 I230
302 KEY_CANCEL 223 0xe04a I231
303 KEY_BRIGHTNESSDOWN 224 0xe04c I232
304 KEY_BRIGHTNESSUP 225 0xe054 I233
305 KEY_MEDIA 226 0xe06d 0xe050 MediaSelect I234 mediaselect
306 KEY_SWITCHVIDEOMODE 227 0xe056 I235
307 KEY_KBDILLUMTOGGLE 228 0xe057 I236
308 KEY_KBDILLUMDOWN 229 0xe058 I237
309 KEY_KBDILLUMUP 230 0xe059 I238
310 KEY_SEND 231 0xe05a I239
311 KEY_REPLY 232 0xe064 I240
312 KEY_FORWARDMAIL 233 0xe00e I241
313 KEY_SAVE 234 0xe055 I242
314 KEY_DOCUMENTS 235 0xe070 I243
315 KEY_BATTERY 236 0xe071 I244
316 KEY_BLUETOOTH 237 0xe072 I245
317 KEY_WLAN 238 0xe073 I246
318 KEY_UWB 239 0xe074 I247
319 KEY_UNKNOWN 240 I248
320 KEY_VIDEO_NEXT 241 I249
321 KEY_VIDEO_PREV 242 I250
322 KEY_BRIGHTNESS_CYCLE 243 I251
323 KEY_BRIGHTNESS_ZERO 244 I252
324 KEY_DISPLAY_OFF 245 I253
325 KEY_WIMAX 246
326 247
327 248
328 249
329 250
330 251
331 252
332 253
333 254
334 255 0xe012
335 BTN_MISC 0x100
336 BTN_0 0x100 VK_LBUTTON 0x01
337 BTN_1 0x101 VK_RBUTTON 0x02
338 BTN_2 0x102 VK_MBUTTON 0x04
339 BTN_3 0x103 VK_XBUTTON1 0x05
340 BTN_4 0x104 VK_XBUTTON2 0x06
341 BTN_5 0x105
342 BTN_6 0x106
343 BTN_7 0x107
344 BTN_8 0x108
345 BTN_9 0x109
346 BTN_MOUSE 0x110
347 BTN_LEFT 0x110
348 BTN_RIGHT 0x111
349 BTN_MIDDLE 0x112
350 BTN_SIDE 0x113
351 BTN_EXTRA 0x114
352 BTN_FORWARD 0x115
353 BTN_BACK 0x116
354 BTN_TASK 0x117
355 BTN_JOYSTICK 0x120
356 BTN_TRIGGER 0x120
357 BTN_THUMB 0x121
358 BTN_THUMB2 0x122
359 BTN_TOP 0x123
360 BTN_TOP2 0x124
361 BTN_PINKIE 0x125
362 BTN_BASE 0x126
363 BTN_BASE2 0x127
364 BTN_BASE3 0x128
365 BTN_BASE4 0x129
366 BTN_BASE5 0x12a
367 BTN_BASE6 0x12b
368 BTN_DEAD 0x12f
369 BTN_GAMEPAD 0x130
370 BTN_A 0x130
371 BTN_B 0x131
372 BTN_C 0x132
373 BTN_X 0x133
374 BTN_Y 0x134
375 BTN_Z 0x135
376 BTN_TL 0x136
377 BTN_TR 0x137
378 BTN_TL2 0x138
379 BTN_TR2 0x139
380 BTN_SELECT 0x13a
381 BTN_START 0x13b
382 BTN_MODE 0x13c
383 BTN_THUMBL 0x13d
384 BTN_THUMBR 0x13e
385 BTN_DIGI 0x140
386 BTN_TOOL_PEN 0x140
387 BTN_TOOL_RUBBER 0x141
388 BTN_TOOL_BRUSH 0x142
389 BTN_TOOL_PENCIL 0x143
390 BTN_TOOL_AIRBRUSH 0x144
391 BTN_TOOL_FINGER 0x145
392 BTN_TOOL_MOUSE 0x146
393 BTN_TOOL_LENS 0x147
394 BTN_TOUCH 0x14a
395 BTN_STYLUS 0x14b
396 BTN_STYLUS2 0x14c
397 BTN_TOOL_DOUBLETAP 0x14d
398 BTN_TOOL_TRIPLETAP 0x14e
399 BTN_TOOL_QUADTAP 0x14f
400 BTN_WHEEL 0x150
401 BTN_GEAR_DOWN 0x150
402 BTN_GEAR_UP 0x151
403 KEY_OK 0x160
404 KEY_SELECT 0x161 VK_SELECT 0x29 XK_Select 0xff60 Select SELE
405 KEY_GOTO 0x162
406 KEY_CLEAR 0x163 NumpadClear CLR
407 KEY_POWER2 0x164
408 KEY_OPTION 0x165
409 KEY_INFO 0x166
410 KEY_TIME 0x167
411 KEY_VENDOR 0x168
412 KEY_ARCHIVE 0x169
413 KEY_PROGRAM 0x16a
414 KEY_CHANNEL 0x16b
415 KEY_FAVORITES 0x16c VK_BROWSER_FAVOURITES 0xab
416 KEY_EPG 0x16d
417 KEY_PVR 0x16e
418 KEY_MHP 0x16f
419 KEY_LANGUAGE 0x170
420 KEY_TITLE 0x171
421 KEY_SUBTITLE 0x172
422 KEY_ANGLE 0x173
423 KEY_ZOOM 0x174 VK_ZOOM 0xfb
424 KEY_MODE 0x175
425 KEY_KEYBOARD 0x176
426 KEY_SCREEN 0x177
427 KEY_PC 0x178
428 KEY_TV 0x179
429 KEY_TV2 0x17a
430 KEY_VCR 0x17b
431 KEY_VCR2 0x17c
432 KEY_SAT 0x17d
433 KEY_SAT2 0x17e
434 KEY_CD 0x17f
435 KEY_TAPE 0x180
436 KEY_RADIO 0x181
437 KEY_TUNER 0x182
438 KEY_PLAYER 0x183
439 KEY_TEXT 0x184
440 KEY_DVD 0x185
441 KEY_AUX 0x186
442 KEY_MP3 0x187
443 KEY_AUDIO 0x188
444 KEY_VIDEO 0x189
445 KEY_DIRECTORY 0x18a
446 KEY_LIST 0x18b
447 KEY_MEMO 0x18c
448 KEY_CALENDAR 0x18d
449 KEY_RED 0x18e
450 KEY_GREEN 0x18f
451 KEY_YELLOW 0x190
452 KEY_BLUE 0x191
453 KEY_CHANNELUP 0x192
454 KEY_CHANNELDOWN 0x193
455 KEY_FIRST 0x194
456 KEY_LAST 0x195
457 KEY_AB 0x196
458 KEY_NEXT 0x197
459 KEY_RESTART 0x198
460 KEY_SLOW 0x199
461 KEY_SHUFFLE 0x19a
462 KEY_BREAK 0x19b BREA
463 KEY_BREAK 0x19b BRK
464 KEY_PREVIOUS 0x19c
465 KEY_DIGITS 0x19d
466 KEY_TEEN 0x19e
467 KEY_TWEN 0x19f
468 KEY_VIDEOPHONE 0x1a0
469 KEY_GAMES 0x1a1
470 KEY_ZOOMIN 0x1a2
471 KEY_ZOOMOUT 0x1a3
472 KEY_ZOOMRESET 0x1a4
473 KEY_WORDPROCESSOR 0x1a5
474 KEY_EDITOR 0x1a6
475 KEY_SPREADSHEET 0x1a7
476 KEY_GRAPHICSEDITOR 0x1a8
477 KEY_PRESENTATION 0x1a9
478 KEY_DATABASE 0x1aa
479 KEY_NEWS 0x1ab
480 KEY_VOICEMAIL 0x1ac
481 KEY_ADDRESSBOOK 0x1ad
482 KEY_MESSENGER 0x1ae
483 KEY_DISPLAYTOGGLE 0x1af
484 KEY_SPELLCHECK 0x1b0
485 KEY_LOGOFF 0x1b1
486 KEY_DOLLAR 0x1b2
487 KEY_EURO 0x1b3
488 KEY_FRAMEBACK 0x1b4
489 KEY_FRAMEFORWARD 0x1b5
490 KEY_CONTEXT_MENU 0x1b6
491 KEY_MEDIA_REPEAT 0x1b7
492 KEY_DEL_EOL 0x1c0
493 KEY_DEL_EOS 0x1c1
494 KEY_INS_LINE 0x1c2
495 KEY_DEL_LINE 0x1c3
496 KEY_FN 0x1d0 Function 0x3f Fn
497 KEY_FN_ESC 0x1d1
498 KEY_FN_F1 0x1d2
499 KEY_FN_F2 0x1d3
500 KEY_FN_F3 0x1d4
501 KEY_FN_F4 0x1d5
502 KEY_FN_F5 0x1d6
503 KEY_FN_F6 0x1d7
504 KEY_FN_F7 0x1d8
505 KEY_FN_F8 0x1d9
506 KEY_FN_F9 0x1da
507 KEY_FN_F10 0x1db
508 KEY_FN_F11 0x1dc
509 KEY_FN_F12 0x1dd
510 KEY_FN_1 0x1de
511 KEY_FN_2 0x1df
512 KEY_FN_D 0x1e0
513 KEY_FN_E 0x1e1
514 KEY_FN_F 0x1e2
515 KEY_FN_S 0x1e3
516 KEY_FN_B 0x1e4
517 KEY_BRL_DOT1 0x1f1
518 KEY_BRL_DOT2 0x1f2
519 KEY_BRL_DOT3 0x1f3
520 KEY_BRL_DOT4 0x1f4
521 KEY_BRL_DOT5 0x1f5
522 KEY_BRL_DOT6 0x1f6
523 KEY_BRL_DOT7 0x1f7
524 KEY_BRL_DOT8 0x1f8
525 KEY_BRL_DOT9 0x1f9
526 KEY_BRL_DOT10 0x1fa
527 KEY_NUMERIC_0 0x200
528 KEY_NUMERIC_1 0x201
529 KEY_NUMERIC_2 0x202
530 KEY_NUMERIC_3 0x203
531 KEY_NUMERIC_4 0x204
532 KEY_NUMERIC_5 0x205
533 KEY_NUMERIC_6 0x206
534 KEY_NUMERIC_7 0x207
535 KEY_NUMERIC_8 0x208
536 KEY_NUMERIC_9 0x209
537 KEY_NUMERIC_STAR 0x20a NumpadStar
538 KEY_NUMERIC_POUND 0x20b NumpadHash
539 KEY_RFKILL 0x20c

View File

@@ -1,11 +0,0 @@
osx2win32.*
osx2win32_name.*
osx2xkb.*
osx2xkb_name.*
html2win32.*
html2win32_name.*
osx.*
osx_name.*
stdc
stdc++
node_modules/

View File

@@ -1,118 +0,0 @@
TESTS := stdc stdc++ python2 python3 javascript
check: $(TESTS)
@set -e; for fn in $(TESTS); do \
./$$fn; \
echo $$fn: OK; \
done
@echo Done.
GEN := ../tools/keymap-gen
DATA := ../data/keymaps.csv
SOURCES := $(GEN) $(DATA)
.DELETE_ON_ERROR:
stdc: stdc.c osx2win32.h osx2win32_name.h \
osx2xkb.h osx2xkb_name.h \
html2win32.h html2win32_name.h \
osx.h osx_name.h
$(CC) -Wall -o $@ $^
osx2win32.h: $(SOURCES)
$(GEN) --lang stdc code-map $(DATA) osx win32 > $@
osx2win32_name.h: $(SOURCES)
$(GEN) --lang stdc name-map $(DATA) osx win32 > $@
osx2xkb.h: $(SOURCES)
$(GEN) --lang stdc code-map $(DATA) osx xkb > $@
osx2xkb_name.h: $(SOURCES)
$(GEN) --lang stdc name-map $(DATA) osx xkb > $@
html2win32.h: $(SOURCES)
$(GEN) --lang stdc code-map $(DATA) html win32 > $@
html2win32_name.h: $(SOURCES)
$(GEN) --lang stdc name-map $(DATA) html win32 > $@
osx.h: $(SOURCES)
$(GEN) --lang stdc code-table $(DATA) osx > $@
osx_name.h: $(SOURCES)
$(GEN) --lang stdc name-table $(DATA) osx > $@
stdc++: stdc++.cc osx2win32.hh osx2win32_name.hh \
osx2xkb.hh osx2xkb_name.hh \
html2win32.hh html2win32_name.hh \
osx.hh osx_name.hh
$(CXX) -Wall -std=c++11 -o $@ $^
osx2win32.hh: $(SOURCES)
$(GEN) --lang stdc++ code-map $(DATA) osx win32 > $@
osx2win32_name.hh: $(SOURCES)
$(GEN) --lang stdc++ name-map $(DATA) osx win32 > $@
osx2xkb.hh: $(SOURCES)
$(GEN) --lang stdc++ code-map $(DATA) osx xkb > $@
osx2xkb_name.hh: $(SOURCES)
$(GEN) --lang stdc++ name-map $(DATA) osx xkb > $@
html2win32.hh: $(SOURCES)
$(GEN) --lang stdc++ code-map $(DATA) html win32 > $@
html2win32_name.hh: $(SOURCES)
$(GEN) --lang stdc++ name-map $(DATA) html win32 > $@
osx.hh: $(SOURCES)
$(GEN) --lang stdc++ code-table $(DATA) osx > $@
osx_name.hh: $(SOURCES)
$(GEN) --lang stdc++ name-table $(DATA) osx > $@
python2: osx2win32.py osx2win32_name.py \
osx2xkb.py osx2xkb_name.py \
html2win32.py html2win32_name.py \
osx.py osx_name.py
osx2win32.py: $(SOURCES)
$(GEN) --lang python2 code-map $(DATA) osx win32 > $@
osx2win32_name.py: $(SOURCES)
$(GEN) --lang python2 name-map $(DATA) osx win32 > $@
osx2xkb.py: $(SOURCES)
$(GEN) --lang python2 code-map $(DATA) osx xkb > $@
osx2xkb_name.py: $(SOURCES)
$(GEN) --lang python2 name-map $(DATA) osx xkb > $@
html2win32.py: $(SOURCES)
$(GEN) --lang python2 code-map $(DATA) html win32 > $@
html2win32_name.py: $(SOURCES)
$(GEN) --lang python2 name-map $(DATA) html win32 > $@
osx.py: $(SOURCES)
$(GEN) --lang python2 code-table $(DATA) osx > $@
osx_name.py: $(SOURCES)
$(GEN) --lang python2 name-table $(DATA) osx > $@
javascript: node_modules/babel-core \
node_modules/babel-plugin-transform-es2015-modules-commonjs \
osx2win32.js osx2win32_name.js \
osx2xkb.js osx2xkb_name.js \
html2win32.js html2win32_name.js \
osx.js osx_name.js
node_modules/babel-core:
npm install babel-core
node_modules/babel-plugin-transform-es2015-modules-commonjs:
npm install babel-plugin-transform-es2015-modules-commonjs
osx2win32.js: $(SOURCES)
$(GEN) --lang js code-map $(DATA) osx win32 > $@
osx2win32_name.js: $(SOURCES)
$(GEN) --lang js name-map $(DATA) osx win32 > $@
osx2xkb.js: $(SOURCES)
$(GEN) --lang js code-map $(DATA) osx xkb > $@
osx2xkb_name.js: $(SOURCES)
$(GEN) --lang js name-map $(DATA) osx xkb > $@
html2win32.js: $(SOURCES)
$(GEN) --lang js code-map $(DATA) html win32 > $@
html2win32_name.js: $(SOURCES)
$(GEN) --lang js name-map $(DATA) html win32 > $@
osx.js: $(SOURCES)
$(GEN) --lang js code-table $(DATA) osx > $@
osx_name.js: $(SOURCES)
$(GEN) --lang js name-table $(DATA) osx > $@
clean:
rm -rf node_modules
rm -f osx2win32.*
rm -f osx2win32_name.*
rm -f osx2xkb.*
rm -f osx2xkb_name.*
rm -f html2win32.*
rm -f html2win32_name.*
rm -f osx.*
rm -f osx_name.*
rm -f stdc stdc++

View File

@@ -1,53 +0,0 @@
#!/usr/bin/env node
/*
* Keycode Map Generator JavaScript Tests
*
* Copyright 2017 Pierre Ossman for Cendio AB
*
* This file is dual license under the terms of the GPLv2 or later
* and 3-clause BSD licenses.
*/
"use strict";
var assert = require('assert');
var babel = require('babel-core');
var fs = require('fs');
function include(fn) {
var options = {
plugins: ["transform-es2015-modules-commonjs"]
};
var code = babel.transformFileSync(fn, options).code;
fs.writeFileSync("." + fn + "_nodejs.js", code);
var imp = require("./." + fn + "_nodejs.js");
fs.unlinkSync("./." + fn + "_nodejs.js");
return imp
}
var code_map_osx_to_win32 = include("osx2win32.js").default;
var name_map_osx_to_win32 = include("osx2win32_name.js").default;
var code_map_osx_to_xkb = include("osx2xkb.js").default;
var name_map_osx_to_xkb = include("osx2xkb_name.js").default;
var code_map_html_to_win32 = include("html2win32.js").default;
var name_map_html_to_win32 = include("html2win32_name.js").default;
var code_table_osx = include("osx.js").default;
var name_table_osx = include("osx_name.js").default;
assert.equal(code_map_osx_to_win32[0x1d], 0x30);
assert.equal(name_map_osx_to_win32[0x1d], "VK_0");
assert.equal(code_map_osx_to_xkb[0x1d], "AE10");
assert.equal(name_map_osx_to_xkb[0x1d], "AE10");
assert.equal(code_map_html_to_win32["ControlLeft"], 0x11);
assert.equal(name_map_html_to_win32["ControlLeft"], "VK_CONTROL");
assert.equal(code_table_osx[0x1d], 0x3b);
assert.equal(name_table_osx[0x1d], "Control");

Some files were not shown because too many files have changed in this diff Show More