Commit Graph

74717 Commits (15b015690b62c7c463fa6f5cdb2bc669749eefc3)

Author SHA1 Message Date
Philippe Mathieu-Daudé 15b015690b tests/qemu-iotests/check: Allow use of python3 interpreter
As we want to enforce a unique and explicit Python 3 interpreter,
we need let this script handle 'python3' too.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé bc7f3b0b0d scripts/checkpatch.pl: Only allow Python 3 interpreter
Since commit ddf9069963 QEMU requires Python >= 3.5.

PEP 0394 [*] states that 'python3' should be available and
that 'python' is optional.

To avoid problem with unsupported versions, enforce the
shebang interpreter to Python 3.

[*] https://www.python.org/dev/peps/pep-0394/

Reported-by: John Snow <jsnow@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-2-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 5427ecd4ab tests/acceptance/migration: Default to -nodefaults
We don't need the default options to run this test.

This fixes errors when running a binary built with
--without-default-devices such:

  ERROR: qemu-system-arm: Unsupported NIC model: virtio-net-pci

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200129212345.20547-28-philmd@redhat.com>
[PMD: Rebased]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé a091864943 tests/acceptance/migration: Add the 'migration' tag
By using an Avocado tag, we can run all tests described by that
tag as once:

  $ avocado --show=app run -t migration tests/acceptance/
  JOB ID     : 165477737509503fcfa6d7108057a0a18f2a6559
  JOB LOG    : avocado/job-results/job-2020-02-04T17.29-1654777/job.log
   (1/3) tests/acceptance/migration.py:Migration.test_migration_with_tcp_localhost: PASS (0.38 s)
   (2/3) tests/acceptance/migration.py:Migration.test_migration_with_unix: PASS (0.33 s)
   (3/3) tests/acceptance/migration.py:Migration.test_migration_with_exec: PASS (0.07 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200204163304.14616-1-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana 2e768cb682 tests/acceptance/migration: Test EXEC transport when migrating
Along with VM migration via TCP, we can use migration through
the EXEC transport protocol

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-3-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana b2cf8d4704 tests/acceptance/migration: Test UNIX transport when migrating
Along with VM migration via TCP, we can use migration through
the UNIX transport protocol

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-3-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana 63adf16d4f tests/acceptance/migration: Factor out do_migrate()
We are going to reuse this code when testing different transport
methods, so factor it out first.

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-2-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana d7c9a83344 tests/acceptance/migration: Factor out assert_migration()
We are going to reuse this code when testing different transport
methods, so factor it out first

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-2-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé b79e55f09b tests/acceptance/version: Default to -nodefaults
We don't need the default options to run this test.

This fixes errors when running a binary built with
--without-default-devices such:

  ERROR: qemu-system-arm: Unsupported NIC model: virtio-net-pci

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200129212345.20547-29-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 1d77f1b14f tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine
As we only read the serial console, we don't need to force a
VGA display. This fixes when running a binary built with
--without-default-devices:

  ERROR: qemu-system-alpha: standard VGA not available

We also need the '-nodefaults' argument to avoid:

  ERROR: qemu-system-alpha: Unsupported NIC model: e1000

Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200129212345.20547-27-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 2d6a6e238a tests/acceptance/virtio_check_params: Disable the test
This test fails on various CI:

- Using QEMU 4.0:
  tests/acceptance/x86_cpu_model_versions.py:X86CPUModelAliases.test_none_alias:  ERROR: 'alias-of' (0.45 s)

- On OSX
  Unexpected error in object_property_find() at qom/object.c:1201:
  qemu-system-x86_64: -device virtio-blk-pci,id=scsi0,drive=drive0: can't apply global virtio-blk-device.scsi=true: Property '.scsi' not found

- When removing unavailable machine:
  VirtioMaxSegSettingsCheck.test_machine_types: ERROR: list.remove(x): x not in list (0.12 s)

- Using Xen:
  xencall: error: Could not obtain handle on privileged command interface: No such file or directory
  xen be core: xen be core: can't open xen interface

- On PPC:
  TestFail: machine type pseries-2.8: No Transactional Memory support in TCG, try appending -machine cap-htm=off

- On S390X configured with --without-default-devices:
  ERROR: qemu-system-s390x: -device virtio-scsi-pci,id=scsi0: 'virtio-scsi-pci' is not a valid device model name

Disable it for now.

Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20200206171715.25041-1-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 77f50e826f tests/acceptance/virtio_check_params: Default to -nodefaults
We don't need the default options to run this test.

This fixes errors when running a binary built with
--without-default-devices such:

  ERROR: qemu-system-s390x: Unknown device 'virtio-net-ccw' for bus 'virtual-css-bus'

Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200129212345.20547-25-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé e435f66bd3 tests/acceptance/virtio_check_params: List machine being tested
Add logging for easier debugging of failures:

  $ avocado --show=machine run tests/acceptance/virtio_check_params.py
   (1/1) tests/acceptance/virtio_check_params.py:VirtioMaxSegSettingsCheck.test_machine_types:
  machine: {'name': 'pc-i440fx-2.12', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-2.0', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-q35-4.2', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-2.5', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-4.2', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  ...

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20200129212345.20547-19-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 0400937be1 tests/acceptance/virtio_check_params: Improve exception logging
Message-Id: <20200129212345.20547-18-philmd@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Denis Plotnikov 47d4c6e696 tests: rename virtio_seg_max_adjust to virtio_check_params
Since, virtio_seg_max_adjust checks not only seg_max, but also
virtqueue_size parameter, let's make the test more general and
add new parameters to be checked there in the future.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-Id: <20200129140702.5411-5-dplotnikov@virtuozzo.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Thomas Huth b0065e1f19 tests/acceptance: Add boot tests for some of the QEMU advent calendar images
The 2018 edition of the QEMU advent calendar 2018 featured Linux images
for various non-x86 machines. We can use them for a boot tests in our
acceptance test suite.

Let's also make sure that we build the corresponding machines in Travis.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200124170325.30072-1-thuth@redhat.com>
[PMD: Rebased, --python=python3 parameter dropped in commit 5311cb12e]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé b6f0a43463 tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'
Avocado tags are handy to automatically select tests matching
the tags. Since this test also runs U-Boot, tag it.

We can run all the tests using U-Boot as once with:

  $ avocado --show=app run -t u-boot tests/acceptance/
  JOB LOG    : avocado/job-results/job-2020-01-21T00.16-ee9344e/job.log
   (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2: PASS (16.59 s)
   (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot: PASS (0.47 s)
   (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot: PASS (2.43 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 19.78 s

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200120235159.18510-8-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Liam Merwick 4526038872 tests/boot_linux_console: fix extract_from_deb() comment
The second param in extract_from_deb() is 'path' not 'file'

Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1580142994-1836-4-git-send-email-liam.merwick@oracle.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Liam Merwick 921a9f6db8 tests/boot_linux_console: use os.path for filesystem paths
Change extract_from_deb() to use os.path routines to manipulate the
filesystem path returned when extracting a file.

Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1580914565-19675-7-git-send-email-liam.merwick@oracle.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Liam Merwick 76a901d295 tests/boot_linux_console: add extract_from_rpm method
Add a method to extract a specified file from an RPM to the test's
working directory and return the path to the extracted file.

Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1580914565-19675-5-git-send-email-liam.merwick@oracle.com>
[PMD: Rebased]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Liam Merwick 8a83708892 travis.yml: install rpm2cpio for acceptance tests
The extract_from_rpm() method added for the PVH acceptance tests needs
rpm2cpio to extract a vmlinux binary from an RPM.

Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1580914565-19675-4-git-send-email-liam.merwick@oracle.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 647eb26a6c Acceptance tests: Add interrupt_interactive_console_until_pattern()
We need a function to interrupt interactive consoles.

Example: Interrupt U-Boot to set different environment values.

Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200120235159.18510-4-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé a91ba1d3ef Acceptance tests: Extract _console_interaction()
Since we are going to re-use the code shared between
wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
extract the common part into a local function.

Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200120235159.18510-3-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé 746f244d97 python/qemu/machine: Allow to use other serial consoles than default
Currently the QEMU Python module limits the QEMUMachine class to
use the first serial console.

Some machines/guest might use another console than the first one as
the 'boot console'. For example the Raspberry Pi uses the second
(AUX) console.

To be able to use the Nth console as default, we simply need to
connect all the N - 1 consoles to the null chardev.

Add an index argument, so we can use a specific serial console as
default.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200120235159.18510-5-f4bug@amsat.org>
[PMD: zero-initialize _console_index in __init__()]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Lukáš Doktor 52c68b8abb python: Treat None-return of greeting cmd
In case qemu process dies the "monitor.cmd" returns None which gets
passed to the "__negotiate_capabilities" and leads to unhandled
exception. Let's only check the resp in case it has a value.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200120071202.30646-1-ldoktor@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Thomas Huth a1e8bd2f5c qemu-deprecated: Remove text about Python 2
Python 2 support has been removed, so we should now also remove
the announcement text for the deprecation.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20200109095116.18201-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta 51aa9d127d python/qemu: accel: Fix kvm_available() on ppc64le
On ppc64le, the accel.kvm_available() check may wrongly
return False because the host arch (as returned by os.uname[4])
and the target arch (ppc64) mismatch. In order to solve this
it is added an ppc64le -> ppc64 mapping which is used as an
fallback verification.

Fixes: 53a049d7d7
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200205203250.30526-5-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta 50d189c97a python/qemu: qmp: Remove unnused attributes
The `error` and `timeout` attributes in QEMUMonitorProtocol are
not used, so this delete them.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191227134101.244496-6-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta 07608e807d python/qemu: qmp: Make QEMUMonitorProtocol a context manager
This implement the __enter__ and __exit__ functions on
QEMUMonitorProtocol class so that it can be used on 'with'
statement and the resources will be free up on block end:

with QEMUMonitorProtocol(socket_path) as qmp:
    qmp.connect()
    qmp.command('query-status')

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20200204141111.3207-5-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta 54aafc2fc6 python/qemu: qmp: Make accept()'s timeout configurable
Currently the timeout of QEMUMonitorProtocol.accept() is
hard-coded to 15.0 seconds. This added the parameter `timeout`
so the value can be configured by the user.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20200204141111.3207-4-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta a0a261e472 python/qemu: Delint the qmp module
This clean up the pylint-3 report on qmp:

************* Module qemu.qmp
python/qemu/qmp.py:1:0: C0111: Missing module docstring (missing-docstring)
python/qemu/qmp.py:17:0: C0111: Missing class docstring (missing-docstring)
python/qemu/qmp.py:21:0: C0111: Missing class docstring (missing-docstring)
python/qemu/qmp.py:25:0: C0111: Missing class docstring (missing-docstring)
python/qemu/qmp.py:29:0: C0111: Missing class docstring (missing-docstring)
python/qemu/qmp.py:33:0: C0111: Missing class docstring (missing-docstring)
python/qemu/qmp.py:33:0: R0205: Class 'QEMUMonitorProtocol' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
python/qemu/qmp.py:80:4: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
python/qemu/qmp.py:131:4: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
python/qemu/qmp.py:159:4: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
python/qemu/qmp.py:245:4: C0111: Missing method docstring (missing-docstring)
python/qemu/qmp.py:249:4: C0111: Missing method docstring (missing-docstring)
python/qemu/qmp.py:252:4: C0111: Missing method docstring (missing-docstring)
python/qemu/qmp.py:255:4: C0111: Missing method docstring (missing-docstring)

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191227134101.244496-3-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Wainer dos Santos Moschetta 10e163c039 python/qemu: qmp: Replace socket.error with OSError
The socket.error is deprecated from Python 3.3, instead it is
made a link to OSError. This change replaces the occurences
of socket.error with OSError.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20191227134101.244496-2-wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Peter Maydell 863d2ed582 Block patches:
- Drop BDRV_SECTOR_SIZE from qcow2
 - Allow Python iotests to be added to the auto group
   (and add some)
 - Fix for the backup job
 - Fix memleak in bdrv_refresh_filename()
 - Use GStrings in two places for greater efficiency (than manually
   handling string allocation)
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl48C4YSHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9A1GsH/2qYR3qqZM6y3fJB0sfaARaKiTS9ac9f
 QQTAs2TEgXkzjdXUgIZu2ieJXPKDK+li7wYGVuMsaHPviscbJH3dKcDprjASHalR
 u7N913u+PB0eY3aTkHyZZwdoxCGQvMAvrHqkyAXrSdgqssodZ7Uj9/jjpGg0z3Io
 JE/eu11RdrOX7eOSZbEAYuaRg4VtrBuYIqwSihlRk0cBxjAScfW2ppa2P/Q8+VPa
 v9bTlFvWQALjG/b0uZx5EgbGztyYzy6PuhdDb+B0S9IsGRidtsxCYZ4pmwQuroJ4
 9mITIj0OAzN3Nw80SqTEmW0jtBecmiOOHrCxEREWYjNSP1MbVF5qSTI=
 =abrL
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-02-06' into staging

Block patches:
- Drop BDRV_SECTOR_SIZE from qcow2
- Allow Python iotests to be added to the auto group
  (and add some)
- Fix for the backup job
- Fix memleak in bdrv_refresh_filename()
- Use GStrings in two places for greater efficiency (than manually
  handling string allocation)

# gpg: Signature made Thu 06 Feb 2020 12:50:14 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-02-06:
  iotests: add test for backup-top failure on permission activation
  block/backup-top: fix failure path
  qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value
  qcow2: Don't require aligned offsets in qcow2_co_copy_range_from()
  qcow2: Use bs->bl.request_alignment when updating an L1 entry
  qcow2: Tighten cluster_offset alignment assertions
  qcow2: Don't round the L1 table allocation up to the sector size
  iotests: Enable more tests in the 'auto' group to improve test coverage
  iotests: Skip Python-based tests if QEMU does not support virtio-blk
  iotests: Check for the availability of the required devices in 267 and 127
  iotests: Test 183 does not work on macOS and OpenBSD
  iotests: Test 041 only works on certain systems
  iotests: remove 'linux' from default supported platforms
  qcow2: Use a GString in report_unsupported_feature()
  block: fix memleaks in bdrv_refresh_filename
  block: Use a GString in bdrv_perm_names()
  qcow2: Assert that host cluster offsets fit in L2 table entries

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-06 16:22:05 +00:00
Peter Maydell 2021b7c971 trivial patches 20200206
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAl48C08SHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748M3UQAKf5Fvp3gc2M5hqCENmK1sISs5Xnh/In
 vIVRNJwLsC5PPBmgHSRm+rP0lcS3c9Nr9BwyOm3ppxmiBXzAjgC/fQOxK7KjTqyQ
 s2eys658K7+kWyZhqc/VE/A1xnPbOY2WDjwTgW6QPZAXmOel2+WcTXoXCpBdgFNt
 SOG1EpcxXkCx6H3ZDmrEZBB4Fn/iZO00njIDsinL53MZv0aeIu1iTGFq1bsxcL7z
 RzCqRF3HT5XvV0CCjin73eSBzQ29IZHf0MxsQ4uuHv78ZRDqdFbhmw5TZ+FHAcsX
 oThe2fPyC3xdLDRdJH0dyA22LoHCE0g+JuZhQb0PTLDIdOGRkTC3L3XHS4saLW5N
 UlzRyKCxqs9M6S/dHCJ/HbwOgDi5cViLptajMSWTgTVpTeIvLIwEaCUs2GPDzGP/
 X3Bgkg4CcTlB9pa4UL7LmDmn5VTpjHJePAKMsHEuQPEaTSDUVtLj/lQOdcwEPqQR
 mnSBYtSKRWBy2C5LEhO1a0WAGVZfvYpt61yrOYCNkuy5DChsleJcUWXstHz+OmXn
 VHy6mxcP/uj+OddHGi5xi1Evz5JhXBNQOAsLPRopeLqV+Mk4i2hqifRFRf3HCqbO
 LaKqfIbWkI+TKkHo03+5ny/JZ3oaJ7TX6TjfBeA66NeXB1pcrgvs1h8aTcdkIRRk
 MLkfGdnoRiUq
 =Occ1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

trivial patches 20200206

# gpg: Signature made Thu 06 Feb 2020 12:49:19 GMT
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  MAINTAINERS: Cc the qemu-arm@nongnu.org for the ARM machines
  aspeed/i2c: Prevent uninitialized warning
  hw/pci/pci_bridge: Fix typo in comment
  qemu-img: Place the '-i aio' option in alphabetical order
  qemu-options: replace constant 1 with HAS_ARG
  MAINTAINERS: Cover hppa-softmmu.mak in the HP-PARISC Machines section
  hw/i386/vmmouse: Fix crash when using the vmmouse on a machine without vmport
  hw/bt: Remove empty Kconfig file
  hw/timer/m48t59: Convert debug printf()s to trace events
  MAINTAINERS: update Leif Lindholm's address
  monitor: fix memory leak in monitor_fdset_dup_fd_find_remove
  hw/smbios/smbios: Remove unused include

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-06 12:57:54 +00:00
Vladimir Sementsov-Ogievskiy a541fcc27c iotests: add test for backup-top failure on permission activation
This test checks that bug is really fixed by previous commit.

Cc: qemu-stable@nongnu.org # v4.2.0
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20200121142802.21467-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Vladimir Sementsov-Ogievskiy 0df62f45c1 block/backup-top: fix failure path
We can't access top after call bdrv_backup_top_drop, as it is already
freed at this time.

Also, no needs to unref target child by hand, it will be unrefed on
bdrv_close() automatically.

So, just do bdrv_backup_top_drop if append succeed and one bdrv_unref
otherwise.

Note, that in !appended case bdrv_unref(top) moved into drained section
on source. It doesn't really matter, but just for code simplicity.

Fixes: 7df7868b96
Cc: qemu-stable@nongnu.org # v4.2.0
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20200121142802.21467-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia 3afea40243 qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value
This replaces all remaining instances in the qcow2 code.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: b5f74b606c2d9873b12d29acdb7fd498029c4025.1579374329.git.berto@igalia.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia 25ae71db55 qcow2: Don't require aligned offsets in qcow2_co_copy_range_from()
qemu-img's convert_co_copy_range() operates at the sector level and
block_copy() operates at the cluster level so this condition is always
true, but it is not necessary to restrict this here, so let's leave it
to the driver implementation return an error if there is any.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: a4264aaee656910c84161a2965f7a501437379ca.1579374329.git.berto@igalia.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia da86f8cbad qcow2: Use bs->bl.request_alignment when updating an L1 entry
When updating an L1 entry the qcow2 driver writes a (512-byte) sector
worth of data to avoid a read-modify-write cycle. Instead of always
writing 512 bytes we should follow the alignment requirements of the
storage backend.

(the only exception is when the alignment is larger than the cluster
size because then we could be overwriting data after the L1 table)

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 71f34d4ae4b367b32fb36134acbf4f4f7ee681f4.1579374329.git.berto@igalia.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia 344ffea951 qcow2: Tighten cluster_offset alignment assertions
qcow2_alloc_cluster_offset() and qcow2_get_cluster_offset() always
return offsets that are cluster-aligned so don't just check that they
are sector-aligned.

The check in qcow2_co_preadv_task() is also replaced by an assertion
for the same reason.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 558ba339965f858bede4c73ce3f50f0c0493597d.1579374329.git.berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia ef97d608c7 qcow2: Don't round the L1 table allocation up to the sector size
The L1 table is read from disk using the byte-based bdrv_pread() and
is never accessed beyond its last element, so there's no need to
allocate more memory than that.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: b2e27214ec7b03a585931bcf383ee1ac3a641a10.1579374329.git.berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth ce95a15e42 iotests: Enable more tests in the 'auto' group to improve test coverage
According to Kevin, tests 030, 040 and 041 are among the most valuable
tests that we have, so we should always run them if possible, even if
they take a little bit longer.

According to Max, it would be good to have a test for iothreads and
migration. 127 and 256 seem to be good candidates for iothreads. For
migration, let's enable 181 and 203 (which also tests iothreads).
(091 would be a good candidate for migration, too, but Alex Bennée
reported that this test fails on ZFS file systems, so it can't be
included yet)

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-7-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth cd2058289b iotests: Skip Python-based tests if QEMU does not support virtio-blk
We are going to enable some of the python-based tests in the "auto" group,
and these tests require virtio-blk to work properly. Running iotests
without virtio-blk likely does not make too much sense anyway, so instead
of adding a check for the availability of virtio-blk to each and every
test (which does not sound very appealing), let's rather add a check for
this a central spot in the "check" script instead (so that it is still
possible to run "make check" for qemu-system-tricore for example).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-6-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth 9bdabfbe72 iotests: Check for the availability of the required devices in 267 and 127
We are going to enable 127 in the "auto" group, but it only works if
virtio-scsi and scsi-hd are available - which is not the case with
QEMU binaries like qemu-system-tricore for example, so we need a
proper check for the availability of these devices here.

A very similar problem exists in iotest 267 - it has been added to
the "auto" group already, but requires virtio-blk and thus currently
fails with qemu-system-tricore for example. Let's also add aproper
check there.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-5-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth 30ad36f55f iotests: Test 183 does not work on macOS and OpenBSD
In the long run, we might want to add test 183 to the "auto" group
(but it still fails occasionally, so we cannot do that yet). However,
when running 183 in Cirrus-CI on macOS, or with our vm-build-openbsd
target, it currently always fails with an "Timeout waiting for return
on handle 0" error.

Let's mark it as supported only on systems where the test is working
most of the time (i.e. Linux, FreeBSD and NetBSD).

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-4-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth 877d18f2aa iotests: Test 041 only works on certain systems
041 works fine on Linux, FreeBSD, NetBSD and OpenBSD, but fails on macOS.
Let's mark it as only supported on the systems where we know that it is
working fine.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-3-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
John Snow 72b2903056 iotests: remove 'linux' from default supported platforms
verify_platform will check an explicit whitelist and blacklist instead.
The default will now be assumed to be allowed to run anywhere.

For tests that do not specify their platforms explicitly, this has the effect of
enabling these tests on non-linux platforms. For tests that always specified
linux explicitly, there is no change.

For Python tests on FreeBSD at least; only seven python tests fail:
045 147 149 169 194 199 211

045 and 149 appear to be misconfigurations,
147 and 194 are the AF_UNIX path too long error,
169 and 199 are bitmap migration bugs, and
211 is a bug that shows up on Linux platforms, too.

This is at least good evidence that these tests are not Linux-only. If
they aren't suitable for other platforms, they should be disabled on a
per-platform basis as appropriate.

Therefore, let's switch these on and deal with the failures.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20200121095205.26323-2-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia 7cdca2e233 qcow2: Use a GString in report_unsupported_feature()
This is a bit more efficient than having to allocate and free memory
for each item.

The default size (60) is enough for all the existing incompatible
features or the "Unknown incompatible feature" message.

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20200115135626.19442-1-berto@igalia.com
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Pan Nengyuan cb8956144c block: fix memleaks in bdrv_refresh_filename
If we call the qmp 'query-block' while qemu is working on
'block-commit', it will cause memleaks, the memory leak stack is as
follow:

Indirect leak of 12360 byte(s) in 3 object(s) allocated from:
    #0 0x7f80f0b6d970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970)
    #1 0x7f80ee86049d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d)
    #2 0x55ea95b5bb67 in qdict_new /mnt/sdb/qemu-4.2.0-rc0/qobject/qdict.c:29
    #3 0x55ea956cd043 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6427
    #4 0x55ea956cc950 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6399
    #5 0x55ea956cc950 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6399
    #6 0x55ea956cc950 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6399
    #7 0x55ea958818ea in bdrv_block_device_info /mnt/sdb/qemu-4.2.0-rc0/block/qapi.c:56
    #8 0x55ea958879de in bdrv_query_info /mnt/sdb/qemu-4.2.0-rc0/block/qapi.c:392
    #9 0x55ea9588b58f in qmp_query_block /mnt/sdb/qemu-4.2.0-rc0/block/qapi.c:578
    #10 0x55ea95567392 in qmp_marshal_query_block qapi/qapi-commands-block-core.c:95

Indirect leak of 4120 byte(s) in 1 object(s) allocated from:
    #0 0x7f80f0b6d970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970)
    #1 0x7f80ee86049d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d)
    #2 0x55ea95b5bb67 in qdict_new /mnt/sdb/qemu-4.2.0-rc0/qobject/qdict.c:29
    #3 0x55ea956cd043 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6427
    #4 0x55ea956cc950 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6399
    #5 0x55ea956cc950 in bdrv_refresh_filename /mnt/sdb/qemu-4.2.0-rc0/block.c:6399
    #6 0x55ea9569f301 in bdrv_backing_attach /mnt/sdb/qemu-4.2.0-rc0/block.c:1064
    #7 0x55ea956a99dd in bdrv_replace_child_noperm /mnt/sdb/qemu-4.2.0-rc0/block.c:2283
    #8 0x55ea956b9b53 in bdrv_replace_node /mnt/sdb/qemu-4.2.0-rc0/block.c:4196
    #9 0x55ea956b9e49 in bdrv_append /mnt/sdb/qemu-4.2.0-rc0/block.c:4236
    #10 0x55ea958c3472 in commit_start /mnt/sdb/qemu-4.2.0-rc0/block/commit.c:306
    #11 0x55ea94b68ab0 in qmp_block_commit /mnt/sdb/qemu-4.2.0-rc0/blockdev.c:3459
    #12 0x55ea9556a7a7 in qmp_marshal_block_commit qapi/qapi-commands-block-core.c:407

Fixes: bb808d5f5c
Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Message-id: 20200116085600.24056-1-pannengyuan@huawei.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Alberto Garcia e2a7423a11 block: Use a GString in bdrv_perm_names()
This is a bit more efficient than having to allocate and free memory
for each new permission.

The default size (30) is enough for "consistent read, write, resize".

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 20200110171518.22168-1-berto@igalia.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00