Commit Graph

4182 Commits (8961be33e8ca7e809c603223803ea66ef7ea5be7)

Author SHA1 Message Date
Alex Bennée be5e1f527a docker: par down QEMU_CONFIGURE_OPTS in debian-tricore-cross
This image isn't going to build anything significant as it is just
intended for building test cases. In case it does end up getting
inadvertently included in a build lets aim for the minimal possible
product.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée e043d73212 docker: base debian-tricore on qemu:debian9
We need both git and a working compiler to build the tools. Although
the qemu:debian9 image also has a bunch of extra dependencies it would
be fairly unusual for a user not to already have this layer available
for one of our many other docker images so lets not complicate things.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée 1818d50179 tests/.gitignore: don't ignore docker tests
The .gitignore was being a little over enthusiastic hiding files.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Peter Maydell 5c29b203df Block layer patches:
- vvfat: Disable debug message by default
 - qemu-iotests fixes
 - Fix typos in comments
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbVgX4AAoJEH8JsnLIjy/WAJkQAIO61ydCkVEGch9GBThPHKlH
 wqmE2jVfiM08aFKrfNXvIyJTvYskZ78PCFSLwKp2SSJsggkmSc2RyQUkZJEw1ECx
 wS3rW6OynwwfcDBJkUiM2J21r0eByuxZpn1hQUv7sX1ZaxXmmwB4fGS4F+fNi2j1
 kxQIvWbsxswB58v0jcpZiidJGgrBlxxR3TyBN2d2bnfLEJSRUoCHUqq7d1KIgQA2
 YfVKEt0La6w9IuXWrfzXps3sZlMWoJn8AoEjHzTUEnxWZE0TgsQPfFEbkxeCnaYv
 /JqeVw6ErJm46swgxWtPEkK3vjBBAum5wZ6USPi/KDhULVRh4djVrAzhUN1Y87YR
 p2U1GEnpCPRL6bgq22eHD6ub70wn02FpgCuHIsK044xMjOvGhpsCnPUMlkNHBAZU
 BCroEJ3OWOxZaedb4vPaVicOAnoBuxJWzuUBpbrX/a6w7r8vP/Xy540NHcL3RetO
 +N1YgtTuc8J7Kleo35MvQAZtEfwcOt0JOeGED0facpqcQunVY7u7lAbbH7gqvhbk
 biuw7+u7lvvy3jYwVmBP+KYU686voqE6o9zE42dtUPbtdqDPdQ6Cf6/Iwipu/RKI
 UZVVNP0gzXhSG5pMRKbbOFgHa1wU+tF7iedckGwsuC+97Scq5kekT+nKlwn4RDI1
 o8Vd8Uql8lz88rbKdirX
 =YWeJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- vvfat: Disable debug message by default
- qemu-iotests fixes
- Fix typos in comments

# gpg: Signature made Mon 23 Jul 2018 17:44:40 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  block/vvfat: Disable debug message by default
  iotests: Disallow compat=0.10 in 223
  iotest: Fix filtering order in 226
  iotests: remove LUKS support from test 226
  qemu-img: avoid overflow of min_sparse parameter
  block: Fix typos in comments (found by codespell)
  qemu-iotests: Use host_device instead of file in 149

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-23 18:37:39 +01:00
Max Reitz 092b9c408f iotests: Disallow compat=0.10 in 223
223 tests persistent dirty bitmaps which are not supported in
compat=0.10, so that option is unsupported for this test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Tested-by: John Snow <jsnow@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-23 16:50:43 +02:00
Max Reitz d6e4ca9021 iotest: Fix filtering order in 226
The test directory should be filtered before the image format, otherwise
the test will fail if the image format is part of the test directory,
like so:

[...]
-can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory
+can't open: Could not open '/tmp/test-IMGFMT/t.IMGFMT': Is a directory
[...]

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-23 16:50:43 +02:00
John Snow e05eb9f29b iotests: remove LUKS support from test 226
This test doesn't actually care about the format anyway, it just
supports "all formats" as a convenience. LUKS however does not use a
simple image filename which confuses this iotest.

We can simply skip the test for formats that use IMGOPTSSYNTAX for
their filenames without missing much coverage.

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-23 16:50:43 +02:00
Kevin Wolf 4e04f3d91a qemu-iotests: Use host_device instead of file in 149
The test case uses block devices with driver=file, which causes the test
to fail after commit 230ff73904 added a deprecation warning for this.
Fix the test case to use driver=host_device and update the reference
output accordingly.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-23 16:50:43 +02:00
Peter Xu 62aa1d887f monitor: Fix unsafe sharing of @cur_mon among threads
@cur_mon is null unless the main thread is running monitor code, either
HMP code within monitor_read(), or QMP code within
monitor_qmp_dispatch().

Use of @cur_mon outside the main thread is therefore unsafe.

Most of its uses are in monitor command handlers.  These run in the main
thread.

However, there are also uses hiding elsewhere, such as in
error_vprintf(), and thus error_report(), making these functions unsafe
outside the main thread.  No such unsafe uses are known at this time.
Regardless, this is an unnecessary trap.  It's an ancient trap, though.

More recently, commit cf869d5317 "qmp: support out-of-band (oob)
execution" spiced things up: the monitor I/O thread assigns to @cur_mon
when executing commands out-of-band.  Having two threads save, set and
restore @cur_mon without synchronization is definitely unsafe.  We can
end up with @cur_mon null while the main thread runs monitor code, or
non-null while it runs non-monitor code.

We could fix this by making the I/O thread not mess with @cur_mon, but
that would leave the trap armed and ready.

Instead, make @cur_mon thread-local.  It's now reliably null unless the
thread is running monitor code.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[peterx: update subject and commit message written by Markus]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180720033451.32710-1-peterx@redhat.com>
2018-07-23 14:00:03 +02:00
Peter Maydell 59b5552f02 Bug fixes.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJbTgXfAAoJEL/70l94x66DTbYH/3NutBAkNZKX7EImj/d0I1O8
 nERMVH1R70KBcugdsjhaBfTRoATDXdrBng4MBqloIK9dEMT3g6D4TFZJLU+WAjOc
 8sItx0BrUR7Sl8SnAvWNFoqVtvVancFiLnu11DsFGM0l8mJHRlZSkQZ0Fd0FL2W/
 OPnW7t6F7B2bc1VlPfSs093FVCoD3S+lJmbj64dwNrn8+fOX918V6gSaYQe92aIY
 pSbJjkRDx2iULmzMY8QH4OQiHgnd/Pijj+D628DMrUc0iW1Rsw5V2Yq7SMY6zoa8
 MoI/YDwX6eRMU2mq74BrKlULZrpmQn+6ZCdZTvXzLwc2zpKD4puO4FuMBOA7yx4=
 =GcxI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

Bug fixes.

# gpg: Signature made Tue 17 Jul 2018 16:06:07 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  Document command line options with single dash
  opts: remove redundant check for NULL parameter
  i386: only parse the initrd_filename once for multiboot modules
  i386: fix regression parsing multiboot initrd modules
  virtio-scsi: fix hotplug ->reset() vs event race
  qdev: add HotplugHandler->post_plug() callback
  hw/char/serial: retry write if EAGAIN
  PC Chipset: Improve serial divisor calculation
  vhost-user-test: added proper TestServer *dest initialization in test_migrate()
  hyperv: ensure VP index equal to QEMU cpu_index
  hyperv: rename vcpu_id to vp_index
  accel: Fix typo and grammar in comment
  dump: add kernel_gs_base to QEMU CPU state

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-17 17:06:32 +01:00
Emanuele Giuseppe Esposito 9ee8a692f1 vhost-user-test: added proper TestServer *dest initialization in test_migrate()
server->bus in _test_server_free() could be NULL, since TestServer
*dest in test_migrate() was not properly initialized like TestServer *s.
Added init_virtio_dev(dest) and uninit_virtio_dev(dest), so the fields
are properly set and when test_server_free(dest); is called, they can
be correctly freed.

The reason for that is init_virtio_dev() calls qpci_init_pc(), that
creates a QPCIBusPC * (returned as QPCIBus *), while test_server_free()
calls qpci_free_pc(), that frees the QPCIBus *. Not calling
init_virtio_dev() would leave the QPCIBus * of TestServer unset.

Problem came out once I modified  pci-pc.c and pci-pc.h, modifying
QPCIBusPC by adding another field before QPCIBus bus. Re-running the
tests showed vhost-user-test failing.

Signed-off-by: Emanuele Giuseppe Esposito <e.emanuelegiuseppe@gmail.com>
Message-Id: <1530022733-29581-1-git-send-email-esposem@usi.ch>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-07-16 16:58:16 +02:00
Markus Armbruster 361ac948a5 cli qmp: Mark --preconfig, exit-preconfig experimental
Committing to the current --preconfig / exit-preconfig interface
before it has seen any use is premature.  Mark both as experimental,
the former in documentation, the latter by renaming it to
x-exit-preconfig.

See the previous commit for more detailed rationale.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180705091402.26244-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
[Straightforward conflict with commit 514337c142 resolved]
2018-07-16 15:38:19 +02:00
Peter Lieven 8dcd3c9b91 qemu-img: align result of is_allocated_sectors
We currently don't enforce that the sparse segments we detect during convert are
aligned. This leads to unnecessary and costly read-modify-write cycles either
internally in Qemu or in the background on the storage device as nearly all
modern filesystems or hardware have a 4k alignment internally.

This patch modifies is_allocated_sectors so that its *pnum result will always
end at an alignment boundary. This way all requests will end at an alignment
boundary. The start of all requests will also be aligned as long as the results
of get_block_status do not lead to an unaligned offset.

The number of RMW cycles when converting an example image [1] to a raw device that
has 4k sector size is about 4600 4k read requests to perform a total of about 15000
write requests. With this path the additional 4600 read requests are eliminated while
the number of total write requests stays constant.

[1] https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-amd64-disk1.vmdk

Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-12 18:24:08 +02:00
John Snow 2d4cb49dda iotests: add test 226 for file driver types
Test that we're rejecting what we ought to for file,
host_driver and host_cdrom drivers. Test that we're
seeing the deprecated message for block and chardevs
on the file driver.

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-12 18:24:08 +02:00
Fam Zheng 2f9d4083f7 iotests: nbd: Stop qemu-nbd before remaking image
197 is one example where _make_test_img is used twice without stopping
the NBD server in between. An error will occur like this:

    @@ -26,9 +26,13 @@

     === Partial final cluster ===

    +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "resize" lock
    +Is another process using the image?
     Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024
    +Failed to find an available port: Address already in use
     read 1024/1024 bytes at offset 0

Patch _make_test_img to stop the old qemu-nbd before starting a new one,
which fixes this problem, and similarly 215.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-12 18:24:08 +02:00
Fam Zheng 0e1a582750 iotests: 153: Fix dead code
This step was left behind my mistake. As suggested by the echoed text,
the intention was to test two devices with the same image, with
different options. The behavior should be the same as two QEMU
processes. Complete it.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-12 18:24:08 +02:00
Peter Maydell 7851f1a706 Block layer patches:
- Copy offloading fixes for when the copy increases the image size
 - Temporary revert of the removal of deprecated -drive options
 - Fix request serialisation in the image fleecing scenario
 - Fix copy-on-read crash with unaligned image size
 - Fix another drain crash
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbRNLQAAoJEH8JsnLIjy/WOaQQALlZk01JohETuwGG6HGl0LdI
 jEEm+N0J+BlGOVjoGU67OKGidUCl5WvBsQTlyYkmlaToGuk/njWxCa/GA6+iNRnt
 MDq7Ovr8uZI3D+0Fuc6xg/6NBiLkukgh0Q9gMWkzn3jaNWzO2WcTr8WXwepvP6sj
 YtPhEQOXTT3sXf/MFY8ig7qRrZ6f7LFOoKu7LMnrD+QWDo8TY3QLZaxP9OUFHH7S
 A6J0LIfuRZlq79a7SgrRkCR2ddtgYyBQ+zD7PD5kf1vLW4+dOhDOutQEsZCMCPgR
 ft99kNhrZcJGN6n2r8/oVcvRkw5c4I1JPgakm/GoW/NllfPMebuPospKaS4wiJnB
 zI4YOtmco4Mfxkw/wK+Ep/bPCpxEF43uDcpPiEjsNADrdLq0eKnPn5ctwSyWlGvn
 ayQWxDoKoYckn/ccjtLxJ2xPws8433cTXrBdIKnJadWxi3iRNzlIKHRuEfXf9zQt
 G+Nq7ruysT9TPf9ifuCHcZnTsi3SLYLsjCj7pAgBkazBYE2cCI3eKN8kxsDJi7qv
 cWzFCpwE28pHRJ6FwtdzBVkNcfTlC/XopR1M66OzYZlLqR/4hbNhyHL3hBV+yfrM
 fC7mPi81ttI6e+JAgC6K8t3Ey242MjSzUYa7pJUNws7RpqUhfhr6EXXbBceJKsVW
 F8qKZoiIEK7wDacUiEiE
 =FXOo
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Copy offloading fixes for when the copy increases the image size
- Temporary revert of the removal of deprecated -drive options
- Fix request serialisation in the image fleecing scenario
- Fix copy-on-read crash with unaligned image size
- Fix another drain crash

# gpg: Signature made Tue 10 Jul 2018 16:37:52 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (24 commits)
  block: Use common write req handling in truncate
  block: Fix bdrv_co_truncate overlap check
  block: Use common req handling in copy offloading
  block: Use common req handling for discard
  block: Fix handling of image enlarging write
  block: Extract common write req handling
  block: Use uint64_t for BdrvTrackedRequest byte fields
  block: Use BdrvChild to discard
  block: Add copy offloading trace points
  block: Prefix file driver trace points with "file_"
  Revert "block: Remove deprecated -drive geometry options"
  Revert "block: Remove deprecated -drive option addr"
  Revert "block: Remove deprecated -drive option serial"
  Revert "block: Remove dead deprecation warning code"
  block/blklogwrites: Make sure the log sector size is not too small
  qapi/block-core.json: Add missing documentation for blklogwrites log-append option
  block/backup: fix fleecing scheme: use serialized writes
  block: add BDRV_REQ_SERIALISING flag
  block: split flags in copy_range
  block/io: fix copy_range
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-10 17:28:29 +01:00
Peter Xu 3e81f73c7a tests: hide stderr for postcopy recovery test
We dumped something when network failure happens.  We should avoid those
messages to be dumped when running the tests:

  $ ./tests/migration-test -p /x86_64/migration/postcopy/recovery
  /x86_64/migration/postcopy/recovery: qemu-system-x86_64: check_section_footer: Read section footer failed: -5
  qemu-system-x86_64: Detected IO failure for postcopy. Migration paused.
  qemu-system-x86_64: Detected IO failure for postcopy. Migration paused.
  OK

After the patch:

  $ ./tests/migration-test -p /x86_64/migration/postcopy/recovery
  /x86_64/migration/postcopy/recovery: OK

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-11-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-07-10 15:23:22 +01:00
Peter Xu d5f496407d tests: add postcopy recovery test
Test the postcopy recovery procedure by emulating a network failure
using migrate-pause command.

Tested-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-10-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-07-10 15:20:08 +01:00
Peter Xu 2f6d313836 tests: introduce wait_for_migration_status()
It's generalized from wait_for_migration_complete() to allow us to wait
for any migration status besides failure.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-9-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-07-10 15:18:03 +01:00
Peter Xu 2f7074c6fd tests: introduce migrate_query*() helpers
Introduce helpers to query migration states and use it.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-8-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-07-10 15:18:03 +01:00
Peter Xu 7e1d74271c tests: allow migrate() to take extra flags
For example, we can pass in '"resume": true' to resume a migration.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-7-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-07-10 15:18:03 +01:00
Peter Xu d131662a1a tests: introduce migrate_postcopy_* helpers
Separate the old postcopy UNIX socket test into three steps, provide a
helper for each step.  With these helpers, we can do more compliated
tests like postcopy recovery, while keep the codes shared.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180710091902.28780-6-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Fix up merge with 2e295789 / Skip tests for ppc tcg
2018-07-10 15:17:40 +01:00
Cornelia Huck 6703db131f Revert "block: Remove deprecated -drive geometry options"
This reverts commit a7aff6dd10.

Hold off removing this for one more QEMU release (current libvirt
release still uses it.)

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10 14:36:12 +02:00
Cornelia Huck 44e8b4689c Revert "block: Remove deprecated -drive option serial"
This reverts commit b008326744.

Hold off removing this for one more QEMU release (current libvirt
release still uses it.)

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10 14:36:11 +02:00
Fam Zheng e79c4cd190 iotests: 222: Don't run with luks
Luks needs special parameters to operate the image. Since this test is
focusing on image fleecing, skip skip that format.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10 11:55:11 +02:00
Kevin Wolf b0ddcbbb36 block: Fix copy-on-read crash with partial final cluster
If the virtual disk size isn't aligned to full clusters,
bdrv_co_do_copy_on_readv() may get pnum == 0 before having the full
cluster completed, which will let it run into an assertion failure:

qemu-io: block/io.c:1203: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed.

Check for EOF, assert that we read at least as much as the read request
originally wanted to have (which is true at EOF because otherwise
bdrv_check_byte_request() would already have returned an error) and
return success early even though we couldn't copy the full cluster.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10 10:36:15 +02:00
Kevin Wolf b994c5bc51 test-bdrv-drain: Test bdrv_append() to drained node
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10 10:36:15 +02:00
Max Reitz 85456e0d16 iotests: Add VMDK backing file correlation test
This new test verifies that VMDK backing file reads fail when the
backing file has a non-matching CID.  This includes non-VMDK backing
files.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180702210721.4847-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-07-09 19:43:24 +02:00
Peter Maydell 086ede32af ptimer: Add TRIGGER_ONLY_ON_DECREMENT policy option
The CMSDK timer behaviour is that an interrupt is triggered when the
counter counts down from 1 to 0; however one is not triggered if the
counter is manually set to 0 by a guest write to the counter register.
Currently ptimer can't handle this; add a policy option to allow
a ptimer user to request this behaviour.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Message-id: 20180703171044.9503-2-peter.maydell@linaro.org
2018-07-09 14:51:34 +01:00
Dr. David Alan Gilbert 5fd4a9c973 tests/migration: Skip tests for ppc tcg
PPC tcg seems to be failing migration tests quite regularly;
we believe this is TCG bugs in dirty bit updating; it's
not clear why PPC fails more but lets skip for the moment.

$ ./tests/migration-test
/ppc64/migration/deprecated: OK
/ppc64/migration/bad_dest: Skipping test: kvm_hv not available OK
/ppc64/migration/postcopy/unix: Skipping test: kvm_hv not available OK
/ppc64/migration/precopy/unix: Skipping test: kvm_hv not available OK

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Message-id: 20180706143105.93472-1-dgilbert@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-06 16:18:31 +01:00
Peter Maydell cee35138b5 Code coverage and other build tweaks
- revert 208ecb3e (and drop filter for mingw, tweak for check-tcg)
   - some travis speed-ups
   - modernise code coverage support
   - docker image cleanups
   - clean-up binfmt_misc docker infrastructure
   - add debian-powerpc-user-cross image for ppc32 build
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAls+QIIACgkQ+9DbCVqe
 KkT3nAf+JQIohMR4v+YPbuFGshzy89P2KrgTxWfA1EjVkfxCB2ida9vl5SzL4G3l
 jvINZhz6U2eUn/4Csx5Pempn3GUO/k7FKN0oG4s7Xu+KQQpCCRBIxdE/SchRKk+O
 Y9fYPTc8pvYEplAUzwwICFJtOBY0eFIYzA8oTZ+/lQ5YAsgC1ud31LdeN3/lh4J6
 BPZ22WEydlC6r+ihh5HkL6+9HQN3+uPZuK3VqP1UrrZRvoL/LKcGip4WCh2iUxqE
 RdmNBwo6DPPsBUooBbkPZojGGAo3RTGV6kiekAbMcidd3xkNt+4WuBnChzn8xMyP
 j3W0pokX9gJ3gI4kxrzqk4ObQE/A5g==
 =N4gA
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3' into staging

Code coverage and other build tweaks

  - revert 208ecb3e (and drop filter for mingw, tweak for check-tcg)
  - some travis speed-ups
  - modernise code coverage support
  - docker image cleanups
  - clean-up binfmt_misc docker infrastructure
  - add debian-powerpc-user-cross image for ppc32 build

# gpg: Signature made Thu 05 Jul 2018 17:00:02 BST
# gpg:                using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3:
  docker: add linux-user powered cross builder for QEMU
  docker: add special rule for deboostrapped images
  docker: add special handling for FROM:debian-%-user targets
  docker: debian-bootstrap.pre allow customising of variant/url
  docker: drop QEMU build-dep from bootstrap
  docker: Do not run tests in 'intermediate' images
  docker: Clean the MXE base image
  docker: ubuntu: Use SDL2
  docker: ubuntu: Update the package list before installing new ones
  linux-user: add gcov support to preexit_cleanup
  linux-user: introduce preexit_cleanup
  build-system: add coverage-report target
  build-system: add clean-coverage target
  travis: add gcovr summary for GCOV build
  docker: add gcovr to travis image
  .gitignore: add .gcov files
  build-system: remove per-test GCOV reporting
  travis: test out-of-tree builds
  travis: do not waste time cloning unused submodules
  Revert "Makefile: Rename TARGET_DIRS to TARGET_LIST"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-05 18:24:28 +01:00
Alex Bennée 19c9a18f45 docker: add linux-user powered cross builder for QEMU
We can't use cross compilers in the current Debian stable and Debian
sid is sketchy as hell. So for powerpc fall back to dog-fooding our
own linux-user to do the build.

As we can only build the base image with a suitably configured
source tree we fall back to checking for its existence when we can't
build it from scratch. However this does mean you don't have to keep
a static powerpc-linux-user in your active configuration just to
update the cross build image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 16:57:39 +01:00
Alex Bennée dc338fdb07 docker: add special rule for deboostrapped images
We might as well have a custom rule for this. For one thing the
dependencies are different. As the primary dependency for
docker-image-% could never be docker-image-debian-bootstrap we can
drop that test in the main rule as well.

Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults
now. We also error out if the EXECUTABLE file isn't there. We should
really do this with a dependency on any source rules but currently
subdir-FOO-linux-user isn't enough on a clean build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée 547cb45ea3 docker: add special handling for FROM:debian-%-user targets
These will have been build with debootstrap so we need to check
against the debian-bootstrap dockerfile. This does mean sticking to
debian-FOO-user as the naming conventions for boot-strapped images.
The actual cross image is built on top.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée 300cf467fd docker: debian-bootstrap.pre allow customising of variant/url
We default to the buildd variant as most of our images are for
building. However lets give the user the ability to specify "minbase"
if they want to create a simple base image for experimentation.

Allowing the tweaking of DEB_URL means we can also bootstrap other
Debian based OS's. For example:

  make docker-binfmt-image-debian-ubuntu-bionic-arm64 \
       DEB_ARCH=arm64 DEB_TYPE=bionic \
       DEB_VARIANT=minbase DEB_URL=http://ports.ubuntu.com/ \
       EXECUTABLE=./aarch64-linux-user/qemu-aarch64

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée e6bfdeca8b docker: drop QEMU build-dep from bootstrap
This is best done with any child images that actually need it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé 272e551b43 docker: Do not run tests in 'intermediate' images
We can still build the DOCKER_INTERMEDIATE_IMAGES images,
but they won't appear in 'make test*@$IMAGE'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé 48feb682db docker: Clean the MXE base image
Using the duplicated same package is confusing.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé daf999f77a docker: ubuntu: Use SDL2
Do not test the deprecated API versions (see cabd358407).
Debian MXE MinGW cross images are already using SDL2.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé beac6a98f6 docker: ubuntu: Update the package list before installing new ones
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.

See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get

This fixes:

  $ make docker-image-ubuntu V=1
  ./tests/docker/docker.py build qemu:ubuntu tests/docker/dockerfiles/ubuntu.docker   --add-current-user
  Sending build context to Docker daemon  3.072kB
  [...]
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa-dev_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  The command '/bin/sh -c apt-get -y install $PACKAGES' returned a non-zero code: 100
  tests/docker/Makefile.include:40: recipe for target 'docker-image-ubuntu' failed
  make: *** [docker-image-ubuntu] Error 1

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Alex Bennée 95f7aabf49 docker: add gcovr to travis image
Useful for debugging if nothing else as the gcovr on the Travis images
are a little old.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-07-05 15:59:41 +01:00
Alex Bennée 31d2dda3c4 build-system: remove per-test GCOV reporting
I'm not entirely sure who's using this information and certainly in a
CI environment it just washes over as additional noise. Later patches
will provide new reporting options so a user who wants to analyse
individual tests will be able to use that to get the information.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-07-05 15:59:41 +01:00
Alex Bennée 2b1f35b9a8 Revert "Makefile: Rename TARGET_DIRS to TARGET_LIST"
This reverts commit 208ecb3e1a. This was
causing problems by making DEF_TARGET_LIST pointless and having to
jump through hoops to build on mingw with a dully enabled config.
This includes a change to fix the per-guest TCG test probe which was
added after 208ecb3 and used TARGET_LIST.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
2018-07-05 15:59:41 +01:00
Peter Maydell 1daf14ec9e Block layer patches:
- qcow2: Use worker threads for compression to improve performance of
   'qemu-img convert -W' and compressed backup jobs
 - blklogwrites: New filter driver to log write requests to an image in
   the dm-log-writes format
 - file-posix: Fix image locking during image creation
 - crypto: Fix memory leak in error path
 - Error out instead of silently truncating node names
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbPfHhAAoJEH8JsnLIjy/Wxp8P/jeEJ/jJ0N2doix5KJSvtkdP
 dYx4cJxHg9tMSnmBfA49qc1ilgbzZqjbF/wH3Ko3zPsmFP0xsxx+9zE4c3iL7HA6
 6E4/r3szznO8DIxdgzAIeHjZc7YPpsY3alrthT55eR/FnDyc8zofi/iUMgvTKPYA
 8TnWqgi2fdaVwNy+lIRJFdhzzQNtPxOEO+0DFHaOZvQ9vlc5DGPC+Hj3Qc11GK8M
 u7orkeYqQPknxy0hJKPxtWHvzCQUJMcSSs6PbuslnzOerYiQLmx+RVIwMhXfVZjV
 lXe2SppAszBujtcIENhZlj1cECs7MrWTmFDcWvBA+Mh/JhFEWwykmlQHYrjCqdvw
 QyWhVLgP/6jQDaBls6LADSZDxX7i0sV27DzVGN4gYUX/KcyVPEDROm90MyE8nxN0
 Y5hhujTkzu94Zvd/OlKZRs4tPxmbRrd2SWy0id8Kj5/gO/+UWECOZCrPrlB5uzec
 bsxHoeXLVe2/56JkCIiOw3hLILMY6gPLJgeaQjz6hu1oZJqYuoof8grVFjUCSH0C
 BBlz8O4upHdPIKNRl4rmdgwasg5YIiJ7SFl4h7KMCD0elkKmo3SyTKosagvqrpVN
 JvW1YuosCedcfYIKNPnZdEGWDKuTaZtcnZnM9IRY6hr8ySMw1LloFsgXsJ3tqTwe
 wb1Wjm4Qlz5BaoB7o/Os
 =z5AC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Use worker threads for compression to improve performance of
  'qemu-img convert -W' and compressed backup jobs
- blklogwrites: New filter driver to log write requests to an image in
  the dm-log-writes format
- file-posix: Fix image locking during image creation
- crypto: Fix memory leak in error path
- Error out instead of silently truncating node names

# gpg: Signature made Thu 05 Jul 2018 11:24:33 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  file-posix: Unlock FD after creation
  file-posix: Fix creation locking
  block/blklogwrites: Add an option for the update interval of the log superblock
  block/blklogwrites: Add an option for appending to an old log
  block/blklogwrites: Change log_sector_size from int64_t to uint64_t
  block/crypto: Fix memory leak in create error path
  block: Don't silently truncate node names
  block: Add blklogwrites
  block: Move two block permission constants to the relevant enum
  qcow2: add compress threads
  qcow2: refactor data compression
  qemu-img: allow compressed not-in-order writes

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-05 15:53:04 +01:00
Peter Maydell 4fd1cbaf14 Monitor patches for 2018-07-03
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbO+iNAAoJEDhwtADrkYZTX9gQAJFT0DPtUYQsqbYAhrUIJo/a
 lT5OAnMnMou6Qu0XHjSYrqdZu7Ega00rvyeYgtB54lmLcOVrjAxF0SwcPcoOuz7K
 O9o00ZmwuFPUugJuYBLDRqJryejbZTCGSOg3cp+YlV6naW6Omck97iu7G/MZWLvB
 FJvIIUaPIAGklRhSodhC+yePp1PNbfGVcpDjAxn4e1UQ/2M6gwSFx9iPbNl2WotN
 CXjRHmkB4ZcxwcDtzzFLwkXTDkryZE27sOgFjDG5xFWO/oCS+Px7a0vME8YM+j2r
 eNa93g16ZohNqHb9t6GarlqrqNqbQF2t6JAPpmlIX5qb3Dtb87MQ/ExdN+FJ4btD
 IfBkTnXbs7qkZpConRlrhdd2J01ChnCdcmddUeenlq+8mIMjQJXH2RCwRVKyoZYs
 4Z74XWy2A+xMJgjqY9WO7E4jy4QVDSckKIm6je/O8fXzXWxthi4H1/2EN5nAv1hY
 zNxP/SW0vpqiR/cVQAr2N6VGz2/m9cWQIf0NwFc765gypPBbCL5HLrb+/xC86Bi3
 ylg9YlTw84oXtpSauTYu3oE989iDndu/s9BEUe5fW5g+8azs+h5SHVTwq0Y8dzSZ
 n2FsGxr76BlOIu5i1oIYNUUhAoU3txddueT4SNHE5Iaz1x6fAKnMP2hxoo5DWDzn
 1Rzgq4ATB8H+V6Og942n
 =/2SW
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-monitor-2018-07-03-v2' into staging

Monitor patches for 2018-07-03

# gpg: Signature made Tue 03 Jul 2018 22:20:13 BST
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-monitor-2018-07-03-v2: (32 commits)
  qapi: Polish command flags documentation in qapi-code-gen.txt
  monitor: Improve some comments
  qmp: Clean up capability negotiation after commit 02130314d8
  qobject: Let qobject_from_jsonf() fail instead of abort
  qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()
  qmp: Add some comments around null responses
  qmp: Simplify monitor_qmp_respond()
  qmp: Replace get_qmp_greeting() by qmp_greeting()
  qmp: Replace monitor_json_emitter{,raw}() by qmp_{queue,send}_response()
  qmp: Use QDict * instead of QObject * for response objects
  qmp: De-duplicate error response building
  qobject: New qdict_from_jsonf_nofail()
  monitor: Peel off @mon_global wrapper
  monitor: Rename use_io_thr to use_io_thread
  qmp: Don't let JSON errors jump the queue
  qmp: Don't let malformed in-band commands jump the queue
  tests/qmp-test: Demonstrate QMP errors jumping the queue
  qmp: Simplify code around monitor_qmp_dispatch_one()
  qmp: Always free QMPRequest with qmp_request_free()
  qmp: Revert change to handle_qmp_command tracepoint
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-05 11:25:14 +01:00
Peter Maydell 5dafaf4fbc QAPI patches for 2018-07-03
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbO+InAAoJEDhwtADrkYZTtsQQAI2IEFWwsIWtjh+JO2xWHjTC
 sXrzD2m8g50xledX1mKgQdA9NujvrSreQBFEgXvOEz65K8hl1HIqNp9C4NeOPUGn
 vVxDGRrMz1rGRrBXXM9mf2v3ZSMw0mVAmpBzQ2lO/fPsGEAThJ4451c0mHiMhBXi
 Ez2v3Qod597ITWCG73DccCN7ehl+3HCpqAyvBjVjtolmnt0h4oefqb8OV1l59kl3
 dywlnEoMcLq80uznAkkCnQ89yZR6ygrZIYfyMynpkTo4quf/ZB8eHF8Au4Jyynby
 1cAPAs0tfjmfvSpSEGvWyhmuu8jKm9pGnby7Me18HV/dkyNWzQWWMpzWpYHxiu41
 fa0SpBpw4g112SaeNOOt0g+SDpcdtvTheWmyPbzSL1yTDJ2NTppbl2zpNo9BHY3X
 jFSXpNi74RTB4KDXx+t/QXJepo3f35dTOOoOvodhT4EHzTIkPQoalKbKZij6zDHo
 CoBRrJQ8WQIxaYuiYHCDdRxTBbq44XbW30rWDnjmUBcLW6uMuUOiZxUh+n2GQjP7
 dEZel23zslqPdFexnTX32miIfkiHABKWIr7EzZ4NC8P/0U8McKD/lxGP8IHrs9Lo
 D8UYgfOGse2w5MWXCxSwq+VY5aFPasYrpVpi31gnCzLtJ9Q8qBMoqIGdyFKHzxXj
 qTvkP1iSHnXjSsq0KryZ
 =uZ84
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2018-07-03' into staging

QAPI patches for 2018-07-03

# gpg: Signature made Tue 03 Jul 2018 21:52:55 BST
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2018-07-03:
  qapi: add conditions to SPICE type/commands/events on the schema
  qapi: add conditions to VNC type/commands/events on the schema
  qapi: add 'If:' section to generated documentation
  qapi-types: add #if conditions to types & visitors
  qapi/events: add #if conditions to events
  qapi/commands: add #if conditions to commands
  qapi-introspect: add preprocessor conditions to generated QLit
  qapi-introspect: modify to_qlit() to append ',' on level > 0
  qapi: add #if/#endif helpers
  qapi: mcgen() shouldn't indent # lines
  qapi: add 'ifcond' to visitor methods
  qapi: leave the ifcond attribute undefined until check()
  qapi: pass 'if' condition into QAPISchemaEntity objects
  qapi: add 'if' to top-level expressions

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-05 10:31:36 +01:00
Kevin Wolf 824808dd77 block: Don't silently truncate node names
If the user passes a too long node name string, we silently truncate it
to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart
from surprising the user when the node has a different name than
requested, this also bypasses the check for duplicate names, so that the
same name can be assigned to multiple nodes.

Fix this by just making too long node names an error.

Reported-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-05 10:29:19 +02:00
Peter Maydell 8beb8cc64d qemu-ga patch queue for soft-freeze
* add systemd suspend support
 * add used/total space stats for guest-get-fsinfo
 * fixes for guest-get-fsinfo over PCI bridges
 * MSI installer and schema doc fixes
 * guard against unbounded allocations in guest-file-read
 * add some additional qga test cases
 -----BEGIN PGP SIGNATURE-----
 
 iQE3BAABCAAhBQJbO+BsGhxtZHJvdGhAbGludXgudm5ldC5pYm0uY29tAAoJEDNT
 yc7xCLWEUaoH/0gaXLAV2XoijUeVbce+VKXgAVMCDXnew0mhX3KuOXy3j1xePcdz
 asNb/pOrv0YRo0xAP7VS5Guxk7oIPwq052Fcb6ncMCw7KZNoh4KaBrs0Zbwz1DQU
 XPA39bQIl+gbG7jQJlNyM9FYtgohZucy1n91dUF0TzdYreoR0vTq0zwUb4XvrZ0m
 a0r01FuBXGb3YttSEFzYGTEhJcEm22OX/FuwoyZRoZ6OKKeLl+IvdwNPq+v3oJvu
 cHqX3nwf7NRVac9/kqi+th+sGYKppuq4CMoSkeZLJpIpc6k+ISUBhiiiiypN44xQ
 V5/Z05+YYZRmQxHUSIKqJl8AgzBRRjoGj2Y=
 =EbNE
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mdroth/tags/qga-pull-2018-07-03-tag' into staging

qemu-ga patch queue for soft-freeze

* add systemd suspend support
* add used/total space stats for guest-get-fsinfo
* fixes for guest-get-fsinfo over PCI bridges
* MSI installer and schema doc fixes
* guard against unbounded allocations in guest-file-read
* add some additional qga test cases

# gpg: Signature made Tue 03 Jul 2018 21:45:32 BST
# gpg:                using RSA key 3353C9CEF108B584
# gpg: Good signature from "Michael Roth <flukshun@gmail.com>"
# gpg:                 aka "Michael Roth <mdroth@utexas.edu>"
# gpg:                 aka "Michael Roth <mdroth@linux.vnet.ibm.com>"
# Primary key fingerprint: CEAC C9E1 5534 EBAB B82D  3FA0 3353 C9CE F108 B584

* remotes/mdroth/tags/qga-pull-2018-07-03-tag:
  qga: removing bios_supports_mode
  qga: systemd hibernate/suspend/hybrid-sleep support
  qga: removing switch statements, adding run_process_child
  qga: guest_suspend: decoupling pm-utils and sys logic
  qga: bios_supports_mode: decoupling pm-utils and sys logic
  qga: refactoring qmp_guest_suspend_* functions
  qemu-ga: make get-fsinfo work over pci bridges
  qga-win: Fixing msi upgrade disallow in WiX file
  qga/schema: fix documentation for GuestOSInfo
  test-qga: add trivial tests for some commands
  qga-win: add driver path usage to GuestFilesystemInfo
  qga: add mountpoint usage info to GuestFilesystemInfo
  qga: check bytes count read by guest-file-read
  qga: unset frozen state if no mount points are frozen

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-05 08:21:25 +01:00
Markus Armbruster d43b16945a qmp: Use QDict * instead of QObject * for response objects
By using the more specific type, we get fewer downcasts.  The
downcasts are safe, but not obviously so, at least not locally.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-24-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster 69240fe62d qmp: Don't let malformed in-band commands jump the queue
handle_qmp_command() reports certain errors right away.  This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
the previous commit demonstrates.

To fix, we need to delay errors until dispatch.  Do that for semantic
errors, mostly by reverting ill-advised parts of commit cf869d5317
"qmp: support out-of-band (oob) execution".  Bonus: doesn't run
qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
again in do_qmp_dispatch().  That's also due to commit cf869d5317.

The next commit will fix queue jumping for syntax errors.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-18-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster e8f4a22168 tests/qmp-test: Demonstrate QMP errors jumping the queue
When OOB is enabled, out-of-band commands are executed right away,
everything else is queued.  This lets out-of-band commands "jump the
queue".

However, certain errors are always reported right away, and therefore
can jump the queue even when the erroneous input does not request
out-of-band execution.  These errors are pretty unlikely to occur in
production, but it's wrong all the same.  Mark FIXME.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180703085358.13941-17-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster 00ecec151d qmp: Redo how the client requests out-of-band execution
Commit cf869d5317 "qmp: support out-of-band (oob) execution" added a
general mechanism for command-independent arguments just for an
out-of-band flag:

    The "control" key is introduced to store this extra flag.  "control"
    field is used to store arguments that are shared by all the commands,
    rather than command specific arguments.  Let "run-oob" be the first.

However, it failed to reject unknown members of "control".  For
instance, in QMP command

    {"execute": "query-name", "id": 42, "control": {"crap": true}}

"crap" gets silently ignored.

Instead of fixing this, revert the general "control" mechanism
(because YAGNI), and do it the way I initially proposed, with key
"exec-oob".  Simpler code, simpler interface.

An out-of-band command

    {"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}}

becomes

    {"exec-oob": "migrate-pause", "id": 42}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-13-armbru@redhat.com>
[Commit message typo fixed]
2018-07-03 23:18:56 +02:00
Markus Armbruster 674ed7228f qmp qemu-ga: Fix qemu-ga not to accept "control"
Commit cf869d5317 "qmp: support out-of-band (oob) execution"
accidentally made qemu-ga accept and ignore "control".  Fix that.

Out-of-band execution in a monitor that doesn't support it now fails
with

    {"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}

instead of

    {"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}

The old description is suboptimal when out-of-band cannot not be
enabled, or the command doesn't support out-of-band execution.

The new description is a bit unspecific, but it'll do.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-12-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster d4d7ed731c tests/test-qga: Demonstrate the guest-agent ignores "control"
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-11-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster 0fa39d0b03 qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"
Commit cf869d5317 "qmp: support out-of-band (oob) execution" changed
how we check "id":

    Note that in the patch I exported qmp_dispatch_check_obj() to be
    used to check the request earlier, and at the same time allowed
    "id" field to be there since actually we always allow that.

The part after "and" is ill-advised: it makes qemu-ga accept and
ignore "id".  Revert.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-10-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster b5f8431040 tests/test-qga: Demonstrate the guest-agent ignores "id"
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-9-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster 2970b4461f tests/qmp-test: Test in-band command doesn't overtake
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-7-armbru@redhat.com>
2018-07-03 23:18:56 +02:00
Markus Armbruster 97ca0712c8 qmp: Get rid of x-oob-test command
tests/qmp-test tests an out-of-band command overtaking a slow in-band
command.  To do that, it needs:

1. An in-band command that *reliably* takes long enough to be
   overtaken.

2. An out-of-band command to do the overtaking.

3. To avoid delays, a way to make the in-band command complete quickly
   after it was overtaken.

To satisfy these needs, commit 469638f9cb provides the rather
peculiar oob-capable QMP command x-oob-test:

* With "lock": true, it waits for a global semaphore.

* With "lock": false, it signals the global semaphore.

To satisfy 1., the test runs x-oob-test in-band with "lock": true.
To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.

Note that waiting for a semaphore violates the rules for oob-capable
commands.  Running x-oob-test with "lock": true hangs the monitor
until you run x-oob-test with "lock": false on another monitor (which
you might not have set up).

Having an externally visible QMP command that may hang the monitor is
not nice.  Let's apply a little more ingenuity to the problem.  Idea:
have an existing command block on reading a FIFO special file, unblock
it by opening the FIFO for writing.

For 1., use

    {"execute": "blockdev-add",  "id": ID1,
     "arguments": {
        "driver": "blkdebug", "node-name": ID1, "config": FIFO,
        "image": { "driver": "null-co"}}}

where ID1 is an arbitrary string, and FIFO is the name of the FIFO.

For 2., use

    {"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}

where ID2 is a different arbitrary string.  Since there's no migration
to pause, the command will fail, but that's fine; instant failure is
still a test of out-of-band responses overtaking in-band commands.

For 3., open FIFO for writing.

Drop QMP command x-oob-test.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-6-armbru@redhat.com>
[Error checking tweaked]
2018-07-03 23:15:43 +02:00
Tomáš Golembiovský 509b97fd07 test-qga: add trivial tests for some commands
These commands did not get their tests in the original commits:
- guest-get-host-name
- guest-get-timezone
- guest-get-users

Trivial tests that mostly only call the commands were added.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* replace QDECREF() with qobject_unref()
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
2018-07-03 15:20:26 -05:00
Marc-André Lureau 901a34a400 qapi: add 'If:' section to generated documentation
The documentation is generated only once, and doesn't know C
pre-conditions. Add 'If:' sections for top-level entities.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-13-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Marc-André Lureau 1f7b9f3181 qapi/commands: add #if conditions to commands
Wrap generated code with #if/#endif using an 'ifcontext' on
QAPIGenCSnippet objects.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180703155648.11933-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Line breaks tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Marc-André Lureau fbf09a2fa4 qapi: add 'ifcond' to visitor methods
Modify the test visitor to check correct passing of values.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-5-marcandre.lureau@redhat.com>
[Accidental change to roms/seabios dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau 967c885108 qapi: add 'if' to top-level expressions
Accept 'if' key in top-level elements, accepted as string or list of
string type. The following patches will modify the test visitor to
check the value is correctly saved, and generate #if/#endif code (as a
single #if/endif line or a series for a list).

Example of 'if' key:
{ 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
  'if': 'defined(TEST_IF_STRUCT)' }

The generated code is for now *unconditional*. Later patches generate
the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-2-marcandre.lureau@redhat.com>
[Commit message and Documentation improved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:21:24 +02:00
Richard W.M. Jones e1a6dc91dd crypto: Implement TLS Pre-Shared Keys (PSK).
Pre-Shared Keys (PSK) is a simpler mechanism for enabling TLS
connections than using certificates.  It requires only a simple secret
key:

  $ mkdir -m 0700 /tmp/keys
  $ psktool -u rjones -p /tmp/keys/keys.psk
  $ cat /tmp/keys/keys.psk
  rjones:d543770c15ad93d76443fb56f501a31969235f47e999720ae8d2336f6a13fcbc

The key can be secretly shared between clients and servers.  Clients
must specify the directory containing the "keys.psk" file and a
username (defaults to "qemu").  Servers must specify only the
directory.

Example NBD client:

  $ qemu-img info \
    --object tls-creds-psk,id=tls0,dir=/tmp/keys,username=rjones,endpoint=client \
    --image-opts \
    file.driver=nbd,file.host=localhost,file.port=10809,file.tls-creds=tls0,file.export=/

Example NBD server using qemu-nbd:

  $ qemu-nbd -t -x / \
    --object tls-creds-psk,id=tls0,endpoint=server,dir=/tmp/keys \
    --tls-creds tls0 \
    image.qcow2

Example NBD server using nbdkit:

  $ nbdkit -n -e / -fv \
    --tls=on --tls-psk=/tmp/keys/keys.psk \
    file file=disk.img

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-07-03 13:04:38 +01:00
Peter Maydell 9b75dcb15f nbd patches for 2018-07-02
Bug fixes and iotest exposure of fleecing via NBD (serving a
 read-only point-in-time view via blockdev-backup sync:none,
 as well as serving dirty bitmaps over NBD), including a new
 x-dirty-bitmap parameter when opening NBD clients as the
 counterpart to x-nbd-server-add-bitmap. Also a random fix
 for iscsi block_status spotted by Coverity that missed other
 miscellaneous trees.
 
 - Eric Blake: nbd/server: Fix dirty bitmap logic regression
 - Eric Blake: iscsi: Avoid potential for get_status overflow
 - John Snow/Vladimir Sementsov-Ogievskiy: 0/2 block: formalize and test fleecing
 - Eric Blake: 0/2 test NBD bitmap export
 -----BEGIN PGP SIGNATURE-----
 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
 
 iQEcBAABCAAGBQJbOtJPAAoJEKeha0olJ0NqEvwH/3FwWnlBdBvdYGgPjzGE1Atm
 ofCKcyxE/2VJtxeWlZQHzs1VqSq81s7am5SdzOrIWnQekvHFcLu6/71RABiauzMd
 neCvVOrXOVdktj1i1Z2Gg4BgjDmqbTDlo5ssVh/oXP0Zebi6OZFfQrB7y3cGBvui
 4XI7lW9qJxt6F1FlKloXnofWRDENyo5vgdz6QjQXfauthw2T5045RIPTfiz03FCp
 fbs+6K0+bKxfPdNLrqxxOZo/loYnEXbDYv6VBAIWBqztXVnMHxCqnh0YN05jwsfF
 TRW0/YT8lpWarOZ1soIC6a/OGXQZbxgRhZ+Zr+Wa2jw0YNHJanU9isxi37aUqQo=
 =Xivx
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-07-02' into staging

nbd patches for 2018-07-02

Bug fixes and iotest exposure of fleecing via NBD (serving a
read-only point-in-time view via blockdev-backup sync:none,
as well as serving dirty bitmaps over NBD), including a new
x-dirty-bitmap parameter when opening NBD clients as the
counterpart to x-nbd-server-add-bitmap. Also a random fix
for iscsi block_status spotted by Coverity that missed other
miscellaneous trees.

- Eric Blake: nbd/server: Fix dirty bitmap logic regression
- Eric Blake: iscsi: Avoid potential for get_status overflow
- John Snow/Vladimir Sementsov-Ogievskiy: 0/2 block: formalize and test fleecing
- Eric Blake: 0/2 test NBD bitmap export

# gpg: Signature made Tue 03 Jul 2018 02:33:03 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-07-02:
  iotests: New test 223 for exporting dirty bitmap over NBD
  nbd/client: Add x-dirty-bitmap to query bitmap from server
  iotests: add 222 to test basic fleecing
  blockdev: enable non-root nodes for backup source
  iscsi: Avoid potential for get_status overflow
  nbd/server: Fix dirty bitmap logic regression

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-07-03 10:47:02 +01:00
Markus Armbruster c069821220 qmp: Say "out-of-band" instead of "Out-Of-Band"
Affects documentation and a few error messages.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-2-armbru@redhat.com>
2018-07-03 11:46:54 +02:00
Eric Blake a1532a225a iotests: New test 223 for exporting dirty bitmap over NBD
Although this test is NOT a full test of image fleecing (as it
intentionally uses just a single block device directly exported
over NBD, rather than trying to set up a blockdev-backup job with
multiple BDS involved), it DOES prove that qemu as a server is
able to properly expose a dirty bitmap over NBD.

When coupled with image fleecing, it is then possible for a
third-party client to do an incremental backup by using
qemu-img map with the x-dirty-bitmap option to learn which parts
of the file are dirty (perhaps confusingly, they are the portions
mapped as "data":false - which is part of the reason this is
still in the x- experimental namespace), along with another
normal client (perhaps 'qemu-nbd -c' to expose the server over
/dev/nbd0 and then just use normal I/O on that block device) to
read the dirty sections.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180702191458.28741-3-eblake@redhat.com>
Tested-by: John Snow <jsnow@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2018-07-02 19:50:37 -05:00
John Snow bacebdedbf iotests: add 222 to test basic fleecing
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20180702194630.9360-3-jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-07-02 15:24:07 -05:00
Philippe Mathieu-Daudé 68dbb6d05d tests/crypto: Use the IEC binary prefix definitions
It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-45-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-07-02 15:41:17 +02:00
Philippe Mathieu-Daudé d23b6caadb hw: Use IEC binary prefix definitions from "qemu/units.h"
Code change produced with:

  $ git ls-files | egrep '\.[ch]$' | \
    xargs sed -i -e 's/\(\W[KMGTPE]\)_BYTE/\1iB/g'

Suggested-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: David Gibson <david@gibson.dropbear.id.au> (ppc parts)
Message-Id: <20180625124238.25339-6-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-07-02 15:41:10 +02:00
Peter Xu cbc4ae2d1a tests: iotests: drop some stderr line
In my Out-Of-Band test, "check -qcow2 060" fail with this:

  --- /home/peterx/git/qemu/tests/qemu-iotests/060.out
  +++ /home/peterx/git/qemu/bin/tests/qemu-iotests/060.out.bad
  @@ -427,8 +427,8 @@
  QMP_VERSION
  {"return": {}}
  qcow2: Image is corrupt: L2 table offset 0x2a2a2a00 unaligned (L1
  index: 0); further non-fatal corruption events will be suppressed
  -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "", "msg": "L2 table offset 0x2a2a2a0
  0 unaligned (L1 index: 0)", "node-name": "drive", "fatal": false}}
  read failed: Input/output error
  +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "", "msg": "L2 table offset 0x2a2a2a0
  0 unaligned (L1 index: 0)", "node-name": "drive", "fatal": false}}
  {"return": ""}
  {"return": {}}
  {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP},
  "event": "SHUTDOWN", "data": {"guest": false}}

The order of the event and the in/out error line is swapped.  I didn't
dig up the reason, but AFAIU what we want to verify is the event rather
than stderr.  Let's drop the stderr line directly for this test.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180620073223.31964-5-peterx@redhat.com>
[Commit message touched up]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-30 17:50:48 +02:00
Peter Maydell ce59ecc411 Block layer patches:
- Make truncate operations asynchronous (so that preallocation in
   blockdev-create doesn't block the main loop any more)
 - usb-storage: Add rerror/werror properties
 - nvme: Add num_queues property
 - qemu-img convert: Copy offloading fixes (including data corruption fix)
 - qcow2: Fix cluster leak on temporary write error
 - Use byte-based functions instead of bdrv_co_readv/writev()
 - Various small fixes and cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbNj1iAAoJEH8JsnLIjy/Wo0kQAIFv9WtDE1Y5bPRxM3lWCVLU
 paEz8XnURl3KtFnSMH6tG4CNz/ZlnjV2bqeDxF2u4bFxsMS39lpah0BdzG7cfjNS
 qSotKTLNg1hxRefyqpvZ291Atb8kSgQdw4vtZWVmwLgfSM1S6VwL+7QLPaCO0nwH
 8OELp44acD/Avpuum3ztXGdzf/bHX6/wRhba6q1avgP6A0o83aQaG39wTd9pOgzy
 ZWCuLhrThaNXckXUFWTkZRmxh7B+XyOGdfM/jGR6uUdvCs3b6HAQmkkwVsmj4d1b
 gau7nH0BGH7wiI+yCkhdMd7IwFxSLVKrBo5oKC9JzqDCjL5gEDIr1FoWmJPqQIog
 E1h9x44t3uNVcS8tpSis2bjKM4IGS4DgSEEynFLOqHJYCcNF6tFNnJZPouzw+Zt5
 HOYS3iUzMRjYvkTnnxZSYyXM5llu5kXGa0pkFA5nbiufYJ530aJX0rtqPSpeIzMC
 D9l4Rkq1iM9oh/zKUenC2beITRPxF7iBFbEuWX58NgTTgaP0OfT3tYT/ki14kTm4
 iROMu+CotBZHkkDB0x7VlNuWeoabw1DvR+Ipu7GUsZsLhk40owuslZeDSl6+C0Wm
 ABouiYgHNo5xch0znomTnAM4NBl3Izs+Tim8wNvFH6zrGaXQ8FnxvW7W0sUQnDMK
 ydQbf7Mmwv/yaG23Bik2
 =fNzf
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Make truncate operations asynchronous (so that preallocation in
  blockdev-create doesn't block the main loop any more)
- usb-storage: Add rerror/werror properties
- nvme: Add num_queues property
- qemu-img convert: Copy offloading fixes (including data corruption fix)
- qcow2: Fix cluster leak on temporary write error
- Use byte-based functions instead of bdrv_co_readv/writev()
- Various small fixes and cleanups

# gpg: Signature made Fri 29 Jun 2018 15:08:34 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (29 commits)
  block: Remove unused sector-based vectored I/O
  vhdx: Switch to byte-based calls
  replication: Switch to byte-based calls
  qcow: Switch to a byte-based driver
  qcow: Switch qcow_co_writev to byte-based calls
  qcow: Switch qcow_co_readv to byte-based calls
  qcow: Switch get_cluster_offset to be byte-based
  parallels: Switch to byte-based calls
  file-posix: Fix EINTR handling
  iscsi: Don't blindly use designator length in response for memcpy
  qcow2: Fix src_offset in copy offloading
  file-posix: Implement co versions of discard/flush
  qemu-iotests: Test qcow2 not leaking clusters on write error
  qcow2: Free allocated clusters on write error
  qemu-iotests: Update 026.out.nocache reference output
  block/crypto: Simplify block_crypto_{open,create}_opts_init()
  block: Move request tracking to children in copy offloading
  qcow2: Remove dead check on !ret
  file-posix: Make .bdrv_co_truncate asynchronous
  block: Use tracked request for truncate
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-29 18:29:15 +01:00
Peter Maydell 75507f1aba glib: update the min required version
This updates the minimum required glib version to 2.40
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJbNhcKAAoJEL6G67QVEE/f1gMP+wV4J+V9WJOxcXbNjP9bb1K4
 gsvZrv2pKCVbcJW2o3Mc3POXExX1E2GsrAl639zpDHbNVlVFvQC51GaRO1Fc6lwh
 7NRFHen0Ee6I7TvWlVXMy1YPXPZ/8EJ/1KuZerYUUyKO/R8ojEt/TbELwfl3P1LF
 VZGWgs+GpwUYwiG4ZmYPQ0VsT/munZPlaK1mRQSDbGqX0KG1Vy8Q+mgbGAm+S6xh
 39HtM7ecdfXbVEAnoMsp9+kRi3zXpD5zSAyVBZN2RDktMt/EHxF0pPJCqX7oq0Rn
 DehXDkzaqF2ghzrSAIT3rbKBhzYIY/ny/vxnyQZ/Px0GrBdKwuUJ+pYcF2akS2hV
 s98VWRw/tqt8wQKYLF/wmeL7eE/tKAHSUqFc3Ta0Rvgfq0z9Mp3b73vuswBpfjWL
 +GiASRvg96n/1yqmywtCd7KtF5riOo1iyvfMFCdQ+nBHQok/6K/oWfZPyoZsCAIa
 2GJBfmHzkkc98QeNO0Dp/+ckSyKIBuyTV1bDyq/8Yz7IwEd1PfRWooEgVFSHO7MU
 6ddCECvKVvP0S03r5dlw4Imio39gPpeZBMmSE4ZOh/hRa89T4UqIUvSLwzat7OhA
 DH1NhJsj7G1jaIxENfzBhpggg++rHCVcfc2cv28Tl7i7twyRsb7CveYA6TwD+UIg
 7JT+KswUo7W9MrkKBHC+
 =8tKK
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange/tags/min-glib-pull-request' into staging

glib: update the min required version

This updates the minimum required glib version to 2.40

# gpg: Signature made Fri 29 Jun 2018 12:24:58 BST
# gpg:                using RSA key BE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>"
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>"
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange/tags/min-glib-pull-request:
  glib: enforce the minimum required version and warn about old APIs
  glib: bump min required glib library version to 2.40
  util: remove redundant include of glib.h and add osdep.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-29 15:04:20 +01:00
Fam Zheng e06f4639d8 qcow2: Fix src_offset in copy offloading
Not updating src_offset will result in wrong data being written to dst
image.

Reported-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-29 14:20:56 +02:00
Kevin Wolf ae376c6255 qemu-iotests: Test qcow2 not leaking clusters on write error
This adds a test for a temporary write failure, which simulates the
situation after werror=stop/enospc has stopped the VM. We shouldn't
leave leaked clusters behind in such cases.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-06-29 14:20:56 +02:00
Kevin Wolf 93a3642efc qemu-iotests: Update 026.out.nocache reference output
Commit abf754fe40 updated 026.out, but forgot to also update
026.out.nocache.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-06-29 14:20:56 +02:00
Daniel P. Berrangé e71e8cc035 glib: enforce the minimum required version and warn about old APIs
There are two useful macros that can be defined before including
glib.h that are related to the min required glib version

 - GLIB_VERSION_MIN_REQUIRED

   When this is defined, if code uses an API that was deprecated
   in this version, or older, a compiler warning will be emitted.
   This alerts maintainers to update their code to whatever new
   replacement API is now recommended best practice.

 - GLIB_VERSION_MAX_ALLOWED

   When this is defined, if code uses an API that was introduced
   in a version that is newer than the declared version, a compiler
   warning will be emitted. This alerts maintainers if new code
   accidentally uses functionality that won't be available on some
   supported platforms.

The GLIB_VERSION_MAX_ALLOWED constant makes it a bit harder to opt
in to using specific new APIs with a GLIB_CHECK_VERSION conditional.
To workaround this Pragmas can be used to temporarily turn off the
-Wdeprecated-declarations compiler warning, while a static inline
compat function is implemented. This workaround is illustrated with the
implementation of the g_strv_contains method to satisfy the test suite.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-06-29 12:22:28 +01:00
Daniel P. Berrangé e7b3af8159 glib: bump min required glib library version to 2.40
Per supported platforms doc[1], the various min glib on relevant distros is:

  RHEL-7: 2.50.3
  Debian (Stretch): 2.50.3
  Debian (Jessie): 2.42.1
  OpenBSD (Ports): 2.54.3
  FreeBSD (Ports): 2.50.3
  OpenSUSE Leap 15: 2.54.3
  SLE12-SP2: 2.48.2
  Ubuntu (Xenial): 2.48.0
  macOS (Homebrew): 2.56.0

This suggests that a minimum glib of 2.42 is a reasonable target.

The GLibC compile farm, however, uses Ubuntu 14.04 (Trusty) which only
has glib 2.40.0, and this is needed for testing during merge. Thus an
exception is made to the documented platform support policy to allow for
all three current LTS releases to be supported.

Docker jobs that not longer satisfy this new min version are removed.

[1] https://qemu.weilnetz.de/doc/qemu-doc.html#Supported-build-platforms

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-06-29 12:22:28 +01:00
Thomas Huth 28a3cfc10b tests/boot-serial: Do not delete the output file in case of errors
Peter reported that the boot-serial tester sometimes runs into timeouts
with SPARC guests. It's currently completely unclear whether this is due
to too much load on the host machine (so that the guest really just ran
too slow), or whether there is something wrong with the guest's firmware
boot. For further debugging, we need the serial output of the guest in
case of errors, so instead of unlinking the file immediately, this is
now only done in case of success. In case of error, print the name of the
file with the serial output via g_error() (which then also calls abort()
internally to mark the test as failed).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1526977831-31129-1-git-send-email-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-06-29 13:02:50 +02:00
Emilio G. Cota 70c31264af tests/atomic_add-bench: add -m option to use mutexes
This allows us to use atomic-add-bench as a microbenchmark
for evaluating qemu_mutex_lock's performance.

Signed-off-by: Emilio G. Cota <cota@braap.org>
[cherry picked from https://github.com/cota/qemu/commit/f04f34df]
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180425025459.5258-2-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-06-28 19:05:31 +02:00
Paolo Bonzini ae79c2db15 ahci: fix FIS I bit and PIO Setup FIS interrupt
The "I" bit in PIO Setup and D2H FISes is exclusively a device concept
and the irqstatus register in the controller does not matter.  The SATA
spec says when it should be one; for D2H FISes in practice it is always
set, while the PIO Setup FIS has several subcases that are documented in
the patch.

Also, the PIO Setup FIS interrupt is actually generated _after_ data
has been received.

Someone should probably spend some time reading the SATA specification and
figuring out the more obscure fields in the PIO Setup FIS, but this is enough
to fix SeaBIOS booting from ATAPI CD-ROMs over an AHCI controller.

Fixes: 956556e131
Reported-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20180622165159.19863-1-pbonzini@redhat.com
[Minor edit to avoid ATAPI comment ambiguity. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-25 16:50:48 -04:00
Matthias Maier 0d6b93deee Revert commit d4e5ec877c
This commit removes the PYTHON_UTF8 workaround. The problem with setting

  LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8

is that the en_US.UTF-8 locale might not be available. In this case
setting above locales results in build errors even though another UTF-8
locale was originally set [1]. The only stable way of fixing the
encoding problem is by specifying the encoding in Python, like the
previous commit does.

[1] https://bugs.gentoo.org/657766

Signed-off-by: Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com>
Signed-off-by: Matthias Maier <tamiko@43-1.org>
Message-Id: <20180618175958.29073-3-armbru@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:47 +02:00
Anton Nefedov 800877bb16 qapi: allow empty branches in flat unions
It often happens that just a few discriminator values imply extra data in
a flat union. Existing checks did not make possible to leave other values
uncovered. Such cases had to be worked around by either stating a dummy
(empty) type or introducing another (subset) discriminator enumeration.

Both options create redundant entities in qapi files for little profit.

With this patch it is not necessary anymore to add designated union
fields for every possible value of a discriminator enumeration.

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Message-Id: <1529311206-76847-2-git-send-email-anton.nefedov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:46 +02:00
Max Reitz fe170d8bfa tests: Add QDict clone-flatten test
This new test verifies that qdict_flatten() does not modify a shallow
clone of the given QDict.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20180611205203.2624-8-mreitz@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:46 +02:00
Peter Maydell de44c04442 Add check-tcg machinary
This restores the ability to run TCG smoke tests by using our docker
 infrastructure to support cross building simple tests. It represents
 the first step to making better cross-architecture testing available
 straight from the source tree ;-)
 
 v2
   - fix quoting of target_compiler
   - make docker.py Py3 safe
   - tweak .travis.yml recipe
   - don't probe docker when HAVE_USER_DOCKER not set
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAlsrRHEACgkQ+9DbCVqe
 KkSRhAf9EkJLIkB0uUs6NXu8ZKbMoAFlpQtbHZ36/YvTWHW+oDOBjYkZyrRlsTMR
 lIpm51wSAtVOJnScFASKbYQAD4ZTw2CUjwRqkCUhWI/vJOR7NvzZHfF+SHCNl8QE
 un7N/n3pWga7vY3j5Pd9cXz14im42Lcyl/xL5juh0lpCo+dw9OTdMT49sPF3YJGC
 N7uK3gIqFg24nofCAZq/YyUGzfftSkZ8x6D9k9fhHyPUGh+28sE2Ahptb6jGAYnr
 IcfHSUFkHpvliEJCJ50ho3iNF/JvV57gluW/LLQua35XBJ7sgiXHlhJP6Fnkobps
 Z8tzaEFGg96fFOreSs0PkqTrLhMO5w==
 =BIF8
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-tcg-testing-revivial-210618-2' into staging

Add check-tcg machinary

This restores the ability to run TCG smoke tests by using our docker
infrastructure to support cross building simple tests. It represents
the first step to making better cross-architecture testing available
straight from the source tree ;-)

v2
  - fix quoting of target_compiler
  - make docker.py Py3 safe
  - tweak .travis.yml recipe
  - don't probe docker when HAVE_USER_DOCKER not set

# gpg: Signature made Thu 21 Jun 2018 07:23:45 BST
# gpg:                using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-tcg-testing-revivial-210618-2: (57 commits)
  .travis.yml: add check-tcg test
  tests/docker/Makefile.include: only force SID to NOCACHE if old
  docker: docker.py adding age check command
  tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1
  docker: docker.py add check sub-command
  docker: docker.py don't conflate checksums for extra_files
  docker: docker.py use "version" to probe usage
  tests: add top-level make dependency for docker builds
  tests/tcg/i386: extend timeout for runcom test
  tests/tcg: override runners for broken tests
  tests/tcg: add run, diff, and skip helper macros
  tests/Makefile.include: add [build|clean|check]-tcg targets
  Makefile.target: add (clean-/build-)guest-tests targets
  tests/tcg/Makefile: update to be called from Makefile.target
  tests/tcg: enable building for PowerPC
  docker: move debian-powerpc-cross to sid based build
  tests/tcg: enable building for RISCV64
  tests/tcg: enable building for mips64
  tests/tcg: enable building for sparc64
  tests/tcg: enable building for sh4
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-22 10:57:47 +01:00
Peter Maydell 409ef9eb4a nbd patches for 2018-06-20
Add experimental x-nbd-server-add-bitmap to expose a disabled
 bitmap over NBD, in preparation for a pull model incremental
 backup scheme. Also fix a corner case protocol issue with
 NBD_CMD_BLOCK_STATUS, and add new NBD_CMD_CACHE.
 
 - Eric Blake: tests: Simplify .gitignore
 - Eric Blake: nbd/server: Reject 0-length block status request
 - Vladimir Sementsov-Ogievskiy: 0/6 NBD export bitmaps
 - Vladimir Sementsov-Ogievskiy: nbd/server: introduce NBD_CMD_CACHE
 -----BEGIN PGP SIGNATURE-----
 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
 
 iQEcBAABCAAGBQJbK7wDAAoJEKeha0olJ0NqJuMH/0o7wzP3HWIO1pJZvhRA81AP
 ZZqZy/8xO0B++keCxzgTbdr9yf6RDtYGjDrNuOcVzqfA1k9E1Cl2pPjS93TzPlzZ
 PEgtG6bLXL2jCUi/8/EbZ/TLuoQx5YJc7eXTNREgXDRMLiTEg5Kcmg1n0efhr8Sl
 5/bRMDmW1+atO4dJS8bEW5WLuy4IoB823cSHjWoPFRHeNCwmFREwTx2xVULVje84
 JZzHYxyaBRquYLKIAew1WidhKVC4OiYtm4F5PxGBB/ZCVkeoDBX5uqZSjOUVt8y7
 uOhGYQGW/qSj/s3x74NEL6IPo57Iay/Wq065DbolDx7s0y5t38TocrBnakzOBFU=
 =VK6J
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-06-20-v2' into staging

nbd patches for 2018-06-20

Add experimental x-nbd-server-add-bitmap to expose a disabled
bitmap over NBD, in preparation for a pull model incremental
backup scheme. Also fix a corner case protocol issue with
NBD_CMD_BLOCK_STATUS, and add new NBD_CMD_CACHE.

- Eric Blake: tests: Simplify .gitignore
- Eric Blake: nbd/server: Reject 0-length block status request
- Vladimir Sementsov-Ogievskiy: 0/6 NBD export bitmaps
- Vladimir Sementsov-Ogievskiy: nbd/server: introduce NBD_CMD_CACHE

# gpg: Signature made Thu 21 Jun 2018 15:53:55 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-06-20-v2:
  nbd/server: introduce NBD_CMD_CACHE
  docs/interop: add nbd.txt
  qapi: new qmp command nbd-server-add-bitmap
  nbd/server: implement dirty bitmap export
  nbd/server: add nbd_meta_empty_or_pattern helper
  nbd/server: refactor NBDExportMetaContexts
  nbd/server: fix trace
  nbd/server: Reject 0-length block status request
  tests: Simplify .gitignore

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-22 09:58:29 +01:00
Peter Maydell 33836a7315 TCG patch queue:
Workaround macos assembler lossage.
 Eliminate tb_lock.
 Fix TB code generation overflow.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJbJBZIAAoJEGTfOOivfiFfy0gH/1brodMhJbTS6/k9+FyXWEy5
 zYjCGKKlMZk//Y+4wcF5tXY/qDRNWk80j6KyxumNp3gCBehx6u59EEsrJRQaxBHm
 nYbDoE3Fy0J4KgRzdGmkYtl89XDK1++Ea9uL9N/stg2MSodzqoV6uudLYr/f+nRj
 4MkS+7BI+aJ4/XIKLU+/+cRo+5FdD0hNEabjlUxTOSrfJbr/YxbnVINX01A4yD6q
 LSzwLAEqpJehFBQjeSLu93ztrapj/1vEaguPOf04F6pXgOLpvSPlPahqwwk4qRwS
 OFgWwSPby3jrNLYZcufx2cY5pG3i4wDGK3z/B35hnDEGwYp1fNt6xdq+EzmHhaM=
 =ibt/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20180615' into staging

TCG patch queue:

Workaround macos assembler lossage.
Eliminate tb_lock.
Fix TB code generation overflow.

# gpg: Signature made Fri 15 Jun 2018 20:40:56 BST
# gpg:                using RSA key 64DF38E8AF7E215F
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>"
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-tcg-20180615:
  tcg: Reduce max TB opcode count
  tcg: remove tb_lock
  translate-all: remove tb_lock mention from cpu_restore_state_from_tb
  cputlb: remove tb_lock from tlb_flush functions
  translate-all: protect TB jumps with a per-destination-TB lock
  translate-all: discard TB when tb_link_page returns an existing matching TB
  translate-all: introduce assert_no_pages_locked
  translate-all: add page_locked assertions
  translate-all: use per-page locking in !user-mode
  translate-all: move tb_invalidate_phys_page_range up in the file
  translate-all: work page-by-page in tb_invalidate_phys_range_1
  translate-all: remove hole in PageDesc
  translate-all: make l1_map lockless
  translate-all: iterate over TBs in a page with PAGE_FOR_EACH_TB
  tcg: move tb_ctx.tb_phys_invalidate_count to tcg_ctx
  tcg: track TBs with per-region BST's
  qht: return existing entry when qht_insert fails
  qht: require a default comparison function
  tcg/i386: Use byte form of xgetbv instruction

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-21 17:54:26 +01:00
Eric Blake ac5de4984d tests: Simplify .gitignore
Commit 0bcc8e5b was yet another instance of 'git status' reporting
dirty files after an in-tree build, thanks to the new binary
tests/check-block-qdict.

Instead of piecemeal exemptions of each new binary as they are
added, let's use git's negative globbing feature to exempt ALL
files that have a 'test-' or 'check-' prefix, except for the ones
ending in '.c' or '.sh'.  We still have a couple of generated
files that then need (re-)exclusion, but the overall list is a
LOT shorter, and less prone to needing future edits.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180619203918.65450-1-eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-21 09:21:19 -05:00
Alex Bennée 5499fa9e44 tests/docker/Makefile.include: only force SID to NOCACHE if old
Now we can check the age of a docker image we can be a little more
intelligent about re-building Sid images and only force NOCACHE if
it is "old".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée 7b882245f4 docker: docker.py adding age check command
This is useful for querying if an image is too old.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 22:13:39 +01:00
Alex Bennée cff5477330 tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1
As we now ensure all the images we are going to use are built in the
top level make file lets not over complicate things by running the
full script again. We do run the check script just in case someone
deletes the docker image while we are running.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée f97da1f717 docker: docker.py add check sub-command
This command allows you to check if we need to re-build a docker
image. If the image isn't in the repository or the checksums don't
match then we return false and some text (for processing in
makefiles).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée f9172822e7 docker: docker.py don't conflate checksums for extra_files
This just gets confusing especially as the helper function doesn't
even take into account any extra files (or the executable). Currently
the actual check just ignores them and also passes the result through
_dockerfile_preprocess so we fix that too.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 22:13:39 +01:00
Alex Bennée 83405c4517 docker: docker.py use "version" to probe usage
The "images" command is a fairly heavyweight command to run as it
involves searching the whole docker file-system inventory. On a
machine with a lot of images this makes start-up fairly expensive.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée 1a7fab926b tests: add top-level make dependency for docker builds
One problem with satisfying your docker dependencies in a sub-make it
you might end up trying to satisfy the dependency multiple times. This
is especially a problem with debian-sid based cross compilers and CI
setups. We solve this by doing a docker build pass at the top level
before any sub-makes are called.

We still need to satisfy dependencies in the Makefile.target call so
people can run tests from individual target directories. We introduce
a new Makefile.probe which gets called for each PROBE_TARGET and
allows us to build up the list. It does require multiply including
config-target.mak which shouldn't cause any issues as it shouldn't
define anything that clashes with config-host.mak. However we undefine
a few key variables each time around.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:32 +01:00
Alex Bennée ab93256144 tests/tcg/i386: extend timeout for runcom test
The Travis hardware can be a little slow and the runcom test is fairly
heavy in calculating pi. Lets double the timeout so we don't trip up
during CI by mistake.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 83749e649e tests/tcg: override runners for broken tests
To get a clean run of check-tcg these tests are currently skipped:

  - hello-mips for mips
  - linux-test for sparc

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 607bf9b5a5 tests/tcg: add run, diff, and skip helper macros
As we aren't using the default runners for all the test cases it is
easy to miss out things like timeouts. To help with this we add some
helpers and use them so we only need to make core changes in one
place.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 8ba0a8206a tests/Makefile.include: add [build|clean|check]-tcg targets
This will ensure all linux-user targets build their guest test
programs and ensure check-tcg will run the respective tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée c722a9e492 Makefile.target: add (clean-/build-)guest-tests targets
Now all the build infrastructure is in place we can build tests for
each guest that we support. That support mainly depends on having
cross compilers installed or docker setup. To keep all the logic for
that together we put the rules in tests/tcg/Makefile.include and
include it from the main Makefile.target.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 7e97017e7d tests/tcg/Makefile: update to be called from Makefile.target
This make is now invoked from each individual target make with the
appropriate CC and EXTRA_CFLAGS set for each guest. It then includes
additional Makefile.targets from:

  - tests/tcg/multiarch (always)
  - tests/tcg/$(TARGET_BASE_ARCH) (if available)
  - tests/tcg/$(TARGET_NAME)

The order is important as the later Makefile's may want to suppress
TESTS from its base arch profile. Each included Makefile.target is
responsible for adding TESTS as well as defining any special build
instructions for individual tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 97103c256e tests/tcg: enable building for PowerPC
Now we have restored debian-image-powerpc-cross using Debian SID
compilers we can build for 32 bit powerpc. Although PPC32 supports a
range of pages sizes currently only 4k works so the others are
commented out for now.

We can also merge the ppc64 support under the base architecture
directory to avoid too much proliferation of directories.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 08f56d8c9c docker: move debian-powerpc-cross to sid based build
The original Jessie based cross builder hasn't worked for a while. The
state of the libraries is still perilous for cross-building QEMU but
we can use it for building TCG tests.

The debian-apt-fake.sh script can also be dropped as it is no longer
used.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée e100a96755 tests/tcg: enable building for RISCV64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 33b2c4b50f tests/tcg: enable building for mips64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée cc6c7365b8 tests/tcg: enable building for sparc64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée a5ec3e36dc tests/tcg: enable building for sh4
As before, using Debian SID compilers. While the compiler can be
coerced into generating big-endian code it seems the linker can't deal
with it so we only enable the building for little endian SH4.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 21d71c8c41 tests/tcg: enable building for m68k
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 3ae8a1000b tests/tcg: enable building for HPPA
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 4f2cbcfb4d tests/tcg/alpha: add Alpha specific tests
These tests did use their own crt.o stub however that is a little
stone age so we drop crt.S and just statically link to the cross
compilers libraries.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 409c1c9cbd tests/tcg: enable building for Alpha
We can't use our normal Debian based compilers as Alpha isn't an
officially supported architecture. However it is available as a port
and fortunately cross compilers for all these targets are included in
Debian Sid, the perpetual rolling/unstable/testing version of Debian.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 1e2107092d tests/tcg: enable building for ppc64
Currently this just enables building the multiarch tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 78353f49d2 tests/tcg: enable building for s390x
This doesn't add any additional tests but enables building the
multiarch tests for s390x.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
2018-06-20 20:22:34 +01:00
Alex Bennée 2ac93f19b6 tests/tcg/mips: include common mips hello-mips
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 6bf77518dd tests/docker/Makefile.include: fix mipsel-cross dependancy
This got broken in commit 4319db7 but generally only shows up when you
try and do massive parallel builds on fresh machines.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 7086ffbb93 tests/tcg: enable building for MIPS
This doesn't add any additional tests but enables building the
multiarch tests for MIPS using docker cross compilers. We don't have a
cross compiler for mips64 big endian though.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 106ea2db12 tests/tcg: move MIPS specific tests into subdir
These only need to be built for MIPS guests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 8ec8a55e3f tests/tcg/arm: add fcvt test cases for AArch32/64
This runs through the usual float to float conversions and crucially
also runs with ARM Alternative Half Precision Format.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 29e0436e3d tests/tcg: enable building for AArch64
We only have compilers for the (default) little endian variants.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 65eab0f8cb tests/tcg/arm: fix up test-arm-iwmmxt test
We need to rename the source file to a .S so we can do a single-line
assemble and link invocation. We also specify the additional CFLAGS
for the compile as it's a non-standard ARM binary.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[rth: force fpu configuration]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée ab4aac5023 tests/tcg: enable building for ARM
This allows us to use the docker cross compiler image to build these
tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée c348722c53 tests/tcg: move ARM specific tests into subdir
These only need to be built for ARM guests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée e571ba67cb tests/tcg/i386/test-i386: fix printf format
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 9b8381d1ed tests/tcg/i386/test-i386: use modern vector_size attributes
The compiler complains about the old __mode__ style attributes.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 3a082ec01b tests/tcg/x86_64: add Makefile.target
The sources for x86_64 are shared in the i386 directory which will be
included thanks to TARGET_BASE_ARCH. However not all sources build so
we need to filter out the ones we can't build in the 64 bit world and
those that can't be built for 32 bit.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 25f9e7e8d7 tests/tcg/i386: add runner for test-i386-fprem
The runner needs to compare against a reference run. We also only run
this test when SPEED=slow as it takes a while.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 9ae02bf889 tests/tcg/i386: fix test-i386
We don't include anything from qemu itself for the build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Fam Zheng 553a5a6046 tests/tcg/i386: Build fix for hello-i386
We have -Werror=missing-prototype, add a dummy prototype to avoid that
warning.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 8b17219e7d tests/tcg: enable building for i386
While you can construct a compile command that does work using the
x86_64 host compiler that most people use this is flakey. Different
distros handle this is different ways so we default to using a known
good i386 compiler via docker.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 4132431f24 tests/tcg: move i386 specific tests into subdir
These only need to be built for i386 guests. This includes a stub
tests/tcg/i386/Makfile.target which absorbs some of what was in
tests/tcg/Makefile.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 07c85b696a tests/tcg/multiarch: move most output to stdout
The default test run outputs to stdout so it can be re-directed.
Errors are still reported to stderr.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2018-06-20 20:22:34 +01:00
Alex Bennée 5aa632d5d9 tests/tcg/multiarch: don't hard code paths/ports for linux-test
The fixed path and ports get in the way of running our tests and
builds in parallel. Instead of using TESTPATH we use mkdtemp() and
instead of a fixed port we allow the kernel to assign one and query it
afterwards.

Ideally test directory creation should be common functionally across
all TCG tests but this could complicate an already huge patch series
so we mark it as a TODO for next time.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 875fcce661 tests/tcg/multiarch: enable additional linux-test tests
Un-comment the remaining tests.

I removed the itimer value tests because I'm fairly sure a re-arming
timer will always have a different value in it when you grab it.

I've also fixed up the clone thread flags as QEMU will only allow a
clone to use flags which match glibc. However the test is still racey
so it remains disabled by default - it can be run by passing any
additional parameters on the command line.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Fam Zheng 5e13cbd51d tests/tcg/multiarch: Build fix for linux-test
To keep the compiler happy, and to fit in our buildsys flags:

- Make local functions "static"
- #ifdef out unused functions
- drop cutils/osdep dependencies

Signed-off-by: Fam Zheng <famz@redhat.com>
[AJB: drop cutils/osdep dependencies]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée dd28bebd02 tests/tcg: move architecture independent tests into subdir
We will want to build these for all supported guest architectures so
lets move them all into one place. We also drop test_path at this
point because it needs qemu utils and glib bits which is hard to
support for cross compiling.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-20 20:22:34 +01:00
Alex Bennée e27cae268f docker: Makefile.include introduce DOCKER_SCRIPT
Define this in one place to make it easy to re-use.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 50b7273854 docker: allow "cc" command to run in user context
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 99cfdb8662 docker: extend "cc" command to accept compiler
When calling our cross-compilation images we want to call something
other than the default cc.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
2018-06-20 20:22:34 +01:00
Alex Bennée 5e03c2d816 docker: Add "cc" subcommand
Signed-off-by: Fam Zheng <famz@redhat.com>
[AJB: add if args.paths check]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 7a5d936b6f docker: docker.py wrap StringIO import for python3
Although the docker.py is nominally python2 we actually invoke it with
the configured python from the configure script.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 16:57:34 +01:00
Peter Maydell 0f01b9fdd4 Block layer patches:
- Active mirror (blockdev-mirror copy-mode=write-blocking)
 - bdrv_drain_*() fixes and test cases
 - Fix crash with scsi-hd and drive_del
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbJ+FaAAoJEH8JsnLIjy/WIRoP/29yh6sRfk+bgxxKczctZLSr
 hJYkf62oZG0NjwAqjA+P9TJjUFLuRS62cVE5IljvcKjcAaXIavJy5ZIqHx/T2BS2
 faaUMOjlhbMtH8Emeun8BoTzPXKME3m7IifxTvc1g40UkeTbeGFF8P3wZ32QtduO
 OwxkT4qM0tiV4rin2Gds4IGokigx9qJKAN3i2PJNlUYg9kXs7tjdVuJp73OK3ZVb
 w0uYvPsiY65GJ3VrLxyPrH3wCWMnI7Ep4ekbjYTusoV57iZdQ6e3kDwqAgCGNPaE
 BaCQx7Aza7D6EDE+fOKIegyh7AyUy+oA1kcA5Z4u4qsasvYIYm7bQIFh+ohRHHpf
 +SbvAY1dockOYuN6V6K2EkIU6jNFmgUFHAU9jHb7QBlgYDfsTBNOPgK6po3NtjJM
 Scv9aB7aHGmTvRFYb1LHHJWsiMOaWgXgCFelAzM6PfhSvPPE8110doGS7dqbDAI+
 PThdtvXoNACuvwNUKGc5I16iO9j3p1aL4RYz2e/9xVlsATwUYiU3D1ICpw/ejb4z
 gfwseeFxdLfWV/bNylYd9FZJSs2NoZhNrw+qghIkJ9hTO2YJguCLqQTtJzT2gTXT
 YVjEhTFGcOtTgG3Z+s5PhHwkWBCVmJWUFxh0eDa4ejfHqc1e+A2Tqxh4LgnsloaK
 WgHFympxV6Sb1s7V8i7V
 =6Krl
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Active mirror (blockdev-mirror copy-mode=write-blocking)
- bdrv_drain_*() fixes and test cases
- Fix crash with scsi-hd and drive_del

# gpg: Signature made Mon 18 Jun 2018 17:44:10 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (35 commits)
  iotests: Add test for active mirroring
  block/mirror: Add copy mode QAPI interface
  block/mirror: Add active mirroring
  job: Add job_progress_increase_remaining()
  block/mirror: Add MirrorBDSOpaque
  block/dirty-bitmap: Add bdrv_dirty_iter_next_area
  test-hbitmap: Add non-advancing iter_next tests
  hbitmap: Add @advance param to hbitmap_iter_next()
  block: Generalize should_update_child() rule
  block/mirror: Use source as a BdrvChild
  block/mirror: Wait for in-flight op conflicts
  block/mirror: Use CoQueue to wait on in-flight ops
  block/mirror: Convert to coroutines
  block/mirror: Pull out mirror_perform()
  block: fix QEMU crash with scsi-hd and drive_del
  test-bdrv-drain: Test graph changes in drain_all section
  block: Allow graph changes in bdrv_drain_all_begin/end sections
  block: ignore_bds_parents parameter for drain functions
  block: Move bdrv_drain_all_begin() out of coroutine context
  block: Allow AIO_WAIT_WHILE with NULL ctx
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-19 16:04:43 +01:00
Max Reitz e38da02091 iotests: Add test for active mirroring
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20180613181823.13618-15-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-18 17:05:17 +02:00
Max Reitz 269576848e test-hbitmap: Add non-advancing iter_next tests
Add a function that wraps hbitmap_iter_next() and always calls it in
non-advancing mode first, and in advancing mode next.  The result should
always be the same.

By using this function everywhere we called hbitmap_iter_next() before,
we should get good test coverage for non-advancing hbitmap_iter_next().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20180613181823.13618-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-18 17:04:56 +02:00
Max Reitz a33fbb4f8b hbitmap: Add @advance param to hbitmap_iter_next()
This new parameter allows the caller to just query the next dirty
position without moving the iterator.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20180613181823.13618-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-18 17:04:55 +02:00
Kevin Wolf 19f7a7e574 test-bdrv-drain: Test graph changes in drain_all section
This tests both adding and remove a node between bdrv_drain_all_begin()
and bdrv_drain_all_end(), and enabled the existing detach test for
drain_all.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf 57320ca961 test-bdrv-drain: Test that bdrv_drain_invoke() doesn't poll
This adds a test case that goes wrong if bdrv_drain_invoke() calls
aio_poll().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf 231281ab42 test-bdrv-drain: Graph change through parent callback
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf ebd3183761 test-bdrv-drain: Test node deletion in subtree recursion
If bdrv_do_drained_begin() polls during its subtree recursion, the graph
can change and mess up the bs->children iteration. Test that this
doesn't happen.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Max Reitz 4c8158e359 test-bdrv-drain: Add test for node deletion
This patch adds two bdrv-drain tests for what happens if some BDS goes
away during the drainage.

The basic idea is that you have a parent BDS with some child nodes.
Then, you drain one of the children.  Because of that, the party who
actually owns the parent decides to (A) delete it, or (B) detach all its
children from it -- both while the child is still being drained.

A real-world case where this can happen is the mirror block job, which
may exit if you drain one of its children.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf 89bd030533 block: Really pause block jobs on drain
We already requested that block jobs be paused in .bdrv_drained_begin,
but no guarantee was made that the job was actually inactive at the
point where bdrv_drained_begin() returned.

This introduces a new callback BdrvChildRole.bdrv_drained_poll() and
uses it to make bdrv_drain_poll() consider block jobs using the node to
be drained.

For the test case to work as expected, we have to switch from
block_job_sleep_ns() to qemu_co_sleep_ns() so that the test job is even
considered active and must be waited for when draining the node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf 6d0252f2f9 tests/test-bdrv-drain: bdrv_drain_all() works in coroutines now
Since we use bdrv_do_drained_begin/end() for bdrv_drain_all_begin/end(),
coroutine context is automatically left with a BH, preventing the
deadlocks that made bdrv_drain_all*() unsafe in coroutine context. Now
that we even removed the old polling code as dead code, it's obvious
that it's compatible now.

Enable the coroutine test cases for bdrv_drain_all().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf 79ab8b21dc block: Use bdrv_do_drain_begin/end in bdrv_drain_all()
bdrv_do_drain_begin/end() implement already everything that
bdrv_drain_all_begin/end() need and currently still do manually: Disable
external events, call parent drain callbacks, call block driver
callbacks.

It also does two more things:

The first is incrementing bs->quiesce_counter. bdrv_drain_all() already
stood out in the test case by behaving different from the other drain
variants. Adding this is not only safe, but in fact a bug fix.

The second is calling bdrv_drain_recurse(). We already do that later in
the same function in a loop, so basically doing an early first iteration
doesn't hurt.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-06-18 15:03:25 +02:00
Kevin Wolf bb67568954 test-bdrv-drain: bdrv_drain() works with cross-AioContext events
As long as nobody keeps the other I/O thread from working, there is no
reason why bdrv_drain() wouldn't work with cross-AioContext events. The
key is that the root request we're waiting for is in the AioContext
we're polling (which it always is for bdrv_drain()) so that aio_poll()
is woken up in the end.

Add a test case that shows that it works. Remove the comment in
bdrv_drain() that claims otherwise.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-18 15:03:25 +02:00
Cleber Rosa c1cc73f407 Acceptance tests: add Linux kernel boot and console checking test
This test boots a Linux kernel, and checks that the given command
line was effective in two ways:

 * It makes the kernel use the set "console device" as a console
 * The kernel records the command line as expected in the console

Given that way too many error conditions may occur, and detecting the
kernel boot progress status may not be trivial, this test relies on a
timeout to handle unexpected situations.  Also, it's *not* tagged as a
quick test for obvious reasons.

It may be useful, while interactively running/debugging this test, or
tests similar to this one, to show some of the logging channels.
Example:

 $ avocado --show=QMP,console run boot_linux_console.py

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20180530184156.15634-6-crosa@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-15 16:10:11 -03:00
Cleber Rosa 7b1bd11cff Acceptance tests: add quick VNC tests
This patch adds a few simple behavior tests for VNC.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20180530184156.15634-4-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-15 16:10:11 -03:00
Cleber Rosa c3d7e8c90d Add functional/acceptance tests infrastructure
This patch adds the very minimum infrastructure necessary for writing
and running functional/acceptance tests, including:

 * Documentation
 * The avocado_qemu.Test base test class
 * One example tests (version.py)

Additional functionality is expected to be added along the tests that
require them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20180530184156.15634-2-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[ehabkost: fix typo on testing.rst]
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-15 16:10:11 -03:00
Emilio G. Cota 32359d529f qht: return existing entry when qht_insert fails
The meaning of "existing" is now changed to "matches in hash and
ht->cmp result". This is saner than just checking the pointer value.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by:  Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-15 07:42:55 -10:00
Emilio G. Cota 61b8cef1d4 qht: require a default comparison function
qht_lookup now uses the default cmp function. qht_lookup_custom is defined
to retain the old behaviour, that is a cmp function is explicitly provided.

qht_insert will gain use of the default cmp in the next patch.

Note that we move qht_lookup_custom's @func to be the last argument,
which makes the new qht_lookup as simple as possible.
Instead of this (i.e. keeping @func 2nd):
0000000000010750 <qht_lookup>:
   10750:       89 d1                   mov    %edx,%ecx
   10752:       48 89 f2                mov    %rsi,%rdx
   10755:       48 8b 77 08             mov    0x8(%rdi),%rsi
   10759:       e9 22 ff ff ff          jmpq   10680 <qht_lookup_custom>
   1075e:       66 90                   xchg   %ax,%ax

We get:
0000000000010740 <qht_lookup>:
   10740:       48 8b 4f 08             mov    0x8(%rdi),%rcx
   10744:       e9 37 ff ff ff          jmpq   10680 <qht_lookup_custom>
   10749:       0f 1f 80 00 00 00 00    nopl   0x0(%rax)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2018-06-15 07:42:55 -10:00
Kevin Wolf b008326744 block: Remove deprecated -drive option serial
The -drive option serial was deprecated in QEMU 2.10. It's time to
remove it.

Tests need to be updated to set the serial number with -global instead
of using the -drive option.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-06-15 14:49:44 +02:00
Kevin Wolf a7aff6dd10 block: Remove deprecated -drive geometry options
The -drive options cyls, heads, secs and trans were deprecated in
QEMU 2.10. It's time to remove them.

hd-geo-test tested both the old version with geometry options in -drive
and the new one with -device. Therefore the code using -drive doesn't
have to be replaced there, we just need to remove the -drive test cases.
This in turn allows some simplification of the code.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2018-06-15 14:49:44 +02:00
Markus Armbruster 2860b2b2cb block: Fix -blockdev / blockdev-add for empty objects and arrays
-blockdev and blockdev-add silently ignore empty objects and arrays in
their argument.  That's because qmp_blockdev_add() converts the
argument to a flat QDict, and qdict_flatten() eats empty QDict and
QList members.  For instance, we ignore an empty BlockdevOptions
member @cache.  No real harm, as absent means the same as empty there.

Thus, the flaw puts an artificial restriction on the QAPI schema: we
can't have potentially empty objects and arrays within
BlockdevOptions, except when they're optional and "empty" has the same
meaning as "absent".

Our QAPI schema satisfies this restriction (I checked), but it's a
trap for the unwary, and a temptation to employ awkward workarounds
for the wary.  Let's get rid of it.

Change qdict_flatten() and qdict_crumple() to treat empty dictionaries
and lists exactly like scalars.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Markus Armbruster bef96b1549 check-block-qdict: Cover flattening of empty lists and dictionaries
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Markus Armbruster cddec03683 check-block-qdict: Rename qdict_flatten()'s variables for clarity
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Markus Armbruster 0bcc8e5bd8 qobject: Move block-specific qdict code to block-qdict.c
Pure code motion, except for two brace placements and a comment
tweaked to appease checkpatch.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Max Reitz 609f45ea95 block: Add block-specific QDict header
There are numerous QDict functions that have been introduced for and are
used only by the block layer.  Move their declarations into an own
header file to reflect that.

While qdict_extract_subqdict() is in fact used outside of the block
layer (in util/qemu-config.c), it is still a function related very
closely to how the block layer works with nested QDicts, namely by
sometimes flattening them.  Therefore, its declaration is put into this
header as well and util/qemu-config.c includes it with a comment stating
exactly which function it needs.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20180509165530.29561-7-mreitz@redhat.com>
[Copyright note tweaked, superfluous includes dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Eric Blake c6a9d2f6f9 iotests: Add test 221 to catch qemu-img map regression
Although qemu-img creates aligned files (by rounding up), it
must also gracefully handle files that are not sector-aligned.
Test that the bug fixed in the previous patch does not recur.

It's a bit annoying that we can see the (implicit) hole past
the end of the file on to the next sector boundary, so if we
ever reach the point where we report a byte-accurate size rather
than our current behavior of always rounding up, this test will
probably need a slight modification.

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15 14:49:44 +02:00
Peter Maydell 3b2a4d3901 Fixes in syscall numbers,
disable the build of binaries not needed for linux-user,
 update of qemu-binfmt-conf.sh and cleanup around is_error()
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbH6cOAAoJEPMMOL0/L748Qx0P/154ARvl8Rfa1JyxFp87vcWW
 N/WbtxsqKJupdaGSkdZQTOIBQ9Xf51z1EkKaD++oEgN2bYtw0CUQ5MVlfOw2uCoY
 0SnSGrB5qHcEkoN5p+0CwOgQlSoGLNKRZKdkqFIOkXlQUe20DmKqWROB1N5Jpt7x
 ezsaMKKTu+qSd4nlcF2wyJxeW7v6ofJkOsVRZHktxOPBYqbcwV4rQV3Mg+3ZCumt
 6NxvrRvdCLOPMqKJeMSXUWfAE194PiIwtJwwdPG8GbNpV/w/hfse0tAeUZASZoSd
 xBhS6Ne8T19lCpn62ENrtQj5l9zV5Dt1ZVFLMRGFrHDaZvnsr5UKLmr2yDCwvETN
 D023PHWfNaWt+fWoCSVQ8Kr3rB3UgTryEQTnuKiCg6MCj8hbbS90wn+26dlXo5XQ
 dBatlMuLsw6ASPyqPl3alPogS+ies6/sK52EEJU5KQJ3bxcFcETX5bBO1QEDAhJs
 Eo0nDyAjcTIXv7ICQQaVyJnb1QGw0TK7KBKq5wACkvHMqEc++Q8U54NiA8O/nsIu
 XCZlsO0pktJgT0EftS+alnygMAy158eO8u49egVWDDjBdKri6kgu9poljCKZGwTN
 CfAO0+kjfnrCFuCub0q4Ssp641rjeVF20uQ8O8bx4qqWTILD6PtKK5FiYHq2jYnN
 cBYRvu7w2N741o2LXiRD
 =oWiH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-3.0-pull-request' into staging

Fixes in syscall numbers,
disable the build of binaries not needed for linux-user,
update of qemu-binfmt-conf.sh and cleanup around is_error()

# gpg: Signature made Tue 12 Jun 2018 11:57:18 BST
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>"
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>"
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>"
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-3.0-pull-request:
  linux-user/sparc64: Add inotify_rm_watch and tee syscalls
  linux-user/microblaze: Fix typo in accept4 syscall
  linux-user/hppa: Fix typo in mknodat syscall
  linux-user/alpha: Fix epoll syscalls
  qemu-binfmt-conf.sh: ignore the OS/ABI field
  linux-user: disable qemu-bridge-helper and socket_scm_helper build
  linux-user: Use is_error() to avoid warnings and make the code clearer
  linux-user: Export use is_error(), use it to avoid warnings

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-12 16:42:53 +01:00
Peter Maydell 2ab09bf2f9 usb: bug fix collection, doc update.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJbH6QBAAoJEEy22O7T6HE4ZwMP/RHEknxR+tGKYfUxtVP24+ta
 tKMVuewqnDuIjUQsYj/tMvtGEvPd827Cy1X+ocqNq1Gk9aJez+DWWoDlrSXlLyiG
 r4EBMmOasFdmp1D9iQEDUF0pbnbXQv7kqhnM7XbyO+rjNe1rX7HElmRXtcjBcY2I
 QCIpekUBVGzFO97pVzB66gwYpJvmQu2YS2783NaujeS9VEEUFy6RFCFhfNwxhA4W
 C264zFVk68oK1fL+71Y3Kq0WKmetBwfW2ozcBAnqLB5z0hGd9bHJocaLwqgaaQWC
 uaGUJS70Q7iWZ8NwOYvrpK/2ZetjPAeU6ERXeR7tK6yfkjV9UmpPXQ0aqu1OSgEr
 VjuMPEDK0tL8pgcuhhxkJOD6S/HNyCe8y7WLeHKpd21Mv+R2yB1zs2xsWrdMqreZ
 2YXUpgvIaZJBycO8vzmzWwiSNInOGO/3Vu1kpUP4T/rdmFZ1q5f8tGgwMlq/JsU4
 Ce/p0BCcCXvcG+2x5uouffIuTur4DUPn93BCUmjy/TuyggtXVNoQHEVGrMDjiyO/
 9RFVUgJl32mTbIqfVaWkweTch/yXm4XVrlbcTW3VhCCVWyA9O0m+MwyjIoNeXn8L
 yPLF+3ndYgMoTJBFZ20HWEGhZ2+I54fhn/iRW6Nq7qtZ2rOP//K2CqyEp/CuQWis
 vxb796sHEPPUeTCoCiVY
 =sxQ0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/usb-20180612-pull-request' into staging

usb: bug fix collection, doc update.

# gpg: Signature made Tue 12 Jun 2018 11:44:17 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/usb-20180612-pull-request:
  usb-mtp: Return error on suspicious TYPE_DATA packet from initiator
  usb-hcd-xhci-test: add a test for ccid hotplug
  usb-ccid: fix bus leak
  object: fix OBJ_PROP_LINK_UNREF_ON_RELEASE ambivalence
  bus: do not unref the added child bus on realize
  usb/dev-mtp: Fix use of uninitialized values
  usb: correctly handle Zero Length Packets
  usb: update docs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-12 15:34:34 +01:00
Peter Maydell 59f3a1c6ac pc: fixes
A couple of fixes to acpi and nvdimm.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJbHsufAAoJECgfDbjSjVRp7rUH/ivy7ZOfJU98VemRJK325KJ2
 AzsJgn3jKBvlaq4WtsQdspi+eTi9p0kupFJqFxRdpDgPnYjjegDGOjY0RrnhPL19
 lXpwWixZvWAL5zZ60p8RYZxYP9AeFUDQ8wX091uB8phQfBdTUwxCE88bznv2ZUbz
 He3rEMJpMZIorUe8P+RVrUNWHcIdvCZkqYhxt0nZR4YkeaANVCngXGaexBzxdRXR
 oT3cKsmshFPMV+LUr3/os3I5YiOSLCWflTCgnlhhW0rKPbDWFJOtqF78bjLJrR/p
 9SuqW0vmB3JcXXsSWUFvuBo4K0hXgNZEI2hrFVxhkXtw5Ij35ufCCizhmalAaVk=
 =m9VP
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

pc: fixes

A couple of fixes to acpi and nvdimm.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Mon 11 Jun 2018 20:21:03 BST
# gpg:                using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>"
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
  nvdimm: make persistence option symbolic
  hw/i386: Update SSDT table used by "make check"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-12 12:42:15 +01:00
Peter Maydell 5eca450b2e Python queue, 2018-06-11
* Make code compatible with Python 3 using 'futurize --stage1'
 * Require Python >= 2.7 and remove Python 2.6 compatibility
   modules
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJbHrRGAAoJECgHk2+YTcWmeV8P/ReYBQvv4QxFrGgFJs0unWR3
 OLU/zZ3Uaf1U8Q/oJ/pLSYK+XFWJYC9utXApTpBr4Xaxd1eavHl/EpjS/iHaL8VB
 HguuvA2SdlU9+5ZJMk8IdG1Hft/cYLiLNCMJRQzrAL/CL/ToZOQyOPU0NNmyk0S5
 J3cug4VCDJMs+unKPHL/ThxXoWS1sRq7DZyYeaFBqOuDBLf7ZXSGOWPTC4KKwGbl
 N7sXxhDUbXFnY6HYpdShB8gACeI4P0WN0UCtGkvMwjF17DbHMohgoaXkzUks4wCI
 gJF7m0tFY9rR2w/RPiHeZmPcuWsxFxKjLCE93qrxNSoq7DWQyS6UprIf1CaW6rgz
 rPJAbxjHedKhW3M4gW9utpqYlSweTg7kQP1jLStZATvCnRgjQ5BtDcBJFrXa0zdb
 6+py9MlvE0ysiXPl6vmbcn8VzDq+TnmyXMj2zrpcXMLJphnUcJlD1EgbXMTqXRY5
 HUN61ZWyc6nj3go52QaC1hupcCSf4macgoXWh8BORuHQZcL5xom4rOJL4MwhQIu/
 5crYp/Xa9SA1FU7c0we8Np53feSU+5PGY5VyHJJyRoVSEriJnvvN9kCBpziln+Y/
 v0CIFt7ohVB1OzsdNvFLSsRrlPd1RReiGOE9ixgcb37mIKBPWc2u73Mws37lxxiC
 BO64VpDUXdwpyIrHXxTU
 =d1wy
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-06-11

* Make code compatible with Python 3 using 'futurize --stage1'
* Require Python >= 2.7 and remove Python 2.6 compatibility
  modules

# gpg: Signature made Mon 11 Jun 2018 18:41:26 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
  python: Remove scripts/ordereddict.py
  python: Remove scripts/argparse.py
  configure: Require Python 2.7 or newer
  python: futurize -f lib2to3.fixes.fix_numliterals
  python: futurize -f lib2to3.fixes.fix_except
  python: futurize -f lib2to3.fixes.fix_renames
  python: futurize -f lib2to3.fixes.fix_tuple_params
  python: futurize -f lib2to3.fixes.fix_reduce
  python: futurize -f lib2to3.fixes.fix_standarderror
  python: futurize -f lib2to3.fixes.fix_has_key
  python: futurize -f libfuturize.fixes.fix_next_call
  python: futurize -f libfuturize.fixes.fix_absolute_import
  python: futurize -f libfuturize.fixes.fix_print_with_import

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-12 11:56:21 +01:00
Marc-André Lureau 1a3ff20e67 usb-hcd-xhci-test: add a test for ccid hotplug
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20180531195119.22021-5-marcandre.lureau@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2018-06-12 12:07:30 +02:00
Ross Zwisler 11c39b5cd9 nvdimm: make persistence option symbolic
Replace the "nvdimm-cap" option which took numeric arguments such as "2"
with a more user friendly "nvdimm-persistence" option which takes symbolic
arguments "cpu" or "mem-ctrl".

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Michael S. Tsirkin <mst@redhat.com>
Suggested-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2018-06-11 22:19:57 +03:00
Ross Zwisler 0f3ef1c062 hw/i386: Update SSDT table used by "make check"
This commit:

commit aa78a16d86 ("hw/i386: Rename 2.13 machine types to 3.0")

updated the name used to create the q35 machine, which in turn changed the
SSDT table which is generated when we run "make check":

  acpi-test: Warning! SSDT mismatch. Actual [asl:/tmp/asl-QZDWJZ.dsl,
  aml:/tmp/aml-T8JYJZ], Expected [asl:/tmp/asl-DTWVJZ.dsl,
  aml:tests/acpi-test-data/q35/SSDT.dimmpxm].

Here's the only difference, aside from the checksum:

  <     Name (MEMA, 0x07FFF000)
  ---
  >     Name (MEMA, 0x07FFE000)

Update the binary table that we compare against so it reflects this name
change.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Cornelia Huck <cohuck@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Fixes: commit aa78a16d86 ("hw/i386: Rename 2.13 machine types to 3.0")
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2018-06-11 22:19:57 +03:00
Max Reitz c50abd175a iotests: Add case for a corrupted inactive image
Reviewed-by: John Snow <jsnow@redhat.com>
Tested-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180606193702.7113-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Thomas Huth 46e8d272ba qemu-img: Remove deprecated -s snapshot_id_or_name option
It has been marked as deprecated since QEMU v2.0 already, so it
is time now to finally remove it.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 1528288551-31641-1-git-send-email-thuth@redhat.com
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz 83f90b535a iotests: Fix 219's timing
219 has two issues that may lead to sporadic failure, both of which are
the result of issuing query-jobs too early after a job has been
modified.  This can then lead to different results based on whether the
modification has taken effect already or not.

First, query-jobs is issued right after the job has been created.
Besides its current progress possibly being in any random state (which
has already been taken care of), its total progress too is basically
arbitrary, because the job may not yet have been able to determine it.
This patch addresses this by just filtering the total progress, like
what has been done for the current progress already.  However, for more
clarity, the filtering is changed to replace the values by a string
'FILTERED' instead of deleting them.

Secondly, query-jobs is issued right after a job has been resumed.  The
job may or may not yet have had the time to actually perform any I/O,
and thus its current progress may or may not have advanced.  To make
sure it has indeed advanced (which is what the reference output already
assumes), keep querying it until it has.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180606190628.8170-1-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Vladimir Sementsov-Ogievskiy c1bac161bb iotests: improve pause_job
It's possible, that job was finished during waiting. In this case we
will see error message "Timeout waiting for job to pause" which is not
very informative. So, let's check during waiting iteration that the job
exists.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180601115923.17159-1-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz 0682854f89 iotests: Test post-backing convert target behavior
This adds a test case to 122 for what happens when you convert to a
target with a backing file that is shorter than the target, and the
image format does not support efficient zero writes (as is the case with
qcow2 v2).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180501165750.19242-3-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz 28036a7f70 iotests: Add test for rebasing with relative paths
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509182002.8044-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz e4ca4e981a iotests: Let 216 make use of qemu-io's exit code
As a showcase of how you can use qemu-io's exit code to determine
success or failure (same for qemu-img), this test is changed to use
qemu_io_silent() instead of qemu_io(), and to assert the exit code
instead of logging the filtered result.

One real advantage of this is that in case of an error, you get a
backtrace that helps you locate the issue in the test file quickly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509194302.21585-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz 745f2bf4a5 iotests.py: Add qemu_io_silent
With qemu-io now returning a useful exit code, some tests may find it
sufficient to just query that instead of logging (and filtering) the
whole output.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509194302.21585-5-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz b41ad73a3b iotests: Repairing error during snapshot deletion
This adds a test for an I/O error during snapshot deletion, and maybe
more importantly, for how to repair the resulting image.  If the
snapshot has been deleted before the error occurs, the only negative
result will be leaked clusters -- and those should be repairable with
qemu-img check -r leaks.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509200059.31125-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz dee6ddd8a6 iotests: Rework 113
This test case has been broken since 398e6ad014 (roughly half a
year).  qemu-img amend requires its output image to be R/W, so it opens
it as such; the node is then turned into an read-only node automatically
which is now accompanied by a warning, however.  This warning has not
been part of the reference output.

For one thing, this warning shows that we cannot keep the test case as
it is.  We would need a format that has no create_opts but that does
have write support -- we do not have such a format, though.

Another thing is that qemu now actually checks whether an image format
supports amendment instead of whether it has create_opts (since the
former always implies the latter).  So we can now use any format that
does not support amendment (even if it supports creation) and thus test
the same code path.

The reason nobody has noticed the breakage until now of course is the
fact that nobody runs the iotests for nbd+bochs.  There actually was
never any reason to set the protocol to "nbd" but because that was
technically correct; functionally it made no difference.  So that is the
first thing we are going to change: Make the protocol "file" instead so
that people might actually notice breakage here.

Secondly, now that bochs no longer works for the amend test case, we
have to change the format there anyway.  Set let us just bend the truth
a bit, declare this test a raw test.  In fact, that does not even
concern the bochs test cases, other than the output now reading 'bochs'
instead of 'IMGFMT'.

So with this test now being a raw test, we can rework the amend test
case to use raw instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20180509210023.20283-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz e53995eb19 iotests: Test help option for unsupporting formats
This adds test cases to 082 for qemu-img create/convert/amend "-o help"
on formats that do not support creation or amendment, respectively.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509210023.20283-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz 5164135142 qemu-img: Add print_amend_option_help()
The more generic print_block_option_help() function is not really
suitable for qemu-img amend, for a couple of reasons:
(1) We do not need to append the protocol-level options, as amendment
    happens only on one node and does not descend downwards to its
    children.
(2) print_block_option_help() says those options are "supported".  For
    option amendment, we do not really know that.  So this new function
    explicitly says that those options are the creation options, and not
    all of them may be supported.
(3) If the driver does not support option amendment, we should not print
    anything (except for an error message that amendment is not
    supported).

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1537956
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509210023.20283-5-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz d1402b5026 block: Add Error parameter to bdrv_amend_options
Looking at the qcow2 code that is riddled with error_report() calls,
this is really how it should have been from the start.

Along the way, turn the target_version/current_version comparisons at
the beginning of qcow2_downgrade() into assertions (the caller has to
make sure these conditions are met), and rephrase the error message on
using compat=1.1 to get refcount widths other than 16 bits.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180509210023.20283-3-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Max Reitz f45b638f9f iotests: Add creation test to 153
This patch adds a test case to 153 which tries to overwrite an image
(using qemu-img create) while it is in use.  Without the original user
explicitly sharing the necessary permissions (writing and truncation),
this should not be allowed.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 20180509215336.31304-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11 16:18:45 +02:00
Laurent Vivier 206d1b9c38 linux-user: disable qemu-bridge-helper and socket_scm_helper build
linux-user targets don't need them, and if we ask to build statically
linked binaries, some static libraries they need are not available.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180605160958.5434-1-laurent@vivier.eu>
2018-06-11 14:40:11 +02:00
Eduardo Habkost c788341244 python: Remove scripts/ordereddict.py
Python 2.7 (the minimum Python version we require) provides
collections.OrderedDict on the standard library, so we don't need
to carry our own implementation.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608175252.25110-3-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 16:40:54 -03:00
Eduardo Habkost 4803c5cde8 python: futurize -f lib2to3.fixes.fix_numliterals
Convert octal literals into the new syntax.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f lib2to3.fixes.fix_numliterals $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-11-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost cd67daaa93 python: futurize -f lib2to3.fixes.fix_renames
Change sys.maxint to sys.maxsize.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f lib2to3.fixes.fix_renames $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-9-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost 8d2860cb91 python: futurize -f lib2to3.fixes.fix_reduce
Handle the move of reduce() to functools.reduce().

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f lib2to3.fixes.fix_reduce $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-7-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost d7a4228ebb python: futurize -f lib2to3.fixes.fix_has_key
Change "dict.has_key(key)" to "key in dict"

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f lib2to3.fixes.fix_has_key $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-5-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost d24d523c14 python: futurize -f libfuturize.fixes.fix_next_call
Change obj.next() calls to next(obj).

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f libfuturize.fixes.fix_next_call $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-4-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost 068cf7a44c python: futurize -f libfuturize.fixes.fix_absolute_import
Make implicit relative imports explicit and add "from __future__ import
absolute_import" at the top of each relevant module.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f libfuturize.fixes.fix_absolute_import $py

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-3-ehabkost@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost f03868bd56 python: futurize -f libfuturize.fixes.fix_print_with_import
Change all Python code to use print as a function.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f libfuturize.fixes.fix_print_with_import $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-2-ehabkost@redhat.com>
[ehabkost: fixup tests/docker/docker.py]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
John Snow 956556e131 ahci: move PIO Setup FIS before transfer, fix it for ATAPI commands
The PIO Setup FIS is written in the PIO:Entry state, which comes before
the ATA and ATAPI data transfer states.  As a result, the PIO Setup FIS
interrupt is now raised before DMA ends for ATAPI commands, and tests have
to be adjusted.

This is also hinted by the description of the command header in the AHCI
specification, where the "A" bit is described as

    When ‘1’, indicates that a PIO setup FIS shall be sent by the device
    indicating a transfer for the ATAPI command.

and also by the description of the ACMD (ATAPI command region):

    The ATAPI command must be either 12 or 16 bytes in length. The length
    transmitted by the HBA is determined by the PIO setup FIS that is sent
    by the device requesting the ATAPI command.

QEMU, which conflates the "generator" and the "receiver" of the FIS into
one device, always uses ATAPI_PACKET_SIZE, aka 12, for the length.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20180606190955.20845-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-08 13:36:11 -04:00
John Snow 27e4648ce9 libqos/ahci: track sector size
It's not always 512, and it does wind up mattering for PIO tranfers,
because this means DRQ blocks are four times as big for ATAPI.
Replace an instance of 2048 with the correct define, too.

This patch by itself winds changing no behavior. fis->count is ignored
for CMD_PACKET, and sect_count only gets used in non-ATAPI cases.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20180606190955.20845-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-08 13:17:59 -04:00
Thomas Huth 49d43a5978 tests/cdrom-test: Test that -cdrom parameter is working
Commit 1454509726 recently broke the "-cdrom" parameter
on a couple of boards without us noticing it immediately. Thus let's
add a test which checks that "-cdrom" can at least be used to start
QEMU with certain machine types.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Hervé Poussineau <hpoussin@reactos.org>
Acked-By: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-08 13:17:39 -04:00
Thomas Huth 42f455054f tests/cdrom-test: Test booting from CD-ROM ISO image file
We already have the code for a boot file in tests/boot-sector.c,
so if the genisoimage program is available, we can easily create
a bootable CD ISO image that we can use for testing whether our
CD-ROM emulation and the BIOS CD-ROM boot works correctly.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Hervé Poussineau <hpoussin@reactos.org>
Acked-By: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-08 13:17:39 -04:00
Thomas Huth b1dd6d2d27 tests/boot-sector: Add magic bytes to s390x boot code header
We're going to use the s390x boot code for testing CD-ROM booting.
But the ISO loader of the s390-ccw bios is a little bit more picky
than the network loader and expects some magic bytes in the header
of the file (see linux_s390_magic in pc-bios/s390-ccw/bootmap.c), so
we've got to add them in our boot code here, too.

Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Hervé Poussineau <hpoussin@reactos.org>
Acked-By: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2018-06-08 13:17:39 -04:00
Cédric Le Goater 5141d4158c misc: add pca9552 LED blinker model
Specs are available here :

    https://www.nxp.com/docs/en/application-note/AN264.pdf

This is a simple model supporting the basic registers for led and GPIO
mode. The device also supports two blinking rates but not the model
yet.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20180530064049.27976-7-clg@kaod.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-08 13:15:32 +01:00
Stefan Berger 70663851ed test: Add swtpm migration test for the TPM TIS interface
Add a test case for testing swtpm migration with the TPM TIS
interface.

Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-06 15:44:12 -04:00
Stefan Berger ea71a33691 test: Pass TPM interface model to functions creating command line
Pass the TPM interface model, such as 'tpm-crb', through to the functions
that create the command line for QEMU.

Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-06 15:44:07 -04:00
Stefan Berger 2b4ccb87ec test: Move common TPM test functions to tpm-tests.c
Move common TPM test functions from tpm-crb-swtpm-test.c to tpm-tests.c
so that for example test cases with the TPM TIS interface can use the
same code. Prefix all funcions with 'tpm_test_'.

Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-06 15:44:01 -04:00
Stefan Berger b1e4b7c6b2 test: Move reusable code from tpm-crb-swtpm-test.c to tpm-util.c
Move code we can reuse from tpm-crb-swtpm-test.c into tpm-util.c
and prefix functions with 'tpm_util_'.

Remove some unnecessary #include's.

Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-06 15:43:54 -04:00
Philippe Mathieu-Daudé 36dc5fedab docker: add debian/tricore image
TriCore binutils is built from Bastian Koppelmann repository.

Note: There is no TriCore compiler in this image (only assembler/linker).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: base of Debian9, add to Makefile.include]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:43 +01:00
Paolo Bonzini cabd358407 docker: test-mingw: use SDL2 and GTK+3
Do not test the deprecated API versions.  debian-win32-cross and debian-win64-cross
are already using SDL2 (they do not cover GTK+ at all).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[AJB: fix merge conflicts]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:42 +01:00
Fam Zheng 8bce19d3a6 docker: Add fedora-i386-cross image
It has some basic *-devel.i686 packages to be used with "gcc -m32" as a
32 bit cross build environment.

Signed-off-by: Fam Zheng <famz@redhat.com>
[AJB: add glibc-static]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-05 16:25:42 +01:00
Fam Zheng 208ecb3e1a Makefile: Rename TARGET_DIRS to TARGET_LIST
To be more accurate on its purpose and make code that looks for a certain
target out of this variable more readable.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:42 +01:00
Alex Bennée 15df9d3783 docker: add "probe" command for configure
This is a helper function for the configure script. It replies yes,
sudo or no to inform the user if non-interactive docker support is
available. We trap the Exception to fail gracefully.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
2018-06-05 16:25:42 +01:00
Peter Maydell 0d514fa234 Pull request
* Copy offloading for qemu-img convert (iSCSI, raw, and qcow2)
 
    If the underlying storage supports copy offloading, qemu-img convert will
    use it instead of performing reads and writes.  This avoids data transfers
    and thus frees up storage bandwidth for other purposes.  SCSI EXTENDED COPY
    and Linux copy_file_range(2) are used to implement this optimization.
 
  * Drop spurious "WARNING: I\/O thread spun for 1000 iterations" warning
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJbFSBoAAoJEJykq7OBq3PISpEIAIcMao4/rzinAWXzS+ncK9LO
 6FtRVpgutpHaWX2ayySaz5n2CdR3cNMrpCI7sjY2Kw0lrdkqxPgl5n0SWD+VCl4W
 7+JLz/uF0iUV8X+99e7WGAjZbm9LSlxgn5AQKfrrwyPf0ZfzoYQ5nBMcQ6xjEeQP
 48j2WqJqN9/u8RBD07o11yn0+CE5g56/f12xVjR5ASVodzsAmcZ2OQRMQbM01isU
 1mBekJQkDxJkt5l13Rql8+t+vWz8/9BEW2c/eIDKvoayMqYJpdfKv4DqLloIuHnc
 3RkquA0zUuKtl7xEnEkH/We7fi4QPGW/vyBN7ychS/zKzZFQrXmwqrAuFSw3dKU=
 =vZp+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

 * Copy offloading for qemu-img convert (iSCSI, raw, and qcow2)

   If the underlying storage supports copy offloading, qemu-img convert will
   use it instead of performing reads and writes.  This avoids data transfers
   and thus frees up storage bandwidth for other purposes.  SCSI EXTENDED COPY
   and Linux copy_file_range(2) are used to implement this optimization.

 * Drop spurious "WARNING: I\/O thread spun for 1000 iterations" warning

# gpg: Signature made Mon 04 Jun 2018 12:20:08 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  main-loop: drop spin_counter
  qemu-img: Convert with copy offloading
  block-backend: Add blk_co_copy_range
  iscsi: Implement copy offloading
  iscsi: Create and use iscsi_co_wait_for_task
  iscsi: Query and save device designator when opening
  file-posix: Implement bdrv_co_copy_range
  qcow2: Implement copy offloading
  raw: Implement copy offloading
  raw: Check byte range uniformly
  block: Introduce API for copy offloading

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-04 18:34:04 +01:00
Peter Maydell cfe672510d -----BEGIN PGP SIGNATURE-----
iQFEBAABCAAuFiEEUAN8t5cGD3bwIa1WyjViTGqRccYFAlsU5acQHGZhbXpAcmVk
 aGF0LmNvbQAKCRDKNWJMapFxxvYvB/wMK/YRDSLdMkz7D8RfCUWKTIjT60L+Cb1W
 vVSOFHQiU5Igl9F9XKzhIR8PDtpwV4ptbACZ+FZmQkaC9WeAcfgXFvTcXPd8XKTW
 jXk+rRlriFw4iAQ8rW4/b4KNRlzGtirED5KEuwt5D12JNw9LEzsdERjvITXpqxaj
 mh54qdjgyoK1oG+fiDhkxEBaf/H8PU81lgx0Vi3tEli2MMBtdD03TU0Ik0eO1k8e
 5mKGJD0wfer1rboGDoOUv/yTxm4gaB4ljHRJfJh7ZcipMYJuffxVyQJbFdsJkteW
 Wa8MYe1b2B5DPSl71n9xRBfIT0/f/a8hzDzOGbAJrGsEDvG6+Ho+
 =A2XO
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging

# gpg: Signature made Mon 04 Jun 2018 08:09:27 BST
# gpg:                using RSA key CA35624C6A9171C6
# gpg: Good signature from "Fam Zheng <famz@redhat.com>"
# Primary key fingerprint: 5003 7CB7 9706 0F76 F021  AD56 CA35 624C 6A91 71C6

* remotes/famz/tags/docker-pull-request:
  docker: Update fedora image to 28
  tests/docker/Makefile.include: handle empty TARGET_LIST

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-04 14:24:16 +01:00
Peter Maydell f67c9b693a acpi, vhost, misc: fixes, features
vDPA support, fix to vhost blk RO bit handling, some include path
 cleanups, NFIT ACPI table.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJbEXNvAAoJECgfDbjSjVRpc8gH/R8xrcFrV+k9wwbgYcOcGb6Y
 LWjseE31pqJcxRV80vLOdzYEuLStZQKQQY7xBDMlA5vdyvZxIA6FLO2IsiJSbFAk
 EK8pclwhpwQAahr8BfzenabohBv2UO7zu5+dqSvuJCiMWF3jGtPAIMxInfjXaOZY
 odc1zY2D2EgsC7wZZ1hfraRbISBOiRaez9BoGDKPOyBY9G1ASEgxJgleFgoBLfsK
 a1XU+fDM6hAVdxftfkTm0nibyf7PWPDyzqghLqjR9WXLvZP3Cqud4p8N29mY51pR
 KSTjA4FYk6Z9EVMltyBHfdJs6RQzglKjxcNGdlrvacDfyFi79fGdiosVllrjfJM=
 =3+V0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

acpi, vhost, misc: fixes, features

vDPA support, fix to vhost blk RO bit handling, some include path
cleanups, NFIT ACPI table.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Fri 01 Jun 2018 17:25:19 BST
# gpg:                using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>"
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream: (31 commits)
  vhost-blk: turn on pre-defined RO feature bit
  ACPI testing: test NFIT platform capabilities
  nvdimm, acpi: support NFIT platform capabilities
  tests/.gitignore: add entry for generated file
  arch_init: sort architectures
  ui: use local path for local headers
  qga: use local path for local headers
  colo: use local path for local headers
  migration: use local path for local headers
  usb: use local path for local headers
  sd: fix up include
  vhost-scsi: drop an unused include
  ppc: use local path for local headers
  rocker: drop an unused include
  e1000e: use local path for local headers
  ioapic: fix up includes
  ide: use local path for local headers
  display: use local path for local headers
  trace: use local path for local headers
  migration: drop an unused include
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-04 10:15:16 +01:00
Fam Zheng 63dba573fa docker: Update fedora image to 28
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180601023557.9770-1-famz@redhat.com>
2018-06-04 14:39:18 +08:00
Alex Bennée 5609c5127d tests/docker/Makefile.include: handle empty TARGET_LIST
If the user doesn't specify a TARGET_LIST they get the current
configuration but with spaces and hilarity ensues. This adds some make
magic to turn the TARGET_LIST back into a comma separated list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20180521103504.26432-1-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-06-04 14:39:18 +08:00
Peter Maydell afd76ffba9 * Linux header upgrade (Peter)
* firmware.json definition (Laszlo)
 * IPMI migration fix (Corey)
 * QOM improvements (Alexey, Philippe, me)
 * Memory API cleanups (Jay, me, Tristan, Peter)
 * WHPX fixes and improvements (Lucian)
 * Chardev fixes (Marc-André)
 * IOMMU documentation improvements (Peter)
 * Coverity fixes (Peter, Philippe)
 * Include cleanup (Philippe)
 * -clock deprecation (Thomas)
 * Disable -sandbox unless CONFIG_SECCOMP (Yi Min Zhao)
 * Configurability improvements (me)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAlsRd2UUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroPG8Qf+M85E8xAQ/bhs90tAymuXkUUsTIFF
 uI76K8eM0K3b2B+vGckxh1gyN5O3GQaMEDL7vITfqbX+EOH5U2lv8V9JRzf2YvbG
 Zahjd4pOCYzR0b9JENA1r5U/J8RntNrBNXlKmGTaXOaw9VCXlZyvgVd9CE3z/e2M
 0jSXMBdF4LB3UzECI24Va8ejJxdSiJcqXA2j3J+pJFxI698i+Z5eBBKnRdo5TVe5
 jl0TYEsbS6CLwhmbLXmt3Qhq+ocZn7YH9X3HjkHEdqDUeYWyT9jwUpa7OHFrIEKC
 ikWm9er4YDzG/vOC0dqwKbShFzuTpTJuMz5Mj4v8JjM/iQQFrp4afjcW2g==
 =RS/B
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Linux header upgrade (Peter)
* firmware.json definition (Laszlo)
* IPMI migration fix (Corey)
* QOM improvements (Alexey, Philippe, me)
* Memory API cleanups (Jay, me, Tristan, Peter)
* WHPX fixes and improvements (Lucian)
* Chardev fixes (Marc-André)
* IOMMU documentation improvements (Peter)
* Coverity fixes (Peter, Philippe)
* Include cleanup (Philippe)
* -clock deprecation (Thomas)
* Disable -sandbox unless CONFIG_SECCOMP (Yi Min Zhao)
* Configurability improvements (me)

# gpg: Signature made Fri 01 Jun 2018 17:42:13 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (56 commits)
  hw: make virtio devices configurable via default-configs/
  hw: allow compiling out SCSI
  memory: Make operations using MemoryRegionIoeventfd struct pass by pointer.
  char: Remove unwanted crlf conversion
  qdev: Remove DeviceClass::init() and ::exit()
  qdev: Simplify the SysBusDeviceClass::init path
  hw/i2c: Use DeviceClass::realize instead of I2CSlaveClass::init
  hw/i2c/smbus: Use DeviceClass::realize instead of SMBusDeviceClass::init
  target/i386/kvm.c: Remove compatibility shim for KVM_HINTS_REALTIME
  Update Linux headers to 4.17-rc6
  target/i386/kvm.c: Handle renaming of KVM_HINTS_DEDICATED
  scripts/update-linux-headers: Handle kernel license no longer being one file
  scripts/update-linux-headers: Handle __aligned_u64
  virtio-gpu-3d: Define VIRTIO_GPU_CAPSET_VIRGL2 elsewhere
  gdbstub: Prevent fd leakage
  docs/interop: add "firmware.json"
  ipmi: Use proper struct reference for KCS vmstate
  vmstate: Add a VSTRUCT type
  tcg: remove softfloat from --disable-tcg builds
  qemu-options: Mark the non-functional -clock option as deprecated
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-06-01 18:24:16 +01:00
Ross Zwisler cab27afa10 ACPI testing: test NFIT platform capabilities
Add testing for the newly added NFIT Platform Capabilities Structure.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2018-06-01 19:20:38 +03:00
Ross Zwisler 2e0c56cdde tests/.gitignore: add entry for generated file
After a "make check" we end up with the following:

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	tests/test-block-backend

nothing added to commit but untracked files present (use "git add" to track)

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Fixes: commit ad0df3e0fd ("block: test blk_aio_flush() with blk->root == NULL")
Cc: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-06-01 19:20:38 +03:00
Stefan Hajnoczi 21891a5a30 main-loop: drop spin_counter
Commit d759c951f3 ("replay: push
replay_mutex_lock up the call tree") removed the !timeout lock
optimization in the main loop.

The idea of the optimization was to avoid ping-pongs between threads by
keeping the Big QEMU Lock held across non-blocking (!timeout) main loop
iterations.

A warning is printed when the main loop spins without releasing BQL for
long periods of time.  These warnings were supposed to aid debugging but
in practice they just alarm users.  They are considered noise because
the cause of spinning is not shown and is hard to find.

Now that the lock optimization has been removed, there is no danger of
hogging the BQL.  Drop the spin counter and the infamous warning.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-06-01 16:01:29 +01:00
Alex Bennée 67659ab1eb docker: update Travis docker image
This is still poorly documented by Travis but according to:

  https://docs.travis-ci.com/user/common-build-problems/#Running-a-Container-Based-Docker-Image-Locally

their reference images are now hosted on Docker Hub. So we update the
FROM line to refer to the new default image. We also need a few
additional tweaks:

  - re-enable deb-src lines for our build-dep install
  - add explicit PATH definition for tools
  - force the build USER to be Travis
  - add clang to FEATURES for our test-clang machinery

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-01 15:00:54 +01:00
Philippe Mathieu-Daudé bcaf457786 docker: do not display deprecated images in 'make docker' help
the 'debian' base image is deprecated since 3e11974988

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Philippe Mathieu-Daudé 1a06f5c910 docker: sort images list displayed by 'make docker'
we can now directly see different version sort consecutively.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Marc-André Lureau eeaa671505 mux: fix ctrl-a b again
Commit fb5e19d2e1 originally fixed the
regression, but was inadvertently broken again in merge commit
2d6752d38d.

Fixes:
https://bugs.launchpad.net/qemu/+bug/1654137

Cc: qemu-stable@nongnu.org
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180515152500.19460-3-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-06-01 15:13:46 +02:00
Peter Maydell a3ac12fba0 NUMA queue, 2018-05-30
* New command-line option: --preconfig
   This option allows pausing QEMU and allow the configuration
   using QMP commands before running board initialization code.
 * New QMP set-numa-node, now made possible because of --preconfig
 * Small update on -numa error messages
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJbDy2jAAoJECgHk2+YTcWm8OgP/As0judZ7JwT5F5fR4nxtQPr
 EODeEI+e/IahGH58Bafx24Fvxy0HZIgwqBDnPgclA8d3ebwmDQIejgk5/00zq2xv
 cktRpzXGSBGJCVNctVz8xqN91m0lPtVgeRvXUJPn3hthXRSLO4p0vbyOW8g/C+O2
 +dEcGqifAQatyxs9gYmmWoUia8zle/v1bd2v/x/DwiFW9cd47yDr3+ChHh6+nx0W
 uh2zymD+ykVeNJ9WSxc3k8zTzQnuxbDK1fNbZsztk9KQDMWG3+u7KRNoJ86/7hKL
 RMUfRCUMBgemuZsrFF8wSsha27e9VgCw4oR8dQ5AnTgMjK6nch/839XuFWh7j5qu
 ntS4vt1v6IczdflKXX7IUILAgvVffLh2SCGWlKq9Q1eKo+CZTCea+iSbb2QobAE+
 9TyuqXVKCsGKLoqkbK39d1UjZFhDQJhMsyuOHreKEINmJRmLI5qrNbEaaysb+mMB
 DnFE1NiWjE+6X4w9U1l8lnKNUakOPNDG8cbgtuiEKDC9e3OZeiS8yyPLaQajELSR
 Ig7N2GxrLMSfS3LNMSwkACILzVeu03aFoHyCPrpUqD8mpxidN0mmFVup8dawtkJ1
 A5dHrKDmzlOT6DivJOgT8/Os/wMo3ErPcvyu69rTUO9PKSbwcFPwi6bMMDzgb4xO
 kAles9X7HrHxSeJiggRW
 =RHIj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/numa-next-pull-request' into staging

NUMA queue, 2018-05-30

* New command-line option: --preconfig
  This option allows pausing QEMU and allow the configuration
  using QMP commands before running board initialization code.
* New QMP set-numa-node, now made possible because of --preconfig
* Small update on -numa error messages

# gpg: Signature made Thu 31 May 2018 00:02:59 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/numa-next-pull-request:
  tests: functional tests for QMP command set-numa-node
  qmp: add set-numa-node command
  qmp: permit query-hotpluggable-cpus in preconfig state
  tests: extend qmp test with preconfig checks
  cli: add --preconfig option
  tests: qapi-schema tests for allow-preconfig
  qapi: introduce new cmd option "allow-preconfig"
  hmp: disable monitor in preconfig state
  qapi: introduce preconfig runstate
  numa: split out NumaOptions parsing into set_numa_options()
  numa: postpone options post-processing till machine_run_board_init()
  numa: clarify error message when node index is out of range in -numa dist, ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-31 11:12:36 +01:00
Igor Mammedov c35665e1ee tests: functional tests for QMP command set-numa-node
* start QEMU with 2 unmapped cpus,
 * while in preconfig state
    * add 2 numa nodes
    * assign cpus to them
 * exit preconfig and in running state check that cpus
   are mapped correctly.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <1526556607-268163-1-git-send-email-imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30 13:19:14 -03:00
Igor Mammedov fb1e58f72b tests: extend qmp test with preconfig checks
Add permission checks for commands at 'preconfig' stage.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <1526556524-267991-1-git-send-email-imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30 13:19:14 -03:00
Igor Mammedov 7b13f2c27a tests: qapi-schema tests for allow-preconfig
use new allow-preconfig parameter in tests and make sure that
the QAPISchema can parse allow-preconfig correctly

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1526058959-41425-1-git-send-email-imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30 13:19:14 -03:00
Igor Mammedov d6fe3d02e9 qapi: introduce new cmd option "allow-preconfig"
New option will be used to allow commands, which are prepared/need
to run, during preconfig state. Other commands that should be able
to run in preconfig state, should be amended to not expect machine
in initialized state or deal with it.

For compatibility reasons, commands that don't use new flag
'allow-preconfig' explicitly are not permitted to run in
preconfig state but allowed in all other states like they used
to be.

Within this patch allow following commands in preconfig state:
   qmp_capabilities
   query-qmp-schema
   query-commands
   query-command-line-options
   query-status
   exit-preconfig
to allow qmp connection, basic introspection and moving to the next
state.

PS:
set-numa-node and query-hotpluggable-cpus will be enabled later in
a separate patches.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <1526057503-39287-1-git-send-email-imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[ehabkost: Changed "since 2.13" to "since 3.0"]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30 13:19:09 -03:00
Kevin Wolf 3fb588a0f2 block/create: Mark blockdev-create stable
We're ready to declare the blockdev-create job stable. This renames the
corresponding QMP command from x-blockdev-create to blockdev-create.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 0c46a69a5e qemu-iotests: Rewrite 213 for blockdev-create job
This rewrites the test case 213 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

All of the test cases stay the same as before, but in order to be able
to implement proper job handling, the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 2d7abfbeb7 qemu-iotests: Rewrite 212 for blockdev-create job
This rewrites the test case 212 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

All of the test cases stay the same as before, but in order to be able
to implement proper job handling, the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf abbab72cad qemu-iotests: Rewrite 211 for blockdev-create job
This rewrites the test case 211 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

All of the test cases stay the same as before, but in order to be able
to implement proper job handling, the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 5ba141dc6f qemu-iotests: Rewrite 210 for blockdev-create job
This rewrites the test case 210 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

All of the test cases stay the same as before, but in order to be able
to implement proper job handling, the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 00af19359e qemu-iotests: Rewrite 207 for blockdev-create job
This rewrites the test case 207 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

Most of the test cases stay the same as before (the exception being some
improved 'size' options that allow distinguishing which command created
the image), but in order to be able to implement proper job handling,
the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 4de110f8fd qemu-iotests: Rewrite 206 for blockdev-create job
This rewrites the test case 206 to work with the new x-blockdev-create
job rather than the old synchronous version of the command.

All of the test cases stay the same as before, but in order to be able
to implement proper job handling, the test case is rewritten in Python.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:18 +02:00
Kevin Wolf 5a259e868b qemu-iotests: iotests.py helper for non-file protocols
This adds two helper functions that are useful for test cases that make
use of a non-file protocol (specifically ssh).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:16 +02:00
Kevin Wolf fc47d8513b qemu-iotests: Add VM.run_job()
Add an iotests.py function that runs a job and only returns when it is
destroyed. An error is logged when the job failed and job-finalize and
job-dismiss commands are issued if necessary.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:14 +02:00
Kevin Wolf 6b605adec4 qemu-iotests: Add iotests.img_info_log()
This adds a filter function to postprocess 'qemu-img info' input
(similar to what _img_info does), and an img_info_log() function that
calls 'qemu-img info' and logs the filtered output.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:12 +02:00
Kevin Wolf e234398a8e qemu-iotests: Add VM.qmp_log()
This adds a helper function that logs both the QMP request and the
received response before returning it.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-30 13:31:12 +02:00
Kevin Wolf 5ad1dbf76a qemu-iotests: Add VM.get_qmp_events_filtered()
This adds a helper function that returns a list of QMP events that are
already filtered through filter_qmp_event().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:09 +02:00
Kevin Wolf e5ab4347f9 block/create: Make x-blockdev-create a job
This changes the x-blockdev-create QMP command so that it doesn't block
the monitor and the main loop any more, but starts a background job that
performs the image creation.

The basic job as implemented here is all that is necessary to make image
creation asynchronous and to provide a QMP interface that can be marked
stable, but it still lacks a few features that jobs usually provide: The
job will ignore pause commands and it doesn't publish more than very
basic progress yet (total-progress is 1 and current-progress advances
from 0 to 1 when the driver callbacks returns). These features can be
added later without breaking compatibility.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:07 +02:00
Kevin Wolf 1266c9b9f5 job: Add error message for failing jobs
So far we relied on job->ret and strerror() to produce an error message
for failed jobs. Not surprisingly, this tends to result in completely
useless messages.

This adds a Job.error field that can contain an error string for a
failing job, and a parameter to job_completed() that sets the field. As
a default, if NULL is passed, we continue to use strerror(job->ret).

All existing callers are changed to pass NULL. They can be improved in
separate patches.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-05-30 13:31:01 +02:00
Peter Maydell b706ec24f1 Merge tpm 2018/05/23 v4
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJbCMvcAAoJEHWtZYAqC0IRaZkIAKYYlLTjuiSXjOi3pwAIK9MS
 kBXqEqQcAXeKXFVYfyrzxQGJK8vVm+I/P+L+Q3jOLanRWM+gZsxGpct7fDHAQQMY
 iY+o5RAZUJH9rpr9mgvim332A52B/TmPl3viU6SFVM8y7PSOm3Tm7NsC17ak7XDF
 rEY2q5Cou08SU5rTz6Dif2adoz9/YkITMIamz2nVmnjxS5zDCfe+DEh+TrQJXGt5
 3n+Ywva7AzYCvuunKIscYD1QBOu7pMMOTduGkABduKyWr1oLeipG1QU/VTqz9S7O
 xDvySSZMYUohj2D71J7brc05gmMQb668geDIxuKmnXrSYgSoraJatGvfV0UTxOA=
 =BJKH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2018-05-23-4' into staging

Merge tpm 2018/05/23 v4

# gpg: Signature made Sat 26 May 2018 03:52:12 BST
# gpg:                using RSA key 75AD65802A0B4211
# gpg: Good signature from "Stefan Berger <stefanb@linux.vnet.ibm.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B818 B9CA DF90 89C2 D5CE  C66B 75AD 6580 2A0B 4211

* remotes/stefanberger/tags/pull-tpm-2018-05-23-4:
  test: Add test cases that use the external swtpm with CRB interface
  docs: tpm: add VM save/restore example and troubleshooting guide
  tpm: extend TPM TIS with state migration support
  tpm: extend TPM emulator with state migration support

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-29 09:57:09 +01:00
Stefan Berger b21373d071 test: Add test cases that use the external swtpm with CRB interface
Add a test program for testing the CRB with the external swtpm.

The 1st test case extends a PCR and reads back the value and compares
it against an expected return packet.

The 2nd test case repeats the 1st test case and then migrates the
external swtpm's state along with the VM state to a destination
QEMU and swtpm and checks that the PCR has the expected value now.

The test cases require 'swtpm' to be installed on the system and
in the PATH and 'swtpm' must support the --tpm2 option. If this is
not the case, the test will be skipped.

Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-05-25 22:47:27 -04:00
Michael S. Tsirkin fa0d421f39 libqtest: fail if child coredumps
Right now tests report OK status if QEMU crashes during cleanup.
Let's catch that case and fail the test.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
2018-05-25 17:08:39 +03:00
Tiwei Bie e3af2928f8 vhost-user-bridge: support host notifier
This patch introduces the host notifier support in
vhost-user-bridge. A new option (-H) is added to use
the host notifier. This is mainly used to test the
host notifier implementation in vhost user.

Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2018-05-24 21:14:11 +03:00
Kevin Wolf bdebdc712b qemu-iotests: Test job-* with block jobs
This adds a test case that tests the new job-* QMP commands with
mirror and backup block jobs.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-23 14:30:52 +02:00
Kevin Wolf 62a9428812 iotests: Move qmp_to_opts() to VM
qmp_to_opts() used to be a method of QMPTestCase, but recently we
started to add more Python test cases that don't make use of
QMPTestCase. In order to make the method usable there, move it to VM.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-05-23 14:30:51 +02:00