Code movement to pull the conversion from Qdict to BlockdevOptionsRbd
into a helper function.
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NJeff Cody <jcody@redhat.com>
Message-id: 5b49a980f2cde6610ab1df41bb0277d00b5db893.1536704901.git.jcody@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Merge tpm 2018/09/07 v1

# gpg: Signature made Fri 07 Sep 2018 21:38:06 BST
# gpg:                using RSA key 75AD65802A0B4211
# gpg: Good signature from "Stefan Berger <stefanb@linux.vnet.ibm.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B818 B9CA DF90 89C2 D5CE  C66B 75AD 6580 2A0B 4211

* remotes/stefanberger/tags/pull-tpm-2018-09-07-1:
  tests: Fix signalling race condition in TPM tests
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

ppc patch queue 2018-09-07

Here's another pull request for qemu-3.1.  No real theme here, just an
assortment of various fixes.  Probably the most notable thing is the
removal of the ppcemb target which has been deprecated for some time
now.

# gpg: Signature made Fri 07 Sep 2018 08:30:02 BST
# gpg:                using RSA key 6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>"
# gpg:                 aka "David Gibson (Red Hat) <dgibson@redhat.com>"
# gpg:                 aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>"
# gpg:                 aka "David Gibson (kernel.org) <dwg@kernel.org>"
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E  87DC 6C38 CACA 20D9 B392

* remotes/dgibson/tags/ppc-for-3.1-20180907:
  target-ppc: Extend HWCAP2 bits for ISA 3.0
  target/ppc/kvm: set vcpu as online/offline
  Fix a deadlock case in the CPU hotplug flow
  spapr: Correct reference count on spapr-cpu-core
  mac_newworld: implement custom FWPathProvider
  uninorth: add ofw-addr property to allow correct fw path generation
  mac_oldworld: implement custom FWPathProvider
  grackle: set device fw_name and address for correct fw path generation
  macio: add addr property to macio IDE object
  macio: add macio bus to help with fw path generation
  macio: move MACIOIDEState type declarations to macio.h
  spapr_pci: fix potential NULL pointer dereference
  spapr: fix leak of rev array
  ppc: Remove deprecated ppcemb target
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

A misc collection of RISC-V related patches for 3.1.

# gpg: Signature made Wed 05 Sep 2018 23:06:55 BST
# gpg:                using RSA key 21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-riscv-pullreq-20180905:
  riscv: remove define cpu_init()
  hw/riscv/spike: Set the soc device tree node as a simple-bus
  hw/riscv/virtio: Set the soc device tree node as a simple-bus
  target/riscv: call gen_goto_tb on DISAS_TOO_MANY
  target/riscv: optimize indirect branches
  target/riscv: optimize cross-page direct jumps in softmmu
  RISC-V: Simplify riscv_cpu_local_irqs_pending
  RISC-V: Use atomic_cmpxchg to update PLIC bitmaps
  RISC-V: Improve page table walker spec compliance
  RISC-V: Update address bits to support sv39 and sv48
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

vga: virtio reset fix, virtio iommu support.

# gpg: Signature made Mon 03 Sep 2018 07:57:32 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20180903-pull-request:
  virtio-gpu: add iommu support
  virtio-gpu: pass down VirtIOGPU pointer to a bunch of functions
  use dpy_gfx_update_full
  Revert "virtio-gpu: fix crashes upon warm reboot with vga mode"
  virtio-vga: fix reset
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Monitor patches for 2018-09-01

# gpg: Signature made Sat 01 Sep 2018 12:06:52 BST
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-monitor-2018-09-01:
  monitor: no need to save need_resume
  Revert "qmp: isolate responses into io thread"
  qmp: constify qmp_is_oob()
  monitor: consitify qmp_send_response() QDict argument
  monitor: accept input on resume
  monitor: simplify monitor_qmp_setup_handlers_bh
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Block patches:
- (Block) job exit refactoring, part 1
  (removing job_defer_to_main_loop())
- test-bdrv-drain leak fix

# gpg: Signature made Fri 31 Aug 2018 15:30:33 BST
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/xanclic/tags/pull-block-2018-08-31-v2:
  jobs: remove job_defer_to_main_loop
  jobs: remove ret argument to job_completed; privatize it
  block/backup: make function variables consistently named
  jobs: utilize job_exit shim
  block/mirror: utilize job_exit shim
  block/commit: utilize job_exit shim
  jobs: add exit shim
  jobs: canonize Error object
  jobs: change start callback to run callback
  tests: fix bdrv-drain leak
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Markus spotted some issues with this new test case which
unfortunately I didn't notice had been flagged until after
I'd applied the pull request. Revert the relevant commit.

This reverts commit 2b70ea92.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios.for-upstream-20180831' into staging

qemu-openbios.for-upstream queue

# gpg: Signature made Fri 31 Aug 2018 12:26:55 BST
# gpg:                using RSA key 5BC2C56FAE0F321F
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>"
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C  C9C4 5BC2 C56F AE0F 321F

* remotes/mcayland/tags/qemu-openbios.for-upstream-20180831:
  Update OpenBIOS images to a1280807 built from submodule.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Removal of deprecated options and improvements for the qtests

# gpg: Signature made Fri 31 Aug 2018 09:10:23 BST
# gpg:                using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>"
# gpg:                 aka "Thomas Huth <thuth@redhat.com>"
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>"
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>"
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2018-08-31:
  tests: add a qmp success-response test
  tests: add qmp/qom-set-without-value test
  tests: add qmp/object-add-without-props test
  tests: add qmp_assert_error_class()
  tests/libqos: Utilize newer glib spawn check
  net: Remove the deprecated -tftp, -bootp, -redir and -smb options
  Remove the deprecated options -startdate, -localtime and -rtc-td-hack
  Remove the deprecated -nodefconfig option
  Remove the deprecated -balloon option
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

- various fixes and improvements in the tcg code
- split off the individual virtio-ccw devices into separate files

# gpg: Signature made Wed 29 Aug 2018 10:38:03 BST
# gpg:                using RSA key DECF6B93C6F02FAF
# gpg: Good signature from "Cornelia Huck <conny@cornelia-huck.de>"
# gpg:                 aka "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"
# gpg:                 aka "Cornelia Huck <cohuck@kernel.org>"
# gpg:                 aka "Cornelia Huck <cohuck@redhat.com>"
# Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0  18CE DECF 6B93 C6F0 2FAF

* remotes/cohuck/tags/s390x-20180829:
  target/s390x: use regular spaces in translate.c
  hw/s390x: Move virtio-ccw-blk code to a separate file
  hw/s390x: Move virtio-ccw-net code to a separate file
  hw/s390x: Move virtio-ccw-input code to a separate file
  hw/s390x: Move virtio-ccw-gpu code to a separate file
  hw/s390x: Move vhost-vsock-ccw code to a separate file
  hw/s390x: Move virtio-ccw-crypto code to a separate file
  hw/s390x: Move virtio-ccw-9p code to a separate file
  hw/s390x: Move virtio-ccw-rng code to a separate file
  hw/s390x: Move virtio-ccw-scsi code to a separate file
  hw/s390x: Move virtio-ccw-balloon code to a separate file
  hw/s390x: Move virtio-ccw-serial code to a separate file
  hw/s390x/virtio-ccw: Consolidate calls to virtio_ccw_unrealize()
  target/s390x: fix PACK reading 1 byte less and writing 1 byte more
  target/s390x: add EX support for TRT and TRTR
  target/s390x: fix IPM polluting irrelevant bits
  target/s390x: fix CSST decoding and runtime alignment check
  target/s390x: add BAL and BALR instructions
  tests/tcg: add a simple s390x test
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

QAPI patches for 2018-08-28

# gpg: Signature made Tue 28 Aug 2018 17:23:32 BST
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2018-08-28:
  qapi: Add comments to aid debugging generated introspection
  qapi: Minor introspect.py cleanups
  qapi: Update docs for generator changes since commit 9ee86b85
  qapi: Emit a blank line before dummy declaration
  qapi: Drop qapi_event_send_FOO()'s Error ** argument
  qapi: Fix build_params() for empty parameter list
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

This patch fixes a race condition and test failure where the main process
waits for the signal of a thread but the thread already sent that signal
via a condition. Since these signals are non-sticky, we need to introduce a
separate variable to make this signal sticky.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

This adds the HWCAP2 bit to detect if a linux user process is
running on an ISA 3.0 compliant cpu like POWER9. This can be
verified using a simple test program that prints the value in
the auxiliary vector for AT_HWCAP2 as shown below.

Before:
  $ qemu-ppc64le -cpu power8 test
  0x8c000000

  $ qemu-ppc64le -cpu power9 test
  0x8c000000

After:
  $ qemu-ppc64le -cpu power8 test
  0x8c000000

  $ qemu-ppc64le -cpu power9 test
  0x8c800000
Signed-off-by: NSandipan Das <sandipan@linux.ibm.com>
Reviewed-by: NLaurent Vivier <laurent@vivier.eu>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>

cpu_init() was removed since 2.12, so drop the define that is now unused.
Signed-off-by: NIgor Mammedov <imammedo@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NMichael Clark <mjc@sifive.com>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

To allow Linux to enumerate devices on the /soc/ node set it as a
"simple-bus".
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

To allow Linux to enumerate devices on the /soc/ node set it as a
"simple-bus".
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

Performance impact of this and the previous commits, measured with
the very-easy-to-cross-compile rv8-bench:
  https://github.com/rv8-io/rv8-bench

Host: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz

- Key:
  before: master
  after1,2,3: the 3 commits in this series (i.e. 3 is this commit)

- User-mode:

 bench      before  after1  after2  after3  final speedup
---------------------------------------------------------
 aes        1.12s   1.12s   1.10s   1.00s   1.12
 bigint     0.78s   0.78s   0.78s   0.78s   1
 dhrystone  0.96s   0.97s   0.49s   0.49s   1.9591837
 miniz      1.94s   1.94s   1.88s   1.86s   1.0430108
 norx       0.51s   0.51s   0.49s   0.48s   1.0625
 primes     0.85s   0.85s   0.84s   0.84s   1.0119048
 qsort      4.87s   4.88s   1.86s   1.86s   2.6182796
 sha512     0.76s   0.77s   0.64s   0.64s   1.1875

(after1 only applies to softmmu, so no surprises here)

- Full-system (fedora):

 bench      before  after1  after2  after3  final speedup
---------------------------------------------------------
 aes        2.68s   2.54s   2.60s   2.34s   1.1452991
 bigint     1.61s   1.56s   1.55s   1.64s   0.98170732
 dhrystone  1.78s   1.67s   1.25s   1.24s   1.4354839
 miniz      3.53s   3.35s   3.28s   3.35s   1.0537313
 norx       1.13s   1.09s   1.07s   1.06s   1.0660377
 primes     15.37s  15.41s  15.20s  15.37s  1
 qsort      7.20s   6.71s   3.85s   3.96s   1.8181818
 sha512     1.07s   1.04s   0.90s   0.90s   1.1888889

SoftMMU slows things down, so the numbers are less sensitive.
Cross-page jumps improve things a little bit, though.

Note that I'm not showing here averages, just results from a
single run, so with primes there isn't much to worry about.
Signed-off-by: NEmilio G. Cota <cota@braap.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

Signed-off-by: NEmilio G. Cota <cota@braap.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

Signed-off-by: NEmilio G. Cota <cota@braap.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

Set the newly added register(KVM_REG_PPC_ONLINE) to indicate if the vcpu is
online(1) or offline(0)

KVM will use this information to set the RWMR register, which controls the PURR
and SPURR accumulation.

CC: paulus@samba.org
Signed-off-by: NNikunj A Dadhania <nikunj@linux.ibm.com>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>

This commit is intended to improve readability.
There is no change to the logic.

Cc: Sagar Karandikar <sagark@eecs.berkeley.edu>
Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Alistair Francis <Alistair.Francis@wdc.com>
Signed-off-by: NMichael Clark <mjc@sifive.com>
Reviewed-by: NAlistair Francis <alistair.francis@wdc.com>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

The PLIC previously used a mutex to protect against concurrent
access to the claimed and pending bitfields. Instead of using
a mutex, we update the bitfields using atomic_cmpxchg.

Rename sifive_plic_num_irqs_pending to sifive_plic_irqs_pending
and add an early out if any interrupts are pending as the
count of pending interrupts is not used.

Cc: Sagar Karandikar <sagark@eecs.berkeley.edu>
Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Alistair Francis <Alistair.Francis@wdc.com>
Signed-off-by: NMichael Clark <mjc@sifive.com>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

- Inline PTE_TABLE check for better readability
- Change access checks from ternary operator to if
- Improve readibility of User page U mode and SUM test
- Disallow non U mode from fetching from User pages
- Add reserved PTE flag check: W or W|X
- Add misaligned PPN check
- Set READ protection for PTE X flag and mstatus.mxr
- Use memory_region_is_ram in pte update

Cc: Sagar Karandikar <sagark@eecs.berkeley.edu>
Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Alistair Francis <Alistair.Francis@wdc.com>
Signed-off-by: NMichael Clark <mjc@sifive.com>
Reviewed-by: NAlistair Francis <alistair.francis@wdc.com>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

Cc: Sagar Karandikar <sagark@eecs.berkeley.edu>
Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Alistair Francis <Alistair.Francis@wdc.com>
Signed-off-by: NPalmer Dabbelt <palmer@sifive.com>
Reviewed-by: NAlistair Francis <alistair.francis@wdc.com>
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>

Switch from cpu_physical_memory_map to dma_memory_map,
so iommu mappings for virtio-gpu objects work properly.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>
Message-Id: <20180829122101.29852-3-kraxel@redhat.com>

No functional change, just preparation for a followup patch
which needs a VirtIOGPU pointer.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>
Message-Id: <20180829122101.29852-2-kraxel@redhat.com>

We need to set cs->halted to 1 before calling ppc_set_compat. The reason
is that ppc_set_compat kicks up the new thread created to manage the
hotplugged KVM virtual CPU and the code drives directly to KVM_RUN
ioctl. When cs->halted is 1, the code:

int kvm_cpu_exec(CPUState *cpu)
...
     if (kvm_arch_process_async_events(cpu)) {
         atomic_set(&cpu->exit_request, 0);
         return EXCP_HLT;
     }
...

returns before it reaches KVM_RUN, giving time to the main thread to
finish its job. Otherwise we can fall in a deadlock because the KVM
thread will issue the KVM_RUN ioctl while the main thread is setting up
KVM registers. Depending on how these jobs are scheduled we'll end up
freezing QEMU.

The following output shows kvm_vcpu_ioctl sleeping because it cannot get
the mutex and never will.
PS: kvm_vcpu_ioctl was triggered kvm_set_one_reg - compat_pvr.

STATE: TASK_UNINTERRUPTIBLE|TASK_WAKEKILL

PID: 61564  TASK: c000003e981e0780  CPU: 48  COMMAND: "qemu-system-ppc"
 #0 [c000003e982679a0] __schedule at c000000000b10a44
 #1 [c000003e98267a60] schedule at c000000000b113a8
 #2 [c000003e98267a90] schedule_preempt_disabled at c000000000b11910
 #3 [c000003e98267ab0] __mutex_lock at c000000000b132ec
 #4 [c000003e98267bc0] kvm_vcpu_ioctl at c00800000ea03140 [kvm]
 #5 [c000003e98267d20] do_vfs_ioctl at c000000000407d30
 #6 [c000003e98267dc0] ksys_ioctl at c000000000408674
 #7 [c000003e98267e10] sys_ioctl at c0000000004086f8
 #8 [c000003e98267e30] system_call at c00000000000b488

crash> struct -x kvm.vcpus 0xc000003da0000000
vcpus = {0xc000003db4880000, 0xc000003d52b80000, 0xc0000039e9c80000, 0xc000003d0e200000, 0xc000003d58280000, 0x0, 0x0, ...}

crash> struct -x kvm_vcpu.mutex.owner 0xc000003d58280000
  mutex.owner = {
    counter = 0xc000003a23a5c881 <- flag 1: waiters
  },

crash> bt 0xc000003a23a5c880
PID: 61579  TASK: c000003a23a5c880  CPU: 9   COMMAND: "CPU 4/KVM"
(active)

crash> struct -x kvm_vcpu.mutex.wait_list 0xc000003d58280000
  mutex.wait_list = {
    next = 0xc000003e98267b10,
    prev = 0xc000003e98267b10
  },

crash> struct -x mutex_waiter.task 0xc000003e98267b10
  task = 0xc000003e981e0780

The following command-line was used to reproduce the problem (note: gdb
and trace can change the results).

 $ qemu-ppc/build/ppc64-softmmu/qemu-system-ppc64 -cpu host \
     -enable-kvm -m 4096 \
     -smp 4,maxcpus=8,sockets=1,cores=2,threads=4 \
     -display none -nographic \
     -drive file=disk1.qcow2,format=qcow2
 ...
 (qemu) device_add host-spapr-cpu-core,core-id=4
[no interaction is possible after it, only SIGKILL to take the terminal
back]
Signed-off-by: NJose Ricardo Ziviani <joserz@linux.ibm.com>
Reviewed-by: NGreg Kurz <groug@kaod.org>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>

Now that the job infrastructure is handling the job_completed call for
all implemented jobs, we can remove the interface that allowed jobs to
schedule their own completion.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-10-jsnow@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Jobs are now expected to return their retcode on the stack, from the
.run callback, so we can remove that argument.

job_cancel does not need to set -ECANCELED because job_completed will
update the return code itself if the job was canceled.

While we're here, make job_completed static to job.c and remove it from
job.h; move the documentation of return code to the .run() callback and
to the job->ret property, accordingly.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 20180830015734.19765-9-jsnow@redhat.com
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Rename opaque_job to job to be consistent with other job implementations.
Rename 'job', the BackupBlockJob object, to 's' to also be consistent.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-8-jsnow@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Utilize the job_exit shim by not calling job_defer_to_main_loop, and
where applicable, converting the deferred callback into the job_exit
callback.

This converts backup, stream, create, and the unit tests all at once.
Most of these jobs do not see any changes to the order in which they
clean up their resources, except the test-blockjob-txn test, which
now puts down its bs before job_completed is called.

This is safe for the same reason the reordering in the mirror job is
safe, because job_completed no longer runs under two locks, making
the unref safe even if it causes a flush.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-7-jsnow@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Change the manual deferment to mirror_exit into the implicit
callback to job_exit and the mirror_exit callback.

This does change the order of some bdrv_unref calls and job_completed,
but thanks to the new context in which we call .exit, this is safe to
defer the possible flushing of any nodes to the job_finalize_single
cleanup stage.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 20180830015734.19765-6-jsnow@redhat.com
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Change the manual deferment to commit_complete into the implicit
callback to job_exit, renaming commit_complete to commit_exit.

This conversion does change the timing of when job_completed is
called to after the bdrv_replace_node and bdrv_unref calls, which
could have implications for bjob->blk which will now be put down
after this cleanup.

Kevin highlights that we did not take any permissions for that backend
at job creation time, so it is safe to reorder these operations.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-5-jsnow@redhat.com
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

All jobs do the same thing when they leave their running loop:
- Store the return code in a structure
- wait to receive this structure in the main thread
- signal job completion via job_completed

Few jobs do anything beyond exactly this. Consolidate this exit
logic for a net reduction in SLOC.

More seriously, when we utilize job_defer_to_main_loop_bh to call
a function that calls job_completed, job_finalize_single will run
in a context where it has recursively taken the aio_context lock,
which can cause hangs if it puts down a reference that causes a flush.

You can observe this in practice by looking at mirror_exit's careful
placement of job_completed and bdrv_unref calls.

If we centralize job exiting, we can signal job completion from outside
of the aio_context, which should allow for job cleanup code to run with
only one lock, which makes cleanup callbacks less tricky to write.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-4-jsnow@redhat.com
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Jobs presently use both an Error object in the case of the create job,
and char strings in the case of generic errors elsewhere.

Unify the two paths as just j->err, and remove the extra argument from
job_completed. The integer error code for job_completed is kept for now,
to be removed shortly in a separate patch.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 20180830015734.19765-3-jsnow@redhat.com
[mreitz: Dropped a superfluous g_strdup()]
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Presently we codify the entry point for a job as the "start" callback,
but a more apt name would be "run" to clarify the idea that when this
function returns we consider the job to have "finished," except for
any cleanup which occurs in separate callbacks later.

As part of this clarification, change the signature to include an error
object and a return code. The error ptr is not yet used, and the return
code while captured, will be overwritten by actions in the job_completed
function.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20180830015734.19765-2-jsnow@redhat.com
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Spotted by ASAN:

=================================================================
==5378==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 65536 byte(s) in 1 object(s) allocated from:
    #0 0x7f788f83bc48 in malloc (/lib64/libasan.so.5+0xeec48)
    #1 0x7f788c9923c5 in g_malloc (/lib64/libglib-2.0.so.0+0x523c5)
    #2 0x5622a1fe37bc in coroutine_trampoline /home/elmarco/src/qq/util/coroutine-ucontext.c:116
    #3 0x7f788a15d75f in __correctly_grouped_prefixwc (/lib64/libc.so.6+0x4c75f)

(Broken in commit 4c8158e3.)
Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20180809114417.28718-3-marcandre.lureau@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Signed-off-by: NMark Cave-Ayland <mark.cave-ayland@ilande.co.uk>

Verify the usage of this schema feature and the API behaviour.  This
should be the only case where qmp_dispatch() returns NULL.
Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>