* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Remove
  qemuCaps parameters & use cached data
* src/qemu/qemu_driver.c: Don't create & pass qemuCaps to
  migration methods

* src/qemu/qemu_hotplug.h, src/qemu/qemu_hotplug.c: Remove
  qemuCaps parameters from all methods
* src/qemu/qemu_driver.c: Don't create & pass qemuCaps to
  hotplug methods

Clang 2.8 wasn't quite able to follow that persistentDef was
assigned earlier if (flags & VIR_DOMAIN_MEM_CONFIG) is true.
Silence this false positive, to make clang analysis easier to use.

* src/qemu/qemu_driver.c (qemudDomainSetMemoryFlags): Add an
annotation to silence clang's claim of a NULL dereference.

This code has had problems historically.  As originally
written, in commit 6bcf2501 (Jun 08), it could call unlink
on a random string, nuking an unrelated file.

Then commit 182a80b9 (Sep 09), the code was rewritten to
allocate tmp, with both a use-after-free bug and a chance to
call unlink(NULL).

Commit e206946d (Mar 11) fixed the use-after-free, but not the
NULL dereference.  Thanks to clang for catching this!

* src/qemu/qemu_driver.c (qemudDomainMemoryPeek): Don't call
unlink on NULL.

This commit fixes
qemu/qemu_driver.c: In function 'qemuDomainModifyDeviceFlags':
qemu/qemu_driver.c:4041:8: warning: 'ret' may be used uninitialized in this
function [-Wuninitialized]
qemu/qemu_driver.c:4013:9: note: 'ret' was declared here

The variable is set to -1 so that the error paths are taken when the code
to set it didn't get a chance to run. Without initializing it, we could
return some an undefined value from this function.

While I was at it, I made a trivial whitespace change in the same function
to improve readability.

Support update of disks by MODIFY_CONFIG

This patch includes changes for qemu's disk to support
virDomainUpdateDeviceFlags() with VIR_DOMAIN_DEVICE_MODIFY_CONFIG.

This patch adds support for CDROM/foppy disk types.
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

* src/qemu/qemu_driver.c
(qemuDomainUpdateDeviceConfig): support cdrom/floppy.

Support changes of disks by MODIFY_CONFIG for qemu.

This patch includes patches for qemu's disk to support
virDomainAt(De)tachDeviceFlags with VIR_DOMAIN_DEVICE_MODIFY_CONFIG.

Other devices can be added incrementally.
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

* /src/conf/domain_conf.c
(virDomainDiskIndexByName): returns array index of disk in vmdef.
(virDomainDiskRemoveByName): removes a disk which has the name in vmdef.
* src/qemu/qemu_driver.c
(qemuDomainAttachDeviceConfig): add support for Disks.
(qemuDomainDetachDeviceConfig): add support for Disks.

This patch adds functions for modify domain's persistent definition.
To do error recovery in easy way, we use a copy of vmdef and update it.

The whole sequence will be:

  make a copy of domain definition.

  if (flags & MODIFY_CONFIG)
      update copied domain definition
  if (flags & MODIF_LIVE)
      do hotplug.
  if (no error)
      save copied one to the file and update cached definition.
  else
      discard copied definition.

This patch is mixuture of Eric Blake's work and mine.
From: Eric Blake <eblake@redhat.com>
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

(virDomainObjCopyPersistentDef): make a copy of persistent vm definition
(qemuDomainAttach/Detach/UpdateDeviceConfig) : callbacks. now empty
(qemuDomainModifyDeviceFlags): add support for MODIFY_CONFIG and MODIFY_CURRENT

This patch strips reusable part of qemuDomainUpdateDeviceFlags()
and consolidate it to qemuDomainModifyDeviceFlags().
No functional changes.

* src/qemu/qemu_driver.c
(qemuDomainChangeDiskMediaLive) : pulled out code for updating disks.
(qemuDomainUpdateDeviceLive) : core of UpdateDevice, extracted from
UpdateDeviceFlags()
(qemuDomainModifyDeviceFlags): add support for updating device in live domain.
(qemuDomainUpdateDeviceFlags): reworked as a wrapper function of
qemuDomainModifyDeviceFlags()
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

clean up At(De)tachDeviceFlags() for consolidation.

qemuDomainAttachDeviceFlags()/qemuDomainDetachFlags()/
qemuDomainUpdateDeviceFlags() has similar logics and copied codes.

This patch series tries to unify them to use shared code when it can.
At first, clean up At(De)tachDeviceFlags() and devide it into functions.

By this, this patch pulls out shared components between functions.
Based on patch series by Eric Blake, I added some modification as
switch-case with QEMU_DEVICE_ATTACH, QEMU_DEVICE_DETACH, QEMU_DEVICE_UPDATE

* src/qemu/qemu_driver.c
(qemuDomainAt(De)tachDeviceFlags) : pulled out to qemuDomainModifyDeviceFlags()
(qemuDomainModifyDeviceFlags) : implements generic code for modifying domain.
(qemuDomainAt(De)tachDeviceFlagsLive) : code for at(de)taching devices to
domain in line. no changes in logic from old code.
(qemuDomainAt(De)tachDeviceDiskLive) : for at(de)taching Disks.
(qemuDomainAt(De)tachDeviceControllerLive) : for at(de)taching Controllers
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Centralize device modification in the more flexible APIs, to allow future
honoring of additional flags.  Explicitly reject the
VIR_DOMAIN_DEVICE_MODIFY_FORCE flag on attach/detach.

Based on Eric Blake<eblake@redhat.com>'s work.

* src/qemu/qemu_driver.c
(qemudDomainAttachDevice)(qemudDomainAttachDeviceFlags): Swap bodies,rename...
(qemudDomainDetachDevice, qemudDomainDetachDeviceFlags): Likewise.
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch does the following two things:
1. hold an extra reference while handling watchdog event
   If the domain is not persistent, and qemu quits unexpectedly before
   calling processWatchdogEvent(), vm will be freed and the function
   processWatchdogEvent() will be dangerous.

2. unlock qemu driver and vm before returning from processWatchdogEvent()
   When the function processWatchdogEvent() failed, we only free wdEvent,
   but forget to unlock qemu driver and vm, free dumpfile.

We do not lock qemu_driver when calling virThreadPoolNew(). If it failed,
we will unlock qemu_driver. It is dangerous.

We may use this pool during auto starting domains. So we must create it before
calling qemuAutostartDomains(). Otherwise, libvirtd will crash.

This patch enables the migration of Qemu VMs between hosts of different endianess. I tested this by migrating a i686 VM between a x86 and ppc64 host.

I am converting the 'int's in the VM's state header to uint32_t assuming this doesn't break compatibility with existing deployments other than Linux.

The methods qemuDomain{Get,Set}{Memory,Blkio,Scheduler}Parameters
all forgot to do a check on virDomainIsActive(), resulting in bogus
error messages from later parts of their impl

* src/qemu/qemu_driver.c: Add missing checks on virDomainIsActive()

This patch implements the code to support virDomainSetMaxMemory API,
and to support VIR_DOMAIN_MEM_MAXIMUM flag in qemudDomainSetMemoryFlags function.
As a result, we can change the maximum memory size of inactive QEMU guests.
Signed-off-by: NTaku Izumi <izumi.taku@jp.fujitsu.com>

* tools/virsh.pod: Fix grammar, and clarify wording.
* src/qemu/qemu_driver.c (qemudDomainObjStart): Drop redundant
condition.

This patch adds virDomainSetMemoryFlags(,,VIR_DOMAIN_MEM_CURRENT) support
code to qemu driver.

Also, change virDomainObjIsActive to return bool, given its usage.
Signed-off-by: NTaku Izumi <izumi.taku@jp.fujitsu.com>

1) Both "qemuDomainStartWithFlags" and "qemuAutostartDomain" try to
restore the domain from managedsave'ed image if it exists (by
invoking "qemuDomainObjRestore"), but it unlinks the image even
if restoring fails, which causes data loss. (This problem exists
for "virsh managedsave dom; virsh start dom").

The fix for is to unlink the managed state file only if restoring
succeeded.

2) For "virsh save dom; virsh restore dom;", it can cause data
corruption if one reuse the saved state file for restoring. Add
doc to tell user about it.

3) In "qemuDomainObjStart", if "managed_save" is NULL, we shouldn't
fallback to start the domain, skipping it to cleanup as a incidental
fix. Discovered by Eric.

Even with -Wuninitialized (which is part of autobuild.sh
--enable-compile-warnings=error), gcc does NOT catch this
use of an uninitialized variable:

{
  if (cond)
    goto error;
  int a = 1;
error:
  printf("%d", a);
}

which prints 0 (supposing the stack started life wiped) if
cond was true.  Clang will catch it, but we don't use clang
as often.  Using gcc -Wjump-misses-init catches it, but also
gives false positives:

{
  if (cond)
    goto error;
  int a = 1;
  return a;
error:
  return 0;
}

Here, a was never used in the scope of the error block, so
declaring it after goto is technically fine (and clang agrees).
However, given that our HACKING already documents a preference
to C89 decl-before-statement, the false positive warning is
enough of a prod to comply with HACKING.

[Personally, I'd _really_ rather use C99 decl-after-statement
to minimize scope, but until gcc can efficiently and reliably
catch scoping and uninitialized usage bugs, I'll settle with
the compromise of enforcing a coding standard that happens to
reject false positives if it can also detect real bugs.]

* acinclude.m4 (LIBVIRT_COMPILE_WARNINGS): Add -Wjump-misses-init.
* src/util/util.c (__virExec): Adjust offenders.
* src/conf/domain_conf.c (virDomainTimerDefParseXML): Likewise.
* src/remote/remote_driver.c (doRemoteOpen): Likewise.
* src/phyp/phyp_driver.c (phypGetLparNAME, phypGetLparProfile)
(phypGetVIOSFreeSCSIAdapter, phypVolumeGetKey)
(phypGetStoragePoolDevice)
(phypVolumeGetPhysicalVolumeByStoragePool)
(phypVolumeGetPath): Likewise.
* src/vbox/vbox_tmpl.c (vboxNetworkUndefineDestroy)
(vboxNetworkCreate, vboxNetworkDumpXML)
(vboxNetworkDefineCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (getCapsObject)
(xenapiDomainDumpXML): Likewise.
* src/xenapi/xenapi_utils.c (createVMRecordFromXml): Likewise.
* src/security/security_selinux.c (SELinuxGenNewContext):
Likewise.
* src/qemu/qemu_command.c (qemuBuildCommandLine): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Likewise.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextGetPtyPaths):
Likewise.
* src/qemu/qemu_driver.c (qemudDomainShutdown)
(qemudDomainBlockStats, qemudDomainMemoryPeek): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendCreateIfaceIQN): Likewise.
* src/node_device/node_device_udev.c (udevProcessPCI): Likewise.

We create a temporary file to save memory, and we will remove it after reading
memory to buffer. But we free the variable that contains the temporary filename
before we remove it. So we should free tmp after unlinking it.

qemuDriverUnlock() wasn't called on 2 exit paths
* src/qemu/qemu_driver.c: fix qemudDomainSave() to always unlock
  the driver before exiting on error

Currently libvirt's default logging is limited and it is difficult to
determine what was happening when a proglem occurred (especially on a
machines where one don't know the detail.)  This patch helps to do that
by making additional logging available for the following events:

  creating/defining/undefining domains
  creating/defining/undefining/starting/stopping networks
  creating/defining/undefining/starting/stopping storage pools
  creating/defining/undefining/starting/stopping storage volumes.

* AUTHORS: add Naoya Horiguchi
* src/network/bridge_driver.c src/qemu/qemu_driver.c
  src/storage/storage_driver.c: provide more VIR_INFO logging

When domain startup, setting cpu affinity and cpu shares according
to the cputune xml specified in domain xml.

Modify "qemudDomainPinVcpu" to update domain config for vcpupin,
and modify "qemuSetSchedulerParameters" to update domain config
for cpu shares.

v1 - v2:
   * Use "VIR_ALLOC_N" instead of "VIR_ALLOC_VAR"
   * But keep raising error when it fails on adding vcpupin xml
     entry, as I still don't have a better idea yet.

The O_NONBLOCK flag doesn't work as desired on plain files
or block devices. Introduce an I/O helper program that does
the blocking I/O operations, communicating over a pipe that
can support O_NONBLOCK

* src/fdstream.c, src/fdstream.h: Add non-blocking I/O
  on plain files/block devices
* src/Makefile.am, src/util/iohelper.c: I/O helper program
* src/qemu/qemu_driver.c, src/lxc/lxc_driver.c,
  src/uml/uml_driver.c, src/xen/xen_driver.c: Update for
  streams API change

Regression introduced in commit 6034ddd5.

* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Jump to correct
label.

This points out that core dumps (still) don't work for root-squash
NFS, since the fd is not opened correctly.  This patch should not
introduce any functionality change, it is just a refactoring to
avoid duplicated code.

* src/qemu/qemu_migration.h (qemuMigrationToFile): New prototype.
* src/qemu/qemu_migration.c (qemuMigrationToFile): New function.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag, doCoreDump): Use
it.

Direct access to an open file is so much simpler than passing
everything through a pipe!

* src/qemu/qemu_driver.c (qemudOpenAsUID)
(qemudDomainSaveImageClose): Delete.
(qemudDomainSaveImageOpen): Rename...
(qemuDomainSaveImageOpen): ...and drop read_pid argument.  Use
virFileOpenAs instead of qemudOpenAsUID.
(qemudDomainSaveImageStartVM, qemudDomainRestore)
(qemudDomainObjRestore): Rename...
(qemuDomainSaveImageStartVM, qemuDomainRestore)
(qemDomainObjRestore): ...and simplify accordingly.
(qemudDomainObjStart, qemuDriver): Update callers.

* src/storage/storage_backend.c (createRawFileOpHook): Change
signature.
(struct createRawFileOpHookData): Delete unused struct.
(virStorageBackendCreateRaw): Adjust caller.
* src/qemu/qemu_driver.c (struct fileOpHookData): Delete unused
struct.
(qemudDomainSaveFileOpHook): Rename...
(qemuDomainSaveFileOpHook): ...and change signature.
(qemudDomainSaveFlag): Adjust caller.

This patch intentionally doesn't change indentation, in order to
make it easier to review the real changes.

* src/util/util.h (VIR_FILE_OP_RETURN_FD, virFileOperationHook):
Delete.
(virFileOperation): Rename...
(virFileOpenAs): ...and reduce parameters.
* src/util/util.c (virFileOperationNoFork, virFileOperation):
Rename and simplify.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Adjust caller.
* src/storage/storage_backend.c (virStorageBackendCreateRaw):
Likewise.
* src/libvirt_private.syms: Reflect rename.

This makes root-squash NFS saves more efficient.

* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Use new
virFileOperation flag to open fd only once.

This allows direct saves (no compression, no root-squash NFS) to use
the more efficient fd: migration, which in turn avoids a race where
qemu exec: migration can sometimes fail because qemu does a generic
waitpid() that conflicts with the pclose() used by exec:.  Further
patches will solve compression and root-squash NFS.

* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Use new function
when there is no compression.

Otherwise, if something like doStopVcpus fails after the first
restore, a second restore is attempted and throws a useless
warning.

* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Avoid second
restore of state label.

Add missing open curly brace between function declaration of non-linux
variant of qemudDomainInterfaceStats() and its body.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Steps to reproduce this bug:
1. virsh attach-disk domain --source diskimage --target sdb --sourcetype file --driver qemu --subdriver qcow2
   error: Failed to attach disk
   error: operation failed: adding scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 device failed: Property 'scsi-disk.drive' can't find value 'drive-scsi0-0-1'
2. service libvirtd restart
   Stopping libvirtd daemon:                                  [  OK  ]
   Starting libvirtd daemon:                                  [  OK  ]
3. virsh attach-disk domain --source diskimage --target sdb --sourcetype file --driver qemu --subdriver raw
   error: Failed to attach disk
   error: operation failed: adding lsi,id=scsi0,bus=pci.0,addr=0x6 device failed: Duplicate ID 'scsi0' for device

The reason is that we create a new scsi controller but we do not update
/var/run/libvirt/qemu/domain.xml.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

Enhance the QEMU migration monitoring loop, so that it can get
a signal to change migration speed on the fly

* src/qemu/qemu_domain.h: Add signal for changing speed on the fly
* src/qemu/qemu_driver.c: Wire up virDomainMigrateSetSpeed driver
* src/qemu/qemu_migration.c: Support signal for changing speed

It is possible to set a migration speed limit when starting
migration. This new API allows the speed limit to be changed
on the fly to adjust to changing conditions

* src/driver.h, src/libvirt.c, src/libvirt_public.syms,
  include/libvirt/libvirt.h.in: Add virDomainMigrateSetMaxSpeed
* src/esx/esx_driver.c, src/lxc/lxc_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/vmware/vmware_driver.c, src/xen/xen_driver.c,
  src/libxl/libxl_driver.c: Stub new API

THREADS.txt states that the contents of vm should not be read or
modified while the vm lock is not held, but that the lock must not
be held while performing a monitor command.  This fixes all the
offenders that I could find.

* src/qemu/qemu_process.c (qemuProcessStartCPUs)
(qemuProcessInitPasswords, qemuProcessStart): Don't modify or
refer to vm state outside lock.
* src/qemu/qemu_driver.c (qemudDomainHotplugVcpus): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphicsPasswords):
Likewise.