qemu treats blockjob bandwidth as a 64-bit number, in the units
of bytes/second.  But we stupidly modeled block job bandwidth
after migration bandwidth, which in turn was an 'unsigned long'
and therefore subject to 32-bit vs. 64-bit interpretations, and
with a scale of MiB/s.  Our code already has to convert between
the two scales, and report overflow as appropriate; although
this conversion currently lives in the monitor code.  In fact,
our conversion code limited things to 63 bits, because we
checked against LLONG_MAX and reject what would be negative
bandwidth if treated as signed.

On the bright side, our use of MiB/s means that even with a
32-bit unsigned long, we still have no problem representing a
bandwidth of 2GiB/s, which is starting to be more feasible as
10-gigabit or even faster interfaces are used.  And once you
get past the physical speeds of existing interfaces, any larger
bandwidth number behaves the same - effectively unlimited.
But on the low side, the granularity of 1MiB/s tuning is rather
coarse.  So the new virDomainBlockJob API decided to go with
a direct 64-bit bytes/sec number instead of the scaled number
that prior blockjob APIs had used.  But there is no point in
rounding this number to MiB/s just to scale it back to bytes/s
for handing to qemu.

In order to make future code sharing possible between the old
virDomainBlockRebase and the new virDomainBlockCopy, this patch
moves the scaling and overflow detection into the driver code.
Several of the block job calls that can set speed are fed
through a common interface, so it was easier to adjust all block
jobs at once, for consistency.  This patch is just code motion;
there should be no user-visible change in behavior.

* src/qemu/qemu_monitor.h (qemuMonitorBlockJob)
(qemuMonitorBlockCommit, qemuMonitorDriveMirror): Change
parameter type and scale.
* src/qemu/qemu_monitor.c (qemuMonitorBlockJob)
(qemuMonitorBlockCommit, qemuMonitorDriveMirror): Move scaling
and overflow detection...
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl)
(qemuDomainBlockRebase, qemuDomainBlockCommit): ...here.
(qemuDomainBlockCopy): Use bytes/sec.
Signed-off-by: NEric Blake <eblake@redhat.com>

Another layer of overly-multiplexed code that deserves to be
split into obviously separate paths for query vs. modify.
This continues the cleanup started in commit cefe0ba3.

In the process, make some tweaks to simplify the logic when
parsing the JSON reply.  There should be no user-visible
semantic changes.

* src/qemu/qemu_monitor.h (qemuMonitorBlockJob): Drop parameter.
(qemuMonitorBlockJobInfo): New prototype.
(BLOCK_JOB_INFO): Drop enum.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONBlockJob)
(qemuMonitorJSONBlockJobInfo): Likewise.
* src/qemu/qemu_monitor.c (qemuMonitorBlockJob): Split...
(qemuMonitorBlockJobInfo): ...into second function.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONBlockJob): Move
block info portions...
(qemuMonitorJSONGetBlockJobInfo): ...here, and rename...
(qemuMonitorJSONBlockJobInfo): ...and export.
(qemuMonitorJSONGetBlockJobInfoOne): Alter return semantics.
* src/qemu/qemu_driver.c (qemuDomainBlockPivot)
(qemuDomainBlockJobImpl, qemuDomainGetBlockJobInfo): Adjust
callers.
* src/qemu/qemu_migration.c (qemuMigrationDriveMirror)
(qemuMigrationCancelDriveMirror): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit fba6bc47 introduced support for the 'invtsc' feature,
which blocks migration. We should not include it in the
host-model CPU by default, because it's intended to be used
with migration.

https://bugzilla.redhat.com/show_bug.cgi?id=1138221

This commit is rather big. Firstly, the in memory config
representation is adjusted like if security_driver was set to "none".
The rest is then just adaptation to the new code that will generate
different seclabels.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1027096#c8

There are two ways in which security model can make it way into
<seclabel/>. One is as the @model attribute, the second one is
via security_driver knob in qemu.conf. Then, while parsing
<seclabel/> several checks and fix ups of old, stale combinations
are performed. However, iff @model is specified. They are not
done in the latter case. So it's still possible to feed libvirt
with senseless combinations (if qemu.conf is adjusted correctly).

One example of a seclabel that needs some adjustment (in case
security_driver=none in qemu.conf) is:

    <seclabel type='dynamic' relabel='yes'/>

The fixup code is copied from virSecurityLabelDefParseXML
(covering the former case) into virSecurityLabelDefsParseXML
(which handles the latter case).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The qemu implementation for virDomainGetBlockJobInfo() has a
minor bug: it grabs the qemu job with intent to QEMU_JOB_MODIFY,
which means it cannot be run in parallel with any other
domain-modifying command.  Among others, virDomainBlockJobAbort()
is such a modifying command, and it defaults to being
synchronous, and can wait as long as several seconds to ensure
that the job has actually finished.  Due to the job rules, this
means a user cannot obtain status about the job during that
timeframe, even though we know that some client management code
exists which is using a polling loop on status to see when a job
finishes.

This bug has been present ever since blockpull support was first
introduced (commit b976165c, v0.9.4 in Jul 2011), all because we
stupidly tried to cram too much multiplexing through a single
helper routine, but was made worse in 97c59b9c (v1.2.7) when
BlockJobAbort was fixed to wait longer.  It's time to disentangle
some of the mess in qemuDomainBlockJobImpl, and in the process
relax block job query to use QEMU_JOB_QUERY, since it can safely
be used in parallel with any long running modify command.

Technically, there is one case where getting block job info can
modify domain XML - we do snooping to see if a 2-phase job has
transitioned into the second phase, for an optimization in the
case of old qemu that lacked an event for the transition.  I
claim this optimization is safe (the jobs are all about modifying
qemu state, not necessarily xml state); but if it proves to be
a problem, we could use the difference between the capabilities
QEMU_CAPS_BLOCKJOB_{ASYNC,SYNC} to determine whether we even
need snooping, and only request a modifying job in the case of
older qemu.

* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Move info
handling...
(qemuDomainGetBlockJobInfo): ...here, and relax job type.
(qemuDomainBlockJobAbort, qemuDomainBlockJobSetSpeed)
(qemuDomainBlockRebase, qemuDomainBlockPull): Adjust callers.
Signed-off-by: NEric Blake <eblake@redhat.com>

The existing virDomainBlockRebase code rejected the combination of
_RELATIVE and _COPY flags, but only by accident.  It makes sense
to add support for the combination someday, at least for the case
of _SHALLOW and not _REUSE_EXT; but to implement it, libvirt would
have to pre-create the file with a relative backing name, and I'm
not ready to code that in yet.

Meanwhile, the code to forward on to the block copy code is getting
longer, and reorganizing the function to have the block pull done
early makes it easier to add even more block copy prep code.

This patch should have no semantic difference other than the quality
of the error message on the unsupported flag combination.  Pre-patch:

error: unsupported flags (0x10) in function qemuDomainBlockCopy

Post-patch:

error: argument unsupported: Relative backing during copy not supported yet

* src/qemu/qemu_driver.c (qemuDomainBlockRebase): Reorder code,
and improve error message of relative copy.
Signed-off-by: NEric Blake <eblake@redhat.com>

Now that hanging brace offenders have been fixed, we can automate
the check, and document our style.  Done as a separate commit from
code changes, to make it easier to just backport code changes, if
that is ever needed.

* cfg.mk (sc_curly_braces_style): Catch hanging braces.
* docs/hacking.html.in: Document it.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Our style overwhelmingly uses hanging braces (the open brace
hangs at the end of the compound condition, rather than on
its own line), with the primary exception of the top level function
body.  Fix the few remaining outliers, before adding a syntax
check in a later patch.

* src/interface/interface_backend_netcf.c (netcfStateReload)
(netcfInterfaceClose, netcf_to_vir_err): Correct use of { in
compound statement.
* src/conf/domain_conf.c (virDomainHostdevDefFormatSubsys)
(virDomainHostdevDefFormatCaps): Likewise.
* src/network/bridge_driver.c (networkAllocateActualDevice):
Likewise.
* src/util/virfile.c (virBuildPathInternal): Likewise.
* src/util/virnetdev.c (virNetDevGetVirtualFunctions): Likewise.
* src/util/virnetdevmacvlan.c
(virNetDevMacVLanVPortProfileCallback): Likewise.
* src/util/virtypedparam.c (virTypedParameterAssign): Likewise.
* src/util/virutil.c (virGetWin32DirectoryRoot)
(virFileWaitForDevices): Likewise.
* src/vbox/vbox_common.c (vboxDumpNetwork): Likewise.
* tests/seclabeltest.c (main): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Done as a separate commit in case earlier cleanups are backported
independently.

* cfg.mk (sc_require_space_before_label): New rule.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This patch focuses on all remaining problems, where there weren't
enough issues to warrant splitting it further.

* src/remote/remote_driver.c (doRemoteOpen): Correct use of {}.
* src/security/virt-aa-helper.c (vah_add_path, valid_path, main):
Likewise.
* src/rpc/virnetsocket.c (virNetSocketNewConnectLibSSH2):
Likewise.
* src/esx/esx_vi_types.c (esxVI_Type_FromString): Likewise.
* src/uml/uml_driver.c (umlDomainDetachDevice): Likewise.
* src/util/viralloc.c (virShrinkN): Likewise.
* src/util/virbuffer.c (virBufferURIEncodeString): Likewise.
* src/util/virdbus.c (virDBusCall): Likewise.
* src/util/virnetdev.c (virNetDevValidateConfig): Likewise.
* src/util/virnetdevvportprofile.c
(virNetDevVPortProfileGetNthParent): Likewise.
* src/util/virpci.c (virPCIDeviceIterDevices)
(virPCIDeviceWaitForCleanup)
(virPCIDeviceIsBehindSwitchLackingACS): Likewise.
* src/util/virsocketaddr.c (virSocketAddrGetNumNetmaskBits):
Likewise.
* src/util/viruri.c (virURIParseParams): Likewise.
* daemon/stream.c (daemonStreamHandleAbort): Likewise.
* tests/testutils.c (virtTestResult): Likewise.
* tests/cputest.c (cpuTestBaseline): Likewise.
* tools/virsh-domain.c (cmdDomPMSuspend): Likewise.
* tools/virsh-host.c (cmdNodeSuspend): Likewise.
* src/esx/esx_vi_generator.py (Type.generate_typefromstring):
Tweak generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This patch focuses on drivers that had several issues.

* src/lxc/lxc_fuse.c (lxcProcGetattr, lxcProcReadMeminfo): Correct
use of {}.
* src/lxc/lxc_driver.c (lxcDomainMergeBlkioDevice): Likewise.
* src/phyp/phyp_driver.c (phypConnectNumOfDomainsGeneric)
(phypUUIDTable_Init, openSSHSession, phypStoragePoolListVolumes)
(phypConnectListStoragePools, phypDomainSetVcpusFlags)
(phypStorageVolGetXMLDesc, phypStoragePoolGetXMLDesc)
(phypConnectListDefinedDomains): Likewise.
* src/vbox/vbox_common.c (vboxAttachSound, vboxDumpDisplay)
(vboxDomainRevertToSnapshot, vboxDomainSnapshotDelete): Likewise.
* src/vbox/vbox_tmpl.c (vboxStorageVolGetXMLDesc): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This patch focuses on code related to xen.

* src/libxl/libxl_conf.c (libxlCapsInitGuests)
(libxlMakeDomBuildInfo): Correct use of {}.
* src/xen/xen_hypervisor.c (virXen_getvcpusinfo)
(xenHypervisorMakeCapabilitiesInternal): Likewise.
* src/xen/xend_internal.c (xenDaemonOpen)
(xenDaemonDomainMigratePerform, xend_detect_config_version)
(xenDaemonDetachDeviceFlags, xenDaemonDomainMigratePerform)
(xenDaemonDomainBlockPeek): Likewise.
* src/xenapi/xenapi_driver.c (xenapiConnectListDomains)
(xenapiDomainLookupByUUID, xenapiDomainGetOSType): Likewise.
* src/xenconfig/xen_common.c (xenParseCPUFeatures, xenFormatNet):
Likewise.
* src/xenconfig/xen_sxpr.c (xenParseSxpr, xenFormatSxprNet)
(xenFormatSxpr): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This commit focuses on the qemu driver.

* src/qemu/qemu_command.c (qemuParseISCSIString)
(qemuParseCommandLineDisk, qemuParseCommandLine)
(qemuBuildSmpArgStr, qemuBuildCommandLine)
(qemuParseCommandLineDisk, qemuParseCommandLineSmp): Correct use
of {}.
* src/qemu/qemu_capabilities.c (virQEMUCapsProbeCPUModels):
Likewise.
* src/qemu/qemu_driver.c (qemuDomainCoreDumpWithFormat)
(qemuDomainRestoreFlags, qemuDomainGetInfo)
(qemuDomainMergeBlkioDevice): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainAttachNetDevice): Likewise.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextCreateSnapshot)
(qemuMonitorTextLoadSnapshot, qemuMonitorTextDeleteSnapshot):
Likewise.
* src/qemu/qemu_process.c (qemuProcessStop): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This patch focuses on code shared between multiple drivers.

* src/conf/domain_conf.c (virDomainFSDefParseXML)
(virSysinfoParseXML, virDomainNetDefParseXML)
(virDomainWatchdogDefParseXML)
(virDomainRedirFilterUSBDevDefParseXML): Correct use of {}.
* src/conf/interface_conf.c (virInterfaceDefParseDhcp)
(virInterfaceDefParseIp, virInterfaceVlanDefFormat)
(virInterfaceDefParseStartMode, virInterfaceDefParseBondMode)
(virInterfaceDefParseBondMiiCarrier)
(virInterfaceDefParseBondArpValid): Likewise.
* src/conf/node_device_conf.c (virNodeDevCapStorageParseXML):
Likewise.
* src/conf/nwfilter_conf.c (virNWFilterRuleDetailsParse)
(virNWFilterRuleParse, virNWFilterDefParseXML): Likewise.
* src/conf/secret_conf.c (secretXMLParseNode): Likewise.
* src/cpu/cpu_x86.c (x86Baseline, x86FeatureLoad, x86ModelLoad):
Likewise.
* src/network/bridge_driver.c (networkKillDaemon)
(networkDnsmasqConfContents): Likewise.
* src/node_device/node_device_hal.c (dev_refresh): Likewise.
* src/nwfilter/nwfilter_gentech_driver.c (virNWFilterInstantiate):
Likewise.
* src/nwfilter/nwfilter_ebiptables_driver.c
(_iptablesCreateRuleInstance): Likewise.
* src/storage/storage_backend_disk.c
(virStorageBackendDiskBuildPool): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

We lacked of HOME environment variable,
set 'HOME=/' as default.

The kernel sets up $HOME for the init process.
Therefore any init can assume that $HOME is set.
libvirt currently violates that implicit rule.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

libcap-ng >= 0.7.4 fails when it can't read /sys/kernel/cap_last_cap
and thus running a qemu guest fails.

Allow reading cap_last_cap in the libvirt-qemu apparmor abstraction.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

When FIPS mode is on, gnutls_dh_params_generate2 will fail if 1024 is
specified as the prime's number of bits, a bigger value works in both
cases.
Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

Memory is allocated for 'mnt_src' by VIR_STRDUP in the loop. Next
loop it will be allocated again. So we need to free 'mnt_src'
before continue the loop.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

Need to free 'root' and 'opts' before 'return -1' if symlink fails.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

If virSecurityManagerClearSocketLabel() fails, 'agent' won't
be freed before jumping to cleanup.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

Need to free 'uri_out' on error path.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

If the condition 'ret < 0' is true, the code will jump to
'cleanup' and 'conf' won't be freed.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

Coverity determined that 'log' and 'newenv' were not freed in
some cases. Free them in 'error' branch and normal branch.
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

Add testsuite coverage for the recent commit 0e1a1a8c.

* tests/commandhelper.c (main): Output umask.
* tests/commandtest.c (test15): Also test umask.
(mymain): Add test.
* tests/commanddata/*.log: Update expected output.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 0e1a1a8c introduced umask for virCommand, but the variables
used emit a warning on older compilers about shadowed global
declaration.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1136788Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Add umask to _virCommand, allow user to set umask to command.
Set umask(002) to qemu process to overwrite the default umask
of 022 set by many distros, so that unix sockets created for
virtio-serial has expected permissions.

Fix problem reported here:
https://sourceware.org/bugzilla/show_bug.cgi?id=13078#c11
https://bugzilla.novell.com/show_bug.cgi?id=888166

To use virtio-serial device, unix socket created for chardev with
default umask(022) has insufficient permissions.
e.g.:
-device virtio-serial \
-chardev socket,path=/tmp/foo,server,nowait,id=foo \
-device virtserialport,chardev=foo,name=org.fedoraproject.port.0

srwxr-xr-x 1 qemu qemu 0 21. Jul 14:19 /tmp/somefile.sock

Other users in the same group (like real user, test engines, etc)
cannot write to this socket.
Signed-off-by: NChunyan Liu <cyliu@suse.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

%systemd_preun macro cannot be split into several lines.

https://bugzilla.redhat.com/show_bug.cgi?id=1136736Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The 'elem' variable along with the domain object would be leaked when
taking the error path.

Found by coverity.

Currently, there is one flag passed in during macvtap creation
(withTap) -- Let's convert this field to an unsigned int flag
field for future expansion.
Signed-off-by: NMatthew Rosato <mjrosato@linux.vnet.ibm.com>
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1135388

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

* docs/news.html.in libvirt.spec.in: update for release
* po/*.po*: new localizations and regenerate pos

While qemu definitely caps granularity to 64 MiB, it places no
limits on buf-size.  On a machine beefy enough for lots of
memory, a buf-size larger than 2 GiB is feasible, so we should
pass a 64-bit parameter.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BLOCK_COPY_BUF_SIZE):
Allow 64 bits.
Signed-off-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The cleanup in commit cf976d9d used secdef->label to label the tap
FDs, but that is not possible since it's process-only label (svirt_t)
and not a object label (e.g. svirt_image_t).  Starting a domain failed
with EPERM, but simply using secdef->imagelabel instead of
secdef->label fixes it.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Since 1b807f92, connecting with virsh to an already running session
libvirtd fails with:
$ virsh list --all
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to
'/run/user/1000/libvirt/libvirt-sock': Transport endpoint is already
connected

This is caused by a logic error in virNetSocketNewConnectUnix: even if
the connection to the daemon socket succeeded, we still try to spawn the
daemon and then connect to it.
This commit changes the logic to not try to spawn libvirtd if we
successfully connected to its socket.

Most of this commit is whitespace changes, use of -w is recommended to
look at it.

Currently, after calling commands to create a new volumes,
virStorageBackendZFSCreateVol calls virStorageBackendZFSFindVols that
calls virStorageBackendZFSParseVol.

virStorageBackendZFSParseVol checks if a volume already exists by
trying to get it using virStorageVolDefFindByName.

For a just created volume it returns NULL, so volume is reported as
new and appended to pool->volumes. This causes a volume to be listed
twice as storageVolCreateXML appends this new volume to the list as
well.

Fix that by passing a new volume definition to
virStorageBackendZFSParseVol so it could determine if it needs to add
this volume to the list.

In qemuDomainSnapshotCreateDiskActive() if we jumped to cleanup from a
failed actions = virJSONValueNewArray(), then 'cfg' would be NULL.

So just return -1, which in turn removes the need for cleanup: