A mild case of dyslexia caused that commit
012f9b19 specifies wrong mask for the
smap cpu feature flag. This patch fixes that mistake.

Qemu has added some new feature flags. This patch adds them to libvirt.

The new features are for the cpuid function 0x7 that takes an argument
in the ecx register. Currently only 0x0 is used as the argument so I was
lazy and I just clear the registers to 0 before calling cpuid. In future
when there maybe will be some other possible arguments, we will need to
improve the cpu detection code to take this into account.

There are some descriptions not right in PowerPC CPU model driver.
This patch is to fix them.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>
Acked-by: NMichal Privoznik <mprivozn@redhat.com>

Currently, the CPU model driver is not implemented for PowerPC.
Host's CPU information is needed to exposed to guests' XML file some
time.

This patch is to implement the callback functions of CPU model driver.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>
Acked-by: NMichal Privoznik <mprivozn@redhat.com>

CPU version can be got by PVR on PowerPC. So this PVR is defined in
the CPU data in cpuData structure.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>
Acked-by: NMichal Privoznik <mprivozn@redhat.com>

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

Update the CPU helper APIs to use virReportError instead
of the virCPUReportError custom macor
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Adding CPU encoder/decoder for armhf to avoid runtime error messages.
Signed-off-by: NChuck Short <chuck.short@canonical.com>

Adding CPU encoder/decoder for s390 to avoid runtime error messages.
Signed-off-by: NThang Pham <thang.pham@us.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

This patch modifies the CPU comparrison function to report the
incompatibilities in more detail to ease identification of problems.

* src/cpu/cpu.h:
    cpuGuestData(): Add argument to return detailed error message.
* src/cpu/cpu.c:
    cpuGuestData(): Add passthrough for error argument.
* src/cpu/cpu_x86.c
    x86FeatureNames(): Add function to convert a CPU definition to flag
                       names.
    x86Compute(): - Add error message parameter
                  - Add macro for reporting detailed error messages.
                  - Improve error reporting.
                  - Simplify calculation of forbidden flags.
    x86DataIteratorInit():
    x86cpuidMatchAny(): Remove functions that are no longer needed.
* src/qemu/qemu_command.c:
    qemuBuildCpuArgStr(): - Modify for new function prototype
                          - Add detailed error reports
                          - Change error code on incompatible processors
                            to VIR_ERR_CONFIG_UNSUPPORTED instead of
                            internal error
* tests/cputest.c:
    cpuTestGuestData(): Modify for new function prototype

Return statements with parameter enclosed in parentheses were modified
and parentheses were removed. The whole change was scripted, here is how:

List of files was obtained using this command:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$'

Found files were modified with this command:
sed -i -e                                                                 \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

Then checked for nonsense.

The whole command looks like this:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$' | xargs sed -i -e                            \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

Using inheritance, this patch cleans up the cpu_map.xml file and also
sorts all CPU features according to the feature and registry
values. Model features are sorted the same way as foeatures in the
specification.
Also few models that are related were organized together and parts of
the XML are marked with comments

This patch adds support for the new tsc-deadline feature flag
and a new model to the supported model list describing the
Intel Sandy Bridge platform.

AMD Bulldozer (or Opteron_G4 as called in QEMU) was added to the list
of cpu models, flags were taken from upstream qemu cpu specifications
and should be sorted by bit values (or first occurence in the feature
specification part of cpu_map.xml).

Based on QEMU upstream commit 885bb0369a4f0abe2c0185178f3cb347cb02cdf1.

We found few more AMD-specific features in cpu64-rhel* models that
made it impossible to start qemu guest on Intel host (with this
setting) even though qemu itself starts correctly with them.
This impacts one test, thus the fix in tests/cputestdata/.

Some new cpu features were added to qemu. This patch adds some of them
to our CPU map.

In qemu there are 2 cpu models (cpu64-rhel5 and cpu64-rhel6) not
supported by libvirt. This patch adds the support with the flags
specifications from /usr/share/qemu-kvm/cpu-model/cpu-x86_64.conf
The only difference is that AMD-specific features are removed so
the processor type is not vendor-specific. Those features are either
emulated or ignored by qemu if host CPU doesn't support them.

Our HACKING discourages use of malloc and free, for at least
a couple of years now.  But we weren't enforcing it, until now :)

For now, I've exempted python and tests, and will clean those up
in subsequent patches.  Examples should be permanently exempt,
since anyone copying our examples won't have use of our
internal-only memory.h via libvirt_util.la.

* cfg.mk (sc_prohibit_raw_allocation): New rule.
(exclude_file_name_regexp--sc_prohibit_raw_allocation): and
exemptions.
* src/cpu/cpu.c (cpuDataFree): Avoid false positive.
* src/conf/network_conf.c (virNetworkDNSSrvDefParseXML): Fix
offenders.
* src/libxl/libxl_conf.c (libxlMakeDomBuildInfo, libxlMakeVfb)
(libxlMakeDeviceModelInfo): Likewise.
* src/rpc/virnetmessage.c (virNetMessageSaveError): Likewise.
* tools/virsh.c (_vshMalloc, _vshCalloc): Likewise.

Recently (or not so recently) QEMU added the kvm32 and kvm64
architectures, representing a least common denominator of all
hosts that can run KVM.  Add them to the machine map.

Also, some features that TCG supports were added to qemu64.
Add them to the cpu_map.xml whenever KVM is guaranteed to support
those.  We still have to leave some out, because they would not
be available to guests running on older hosts.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

In preparation for the patch to include Murmurhash3, which
introduces a virhashcode.h and virhashcode.c files, rename
the existing hash.h and hash.c to virhash.h and virhash.c
respectively.

VIR_DOMAIN_XML_UPDATE_CPU flag for virDomainGetXMLDesc may be used to
get updated custom mode guest CPU definition in case it depends on host
CPU. This patch implements the same behavior for host-model and
host-passthrough CPU modes.

The mode can be either of "custom" (default), "host-model",
"host-passthrough". The semantics of each mode is described in the
following examples:

- guest CPU is a default model with specified topology:
    <cpu>
      <topology sockets='1' cores='2' threads='1'/>
    </cpu>

- guest CPU matches selected model:
    <cpu mode='custom' match='exact'>
      <model>core2duo</model>
    </cpu>

- guest CPU should be a copy of host CPU as advertised by capabilities
  XML (this is a short cut for manually copying host CPU specification
  from capabilities to domain XML):
    <cpu mode='host-model'/>

  In case a hypervisor does not support the exact host model, libvirt
  automatically falls back to a closest supported CPU model and
  removes/adds features to match host. This behavior can be disabled by
    <cpu mode='host-model'>
      <model fallback='forbid'/>
    </cpu>

- the same as previous returned by virDomainGetXMLDesc with
  VIR_DOMAIN_XML_UPDATE_CPU flag:
    <cpu mode='host-model' match='exact'>
      <model fallback='allow'>Penryn</model>       --+
      <vendor>Intel</vendor>                         |
      <topology sockets='2' cores='4' threads='1'/>  + copied from
      <feature policy='require' name='dca'/>         | capabilities XML
      <feature policy='require' name='xtpr'/>        |
      ...                                          --+
    </cpu>

- guest CPU should be exactly the same as host CPU even in the aspects
  libvirt doesn't model (such domain cannot be migrated unless both
  hosts contain exactly the same CPUs):
    <cpu mode='host-passthrough'/>

- the same as previous returned by virDomainGetXMLDesc with
  VIR_DOMAIN_XML_UPDATE_CPU flag:
    <cpu mode='host-passthrough' match='minimal'>
      <model>Penryn</model>                        --+ copied from caps
      <vendor>Intel</vendor>                         | XML but doesn't
      <topology sockets='2' cores='4' threads='1'/>  | describe all
      <feature policy='require' name='dca'/>         | aspects of the
      <feature policy='require' name='xtpr'/>        | actual guest CPU
      ...                                          --+
    </cpu>

In case a hypervisor doesn't support the exact CPU model requested by a
domain XML, we automatically fallback to a closest CPU model the
hypervisor supports (and make sure we add/remove any additional features
if needed). This patch adds 'fallback' attribute to model element, which
can be used to disable this automatic fallback.

Add support for newly supported Intel cpu features. Newly supported
flags are: pclmuldq, dtes64, smx, fma, pdcm, movbe, xsave, osxsave and
avx. This adds support for Intel's Sandy Bridge platform.

To add support for running libvirt on PowerPC, a CPU driver for the
PowerPC platform must be added.
Most generic cpu driver routines such as CPU compare, decode, etc
are based on CPUID comparison and are not relevant for non-x86
platforms.
Here, we introduce stubs for relevant PowerPC routines invoked by libvirt.
Signed-off-by: NPrerna Saxena <prerna@linux.vnet.ibm.com>
Signed-off-by: NAnton Blanchard <anton@au.ibm.com>

Auto-indent makes life a bit easier; this patch also drops unused
arguments and replaces a misspelled flag name with two entry points
instead, so that callers don't have to worry about how much spacing
is present when embedding cpu elements.

* src/conf/cpu_conf.h (virCPUFormatFlags): Delete.
(virCPUDefFormat): Drop unused argument.
(virCPUDefFormatBuf): Alter signature.
(virCPUDefFormatBufFull): New prototype.
* src/conf/cpu_conf.c (virCPUDefFormatBuf): Split...
(virCPUDefFormatBufFull): ...into new function.
(virCPUDefFormat): Adjust caller.
* src/conf/domain_conf.c (virDomainDefFormatInternal): Likewise.
* src/conf/capabilities.c (virCapabilitiesFormatXML): Likewise.
* src/cpu/cpu.c (cpuBaselineXML): Likewise.
* tests/cputest.c (cpuTestCompareXML): Likewise.

Commit 498d7833 cleans up some of virtual file names for parsing strings
in memory. This patch cleans up (hopefuly) the rest forgotten by the
first patch.

This patch also changes all of the previously modified "filenames" to
valid URI's replacing spaces for underscores.

Changes to v1:
- Replace all spaces for underscores, so that the strings form valid
  URI's
- Replace spaces in places changed by commit 498d7833

Repetitive patterns should be factored.  The sign of a good
factorization is a change that kills 5x more lines than it adds :)

* src/conf/domain_conf.c (virDomainDeviceDefParse)
(virDomainSnapshotDefParseString): Use new convenience macros.
* src/conf/storage_conf.c (virStoragePoolDefParseSourceString):
Likewise.
* src/cpu/cpu.c (cpuCompareXML, cpuBaselineXML): Likewise.
* src/esx/esx_vi.c (esxVI_Context_Execute): Likewise.
* src/qemu/qemu_migration.c (qemuMigrationCookieXMLParseStr):
Likewise.
* src/security/virt-aa-helper.c (caps_mockup): Likewise.
* src/test/test_driver.c (testOpenFromFile): Likewise.
* tests/cputest.c (cpuTestLoadXML, cpuTestLoadMultiXML):
Likewise.
* tools/virsh.c (cmdFreecell, makeCloneXML, cmdVNCDisplay)
(cmdTTYConsole, cmdDetachInterface, cmdDetachDisk)
(cmdSnapshotCreate, cmdSnapshotCreateAs, cmdSnapshotCurrent)
(cmdSnapshotList, cmdSnapshotParent): Likewise.

These VIR_XXXX0 APIs make us confused, use the non-0-suffix APIs instead.

How do these coversions works? The magic is using the gcc extension of ##.
When __VA_ARGS__ is empty, "##" will swallow the "," in "fmt," to
avoid compile error.

example: origin				after CPP
	high_level_api("%d", a_int)	low_level_api("%d", a_int)
	high_level_api("a  string")	low_level_api("a  string")

About 400 conversions.

8 special conversions:
VIR_XXXX0("") -> VIR_XXXX("msg") (avoid empty format) 2 conversions
VIR_XXXX0(string_literal_with_%) -> VIR_XXXX(%->%%) 0 conversions
VIR_XXXX0(non_string_literal) -> VIR_XXXX("%s", non_string_literal)
  (for security) 6 conversions
Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>

We already have virAsprintf, so picking a similar name helps for
seeing a similar purpose.  Furthermore, the prefix V before printf
generally implies 'va_list', even though this variant was '...', and
the old name got in the way of adding a new va_list version.

global rename performed with:

$ git grep -l virBufferVSprintf \
  | xargs -L1 sed -i 's/virBufferVSprintf/virBufferAsprintf/g'

then revert the changes in ChangeLog-old.

And from all related macros and functions.

Two-argument free functions are uncommon; match the style elsewhere
by caching the callback at creation.

* src/util/hash.h (virHashCreate, virHashFree): Move deallocator
argument to creation.
* cfg.mk (useless_free_options): Add virHashFree.
* src/util/hash.c (_virHashTable): Track deallocator.
(virHashCreate, virHashFree): Update to new signature.
* src/conf/domain_conf.c (virDomainObjListDeinit)
(virDomainObjListInit, virDomainDiskDefForeachPath)
(virDomainSnapshotObjListDeinit, virDomainSnapshotObjListInit):
Update callers.
* src/conf/nwfilter_params.c (virNWFilterHashTableFree)
(virNWFilterHashTableCreate): Likewise.
* src/conf/nwfilter_conf.c (virNWFilterTriggerVMFilterRebuild):
Likewise.
* src/cpu/cpu_generic.c (genericHashFeatures, genericBaseline):
Likewise.
* src/xen/xm_internal.c (xenXMOpen, xenXMClose): Likewise.
* src/nwfilter/nwfilter_learnipaddr.c (virNWFilterLearnInit)
(virNWFilterLearnShutdown): Likewise.
* src/qemu/qemu_command.c (qemuDomainPCIAddressSetCreate)
(qemuDomainPCIAddressSetFree): Likewise.
* src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Likewise.

* src/cpu/cpu_x86.c (x86ModelLoad): Free data before overwriting.

So far, CPUID data were stored in two different data structures. First
of them was a structure allowing direct access for CPUID data according
to function number and the second was a plain array of struct
cpuX86cpuid. This was a silly design which resulted in converting data
from one type to the other and back again or implementing similar
functionality for both data structures.

The patch leaves only the direct access structure. This makes the code
both smaller and more maintainable since operations on different objects
can use common low-level operations.

All 57 tests for cpu subsystem still pass after this rewrite.

* bootstrap.conf (gnulib_modules): Add configmake.
* daemon/Makefile.am (libvirtd_CFLAGS): Drop defines provided by
gnulib.
* src/Makefile.am (INCLUDES): Likewise.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (virsh_CFLAGS): Likewise.
* daemon/libvirtd.c (qemudInitPaths, usage, main): Update
clients.
* src/cpu/cpu_map.c (CPUMAPFILE): Likewise.
* src/driver.c (DEFAULT_DRIVER_DIR): Likewise.
* src/internal.h (_): Likewise.
* src/libvirt.c (virInitialize): Likewise.
* src/lxc/lxc_conf.h (LXC_CONFIG_DIR, LXC_STATE_DIR, LXC_LOG_DIR):
Likewise.
* src/lxc/lxc_conf.c (lxcCapsInit, lxcLoadDriverConfig):
Likewise.
* src/network/bridge_driver.c (NETWORK_PID_DIR)
(NETWORK_STATE_DIR, DNSMASQ_STATE_DIR, networkStartup): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterDriverStartup):
Likewise.
* src/qemu/qemu_conf.c (qemudLoadDriverConfig): Likewise.
* src/qemu/qemu_driver.c (qemudStartup): Likewise.
* src/remote/remote_driver.h (LIBVIRTD_PRIV_UNIX_SOCKET)
(LIBVIRTD_PRIV_UNIX_SOCKET_RO, LIBVIRTD_CONFIGURATION_FILE)
(LIBVIRT_PKI_DIR): Likewise.
* src/secret/secret_driver.c (secretDriverStartup): Likewise.
* src/security/security_apparmor.c (VIRT_AA_HELPER): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/storage_backend_disk.c (PARTHELPER): Likewise.
* src/storage/storage_driver.c (storageDriverStartup): Likewise.
* src/uml/uml_driver.c (TEMPDIR, umlStartup): Likewise.
* src/util/hooks.c (LIBVIRT_HOOK_DIR): Likewise.
* tools/virsh.c (main): Likewise.
* docs/hooks.html.in: Likewise.

When only some host CPUs given to cpuBaseline contain <vendor> element,
baseline CPU should not contain it. Otherwise the result would not be
compatible with the host CPUs without vendor. CPU vendors are still
taken into account when computing baseline CPU, it's just removed from
the result.

Recent CPU models were specified using invalid vendor element
<vendor>NAME</vendor>, which was silently ignored due to a bug in the
code which was parsing it.