There's this question on the list that is asked over and over again.
How do I get {cpu, memory, ...} usage in percentage? Or its modified
version: How do I plot nice graphs like virt-manager does?

It would be nice if we have an example to inspire people. And that's
what domtop should do. Yes, it could be written in different ways, but
I've chosen this one as I think it show explicitly what users need to
implement in order to imitate virt-manager's graphing.

Note: The usage is displayed from host perspective. That is, how much
host CPUs the domain is using. But it should be fairly simple to
switch do just guest CPU usage if needed.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

There's no need to use it since we have this shiny functions
that even checks for conversion and overflow errors.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The virNodeParseSocket() function tries to get socked ID from
'topology/physical_package_id' file. However, on some architectures
the file contains the -1 constant which makes in turn libvirt think
the info extraction was unsuccessful. If that's the case, we need to
overwrite the obtained integer with zero like we are doing for other
architectures.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Roman Bogorodskiy reported a syntax-check failure when using
FreeBSD; complaining that:

prohibit_empty_first_line
tools/libvirt_win_icon_16x16.ico:1:
tools/libvirt_win_icon_32x32.ico:1:
tools/libvirt_win_icon_48x48.ico:1:
tools/libvirt_win_icon_64x64.ico:1:
maint.mk: Prohibited empty first line

In reality, the first 'line' of that file is NOT empty; but since
it is a binary file, awk is not required to handle it gracefully.
The simplest solution is to exempt all image files from syntax
checks in the first place - after all, we only store them in git
because they are inconvenient to regenerate, but they are not our
preferred format for making modifications, and syntax check should
only cover files that we are likely to modify.

* cfg.mk (VC_LIST_ALWAYS_EXCLUDE_REGEX): Exempt images.
(exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF): Simplify.
(exclude_file_name_regexp--sc_trailing_blank): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Based on discussion with Eric:

https://www.redhat.com/archives/libvir-list/2014-March/msg01001.htmlSigned-off-by: NMartin Kletzander <mkletzan@redhat.com>

I accidentally typed 'make' in the srcdir of a VPATH build, and
was surprised to see this:

$ make
/bin/sh: s/^[ +-]//;s/ .*//: No such file or directory
INFO: gnulib update required; running ./autogen.sh first
make: -n: Command not found
./autogen.sh
I am going to run ./configure with no arguments - if you wish
to pass any to it, please specify them on the ./autogen.sh command line.
running bootstrap...
./bootstrap: Bootstrapping from checked-out libvirt sources...
./bootstrap: getting gnulib files...

Oops - we're trying to execute some fairly bogus command names,
and then trying to configure in-tree (which breaks all existing
VPATH builds, since automake refuses to do a VPATH build if it
detects an in-tree configure).  The third line (executing "-n")
is fixed by updating to the latest gnulib; the rest of the problem
is fixed by copying the same filtering in our cfg.mk as what
gnulib just added, so that we avoid any $(shell) invocations which
in turn depend on variables that are only populated by a working
Makefile.  With that in place, we are back to the much nicer:

$ make
There seems to be no Makefile in this directory.
You must run ./configure before running 'make'.
make: *** [abort-due-to-no-makefile] Error 1

Additionally, although harder to see - there was a trailing space in
the message warning us that autogen would run an in-tree configure.

* .gnulib: Update to latest, in part for maint.mk improvements.
* cfg.mk (_update_required): Don't check for update in
unconfigured directory.
* autogen.sh (no_git): Drop trailing space.
Signed-off-by: NEric Blake <eblake@redhat.com>

Reusing the maint.mk code allows for a more efficient syntax check
(fewer grep processes), and a more compact representation of what
we are really checking for in commit 1919e355.

* cfg.mk (sc_require_locale_h): Use maint.mk loop instead of
rolling our own.
Signed-off-by: NEric Blake <eblake@redhat.com>

In the past we had some issues where setlocale() was called without
corresponding include of locale.h. While on some systems this may
work, on others the compilation failed. We should have a syntax-check
rule for that to prevent this from happening again.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

My future work will modify the metadata crawler function to use the
storage driver file APIs to access the files instead of accessing them
directly so that we will be able to request the metadata for remote
files too. To avoid linking the storage driver to every helper file
using the utils code, the backing chain traversal function needs to be
moved to the storage driver source.

Additionally the virt-aa-helper and virstoragetest programs need to be
linked with the storage driver as a result of this change.

Some syntax-check rules use GNU sed specific regexps, so make
sure we're using $(SED) instead of sed, which might not be a
GNU sed.

Now that all clients have been adjusted, ensure that no future
misuse of readdir is introduced into the code base.

* cfg.mk (sc_prohibit_readdir): New rule.
* src/util/virfile.c (virDirRead): Exempt the wrapper.
Signed-off-by: NEric Blake <eblake@redhat.com>

Use a syntax-check rule to force PCI, USB and SCSI instead
of Pci, Usb and Scsi.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In 'make syntax-check', we have a rule that prevents layering
violations between the various files in src.  However, we
forgot to treat conf/ and the more recently-added access/ as
lower-level directories, and were not detecting cases where
they might have used a driver file.  Also, it's not nice that
qemu can use storage/ but none of the other drivers could do so.

* cfg.mk (sc_prohibit_cross_inclusion): Tighten rules for conf/
and access/, let all other drivers use storage/.
Signed-off-by: NEric Blake <eblake@redhat.com>

Using any of these chars [:*?"<>|] in a filename is forbidden on
Windows and breaks git operations on Windows as git is not able
to create those files/directories on clone or pull.

Because some of them can be used in UNIX filenames they tend to
creep into filenames; especially : in PCI/SCSI device names that
are used as filenames in test cases.

Indent top-level labels by one space.

Add the rule to HACKING and enforce it by syntax-check.

Although not explicitly requested, we are using K&R (or Kernel)
indentation for curly braces around functions in HACKING file and most
of the code.  Using grep -P, this patch add the syntax-check rule for
it (while skipping all the false positives with foreach constructs).
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Just to align the backslashes with most of the file.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Reported by Martin Kletzander

Addition of vshConnect() makes virConnectOpen() functions obsolete in
virsh.  Thus all virsh-*.[ch] files should be left only with
vshConnect() in the case of need.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

This rule wouldn't be able to find any case of a hardcoded indent that
was in the middle of a string, but then virBuffer doesn't add
indentation in the middle of a string either.

Some of these are leftovers from renaming the files, others
are just typos.

Also introduce an ugly awk script to enforce this.

Mock the /sys/bus/usb directory and test the finding
(and not finding) of some USB devices.

The previous OOM testing support would re-run the entire "main"
method each iteration, failing a different malloc each time.
When a test suite has 'n' allocations, the number of repeats
requires is  (n * (n + 1) ) / 2.  This gets very large, very
quickly.

This new OOM testing support instead integrates at the
virtTestRun level, so each individual test case gets repeated,
instead of the entire test suite. This means the values of
'n' are orders of magnitude smaller.

The simple usage is

   $ VIR_TEST_OOM=1 ./qemuxml2argvtest
   ...
   29) QEMU XML-2-ARGV clock-utc                                         ... OK
       Test OOM for nalloc=36 .................................... OK
   30) QEMU XML-2-ARGV clock-localtime                                   ... OK
       Test OOM for nalloc=36 .................................... OK
   31) QEMU XML-2-ARGV clock-france                                      ... OK
       Test OOM for nalloc=38 ...................................... OK
   ...

the second lines reports how many mallocs have to be failed, and thus
how many repeats of the test will be run.

If it crashes, then running under valgrind will often show the problem

  $ VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest

When debugging problems it is also helpful to select an individual
test case

  $ VIR_TEST_RANGE=30 VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest

When things get really tricky, it is possible to request that just
specific allocs are failed. eg to fail allocs 5 -> 12, use

  $ VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-12 ../run valgrind ./qemuxml2argvtest

In the worse case, you might want to know the stack trace of the
alloc which was failed then VIR_TEST_OOM_TRACE can be set. If it
is set to 1 then it will only print if it thinks a mistake happened.
This is often not reliable, so setting it to 2 will make it print
the stack trace for every alloc that is failed.

  $ VIR_TEST_OOM_TRACE=2 VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-5 ../run valgrind ./qemuxml2argvtest
  30) QEMU XML-2-ARGV clock-localtime                                   ... OK
      Test OOM for nalloc=36 !virAllocN
  /home/berrange/src/virt/libvirt/src/util/viralloc.c:180
  virHashCreateFull
  /home/berrange/src/virt/libvirt/src/util/virhash.c:144
  virDomainDefParseXML
  /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11745
  virDomainDefParseNode
  /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12646
  virDomainDefParse
  /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12590
  testCompareXMLToArgvFiles
  /home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:106
  virtTestRun
  /home/berrange/src/virt/libvirt/tests/testutils.c:250
  mymain
  /home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:418 (discriminator 2)
  virtTestMain
  /home/berrange/src/virt/libvirt/tests/testutils.c:750
  ??
  ??:0
  _start
  ??:?
   FAILED
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use the new storage driver APIs to delete snapshot backing files in case
of failure instead of directly relying on "unlink". This will help us in
the future when we will be adding network based storage without local
representation in the host.

Introduce Wireshark dissector plugin which adds support to Wireshark
for dissecting libvirt RPC protocol.
Added following files to build Wireshark dissector from libvirt source
tree.
* tools/wireshark/*: Source tree of Wireshark dissector plugin.

Added followings to configure.ac or Makefile.am.
configure.ac
* --with-wireshark-dissector: Enable support for building Wireshark
  dissector.
* --with-ws-plugindir: Specify wireshark plugin directory that dissector
  will installed.
* Added tools/wireshark/{Makefile,src/Makefile} to  AC_CONFIG_FILES.
Makefile.am
* Added tools/wireshark/ to SUBDIR.

Finish the cleanup of libvirt.c; all uses of virLib*Error have
now been converted to more canonical conventions.

* src/libvirt.c: Use virReportError in remaining errors.
(virLibConnError, virLibDomainError): Delete unused macros.
* cfg.mk (msg_gen_function): Drop unused names.
Signed-off-by: NEric Blake <eblake@redhat.com>

This partially reverts 5eb4b042 and 62774afb.

Rewrite the domsuspend example from scratch. This time do it right.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The domain events demo program isn't really tied to domain
events anymore, so rename it to object events.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Added a parent class virObjectEvent for future event types

Leave virDomainEventRegister and its Deregister brother as these are
legacy functions only for domain lifecycle events.

Kill the use of atoi() and introduce syntax check to forbid it and it's
friends (atol, atoll, atof, atoq).

Also fix a typo in variable name holding the cylinders count of a disk
pool (apparently unused).

examples/domsuspend/suspend.c will need a larger scale refactor as the
whole example file is broken thus it will be exempted from the syntax
check for now.

The python binding now lives in

  http://libvirt.org/git/?p=libvirt-python.git

that repo also provides an RPM which is upgrade compatible
with the old libvirt-python sub-RPM.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This addition, however, requires some refactoring to be done.  First of
all, to match the best practice we should detach the device prior
resetting it. That's why testVirPCIDeviceDetach is detaching all devices
within 0000:00:01.0 and 0000:00:03.0 range. Then, the brand new test
will reset the 0000:00:02.0 device, so the last testVirPCIDeviceReattach
can reattach all the devices back.

In order to perform a PCI device reset, the dummy config file is not
sufficient anymore and must be replaced with real PCI config (binary
mess). Such config files are to be stored under tests/virpcitestdata/
and ought to have '.config' suffix.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This commit introduces yet another test under virpcitest:
virPCIDeviceDetach. However, in order to be able to do this, the
virpcimock needs to be extended to model the kernel behavior on PCI
device binding and unbinding (create 'driver' symlinks under the device
tree, check for device ID in driver's ID table, etc.)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Among with this test introduce virpcimock as we need to mock some
syscalls, e.g. redirect open() of a file under /sys/bus/pci to a
stub sysfs tree.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

To make it easier to forbid future attempts at a confusing typedef
name ending in Ptr that isn't actually a pointer, insist that we
follow our preferred style of 'typedef foo *fooPtr'.

* cfg.mk (sc_forbid_const_pointer_typedef): Enforce consistent
style, to prevent issue fixed in previous storage patch.
* src/conf/capabilities.h (virCapsPtr): Fix offender.
* src/security/security_stack.c (virSecurityStackItemPtr):
Likewise.
* tests/qemucapabilitiestest.c (testQemuDataPtr): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

On RHEL 5, make syntax-check was failing because even strings like
'int isTempChain' matched the 'int i' rule.  To be honest, I haven't
found the root cause, but the change added makes it work as expected
and keeps the proper behavior on newer systems as well.

The use of getenv is typically insecure, and we want people
to use our wrappers, to force them to think about setuid
needs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

'const fooPtr' is the same as 'foo * const' (the pointer won't
change, but it's contents can).  But in general, if an interface
is trying to be const-correct, it should be using 'const foo *'
(the pointer is to data that can't be changed).

Now that the code base has been cleaned, enforce it with a syntax
checker.

* cfg.mk (sc_forbid_const_pointer_typedef): New rule.
Signed-off-by: NEric Blake <eblake@redhat.com>