The current virRandomBits() API is only usable if the caller wants
a random number in the range [0, n-1) where n is a power of two.
This adds a virRandom() API which generates a double in the
range [0.0,1.0) with 48 bits of entropy. It then also adds a
virRandomInt(uint32_t max) API which generates an unsigned
in the range [0,@max)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Fixes for some typos that somehow didn't get to the final push of the
commit 43bfa23e.

libvirt creates invalid commands if wrong locale is selected. For
example with locale that uses comma as a decimal point, JSON commands
created with decimal numbers are invalid because comma separates the
entries in JSON. Fortunately even when decimal point is affected,
thousands grouping is not, because for grouping to be enabled with
*printf, there has to be an apostrophe flag specified (and supported).

This patch adds specific internal function for converting doubles to
strings with C locale.

This patch introduces a new error code VIR_ERR_OPERATION_UNSUPPORTED to
mark error messages regarding operations that failed due to lack of
support on the hypervisor or other than libvirt issues.

The code is first used in reporting error if qemu does not support block
IO tuning variables yielding error message:
error: Unable to get block I/O throttle parameters
error: Operation not supported: block_io_throttle field
'total_bytes_sec' missing in qemu's output

instead of:
error: Unable to get block I/O throttle parameters
error: internal error cannot read total_bytes_sec

Otherwise, in locations like virobject.c where PROBE is used,
for certain configure options, the compiler warns:

util/virobject.c:110:1: error: 'intptr_t' undeclared (first use in this function)

As long as we are making this header always available, we can
clean up several other files.

* src/internal.h (includes): Pull in <stdint.h>.
* src/conf/nwfilter_conf.h: Rely on internal.h.
* src/storage/storage_backend.c: Likewise.
* src/storage/storage_backend.h: Likewise.
* src/util/cgroup.c: Likewise.
* src/util/sexpr.h: Likewise.
* src/util/virhashcode.h: Likewise.
* src/util/virnetdevvportprofile.h: Likewise.
* src/util/virnetlink.h: Likewise.
* src/util/virrandom.h: Likewise.
* src/vbox/vbox_driver.c: Likewise.
* src/xenapi/xenapi_driver.c: Likewise.
* src/xenapi/xenapi_utils.c: Likewise.
* src/xenapi/xenapi_utils.h: Likewise.
* src/xenxs/xenxs_private.h: Likewise.
* tests/storagebackendsheepdogtest.c: Likewise.

Both LVM volumes and SCSI LUNs have a globally unique
identifier associated with them. It is useful to be able
to query this identifier to then perform disk locking,
rather than try to figure out a stable pathname.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Prevents libvirt from treating RBD backing stores as files. Without this
patch, creating a domain with a qcow2 overlay on an RBD would fail.

This patch essentially extends 9c7c4a4f,
which allows nbd backing stores, to allow rbd backing stores.

From man poll(2), poll does not set errno=EAGAIN on interrupt, however
it does set errno=EINTR. Have libvirt retry on the appropriate errno.

Under heavy load, a program of mine kept getting libvirt errors 'poll on
socket failed: Interrupted system call'. The signals were SIGCHLD from
processes forked by threads unrelated to those using libvirt.

This patch is in response to:

  https://bugzilla.redhat.com/show_bug.cgi?id=818467

If a caller to virCommandRun doesn't ask for the exitstatus of the
program it's running, the virCommand functions assume that they should
log an error message and return failure if the exit code isn't
0. However, only the commandline and exit status are logged, while
potentially useful information sent by the program to stderr is
discarded.

Fortunately, virCommandRun is already checking if the caller had asked
for stderr to be saved and, if not, sets things up to save it in
*cmd->errbuf. This makes it fairly simple for virCommandWait to
include *cmd->errbuf in the error log (there are still other callers
that don't setup errbuf, and even virCommandRun won't set it up if the
command is being daemonized, so we have to check that it's non-zero).

This introduces a fairly basic reference counted virObject type
and an associated virClass type, that use atomic operations for
ref counting.

In a global initializer (recommended to be invoked using the
virOnceInit API), a virClass type must be allocated for each
object type. This requires a class name, a "dispose" callback
which will be invoked to free memory associated with the object's
fields, and the size in bytes of the object struct.

eg,

   virClassPtr  connclass = virClassNew("virConnect",
                                        sizeof(virConnect),
                                        virConnectDispose);

The struct for the object, must include 'virObject' as its
first member

eg

  struct _virConnect {
    virObject object;

    virURIPtr uri;
  };

The 'dispose' callback is only responsible for freeing
fields in the object, not the object itself. eg a suitable
impl for the above struct would be

  void virConnectDispose(void *obj) {
     virConnectPtr conn = obj;
     virURIFree(conn->uri);
  }

There is no need to reset fields to 'NULL' or '0' in the
dispose callback, since the entire object will be memset
to 0, and the klass pointer & magic integer fields will
be poisoned with 0xDEADBEEF before being free()d

When creating an instance of an object, one needs simply
pass the virClassPtr eg

   virConnectPtr conn = virObjectNew(connclass);
   if (!conn)
      return NULL;
   conn->uri = virURIParse("foo:///bar")

Object references can be manipulated with

   virObjectRef(conn)
   virObjectUnref(conn)

The latter returns a true value, if the object has been
freed (ie its ref count hit zero)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

All callers used the same initialization seed (well, the new
viratomictest forgot to look at getpid()); so we might as well
make this value automatic.  And while it may feel like we are
giving up functionality, I documented how to get it back in the
unlikely case that you actually need to debug with a fixed
pseudo-random sequence.  I left that crippled by default, so
that a stray environment variable doesn't cause a lack of
randomness to become a security issue.

* src/util/virrandom.c (virRandomInitialize): Rename...
(virRandomOnceInit): ...and make static, with one-shot call.
Document how to do fixed-seed debugging.
* src/util/virrandom.h (virRandomInitialize): Drop prototype.
* src/libvirt_private.syms (virrandom.h): Don't export it.
* src/libvirt.c (virInitialize): Adjust caller.
* src/lxc/lxc_controller.c (main): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/util/iohelper.c (main): Likewise.
* tests/seclabeltest.c (main): Likewise.
* tests/testutils.c (virtTestMain): Likewise.
* tests/viratomictest.c (mymain): Likewise.

There are a few issues with the current virAtomic APIs

 - They require use of a virAtomicInt struct instead of a plain
   int type
 - Several of the methods do not implement memory barriers
 - The methods do not implement compiler re-ordering barriers
 - There is no Win32 native impl

The GLib library has a nice LGPLv2+ licensed impl of atomic
ops that works with GCC, Win32, or pthreads.h that addresses
all these problems. The main downside to their code is that
the pthreads impl uses a single global mutex, instead of
a per-variable mutex. Given that it does have a Win32 impl
though, we don't expect anyone to seriously use the pthread.h
impl, so this downside is not significant.

* .gitignore: Ignore test case
* configure.ac: Check for which atomic ops impl to use
* src/Makefile.am: Add viratomic.c
* src/nwfilter/nwfilter_dhcpsnoop.c: Switch to new atomic
  ops APIs and plain int datatype
* src/util/viratomic.h: inline impls of all atomic ops
  for GCC, Win32 and pthreads
* src/util/viratomic.c: Global pthreads mutex for atomic
  ops
* tests/viratomictest.c: Test validate to validate safety
  of atomic ops.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Remove the use of a manually run virLogStartup and
virNodeSuspendInitialize methods. Instead make sure they
are automatically run using VIR_ONCE_GLOBAL_INIT
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add function virCommandNewVAList which is equivalent to the
virCommandNewArgList but with va_list instead of a variable number
of arguments.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Parallels Cloud Server is a cloud-ready virtualization
solution that allows users to simultaneously run multiple virtual
machines and containers on the same physical server.

More information can be found here: http://www.parallels.com/products/pcs/
Also beta version of Parallels Cloud Server can be downloaded there.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

This is a follow up patch of commit f9ce7dad, it modifies all
the files which declare the copyright like "See COPYING.LIB for
the License of this software" to use the detailed/consistent one.

And deserts the outdated comments like:

 * libvirt-qemu.h:
 * Summary: qemu specific interfaces
 * Description: Provides the interfaces of the libvirt library to handle
 *              qemu specific methods
 *
 * Copy:  Copyright (C) 2010, 2012 Red Hat, Inc.

Uses the more compact style like:

 * libvirt-qemu.h: Interfaces specific for QEMU/KVM driver
 *
 * Copyright (C) 2010, 2012 Red Hat, Inc.

Change the permissible minimum value of nodesuspend duration time
to 60 seconds. If option is less than the value, reports error.
Update virsh help and manpage the infomation.

virNetDevTapCreateInBridgePort: Fix return value to -1
virNetDevTapCreate: Fix comments

No check for conn->uri being NULL in virAuthGetConfigFilePath (valid
state) made the client segfault. This happens for example with these
settings:
 - no virtualbox driver installed (modifies conn->uri)
 - no default URI set (VIRSH_DEFAULT_CONNECT_URI="",
   LIBVIRT_DEFAULT_URI="", uri_default="")
 - auth_sock_rw="sasl"
 - virsh run as root

That are unfortunately the settings with fresh Fedora 17 installation
with VDSM.

The check ought to be enough as conn->uri being NULL is valid in later
code and is handled properly.

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

When reporting a system error (VIR_ERR_SYSTEM_ERROR) via
virReportSystemError, we should copy the errno value into
the 'int1' field of the virErrorPtr struct. This allows
callers to detect certain errno conditions & discard the
error

* src/util/virterror.c: Place errno value in int1 field

ensures that initialization will always take place when it is
needed, and guarantees it only occurs once. The problem is that
the code to setup a global initializer with proper error
propagation is tedious. This introduces VIR_ONCE_GLOBAL_INIT
macro to simplify this.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Detected on Cygwin.  Broken in commit 387117ad.

* src/util/virnetdev.c (virNetDevValidateConfig)
(virNetDevReplaceNetConfig): Fix prototypes.
* src/util/virnetlink.c (virNetlinkEventAddClient)
(virNetlinkEventRemoveClient): Likewise.

This removes nearly all the per-file error reporting macros
from the code in src/util/. A few custom macros remain for the
case, where the file needs to report errors with a variety of
different codes or parameters
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virnetdevtap.c and viruri.c files had two error report
messages which were not annotated with _(...)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Nearly every source file does something like

  #define VIR_FROM_THIS VIR_FROM_FOO
  #define virFooReportErorr(code, ...) \
     virReportErrorHelper(VIR_FROM_THIS, code,  __FILE__,    \
                          __FUNCTION__, __LINE__, \
                          __VA_ARGS__)

This creates needless duplication and inconsistent error
reporting function names in each file. It is trivial to
just have virterror_internal.h provide a virReportError
macro that is equivalent

* src/util/virterror_internal.h: Define virReportError(code, ...)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce new members in the virMacAddr 'class'
- virMacAddrSet: set virMacAddr from a virMacAddr
- virMacAddrSetRaw: setting virMacAddr from raw 6 byte MAC address buffer
- virMacAddrGetRaw: writing virMacAddr into raw 6 byte MAC address buffer
- virMacAddrCmp: comparing two virMacAddr
- virMacAddrCmpRaw: comparing a virMacAddr with a raw 6 byte MAC address buffer

then replace raw MAC addresses by replacing

- 'unsigned char *' with virMacAddrPtr
- 'unsigned char ... [VIR_MAC_BUFLEN]' with virMacAddr

and introduce usage of above functions where necessary.

When building with --disable-debug, VIR_DEBUG expands to a nop.
But parameters to VIR_DEBUG can be variables that are passed only
to VIR_DEBUG. In the case the building system complains about unused
variables.

All the callers of virDirCreate are updated incidentally.

Instead of changing the existed virFileMakePath to accept mode
argument and modifying a pile of its uses, this patch introduces
virFileMakePathWithMode, and use it instead of mkdir() to create
the readline history dir.

Commit 9612e4b2 introduced a typo and unused variable that break
non-Linux builds.

* src/util/virfile.c (virFileLoopDeviceAssociate): Fix syntax error.

While it is not currently used elsewhere in libvirt, the code
for finding a free loop device & associating a file with it
is not LXC specific. Move it into the viffile.{c,h} file where
potentially shared code is more commonly kept.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Hello,

This is a patch to fix vm's outbound traffic control problem.

Currently, vm's outbound traffic control by libvirt doesn't go well.
This problem was previously discussed at libvir-list ML, however
it seems that there isn't still any answer to the problem.
http://www.redhat.com/archives/libvir-list/2011-August/msg00333.html

I measured Guest(with virtio-net) to Host TCP throughput with the
command "netperf -H".
Here are the outbound QoS parameters and the results.

outbound average rate[kilobytes/s] : Guest to Host throughput[Mbit/s]
======================================================================
1024  (8Mbit/s)                    : 4.56
2048  (16Mbit/s)                   : 3.29
4096  (32Mbit/s)                   : 3.35
8192  (64Mbit/s)                   : 3.95
16384 (128Mbit/s)                  : 4.08
32768 (256Mbit/s)                  : 3.94
65536 (512Mbit/s)                  : 3.23

The outbound traffic goes down unreasonably and is even not controled.

The cause of this problem is too large mtu value in "tc filter" command run by
libvirt. The command uses burst value to set mtu and the burst is equal to
average rate value if it's not set. This value is too large. For example
if the average rate is set to 1024 kilobytes/s, the mtu value is set to 1024
kilobytes. That's too large compared to the size of network packets.
Here libvirt applies tc ingress filter to Host's vnet(tun) device.
Tc ingress filter is implemented with TBF(Token Buckets Filter) algorithm. TBF
uses mtu value to calculate the amount of token consumed by each packet. With too
large mtu value, the token consumption rate is set too large. This leads to
token starvation and deterioration of TCP throughput.

Then, should we use the default mtu value 2 kilobytes?
The anser is No, because Guest with virtio-net device uses 65536 bytes
as mtu to transmit packets to Host, and the tc filter with the default mtu
value 2k drops packets whose size is larger than 2k. So, the most packets
is droped and again leads to deterioration of TCP throughput.

The appropriate mtu value is 65536 bytes which is equal to the maximum value
of network interface device defined in <linux/netdevice.h>. The value is
not so large that it causes token starvation and not so small that it
drops most packets.
Therefore this patch set the mtu value to 64kb(== 65535 bytes).

Again, here are the outbound QoS parameters and the TCP throughput with
the libvirt patched.

outbound average rate[kilobytes/s] : Guest to Host throughput[Mbit/s]
======================================================================
1024  (8Mbit/s)                    : 8.22
2048  (16Mbit/s)                   : 16.42
4096  (32Mbit/s)                   : 32.93
8192  (64Mbit/s)                   : 66.85
16384 (128Mbit/s)                  : 133.88
32768 (256Mbit/s)                  : 271.01
65536 (512Mbit/s)                  : 547.32

The outbound traffic conforms to the given limit.

Thank you,
Signed-off-by: NEiichi Tsukata <eiichi.tsukata.xh@hitachi.com>

The ignore_value macro is used across libvirt. This patch includes it in
the internal header and cleans all other includes.

In order to retrieve some sysinfo data we need to parse /proc/sysinfo and
/proc/cpuinfo.
Signed-off-by: NThang Pham <thang.pham@us.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Some GNULIB headers (eg unistd.h) will often need to include
winsock2.h for various symbols. There is a rule that winsock2.h
must be included before windows.h. This means that any file
which does

  #ifdef WIN32
  #include <windows.h>
  #endif
  #include <unistd.h>

is potentially broken. A simple rule is that /all/ includes of
windows.h must be matched with a preceding include of winsock2.h
regardless of whether unistd.h is used currently
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

src/util/util.c: virFileOpenAs.

To facilitate future troubleshooting add a bunch more debugging
statements into important startup parts of libvirt