Move it to the actual users.  There are some inclusions of
qemu/host-utils.h in headers, but they are all necessary.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 1454089805-5470-16-git-send-email-peter.maydell@linaro.org

A number of files were including strings.h but not using any
of the functions it provides
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Rebuild the cache_is_cached function by cache_get_by_addr. And
drops the asserts because the caller is also asserting the same
thing.
Signed-off-by: NChenLiang <chenliang88@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ran in
vm:(migrate bandwidth:1GE and xbzrle cache size 8MB)

the test program:

include <stdlib.h>
include <stdio.h>
int main()
 {
        char *buf = (char *) calloc(4096, 4096);
        while (1) {
            int i;
            for (i = 0; i < 4096 * 4; i++) {
                buf[i * 4096 / 4]++;
            }
            printf(".");
        }
 }

before this patch:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":1020,"xbzrle-cache":{"bytes":1108284,
"cache-size":8388608,"cache-miss-rate":0.987013,"pages":18297,"overflow":8,
"cache-miss":1228737},"status":"active","setup-time":10,"total-time":52398,
"ram":{"total":12466991104,"remaining":1695744,"mbps":935.559472,
"transferred":5780760580,"dirty-sync-counter":271,"duplicate":2878530,
"dirty-pages-rate":29130,"skipped":0,"normal-bytes":5748592640,
"normal":1403465}},"id":"libvirt-706"}

18k pages sent compressed in 52 seconds.
cache-miss-rate is 98.7%, totally miss.

after optimizing:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":2054,"xbzrle-cache":{"bytes":5066763,
"cache-size":8388608,"cache-miss-rate":0.485924,"pages":194823,"overflow":0,
"cache-miss":210653},"status":"active","setup-time":11,"total-time":18729,
"ram":{"total":12466991104,"remaining":3895296,"mbps":937.663549,
"transferred":1615042219,"dirty-sync-counter":98,"duplicate":2869840,
"dirty-pages-rate":58781,"skipped":0,"normal-bytes":1588404224,
"normal":387794}},"id":"libvirt-266"}

194k pages sent compressed in 18 seconds.
The value of cache-miss-rate decrease to 48.59%.
Signed-off-by: NChenLiang <chenliang88@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

We call g_free() after cache_fini() in migration_end(), but we don't
call it after cache_fini() in xbzrle_cache_resize(), leaking the
memory.

cache_init() and cache_fini() are a pair.  Since cache_init()
allocates the cache, let cache_fini() free it.  This plugs the leak.
Signed-off-by: NChen Gang <gang.chen.5i5j@gmail.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Push zero'd pages into the XBZRLE cache
    A page that was cached by XBZRLE, zero'd and then XBZRLE'd again
    was being compared against a stale cache value

Don't use 'qemu_put_buffer_async' to put pages from the XBZRLE cache
    Since the cache might change before the data hits the wire
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It is better to fail migration in case of failure to
allocate new cache item
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Some source files #include the same header more than
once for no good reason.  Remove second #includes in
such cases.
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

The page cache frees all data on finish, on resize and
if there is collision on insert. So it should be the caches
responsibility to dup the data that is stored in the cache.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

XBZRLE encoded migration introduced a MRU page cache
meachnism. Unfortunately, cached items where never freed in
case of a collision in the page cache on cache_insert().

This lead to out of memory conditions during XBZRLE migration
if the page cache was small and there where a lot of collisions
in the cache.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Instead of using cache_insert do the update itself
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Add MRU page cache mechanism.
The page are accessed by their address.
Signed-off-by: NBenoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: NPetter Svard <petters@cs.umu.se>
Signed-off-by: NAidan Shribman <aidan.shribman@sap.com>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>