A couple of code paths check the lower bits of CPUTLBEntry::addr_write
against io_mem_ram as a way of looking for a dirty RAM page.  This works
by accident since the value is zero, which matches all clear bits for
TLB_INVALID, TLB_MMIO, and TLB_NOTDIRTY (indicating dirty RAM).

Make it work by design by checking for the proper bits.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Optionally, make memory access helpers take a parameter for CPUState
instead of relying on global env.

On most targets, perform simple moves to reorder registers. On i386,
switch from regparm(3) calling convention to standard stack-based
version.
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

Scripted conversion:
  for file in *.[hc] hw/*.[hc] hw/kvm/*.[hc] linux-user/*.[hc] linux-user/m68k/*.[hc] bsd-user/*.[hc] darwin-user/*.[hc] tcg/*/*.[hc] target-*/cpu.h; do
    sed -i "s/CPUState/CPUArchState/g" $file
  done

All occurrences of CPUArchState are expected to be replaced by QOM CPUState,
once all targets are QOM'ified and common fields have been extracted.
Signed-off-by: NAndreas Färber <afaerber@suse.de>
Reviewed-by: NAnthony Liguori <aliguori@us.ibm.com>

The return value of cpu_register_io_memory() is no longer used anywhere, so
we can remove it and all associated data and code.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Instead of indirecting via io_mem_region, dispatch directly
through the MemoryRegion obtained from the iotlb or phys_page_find().
Signed-off-by: NAvi Kivity <avi@redhat.com>

get_page_addr_code() reads a code tlb entry, but interprets it as an
iotlb entry.  This works by accident since the low bits of a RAM code
tlb entry are clear, and match a RAM iotlb entry.  This accident is
about to unhappen, so fix the code to use an iotlb entry (using the
code entry with TLB_MMIO may fail if the page is a watchpoint).
Signed-off-by: NAvi Kivity <avi@redhat.com>

A step towards eliminating io indices.
Signed-off-by: NAvi Kivity <avi@redhat.com>

We'd like to store the section index in the iotlb, so we can't
adjust it before returning.  Return an unadjusted section and
instead introduce section_addr(), which does the adjustment later.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Commit e58ac72b6a0 ("ioport: change portio_list not to use
memory_region_set_offset()") started using aliases of I/O memory
regions.  Since the IORange used for the I/O was contained in the
target region, the alias information (specifically, the offset
into the region) was lost.  This broke -vga std.

Fix by allocating an independent object to hold the IORange and
also the new offset.

Note that I/O memory regions were conceptually broken wrt aliases
in a different way: an alias can cause the same region to appear
twice in an address space, but we had just one IORange to service it.
This patch fixes that problem as well, since we can now have multiple
IORange/MemoryRegion associations.
Signed-off-by: NAvi Kivity <avi@redhat.com>

When storing large contiguous ranges in phys_map, all values tend to
be the same pointers to a single MemoryRegionSection.  Collapse them
by marking nodes with level > 0 as leaves.  This reduces tree memory
usage dramatically.
Signed-off-by: NAvi Kivity <avi@redhat.com>

They have the same type, unify them.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NAvi Kivity <avi@redhat.com>

Setting multiple pages at once requires backtracking to previous
nodes; easiest to achieve via recursion.
Signed-off-by: NAvi Kivity <avi@redhat.com>

By giving the function the value we want to set, we make it
more flexible for the next patch.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Instead of considering subpage on a per-page basis, split each section
into a subpage head, multipage body, and subpage tail, and register
each separately.  This simplifies the registration functions.
Signed-off-by: NAvi Kivity <avi@redhat.com>

We'll change phys_page_find_alloc() soon, but phys_page_find()
doesn't need to bear the consequences.
Signed-off-by: NAvi Kivity <avi@redhat.com>

We no longer describe memory in terms of individual pages; use sections
throughout instead.

PhysPageDesc no longer used - remove.
Signed-off-by: NAvi Kivity <avi@redhat.com>

This way, if we have several changes in a single transaction, we flush just
once.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Identical except that the second branch knows its not modifying an existing
subpage.
Signed-off-by: NAvi Kivity <avi@redhat.com>

If the first subpage installed in a page is RAM, then we install it as
a full page, instead of a subpage.  Fix by not special casing RAM.

The issue dates to commit db7b5426, which introduced subpages.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Use an expanding vector to store nodes.  Allocation is baroque to g_renew()
potentially invalidating pointers; this will be addressed later.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Instead of storing PhysPageDesc, store pointers to MemoryRegionSections.
The various offsets (phys_offset & ~TARGET_PAGE_MASK,
PHYS_OFFSET & TARGET_PAGE_MASK, region_offset) can all be synthesized
from the information in a MemoryRegionSection.  Adjust phys_page_find()
to synthesize a PhysPageDesc.

The upshot is that phys_map now contains uniform values, so it's easier
to generate and compress.

The end result is somewhat clumsy but this will be improved as we we
propagate MemoryRegionSections throughout the code instead of transforming
them to PhysPageDesc.

The MemoryRegionSection pointers are stored as uint16_t offsets in an
array.  This saves space (when we also compress node pointers) and is
more cache friendly.
Signed-off-by: NAvi Kivity <avi@redhat.com>

This lays the groundwork for storing leaf data in intermediate levels,
saving space.
Signed-off-by: NAvi Kivity <avi@redhat.com>

L1 and the lower levels in l1_phys_map are equivalent, except that L1 has
a different size, and is always allocated.  Simplify the code by removing
L1.  This leaves us with a tree composed solely of L2 tables, but that
problem can be renamed away later.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Instead of incrementally building the memory map, rebuild it every time.
This allows later simplification, since the code need not consider overlaying
a previous mapping.  It is also RCU friendly.

With large memory guests this can get expensive, since the operation is
O(mem size), but this will be optimized later.

As a side effect subpage and L2 leaks are fixed here.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Current memory listeners are incremental; that is, they are expected to
maintain their own state, and receive callbacks for changes to that state.

This patch adds support for stateless listeners; these work by receiving
a ->begin() callback (which tells them that new state is coming), a
sequence of ->region_add() and ->region_nop() callbacks, and then a
->commit() callback which signifies the end of the new state.  They should
ignore ->region_del() callbacks.
Signed-off-by: NAvi Kivity <avi@redhat.com>

The memory and I/O address spaces do different things, so split them into
two memory listeners.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Ignore any regions not belonging to a specified address space.
Signed-off-by: NAvi Kivity <avi@redhat.com>

This transforms memory.c into a library which can then be unit tested
easily, by feeding it inputs and listening to its outputs.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>

It can be derived from the MemoryRegion itself (which is why it is not
used there).
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>

Make cpu_watchpoint_insert accept watchpoints of any power-of-two size
up to the target page size.
Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>

In case of BP_STOP_BEFORE_ACCESS watchpoint check_watchpoint intends to
signal EXCP_DEBUG exception on exit from cpu loop, but later overwrites
exception code by the cpu_resume_from_signal call.

Use cpu_loop_exit with BP_STOP_BEFORE_ACCESS watchpoints.
Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>

Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Reviewed-by: NMeador Inge <meadori@codesourcery.com>

Clarify the comment about tlb_flush()'s flush_global parameter,
so it is clearer what it does and why it is OK that the implementation
currently ignores it.
Reviewed-by: NAndreas F=C3=A4rber <afaerber@suse.de>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The virtio config area in PIO space is a bit special. The initial
header is little endian but the rest (device specific) is guest
native endian.

The PIO accessors for PCI on machines that don't have native IO ports
assume that all PIO is little endian, which works fine for everything
except the above.

A complicated way to fix it would be to split the BAR into two memory
regions with different endianess settings, but this isn't practical
to do, besides, the PIO code doesn't honor region endianness anyway
(I have a patch for that too but it isn't necessary at this stage).

So I decided to go for the quick fix instead which consists of
reverting the swap in virtio-pci in selected places, hoping that when
we eventually do a "v2" of the virtio protocols, we sort that out once
and for all using a fixed endian setting for everything.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NAlexander Graf <agraf@suse.de>
[agraf: keep virtio in libhw and determine endianness through a
        helper function in exec.c]
Reviewed-by: NAnthony Liguori <aliguori@us.ibm.com>

ARM still doesn't support 16GB buffers in 32-bit modes, replace the
16GB by 16MB in the comment.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

We no longer use any of the lower bits of a ram_addr, so we might as well
use them for the io table index.  This increases the number of potential
I/O handlers by a factor of 8.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>

Unlike ->readonly, ->readable is not inherited from aliase, so we can simply
query the memory region.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>

Replace with a MemoryRegion flag.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>

Now that all mmio goes through MemoryRegions, we can convert
io_mem_opaque to be a MemoryRegion pointer, and remove the thunks
that convert from old-style CPU{Read,Write}MemoryFunc to MemoryRegionOps.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>