The address of the gpte was already calculated and stored in ptep_user
before entering cmpxchg_gpte().

This patch makes cmpxchg_gpte() to use that to make it clear that we
are using the same address during walk_addr_generic().

Note that the unlikely annotations are used to show that the conditions
are something unusual rather than for performance.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NMarcelo Tosatti <mtosatti@redhat.com>

Like the following, mmu_notifier can be called after registering
immediately. So, kvm have to initialize kvm->mmu_lock before it.

BUG: spinlock bad magic on CPU#0, kswapd0/342
 lock: ffff8800af8c4000, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
Pid: 342, comm: kswapd0 Not tainted 2.6.39-rc5+ #1
Call Trace:
 [<ffffffff8118ce61>] spin_bug+0x9c/0xa3
 [<ffffffff8118ce91>] do_raw_spin_lock+0x29/0x13c
 [<ffffffff81024923>] ? flush_tlb_others_ipi+0xaf/0xfd
 [<ffffffff812e22f3>] _raw_spin_lock+0x9/0xb
 [<ffffffffa0582325>] kvm_mmu_notifier_clear_flush_young+0x2c/0x66 [kvm]
 [<ffffffff810d3ff3>] __mmu_notifier_clear_flush_young+0x2b/0x57
 [<ffffffff810c8761>] page_referenced_one+0x88/0xea
 [<ffffffff810c89bf>] page_referenced+0x1fc/0x256
 [<ffffffff810b2771>] shrink_page_list+0x187/0x53a
 [<ffffffff810b2ed7>] shrink_inactive_list+0x1e0/0x33d
 [<ffffffff810acf95>] ? determine_dirtyable_memory+0x15/0x27
 [<ffffffff812e90ee>] ? call_function_single_interrupt+0xe/0x20
 [<ffffffff810b3356>] shrink_zone+0x322/0x3de
 [<ffffffff810a9587>] ? zone_watermark_ok_safe+0xe2/0xf1
 [<ffffffff810b3928>] kswapd+0x516/0x818
 [<ffffffff810b3412>] ? shrink_zone+0x3de/0x3de
 [<ffffffff81053d17>] kthread+0x7d/0x85
 [<ffffffff812e9394>] kernel_thread_helper+0x4/0x10
 [<ffffffff81053c9a>] ? __init_kthread_worker+0x37/0x37
 [<ffffffff812e9390>] ? gs_change+0xb/0xb
Signed-off-by: NOGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Document KVM_CAP_NR_VCPUS that can be used by the userspace to determine
maximum number of VCPUs it can create with the KVM_CREATE_VCPU ioctl.

Cc: Avi Kivity <avi@redhat.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Jan Kiszka <jan.kiszka@web.de>
Signed-off-by: NPekka Enberg <penberg@kernel.org>
Signed-off-by: NAvi Kivity <avi@redhat.com>

KVM does not hold any references to rcu protected data when it switches
CPU into a guest mode. In fact switching to a guest mode is very similar
to exiting to userspase from rcu point of view. In addition CPU may stay
in a guest mode for quite a long time (up to one time slice). Lets treat
guest mode as quiescent state, just like we do with user-mode execution.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

* commit '29ce8310': (34 commits)
  rcu: provide rcu_virt_note_context_switch() function.
  rcu: get rid of signed overflow in check_cpu_stall()
  rcu: optimize rcutiny
  rcu: prevent call_rcu() from diving into rcu core if irqs disabled
  rcu: further lower priority in rcu_yield()
  rcu: introduce kfree_rcu()
  rcu: fix spelling
  rcu: call __rcu_read_unlock() in exit_rcu for tree RCU
  rcu: Converge TINY_RCU expedited and normal boosting
  rcu: remove useless ->boosted_this_gp field
  rcu: code cleanups in TINY_RCU priority boosting.
  rcu: Switch to this_cpu() primitives
  rcu: Use WARN_ON_ONCE for DEBUG_OBJECTS_RCU_HEAD warnings
  rcu: mark rcutorture boosting callback as being on-stack
  rcu: add DEBUG_OBJECTS_RCU_HEAD check for alignment
  rcu: Enable DEBUG_OBJECTS_RCU_HEAD from !PREEMPT
  rcu: Add forward-progress diagnostic for per-CPU kthreads
  rcu: add grace-period age and more kthread state to tracing
  rcu: fix tracing bug thinko on boost-balk attribution
  rcu: update tracing documentation for new rcutorture and rcuboost
  ...

Pulling in rcu_virt_note_context_switch().
Signed-off-by: NAvi Kivity <avi@redhat.com>

* commit '29ce8310': (34 commits)
  rcu: provide rcu_virt_note_context_switch() function.
  rcu: get rid of signed overflow in check_cpu_stall()
  rcu: optimize rcutiny
  rcu: prevent call_rcu() from diving into rcu core if irqs disabled
  rcu: further lower priority in rcu_yield()
  rcu: introduce kfree_rcu()
  rcu: fix spelling
  rcu: call __rcu_read_unlock() in exit_rcu for tree RCU
  rcu: Converge TINY_RCU expedited and normal boosting
  rcu: remove useless ->boosted_this_gp field
  rcu: code cleanups in TINY_RCU priority boosting.
  rcu: Switch to this_cpu() primitives
  rcu: Use WARN_ON_ONCE for DEBUG_OBJECTS_RCU_HEAD warnings
  rcu: mark rcutorture boosting callback as being on-stack
  rcu: add DEBUG_OBJECTS_RCU_HEAD check for alignment
  rcu: Enable DEBUG_OBJECTS_RCU_HEAD from !PREEMPT
  rcu: Add forward-progress diagnostic for per-CPU kthreads
  rcu: add grace-period age and more kthread state to tracing
  rcu: fix tracing bug thinko on boost-balk attribution
  rcu: update tracing documentation for new rcutorture and rcuboost
  ...

We introduce em_jmp_far().

We also call this from em_grp45() to stop treating modrm_reg == 5 case
separately in the group 5 emulation.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The prototypes are changed appropriately.

We also replaces "goto grp45;" with simple em_grp45() call.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The opt of emulate_grp1a() is also removed.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Remove inline at this chance.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

In addition, one comma at the end of a statement is replaced with a
semicolon.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

This way, we can avoid checking the user space address many times when
we read the guest memory.

Although we can do the same for write if we check which slots are
writable, we do not care write now: reading the guest memory happens
more often than writing.

[avi: change VERIFY_READ to VERIFY_WRITE]
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

When we optimized walk_addr_generic() by not using the generic guest
memory reader, we replaced copy_from_user() with get_user():

  commit e30d2a170506830d5eef5e9d7990c5aedf1b0a51
  KVM: MMU: Optimize guest page table walk

  commit 15e2ac9a43d4d7d08088e404fddf2533a8e7d52e
  KVM: MMU: Fix 64-bit paging breakage on x86_32

But as Andi pointed out later, copy_from_user() does the same as
get_user() as long as we give a constant size to it.

So we use copy_from_user() to clean up the code.

The only, noticeable, regression introduced by this is 64-bit gpte
reading on x86_32 hosts needed for PAE guests.

But this can be mitigated by implementing 8-byte get_user() for x86_32,
if needed.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Linux doesn't use USPRG0 (now renamed VRSAVE in the architecture, even
when Altivec isn't involved), but a guest might.
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Convert to microseconds when displaying
(with fix from Bharat Bhushan <Bharat.Bhushan@freescale.com>).

This reduces rounding error with large quantities of short exits.
Signed-off-by: NStuart Yoder <stuart.yoder@freescale.com>
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

The exit type setting for mfspr/mtspr is moved from 44x to toplevel SPR
emulation.  This enables it on e500, and makes sure that all SPRs
are covered.

Exit accounting for tlbwe and tlbsx is added to e500.
Signed-off-by: NStuart Yoder <stuart.yoder@freescale.com>
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Return the actual host SVR for now, as we already do for PVR.  Eventually
we may support Qemu overriding PVR/SVR if the situation is appropriate,
once we implement KVM_SET_SREGS on e500.
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Since the emulator now checks segment limits and access rights, it
generates a lot more accesses to the vmcs segment fields.  Undo some
of the performance hit by cacheing those fields in a read-only cache
(the entire cache is invalidated on any write, or on guest exit).
Signed-off-by: NAvi Kivity <avi@redhat.com>

Instead of separate accessors for the segment selector and cached descriptor,
use one accessor for both.  This simplifies the code somewhat.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Avoids a VMREAD.
Signed-off-by: NAvi Kivity <avi@redhat.com>

dump_vmcb isn't used outside this module, make it static.
Shrink text and object by ~1% by standardizing formats.

$ size arch/x86/kvm/svm.o*
   text	   data	    bss	    dec	    hex	filename
  52910	    580	  10072	  63562	   f84a	arch/x86/kvm/svm.o.new
  53563	    580	  10072	  64215	   fad7	arch/x86/kvm/svm.o.old
Signed-off-by: NJoe Perches <joe@perches.com>
Acked-by: NJoerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Fix regression introduced by
  commit e30d2a170506830d5eef5e9d7990c5aedf1b0a51
  KVM: MMU: Optimize guest page table walk

On x86_32, get_user() does not support 64-bit values and we fail to
build KVM at the point of 64-bit paging.

This patch fixes this by using get_user() twice for that condition.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Reported-by: NJan Kiszka <jan.kiszka@web.de>
Signed-off-by: NAvi Kivity <avi@redhat.com>

This patch fixes some sparse warning about "dubious one-bit signed bitfield."
Signed-off-by: NJeff Mahoney <jeffm@suse.com>
Originally-by: NJan Blunck <jblunck@suse.de>
Signed-off-by: NJan Blunck <jblunck@suse.de>
Acked-by: NXiantao Zhang <xiantao.zhang@intel.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The CPUIDs for Centaur are added, and then the features of
PadLock hardware engine on VIA CPU, such as "ace", "ace_en"
and so on, can be passed into the kvm guest.
Signed-off-by: NBrilly Wu <brillywu@viatech.com.cn>
Signed-off-by: NKary Jin <karyjin@viatech.com.cn>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

mmio_index should be taken into account when copying data from
userspace.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Remove unused variable mmio_fault_cr2.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Move all groups into a single field and handle them in a single place.  This
saves bits when we add more group types (3 bits -> 7 groups types).
Signed-off-by: NAvi Kivity <avi@redhat.com>

walk_addr_generic() is a hot path and is also hard for the cpu to predict -
some of the parameters (fetch_fault in particular) vary wildly from
invocation to invocation.

Add unlikely() annotations where appropriate; all walk failures are
considered unlikely, as are cases where we have to mark the accessed or
dirty bit, as they are slow paths both in kvm and on real processors.
Signed-off-by: NAvi Kivity <avi@redhat.com>

For this, em_pushf/popf() are introduced.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

For this, emulate_pusha/popa() are converted to em_pusha/popa().
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

In addition, the RET emulation is changed to call em_pop() to remove
the pop_instruction label.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The following instructions are changed to use opcode::execute.

Group 1 (80-83)
  ADD (00-05), OR (08-0D), ADC (10-15), SBB (18-1D), AND (20-25),
  SUB (28-2D), XOR (30-35), CMP (38-3D)

CMPS (A6-A7), SCAS (AE-AF)

The last two do the same as CMP in the emulator, so em_cmp() is used.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

This patch optimizes the guest page table walk by using get_user()
instead of copy_from_user().

With this patch applied, paging64_walk_addr_generic() has become
about 0.5us to 1.0us faster on my Phenom II machine with NPT on.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

By reserving 0 as an invalid x86_intercept_stage, we no longer
need to store a valid flag in x86_intercept_map.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NAvi Kivity <avi@redhat.com>

While it isn't defined, no need to force a #UD.  If it becomes defined
in the future this can cause wierd problems for the guest.
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NAvi Kivity <avi@redhat.com>