Instead of pinning per-cpu TLB into a DTR, use DTC.  This will free up
one TLB entry for application, or even kernel if access pattern to
per-cpu data area has high temporal locality.

Since per-cpu is mapped at the top of region 7 address, we just need to
add special case in alt_dtlb_miss.  The physical address of per-cpu data
is already conveniently stored in IA64_KR(PER_CPU_DATA).  Latency for
alt_dtlb_miss is not affected as we can hide all the latency.  It was
measured that alt_dtlb_miss handler has 23 cycles latency before and
after the patch.

The performance effect is massive for applications that put lots of tlb
pressure on CPU.  Workload environment like database online transaction
processing or application uses tera-byte of memory would benefit the most.
Measurement with industry standard database benchmark shown an upward
of 1.6% gain.  While smaller workloads like cpu, java also showing small
improvement.
Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

When entering the kernel due to an MCA or INIT, ar.fpsr (ar40)
was not getting set to the kernel default value (remaining
at the user value).  The effect depends on the user setting 
of ar.fpsr.  In the test case, the effect was addresses 
printing with strange hex values.  

Setting ar.fpsr in ia64_set_kernel_registers sets it for both
the MCA and INIT paths.  The user value of ar.fpsr is correctly 
saved (in ia64_state_save) and restored (in ia64_state_restore).

Below is an example of output with very strange hex values.
Anyone know the value of hex 'g'?  :-)

Processes interrupted by INIT - 0 (cpu 14 task 0xdfffg55g7a4c6gA)

Signed-off-by: Russ Anderson (rja@sgi.com)
Signed-off-by: NTony Luck <tony.luck@intel.com>

MCA dispatch code take physical address of GP passed from SAL, then call
DATA_PA_TO_VA twice on GP before call into C code.  The first time is
in ia64_set_kernel_register, the second time is in VIRTUAL_MODE_ENTER.
The gp is changed to a virtual address in region 7 because DATA_PA_TO_VA
is implemented by dep instruction.

However when notify blocks were called from MCA handler code, because
notify blocks are supported by callback function pointers, gp value
value was switched to region 5 again.

The patch set gp register to kernel gp of region 5 at entry of MCA
dispatch.
Signed-off-by: NZou Nan hai <nanhai.zou@intel.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

struct ia64_sal_os_state has three semi-independent sections.  The code
in mca_asm.S assumes that these three sections are contiguous, which
makes it very awkward to add new data to this structure.  Remove the
assumption that the sections are contiguous.  Define a macro to shorten
references to offsets in ia64_sal_os_state.

This patch does not change the way that the code behaves.  It just
makes it easier to update the code in future and to add fields to
ia64_sal_os_state when debugging the MCA/INIT handlers.
Signed-off-by: NKeith Owens <kaos@sgi.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

The OS INIT handler is loading incorrect values into cr.ifa on exit.
This shows up as a hang when resuming after an INIT that is delivered
while a cpu is in user space.  Correct the value loaded into cr.ifa.
Signed-off-by: NKeith Owens <kaos@sgi.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

sos->os_status is set to a default value of IA64_MCA_COLD_BOOT for an
MCA, but then is incorrectly overwritten with IA64_MCA_SAME_CONTEXT (0).
This makes SAL think that all MCAs have been recovered.
Signed-off-by: NKeith Owens <kaos@sgi.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

On return from INIT handler we must convert the address of the
minstate area from a kernel virtual uncached address (0xC...)
to physical uncached (0x8...).  A typo (or thinko?) in the code
converted to physical cached.
Signed-off-by: NTony Luck <tony.luck@intel.com>

Wire the MCA/INIT handler stacks into DTR[2] and track them in
IA64_KR(CURRENT_STACK).  This gives the MCA/INIT handler stacks the
same TLB status as normal kernel stacks.  Reload the old CURRENT_STACK
data on return from OS to SAL.
Signed-off-by: NKeith Owens <kaos@sgi.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

The bulk of the change.  Use per cpu MCA/INIT stacks.  Change the SAL
to OS state (sos) to be per process.  Do all the assembler work on the
MCA/INIT stacks, leaving the original stack alone.  Pass per cpu state
data to the C handlers for MCA and INIT, which also means changing the
mca_drv interfaces slightly.  Lots of verification on whether the
original stack is usable before converting it to a sleeping process.
Signed-off-by: NKeith Owens <kaos@sgi.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

This patch is required to support cpu removal for IPF systems. Existing code
just fakes the real offline by keeping it run the idle thread, and polling
for the bit to re-appear in the cpu_state to get out of the idle loop.

For the cpu-offline to work correctly, we need to pass control of this CPU 
back to SAL so it can continue in the boot-rendez mode. This gives the
SAL control to not pick this cpu as the monarch processor for global MCA
events, and addition does not wait for this cpu to checkin with SAL
for global MCA events as well. The handoff is implemented as documented in 
SAL specification section 3.2.5.1 "OS_BOOT_RENDEZ to SAL return State"
Signed-off-by: NAshok Raj <ashok.raj@intel.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!