When calculating the DCT channel from the syndrome we need to know the
syndrome type (x4 vs x8). On F10h, this is read out from extended PCI
cfg space register F3x180 while on K8 we only support x4 syndromes and
don't have extended PCI config space anyway.

Make the code accessing F3x180 F10h only and fall back to x4 syndromes
on everything else.

Cc: <stable@kernel.org> # .33.x .34.x
Reported-by: NJeffrey Merkey <jeffmerkey@gmail.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

No need for clearing ecc_enable_override and checking it in two places.
Instead, simply check it during probing and act accordingly. Also,
rename the flag bitfields according to the functionality they actually
represent. What is more, make sure original BIOS ECC settings are
restored when the module is unloaded.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Add __percpu sparse annotations to places which didn't make it in one
of the previous patches.  All converions are trivial.

These annotations are to make sparse consider percpu variables to be
in a different address space and warn if accessed without going
through percpu accessors.  This patch doesn't affect normal builds.
Signed-off-by: NTejun Heo <tj@kernel.org>
Acked-by: NBorislav Petkov <borislav.petkov@amd.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Len Brown <lenb@kernel.org>
Cc: Neil Brown <neilb@suse.de>

An unfortunate "WARNING" in the message amd64_edac dumps when the system
doesn't support DRAM ECC or ECC checking is not enabled in the BIOS
used to trigger kerneloops which qualified the message as an OOPS thus
misleading the users. See, e.g.

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/422536
http://bugzilla.kernel.org/show_bug.cgi?id=15238

Downgrade the message level to KERN_NOTICE and fix the formulation.

Cc: stable@kernel.org # .32.x
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>
Acked-by: NDoug Thompson <dougthompson@xmission.com>

Add a missing iterator variable thus fixing the conditional of the
for-loop in amd64_get_scrub_rate().
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Do not access F2x19[0,4] on K8 since they're undefined there.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Clear the override flag after force-loading the module.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Currently, the module does not initialize fully when the DIMMs aren't
ECC but remains still loaded. Propagate the error when no instance of
the driver is properly initialized and prevent further loading.

Reorganize and polish error handling in amd64_edac_init() while at it.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Fix use-after-free errors by pushing all memory-freeing calls to the end
of amd64_remove_one_instance().
Reported-by: NDarren Jenkins <darrenrjenkins@gmail.com>
LKML-Reference: <1261370306.11354.52.camel@ICE-BOX>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Fix the case when amd64_debug_display_dimm_sizes() reports only half the
amount of DRAM on it because it doesn't account for when the single DCT
operates in 128-bit mode and merges chip selects from different DIMMs.
Reported-by: NJohannes Hirte <johannes.hirte@fem.tu-ilmenau.de>
LKML-Reference: <200912112202.48173.johannes.hirte@fem.tu-ilmenau.de>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

The current rd/wrmsr_on_cpus helpers assume that the supplied
cpumasks are contiguous. However, there are machines out there
like some K8 multinode Opterons which have a non-contiguous core
enumeration on each node (e.g. cores 0,2 on node 0 instead of 0,1), see
http://www.gossamer-threads.com/lists/linux/kernel/1160268.

This patch fixes out-of-bounds writes (see URL above) by adding per-CPU
msr structs which are used on the respective cores.

Additionally, two helpers, msrs_{alloc,free}, are provided for use by
the callers of the MSR accessors.

Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Mauro Carvalho Chehab <mchehab@redhat.com>
Cc: Aristeu Rozanski <aris@redhat.com>
Cc: Randy Dunlap <randy.dunlap@oracle.com>
Cc: Doug Thompson <dougthompson@xmission.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>
LKML-Reference: <20091211171440.GD31998@aftab>
Signed-off-by: NH. Peter Anvin <hpa@zytor.com>

drivers/edac/amd64_edac.c: In function 'amd64_edac_init':
drivers/edac/amd64_edac.c:2840: warning: 'ret' may be used uninitialized in this function

Cc: Doug Thompson <dougthompson@xmission.com>
Cc: Mauro Carvalho Chehab <mchehab@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

The routine does the reverse mapping of the error address of a CECC back
to the node id, DRAM controller and chip select of the DIMM which caused
the error. We should lookup the channel using the syndromes _only_ when
the DCTs are ganged so fix that.

Also, add an early exit when there's an error while scanning for the
csrow thus decreasing indentation levels for better readability.

Finally, fixup comments.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Instead of using the whole syndrome tables for channel decoding, use a
set of eigenvectors with which the tables can be generated to search for
the syndrome in error. The algorithm operates independently of symbol
size and can be used for both x4 and x8 syndromes.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

The .probe_valid_hardware low_ops member checked whether the DCTs are in
DDR3 mode and bailed out if so. Now that all the needed changes for DDR3
support is in place, remove it.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Instead of using deeply-nested conditionals for dumping the DIMM type in
debug mode, add a strings array of the supported DIMM types.

This is useful in cases where an edac driver supports multiple DRAM
types and is only defined in debug builds.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

SystemAddress -> sys_addr
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Add cs mode to cs size mapping tables for DDR2 and DDR3 and F10
and all K8 flavors and remove klugdy table of pseudo values. Add a
low_ops->dbam_to_cs member which is family-specific and replaces
low_ops->dbam_map_to_pages since the pages calculation is a one liner
now.

Further cleanups, while at it:

- shorten family name defines
- align amd64_family_types struct members
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Do not read DCLR[01] again since this is done in
amd64_read_mc_registers() earlier. There can be more than two physical
DIMMs present so clamp the channels value to max 2. Also, do not report
DCT data width - it is also done earlier.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Extend f10_debug_display_dimm_sizes to dump the logical DIMMs
configuration on K8 revF too. Remove the ganged arg since we print the
DCT operating mode (ganged vs unganged) earlier.

Also, DCT csrow configuration is relevant therefore dump it as
KERN_DEBUG instead of only on debug builds. Remove misleading DIMM
output since there's no reliable way of mapping of chip selects to
actual physical DIMMs.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Clarify bitfields description, add PCI config function/offset names to
registers for easy reference, simplify code layout, remove unneeded
info.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Carve out the register-specific debug statements into a separate
function, clarify meanings of the single bitfields in the register,
remove irrelevant output and macros.

There should be no functionality change resulting from this patch.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Add a pci config read wrapper for signaling pci config space access
errors instead of them being visible only on a debug build. This is
important on amd64_edac since it uses all those pci config register
values to access the DRAM/DIMM configuration of the nodes.

In addition, the wrapper makes a _lot_ (look at the diffstat!) of
error handling code superfluous and improves much of the overall code
readability by removing error handling details out of the way.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Unify almost identical code into one function and remove NUMA-specific
usage (specifically cpumask_of_node()) in favor of generic topology
methods.

Remove unused defines, while at it.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

cpumask_t -> struct cpumask, and don't put one on the stack.  (Note: this
is actually on the stack unless CONFIG_CPUMASK_OFFSTACK=y).
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Do not shift the TOP_MEM and TOP_MEM2 values by 23 but rather save the
whole 64-bit value read from the MSR. Although the TOP_MEM/TOP_MEM2 bits
are only a subset of the 64bit register, the values are correct since
the remaining bits are Read-As-Zero and no shifting is needed.

Also, cleanup DRAM base/limit debug output.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Make debug info formulations about the DRAM and DCT configuration of the
machine more human readable.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Shift error type bits properly.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

In amd64_edac_init(void) in amd64_edac.c, cache_k8_northbridges() is
called before pci_register_driver. If it fails, should exit with err
directly.
Signed-off-by: NLi Hong <lihong.hi@gmail.com>
Acked-by: NDoug Thompson <dougthompson@xmission.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

This is a proper fix as a follow-up to 66216a7a and 916d11b2.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

On Fam10h and above, F1x[1, 0][7C:40] are DRAM Base/Limit registers
which specify the destination node of a DRAM address. Those address
boundaries are being extracted into ->dram_base[] and ->dram_limit[].
Correct the extraction masks to match the respective address bits.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Different processor families support a different number of chip selects.
Handle this in a family-dependent way with the proper values assigned at
init time (see amd64_set_dct_base_and_mask).

Remove _DCSM_COUNT defines since they're used at one place and originate
from public documentation.

CC: Keith Mannthey <kmannth@us.ibm.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

This allows the errors to be further decoded and mapped to csrows.
Tested with ECC debug dimms and an Rev F cpu based system.
Signed-off-by: NKeith Mannthey <kmannth@us.ibm.com>
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

The check when DRAM interleaving is enabled should be done against the
pvt->dram_IntlvSel field and not against the ->dram_limit.

Simplify first loop and fixup printk formatting while at it.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

The pvt->dram_IntlvEn saves the 3 "Interleave Enable" bits already
right-shifted by 8 so the check in find_mc_by_sys_addr() by shifting the
values to the left 8 bits is wrong.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

K8 DRAM base and limit addresses from F1x40 +8*i and F1x44 + 8*i, where
i in (0..7) are both bits 39-24 and therefore the shifting should be
done by 24 and not by 8.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>

Allocate memory statically for 8-node machines max for simplicity
instead of relying on MAX_NUMNODES which is 0 on !CONFIG_NUMA builds.

Spotted by Jan Beulich.
Signed-off-by: NBorislav Petkov <borislav.petkov@amd.com>