If the machine is booted without any cpu_idle driver set
(b/c disable_cpuidle() has been called) we should follow
other users of cpu_idle API and check the return value
for NULL before using it.
Reported-and-tested-by: NMark van Dijk <mark@internecto.net>
Suggested-by: NJan Beulich <JBeulich@suse.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>

When the system is booted with some cpus offline, the idle
driver is not initialized. When a cpu is set online, the
acpi code call the intel idle init function. Unfortunately
this code introduce a dependency between intel_idle and acpi.

This patch is intended to remove this dependency by using the
notifier of intel_idle. This patch has the benefit of
encapsulating the intel_idle driver and remove some exported
functions.
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: NSrivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>

From an OS point of view, IVB looks like SNB, but quicker.
Signed-off-by: NLen Brown <len.brown@intel.com>

Fix a code indentation in the function intel_idle_cpu_init that looks
confusing.o
Suggested-by: NSrivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Reviewed-by: NSrivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Signed-off-by: NMarcos Paulo de Souza <marcos.mage@gmail.com>
Cc: "Brown, Len" <len.brown@intel.com>
Cc: Len Brown <lenb@kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Commit b66b8b9a ('intel-idle: convert
to x86_cpu_id auto probing') added a distinction between Nehalem and
Westemere processors and changed auto_demotion_disable_flags for the
former to 0.  This was not explained in the commit message, so change
it back.
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Acked-by: NThomas Renninger <trenn@suse.de>
Acked-by: NH. Peter Anvin <hpa@zytor.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Commit b66b8b9a ('intel-idle: convert
to x86_cpu_id auto probing') put two entries for model 0x2f
(Westmere-EX Xeon) in the device ID table and left out model 0x2e
(Nehalem-EX Xeon).
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Acked-by: NThomas Renninger <trenn@suse.de>
Acked-by: NH. Peter Anvin <hpa@zytor.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

With this it should be automatically loaded on suitable systems by
udev.

The old switch () is replaced with a table based approach, this
also cleans up the code.

Cc: Len Brown <lenb@kernel.org>
Signed-off-by: NAndi Kleen <ak@linux.intel.com>
Signed-off-by: NThomas Renninger <trenn@suse.de>
Acked-by: NH. Peter Anvin <hpa@zytor.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Delay the setting up of features (cpuidle, throttling by calling
acpi_processor_start()) to the time when the hotplugged
core got onlined the first time and got fully
initialized.
Signed-off-by: NThomas Renninger <trenn@suse.de>
Signed-off-by: NLen Brown <len.brown@intel.com>

Function split up, should have no functional change.

Provides entry point for physically hotplugged CPUs
to initialize and activate cpuidle.
Signed-off-by: NThomas Renninger <trenn@suse.de>
CC: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
CC: Shaohua Li <shaohua.li@intel.com>
CC: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLen Brown <len.brown@intel.com>

kvm -cpu host passes the original cpuid info to the guest.

Latest kvm version seem to return true for mwait_leaf cpuid
function on recent Intel CPUs. But it does not return mwait
C-states (mwait_substates), instead zero is returned.

While real CPUs seem to always return non-zero values, the intel
idle driver should not get active in kvm (mwait_substates == 0)
case and bail out.
Otherwise a Null pointer exception will happen later when the
cpuidle subsystem tries to get active:
[0.984807] BUG: unable to handle kernel NULL pointer dereference at (null)
[0.984807] IP: [<(null)>] (null)
...
[0.984807][<ffffffff8143cf34>] ? cpuidle_idle_call+0xb4/0x340
[0.984807][<ffffffff8159e7bc>] ? __atomic_notifier_call_chain+0x4c/0x70
[0.984807][<ffffffff81001198>] ? cpu_idle+0x78/0xd0

Reference:
https://bugzilla.novell.com/show_bug.cgi?id=726296

Cc: stable@vger.kernel.org
Signed-off-by: NThomas Renninger <trenn@suse.de>
CC: Bruno Friedmann <bruno@ioda-net.ch>
Signed-off-by: NLen Brown <len.brown@intel.com>

Fix the following warning:

drivers/idle/intel_idle.c: In function 'intel_idle_cpuidle_devices_init':
drivers/idle/intel_idle.c:518:5: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]

By making get_driver_data() return a long instead of an int.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

irq disabling happens earlier in process_32.c:cpu_idle.  Basically,
cpuidle_state->enter is called, cpu irq is disabled.  cpuidle_state->enter
would turn on irq when exiting.

intel_idle doesn't follow this assumption.  Although it doesn't cause real
issue, it misleads developers.  Remove the call to local_irq_disable() at
entry.

[akpm@linux-foundation.org: add comment]
Signed-off-by: NMingming Zhang <mingmingx.zhang@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLen Brown <len.brown@intel.com>

smp_call_function() only lets all other CPUs execute a specific function,
while we expect all CPUs do in intel_idle.  Without the fix, we could have
one cpu which has auto_demotion enabled or has no broadcast timer setup.
Usually we don't see impact because auto demotion just harms power and the
intel_idle init is called in CPU 0, where boradcast timer delivers
interrupt, but this still could be a problem.

Cc: stable@vger.kernel.org
Signed-off-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLen Brown <len.brown@intel.com>

This patch makes the cpuidle_states structure global (single copy)
instead of per-cpu. The statistics needed on per-cpu basis
by the governor are kept per-cpu. This simplifies the cpuidle
subsystem as state registration is done by single cpu only.
Having single copy of cpuidle_states saves memory. Rare case
of asymmetric C-states can be handled within the cpuidle driver
and architectures such as POWER do not have asymmetric C-states.

Having single/global registration of all the idle states,
dynamic C-state transitions on x86 are handled by
the boot cpu. Here, the boot cpu  would disable all the devices,
re-populate the states and later enable all the devices,
irrespective of the cpu that would receive the notification first.

Reference:
https://lkml.org/lkml/2011/4/25/83Signed-off-by: NDeepthi Dharwar <deepthi@linux.vnet.ibm.com>
Signed-off-by: NTrinabh Gupta <g.trinabh@gmail.com>
Tested-by: NJean Pihet <j-pihet@ti.com>
Reviewed-by: NKevin Hilman <khilman@ti.com>
Acked-by: NArjan van de Ven <arjan@linux.intel.com>
Acked-by: NKevin Hilman <khilman@ti.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

This is the first step towards global registration of cpuidle
states. The statistics used primarily by the governor are per-cpu
and have to be split from rest of the fields inside cpuidle_state,
which would be made global i.e. single copy. The driver_data field
is also per-cpu and moved.
Signed-off-by: NDeepthi Dharwar <deepthi@linux.vnet.ibm.com>
Signed-off-by: NTrinabh Gupta <g.trinabh@gmail.com>
Tested-by: NJean Pihet <j-pihet@ti.com>
Reviewed-by: NKevin Hilman <khilman@ti.com>
Acked-by: NArjan van de Ven <arjan@linux.intel.com>
Acked-by: NKevin Hilman <khilman@ti.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

Cpuidle governor only suggests the state to enter using the
governor->select() interface, but allows the low level driver to
override the recommended state. The actual entered state
may be different because of software or hardware demotion. Software
demotion is done by the back-end cpuidle driver and can be accounted
correctly. Current cpuidle code uses last_state field to capture the
actual state entered and based on that updates the statistics for the
state entered.

Ideally the driver enter routine should update the counters,
and it should return the state actually entered rather than the time
spent there. The generic cpuidle code should simply handle where
the counters live in the sysfs namespace, not updating the counters.

Reference:
https://lkml.org/lkml/2011/3/25/52Signed-off-by: NDeepthi Dharwar <deepthi@linux.vnet.ibm.com>
Signed-off-by: NTrinabh Gupta <g.trinabh@gmail.com>
Tested-by: NJean Pihet <j-pihet@ti.com>
Reviewed-by: NKevin Hilman <khilman@ti.com>
Acked-by: NArjan van de Ven <arjan@linux.intel.com>
Acked-by: NKevin Hilman <khilman@ti.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

These files aren't just exporting symbols -- they are also defining
a MODULE_LICENSE etc. so give them the full module.h file.
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

Userspace apps might have to cut off parts off the
idle state name for display reasons.
Switch NHM-C1 to C1-NHM (and others) so that a cut off
name is unique and makes sense to the user.
Signed-off-by: NThomas Renninger <trenn@suse.de>
CC: lenb@kernel.org
Signed-off-by: NLen Brown <len.brown@intel.com>

Just as we had to disable auto-demotion for NHM/WSM,
we need to do the same for Atom (Lincroft version).

In particular, auto-demotion will prevent Lincroft
from entering the S0i3 idle power saving state.

https://bugzilla.kernel.org/show_bug.cgi?id=25252Signed-off-by: NLen Brown <len.brown@intel.com>

Hardware C-state auto-demotion is a mechanism where the HW overrides
the OS C-state request, instead demoting to a shallower state,
which is less expensive, but saves less power.

Modern Linux should generally get exactly the states it requests.
In particular, when a CPU is taken off-line, it must not be demoted, else
it can prevent the entire package from reaching deep C-states.

https://bugzilla.kernel.org/show_bug.cgi?id=25252Signed-off-by: NLen Brown <len.brown@intel.com>

Fix a shutdown regression caused by 2a2d31c8 ("intel_idle: open
broadcast clock event").  The clockevent framework can automatically
shutdown broadcast timers for hotremove CPUs.  And we get a shutdown
regression when we shutdown broadcast timer for hot remove CPU, so just
delete some code.

Also fix some section mismatch.
Reported-by: NAri Savolainen <ari.m.savolainen@gmail.com>
Signed-off-by: NShaohua Li <shaohua.li@intel.com>
Tested-by: NLinus Torvalds <torvalds@linux-foundation.org>
Cc: stable@kernel.org
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

cpuidle/x86/perf: fix power:cpu_idle double end events and throw cpu_idle events from the cpuidle layer

Currently intel_idle and acpi_idle driver show double cpu_idle "exit idle"
events -> this patch fixes it and makes cpu_idle events throwing less complex.

It also introduces cpu_idle events for all architectures which use
the cpuidle subsystem, namely:
  - arch/arm/mach-at91/cpuidle.c
  - arch/arm/mach-davinci/cpuidle.c
  - arch/arm/mach-kirkwood/cpuidle.c
  - arch/arm/mach-omap2/cpuidle34xx.c
  - arch/drivers/acpi/processor_idle.c (for all cases, not only mwait)
  - arch/x86/kernel/process.c (did throw events before, but was a mess)
  - drivers/idle/intel_idle.c (did throw events before)

Convention should be:
Fire cpu_idle events inside the current pm_idle function (not somewhere
down the the callee tree) to keep things easy.

Current possible pm_idle functions in X86:
c1e_idle, poll_idle, cpuidle_idle_call, mwait_idle, default_idle
-> this is really easy is now.

This affects userspace:
The type field of the cpu_idle power event can now direclty get
mapped to:
/sys/devices/system/cpu/cpuX/cpuidle/stateX/{name,desc,usage,time,...}
instead of throwing very CPU/mwait specific values.
This change is not visible for the intel_idle driver.
For the acpi_idle driver it should only be visible if the vendor
misses out C-states in his BIOS.
Another (perf timechart) patch reads out cpuidle info of cpu_idle
events from:
/sys/.../cpuidle/stateX/*, then the cpuidle events are mapped
to the correct C-/cpuidle state again, even if e.g. vendors miss
out C-states in their BIOS and for example only export C1 and C3.
-> everything is fine.
Signed-off-by: NThomas Renninger <trenn@suse.de>
CC: Robert Schoene <robert.schoene@tu-dresden.de>
CC: Jean Pihet <j-pihet@ti.com>
CC: Arjan van de Ven <arjan@linux.intel.com>
CC: Ingo Molnar <mingo@elte.hu>
CC: Frederic Weisbecker <fweisbec@gmail.com>
CC: linux-pm@lists.linux-foundation.org
CC: linux-acpi@vger.kernel.org
CC: linux-kernel@vger.kernel.org
CC: linux-perf-users@vger.kernel.org
CC: linux-omap@vger.kernel.org
Signed-off-by: NLen Brown <len.brown@intel.com>

Intel_idle driver uses CLOCK_EVT_NOTIFY_BROADCAST_ENTER
CLOCK_EVT_NOTIFY_BROADCAST_EXIT
for broadcast clock events. The _ENTER/_EXIT doesn't really open broadcast clock
events, please see processor_idle.c for an example. In some situation, this will
cause boot hang, because some CPUs enters idle but local APIC timer stalls.
Reported-and-tested-by: NYan Zheng <zheng.z.yan@intel.com>
Signed-off-by: NShaohua Li <shaohua.li@intel.com>
cc: stable@kernel.org
Signed-off-by: NLen Brown <len.brown@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>

Having four variables for the same thing:
  idle_halt, idle_nomwait, force_mwait and boot_option_idle_overrides
is rather confusing and unnecessary complex.

if idle= boot param is passed, only set up one variable:
boot_option_idle_overrides

Introduces following functional changes/fixes:
  - intel_idle driver does not register if any idle=xy
    boot param is passed.
  - processor_idle.c will also not register a cpuidle driver
    and get active if idle=halt is passed.
    Before a cpuidle driver with one (C1, halt) state got registered
    Now the default_idle function will be used which finally uses
    the same idle call to enter sleep state (safe_halt()), but
    without registering a whole cpuidle driver.

That means idle= param will always avoid cpuidle drivers to register
with one exception (same behavior as before):
idle=nomwait
may still register acpi_idle cpuidle driver, but C1 will not use
mwait, but hlt. This can be a workaround for IO based deeper sleep
states where C1 mwait causes problems.
Signed-off-by: NThomas Renninger <trenn@suse.de>
cc: x86@kernel.org
Signed-off-by: NLen Brown <len.brown@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>

Add these new power trace events:

 power:cpu_idle
 power:cpu_frequency
 power:machine_suspend

The old C-state/idle accounting events:
  power:power_start
  power:power_end

Have now a replacement (but we are still keeping the old
tracepoints for compatibility):

  power:cpu_idle

and
  power:power_frequency

is replaced with:
  power:cpu_frequency

power:machine_suspend is newly introduced.

Jean Pihet has a patch integrated into the generic layer
(kernel/power/suspend.c) which will make use of it.

the type= field got removed from both, it was never
used and the type is differed by the event type itself.

perf timechart userspace tool gets adjusted in a separate patch.
Signed-off-by: NThomas Renninger <trenn@suse.de>
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Acked-by: NArjan van de Ven <arjan@linux.intel.com>
Acked-by: NJean Pihet <jean.pihet@newoldbits.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: rjw@sisk.pl
LKML-Reference: <1294073445-14812-3-git-send-email-trenn@suse.de>
Signed-off-by: NIngo Molnar <mingo@elte.hu>
LKML-Reference: <1290072314-31155-2-git-send-email-trenn@suse.de>

power_frequency moved to drivers/cpufreq/cpufreq.c which has
to be compiled in, no need to export it.

intel_idle can a be module though...
Signed-off-by: NThomas Renninger <trenn@suse.de>
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Acked-by: NJean Pihet <jean.pihet@newoldbits.com>
Cc: Jean Pihet <j-pihet@ti.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: rjw@sisk.pl
LKML-Reference: <1294073445-14812-2-git-send-email-trenn@suse.de>
Signed-off-by: NIngo Molnar <mingo@elte.hu>
LKML-Reference: <1290072314-31155-2-git-send-email-trenn@suse.de>

We erroneously ignored the Always Running APIC Timer on WSM-EX.
Move the check for ARAT down so that it can apply to any/all models.
Signed-off-by: NLen Brown <len.brown@intel.com>

If we use the LAPIC timer during ATOM C2 on
some nvidia chisets, the system stalls.

https://bugzilla.kernel.org/show_bug.cgi?id=21032Signed-off-by: NLen Brown <len.brown@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>

The mW data in this field is a total fabrication
and serves no purpose other than to mislead
those who might see it in sysfs.  Delete it.
Signed-off-by: NLen Brown <len.brown@intel.com>

A run-time test to invoke leave_mm() for the deepest
supported C-state is redundant, since the appropriate
C-states already have flags with CPUIDLE_FLAG_TLB_FLUSHED set.
Signed-off-by: NLen Brown <len.brown@intel.com>

ATM-C6 was commented out, pending public documentation.

https://bugzilla.kernel.org/show_bug.cgi?id=19762Tested-by: NDennis Jansen <Dennis.Jansen@...>
Signed-off-by: NLen Brown <len.brown@intel.com>

Avoid TLB flush IPIs for the cores in deeper c-states by voluntary leave_mm()
before entering into that state. CPUs tend to flush TLB in those c-states
anyways.

acpi_idle does this with C3-type states, but it was not caried over
when intel_idle was introduced.  intel_idle can apply it
to C-states in addition to those that ACPI might export as C3...
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

intel_idle_cpuidle_devices is a percpu pointer
but was missing __percpu markup.
Signed-off-by: NNamhyung Kim <namhyung@gmail.com>
Acked-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NLen Brown <len.brown@intel.com>

Remove execution permission from source file.
Signed-off-by: NThomas Weber <weber@corscience.de>
Signed-off-by: NLen Brown <len.brown@intel.com>

We have MWAIT constants spread across three different .c files, for no
good reason.  Move them all into a common header file.
Signed-off-by: NH. Peter Anvin <hpa@linux.intel.com>
Reviewed-by: NArjan van de Ven <arjan@linux.intel.com>
Cc: Len Brown <lenb@kernel.org>
LKML-Reference: <tip-*@git.kernel.org>

Signed-off-by: NArjan van de Ven <arjan@linux.intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>