提交 4126c019 编写于 作者: C Colin Cross 提交者: Len Brown

cpuidle: add support for states that affect multiple cpus

On some ARM SMP SoCs (OMAP4460, Tegra 2, and probably more), the
cpus cannot be independently powered down, either due to
sequencing restrictions (on Tegra 2, cpu 0 must be the last to
power down), or due to HW bugs (on OMAP4460, a cpu powering up
will corrupt the gic state unless the other cpu runs a work
around).  Each cpu has a power state that it can enter without
coordinating with the other cpu (usually Wait For Interrupt, or
WFI), and one or more "coupled" power states that affect blocks
shared between the cpus (L2 cache, interrupt controller, and
sometimes the whole SoC).  Entering a coupled power state must
be tightly controlled on both cpus.

The easiest solution to implementing coupled cpu power states is
to hotplug all but one cpu whenever possible, usually using a
cpufreq governor that looks at cpu load to determine when to
enable the secondary cpus.  This causes problems, as hotplug is an
expensive operation, so the number of hotplug transitions must be
minimized, leading to very slow response to loads, often on the
order of seconds.

This file implements an alternative solution, where each cpu will
wait in the WFI state until all cpus are ready to enter a coupled
state, at which point the coupled state function will be called
on all cpus at approximately the same time.

Once all cpus are ready to enter idle, they are woken by an smp
cross call.  At this point, there is a chance that one of the
cpus will find work to do, and choose not to enter idle.  A
final pass is needed to guarantee that all cpus will call the
power state enter function at the same time.  During this pass,
each cpu will increment the ready counter, and continue once the
ready counter matches the number of online coupled cpus.  If any
cpu exits idle, the other cpus will decrement their counter and
retry.

To use coupled cpuidle states, a cpuidle driver must:

   Set struct cpuidle_device.coupled_cpus to the mask of all
   coupled cpus, usually the same as cpu_possible_mask if all cpus
   are part of the same cluster.  The coupled_cpus mask must be
   set in the struct cpuidle_device for each cpu.

   Set struct cpuidle_device.safe_state to a state that is not a
   coupled state.  This is usually WFI.

   Set CPUIDLE_FLAG_COUPLED in struct cpuidle_state.flags for each
   state that affects multiple cpus.

   Provide a struct cpuidle_state.enter function for each state
   that affects multiple cpus.  This function is guaranteed to be
   called on all cpus at approximately the same time.  The driver
   should ensure that the cpus all abort together if any cpu tries
   to abort once the function is called.

update1:

cpuidle: coupled: fix count of online cpus

online_count was never incremented on boot, and was also counting
cpus that were not part of the coupled set.  Fix both issues by
introducting a new function that counts online coupled cpus, and
call it from register as well as the hotplug notifier.

update2:

cpuidle: coupled: fix decrementing ready count

cpuidle_coupled_set_not_ready sometimes refuses to decrement the
ready count in order to prevent a race condition.  This makes it
unsuitable for use when finished with idle.  Add a new function
cpuidle_coupled_set_done that decrements both the ready count and
waiting count, and call it after idle is complete.

Cc: Amit Kucheria <amit.kucheria@linaro.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Trinabh Gupta <g.trinabh@gmail.com>
Cc: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Reviewed-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: NKevin Hilman <khilman@ti.com>
Tested-by: NKevin Hilman <khilman@ti.com>
Signed-off-by: NColin Cross <ccross@android.com>
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NLen Brown <len.brown@intel.com>
上级 3af272ab
...@@ -18,3 +18,6 @@ config CPU_IDLE_GOV_MENU ...@@ -18,3 +18,6 @@ config CPU_IDLE_GOV_MENU
bool bool
depends on CPU_IDLE && NO_HZ depends on CPU_IDLE && NO_HZ
default y default y
config ARCH_NEEDS_CPU_IDLE_COUPLED
def_bool n
...@@ -3,3 +3,4 @@ ...@@ -3,3 +3,4 @@
# #
obj-y += cpuidle.o driver.o governor.o sysfs.o governors/ obj-y += cpuidle.o driver.o governor.o sysfs.o governors/
obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
此差异已折叠。
...@@ -171,7 +171,11 @@ int cpuidle_idle_call(void) ...@@ -171,7 +171,11 @@ int cpuidle_idle_call(void)
trace_power_start_rcuidle(POWER_CSTATE, next_state, dev->cpu); trace_power_start_rcuidle(POWER_CSTATE, next_state, dev->cpu);
trace_cpu_idle_rcuidle(next_state, dev->cpu); trace_cpu_idle_rcuidle(next_state, dev->cpu);
entered_state = cpuidle_enter_state(dev, drv, next_state); if (cpuidle_state_is_coupled(dev, drv, next_state))
entered_state = cpuidle_enter_state_coupled(dev, drv,
next_state);
else
entered_state = cpuidle_enter_state(dev, drv, next_state);
trace_power_end_rcuidle(dev->cpu); trace_power_end_rcuidle(dev->cpu);
trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, dev->cpu); trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, dev->cpu);
...@@ -407,9 +411,16 @@ static int __cpuidle_register_device(struct cpuidle_device *dev) ...@@ -407,9 +411,16 @@ static int __cpuidle_register_device(struct cpuidle_device *dev)
if (ret) if (ret)
goto err_sysfs; goto err_sysfs;
ret = cpuidle_coupled_register_device(dev);
if (ret)
goto err_coupled;
dev->registered = 1; dev->registered = 1;
return 0; return 0;
err_coupled:
cpuidle_remove_sysfs(cpu_dev);
wait_for_completion(&dev->kobj_unregister);
err_sysfs: err_sysfs:
list_del(&dev->device_list); list_del(&dev->device_list);
per_cpu(cpuidle_devices, dev->cpu) = NULL; per_cpu(cpuidle_devices, dev->cpu) = NULL;
...@@ -464,6 +475,8 @@ void cpuidle_unregister_device(struct cpuidle_device *dev) ...@@ -464,6 +475,8 @@ void cpuidle_unregister_device(struct cpuidle_device *dev)
wait_for_completion(&dev->kobj_unregister); wait_for_completion(&dev->kobj_unregister);
per_cpu(cpuidle_devices, dev->cpu) = NULL; per_cpu(cpuidle_devices, dev->cpu) = NULL;
cpuidle_coupled_unregister_device(dev);
cpuidle_resume_and_unlock(); cpuidle_resume_and_unlock();
module_put(cpuidle_driver->owner); module_put(cpuidle_driver->owner);
......
...@@ -32,4 +32,34 @@ extern void cpuidle_remove_state_sysfs(struct cpuidle_device *device); ...@@ -32,4 +32,34 @@ extern void cpuidle_remove_state_sysfs(struct cpuidle_device *device);
extern int cpuidle_add_sysfs(struct device *dev); extern int cpuidle_add_sysfs(struct device *dev);
extern void cpuidle_remove_sysfs(struct device *dev); extern void cpuidle_remove_sysfs(struct device *dev);
#ifdef CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED
bool cpuidle_state_is_coupled(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int state);
int cpuidle_enter_state_coupled(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int next_state);
int cpuidle_coupled_register_device(struct cpuidle_device *dev);
void cpuidle_coupled_unregister_device(struct cpuidle_device *dev);
#else
static inline bool cpuidle_state_is_coupled(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int state)
{
return false;
}
static inline int cpuidle_enter_state_coupled(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int next_state)
{
return -1;
}
static inline int cpuidle_coupled_register_device(struct cpuidle_device *dev)
{
return 0;
}
static inline void cpuidle_coupled_unregister_device(struct cpuidle_device *dev)
{
}
#endif
#endif /* __DRIVER_CPUIDLE_H */ #endif /* __DRIVER_CPUIDLE_H */
...@@ -57,6 +57,7 @@ struct cpuidle_state { ...@@ -57,6 +57,7 @@ struct cpuidle_state {
/* Idle State Flags */ /* Idle State Flags */
#define CPUIDLE_FLAG_TIME_VALID (0x01) /* is residency time measurable? */ #define CPUIDLE_FLAG_TIME_VALID (0x01) /* is residency time measurable? */
#define CPUIDLE_FLAG_COUPLED (0x02) /* state applies to multiple cpus */
#define CPUIDLE_DRIVER_FLAGS_MASK (0xFFFF0000) #define CPUIDLE_DRIVER_FLAGS_MASK (0xFFFF0000)
...@@ -100,6 +101,12 @@ struct cpuidle_device { ...@@ -100,6 +101,12 @@ struct cpuidle_device {
struct list_head device_list; struct list_head device_list;
struct kobject kobj; struct kobject kobj;
struct completion kobj_unregister; struct completion kobj_unregister;
#ifdef CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED
int safe_state_index;
cpumask_t coupled_cpus;
struct cpuidle_coupled *coupled;
#endif
}; };
DECLARE_PER_CPU(struct cpuidle_device *, cpuidle_devices); DECLARE_PER_CPU(struct cpuidle_device *, cpuidle_devices);
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册