Millisecond resolution is possible and there are use cases for it
(automatic testing).

Seconds-based interface is preserved for compatibility.
Signed-off-by: NAri Kauppi <Ext-Ari.Kauppi@nokia.com>
Reviewed-by: NPhil Carmody <ext-phil.2.carmody@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

While handling PRCM IRQs, mask out interrupts that are not enabled in
PRM_IRQENABLE_MPU.  If these are not masked out, non-enabled
interrupts are caught, a WARN() is printed due to no 'handler' and the
events are cleared.  In addition to being noisy, this can also
interfere with independent polling of this register by SR/VP code.

This was noticed using SmartReflex transitions which cause the VPx_*
interrupts to be handled since they are set in PRM_IRQSTATUS_MPU even
but not enabled in PRM_IRQENABLE_MPU.
Acked-by: NMike Turquette <mturquette@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

Add _MASK suffix to CM_FCLKEN_IVA2 bitfieds to conform with the rest
of the usage in cm-regbits-34xx.h of using _SHIFT and _MASK suffixes.
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>
Signed-off-by: NPaul Walmsley <paul@pwsan.com>

When 'enable_off_mode' is 0, the target power state for MPU
and CORE was locally changed to PWRDM_POWER_RET but, the
statistics are updated for idle state originally selected
by the governor.

This patch 'invalidates' the idle states that lead either of
MPU or Core to PWRDM_POWER_OFF state when 'enable_off_mode'
is '0'. The states are valid once 'enable_off_mode' is set
to '1'.

Added function next_valid_state() to check if current state
is valid; else get the next valid state. It is called from
omap3_enter_idle_bm().
Signed-off-by: NSanjeev Premi <premi@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Add usecounting for wakeup and sleep dependencies.  In the current
situation, if several functions add dependencies on the same
clockdomains, when the first dependency removal function is called,
the dependency will be incorrectly removed from the hardware.

Add clkdm_clear_all_wkdeps() and clkdm_clear_all_sleepdeps(), which
provide a fast and usecounting-consistent way to clear all hardware
clockdomain dependencies, since accesses to these registers can be
quite slow.  pm{2,3}4xx.c has been updated to use these new functions.
The original version of this patch did not touch these files, which
previously wrote directly to the wkdep registers, and thus confused
the usecounting code.  This problem was found by Kevin Hilman
<khilman@deeprootsystems.com>.

N.B.: This patch introduces one significant functional difference over
the previous pm34xx.c code: sleepdeps are now cleared during
clockdomain initialization, whereas previously they were left
untouched.  This has been tested by Kevin and confirmed to work.

The original version of this patch also did not take into
consideration that some clockdomains do not have sleep or wakeup
dependency sources, which caused NULL pointer dereferences.  This
problem was debugged and fixed by Kevin Hilman
<khilman@deeprootsystems.com>.
Signed-off-by: NPaul Walmsley <paul@pwsan.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>
Cc: Jouni Högander <jouni.hogander@nokia.com>

Move clockdomain wakeup dependency and sleep dependency data
structures from the powerdomain layer to the clockdomain layer, where
they belong.  These dependencies were originally placed in the
powerdomain layer due to unclear documentation; however, it is clear
now that these dependencies are between clockdomains.  For OMAP2/3,
this is not such a big problem, but for OMAP4 this needs to be fixed.

Thanks to Benoît Cousson <b-cousson@ti.com> for his advice on this
patch.
Signed-off-by: NPaul Walmsley <paul@pwsan.com>
Cc: Benoît Cousson <b-cousson@ti.com>

Module offsets were same for OMAP2 and OMAP3 while they differ for OMAP4.
Hence we need different macros for identifying platform specific offsets.
Signed-off-by: NAbhijit Pagare <abhijitpagare@ti.com>
Signed-off-by: NPaul Walmsley <paul@pwsan.com>
Cc: Benoit Cousson <b-cousson@ti.com>
Cc: Rajendra Nayak <rnayak@ti.com>

Currently, DSS does not wakeup when there is a DMA request.  DSS wake
up event must be enabled so that the DMA request to refill the FIFO
will wake up the CORE domain.
Signed-off-by: NSubramani Venkatesh <subramani.venkatesh@windriver.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Due to OMAP3 erratas 1.157, 1.185 the save of the last pad register
(ETK_D14 and ETK_D15) can fail sometimes when there is simultaneous
OCP access to the SCM register area. Fixed by writing the last
register to the save area.

Also, optimized the delay loop for the HW save to include an udelay(1),
which limits the number of unnecessary HW accesses to SCM register area
during the save.
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Acked-by: NNishanth Menon <nm@ti.com>
Reviewed-by: NAnand Gadiyar <gadiyar@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Wake-up from McBSP ports are needed, especially when the THRESHOLD
dma mode is in use for audio playback.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

This patch removes code blocks that are repeated
in function prcm_setup_regs().
Signed-off-by: NSanjeev Premi <premi@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

OMAP interrupt controller goes to unknown state when there is right
combination of l3,l4 sleep/wake-up transitions, l4 autoidle in
interrupt controller and some interrupt. When this happens, interrupts
are not delivered to ARM anymore and ARM will remain in WFI (wait for
interrupt) until interrupt controller is forced to wake-up
(i.e. lauterbach).
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Enable the auto-idle feature of the SCM block to save some additional
power.
Signed-off-by: NMika Westerberg <ext-mika.1.westerberg@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Suspending drivers may still generate interrupts just before their suspend is
completed. Any pending interrupts here will prevent sleep.
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

`!' has a higher precedence than `&' so parentheses are required.
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Signed-off-by: NTony Lindgren <tony@atomide.com>

Fixes a bug where scheduling is delayed until next wakeup due to race
condition (e.g. interrupt requests scheduling just before omap_sram_idle
is entered.)
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

This patch removes the check to see if some functional clocks are
still enabled before entering sleep.  This is no longer needed when
using safe state (C1) that keeps CORE active.
Signed-off-by: NPeter 'p2' De Schrijver <peter.de-schrijver@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

It is more efficient to use pwrdm_set_next_pwrst for mpu, core and neon
instead of set_pwrdm_state in idle loop. It is anyway known that those are
active in idle loop. So no need to use set_pwrdm_state.
Signed-off-by: NJouni Hogander <jouni.hogander@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Previously omap3_idle_init() was called in device_init, while
omap_pm_init() is called at late_initcall. This causes the cpu idle
driver to call omap_sram_idle before it is properly initialized. This
patch fixes the issue by moving omap3_idle_init into omap3_pm_init.
Signed-off-by: NKalle Jokiniemi <ext-kalle.jokiniemi@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

This patch adds support and enables state C4(MPU RET + CORE RET)
and MPU OFF states (C3 and C5.)
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Basic CPUidle driver for OMAP3 with deepest sleep state supported
being MPU CSWR.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Due to an OMAP3 errata (1.142), on HS/EMU devices SDRC should be
programed to issue automatic self refresh on timeout
of AUTO_CNT = 1 prior to any transition to OFF mode.
This is needed only on sil rev's ES3.0 and above.

This patch enables the above needed WA in the SDRC power register
value stored in scratchpad, so that ROM code restores this value
in SDRC POWER on the wakeup path.
The original SDRC POWER register value is stored and restored back
in omap_sram_idle() function.

This fixes some random crashes observed while stressing suspend
on HS/EMU devices.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKalle Jokiniemi <kalle.jokiniemi@digia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

OMAP 3430 ES3.1 chips have a separate bit for IO daisy-chain
wake up enabling. It needs to be enabled when entering
retention or off state, otherwise waking up might not work
in all situations.
Signed-off-by: NKalle Jokiniemi <kalle.jokiniemi@digia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

MPU and CORE should stay awake if there is CAM domain ACTIVE. This is
because that module doesn't have wake-up capability.

This should replace the patch that is currently in the PM branch.
Signed-off-by: NJouni Hogander <jouni.hogander@nokia.com>
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

OMAP3 can't generate wakeups in this state, thus it is not permitted.
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Using debugfs, export a configurable wakeup timer to be used to
wakeup system from suspend.

If a non-zero value is written to
/debug/pm_debug/wakeup_timer_seconds, A timer wakeup event will wake
the system and resume after the configured number of seconds.
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Allow enable/disable of low-power states during idle.  To
enable low-power idle:

   echo 1 > /debug/pm_debug/sleep_while_idle

 to disable:

   echo 0 > /debug/pm_debug/sleep_while_idle

Also allow enable/disable of OFF-mode.  To enable:

   echo 1 > /debug/pm_debug/enable_off_mode

 to disable:

   echo 0 > /debug/pm_debug/enable_off_mode
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

The SMS_SYSCONFIG register gets reset in off mode, added a
save/restore mechanism for that.
Signed-off-by: NKalle Jokiniemi <kalle.jokiniemi@digia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

The secure sram context save uses dma channels 0 and 1.
In order to avoid collision between kernel DMA transfers and
ROM code dma transfers, we need to reserve DMA channels 0
1 on high security devices.

A bug in ROM code leaves dma irq status bits uncleared.
Hence those irq status bits need to be cleared when restoring
DMA context after off mode.

There was also a faulty parameter given to PPA in the secure
ram context save assembly code, which caused interrupts to
be enabled during secure ram context save. This caused the
save to fail sometimes, which resulted the saved context
to be corrupted, but also left DMA channels in secure mode.
The secure mode DMA channels caused "DMA secure error with
device 0" errors to be displayed.
Signed-off-by: NKalle Jokiniemi <kalle.jokiniemi@digia.com>
Signed-off-by: NJouni Hogander <jouni.hogander@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Fix for ES3.0 bug: SDRC not sending auto-refresh when OMAP wakes-up
from OFF mode (warning for HS devices.)
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

The function omap3_save_secure_ram() is now called only once during
the initialization of the device and consequent sleep cycles will
re-use the same saved contents for secure RAM. Users who need secure
services should do secure RAM saving before entering off-mode, if a
secure service has been accessed after last save.

There are both latency and reliability issues with saving secure RAM
context in the idle path. The context save uses a hardware resource
which takes an order of hundreds of milliseconds to initialize after a
wake up from off-mode, and also there is no way of checking whether it
is ready from kernel side or not. It just crashes if you use it too
quickly

Additional fix to ensure scratchpad save is done after secure
RAM by Roger Quadros.
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NRoger Quadros <ext-roger.quadros@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

For HS/EMU devices, some additional resources need to be
saved/restored for off-mode support.  Namely, saving the secure RAM
and a pointer to it in the scratchpad.
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

For HS/EMU devices, these additional features are also used:

- DMA interrupt disable routine added
- Added DMA controller reset to DMA context restore
Signed-off-by: NTero Kristo <tero.kristo@nokia.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Add context save and restore for CORE powerdomain resources in order
to support off-mode.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Adds a 'save_state' option when calling into SRAM idle function
and adds some minor cleanups of SRAM asm code.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

During the MMU restoration on the restore path from MPU OFF, the page
table entry for the page consisting of the code being executed is
modified to make MMU return VA=PA.

The MMU is then enabled and the original entry is being stored in
scratchpad.  This patch reads the original values stored in
scratchpad, and restores them back.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Expand the powerdomains handled in the idle path to include PER, NEON
and CORE.  This includes properly clearing the previous powerstates,
linking NEON state to MPU state and calling the UART prepare functions
for only the appropraite powerdomain transitions (CORE for UART1,2,
PER for UART3.)
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>

Generalize the copy of SRAM functions into omap_push_sram_idle()
so it can be used on init but also after off-mode transitions.
Signed-off-by: NRajendra Nayak <rnayak@ti.com>
Signed-off-by: NKevin Hilman <khilman@deeprootsystems.com>