The genpd core provides an API now to retrieve the performance state
from DT, use that instead of the ->get_pstate() callback.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>

A device's DT node or its OPP nodes can contain a phandle to other
device's OPP node, in the "required-opps" property.

This patch implements a routine to find that required OPP from the node
that contains the "required-opps" property.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>

"opp-hz" property is optional for power domains now and we shouldn't
error out if it is missing for power domains.

This patch creates two new routines, _get_opp_count() and
_opp_is_duplicate(), by separating existing code from their parent
functions. Also skip duplicate OPP check for power domain OPPs as they
may not have any the "opp-hz" field, but a platform specific performance
state binding to uniquely identify OPP nodes.

By default the debugfs OPP nodes are named using the "rate" value, but
that isn't possible for the power domain OPP nodes and hence they use
the index of the OPP node in the OPP node list instead.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>

This adds the dev_pm_opp_{un}register_get_pstate_helper() helper
routines which will be used to set the get_pstate() callback for a
device. This callback will be later called internally by the OPP core to
get performance state corresponding to an OPP.

This is required temporarily until the time we have proper DT bindings
to include the performance state information.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The genpd framework now provides an API to request device's power
domain to update its performance state.  Use that interface from the
OPP core for devices whose power domains support performance states.

Note that this commit doesn't add any mechanism by which performance
states are made available to the OPP core. That would be done by a
later commit.

Note that the current implementation is restricted to the case where
the device doesn't have separate regulators for itself. We shouldn't
over engineer the code before we have real use case for them. We can
always come back and add more code to support such cases later on.
Tested-by: NRajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The routine is named incorrectly since the first attempt as there is
nothing like a put_opp() helper. We wanted to unregister the set_opp()
helper here and so it should rather be named as
dev_pm_opp_unregister_set_opp_helper().
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

On some i.MX6 platforms which do not have speed grading
check, opp table will not be created in platform code,
so cpufreq driver prints the following error message:

cpu cpu0: dev_pm_opp_get_opp_count: OPP table not found (-19)

However, this is not really an error in this case because the
imx6q-cpufreq driver first calls dev_pm_opp_get_opp_count()
and if it fails, it means that platform code does not provide
OPP and then dev_pm_opp_of_add_table() will be called.

In order to avoid such confusing error message, move it to
debug level.

It is up to the caller of dev_pm_opp_get_opp_count() to check its
return value and decide if it will print an error or not.
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The drivers/base/power/ directory is special and contains code related
to power management core like system suspend/resume, hibernation, etc.
It was fine to keep the OPP code inside it when we had just one file for
it, but it is growing now and already has a directory for itself.

Lets move it directly under drivers/ directory, just like cpufreq and
cpuidle.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The notifier callbacks may want to call some OPP helper routines which
may try to take the same opp_table->lock again and cause a deadlock. One
such usecase was reported by Chanwoo Choi, where calling
dev_pm_opp_disable() leads us to the devfreq's OPP notifier handler,
which further calls dev_pm_opp_find_freq_floor() and it deadlocks.

We don't really need the opp_table->lock to be held across the notifier
call though, all we want to make sure is that the 'opp' doesn't get
freed while being used from within the notifier chain. We can do it with
help of dev_pm_opp_get/put() as well. Let's do it.

Cc: 4.11+ <stable@vger.kernel.org> # 4.11+
Fixes: 5b650b38 "PM / OPP: Take kref from _find_opp_table()"
Reported-by: NChanwoo Choi <cw00.choi@samsung.com>
Tested-by: NChanwoo Choi <cw00.choi@samsung.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Reviewed-by: NChanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

In order to support OPP switching, OPP layer needs to get pointer to the
clock for the device. Simple cases work fine without using the routines
added by this patch (i.e.  by passing connection-id as NULL), but for a
device with multiple clocks available, the OPP core needs to know the
exact name of the clk to use.

Add a new set of APIs to get that done.
Tested-by: NRajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

This code was required while the OPP core was managed with help of RCUs,
but not anymore. Get rid of unnecessary alloc/memcpy operations.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The code was overly complicated here because of the limitations that we
had with RCUs (Couldn't use opp-table and OPPs outside RCU protected
section and can't call sleep-able routines from within that). But that
is long gone now.

Reorganize _generic_set_opp_regulator() in order to avoid using "struct
dev_pm_set_opp_data" and copying data into it for the case where
opp_table->set_opp is not set.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Reading array at given index before checking if index is valid results in
illegal memory access.

The bug was detected using KASAN framework.
Signed-off-by: NAndrzej Hajda <a.hajda@samsung.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Fixes the following sparse warning:

drivers/base/power/opp/core.c:49:18: warning:
 symbol '_find_opp_table_unlocked' was not declared. Should it be static?
Signed-off-by: NWei Yongjun <weiyongjun1@huawei.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

dev_pm_opp_get_max_volt_latency() calls _find_opp_table() two times
effectively.

Merge _get_regulator_count() into dev_pm_opp_get_max_volt_latency() to
avoid that.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

As we don't use RCU locking anymore, there is no need to replace an
earlier OPP node with a new one. Just update the existing one.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The RCU locking isn't well suited for the OPP core. The RCU locking fits
better for reader heavy stuff, while the OPP core have at max one or two
readers only at a time.

Over that, it was getting very confusing the way RCU locking was used
with the OPP core. The individual OPPs are mostly well handled, i.e. for
an update a new structure was created and then that replaced the older
one. But the OPP tables were updated directly all the time from various
parts of the core. Though they were mostly used from within RCU locked
region, they didn't had much to do with RCU and were governed by the
mutex instead.

And that mixed with the 'opp_table_lock' has made the core even more
confusing.

Now that we are already managing the OPPs and the OPP tables with kernel
reference infrastructure, we can get rid of RCU locking completely and
simplify the code a lot.

Remove all RCU references from code and comments.

Acquire opp_table->lock while parsing the list of OPPs though.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Take reference of the OPP table from within _find_opp_table(). Also
update the callers of _find_opp_table() to call
dev_pm_opp_put_opp_table() after they have used the OPP table.

Note that _find_opp_table() increments the reference under the
opp_table_lock.

Now that the OPP table wouldn't get freed until the callers of
_find_opp_table() call dev_pm_opp_put_opp_table(), there is no need to
take the opp_table_lock or rcu_read_lock() around it. Drop them.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

This patch updates dev_pm_opp_find_freq_*() routines to get a reference
to the OPPs returned by them.

Also updates the users of dev_pm_opp_find_freq_*() routines to call
dev_pm_opp_put() after they are done using the OPPs.

As it is guaranteed the that OPPs wouldn't get freed while being used,
the RCU read side locking present with the users isn't required anymore.
Drop it as well.

This patch also updates all users of devfreq_recommended_opp() which was
returning an OPP received from the OPP core.

Note that some of the OPP core routines have gained
rcu_read_{lock|unlock}() calls, as those still use RCU specific APIs
within them.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com> [Devfreq]
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Add kref to struct dev_pm_opp for easier accounting of the OPPs.

Note that the OPPs are freed under the opp_table->lock mutex only.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Migrate all users of _add_opp_table() to use dev_pm_opp_get_opp_table()
to guarantee that the OPP table doesn't get freed while being used.

Also update _managed_opp() to get the reference to the OPP table.

Now that the OPP table wouldn't get freed while these routines are
executing after dev_pm_opp_get_opp_table() is called, there is no need
to take opp_table_lock. Drop them as well.

Now that _add_opp_table(), _remove_opp_table() and the unlocked release
routines aren't used anymore, remove them.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Take reference of the OPP table while adding and removing OPPs, that
helps us remove special checks in _remove_opp_table().
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Now that we have proper kernel reference infrastructure in place for OPP
tables, use it to guarantee that the OPP table isn't freed while being
used by the callers of dev_pm_opp_set_*() APIs.

Make them all return the pointer to the OPP table after taking its
reference and put the reference back with dev_pm_opp_put_*() APIs.

Now that the OPP table wouldn't get freed while these routines are
executing after dev_pm_opp_get_opp_table() is called, there is no need
to take opp_table_lock. Drop them as well.

Remove the rcu specific comments from these routines as they aren't
relevant anymore.

Note that prototypes of dev_pm_opp_{set|put}_regulators() were already
updated by another patch.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Add kref to struct opp_table for easier accounting of the OPP table.

Note that the new routine dev_pm_opp_get_opp_table() takes the reference
from under the opp_table_lock, which guarantees that the OPP table
doesn't get freed unless dev_pm_opp_put_opp_table() is called for the
OPP table.

Two separate release mechanisms are added: locked and unlocked. In
unlocked version the routines aren't required to take/drop
opp_table_lock as the callers have already done that. This is required
to avoid breaking git bisect, otherwise we may get lockdeps between
commits. Once all the users of OPP table are updated the unlocked
version shall be removed.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Add per OPP table lock to protect opp_table->opp_list.

Note that at few places opp_list is used under the rcu_read_lock() and
so a mutex can't be added there for now. This will be fixed by a later
patch.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Split out parts of _add_opp_table() and _remove_opp_table() into
separate routines. This improves readability as well.

Should result in no functional changes.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Let the OPP core provide helpers to register notifiers for any device,
instead of exposing srcu_head outside of the core.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NMyungJoo Ham <myungjoo.ham@samsung.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

There is only one user of dev_pm_opp_get_suspend_opp() and that uses it
to get the OPP rate for the suspend_opp.

Rename dev_pm_opp_get_suspend_opp() as dev_pm_opp_get_suspend_opp_freq()
and return the rate directly from it.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

There is no point in trying to find/allocate the table for every OPP
that is added for a device. It would be far more efficient to allocate
the table only once and pass its pointer to the routines that add the
OPP entry.

Locking is removed from _opp_add_static_v2() and _opp_add_v1() now as
the callers call them with that lock already held.

Call to _remove_opp_table() routine is also removed from _opp_free()
now, as opp_table isn't allocated from within _opp_allocate(). This is
handled by the routines which created the OPP table in the first place.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Later patches would want to remove OPP table (and its OPPs) using the
opp_table pointer instead of 'dev'.

In order to prepare for that, rename _dev_pm_opp_remove_table() as
_dev_pm_opp_find_and_remove_table() split out part of it.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The OPPs which are never successfully added using _opp_add() are not
required to be freed with the _opp_remove() routine, as a simple kfree()
is enough for them.

Introduce a new light weight routine _opp_free(), which will do that.

That also helps us removing the 'notify' parameter to _opp_remove(),
which isn't required anymore.

Note that _opp_free() contains a call to _remove_opp_table() as the OPP
table might have been added for this very OPP only. The
_remove_opp_table() routine returns quickly if there are more OPPs in
the table. This will be simplified in later patches though.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Make the naming consistent with how other routines are named.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

There are two types of duplicate OPPs that get different behavior from
the core:
A) An earlier OPP is marked 'available' and has same freq/voltages as
   the new one.
B) An earlier OPP with same frequency, but is marked 'unavailable' OR
   doesn't have same voltages as the new one.

The OPP core returns 0 for the first one, but -EEXIST for the second.

While the OPP core returns 0 for the first case, its callers don't free
the newly allocated OPP structure which isn't used anymore. Fix that by
returning -EBUSY instead of 0, but make the callers return 0 eventually.

As this isn't a critical fix, its not getting marked for stable kernel.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

If a platform specific OPP driver has called this routine first and set
the regulators, then the second call from cpufreq-dt driver will hit the
WARN_ON(). Remove the WARN_ON(), but continue to return error in such
cases.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The generic set_opp() handler isn't sufficient for platforms with
complex DVFS.  For example, some TI platforms have multiple regulators
for a CPU device. The order in which various supplies need to be
programmed is only known to the platform code and its best to leave it
to it.

This patch implements APIs to register platform specific set_opp()
callback.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Later patches would add support for custom set_opp() callbacks. This
patch separates out the code for _generic_set_opp() handler in order to
prepare for that.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

This patch adds infrastructure to manage multiple regulators and updates
the only user (cpufreq-dt) of dev_pm_opp_set{put}_regulator().

This is preparatory work for adding full support for devices with
multiple regulators.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Pass the entire supply structure instead of all of its fields.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

This is a preparatory step for multiple regulator per device support.
Move the voltage/current variables to a new structure.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

The OPP structure must not be used out of the rcu protected section.
Cache the values to be used in separate variables instead.

Cc: 4.6+ <stable@vger.kernel.org> # 4.6+
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Tested-by: NDave Gerlach <d-gerlach@ti.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>