Add support for QCOM_ICC_TAG_PERF_MODE on RPMh architectures. When
QCOM_ICC_TAG_PERF_MODE is specified using icc_set_tag(), each node in
the requested path will use latency-optimized settings if supported by
the node. The improved latency comes at the cost of higher power.

Change-Id: I83c52e5677f8842b4c17987053d566971b84086f
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Add binding for QCOM_ICC_TAG_PERF_MODE, which can be used with
icc_set_tag() to indicate that the path should use latency-optimized
settings for each node in the path if supported.

Change-Id: I748df7bdaa79784ade4ad9a135e4f91b92b97513
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Some BCMs aren't directly associated with the data path (i.e. ACV) and
therefore don't communicate using BW. Instead, they are simply
enabled/disabled with a simple bit mask. Add support for these.

Change-Id: I9b73e0ce05a09338d22571ff311c5b058fb8e1b2
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Some BCMs that aren't directly associated with NOC ports (i.e. ACV)
publish their width as 0 in cmd_db. The client votes are multiplied by
this width when adjusting for node/BCM width differences. This means a
zero width also zeroes the client votes, making it impossible to vote on
these zero-width BCMs. Set a width floor of 1 to allow aggregation on
these nodes.

Also set a floor on the BCM units. They should never by 0 in cmd_db, but
if they are it would cause a div-by-zero.

Change-Id: I14ba89411ec02077db6d2c27b6de932c5b7dfe1e
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Currently avg and peak bandwidth are voted as allowed max on kernel boot.
Add stub icc_get_bw() callback that return default avg, peak bandwidth.

Change-Id: I5c42f933d770e2c63c8bbc2d8e7afe5b9748ba1d
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>

Add a debug library that provides QCOM-specific interconnect debug
features. This includes support for printing all enabled interconnect
votes to console when entering suspend.

Change-Id: I61302f95c94176f833bd1c88ddc76b88121b3f79
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Add stub icc_set() and icc_aggregate() callbacks that return success
without doing anything. These can be used early in development before
the rest of the system is ready to service RPMh requests. Additionally,
they can be used during probe to avoid unnecessary set/aggregate() calls
each time icc_node_add() is called.

Change-Id: If495ffc8f9839c3d2651c166c62a7936927e3c02
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Build qnoc-qos.c as a module to make the symbols
available and avoid build warnings.

Change-Id: I62ab7a04225d71c036cd78f38259a98f099632ed
Signed-off-by: Odelu Kukatla <okukatla@codeaurora.org>
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Since clk_bulk_prepare_enable function is defined with __must_check option,
add check to return status of clk_bulk_prepare_enable in qcom_icc_set
function.

Change-Id: I69fed5bdda38edfa89af3d899333d6e6f9f939f9
Signed-off-by: Naveen Yadav <naveenky@codeaurora.org>

Move the QoS configuration to happen on the first non-zero bandwidth
request from the consumer driver. This allows configuring QoS also
for consumers like IPA, which can probe asynchronously and their clock
should not be touched until the firmware has completed booting.

Change-Id: I2a8e385d99f3c489d920a7c85e9415d540f6df87
Signed-off-by: Georgi Djakov <gdjako@codeaurora.org>
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

AMC requests made while the RSC is in solver mode result in a -EBUSY
response from the rpmh driver. This is expected and is a non-issue since
in this case we just need the request to apply to the WAKE set instead.
It's only possible to see this for secondary RSCs like the Display RSC,
since the primary Apps RSC will only be in solver during the power
collapse/restore sequences while SW isn't running. Ignore -EBUSY so we
don't return an error and short-circuit before sending the necessary
WAKE/SLEEP requests.

Change-Id: Id34b91c6e46d282be2522434d3c2b4837291b3b7
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Add support for targeting alternate bcm-voters which
live under different RSC devices.

Change-Id: Ia1ad8a4c6c6b8531f0dae67cc7b22e2faa55c418
Signed-off-by: David Dai <daidavid1@codeaurora.org>
Signed-off-by: Georgi Djakov <gdjako@codeaurora.org>
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Introduce support to initialize QoS settings for
QNOC platforms.

Change-Id: I17a237e88f21f7bbc98621d072fff9c1b1bc6403
Signed-off-by: David Dai <daidavid1@codeaurora.org>
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Add a new definition for fast-path TCS configuration.

Change-Id: I610b14dcd893885e5b1c32c434caa7aa7102fd4c
Signed-off-by: Lina Iyer <ilina@codeaurora.org>

In OS-Initiated mode of PSCI, a serialization is established in Linux
between the last CPU powering down the cluster / domain and the CPU
powering up back. The serialization is established by the genpd
framework and therefore it doesn't help to lock around the cache.

Refactor out the core of the flush functionality from the locking
wrappers. Calling the core functionality avoids a spin trylock, which
the RCU framework might consider suspicious under the circumstances.

Change-Id: I242fb4827b71152bda36ea5870393054b8703309
Signed-off-by: Lina Iyer <ilina@codeaurora.org>

Introduce support to compile devicetree sources out-of-tree. This is
useful so that dt-bindings and tools are provided Kbuild.

Change-Id: I3c15e43dd132b01d5ea7a4518aa91e809e83eb48
Signed-off-by: Elliot Berman <eberman@codeaurora.org>

RSC is part the CPU subsystem and powers off the CPU domains when all
the CPUs and no RPMH transactions are pending from any of the drivers.
The RSC needs to flush the 'sleep' and 'wake' votes that are critical
for saving power when all the CPUs are in idle. Doing this every time a
CPU powers down, increases latency to enter idle state. So, we want to
do this, only when it makes sense - when the last CPU is powering down.

Let's make RSC part of the CPU PM domains, by attaching it to the
cluster PD (defined in DT). Registering for PM domain notifications, RSC
driver can be notified that the last CPU is powering down. When the last
CPU is powering down the domain, let's flush the 'sleep' and 'wake'
votes that are stored in the data buffers into the hardware.

Change-Id: Id748413f0f86a4ac73c37e0780b2b22a143a1e61
Signed-off-by: Lina Iyer <ilina@codeaurora.org>

lockdep_assert_irqs_disabled() was added to check rpmh_flush() can
only be invoked when irqs are disabled, this is true for APPS RSC
as the last CPU going to deepest low power mode is writing sleep and
wake TCSes.

However for Display RSC, drivers can invoke rpmh_write_sleep_and_wake()
to immediately write cached sleep and wake sets to TCSes from any CPU.
Conditionally check if RSC controller supports solver mode then do not
check for irqs disabled as such RSC can write sleep and wake TCSes at any
point.

Change-Id: I8ad05cd81af15a0d57353b55d8098cf0e47f79f7
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Let clients call rpmh_write_sleep_and_wake() to immediately
write cached sleep and wake data to the TCSes.

Change-Id: I04a94ee622a39a0aa0e829f4c40190a870fb2fb1
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Allow RPMH requests to succeed even when the always-On subsystem is not
available. In standalone mode, requests are never sent to the TCS
mailbox driver. The command DB provides the information to make the
decision to run in standalone mode or not.

Change-Id: Ia2bc5024da0d7e6607347b1a481b2feb9584059a
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Controllers may be in 'solver' state, where they could be in autonomous
mode executing low power modes for their hardware and as such are not
available for sending active votes. Device driver may notify RPMH API
that the controller is in solver mode and when in such mode, disallow
requests from platform drivers for state change using the RSC.

Change-Id: Ibe777d23fd58489cfb8132c0168af8e29284084d
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Output the TCS state when the debug api is invoked. The state of the
TCS, the contents and the IRQ status is presented. Additionally, crash
the system if any TCS is busy to help with the debug.

Change-Id: I7ef5480cc6da7208d5f82351d863561fcc1abf68
Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org>
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Add sync_state functionality to remove threshold held during initial
boot up and clean up any linger proxy resources that were voted for
from bootloaders.

Change-Id: Ib2fe3f3e54775b85c3ee8107c9aafdcd32841017
Signed-off-by: David Dai <daidavid1@codeaurora.org>
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>

Add a high bandwidth vote for critical NoCs during boot up.

Change-Id: I739df67083e13f542c0021054b1ca4c1f2d9bd09
Signed-off-by: David Dai <daidavid1@codeaurora.org>

The command DB may be initialized outside the AOP, indicated by the
reserved bit 0. Export this information through a function.

Change-Id: I7b8c685cc03d919c5adb1b91134078bd96599bbd
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Enable Command DB, RPMH and PDC drivers.

Change-Id: I83e05a766af794fb47cdf425b80fd25ab71385a5
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>

Enable CONFIG_REGULATOR_DEBUG_CONTROL so that regulators can be
controlled from debugfs for testing purposes.

Change-Id: I92dc148d3bdd60fa76d1db87ff69897ef6b62c2f
Signed-off-by: David Collins <collinsd@codeaurora.org>

Add debug-regulator.ko in modules.list.msm.kalama.  This ensures
that the debug-regulator module is loaded automatically from the
first stage RAM disk image during Linux kernel boot up.

Change-Id: Ia1d857aceaad8fbd9d6976a3e5219d81abd4b3ff
Signed-off-by: David Collins <collinsd@codeaurora.org>

The regulator_lock() and regulator_unlock() functions were unexported in
commit 4c9db393 ("regulator: unexport regulator_lock/unlock()").
The debug-regulator library uses these calls to ensure mutual
exclusion while reading regulator device and regulator consumer
states.  It does not rely upon the per-process reentrancy
provided by regulator_lock() and regulator_unlock().  Therefore,
it is safe to replace the calls with ww_mutex_lock() and
ww_mutex_unlock() respectively.  Replace these calls.

Change-Id: I2dc337d5a126191cce8d2ce707ad910770ab2e1e
Signed-off-by: David Collins <collinsd@codeaurora.org>

Add support to print a list of the regulators found to be
enabled when entering suspend into the kernel log.  This is
useful for identifying which regulator consumers are not voting
as expected during system suspend and resume.

Change-Id: I6089871d7a5945d75fec40fdfe4787efafa1aaa0
Signed-off-by: David Collins <collinsd@codeaurora.org>

Add support for debugfs control of a regulator's state.  This
is useful for testing and debugging.

Change-Id: I0be1fc1e6278df828d5ffce22f425c95b425f041
Signed-off-by: David Collins <collinsd@codeaurora.org>

Add a debug control library which can be used by regulator drivers
to provide a debugfs interface to vote on the enable state,
voltage, mode, and load current of its regulators.  This interface
is useful during live testing and debugging.

Change-Id: Ib7625d3f217d4959cbd9eecb74e357e9e61503b7
Signed-off-by: David Collins <collinsd@codeaurora.org>

Add regulator stubs build config file for kalama.

Change-Id: I59afc05474e5bd0511eda52de2a391d87017d9a7
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>

Add clk-stubs build config file for kalama.

Change-Id: Idfbdf9c5e03213c22da97d70332f1c0e8ad2b1d1
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>