Rather than referencing the cache directly read back the values we are
going to restore, supporting refactoring to use regmap.
Signed-off-by: NMark Brown <broonie@linaro.org>