The cache is based on the full register map so confuses things if used
for a partial map.
Reported-by: NBard Liao <bardliao@realtek.com>
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>