This simplfies the code and slightly reduces the startup time.
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Acked-by: NLiam Girdwood <lrg@slimlogic.co.uk>