Allow them to work when the device is unclocked.
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>