Allow users to keep on specifying their output frequency when disabling
the reference clock.
Reported-by: NKyung Kwee Ryu <Kyung-Kwee.Ryu@wolfsonmicro.com>
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>