correct the sequence when unload this module
Signed-off-by: NBo Shen <voice.shen@atmel.com>
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>