Allow to unregister fixed factor clock.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>