Remove some legacy code and replace it by the clksrc-of code.

Do some cleanup and code consolidation.
Signed-off-by: NYoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>