[ Upstream commit efc2e0bd9594060915696a418564aefd0270b1d6 ]

It is not correct to assign the 24MHz clock oscillator to the GPIO
ports.

Fix it by assigning the proper GPIO clocks instead.
Signed-off-by: NOtavio Salvador <otavio@ossystems.com.br>
Tested-by: NFabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: NHeiko Stuebner <heiko@sntech.de>
Signed-off-by: NSasha Levin <sashal@kernel.org>