The config.speed_hz is just setted at the two lines ahead of it,
obviously we don't need to copy the code to set the config.speed_hz
again. Remove it for a cleanup.

Cc: Mark Brown <broonie@opensource.wolfsonmicro.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NHui Wang <jason77.wang@gmail.com>
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>