also, check for a valid pad_ctrl_ofs before changing the
pad control register.
Signed-off-by: NSascha Hauer <s.hauer@pengutronix.de>