Current ap4evb had issue which couldn't boot sometimes.
This patch fixup it.
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>