merge more of the bf54x and !bf54x gpio code together to
cut down on #ifdef mess
Signed-off-by: NMike Frysinger <vapier.adi@gmail.com>
Signed-off-by: NBryan Wu <cooloney@kernel.org>