The xilinxfb driver is improperly casting a physical address to a
u32, and the probe routine isn't as straight forward as it could be.
(discovered by gcc spitting out warnings on most recent change to
xilinxfb driver).

This patch fixes the cast and simplifies the probe path.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Tested-by: NJohn Linn <john.linn@xilinx.com>