This causes the connector to not be added since i2c init fails
for the adapter.  Fixes:
https://bugs.freedesktop.org/show_bug.cgi?id=31688

Noticed by Ari Savolainen.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: Ari Savolainen <ari.m.savolainen@gmail.com>
Cc: Jean Delvare <khali@linux-fr.org>
Cc: stable@kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>