Sometimes enumeration fails (about 1 in 50 times on my custom board).

The underlying reason is probably electrical but Linux does not have
the problem.

Comparing the Linux / u-boot implementations shows that Linux
retries the error case whereas u-boot aborts early.

Removing the early abort in u-boot fixes the problem.
Signed-off-by: NMartin Fuzzey <martin.fuzzey@flowbird.group>