Without this condition, all controllers will do this delay,
and increase the resume time.

Only enabled and unsuspended port needs this delay, but
Some buggy hardware(like Synopsys usb controller) will
clear suspend bit once they receive/send resume signal,
so it takes resume bit as consideration.

Tested it at Freescale i.mx6q Sabrelite board.
Signed-off-by: NPeter Chen <peter.chen@freescale.com>
Acked-by: NAlan Stern <stern@rowland.harvard.edu>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>