Make debugging real problems easier by not trying to disable an EP that
was not yet enabled.
Reviewed-by: NPeter Chen <peter.chen@nxp.com>
Signed-off-by: NMichał Mirosław <mirq-linux@rere.qmqm.pl>
Signed-off-by: NFelipe Balbi <balbi@kernel.org>