usb_gadget_ops :: udc_stop might be called with driver = NULL since
511f3c53
(usb: gadget: udc-core: fix a regression during gadget driver unbinding)

Because of that, 2nd times insmod goes fail.
This patch fixes it up.
Reported-by: NYusuke Goda <yusuke.goda.sx@renesas.com>
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NFelipe Balbi <balbi@ti.com>