When removing a serial gadget driver, the kernel warning message is outputted.
This patch fixed this issue.

The pch_udc driver did not have disconnection processing of gadget.
Signed-off-by: NToshiharu Okada <toshiharu-linux@dsn.okisemi.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>