Fix the regression caused by commit ad78c918 ("usb: musb: dsps: just
start polling already") which causes polling the ID pin status even in
device-only mode.

Fixes: ad78c918 ("usb: musb: dsps: just start polling already")
Signed-off-by: NBin Liu <b-liu@ti.com>
Signed-off-by: NFelipe Balbi <balbi@ti.com>