Simplify the endpoint sanity check by letting core verify that the
required endpoints are present.

Note that the driver registers four ports but uses five bulk-endpoint
pairs.
Signed-off-by: NJohan Hovold <johan@kernel.org>