Several drivers have the same (and incorrect) code in their
_remove() handler.

Coalesce this into a shared function.
Signed-off-by: NIan Molton <ian@mnementh.co.uk>
Reviewed-by: NSebastian Reichel <sebastian.reichel@collabora.co.uk>
Signed-off-by: NMarcel Holtmann <marcel@holtmann.org>