Only call driver disable when device isn't already disabled, which also
handles the driver not loaded case.
Signed-off-by: NJani Nikula <ext-jani.1.nikula@nokia.com>
Signed-off-by: NTomi Valkeinen <tomi.valkeinen@nokia.com>