The work cancelling has had several hazards, ranging from potentially
executing work after the driver is in OFF state, to executing work after
the driver and relevant memory structures are already removed.

Fix these.
Signed-off-by: NJuuso Oikarinen <juuso.oikarinen@nokia.com>
Reviewed-by: NTeemu Paasikivi <ext-teemu.3.paasikivi@nokia.com>
Signed-off-by: NLuciano Coelho <luciano.coelho@nokia.com>