I'm trying to reduce the WARNs during driver reload and this was one of
them. Also while at it remove the redundant condition from before
unregister_shrinker().

v2:
- fix the error path too and move the unregister to its logical place
(Chris)
- remove redundant condition from before unregister_shrinker()
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>