Prevent "BUG: scheduling while atomic: rmmod" stack trace.

Stop setting spin_locks before calling OS functions to remove devices.

Link: https://lore.kernel.org/r/164375207296.440833.4996145011193819683.stgit@brunhilda.pdev.netReviewed-by: NScott Benesh <scott.benesh@microchip.com>
Reviewed-by: NScott Teel <scott.teel@microchip.com>
Reviewed-by: NKevin Barnett <kevin.barnett@microchip.com>
Signed-off-by: NDon Brace <don.brace@microchip.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>