We should not remove the workqueue, we just need to ensure that the
workqueues are synced. The workqueues are torn down on ctx removal.

Cc: stable@vger.kernel.org
Fixes: 6b06314c ("io_uring: add file set registration")
Reported-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>