Properly handle resource cleanup on unplug/exit.

Spotted by Jonathan Cameron
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>