This makes lifecycle management a bit easier thanks to ref counting, and
it is closer to what the qemu driver does.
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Signed-off-by: NRyan Moeller <ryan@iXsystems.com>