Commit 409de00e changed the logic to only match PCI devices
while moving this before Remove*HostDevice calls.

https://bugzilla.redhat.com/show_bug.cgi?id=1342874