If we detach the kernel drivers on the first set_config, then they will
be still attached when the device gets its initial reset. Causing the drivers
to re-initialize the device after the reset, dirtying the device state.
Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>