Store the group pointer before disassociating the VIN from the group.

Fixes: 3bb4c3bc ("media: rcar-vin: add group allocator functions")
Reported-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NNiklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: NSimon Horman <horms+renesas@verge.net.au>
Signed-off-by: NMauro Carvalho Chehab <mchehab+samsung@kernel.org>