When the DRM device is torn down and the connector is removed, make sure
to detach the panel to make sure there are no dangling pointers.
Signed-off-by: NThierry Reding <treding@nvidia.com>