VT switch back/forth from console to xserver (for example) has potential
to go horribly wrong if a dynamic DP MST connector ends up in the saved
modeset that is restored when switching back to fbcon.

When removing a dynamic connector, don't forget to clean up the saved
state.

v1: original
v2: null out set->fb if no more connectors to avoid making i915 cranky

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1184968
Cc: stable@vger.kernel.org #v3.17+
Signed-off-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>