While we need to clean up unused single ended line outputs we don't want
to do this if the outputs are in differential mode.
Signed-off-by: NMark Brown <broonie@opensource.wolfsonmicro.com>