This reverts commit eba65d17.

This broke audio on Veyron Jerry Chromebooks and I now cannot reproduce
the problem I was trying to fix even with this commit reverted, so it
seems that this was completely the wrong thing to do.
Reported-by: NEnric Balletbo Serra <eballetbo@gmail.com>
Signed-off-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NMark Brown <broonie@kernel.org>