This reverts commit ddb09754.

Linus objected to this originally, I can see why it might be needed, but
given that no one spoke up defending this patch, I'm going to revert it.

If you have hardware that requires this change, please speak up in the
future and defend the patch.
Reported-by: NLinus Torvalds <torvalds@linux-foundation.org>
Cc: Bin Wang <binw@marvell.com>
Cc: Nobuhiro Iwamatsu <nobuhiro.iwamatsu.yj@renesas.com>
Cc: Norbert Ciosek <norbertciosek@gmail.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>