This reverts commit 1e4a8064.

This creates more problems than it solves right now. Compile
testing needs to go in with patches fixing the problems it
uncovers.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>