Remove checking if a unsigned is less than zero

This was found using a static code analysis program called cppcheck.
Signed-off-by: NRickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Acked-by: NSrinivas Kandagatla <srinivas.kandagatla@linaro.org>
Acked-by: NMaxime Coquelin <maxime.coquelin@st.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>