Remove checking if a unsigned is less than zero

This was found using a static code analysis program called cppcheck.
Signed-off-by: NRickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>