This has never been completely sufficient and now happens too late to
paper over the cache coherency problems with the current USB stack.

Cc: Marek Vasut <marex@denx.de>
Signed-off-by: NTom Rini <trini@ti.com>
Signed-off-by: NIlya Yanok <ilya.yanok@cogentembedded.com>