This simplies error and cleanup code paths.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NMark Brown <broonie@linaro.org>