Current ADG driver is over-writing flags. This patch fixes it.
Reported-by: NHiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NMark Brown <broonie@kernel.org>