For gpio controller contain register PDDR, when set one target bit,
current logic will clear all other bits, this is wrong. Use operator
'|=' to fix it.

Fixes: 659d8a62 ("gpio: vf610: add imx7ulp support")
Reviewed-by: NPeng Fan <peng.fan@nxp.com>
Signed-off-by: NHaibo Chen <haibo.chen@nxp.com>
Signed-off-by: NBartosz Golaszewski <brgl@bgdev.pl>