If we are not changing the control register value, avoid writing to it.
Writes to the control register can be very expensive, taking around a
hundred cycles or so.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>