The L2 cache is present on the newer SAMA5D2 and SAMA5D4 families, but
apparently not for the older SAMA5D3.

Solves a build-time regression with the following symptom:

sama5.c:(.init.text+0x48): undefined reference to `outer_cache'

Fixes: 3b5a7ca7 ("ARM: at91: setup outer cache .write_sec() callback if needed")
Signed-off-by: NPeter Rosin <peda@axentia.se>
[claudiu.beznea: delete "At least not always." from commit description]
Signed-off-by: NClaudiu Beznea <claudiu.beznea@microchip.com>
Link: https://lore.kernel.org/r/b7f8dacc-5e1f-0eb2-188e-3ad9a9f7613d@axentia.se