Use extack error reporting mechanism in addition to returning -EINVAL

NL_SET_ERR_* code shamelessy copy/paste/adjusted from act_pedit &
sch_cake and used as reference as to what I should have done in the
first place.
Signed-off-by: NKevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>