Otherwise, a xfrm policy with sport/dport being set cannot be matched.
Signed-off-by: NMartynas Pumputis <martynas@weave.works>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>