net/packet: fix 4gb buffer limit due to overflow check

mainline inclusion
from mainline-v5.0
commit fc62814d690c
category: bugfix
bugzilla: 9556
CVE: NA

-------------------------------------------------

When calculating rb->frames_per_block * req->tp_block_nr the result
can overflow. Check it for overflow without limiting the total buffer
size to UINT_MAX.

This change fixes support for packet ring buffers >= UINT_MAX.

Fixes: 8f8d28e4 ("net/packet: fix overflow in check for tp_frame_nr")
Signed-off-by: NKal Conley <kal.conley@dectris.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NShangli <shangli1@huawei.com>
Reviewed-by: NMao Wenan <maowenan@huawei.com>
Signed-off-by: NYang Yingliang <yangyingliang@huawei.com>

net/packet/af_packet.c

@@ -4275,7 +4275,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
 		rb->frames_per_block = req->tp_block_size / req->tp_frame_size;
 		if (unlikely(rb->frames_per_block == 0))
 			goto out;
-		if (unlikely(req->tp_block_size > UINT_MAX / req->tp_block_nr))
+		if (unlikely(rb->frames_per_block > UINT_MAX / req->tp_block_nr))
 			goto out;
 		if (unlikely((rb->frames_per_block * req->tp_block_nr) !=
 					req->tp_frame_nr))