The wire format dictates that payload_len fits into 4 bytes.  Take u32
instead of size_t to reflect that.

All callers pass a small integer, so no changes required.
Signed-off-by: NIlya Dryomov <idryomov@gmail.com>