Byte-swapping in place is always a little dubious.

Let's instead define this field to always be big-endian, and do the
swapping on demand where we need it.
Signed-off-by: NJ. Bruce Fields <bfields@redhat.com>