Just use the address family from the proc private data instead of copying
it into per-file data.
Signed-off-by: NChristoph Hellwig <hch@lst.de>