The BitArray approach wasn't meaningfully reducing complexity, so we'll
revert to an earlier approach that reduces allocations (by reusing an
existing ReadOnlyCollection<byte> as the backing data when possible).