Tweak how binary files are handled internally.
This commit fixes two issues. The first issue is that if a file contained many NUL bytes without any LF bytes, then the InputBuffer would read the entire file into memory. This is not typically a problem, but if you run rg on /proc, then bad things can happen when reading virtual memory mapping files. Arguably, such files should be ignored, but we should also try to avoid exhausting memory too. We fix this by pushing the `-a/--text` flag option down into InputBuffer, so that it knows to stop immediately if it finds a NUL byte. The other issue this fixes is that binary detection is now applied to every buffer instead of just the first one. This helps avoid detecting too many files as plain text if the first parts of a binary file happen to contain no NUL bytes. This issue still persists somewhat in the memory map searcher, since we probably don't want to search the entire file upfront for NUL bytes before actually performing our search. Instead, we search the first 10KB for now. Fixes #52, Fixes #311
Showing
想要评论请 注册 或 登录