• S
    xread, xwrite: limit size of IO to 8MB · 0b6806b9
    Steffen Prohaska 提交于
    Checking out 2GB or more through an external filter (see test) fails
    on Mac OS X 10.8.4 (12E55) for a 64-bit executable with:
    
        error: read from external filter cat failed
        error: cannot feed the input to external filter cat
        error: cat died of signal 13
        error: external filter cat failed 141
        error: external filter cat failed
    
    The reason is that read() immediately returns with EINVAL when asked
    to read more than 2GB.  According to POSIX [1], if the value of
    nbyte passed to read() is greater than SSIZE_MAX, the result is
    implementation-defined.  The write function has the same restriction
    [2].  Since OS X still supports running 32-bit executables, the
    32-bit limit (SSIZE_MAX = INT_MAX = 2GB - 1) seems to be also
    imposed on 64-bit executables under certain conditions.  For write,
    the problem has been addressed earlier [6c642a].
    
    Address the problem for read() and write() differently, by limiting
    size of IO chunks unconditionally on all platforms in xread() and
    xwrite().  Large chunks only cause problems, like causing latencies
    when killing the process, even if OS X was not buggy.  Doing IO in
    reasonably sized smaller chunks should have no negative impact on
    performance.
    
    The compat wrapper clipped_write() introduced earlier [6c642a] is
    not needed anymore.  It will be reverted in a separate commit.  The
    new test catches read and write problems.
    
    Note that 'git add' exits with 0 even if it prints filtering errors
    to stderr.  The test, therefore, checks stderr.  'git add' should
    probably be changed (sometime in another commit) to exit with
    nonzero if filtering fails.  The test could then be changed to use
    test_must_fail.
    
    Thanks to the following people for suggestions and testing:
    
        Johannes Sixt <j6t@kdbg.org>
        John Keeping <john@keeping.me.uk>
        Jonathan Nieder <jrnieder@gmail.com>
        Kyle J. McKay <mackyle@gmail.com>
        Linus Torvalds <torvalds@linux-foundation.org>
        Torsten Bögershausen <tboegi@web.de>
    
    [1] http://pubs.opengroup.org/onlinepubs/009695399/functions/read.html
    [2] http://pubs.opengroup.org/onlinepubs/009695399/functions/write.html
    
    [6c642a] commit 6c642a87
        compate/clipped-write.c: large write(2) fails on Mac OS X/XNU
    Signed-off-by: NSteffen Prohaska <prohaska@zib.de>
    Signed-off-by: NJunio C Hamano <gitster@pobox.com>
    0b6806b9
t0021-conversion.sh 5.5 KB