• S
    tests: send "bug in the test script" errors to the script's stderr · 165293af
    SZEDER Gábor 提交于
    Some of the functions in our test library check that they were invoked
    properly with conditions like this:
    
      test "$#" = 2 ||
      error "bug in the test script: not 2 parameters to test-expect-success"
    
    If this particular condition is triggered, then 'error' will abort the
    whole test script with a bold red error message [1] right away.
    
    However, under certain circumstances the test script will be aborted
    completely silently, namely if:
    
      - a similar condition in a test helper function like
        'test_line_count' is triggered,
      - which is invoked from the test script's "main" shell [2],
      - and the test script is run manually (i.e. './t1234-foo.sh' as
        opposed to 'make t1234-foo.sh' or 'make test') [3]
      - and without the '--verbose' option,
    
    because the error message is printed from within 'test_eval_', where
    standard output is redirected either to /dev/null or to a log file.
    The only indication that something is wrong is that not all tests in
    the script are executed and at the end of the test script's output
    there is no "# passed all N tests" message, which are subtle and can
    easily go unnoticed, as I had to experience myself.
    
    Send these "bug in the test script" error messages directly to the
    test scripts standard error and thus to the terminal, so those bugs
    will be much harder to overlook.  Instead of updating all ~20 such
    'error' calls with a redirection, let's add a BUG() function to
    'test-lib.sh', wrapping an 'error' call with the proper redirection
    and also including the common prefix of those error messages, and
    convert all those call sites [4] to use this new BUG() function
    instead.
    
    [1] That particular error message from 'test_expect_success' is
        printed in color only when running with or without '--verbose';
        with '--tee' or '--verbose-log' the error is printed without
        color, but it is printed to the terminal nonetheless.
    
    [2] If such a condition is triggered in a subshell of a test, then
        'error' won't be able to abort the whole test script, but only the
        subshell, which in turn causes the test to fail in the usual way,
        indicating loudly and clearly that something is wrong.
    
    [3] Well, 'error' aborts the test script the same way when run
        manually or by 'make' or 'prove', but both 'make' and 'prove' pay
        attention to the test script's exit status, and even a silently
        aborted test script would then trigger those tools' usual
        noticable error messages.
    
    [4] Strictly speaking, not all those 'error' calls need that
        redirection to send their output to the terminal, see e.g.
        'test_expect_success' in the opening example, but I think it's
        better to be consistent.
    Signed-off-by: NSZEDER Gábor <szeder.dev@gmail.com>
    Signed-off-by: NJunio C Hamano <gitster@pobox.com>
    165293af
test-lib-functions.sh 28.8 KB