1. 06 7月, 2010 6 次提交
  2. 30 6月, 2010 1 次提交
  3. 28 6月, 2010 1 次提交
  4. 26 6月, 2010 5 次提交
    • Æ
      tests: Say "pass" rather than "ok" on empty lines for TAP · 335f8787
      Ævar Arnfjörð Bjarmason 提交于
      Lines that begin with "ok" confuse the TAP harness because it can't
      distinguish them from a test counter. Work around the issue by saying
      "pass" instead, which isn't a reserved TAP word.
      Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: NJunio C Hamano <gitster@pobox.com>
      335f8787
    • Æ
      tests: Skip tests in a way that makes sense under TAP · fadb5156
      Ævar Arnfjörð Bjarmason 提交于
      SKIP messages are now part of the TAP plan. A TAP harness now knows
      why a particular test was skipped and can report that information. The
      non-TAP harness built into Git's test-lib did nothing special with
      these messages, and is unaffected by these changes.
      Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: NJunio C Hamano <gitster@pobox.com>
      fadb5156
    • Æ
      test-lib: output a newline before "ok" under a TAP harness · 57e1538a
      Ævar Arnfjörð Bjarmason 提交于
      Some tests in the testsuite will emit a line that doesn't end with a
      newline, right before we're about to output "ok" or "not ok". This
      breaks the TAP output with "Tests out of sequence" errors since a TAP
      harness can't understand this:
      
          ok 1 - A test
          [some output here]ok 2 - Another test
          ok 3 - Yet another test
      
      Work around it by emitting an empty line before we're about to say
      "ok" or "not ok", but only if we're running under --verbose and
      HARNESS_ACTIVE=1 is set, which'll only be the case when running under
      a harnesses like prove(1).
      
      I think it's better to do this than fix each tests by adding `&& echo'
      everywhere. More tests might be added that break TAP in the future,
      and a human isn't going to look at the extra whitespace, since
      HARNESS_ACTIVE=1 always means a harness is reading it.
      
      The tests that had issues were:
      
         t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404,
         t5407, t7402, t7003, t9001
      
      With this workaround the entire test suite runs without errors under:
      
          prove -j 10 ./t[0-9]*.sh :: --verbose
      Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: NJunio C Hamano <gitster@pobox.com>
      57e1538a
    • Æ
      test-lib: Make the test_external_* functions TAP-aware · d998bd4a
      Ævar Arnfjörð Bjarmason 提交于
      Before TAP we just ran the Perl test and assumed that it failed if
      nothing was printed on STDERR. Continue doing that, but introduce a
      `test_external_has_tap' variable which tests can set to indicate that
      they're outputting TAP.
      
      If it's set we won't output a test plan, but trust the external test
      to do so. That way we can make external tests work with a TAP harness,
      but still maintain compatibility with test-lib's own way of tracking
      tests through the test-results directory.
      Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: NJunio C Hamano <gitster@pobox.com>
      d998bd4a
    • Æ
      test-lib: Adjust output to be valid TAP format · 5099b99d
      Ævar Arnfjörð Bjarmason 提交于
      TAP, the Test Anything Protocol, is a simple text-based interface
      between testing modules in a test harness. test-lib.sh's output was
      already very close to being valid TAP. This change brings it all the
      way there. Before:
      
          $ ./t0005-signals.sh
          *   ok 1: sigchain works
          * passed all 1 test(s)
      
      And after:
      
          $ ./t0005-signals.sh
          ok 1 - sigchain works
          # passed all 1 test(s)
          1..1
      
      The advantage of using TAP is that any program that reads the format
      (a "test harness") can run the tests. The most popular of these is the
      prove(1) utility that comes with Perl. It can run tests in parallel,
      display colored output, format the output to console, file, HTML etc.,
      and much more. An example:
      
          $ prove ./t0005-signals.sh
          ./t0005-signals.sh .. ok
          All tests successful.
          Files=1, Tests=1,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.01 cusr  0.02 csys =  0.06 CPU)
          Result: PASS
      
      prove(1) gives you human readable output without being too
      verbose. Running the test suite in parallel with `make test -j15`
      produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
      makes it easy to follow what's going on.
      
      All this patch does is re-arrange the output a bit so that it conforms
      with the TAP spec, everything that the test suite did before continues
      to work. That includes aggregating results in t/test-results/, the
      --verbose, --debug and other options for tests, and the test color
      output.
      
      TAP harnesses ignore everything that they don't know about, so running
      the tests with --verbose works:
      
          $ prove ./t0005-signals.sh :: --verbose --debug
          ./t0005-signals.sh .. Terminated
          ./t0005-signals.sh .. ok
          All tests successful.
          Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.01 cusr  0.01 csys =  0.05 CPU)
          Result: PASS
      
      Just supply the -v option to prove itself to get all the verbose
      output that it suppresses:
      
          $ prove -v ./t0005-signals.sh :: --verbose --debug
          ./t0005-signals.sh ..
          Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
          expecting success:
                  test-sigchain >actual
                  case "$?" in
                  143) true ;; # POSIX w/ SIGTERM=15
                    3) true ;; # Windows
                    *) false ;;
                  esac &&
                  test_cmp expect actual
          Terminated
          ok 1 - sigchain works
          # passed all 1 test(s)
          1..1
          ok
          All tests successful.
          Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.01 cusr  0.01 csys =  0.04 CPU)
          Result: PASS
      
      As a further example, consider this test script that uses a lot of
      test-lib.sh features by Jakub Narebski:
      
          #!/bin/sh
      
          test_description='this is a sample test.
      
          This test is here to see various test outputs.'
      
          . ./test-lib.sh
      
          say 'diagnostic message'
      
          test_expect_success 'true  test' 'true'
          test_expect_success 'false test' 'false'
      
          test_expect_failure 'true  test (todo)' 'true'
          test_expect_failure 'false test (todo)' 'false'
      
          test_debug 'echo "debug message"'
      
          test_done
      
      The output of that was previously:
      
          * diagnostic message                      # yellow
          *   ok 1: true  test
          * FAIL 2: false test                      # bold red
                  false
          *   FIXED 3: true  test (todo)
          *   still broken 4: false test (todo)     # bold green
          * fixed 1 known breakage(s)               # green
          * still have 1 known breakage(s)          # bold red
          * failed 1 among remaining 3 test(s)      # bold red
      
      But is now:
      
          diagnostic message                                    # yellow
          ok 1 - true  test
          not ok - 2 false test                                 # bold red
          #       false
          ok 3 - true  test (todo) # TODO known breakage
          not ok 4 - false test (todo) # TODO known breakage    # bold green
          # fixed 1 known breakage(s)                           # green
          # still have 1 known breakage(s)                      # bold red
          # failed 1 among remaining 3 test(s)                  # bold red
          1..4
      
      All the coloring is preserved when the test is run manually. Under
      prove(1) the test performs as expected, even with --debug and
      --verbose options:
      
          $ prove ./example.sh :: --debug --verbose
          ./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
          Failed 1/4 subtests
                  (1 TODO test unexpectedly succeeded)
      
          Test Summary Report
          -------------------
          ./example.sh (Wstat: 256 Tests: 4 Failed: 1)
            Failed test:  2
            TODO passed:   3
            Non-zero exit status: 1
          Files=1, Tests=4,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.00 cusr  0.01 csys =  0.03 CPU)
          Result: FAIL
      
      The TAP harness itself doesn't get confused by the color output, they
      aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).
      Signed-off-by: NÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: NJunio C Hamano <gitster@pobox.com>
      5099b99d
  5. 24 6月, 2010 2 次提交
  6. 23 6月, 2010 18 次提交
  7. 22 6月, 2010 7 次提交