A patch that changes the filetype (e.g. regular file to symlink) of a path
must be split into a deletion event followed by a creation event, which
means that we need to have two independent metainfo lines for each.
However, the code reused the single set of metainfo lines.

As the blob object names recorded on the index lines are usually not used
nor validated on the receiving end, this is not an issue with normal use
of the resulting patch.  However, when accepting a binary patch to delete
a blob, git-apply verified that the postimage blob object name on the
index line is 0{40}, hence a patch that deletes a regular file blob that
records binary contents to create a blob with different filetype (e.g. a
symbolic link) failed to apply.  "git am -3" also uses the blob object
names recorded on the index line, so it would also misbehave when
synthesizing a preimage tree.

This moves the code to generate metainfo lines around, so that two
independent sets of metainfo lines are used for the split halves.

Additional tests by Jeff King.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

All the other config variables use CamelCase.  This config variable should
not be an exception.
Signed-off-by: NJohannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The type of the size member of filespec is ulong, while strbuf_detach expects
a size_t pointer.  This patch should fix the warning:
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The code was already set up to not really need it, so this just massages
it a bit to remove the use entirely.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This makes all tests pass on a system where 'lstat()' has been hacked to
return bogus data in st_size for symlinks.

Of course, the test coverage isn't complete, but it's a good baseline.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Currently we just skip rewrite diffs for binary files; this
patch makes an exception for files which will be textconv'd,
and actually performs the textconv before generating the
diff.

Conceptually, rewrite diffs should be in the exact same
format as the a non-rewrite diff, except that we refuse to
share any context. Thus it makes very little sense for "git
diff" to show a textconv'd diff, but for "git diff -B" to
show "Binary files differ".
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The current emit_rewrite_diff code always writes a text patch without
checking whether the content is binary. This means that if you end up with
a rewrite diff for a binary file, you get lots of raw binary goo in your
patch.

Instead, if we have binary files, then let's just skip emit_rewrite_diff
altogether. We will already have shown the "dissimilarity index" line, so
it is really about the diff contents. If binary diffs are turned off, the
"Binary files a/file and b/file differ" message should be the same in
either case. If we do have binary patches turned on, there isn't much
point in making a less-efficient binary patch that does a total rewrite;
no human is going to read it, and since binary patches don't apply with
any fuzz anyway, the result of application should be the same.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Some history viewers use the diff plumbing to generate diffs
rather than going through the "git diff" porcelain.
Currently, there is no way for them to specify that they
would like to see the text-converted version of the diff.

This patch adds a "--textconv" option to allow such a
plumbing user to allow text conversion.  The user can then
tell the viewer whether or not they would like text
conversion enabled.

While it may be tempting add a configuration option rather
than requiring each plumbing user to be configured to pass
--textconv, that is somewhat dangerous. Text-converted diffs
generally cannot be applied directly, so each plumbing user
should "opt in" to generating such a diff, either by
explicit request of the user or by confirming that their
output will not be fed to patch.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We treat symlinks as text containing the results of the
symlink, so it doesn't make much sense to text-convert them.

Similarly gitlink components just end up as the text
"Subproject commit $sha1", which we should leave intact.

Note that a typechange may be broken into two parts: the
removal of the old part and the addition of the new. In that
case, we _do_ show the textconv for any part which is the
addition or removal of a file we would ordinarily textconv,
since it is purely acting on the file contents.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Diffs that have been produced with textconv almost certainly
cannot be applied, so we want to be careful not to generate
them in things like format-patch.

This introduces a new diff options, ALLOW_TEXTCONV, which
controls this behavior. It is off by default, but is
explicitly turned on for the "log" family of commands, as
well as the "diff" porcelain (but not diff-* plumbing).

Because both text conversion and external diffing are
controlled by these diff options, we can get rid of the
"plumbing versus porcelain" distinction when reading the
config. This was an attempt to control the same thing, but
suffered from being too coarse-grained.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The original implementation of textconv put the conversion
into fill_mmfile. This was a bad idea for a number of
reasons:

 - it made the semantics of fill_mmfile unclear. In some
   cases, it was allocating data (if a text conversion
   occurred), and in some cases not (if we could use the
   data directly from the filespec). But the caller had
   no idea which had happened, and so didn't know whether
   the memory should be freed

 - similarly, the caller had no idea if a text conversion
   had occurred, and so didn't know whether the contents
   should be treated as binary or not. This meant that we
   incorrectly guessed that text-converted content was
   binary and didn't actually show it (unless the user
   overrode us with "diff.foo.binary = false", which then
   created problems in plumbing where the text conversion
   did _not_ occur)

 - not all callers of fill_mmfile want the text contents. In
   particular, we don't really want diffstat, whitespace
   checks, patch id generation, etc, to look at the
   converted contents.

This patch pulls the conversion code directly into
builtin_diff, so that we only see the conversion when
generating an actual patch. We also then know whether we are
doing a conversion, so we can check the binary-ness and free
the data from the mmfile appropriately (the previous version
leaked quite badly when text conversion was used)
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This function isn't used outside of diff.c; the 'static' was
simply overlooked in the original writing.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

We're going to be adding some parameters to this, so we can't have
any uninitialized data in it.
Signed-off-by: NBrian Downing <bdowning@lavos.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When diffing binary files, it is sometimes nice to see the
differences of a canonical text form rather than either a
binary patch or simply "binary files differ."

Until now, the only option for doing this was to define an
external diff command to perform the diff. This was a lot of
work, since the external command needed to take care of
doing the diff itself (including mode changes), and lost the
benefit of git's colorization and other options.

This patch adds a text conversion option, which converts a
file to its canonical format before performing the diff.
This is less flexible than an arbitrary external diff, but
is much less work to set up. For example:

  $ echo '*.jpg diff=exif' >>.gitattributes
  $ git config diff.exif.textconv exiftool
  $ git config diff.exif.binary false

allows one to see jpg diffs represented by the text output
of exiftool.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

The "diff" gitattribute is somewhat overloaded right now. It
can say one of three things:

  1. this file is definitely binary, or definitely not
     (i.e., diff or !diff)
  2. this file should use an external diff engine (i.e.,
     diff=foo, diff.foo.command = custom-script)
  3. this file should use particular funcname patterns
     (i.e., diff=foo, diff.foo.(x?)funcname = some-regex)

Most of the time, there is no conflict between these uses,
since using one implies that the other is irrelevant (e.g.,
an external diff engine will decide for itself whether the
file is binary).

However, there is at least one conflicting situation: there
is no way to say "use the regular rules to determine whether
this file is binary, but if we do diff it textually, use
this funcname pattern." That is, currently setting diff=foo
indicates that the file is definitely text.

This patch introduces a "binary" config option for a diff
driver, so that one can explicitly set diff.foo.binary. We
default this value to "don't know". That is, setting a diff
attribute to "foo" and using "diff.foo.funcname" will have
no effect on the binaryness of a file. To get the current
behavior, one can set diff.foo.binary to true.

This patch also has one additional advantage: it cleans up
the interface to the userdiff code a bit. Before, calling
code had to know more about whether attributes were false,
true, or unset to determine binaryness. Now that binaryness
is a property of a driver, we can represent these situations
just by passing back a driver struct.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

Both sets of code assume that one specifies a diff profile
as a gitattribute via the "diff=foo" attribute. They then
pull information about that profile from the config as
diff.foo.*.

The code for each is currently completely separate from the
other, which has several disadvantages:

  - there is duplication as we maintain code to create and
    search the separate lists of external drivers and
    funcname patterns

  - it is difficult to add new profile options, since it is
    unclear where they should go

  - the code is difficult to follow, as we rely on the
    "check if this file is binary" code to find the funcname
    pattern as a side effect. This is the first step in
    refactoring the binary-checking code.

This patch factors out these diff profiles into "userdiff"
drivers. A file with "diff=foo" uses the "foo" driver, which
is specified by a single struct.

Note that one major difference between the two pieces of
code is that the funcname patterns are always loaded,
whereas external drivers are loaded only for the "git diff"
porcelain; the new code takes care to retain that situation.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

Many call sites use strbuf_init(&foo, 0) to initialize local
strbuf variable "foo" which has not been accessed since its
declaration. These can be replaced with a static initialization
using the STRBUF_INIT macro which is just as readable, saves a
function call, and takes up fewer lines.
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

Add support for recognition of Objective-C class & instance methods,
C functions, and class implementation/interfaces.
Signed-off-by: NJonathan del Strother <jon.delStrother@bestbefore.tv>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

When "git diff --no-index" is given an absolute pathname, it
would generate a diff header with the absolute path
prepended by the prefix, like:

  diff --git a/dev/null b/foo

Not only is this nonsensical, and not only does it violate
the description of diffs given in git-diff(1), but it would
produce broken binary diffs. Unlike text diffs, the binary
diffs don't contain the filenames anywhere else, and so "git
apply" relies on this header to figure out the filename.

This patch just refuses to use an invalid name for anything
visible in the diff.

Now, this fixes the "git diff --no-index --binary a
/dev/null" kind of case (and we'll end up using "a" as the
basename), but some other insane cases are impossible to
handle. If you do

	git diff --no-index --binary a /bin/echo

you'll still get a patch like

	diff --git a/a b/bin/echo
	old mode 100644
	new mode 100755
	index ...

and "git apply" will refuse to apply it for a couple of
reasons, and the diff is simply bogus.

And that, btw, is no longer a bug, I think. It's impossible
to know whethe the user meant for the patch to be a rename
or not. And as such, refusing to apply it because you don't
know what name you should use is probably _exactly_ the
right thing to do!

Original problem reported by Imre Deak. Test script and problem
description by Jeff King.
Signed-off-by: NJeff King <peff@peff.net>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

On ARM I have the following compilation errors:

    CC fast-import.o
In file included from cache.h:8,
                 from builtin.h:6,
                 from fast-import.c:142:
arm/sha1.h:14: error: conflicting types for 'SHA_CTX'
/usr/include/openssl/sha.h:105: error: previous declaration of 'SHA_CTX' was here
arm/sha1.h:16: error: conflicting types for 'SHA1_Init'
/usr/include/openssl/sha.h:115: error: previous declaration of 'SHA1_Init' was here
arm/sha1.h:17: error: conflicting types for 'SHA1_Update'
/usr/include/openssl/sha.h:116: error: previous declaration of 'SHA1_Update' was here
arm/sha1.h:18: error: conflicting types for 'SHA1_Final'
/usr/include/openssl/sha.h:117: error: previous declaration of 'SHA1_Final' was here
make: *** [fast-import.o] Error 1

This is because openssl header files are always included in
git-compat-util.h since commit 684ec6c6 whenever NO_OPENSSL is not
set, which somehow brings in <openssl/sha1.h> clashing with the custom
ARM version.  Compilation of git is probably broken on PPC too for the
same reason.

Turns out that the only file requiring openssl/ssl.h and openssl/err.h
is imap-send.c.  But only moving those problematic includes there
doesn't solve the issue as it also includes cache.h which brings in the
conflicting local SHA1 header file.

As suggested by Jeff King, the best solution is to rename our references
to SHA1 functions and structure to something git specific, and define those
according to the implementation used.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NShawn O. Pearce <spearce@spearce.org>

[jc: fixes bibtex pattern breakage exposed by this test]
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Pointed out by Brandon Casey.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When multiple regular expressions are concatenated with "\n", they were
traditionally AND'ed together, and only a line that matches _all_ of them
is taken as a match.  This however is unwieldy when multiple regexp
feature is used to specify alternatives.

This fixes the semantics to take the first match.  A nagative pattern, if
matches, makes the line to fail as before.  A match with a positive
pattern will be the final match, and what it captures in $1 is used as the
hunk header comment.

We could write alternatives using "|" in ERE, but the machinery can only
use captured $1 as the hunk header comment (or $0 if there is no match in
$1), so you cannot write:

    "junk ( A | B ) | garbage ( C | D )"

and expect both "junk" and "garbage" to get stripped with the existing
code.  With this fix, you can write it as:

    "junk ( A | B ) \n garbage ( C | D )"

and the way capture works would match the user expectation more
naturally.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Using ERE elements such as "|" (alternation) by backquoting in BRE
is a GNU extension and should not be done in portable programs.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Using ERE elements such as "|" (alternation) by backquoting in BRE
is a GNU extension and should not be done in portable programs.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Currently, the hunk headers produced by 'diff -p' are customizable by
setting the diff.*.funcname option in the config file. The 'funcname' option
takes a basic regular expression. This functionality was designed using the
GNU regex library which, by default, allows using backslashed versions of
some extended regular expression operators, even in Basic Regular Expression
mode. For example, the following characters, when backslashed, are
interpreted according to the extended regular expression rules: ?, +, and |.
As such, the builtin funcname patterns were created using some extended
regular expression operators.

Other platforms which adhere more strictly to the POSIX spec do not
interpret the backslashed extended RE operators in Basic Regular Expression
mode. This causes the pattern matching for the builtin funcname patterns to
fail on those platforms.

Introduce a new option 'xfuncname' which uses extended regular expressions,
and advertise it _instead_ of funcname. Since most users are on GNU
platforms, the majority of funcname patterns are created and tested there.
Advertising only xfuncname should help to avoid the creation of non-portable
patterns which work with GNU regex but not elsewhere.

Additionally, the extended regular expressions may be less ugly and
complicated compared to the basic RE since many common special operators do
not need to be backslashed.

For example, the GNU Basic RE:

    ^[ 	]*\\(\\(public\\|static\\).*\\)$

becomes the following Extended RE:

    ^[ 	]*((public|static).*)$
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is in preparation for allowing extended regular expression patterns.
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This is in preparation for associating a flag with each pattern which will
control how the pattern is interpreted. For example, as a basic or extended
regular expression.
Signed-off-by: NBrandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Otherwise it will always print the class-name rather
than the name of the function inside that class.

While we're at it, reorder the gitattributes manpage to
list the built-in funcname pattern names in alphabetical
order.
Signed-off-by: NAndreas Ericsson <ae@op5.se>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The point of --quiet was to return the status as early as possible without
doing any extra processing.  Well behaved scripts, when they expect to run
many diff operations inside, are supposed to run "update-index --refresh"
upfront; we do not want them to pay the price of iterating over the index
and comparing the contents to fix the stat dirtiness, and we avoided most
of the processing in diffcore_std() when --quiet is in effect.

But scripts that adhere to the good practice won't have to pay any more
price than the necessary lstat(2) that will report stat cleanliness, as
long as only -q is given without any fancier diff options.

More importantly, users who do ask for "--quiet -M --filter=D" (in order
to notice only the deletion, not paths that disappeared only because they
have been renamed away) deserve to get the result they asked for, even it
means they have to pay the extra price; the alternative is to get a cheap
early return that gives a result they did not ask for, which is much
worse.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When we enabled the automatic refreshing of the index to "diff" Porcelain,
we disabled it when --find-copies-harder was asked, but there is no good
reason to do so.  In the following command sequence, the first "diff"
shows an "empty" diff exposing stat dirtyness, while the second one does
not.

    $ >foo
    $ git add foo
    $ touch foo
    $ git diff -C -C
    $ git diff -C

This fixes the inconsistency.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

This new option --dirstat-by-file is the same as --dirstat, but it
counts "impacted files" instead of "impacted lines" (lines that are
added or removed).
Signed-off-by: NHeikki Orsila <heikki.orsila@iki.fi>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The option used to be implemented as if it is a totally independent one,
but "git diff --cumulative" would not mean anything without "--dirstat".

This makes --cumulative imply --dirstat.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

With a new configuration "diff.mnemonicprefix", "git diff" shows the
differences between various combinations of preimage and postimage trees
with prefixes different from the standard "a/" and "b/".  Hopefully this
will make the distinction stand out for some people.

    "git diff" compares the (i)ndex and the (w)ork tree;
    "git diff HEAD" compares a (c)ommit and the (w)ork tree;
    "git diff --cached" compares a (c)ommit and the (i)ndex;
    "git-diff HEAD:file1 file2" compares an (o)bject and a (w)ork tree entity;
    "git diff --no-index a b" compares two non-git things (1) and (2).

Because these mnemonics now have meanings, they are swapped when reverse
diff is in effect and this feature is enabled.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

When the tracked contents have CRLF line endings, colored diff output
shows "^M" at the end of output lines, which is distracting, even though
the pager we use by default ("less") knows to hide them.

The problem is that "less" hides a carriage-return only at the end of the
line, immediately before a line feed.  The colored diff output does not
take this into account, and emits four element sequence for each line:

   - force this color;
   - the line up to but not including the terminating line feed;
   - reset color
   - line feed.

By including the carriage return at the end of the line in the second
item, we are breaking the smart our pager has in order not to show "^M".
This can be fixed by changing the sequence to:

   - force this color;
   - the line up to but not including the terminating end-of-line;
   - reset color
   - end-of-line.

where end-of-line is either a single linefeed or a CRLF pair.  When the
output is not colored, "force this color" and "reset color" sequences are
both empty, so we won't have this problem with or without this patch.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The dirstat code depends on the fact that we always generate diffs with
the names sorted, since it then just does a single-pass walk-over of the
sorted list of names and how many changes there were. The sorting means
that all files are nicely grouped by directory.

That all works fine.

Except when we have rename detection, and suddenly the nicely sorted list
of pathnames isn't all that sorted at all. And now the single-pass dirstat
walk gets all confused, and you can get results like this:

  [torvalds@nehalem linux]$ git diff --dirstat=2 -M v2.6.27-rc4..v2.6.27-rc5
     3.0% arch/powerpc/configs/
     6.8% arch/arm/configs/
     2.7% arch/powerpc/configs/
     4.2% arch/arm/configs/
     5.6% arch/powerpc/configs/
     8.4% arch/arm/configs/
     5.5% arch/powerpc/configs/
    23.3% arch/arm/configs/
     8.6% arch/powerpc/configs/
     4.0% arch/
     4.4% drivers/usb/musb/
     4.0% drivers/watchdog/
     7.6% drivers/
     3.5% fs/

The trivial fix is to add a sorting pass, fixing it to:

  [torvalds@nehalem linux]$ git diff --dirstat=2 -M v2.6.27-rc4..v2.6.27-rc5
    43.0% arch/arm/configs/
    25.5% arch/powerpc/configs/
     5.3% arch/
     4.4% drivers/usb/musb/
     4.0% drivers/watchdog/
     7.6% drivers/
     3.5% fs/

Spot the difference. In case anybody wonders: it's because of a ton of
renames from {include/asm-blackfin => arch/blackfin/include/asm} that just
totally messed up the file ordering in between arch/arm and arch/powerpc.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Find lines with <h1>..<h6> tags.

[jc: while at it, reordered entries to sort alphabetically.]
Signed-off-by: NJohan Herland <johan@herland.net>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

Find classes, functions, and methods definitions.
Signed-off-by: NKirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: NJunio C Hamano <gitster@pobox.com>

The code remembered that the last diff output it saw was an empty line,
and tried to reset that state whenever it sees a context line, a non-blank
new line, or a new hunk.  However, this codepath asks the underlying diff
engine to feed diff without any context, and the "just saw an empty line"
state was not reset if you added a new blank line in the last hunk of your
patch, even if it is not the last line of the file.
Signed-off-by: NJunio C Hamano <gitster@pobox.com>