output_format == DIFFSTAT and with_stat == true does not make sense, and
the way the code is structured it causes trouble.  Avoid it.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The second parameter is not the end of string input; it is
the optional return value to retrieve where the parser stopped.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Actually, it is a diff option now, so you can say

	git diff --check

to ask if what you are about to commit is a good patch.

[jc: this also would work for fmt-patch, but the point is that
 the check is done before making a commit.  format-patch is run
 from an already created commit, and that is too late to catch
 whitespace damaged change.]
Signed-off-by: NJohannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When renaming leading/a/filename to leading/b/filename (and
"filename" is sufficiently long), we tried to squash the rename
to "leading/{a => b}/filename".  However, when "/a" or "/b" part
is empty, we underflowed and tried to print a substring of
length -1.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Remove the need to pipe git diff through git apply to
get the extended headers summary.
Signed-off-by: NSean Estabrooks <seanlkml@sympatico.ca>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

We used to parse "-U" and "--unified" as part of the GIT_DIFF_OPTS
environment variable, but strangely enough we would _not_ parse them as
part of the normal diff command line (where we only accepted "-u").

This adds parsing of -U and --unified, both with an optional numeric
argument. So now you can just say

	git diff --unified=5

to get a unified diff with a five-line context, instead of having to do
something silly like

	GIT_DIFF_OPTS="--unified=5" git diff -u

(that silly format does continue to still work, of course).
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When we cut off the front of a filename to make it fit on the line, we add
a "..." in front. However, the way the "git diff" code was written, we
will never reset the prefix back to the empty string, so every single
filename afterwards will have the "..." prefix, whether appropriate or
not.

You can see this with "git diff v2.6.16.." on the current kernel tree,
since there are filenames with long names that changed there:

 [ snip snip ]
 Documentation/filesystems/vfs.txt                  |  229
 .../firmware_class/firmware_sample_driver.c        |    3
 .../firmware_sample_firmware_class.c               |    1
 ...Documentation/fujitsu/frv/kernel-ABI.txt           |  192
 ...Documentation/hwmon/w83627hf                       |    4
 [ snip snip ]

notice how the two Documentation/firmware** filenames caused the "..." to
be added, but then the later filenames don't want it, and it also screws
up the alignment of the line numbering afterwards.

Trivially fixed by moving the declaration (and initial setting) of the
"prefix" variable into the for-loop where it is used.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This updates the user interface and generated diff data format.

 * "diff --binary" is used to signal that we want an e-mailable
   binary patch.  It implies --full-index and -p.

 * "apply --allow-binary-replacement" acquired a short synonym
   "apply --binary".

 * After the "GIT binary patch\n" header line there is a token
   to record which binary patch mechanism was used, so that we
   can extend it later.  Currently there are two mechanisms
   defined: "literal" and "delta".  The former records the
   deflated postimage and the latter records the deflated delta
   from the preimage to postimage.

   For purely implementation convenience, I added the deflated
   length after these "literal/delta" tokens (otherwise the
   decoding side needs to guess and reallocate the buffer while
   inflating).  Improvement patches are very welcomed.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This adds "binary patch" to the diff output and teaches apply
what to do with them.

On the diff generation side, traditionally, we said "Binary
files differ\n" without giving anything other than the preimage
and postimage object name on the index line.  This was good
enough for applying a patch generated from your own repository
(very useful while rebasing), because the postimage would be
available in such a case.  However, this was not useful when the
recipient of such a patch via e-mail were to apply it, even if
the preimage was available.

This patch allows the diff to generate "binary" patch when
operating under --full-index option.  The binary patch follows
the usual extended git diff headers, and looks like this:

	"GIT binary patch\n"
	<length byte><data>"\n"
	...
	"\n"

Each line is prefixed with a "length-byte", whose value is upper
or lowercase alphabet that encodes number of bytes that the data
on the line decodes to (1..52 -- 'A' means 1, 'B' means 2, ...,
'Z' means 26, 'a' means 27, ...).  <data> is 1 or more groups of
5-byte sequence, each of which encodes up to 4 bytes in base85
encoding.  Because 52 / 4 * 5 = 65 and we have the length byte,
an output line is capped to 66 characters.  The payload is the
same diff-delta as we use in the packfiles.

On the consumption side, git-apply now can decode and apply the
binary patch when --allow-binary-replacement is given, the diff
was generated with --full-index, and the receiving repository
has the preimage blob, which is the same condition as it always
required when accepting an "Binary files differ\n" patch.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Somebody on the #git channel complained that the sha1_to_hex() thing uses
a static buffer which caused an error message to show the same hex output
twice instead of showing two different ones.

That's pretty easily rectified by making it uses a simple LRU of a few
buffers, which also allows some other users (that were aware of the buffer
re-use) to be written in a more straightforward manner.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The patch format shows complete rewrite as deletion of all old lines
followed by addition of all new lines.  Count lines consistenly with
that when doing diffstat.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is the first installment to libify diff brothers.

The updated diff-files uses revision.c::setup_revisions()
infrastructure to parse its command line arguments, which means
the pathname arguments are checked more strictly than before.
The tests are adjusted to separate possibly missing paths from
the rest of arguments with double-dashes, to show the kosher
way.

As Linus pointed out, renaming diff.c to diff-lib.c was simply
stupid, so I am renaming it back.  The new diff-lib.c is to
contain pieces extracted from diff brothers.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When a verbatim rename or copy is detected, we did not show
anything on the "diff --stat" for the filepair.  This makes it
to show the rename information.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Now I am not doing any real "git-diff in C" yet, but this would
help before doing so.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Set maximum filename length for binary files so that scaling won't be
triggered and result in invalid string access.
Signed-off-by: NJonas Fonseca <fonseca@diku.dk>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Just like "patch" format always needs recursive, "diffstat"
format does not make sense without setting recursive.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Just like "patch" format always needs recursive, "diffstat"
format does not make sense without setting recursive.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

With this option, git prepends a diffstat in front of the patch.

Since I really, really do not know what a diffstat of a combined diff
("merge diff") should look like, the diffstat is not generated for these.
Signed-off-by: NJohannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

I missed that "git-diff-* --stat" spits out three-dash separator
on its own without being asked.  Remove it.

When we output commit log followed by diff, perhaps --patch-with-stat,
for downstream consumer, we _would_ want the three-dash between
the message and the diff material, but that logic belongs to the
caller, not diff generator.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Signed-off-by: NJunio C Hamano <junkio@cox.net>

... and a fix for an invalid free():
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Now, you can say "git diff --stat" (to get an idea how many changes are
uncommitted), or "git log --stat".
Signed-off-by: NJohannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

More friendly for human reading I believe, and possibly friendlier to some
parsers (although only by an epsilon).
Signed-off-by: NPetr Baudis <pasky@suse.cz>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This new flag outputs the diff-raw output and diff-patch output
at the same time.  Requested by Cogito.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Nobody except diff-stages used it -- the callers instead filtered
the input to diffcore themselves.  Make diff-stages do that as
well and retire diffcore-pathspec.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

We did not read in the file data before emitting the
total-rewrite diff.  Noticed by Pasky.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

We were leaking filepairs when output-format was set to
NO_OUTPUT.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

git-diff-* --pickaxe-regex will change the -S pickaxe to match
POSIX extended regular expressions instead of fixed strings.

The regex.h library is a rather stupid interface and I like pcre too, but
with any luck it will be everywhere we will want to run Git on, it being
POSIX.2 and all. I'm not sure if we can expect platforms like AIX to
conform to POSIX.2 or if win32 has regex.h. We might add a flag to
Makefile if there is a portability trouble potential.
Signed-off-by: NPetr Baudis <pasky@suse.cz>

Introduce tree-walk.[ch] and move "struct tree_desc" and
associated functions from various places.

Rename DIFF_FILE_CANON_MODE(mode) macro to canon_mode(mode) and
move it to cache.h.  This macro returns the canonicalized
st_mode value in the host byte order for files, symlinks and
directories -- to be compared with a tree_desc entry.
create_ce_mode(mode) in cache.h is similar but is intended to be
used for index entries (so it does not work for directories) and
returns the value in the network byte order.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The speed of the built-in diff generator is nice; but the function names
shown by `diff -p' are /really/ nice.  And I hate having to choose.  So,
we hack xdiff to find the function names and print them.

xdiff has grown a flag to say whether to dig up the function names.  The
builtin_diff function passes this flag unconditionally.  I suppose it
could parse GIT_DIFF_OPTS, but it doesn't at the moment.  I've also
reintroduced the `function name' into the test suite, from which it was
removed in commit 3ce8f089.

The function names are parsed by a particularly stupid algorithm at the
moment: it just tries to find a line in the `old' file, from before the
start of the hunk, whose first character looks plausible.  Still, it's
most definitely a start.
Signed-off-by: NMark Wooding <mdw@distorted.org.uk>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This stops using temporary files when we are using the built-in
diff (including the complete rewrite).
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This fixes up a couple of minor issues with the real built-in
diff to be more usable:

 - Omit ---/+++ header unless we emit diff output;

 - Detect and punt binary diff like GNU does;

 - Honor GIT_DIFF_OPTS minimally (only -u<number> and
   --unified=<number> are currently supported);

 - Omit line count of 1 from "@@ -l,k +m,n @@" hunk header
   (i.e. when k == 1 or n == 1)

 - Adjust testsuite for the lack of -p support.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This uses a simplified libxdiff setup to generate unified diffs _without_
doing  fork/execve of GNU "diff".

This has several huge advantages, for example:

Before:

	[torvalds@g5 linux]$ time git diff v2.6.16.. > /dev/null

	real    0m24.818s
	user    0m13.332s
	sys     0m8.664s

After:

	[torvalds@g5 linux]$ time git diff v2.6.16.. > /dev/null

	real    0m4.563s
	user    0m2.944s
	sys     0m1.580s

and the fact that this should be a lot more portable (ie we can ignore all
the issues with doing fork/execve under Windows).

Perhaps even more importantly, this allows us to do diffs without actually
ever writing out the git file contents to a temporary file (and without
any of the shell quoting issues on filenames etc etc).

NOTE! THIS PATCH DOES NOT DO THAT OPTIMIZATION YET! I was lazy, and the
current "diff-core" code actually will always write the temp-files,
because it used to be something that you simply had to do. So this current
one actually writes a temp-file like before, and then reads it into memory
again just to do the diff. Stupid.

But if this basic infrastructure is accepted, we can start switching over
diff-core to not write temp-files, which should speed things up even
further, especially when doing big tree-to-tree diffs.

Now, in the interest of full disclosure, I should also point out a few
downsides:

 - the libxdiff algorithm is different, and I bet GNU diff has gotten a
   lot more testing. And the thing is, generating a diff is not an exact
   science - you can get two different diffs (and you will), and they can
   both be perfectly valid. So it's not possible to "validate" the
   libxdiff output by just comparing it against GNU diff.

 - GNU diff does some nice eye-candy, like trying to figure out what the
   last function was, and adding that information to the "@@ .." line.
   libxdiff doesn't do that.

 - The libxdiff thing has some known deficiencies. In particular, it gets
   the "\No newline at end of file" case wrong. So this is currently for
   the experimental branch only. I hope Davide will help fix it.

That said, I think the huge performance advantage, and the fact that it
integrates better is definitely worth it. But it should go into a
development branch at least due to the missing newline issue.

Technical note: this is based on libxdiff-0.17, but I did some surgery to
get rid of the extraneous fat - stuff that git doesn't need, and seriously
cutting down on mmfile_t, which had much more capabilities than the diff
algorithm either needed or used. In this version, "mmfile_t" is just a
trivial <pointer,length> tuple.

That said, I tried to keep the differences to simple removals, so that you
can do a diff between this and the libxdiff origin, and you'll basically
see just things getting deleted. Even the mmfile_t simplifications are
left in a state where the diffs should be readable.

Apologies to Davide, whom I'd love to get feedback on this all from (I
wrote my own "fill_mmfile()" for the new simpler mmfile_t format: the old
complex format had a helper function for that, but I did my surgery with
the goal in mind that eventually we _should_ just do

	mmfile_t mf;

	buf = read_sha1_file(sha1, type, &size);
	mf->ptr = buf;
	mf->size = size;
	.. use "mf" directly ..

which was really a nightmare with the old "helpful" mmfile_t, and really
is that easy with the new cut-down interfaces).

[ Btw, as any hawk-eye can see from the diff, this was actually generated
  with itself, so it is "self-hosting". That's about all the testing it
  has gotten, along with the above kernel diff, which eye-balls correctly,
  but shows the newline issue when you double-check it with "git-apply" ]
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This changes diffcore-rename to reuse statistics information
gathered during similarity estimation, and updates the hashtable
implementation used to keep track of the statistics to be
denser.  This seems to give better performance.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Instead of depending of fork() and execve() and doing things in between
the two, make the git diff functions do everything up front, and then do
a single "spawn_prog()" invocation to run the actual external diff
program (if any is even needed).

This actually ends up simplifying the code, and should make it much
easier to make it efficient under broken operating systems (read: Windows).
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Earlier it did not grok the 0{40} SHA1 very well, but what it
needed to do was to find the shortest 0{N} that is not used as a
valid object name to be consistent with the way names of valid
objects are abbreviated.  This makes some users simpler.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This adds "assume unchanged" logic, started by this message in the list
discussion recently:

	<Pine.LNX.4.64.0601311807470.7301@g5.osdl.org>

This is a workaround for filesystems that do not have lstat()
that is quick enough for the index mechanism to take advantage
of.  On the paths marked as "assumed to be unchanged", the user
needs to explicitly use update-index to register the object name
to be in the next commit.

You can use two new options to update-index to set and reset the
CE_VALID bit:

	git-update-index --assume-unchanged path...
	git-update-index --no-assume-unchanged path...

These forms manipulate only the CE_VALID bit; it does not change
the object name recorded in the index file.  Nor they add a new
entry to the index.

When the configuration variable "core.ignorestat = true" is set,
the index entries are marked with CE_VALID bit automatically
after:

 - update-index to explicitly register the current object name to the
   index file.

 - when update-index --refresh finds the path to be up-to-date.

 - when tools like read-tree -u and apply --index update the working
   tree file and register the current object name to the index file.

The flag is dropped upon read-tree that does not check out the index
entry.  This happens regardless of the core.ignorestat settings.

Index entries marked with CE_VALID bit are assumed to be
unchanged most of the time.  However, there are cases that
CE_VALID bit is ignored for the sake of safety and usability:

 - while "git-read-tree -m" or git-apply need to make sure
   that the paths involved in the merge do not have local
   modifications.  This sacrifices performance for safety.

 - when git-checkout-index -f -q -u -a tries to see if it needs
   to checkout the paths.  Otherwise you can never check
   anything out ;-).

 - when git-update-index --really-refresh (a new flag) tries to
   see if the index entry is up to date.  You can start with
   everything marked as CE_VALID and run this once to drop
   CE_VALID bit for paths that are modified.

Most notably, "update-index --refresh" honours CE_VALID and does
not actively stat, so after you modified a file in the working
tree, update-index --refresh would not notice until you tell the
index about it with "git-update-index path" or "git-update-index
--no-assume-unchanged path".

This version is not expected to be perfect.  I think diff
between index and/or tree and working files may need some
adjustment, and there probably needs other cases we should
automatically unmark paths that are marked to be CE_VALID.

But the basics seem to work, and ready to be tested by people
who asked for this feature.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

So far, e.g. git-update-index --refresh was basically uninterruptable
by ctrl-c, since it hooked the SIGINT handler, but that handler would
only unlink the lockfile but not actually quit. This makes it propagate
the signal to the default handler.

Note that I expected it to work without resetting the signal handler to
SIG_DFL, but without that it ended in an infinite loop of tgkill()s -
is my glibc violating SUS or what?
Signed-off-by: NPetr Baudis <pasky@suse.cz>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Earier specifying an abbreviation shorter than minimum fell back
to full 40 letters, which was nonsense.  Make it to fall back to
the minimum number (currently 4).
Signed-off-by: NJunio C Hamano <junkio@cox.net>