This adds "assume unchanged" logic, started by this message in the list
discussion recently:

	<Pine.LNX.4.64.0601311807470.7301@g5.osdl.org>

This is a workaround for filesystems that do not have lstat()
that is quick enough for the index mechanism to take advantage
of.  On the paths marked as "assumed to be unchanged", the user
needs to explicitly use update-index to register the object name
to be in the next commit.

You can use two new options to update-index to set and reset the
CE_VALID bit:

	git-update-index --assume-unchanged path...
	git-update-index --no-assume-unchanged path...

These forms manipulate only the CE_VALID bit; it does not change
the object name recorded in the index file.  Nor they add a new
entry to the index.

When the configuration variable "core.ignorestat = true" is set,
the index entries are marked with CE_VALID bit automatically
after:

 - update-index to explicitly register the current object name to the
   index file.

 - when update-index --refresh finds the path to be up-to-date.

 - when tools like read-tree -u and apply --index update the working
   tree file and register the current object name to the index file.

The flag is dropped upon read-tree that does not check out the index
entry.  This happens regardless of the core.ignorestat settings.

Index entries marked with CE_VALID bit are assumed to be
unchanged most of the time.  However, there are cases that
CE_VALID bit is ignored for the sake of safety and usability:

 - while "git-read-tree -m" or git-apply need to make sure
   that the paths involved in the merge do not have local
   modifications.  This sacrifices performance for safety.

 - when git-checkout-index -f -q -u -a tries to see if it needs
   to checkout the paths.  Otherwise you can never check
   anything out ;-).

 - when git-update-index --really-refresh (a new flag) tries to
   see if the index entry is up to date.  You can start with
   everything marked as CE_VALID and run this once to drop
   CE_VALID bit for paths that are modified.

Most notably, "update-index --refresh" honours CE_VALID and does
not actively stat, so after you modified a file in the working
tree, update-index --refresh would not notice until you tell the
index about it with "git-update-index path" or "git-update-index
--no-assume-unchanged path".

This version is not expected to be perfect.  I think diff
between index and/or tree and working files may need some
adjustment, and there probably needs other cases we should
automatically unmark paths that are marked to be CE_VALID.

But the basics seem to work, and ready to be tested by people
who asked for this feature.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a tricky code and warrants extra commenting.  I wasted
30 minutes trying to break it until I realized why it works.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

The previous round caught the most trivial case well, but broke
down once index file is updated again.  Smudge problematic
entries (they should be very few if any under normal interactive
workflow) before writing a new index file out.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This fixes the longstanding "Racy GIT" problem, which was pretty
much there from the beginning of time, but was first
demonstrated by Pasky in this message on October 24, 2005:

    http://marc.theaimsgroup.com/?l=git&m=113014629716878

If you run the following sequence of commands:

	echo frotz >infocom
        git update-index --add infocom
        echo xyzzy >infocom

so that the second update to file "infocom" does not change
st_mtime, what is recorded as the stat information for the cache
entry "infocom" exactly matches what is on the filesystem
(owner, group, inum, mtime, ctime, mode, length).  After this
sequence, we incorrectly think "infocom" file still has string
"frotz" in it, and get really confused.  E.g. git-diff-files
would say there is no change, git-update-index --refresh would
not even look at the filesystem to correct the situation.

Some ways of working around this issue were already suggested by
Linus in the same thread on the same day, including waiting
until the next second before returning from update-index if a
cache entry written out has the current timestamp, but that
means we can make at most one commit per second, and given that
the e-mail patch workflow used by Linus needs to process at
least 5 commits per second, it is not an acceptable solution.
Linus notes that git-apply is primarily used to update the index
while processing e-mailed patches, which is true, and
git-apply's up-to-date check is fooled by the same problem but
luckily in the other direction, so it is not really a big issue,
but still it is disturbing.

The function ce_match_stat() is called to bypass the comparison
against filesystem data when the stat data recorded in the cache
entry matches what stat() returns from the filesystem.  This
patch tackles the problem by changing it to actually go to the
filesystem data for cache entries that have the same mtime as
the index file itself.  This works as long as the index file and
working tree files are on the filesystems that share the same
monotonic clock.  Files on network mounted filesystems sometimes
get skewed timestamps compared to "date" output, but as long as
working tree files' timestamps are skewed the same way as the
index file's, this approach still works.  The only problematic
files are the ones that have the same timestamp as the index
file's, because two file updates that sandwitch the index file
update must happen within the same second to trigger the
problem.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This starts using the "user.name" and "user.email" config variables if
they exist as the default name and email when committing.  This means
that you don't have to use the GIT_COMMITTER_EMAIL environment variable
to override your email - you can just edit the config file instead.

The patch looks bigger than it is because it makes the default name and
email information non-static and renames it appropriately.  And it moves
the common git environment variables into a new library file, so that
you can link against libgit.a and get the git environment without having
to link in zlib and libcrypt.

In short, most of it is renaming and moving, the real change core is
just a few new lines in "git_default_config()" that copies the user
config values to the new base.

It also changes "git-var -l" to list the config variables.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

With "[core] filemode = false", you can tell git to ignore
differences in the working tree file only in executable bit.

 * "git-update-index --refresh" does not say "needs update" if index
   entry and working tree file differs only in executable bit.

 * "git-update-index" on an existing path takes executable bit
   from the existing index entry, if the path and index entry are
   both regular files.

 * "git-diff-files" and "git-diff-index" without --cached flag
   pretend the path on the filesystem has the same executable
   bit as the existing index entry, if the path and index entry
   are both regular files.

If you are on a filesystem with unreliable mode bits, you may need to
force the executable bit after registering the path in the index.

 * "git-update-index --chmod=+x foo" flips the executable bit of the
   index file entry for path "foo" on.  Use "--chmod=-x" to flip it
   off.

Note that --chmod only works in index file and does not look at nor
update the working tree.

So if you are on a filesystem and do not have working executable bit,
you would do:

 1. set the appropriate .git/config option;

 2. "git-update-index --add new-file.c"

 3. "git-ls-files --stage new-file.c" to see if it has the desired
   mode bits.  If not, e.g. to drop executable bit picked up from the
   filesystem, say "git-update-index --chmod=-x new-file.c".
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a first cut at a very simple parser for a git config file.

The format of the file is a simple ini-file like thing, with simple
variable/value pairs. You can (and should) make the variables have a
simple single-level scope, ie a valid file looks something like this:

	#
	# This is the config file, and
	# a '#' or ';' character indicates
	# a comment
	#

	; core variables
	[core]
		; Don't trust file modes
		filemode = false

	; Our diff algorithm
	[diff]
		external = "/usr/local/bin/gnu-diff -u"
		renames = true

which parses into three variables: "core.filemode" is associated with the
string "false", and "diff.external" gets the appropriate quoted value.

Right now we only react to one variable: "core.filemode" is a boolean that
decides if we should care about the 0100 (user-execute) bit of the stat
information. Even that is just a parsing demonstration - this doesn't
actually implement that st_mode compare logic itself.

Different programs can react to different config options, although they
should always fall back to calling "git_default_config()" on any config
option name that they don't recognize.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Instead of "git status" ignoring (and hiding) potential errors from the
"git-update-index" call, make it exit if it fails, and show the error.

In order to do this, use the "-q" flag (to ignore not-up-to-date files)
and add a new "--unmerged" flag that allows unmerged entries in the index
without any errors.

This also avoids marking the index "changed" if an entry isn't actually
modified, and makes sure that we exit with an understandable error message
if the index is corrupt or unreadable. "read_cache()" no longer returns an
error for the caller to check.

Finally, make die() and usage() exit with recognizable error codes, if we
ever want to check the failure reason in scripts.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This is a long overdue clean-up to the code for parsing and passing
diff options.  It also tightens some constness issues.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Add -m/--modified to show files that have been modified wrt. the index.

[jc: The original came from Brian Gerst on Sep 1st but it only checked
if the paths were cache dirty without actually checking the files were
modified.  I also added the usage string and a new test.]
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Add a check before appending SHA1 signature to write_buffer,
flush it first if necessary.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

This fixes up usage of ".." (without an ending slash) and "." (with or
without the ending slash) in the git diff family.

It also fixes pathspec matching for the case of an empty pathspec, since a
"." in the top-level directory (or enough ".." under subdirectories) will
result in an empty pathspec. We used to not match it against anything, but
it should in fact match everything.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

I have reviewed all occurrences of mmap() in git and fixed three types
of errors/defects:

1) The result is not checked.
2) The file descriptor is closed if mmap() succeeds, but not when it
fails.
3) Various casts applied to -1 are used instead of MAP_FAILED, which is
specifically defined to check mmap() return value.

[jc: This is a second round of Pavel's patch.  He fixed up the problem
that close() potentially clobbering the errno from mmap, which
the first round had.]
Signed-off-by: NPavel Roskin <proski@gnu.org>
Signed-off-by: NJunio C Hamano <junkio@cox.net>

... and make git-diff-files use it too.  This all _should_ make the
diffcore-pathspec.c phase unnecessary, since the diff'ers now all do the
path matching early interally.

An earlier change to optimize directory-file conflict check
broke what "read-tree --emu23" expects.  This is fixed by this
commit.

(1) Introduces an explicit flag to tell add_cache_entry() not to
    check for conflicts and use it when reading an existing tree
    into an empty stage --- by definition this case can never
    introduce such conflicts.

(2) Makes read-cache.c:has_file_name() and read-cache.c:has_dir_name()
    aware of the cache stages, and flag conflict only with paths
    in the same stage.
Signed-off-by: NJunio C Hamano <junkio@cox.net>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Pointed out by Junio, part of my debugging of the rewrite of the
file/dir conflict handling.

This is (imho) more readable, and is also a lot faster.  The expense of
looking up sub-directory beginnings was killing us on things like
"git-diff-cache", even though that one didn't even care at all about the
file vs directory conflicts.

We really only care when somebody tries to add a conflicting name to
stage 0.

We should go through the conflict rules more carefully some day.

When we choose to omit deleted entries, we should subtract
numbers of such entries from the total number in the header.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Oops.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Only when "-u" is used of course.

Use ntohs instead of htons to convert ce_flags to host byte order
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

check_file_directory_conflict can give the wrong answers. This is
because the wrong length is passed to cache_name_pos. The length
passed should be the length of the whole path from the root, not
the length of each path subcomponent.

	$ git-init-db
	defaulting to local storage area
	$ mkdir path && touch path/file
	$ git-update-cache --add path/file
	$ rm path/file
	$ mkdir path/file && touch path/file/f
	$ git-update-cache --add path/file/f  <-- Conflict ignored
	$
Signed-off-by: NDavid Meybohm <dmeybohmlkml@bellsouth.net>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Thomas Glanzmann points out that it doesn't work well with different
clients accessing the repository over NFS - they have different views
on what the "device" for the filesystem is.

Of course, other filesystems may not even have stable inode numbers.
But we don't care. At least for now.

Add <limits.h> to the include files handled by "cache.h", and remove
extraneous #include directives from various .c files. The rule is that
"cache.h" gets all the basic stuff, so that we'll have as few system
dependencies as possible.

This one compares two pathnames that may be partial basenames, not
full paths. We need to get the path sorting right, since a directory
name will sort as if it had the final '/' at the end.

With -u flag, git-checkout-cache picks up the stat information
from newly created file and updates the cache.  This removes the
need to run git-update-cache --refresh immediately after running
git-checkout-cache.
Signed-off-by: NJunio C Hamano <junkio@cox.net>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

 - Raw hashes should be unsigned char.
 - String functions want signed char.
 - Hash and compress functions want unsigned char.

Signed-off By: Brian Gerst <bgerst@didntduck.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

same_name -> ce_same_name()
remove_entry_at() -> remove_cache_entry_at()
Signed-off-by: NBrad Roberts <braddr@puremagic.com>
Signed-off-by: NPetr Baudis <pasky@ucw.cz>

Signed-off-by: NBrad Roberts <braddr@puremagic.com>
Signed-off-by: NPetr Baudis <pasky@ucw.cz>

Changes "if (pointer == 0)" to "if (!pointer)" to match the rest
of the code, noticed by Petr Baudis.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

When "path" exists as a file or a symlink in the index, an
attempt to add "path/file" is refused because it results in file
vs directory conflict.  Similarly when "path/file1",
"path/file2", etc. exist, an attempt to add "path" as a file or
a symlink is refused.  With git-update-cache --replace, these
existing entries that conflict with the entry being added are
automatically removed from the cache, with warning messages.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

And vice versa.  The next commit will introduce an option
--replace to allow replacing existing entries.
Signed-off-by: NJunio C Hamano <junkio@cox.net>

Some commands initialize sha1_file_directory by hand.  There is no
need to do so; sha1_file.c knows how to handle it.

The next patch will remove the variable altogether.
Signed-off-by: NJunio C Hamano <junkio@cox.net>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

It didn't properly mark all cache updates as being dirty, and
causes merge errors due to that. In particular, it didn't notice
when a file was force-removed.

Besides, it was ugly as hell. I've put in place a slightly cleaner
version, but I've not enabled the optimization because I don't
want to be burned again.

Fix update-cache to compare the blob of a symlink against the link-target
and not the file it points to. Also ignore all permissions applied to
links.

Thanks to Greg for recognizing this while he added our list of symlinks
back to the udev repository.
Signed-off-by: NKay Sievers <kay.sievers@vrfy.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Allow to store and track symlink in the repository. A symlink is stored
the same way as a regular file, only with the appropriate mode bits set.
The symlink target is therefore stored in a blob object.
This will hopefully make our udev repository fully functional. :)
Signed-off-by: NKay Sievers <kay.sievers@vrfy.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We now modify the in-memory copy of the index file in "diff-cache", so
we need to add PROT_WRITE. 

Introduce xmalloc and xrealloc to die gracefully with a descriptive
message when out of memory, rather than taking a SIGSEGV. 

Signed-off-by: Christopher Li<chrislgit@chrisli.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This one is about a million times simpler, and much more likely to be
correct too.

Instead of trying to match up a tree object against the index, we just
read in the tree object side-by-side into the index, and just walk the
resulting index file. This was what all the read-tree cleanups were
all getting to.

We use that to specify alternative index files, which can be useful
if you want to (for example) generate a temporary index file to do
some specific operation that you don't want to mess with your main
one with.

It defaults to the regular ".git/index" if it hasn't been specified.

The rev-tree thing just happened to work. It shouldn't have.