Includes tests:
  composite_keys_gpdb_1
  composite_keys_gpdb_3
  composite_keys_gpdb_2
  partitionindexes
  staticselection
  cte_functest
  cte_queries
  create_table_default_distribution
  lastj lastj_hash
  indexapply

Commit 33a1278c merged upstream addition of a typmod argument to
makeConst(). However, one of the calls was botched during the merge, and
the typmod argument was inserted to wrong place in argument list. This
seems to have been harmless because the constant ended up being NULL
anyway, and the bogus values for the other arguments were ignored.

These got in the way in the PostgreSQL 8.3 merge, so let's get them over
with before the big merge lands.

This makes no difference at runtime, as the argument is always passed as
NULL anyway, but let's avoid unnecessary differences vs. upstream.

These changes were lifted from upcoming PostgreSQL 8.3 merge branch, but
are not related to the merge per se, so let's get them out of the way
before the big merge lands. Plus other cosmetic things in neighbouring
code that stuck my eye.

A new regression test was added in the upstream that does work_mem='64kB',
which used to fail without --enable-cassert. There's little reason to not
allow a small value here, so let's just allow it. It doesn't mean that
anyone should use such a small value in production, but that's just a
matter of "don't do that then".

The minimum was changed later in the upstream, too, to be 64 kb regardless
of BLCKSZ.

prev_item variable needs to be reset between the two loops. Otherwise,
if the first item in the latter list (allocatedReaderGangs1) needs to
be removed, things go wrong. I got an assertion failure with
installcheck-good from that:

FailedAssertion(""!(prev != ((void *)0) ? ((prev)->next) == cell : list_head(list) == cell)"", File: ""list.c"", Line: 616)

This got broken in the recent refactoring commit 46dfa750. The rest of the
changes in this commit, introducing the new next_item variable, wasn't
needed for correctness, but IMHO makes the code easier to understand.

I missed this in commit af7b1b51, which reverted the backend shutdown code
closer to the upstream. Without it, auxiliary proc slots are never reused,
so you run out of them fairly quickly, if any (non-permanent) auxiliary
processes are used.

Reported by Gang Xiong, who ran into this with gprecoverseg.

The checks for invalid TIDs are very cheap, a few CPU instructions. It's
better to catch bugs involving invalid TID early, so let's always check
for them.

The LOGs in storeAttrDefault() that were also tied to this GUC seemed
oddly specific. They were probably added long time ago to hunt for some
particular bug, and don't seem generally useful, so I just removed them.

This way we can reuse the csv files in other installcheck suites.

This seems like a copy-paste error since the IDENTIFICATION marker
is a PostgreSQL artifact and Peter Eisentraut probably didn't write
the gpopt Makefile. Remove to make it look like the headers in other
GPDB specific makefiles.

Enhacing code generated ExecVariableList with support of null attributes.
Also, there are some formatting changes based on Google C++ style.

1) add one new type of gang: singleton reader gang.
2) change interface of allocateGang.
3) handling exceptions during gang creation: segment down and segment reset.
4) cleanup some dead code.

This closes #764

This attempts to clean up the autoconf script a bit and follow the
upstream division of generic code in config/ with the actual lookup
configuration in configure.in. Also updated our installation to rely
on a more modern version of autoconf by backporting parts of upstream
commit 7cc514ac. This commit consist of:

  * Decouple --enable-codegen and --with-codegen-prefix to not
    silently ignore prefixes if the enable flag isn't passed.
	Emit a warning if configuring prefixes without codegen. Also
	moves --with-codegen-prefix to require an argument since
	--with-codegen-prefix without an argument is likely to hide
	either a scripting bug or a misunderstanding from the user
  * Move program checks for cmake and apr-1-config to programs.m4
    and allow for path overrides and ensure to use the resolved
	path when invoking cmake for --enable-codegen
  * Propagate the apr-1-config flags and objects to where used via
    Makefile.global rather than performing another lookup
  * Remove check for unused arguments since autoconf does that
    automatically since 2.63
  * Remove backported fseeko handling since that isn't relevant
    for modern autoconf versions
  * Minor help output tidying and spelling fixes

dispatcher/ directory

This commit has no logic change, it just contains movement of code across
files, to make dispatcher code clearer, and easier for unit testing.

Signed-off-by: Kenan Yao

There were a bunch of changes vs. upstream in the way the PGPROC free list
was managed, and the way backend exit was handled. They seemed largely
unnecessary, and somewhat buggy, so I reverted them. Avoiding unnecessary
differences makes merging with upstream easier too.

* The freelist was protected by atomic operations instead of a spinlock.
There was an ABA problem in the implementation, however. In Prepend(), if
another backend grabbed the PGPROC we were just about to grab for ourselves,
and returned it to the freelist before we iterate and notice, we might
set the head of the free list to a PGPROC that's actually already in use.
It's a tight window, and backend startup is quite heavy, so that's unlikely
to happen in practice. Still, it's a bug. Because backend start up is such
a heavy operation, this codepath is not so performance-critical that you
would gain anything from using atomic operations instead of a spinlock, so
just switch back to using a spinlock like in the upstream.

* When a backend exited, the responsibility to recycle the PGPROC entry
to the free list was moved to the postmaster, from the backend itself.
That's not broken per se, AFAICS, but it violates the general principle of
avoiding shared memory access in postmaster.

* There was a dead-man's switch, in the form of the postmasterResetRequired
flag in the PGPROC entry. If a backend died unexpectedly, and the flag
was set, postmaster would restart the whole server. If the flag was not
set, it would clean up only the PGPROC entry that was left behind and
let the system run normally. However, the flag was in fact always set,
except after ProcKill had already run, i.e. when the process had exited
normally. So I don't see the point of that, we might as well rely on the
exit status to signal normal/abnormal exit, like we do in the upstream. That
has worked fine for PostgreSQL.

* There was one more case where the dead-man's switch was activated, even
though the backend exited normally: In AuxiliaryProcKill(), if a filerep
subprocess died, and it didn't have a parent process anymore. That means
that the master filerep process had already died unexpectedly (filerep
subprocesses are children of the are not direct children of postmaster).
That seems unnecessary, however: if the filerep process had died
unexpectedly, the postmaster should wake up to that, and would restart
the server. To play it safe, though, make the subprocess exit with non-zero
exit status in that case, so that the postmaster will wake up to that, if
it didn't notice the master filerep process dying for some reason.

* HaveNFreeProcs() was rewritten by maintaining the number of entries
in the free list in a variable, instead of walking the list to count them.
Presumably to make backend startup cheaper, when max_connections is high.
I kept that, but it's slightly simpler now that we use a spinlock to protect
the free list again: no need to use atomic ops for the variable anymore.

* The autovacFreeProcs list was not used. Autovacuum workers got their
PGPROC entry from the regular free list. Fix that, and also add
missing InitSharedLatch() call to the initialization of the autovacuum
workers list.

This is specially useful for using to get a pointer to a member in a
structure that is an embedded array.

Signed-off-by: NKarthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>

Previously, previous free TID validation was done under the GUC
persistent_integrity_checks. This commit extracts the previous free TID
validation into another GUC validate_previous_free_tid and is enabled by
default. If the validation detects a corruption in the free TID list, we
will now switch to a new free TID list and leave the corrupted one detached
for cleanup during persistent table rebuild or during crash recovery.

Authors: Jimmy Yih and Abhijit Subramanya

These tests use the existing fault injection mechanism built into the
server, to cause errors to happen at strategic places, and checks the
results.

This is almost just a placeholder, there are very few actual tests for
now. But it's a start.

The suite uses plain old pg_regress to run the tests and check the
results. That's enough for the tests included here, but in the future
we'll probably want to do server restarts, crashes, etc. as part of
the suite, and will have to refactor this to something that can do those
things more easily. But let's cross that bridge when we get there.

Also, the test actually leaves the connections to the segments in a
funny state, which shouldn't really happen. The test fails currently
because of that; let's fix it together with the state issue. But
even in this state, this has been useful to me right now, to reproduce
an issue on the merge_8_3_22 branch that I'm working on at the same
time (this test currently causes a PANIC there).

This also isn't hooked up to any top-level targets yet; you have to
run the suite manually from the src/test/dtm directory.

While hacking, I ran into the "Expected a CREATE for file-system object
name" error. But instead of printing the error, it got into an infinite
loop. smgrSortDeletesList() elog(ERROR)'d out of the function, while it
was in the middle of putting the linked list back together, leaving
pendingDeletes list corrupt, with a loop. AbortTransaction() processing
called smgrIsPendingFileSysWork(), which traversed the list, and it got
stuck in the loop.

To avoid that, don't leave the list in an invalid state on error. I don't
know why I ran into the error in the first place, but that's a different
story.

I saw the "nresults < nslots" assertion fail, while hacking on something
else. It happened when a Distributed Prepare command failed, and there were
several error result sets from a segment. I'm not sure how normal it is to
receive multiple ERROR responses to a single query, but the protocol
certainly allows it, and I don't see any explanation for why the code used
to assume that there can be at most 2 result sets from each segment.

Remove that assumption, and make the code cope with more than two result
sets from a segment, by calculating the required size of the array
accurately.

In the passing, remove the NULL-terminator from the array, and change the
callers that depended on it to use the returned size variable instead.
Makes the loops in the callers look less funky.

The code incorrectly called free() on last+1 element of the array. The
array returned by cdbdisp_dispatchRMCommand() always has a NULL element as
terminator, and free(NULL) is a no-op, which is why this didn't outright
crash. But clearly the intention here was to free() the array itself,
otherwise it's leaked.

This is a follow-up to commit b7365f58 which replaced the PostgreSQL
bug report email with the Greenplum one.

Removes unused CVS $Header$ tags and moves comments closer to where
they make sense as well as updates a few comments to match reality.

Rather than storing the full 100kb string in the outfile (which adds
200kb for the header) and passing to diff instead compare the tuple
with the expected value and store the boolean result in the outfile
instead.

This shaves 1.25 seconds off the testsuite on my laptop but the
primary win is to shrink the size of the outfiles. Tests on Pulse
show consistently lower diff times.