Almost everything in this set of commits was already merged via the XML merge
from 9.1 which is in the tree already. The largeobject tests included in this
merge (parallel and serial schedule) are currently left disabled due to test
breakage, this needs to be handled in subsequent commits. The test is altered
to distribute around the oid column (which is alse the default choice by GPDB
in case no explicit distribution column in specified).

Conflicts:
  Makefile
  doc/src/sgml/installation.sgml
  doc/src/sgml/manage-ag.sgml
  src/backend/Makefile
  src/backend/access/Makefile
  src/backend/access/common/Makefile
  src/backend/access/gin/Makefile
  src/backend/access/gist/Makefile
  src/backend/access/hash/Makefile
  src/backend/access/heap/Makefile
  src/backend/access/index/Makefile
  src/backend/access/nbtree/Makefile
  src/backend/bootstrap/Makefile
  src/backend/commands/Makefile
  src/backend/executor/Makefile
  src/backend/executor/execQual.c
  src/backend/lib/Makefile
  src/backend/libpq/Makefile
  src/backend/main/Makefile
  src/backend/nodes/Makefile
  src/backend/optimizer/Makefile
  src/backend/optimizer/geqo/Makefile
  src/backend/optimizer/path/Makefile
  src/backend/optimizer/plan/Makefile
  src/backend/optimizer/prep/Makefile
  src/backend/optimizer/util/Makefile
  src/backend/port/darwin/Makefile
  src/backend/port/nextstep/Makefile
  src/backend/port/sunos4/Makefile
  src/backend/port/win32/Makefile
  src/backend/postmaster/Makefile
  src/backend/rewrite/Makefile
  src/backend/storage/Makefile
  src/backend/storage/buffer/Makefile
  src/backend/storage/file/Makefile
  src/backend/storage/freespace/Makefile
  src/backend/storage/ipc/Makefile
  src/backend/storage/large_object/Makefile
  src/backend/storage/lmgr/Makefile
  src/backend/storage/page/Makefile
  src/backend/storage/smgr/Makefile
  src/backend/tcop/Makefile
  src/backend/utils/adt/Makefile
  src/backend/utils/adt/xml.c
  src/backend/utils/cache/Makefile
  src/backend/utils/error/Makefile
  src/backend/utils/error/elog.c
  src/backend/utils/hash/Makefile
  src/backend/utils/init/Makefile
  src/backend/utils/mb/Makefile
  src/backend/utils/mmgr/Makefile
  src/backend/utils/resowner/Makefile
  src/backend/utils/sort/Makefile
  src/backend/utils/time/Makefile
  src/bin/Makefile
  src/bin/initdb/initdb.c
  src/bin/psql/large_obj.c
  src/include/catalog/catversion.h
  src/include/utils/xml.h
  src/interfaces/Makefile
  src/interfaces/ecpg/compatlib/Makefile
  src/interfaces/ecpg/ecpglib/Makefile
  src/interfaces/ecpg/pgtypeslib/Makefile
  src/interfaces/ecpg/test/Makefile
  src/interfaces/ecpg/test/Makefile.regress
  src/test/regress/expected/xml.out
  src/test/regress/expected/xml_1.out
  src/test/regress/parallel_schedule
  src/test/regress/sql/xml.sql

Additional perl module copy target in gpMgmt/Makefile:
Supporting the commercial build, the new perl modules (atmsort.pm &
explain.pm) supporting gpdiff.pl need to be copied to the bin directory.

Supporting the commercial build, the new perl modules (atmsort.pm &
explain.pm) supporting gpdiff.pl need to be copied to the bin directory.

ORCA uses its own metadata cache to store information about relations,
operators etc. Currently, we always reset the cache when planning a query,
unless the optimizer_release_mdcache GUC is turned off, which is slow.

To make it safe to turn optimizer_release_mdcache off, use the catalog
cache invalidation mechanism to still reset the cache when there are
changes to the catalogs that affect the metadata cache.

The ORCA-facing interface of this is the same as in the previous attempt:
A function that returns true/false indicating whether there has been any
catalog changes whatsoever since last call.

This reverts commit 6c31a3b4. Per
discussion, we will implement the same functionality in a simpler way.

Fix the fake-tidycat definitions of OPFAMILYOID and OPFAMILYAMNAMENSP
syscaches. Remove reference to INHRELID - that syscache removed by one of
the merged upstream 8.3 patches. Re-run tidycat.pl, to correct the "last"
id number in syscache.h.

Added a generation-based Metadata Versioning mechanism
which will be used by ORCA to cache and invalidate catalog
data in its Metadata Cache.
Versioning is disabled by default at this point, until the
Metadata Cache eviction policy is completed.

This patch replaces all occurrences of pgsql-bugs@postgresql.org with
bugs@greenplum.org. Pointers to the pgsql-bugs mailinglist in the code
remain as they re. Regression and BugBuster tests are also patched.

The main difference is that setting up all the test tables and types are
done in one large transaction, to reduce overhead of transaction commits.
Another is that all the objects are created in a separate schema, and
we don't bother to drop them all at the end of the tests. These changes
shave off a few seconds from the test runtime on my laptop.

This makes it easier to modify the script used to generate the tests (which
I will do in the next commit).

This set of commits only bring superficial changes with all functionality
retained from HEAD due to previous merging of the functionality.

Conflicts:
      doc/TODO
      doc/src/FAQ/TODO.html
      doc/src/sgml/config.sgml
      src/backend/utils/adt/pg_lzcompress.c
      src/backend/utils/adt/version.c
      src/backend/utils/adt/xml.c
      src/backend/utils/misc/guc.c
      src/include/utils/xml.h
      src/test/regress/expected/xml.out
      src/test/regress/expected/xml_1.out
      src/test/regress/pg_regress.c
      src/test/regress/sql/xml.sql

These were renamed to 'typname' in GPDB a long time ago because 'typename'
is a reserved keyword in C++. That's a valid concern for header files that
need to be included in C++ code (like src/backend/gpopt), but it's not
necessary for local variables that appear in .c files, as long as we're
careful to compile those files as C code, not C++. So rename local
variables back to 'typename', to reduce diff vs. upstream and make merging
slightly easier.

In the upstream header files, the 'typename' fields were renamed to
'typeName', in a later PostgreSQL version. In GPDB, we've renamed them
to 'typname' instead. Would be good to adopt the upstream naming at some
point, but not in this patch.

This commit eliminates the global new/delete overrides that were causing
compatibility problems (the Allocators.(h/cpp/inl) files have been
completely removed). The GPOS `New()` macro is retained and works the
same way, but has been renamed `GPOS_NEW()` to avoid confusion and
possible name collisions. `GPOS_NEW()` works only for allocating
singleton objects. For allocating arrays, `GPOS_NEW_ARRAY()` is
provided. Because we no longer override the global delete,
objects/arrays allocated by `GPOS_NEW()` and `GPOS_NEW_ARRAY()` must now
be deleted by the new functions `GPOS_DELETE()` and
`GPOS_DELETE_ARRAY()` respectively. All code in GPOS has been
retrofitted for these changes, but Orca and other code that depends on
GPOS should also be changed.

Note that `GPOS_NEW()` and `GPOS_NEW_ARRAY()` should both be
exception-safe and not leak memory when a constructor throws.

Closes #166

I'm trying to eliminate bugbuster test suite in the long run, so I'm
moving any tests from there that seem genuinely useful to the main
regression suite.

Some of the tests were broken, because the necessary test tables were
missing. Just remove those as they didn't test anything that wasn't tested
elsewhere anyway.

when the add-on config file contains GUCs which exist in postgresql.conf.sample,
and are commented out(e.g, #max_resource_queues), these GUC settings would not
take effect because incorrect posgresql.conf would be generated by gpinitsystem.

A --dump-stats flag has been added to gpcrondump.py to dump the
pg_class table tuple counts and pg_statistic catalog statistics
to a SQL file, to later be restored manually or with gpdbrestore.

A --restore-stats flag has been added to gpdbrestore to restore
the dumped statistics.  Passing it the argument "include", or
passing it no argument, will restore the statistics in addition
to performing a normal restore.  Passing it the argument "only"
will restore statistics but will not restore anything else; if
any tables do not exist, using this option will skip restoring
statistics to those tables, print a warning for each, and then
continue restoring statistics to other tables.

If --dump-stats or --restore-stats is used in conjunction with
other flags that include or exclude certain tables or schemas,
the statistics dump or restore will be filtered in the same way.

Authors: James McAtamney and Jimmy Yih

To make the overall suite finish slightly faster. Some of the test
tables used in the tests have been renamed to avoid clashing with each
other.

Loading the tables takes a few seconds, so let's not duplicate that work.

The ones that remain don't seem too useful either, but this commit only
removes the obviously unnecessary ones. Some of them were producing
syntax errors, some were cases that are clearly covered by tests in the
main suite.

None of the test tables existed, and the expected output contained just
"relation does not exist" errors, so it wasn't testing much at all. Looking
at the queries, I don't think they were trying to test anything too
interesting in the first place. I'm pretty sure we have similar aggregate
queries in other test cases. So just remove this.

I'm trying to speed up the regression tests. This doesn't buy us very
much, but every little helps, and makes the test readable anyway.

This further speeds up the diff phase of the regression suit. In
particular, the 'table_functions' regression test.

All the logic is in atmsort.pm now, but atmsort.pl remains as a wrapper
script. In gpdiff.pl, instead of building a command line and invoking a
new process, use atmsort.pm as a perl module. This greatly reduces the
overhead, making gpdiff.pl invocations a lot faster.

HEAD version picked in all cases, only comment changes pulled in from
upstream.

version for Python module

Most of the commits included in this merge we had backported already.
One ForgetDatabaseFsyncRequests() call was missing from dropdb(), and an
error check was added to width_bucket_numeric. Also, add pg_proc entry
for width_bucket_float8; the code was backported already. Update 'numeric'
regression test, per the merged commit.

Since we changed the catalogs, bump catversion, and recreate 4.3.json file.

Conflicts:
	doc/TODO
	doc/src/FAQ/TODO.html
	doc/src/sgml/func.sgml
	src/backend/catalog/information_schema.sql
	src/backend/commands/dbcommands.c
	src/backend/postmaster/bgwriter.c
	src/backend/storage/smgr/md.c
	src/backend/utils/adt/float.c
	src/backend/utils/adt/numeric.c
	src/include/catalog/catversion.h
	src/include/storage/smgr.h
	src/include/utils/builtins.h
	src/test/regress/expected/numeric.out
	src/test/regress/sql/numeric.sql
	src/tools/msvc/Project.pm
	src/tools/msvc/README
	src/tools/msvc/Solution.pm
	src/tools/msvc/build.bat
	src/tools/msvc/mkvcbuild.pl
	src/tools/msvc/pgbison.bat
	src/tools/msvc/pgflex.bat

As promised in previous commit. Upstream patch:

commit bd0a2609
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Fri Jun 1 19:38:07 2007 +0000

    Make CREATE/DROP/RENAME DATABASE wait a little bit to see if other backends
    will exit before failing because of conflicting DB usage.  Per discussion,
    this seems a good idea to help mask the fact that backend exit takes nonzero
    time.  Remove a couple of thereby-obsoleted sleeps in contrib and PL
    regression test sequences.

In the upstream, we turned autovacuum on by default by one of the merged
commits. That change was rejected - autovacuum is still permanently
disabled in gpdb. We might want to test and re-enable it, but let's do that
separately from the 8.3 merge.

The rest of the autovacuum changes changes were applied, to keep the code
in sync with the upstream, although it's dead code when autovacuum can't
be enabled.

This patch removed the 5 seconds wait from CountOtherDBBackends(), that was
backported from later PostgreSQL versions. I'm going to cherry-pick that
change separately, to get that back. (It was introduced in PostgreSQL 8.3,
so we'd get it back later in the merge anyway, but let's take it now to
avoid the regression suite issues that it was originally backpatched for.)

Conflicts:
	doc/src/sgml/config.sgml
	doc/src/sgml/maintenance.sgml
	src/backend/access/transam/twophase.c
	src/backend/commands/dbcommands.c
	src/backend/storage/ipc/procarray.c
	src/backend/storage/lmgr/proc.c
	src/backend/utils/misc/guc.c
	src/backend/utils/misc/postgresql.conf.sample
	src/include/postmaster/autovacuum.h

This file got left over from the commit 29cccef6.

Instead of creating a mock version of every backend object file, only
create mock files as needed by individual tests. Speeds up the build, and
is nicer anyway.

This inverts the logic of the <testname>_REAL_OBJS mechanism. We now link
with the real version of each object file, *except* those listed in
<testname>_MOCK_OBJS variable.

Some tests were already doing this, but it seems nicer to default to using
the real memory manager for all tests. An individual test can still opt-out
by listing the relevant files in <testname>_MOCK_OBJS. Some tests had had
to jump through hoops to mock the memory manager, which is no longer
necessary.

There are a bunch of backend files that are never mocked, but always linked
into test program as is. List those in Makefile.mock, so that every Makefile
doesn't need to list them. Makes the makefiles simpler, and you need less
boilerplate code when adding a new test program.

Not all of these backend files were listed in the _REAL_OBJS variable of
every test, but in those tests that they were not, they were also not used,
so it doesn't matter if we use the real or mock version.

This way we don't need to link the whole backend into client mock programs.
We've gotten away with it so far, but it would cause clashes if one of the
client programs contained a function or global variable with same name as
in the backend. And of course it's completely unnecessary anyway.