CodegenUtils.
This closes #648
Signed-off-by: NFoyzur Rahman <frahman@gmail.com>

In gpconfigurenewsegment, the class ConfExpSegCmd inherits from the class Command.
Command uses a variable of type ExecutionContext which was updated back in December
2011 to include passing the Command object itself to the execute(), interrupt(), and
cancel() ExecutionContext methods. The author of that patch did not update the
gpconfigurenewsegment.ConfExpSegCmd class which overrides those methods. It doesn't
make sense to even override them in the first place so they should just be removed.

Fixes commit f569c1d1.  The query used
in this test to identify missing attributes returned values for both the
primary and mirror segments, and has been modified to avoid doing so.
Also, if an attribute is deleted from every segment this is detected as
an error with the master, rather than an error with each segment, so the
test now only deletes one segment's attribute so it can test for master
and segment error messages.

Authors: Chris Hajas and Jamie McAtamney

- gpcheckcat should return a return code of 0 if schemas are found/dropped
- Backfilled tests for leaked schema logging
- Also cleaned up typo in Makefile

Gpcrondump generates prefix for each different dump file:

For global objects, dump file prefix is gp_global_1_1_*, returned
by method generate_global_prefix().
For statistics dump, file prefix is gp_statistics_1_1_*, returned
by method generate_stats_prefix()

remove scripts directory.
remove LICENSE file since it's the same as GPDB's.
remove dead codes and meaningless comments.
reorder header includings alphabetically.
add a missing unit test for find_replace().
rename testing sql files.
rename source files.
optimize compiling flags.

Currently, the error reporting for duplicate key value violation for unique
indexes does not contain useful information for users when debugging. This
commit backports two commits from PostgreSQL (shown below) and updates our
multikey tuplesort to use the same. Now the error will display the first
instance of a duplicate key violation.

Authors: Jimmy Yih and Abhijit Subramanya

commit b680ae4b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat Aug 1 19:59:41 2009 +0000

    Improve unique-constraint-violation error messages to include the exact
    values being complained of.

    In passing, also remove the arbitrary length limitation in the similar
    error detail message for foreign key violations.

    Itagaki Takahiro

 Conflicts:
	contrib/citext/expected/citext.out
	contrib/citext/expected/citext_1.out
	src/backend/access/common/indextuple.c
	src/backend/access/index/genam.c
	src/backend/access/nbtree/nbtinsert.c
	src/backend/utils/adt/ri_triggers.c
	src/backend/utils/adt/ruleutils.c
	src/include/access/genam.h
	src/include/access/itup.h
	src/include/utils/builtins.h
	src/test/regress/expected/alter_table.out
	src/test/regress/expected/arrays.out
	src/test/regress/expected/create_index.out
	src/test/regress/expected/plpgsql.out
	src/test/regress/expected/transactions.out
	src/test/regress/expected/uuid.out
	src/test/regress/output/constraints.source
	src/test/regress/output/tablespace.source

commit 527f0ae3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat Aug 1 20:59:17 2009 +0000

    Department of second thoughts: let's show the exact key during unique index
    build failures, too.  Refactor a bit more since that error message isn't
    spelled the same.

 Conflicts:
	src/backend/access/nbtree/nbtinsert.c
	src/backend/utils/sort/tuplesort.c
	src/test/regress/expected/alter_table.out
	src/test/regress/expected/create_index.out

Add behave test for gpcheckcat: catalog problems for conflicting db ownership will generate repair scripts that have timestamped names (to avoid overwriting)
- add behave steps for removing and also validating directories that are relative to the current working directory (allows * globbing)
- criteria for determining that repair file was not overwritten:  run gpcheckcat twice, and expect 1 and then 2 repair scripts.
- verify that stdout reports "owner" error
- change criteria for gpcheckcat ownership: relax the pg_type check so that both 4.3STABLE and master report an ownership error for pg_type.

This was causing issues on make check where the logger gets inherited
by other unittests and causes unexpected log outputs.

Authors: Marbin Tan & Stephen Wu

Updates unique index query to find column names by querying pg_attribute instead of string-parsing the index definition.  This allows us to support opclasses as well as special characters in the column name.

The maximal number of distributed by columns is 1600, gpdb should error out when
it exceeds the limitation. Another thing is gpdb should allocate enough memory to
hold those columns, otherwise it will cause memory overflow.

Three functions are available now:
- convert_from(bytea, name) returns text
- convert_to(text, name) returns bytea
- length(bytea, name) returns int

Related commits (partial codes are backported before OSS but not actived)

    commit a9742f12
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Wed Jan 9 23:43:54 2008 +0000

        Remove incorrect (and ill-advised anyway) pfree's in pg_convert_from and
        pg_convert_to.  Per bug #3866 from Andrew Gilligan.

    commit 02138357
    Author: Andrew Dunstan <andrew@dunslane.net>
    Date:   Mon Sep 24 01:29:30 2007 +0000

        Remove "convert 'blah' using conversion_name" facility, because if it
        produces text it is an encoding hole and if not it's incompatible
        with the spec, whatever the spec means (which we're not sure about anyway).

    commit 55613bf9
    Author: Andrew Dunstan <andrew@dunslane.net>
    Date:   Tue Sep 18 17:41:17 2007 +0000

        Close previously open holes for invalidly encoded data to enter the
        database via builtin functions, as recently discussed on -hackers.

        chr() now returns a character in the database encoding. For UTF8 encoded databases
        the argument is treated as a Unicode code point. For other multi-byte encodings
        the argument must designate a strict ascii character, or an error is raised,
        as is also the case if the argument is 0.

        ascii() is adjusted so that it remains the inverse of chr().

        The two argument form of convert() is gone, and the three argument form now
        takes a bytea first argument and returns a bytea. To cover this loss three new
        functions are introduced:
        . convert_from(bytea, name) returns text - converts the first argument from the
          named encoding to the database encoding
        . convert_to(text, name) returns bytea - converts the first argument from the
          database encoding to the named encoding
        . length(bytea, name) returns int - gives the length of the first argument in
          characters in the named encoding

    commit 75c6519f
    Author: Tatsuo Ishii <ishii@postgresql.org>
    Date:   Sun Mar 25 11:56:04 2007 +0000

        Add new encoding EUC_JIS_2004 and SHIFT_JIS_2004,
        along with new conversions among EUC_JIS_2004, SHIFT_JIS_2004 and UTF-8.
        catalog version has been bump up.

Committers: Kuien Liu and Haozhou Wang

more readable format.

Previously, gpcheckcat identified tables with missing attributes but
displayed them in various locations in the output in a
non-standardized format. This commit summarizes all the tables with missing
attributes in one section at the end of the output in the format
"[database].[schema].[table].[segment id]".

Authors: Chris Hajas and Jamie McAtamney

This commit fixes issues below:

extractContent() might forward NULL to marker.
ListBucket() fogets to delete keylist when errors happen.
contents in ListBucketResult are not cleared.
downloader is not destroyed in unit tests.
get_opt_s3() and truncate_options() in unit tests need to be freed.
xmlNodeGetContent() needs to be freed.
xml.ctxt->myDoc needs to be freed.
function for the XML library needs to be cleared.

Thanks to Kuien and Valgrind.

Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

- print out all information about violated indexes in the logs
- changes unique index check to also return the list of violated
  segments in order to support this

- optimize query to find all unique indexes
- omit rows with null values when searching for violated indexes

- more accurate function names in GPObject
- remove duplication of query strings
- make file import for test subject independent of current working
  directory
- update test names to match casing convention

- additionally, minor refactoring in gpcheckcat and unique index check

- Before this we were adding issues to the report but not actually
  setting the checkStatus variable to False

Inheriting from GpTestCase instead of unittest.TestCase gives you:
- access to a logger via self.logger
- the ability to patch methods/objects by calling apply_patches with a list of patches
- automatic teardown of patches after tests are run
- hiding unittest dependency from the actual unit tests (e.g. overriding skip() and providing run_tests() to wrap main())

Also cleaned up unit tests for the unique index violation check

- mock pg.connect instead of patching gpcheckcat global method connect()
- use a magic mock for GV.cfg since we are trusting that it has been implemented properly

- includes unit tests for gpcheckcat covering everything after
  option parsing and everything before writing to stdout/logs
- in the process of rebasing, removes the tests for dropping leaked
  schemas.  These tests will be added back in the later commit
- adds a behave integration test for this check

- also adds unit test for this module
- returns table name, index name, table oid, and segment id of each
  violation

Along the way, fix trailing tabs and split a long error
message into two lines.

This will fix #524

A new version of libXML changes the output format, this patch
adapts the new "expected" file introduced in
PostgreSQL commit 085423e3.
Slight modifications by me.

This patch will not fail the current Pulse run in any way, because
the Pulse systems do not yet use the new libXML library, and therefore
are happy with xml.out and xml_1.out. I stripped down the regression
tests and this is the output without the patch:

parallel group (14 tests):  limit prepare conversion xml rowtypes copy2 polymorphism sequence without_oid temp truncate domain rangefuncs alter_table
     limit                ... ok (0,24 sec)  (diff:0,07 sec)
     copy2                ... ok (2,44 sec)  (diff:0,07 sec)
     temp                 ... ok (3,83 sec)  (diff:0,07 sec)
     domain               ... ok (7,12 sec)  (diff:0,10 sec)
     rangefuncs           ... ok (12,15 sec)  (diff:0,09 sec)
     prepare              ... ok (0,49 sec)  (diff:0,06 sec)
     without_oid          ... ok (3,49 sec)  (diff:0,07 sec)
     conversion           ... ok (1,19 sec)  (diff:0,17 sec)
     truncate             ... ok (4,03 sec)  (diff:0,06 sec)
     alter_table          ... ok (27,88 sec)  (diff:0,32 sec)
     sequence             ... ok (3,18 sec)  (diff:0,08 sec)
     polymorphism         ... ok (3,13 sec)  (diff:0,11 sec)
     rowtypes             ... ok (2,17 sec)  (diff:0,09 sec)
     xml                  ... FAILED (1,69 sec)  (diff:0,39 sec)
test variadic_parameters  ... ok (1,57 sec)  (diff:0,07 sec)
test default_parameters   ... ok (0,68 sec)  (diff:0,07 sec)

=======================
 1 of 97 tests failed.
=======================

This is the output with the patch:

parallel group (14 tests):  limit prepare conversion xml rowtypes copy2 polymorphism sequence without_oid temp truncate domain rangefuncs alter_table
     limit                ... ok (0,21 sec)  (diff:0,10 sec)
     copy2                ... ok (1,95 sec)  (diff:0,06 sec)
     temp                 ... ok (3,41 sec)  (diff:0,26 sec)
     domain               ... ok (6,67 sec)  (diff:0,19 sec)
     rangefuncs           ... ok (11,55 sec)  (diff:0,09 sec)
     prepare              ... ok (0,41 sec)  (diff:0,06 sec)
     without_oid          ... ok (2,91 sec)  (diff:0,06 sec)
     conversion           ... ok (0,96 sec)  (diff:0,15 sec)
     truncate             ... ok (3,62 sec)  (diff:0,09 sec)
     alter_table          ... ok (25,79 sec)  (diff:0,22 sec)
     sequence             ... ok (2,89 sec)  (diff:0,06 sec)
     polymorphism         ... ok (2,50 sec)  (diff:0,09 sec)
     rowtypes             ... ok (1,87 sec)  (diff:0,12 sec)
     xml                  ... ok (1,54 sec)  (diff:0,30 sec)
test variadic_parameters  ... ok (1,43 sec)  (diff:0,07 sec)
test default_parameters   ... ok (0,43 sec)  (diff:0,07 sec)

======================
 All 97 tests passed.
======================

Fix #550

It didn't do anything useful AFAICS.

Currently, the gpconfig utility is exiting with status 0 when one of
the option parsing checks fail.  It should return 1 instead.

* Support the aggregation transition datatype to be "internal"

Backport below commits from upstream:

    commit 22d9ddbb
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Sat Aug 14 15:47:13 2010 +0000

        Fix planner to make a reasonable assumption about the amount of memory space
        used by array_agg(), string_agg(), and similar aggregate functions that use
        "internal" as their transition datatype.  The previous coding thought this
        took *no* extra space, since "internal" is pass-by-value; but actually these
        aggregates typically consume a great deal of space.  Per bug #5608 from
        Itagaki Takahiro, and fix suggestion from Hitoshi Harada.

        Back-patch to 8.4, where array_agg was introduced.

    commit 9e0247ab
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Fri Nov 14 19:47:50 2008 +0000

        In CREATE AGGREGATE, allow the transition datatype to be "internal", but only
        if the user is superuser.  This makes available to extension modules the same
        sort of trick being practiced by array_agg().  The reason for the superuser
        restriction is that you could crash the system by connecting up an
        incompatible pair of internal-using functions as an aggregate.  It shouldn't
        interfere with any legitimate use, since you'd have to be superuser to create
        the internal-using transition and final functions anyway.

Besides, we cannot find regression tests in above commits, so we
add a set of tests for this feature, a part of them come from
legacy extension "intagg" regression tests.

Conflicts:
	src/test/regress/expected/arrays.out
	src/test/regress/sql/arrays.sql

* add tests to #588: run with role of (non-)superuser

    #588: Support the aggregate's transype to be "internal"

Normally, the aggregate's transtype can't be a pseudo-type.
However, we can allow "internal" for functions that want to
pass pointers to private data structures; but allow that only
to superusers. Here we add tests to verify this constraint.