Hash partitioning was never fully implemented, and was never turned
on by default. There has been no effort to complete the feature, so
rather than carrying dead code this removes all support for hash
partitioning. Should we ever want this feature, we will most likely
start from scratch anyways.

As an effect from removing the unsupported MERGE/MODIFY commands,
this previously accepted query is no longer legal:

	create table t (a int, b int)
	distributed by (a)
	partition by range (b) (start () end(2));

The syntax was an effect of an incorrect rule in the parser which
made the start boundary optional for CREATE TABLE when it was only
intended for MODIFY PARTITION.

pg_upgrade was already checking for hash partitions so no new check
was required (upgrade would've been impossible anyways due to hash
algorithm change).

Running ALTER TABLE PARTITION SPLIT on range subpartitions results in both new
partitions to incorrectly have the same partition order value (parruleord in
pg_partition_rule). ALTER TABLE PARTITION SPLIT is accomplished by running
multiple DDLs in sequence:

  1. CREATE TEMP TABLE to match the data type/orientation of the partition we are
     splitting

  2. ALTER TABLE PARTITION EXCHANGE the partition with the new temporary table.
     The temporary table now contains the partition data, and the partition
     table is now empty.

  3. ALTER TABLE DROP PARTITION on the exchanged partition (the new empty table)

  3a. Drop the partitioning rule on the empty partition

  3b. DROP TABLE on the empty partition

At this point (in the old behavior) we remove the partition rule from the in
memory copy of the partition metadata. We need to remove it from the context
here or ADD PARTITION will believe that a partition for the split range already
exists, and will fail to create a new partition.

Now, create two new partitions in the place of the old one. For each partition:

  4a. CREATE TABLE for the new range

  4b. ADD PARTITION - Search for a hole in the partition order to place the
      partition. Open up a hole in the parruleord if needed.

When adding a subpartition, ADD PARTITION relies on the partition rules passed
to it in order to find any holes in the partition range. Previously, the
metadata was not refreshed when adding the second partition, and this resulted
in the ADD PARTITION command creating both tables with the same partition rule
order (parruleord). This commit resolves the issue by refreshing the partition
metadata (PgPartRule) passed to the CREATE TABLE/ADD PARTITION commands upon
each iteration.

The storage/basic/partition tests were a collection of bugfixes from
primarily older version of Greenplum. This moves the valuable tests
over to ICW and removes the ones which are already covered in existing
ICW suites. The decision for each individual tests is elaborated on in
the list below:

* MPP-3379 was testing an ancient bug and MPP-3553 a hypothetical
  bug from before the current partition code was written. Combined
  the two since the added clause will still cover the ancient 3379
  issue and remove the other from Tinc.

* MPP-3625 was mostly already covered by existing tests in partition
  and bfv_partition. Added a test for splitting a non-existing default
  as that was the only thing covered.

* MPP-6297 is testing the pg_get_partition_def() function after the
  partition hierarchy has been amended via ALTER TABLE, which is
  already covered by the partition suite. Additionally it tested
  running pg_dump, which use pg_get_partition_def() heavily, and
  this is covered by our pg_upgrade tests.

* MPP-6379 tested that partitions inherited unique indexes, the test
  was moved to the partition_indexing suite.

* MPP-6489 tested ALTER TABLE .. SET DISTRIBUTED BY for subpartitions
  which didn't seem covered in ICW so moved to alter_distribution_policy
  suite

* MPP-7661 tested for an old bug where pg_dump incorrectly handled
  partition hierarchies created with the EVERY syntax. pg_upgrade
  tests in ICW will run this test on hierarchies from the partition
  suites so remove.

* MPP-7734 tested for excessive memory consumption in the case of a
  very deep partitioning hierarchy. The issue in question was fixed
  in 4.0 and the test clocks in at ~ 1.5 minutes, so remove the test
  to save time in the test pipeline. The test was more of a stress
  test than a regression test at this point, and while thats not of
  no importance, we should run stresstests deliberately and not
  accidentally.

* MPP-6537/6538 tested that new partitions introduced in the hierarchy
  correctly inherited the original owner. Test moved to the partition
  suite.

* MPP-7002 tested that splitting partitions worked after a column
  had been dropped. Since we had a similar test already in the
  partition suite, extend that test to also cover splitting.

* MPP-7164 is testing for partition rank after DDL operations on the
  partition hierarchy. I can see that we are testing something along
  the lines of this in ICW so pulled it across. In the process, fixed
  the test to actually run the SQL properly and not error out on a
  syntax error. Also removed a duplicated file for setting up the views.

* MPP-9548 tested for the case of insert data into a column which
  was dropped and then re-added. Since we already had that covered
  in the partition suite, simply extended the comment on that case
  and added another projection to really cover it completely.

* MPP-7232 tests that pg_get_partition_def() supports renamed partitions
  correctly. Added to ICW but the pg_dump test removed.

* MPP-17740 tested the .. WITH (tablename = <t> ..) syntax which we
  to some extent already covered, but added a separate case to test
  permutations of it.

* MPP-18673 covers an old intermittent bug where concurrent sessions
  splitting partitions would corrupt the relcache. The test in Tinc
  didn't however run concurrently so the underlying bug wasn't being
  tested, so remove the test. If we want a similar test at some point
  it should be an isolationtester suite.

* MPP-17761 tests an old bug where splitting an added CO partition
  failed. Since we didn't have much in terms of testing splitting
  CO, extended the block doing split testing with that.

* MPP-17707-* were essentially the same test but with varying storage
  options. While a lot of this is covered elsehwere, these tests were
  really easy to read so rolled them all into a new suite called
  partition_storage.

* MPP-12775 covered yet another split/exchange scenario. Added a
  short variant to the partition suite.

* MPP-17110 tests for an old regression in attribute encoding for
  added columns in partitioning hierarchies. Removed the part of
  the test that checked compression ratio as AO compression should
  be tested elsewhere.

* The partition_ddl2 test was moved over as partition_ddl more or
  less unchanged.

This also removes unused answer files mpp8031.ans.orca and query99
for which there were no corresponding tests, as well as the data
file used for copying data into the tests (a copy of which already
exists in src/test/regress/data).

Partition exchange for a partition table with sub-partitions should
throw an error as soon as the incompatability is schemas between the
partition table and the candidate table to exchange is found.
Signed-off-by: NJesse Zhang <sbjesse@gmail.com>

I'm not sure we need all of these, but let's at least move them into the
main regression suite. I did leave out a few queries that were identical to
existing tests that I spotted, and merged a few that were almost identical.

The old mechanism was to scan the complete plan, searching for a pattern
with a Join, where the outer side included an Append node. The inner
side was duplicated into an InitPlan, with the pg_partition_oid aggregate
to collect the Oids of all the partitions that can match. That was
inefficient and broken: if the duplicated plan was volatile, you might
choose wrong partitions. And scanning the inner side twice can obviously
be slow, if there are a lot of tuples.

Rewrite the way such plans are generated. Instead of using an InitPlan,
inject a PartitionSelector node into the inner side of the join.

Fixes github issues #2100 and #2116.

The loop to print each constraint's name was broken: it printed the name of
the first constraint multiple times. A test case, as matter of principle.

In the passing, change the set of tests around this error to all use the
same partitioned table, rather than drop and recreate it for each command.
And reduce the number of partitions from 10 to 5. Shaves some milliseconds
from the time to run the test.

And remove the rest of old partition_exchange TINC test. We already had
tests in 'partition' test file, for validating that the values in the
exchanged table are valid for the partition (search for "-- validation").

There were also tests for exchanging a partition in a hash partitioned
table, but the validation hasn't apparently been implemented for hash
partitions, and they're not supported anyway, so that doesn't seem very
interesting to test.

Rename the test tables in 'partition' test, so that they don't clash with
the test tables in 'partition1' test. Change a few validation queries to
not get confused if there are unrelated tables with partitions in the
database. With these changes, we can run 'partition' and 'partition1' in
the same parallel group, which is a more logical grouping.

All of these tests were already in the 'partition' test of the main suite,
in more or less identical form.

I did add one little query to 'partition', to test that START of a partition
range cannot be NULL. The existing test only covered "END (NULL)".

Also, stop testing for long-gone scripts gpaddpart, gpcreatepart and
gpdeletepart utilities.

The different kinds of NOTICE messages regarding table distribution
were using a mix of upper and lower case for 'DISTRIBUTED BY'. Make
them consistent by using upper case for all messages and update the
test files, and atmsort regexes, to match.

The fix removes the assumption that no other partition table exists before the
test starts.

    When we execute an ALTER TABLE statement that splits default partition into a "New" and a new Default partition, we perform the following 5 steps:

     * copy the tuples located in the default partition into a temporary table,
     * drop default partition,
     * create the “New“ partition,
     * create the new DEFAULT partition, and
     * copy all tuples from the temporary table into the newly created partitions.

    The last step is executed in split_rows, where for each tuple in the temporary table,

     * we determine the target partition/table,
     * construct the slot of the target table (if is NULL), and
     * store the tuple into the target table.

    Currently, we allocate the tuple table slot of the target table into a per-tuple memory context. If the size of that memory context exceeds 50KB, then we reset it. This will cause issues, since it will free target table's slot and at the next iteration (copy the next tuple) we will try to access the attributes of a freed (and not NULL) slot. In addition, the target table slot lifespan is much longer than an individual tuple. So it is not correct to allocate the slot in a per-tuple memory context.

    To solve this issue, we allocate target's tuple table slot in PortalHeapMemory. This is the CurrentMemoryContext that is used in split_rows before we start copying tuples. PortalHeapMemory is used already for storing the tuple table slot of the temporary table and the ResultRelInfo of the target tables. PortalHeapMemory is not freed while we copy tuples. After copying all tuples to the new partitions, we drop target tables' slots.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

This is a partial backport of a larger body of work which also already have
been partially backported.

Remove the GPDB-specific "breadcrumbs" mechanism from the parser. It is
made obsolete by the upstream mechanism. We lose context information from
a few errors, which is unfortunate, but seems acceptable. Upstream doesn't
have context information for those errors either.

The backport was originally done by Daniel Gustafsson, on top of the
PostgreSQL 8.3 merge. I tweaked it to apply it to master, before the
merge.

Upstream commit:

  commit b153c092
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Mon Sep 1 20:42:46 2008 +0000

    Add a bunch of new error location reports to parse-analysis error messages.
    There are still some weak spots around JOIN USING and relation alias lists,
    but most errors reported within backend/parser/ now have locations.

The PartitionNode tree returned from RelationBuildPartitionDescByOid
will be NULL in case the OID passed isn't present in pg_partition so
we must abort with error to avoid segfaulting on NULL pointer deref.
Also add a test in the partition suite for this.

Reported by Github user @liruto.

Currently, the error reporting for duplicate key value violation for unique
indexes does not contain useful information for users when debugging. This
commit backports two commits from PostgreSQL (shown below) and updates our
multikey tuplesort to use the same. Now the error will display the first
instance of a duplicate key violation.

Authors: Jimmy Yih and Abhijit Subramanya

commit b680ae4b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat Aug 1 19:59:41 2009 +0000

    Improve unique-constraint-violation error messages to include the exact
    values being complained of.

    In passing, also remove the arbitrary length limitation in the similar
    error detail message for foreign key violations.

    Itagaki Takahiro

 Conflicts:
	contrib/citext/expected/citext.out
	contrib/citext/expected/citext_1.out
	src/backend/access/common/indextuple.c
	src/backend/access/index/genam.c
	src/backend/access/nbtree/nbtinsert.c
	src/backend/utils/adt/ri_triggers.c
	src/backend/utils/adt/ruleutils.c
	src/include/access/genam.h
	src/include/access/itup.h
	src/include/utils/builtins.h
	src/test/regress/expected/alter_table.out
	src/test/regress/expected/arrays.out
	src/test/regress/expected/create_index.out
	src/test/regress/expected/plpgsql.out
	src/test/regress/expected/transactions.out
	src/test/regress/expected/uuid.out
	src/test/regress/output/constraints.source
	src/test/regress/output/tablespace.source

commit 527f0ae3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat Aug 1 20:59:17 2009 +0000

    Department of second thoughts: let's show the exact key during unique index
    build failures, too.  Refactor a bit more since that error message isn't
    spelled the same.

 Conflicts:
	src/backend/access/nbtree/nbtinsert.c
	src/backend/utils/sort/tuplesort.c
	src/test/regress/expected/alter_table.out
	src/test/regress/expected/create_index.out

the cleanup of EState

Users should use error log functionality for error handling in external tables
and during copy since it is more robust and reliable than the old error table
functionality.

This test takes a very long time, so by splitting it and running other tests
in parallel, we can hopefully utilize multiple CPUs better during regression
tests, and finish the tests earlier.

There's no big principle on where exactly I split this, but I did take care
that the new 'partition1' test case doesn't need different expected output
files for the ORCA and non-ORCA case. That always makes maintenance easier.

These tests query pg_locks and fail if there are concurrent transactions
holding locks. Isolate them to separate file, so that we can safely run
other tests in parallel.

Try to put tests into larger groups, and try to run long-running tests
in parallel. This utilizes multiple CPUs better, making the overall suite
finish faster. On my laptop, this reduced the overall runtime of the
greenplum_schedule (i.e. excluding the Postgres tests in parallel_schedule)
from about 4 minutes to 3 minutes.

Exchanging a default partition is currently not allowed because there is
no validation on the data being exchanged. This commit add a GUC to
enable this if the user chooses to do so.