I kept the tests using QuickLZ in TINC, as we don't have QuickLZ support in
the open source version anymore. They ought to be moved to where we keep
the code for reading legacy quicklz-compressed tables.

There were three variants of the same test, combined with 4 compresslevels.
The "*_null" variants insert rows, such that the delta between some rows is
positive, and between others, negative. And it also contains some NULLs.
The other variants only contained positive or postive and negative deltas,
without NULLs. It doesn't seem interesting to have separate tests for the
more narrow cases. If there's something wrong with the handling of positive
or negative deltas, the "*_nulls" tests will catch that too.

OIDS defaults to 'off' for user tables, so this test is effectively the
same as not specifying OIDS at all. (And OIDS=true is not supported on AOCS
tables.)

Commit 714a8375 removed the Python script that referenced these.

There is no tincrepo/mpp/gpdb/tests/utilities/upgrade directory.

We must support older than 80300 in dumping pg_type Oid's in order
to support upgrades from 4.3. Remove bogus check inherited from
upstream PostgreSQL where 8.3 is the earliest version supported by
pg_upgrade. Replace with a check for 8.2 as the base version in
the main binary upgrade dump function and explain the difference
to upstream.

In case there were no auxiliary relations associated with the rel
oid then skip adding the Archive entry to not have blank entries
in the dump with just a header comment.

The conrelid is equal to the oid (it's a join criteria after all).
Perform a little bit of cleanup for clarity (and microoptimization)
and don't project it into the result.

This moves over the queries worth keeping from Bugbuster gpsort to
ICW sort. Many of the queries returned a single row, no rows at all
or even had syntax errors, all of which are uninteresting cases for
testing result sorting. All rows in the test table are distributed
to a single segment to keep the sort order stable during testing.

Also remove pointless ignore blocks from sort and avoid dropping
the schema at the end so shave some time off.

The gp_enable_alter_table_inherit_cols GUC was used to allow a list
of columns to override the attribute discovery for inheritance in
ALTER TABLE INHERIT. According to code comments the only consumer
of this was gpmigrator, but no callsite remains and no support in
pg_dumpall remains either. Remove the leftovers of this to avoid a
potential footgun and get us closer to upstream code.

TINC had a lot of gpmapreduce tests which weren't connected to any
build or pipeline, resurrect (some of) these by porting over to the
mapred suite in ICW.

This is by no means a direct copy of the existing TINC testsuites,
but instead a partial port of the tests that made sense to bring
over. A lot of the tests were highly overlapping, some completely
broken, some were testing functionality ICW already had tests for
and others plain uninteresting.

The gp_external_grant_privileges GUC was needed before 4.0 to let
non superusers create external tables for gphdfs and http protocols.
This GUC was however deprecated during the 4.3 cycle so remove all
traces of it. The utility of the GUC was replaced in 4.0 when rights
management for external tables was implemented with the normal GRANT
REVOKE framework so this has been dead code for quite some time.

Remove GUC, code which handles it, all references to it from the
documentation and a release notes entry.

Release notes entry for the removal of gp_hashagg_compress_spill_files,
gp_eager_hashtable_release, max_work_mem and gp_hash_index.

The gp_eager_hashtable_release GUC was deprecated in version 4.2
in 2011 when the generic eager free framework was implemented. The
leftover gp_eager_hashtable_release was asserted to be true and
never intended to be turned off. The same body of work deprecated
the max_work_mem setting which was bounding the work_mem setting.
While not technically tied to eager hashtable release, remove as
well since it's deprecated, undocumented and not terribly useful.

Relevant commit in closed source repo is 88986b7d

The gp_hashagg_compress_spill_files GUC was deprecated in 2010 when
it was replaced by gp_workfile_compress_algorithm. The leftovers
haven't done anything for quite some time so remove GUC.

Relevant commit in closed source repo is c1ce9f03

Revert commit ffc6226a and 1d31323a. It failed on concourse.

In commit ffc6226a, I removed DROP + CREATE DATABASE from these tests.
That didn't work, you now get "relation already exists" errors from, after
the first test. I'm trying to fix this in the blind one more time, by
changing the tests to use a separate schema, and DROP + CREATE the schema
between each test. That's still a lot cheaper than dropping and creating
a database.

The 'gptest' database was actually used by the tests. CREATE DATABASE is
a fairly expensive operation, so let's avoid doing it without a good
reason.

I don't actually have means to test this properly, so I'm going to push
this in blind and see what Concourse thinks of it. If something goes awry,
I'll revert.

The 'add_part_checksum_with_guc_enable' test tested that a newly-created
table has checksums enabled, when you don't set gp_default_storage_options,
and you explicitly set checksums on in the WITH clause. That's not very
interesting, because checksums default to 'on' anyway. We have plenty of
coverage of that default already.

The 'add_part_checksum' test was marginally useful. It tested that
if you set checksums off in the GUC, but explicitly specify checksums=on
in the CREATE TABLE, checksums really are enabled. And it tested the same
for ALTER TABLE ADD PARTITION. Move that test to the 'dsp' test in the
main regression suite.

REINDEX chooses the new relfilenode for an index independently in each
segment (see also 88f0623e). Therefore, it is not safe to assume, that
after REINDEX, the relfilenode would be the same in all segments, if there
is concurrent activity. These tests failed a few times in the concourse
pipeline. The purpose of this test, per the @description, is to test
permissions, so I'm not sure why these relfilenode checks were tacked on to
it in the first place.

There's retry logic within the monitoring pulse script, but we still
occasionally get hiccups (just network because the monitoring lasts for
a while).
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

* &user_result yields a singleton pointer that is used as an array.
* &text yields a singleton pointer that is used as an array.

- Our rough heuristic was to use it for jobs that tend to take longer
  than an hour
- We think this will address resource concerns, compensating for the
  fact that long-running tasks are no longer gated by ICW
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Don't delay starting long-running tests; kick those off sooner

Until now, the packaging job has served three purposes:
- consolidating the fact that a binary passed all ICW testing (or tests
  at similar levels)
- being a gatekeeper for long-running & resource-intensive jobs, often
  those on Pulse
- producing a wide variety of packages

The packaging job happened to serve the role as gatekeeper because the
pulse jobs run against packaged installers.

Because of this change, the package files in S3 will not necessarily
correspond to binaries which passed ICW. If this is concerning, we could
implement a promotion mechanism for these artifacts.

- move QP_runaway-query and regression_tests_gphdfs_centos to be
  adjacent to other tests that run against binaries (not packages)
- improve pipeline-ing of the gpaddon_src as it's used in multiple
  stages
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>
Signed-off-by: NChumki Roy <croy@pivotal.io>
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Signed-off-by: NChumki Roy <croy@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Similarly to the DROP DATABASE tests in commit 3c2a2200, the ALTER
DATABASE always fails because the connection is active. That is not
interesting to test.

In addition to pipeline changes, Some tests are modified slightly so
that they need less data overall to go OOM.

Commit 92c36a45 moved these to the same parallel group, but that turned
out to be a bad idea. With assertions enabled that can trigger an assertion,
see https://github.com/greenplum-db/gpdb/issues/1859. Also, the test queries
in the 'resource_queue' test assume that there are no other resource queues
active, and sometimes fails because the test_q queue created by the
resource_queue_functions test is active at the same time. Those queries
should be changed to not be sensitive to other resource queues, probably by
qualifying the queries with "WHERE rsqname LIKE 'reg_%'", but for now,
avoid the test failures by running them separately.

Added a test for following scenario:
When analyzedb is run without --skip_root_partition flag like below,
it should also analyze the root partitions.

`analyzedb -d <dbname> -s <schema> -a`

The root parition table gets listed only in the report file (and not in
any state files), hence added appropriate routines for checking the root
partition in the report file.
Signed-off-by: NJemish Patel <jpatel@pivotal.io>

When analyzedb is run without --skip_root_partition flag like below,
it should also analyze the root partitions.

`analyzedb -d <dbname> -s <schema> --full -a`

The expected key format for `leaf_root_dict` used for the check
is a tuple form and the result returned from the SQL was a string
concatenated with ".". Because of this, candidates could not be matched
in the dictionary and we never got the root partition details.

Made the appropriate changes in analyzedb so that it will
identify and analyze on the root partitions.
Signed-off-by: NJemish Patel <jpatel-pivotal@pivotal.io>

Signed-off-by: NMarbin Tan <mtan@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

This was commented out close to 10 years ago so it seems about time
to retire this to the attic.

This applies minor cosmetic cleanup to the partition code stemming
from a read-through. There are no functional changes from this:

  * Remove stale comments and reflow existing ones as well as fix
    some typos
  * Remove spurious whitespace
  * Re-indent and rewrite where the logic isn't clear to improve
    readability (removing !!(foo) construction).

There is no gain in calling truncateStringInfo() on a StringInfo
which was just inited, it will always be no-op. Remove truncates
from the partitioning codepath and save a cycle or two. Also
remove two StringInfos where one were unused and the other could
be replaced with a pstrdup() call.