occurances was a surprisingly common typo, fix all findings except
one in pg_regress.c which will be fixed in a much larger doc patch
as we merge upstream.

Compatible S3 services don't always return error with XML body, check it
before parsing.
Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>

The Bizgres name hasn't been used for a very long time, retire the
last mentions from the reference documentation.

Updates the reference documentation for psql to match the main GPDB
documentation. This only brings in the bare minimum as this reference
page is only used for psql.

From report by Github user yanchaozhong.

The TINC test tested that if you set gp_default_storage_options to
'checksum=False' and create tables, the tables are created without
checksums. We already have tests for that in the 'dsp' test in the main
test suite.

Add the 'checksum' column to a couple of validation queries in the 'dsp'
test, to ensure that it would catch it if setting checksums on/off didn't
work.

These tests are all variations:

1. Open a session to database, do some operations in it like CREATE INDEX,
   DROP INDEX or REINDEX.
2. In another session, attempt to drop the database.

The result is always that dropping the database fails, after a timeout,
with error: ERROR:  database "reindexdb" is being accessed by other users.

That's not a very interesting test case, because you get that same error
regardless of what the other backend is doing. It doesn't even need to
be doing anything, even an idle session has the same effect. Hence, remove
these pointless tests.

The SOMEFILE:SOMEFUNC construction was removed from atmsort with
commit 0bf31cd6, remove leftover
rules relying on it still in the error suite (obviously not needed
either since the suite still works). This brings the test files a
little closer to upstream.

Some tests in the AOCO compression TINC test suite were run with 1MB and
2MB block sizes. That seems excessive. There is no magic boundary, like
e.g. when going from 1 GB to 2 GB, between 1 MB and 2 MB, so it's hard to
see how anything would behave differently in any interesting way between
those two values. This test suite runs for a very long time, so remove
the 1 MB permutations from those cases, to reduce the runtime.

(The test files have 1G and 2G in the names, but that is misleading; these
tests vary the blocksize option to CREATE TABLE command, between 1MB and
2MB. That's megabytes, not gigabytes. That ought to be fixed, but not in
this patch)

All of these tests were already in the 'partition' test of the main suite,
in more or less identical form.

I did add one little query to 'partition', to test that START of a partition
range cannot be NULL. The existing test only covered "END (NULL)".

Also, stop testing for long-gone scripts gpaddpart, gpcreatepart and
gpdeletepart utilities.

Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

The "Execute on" syntax was added when performing a "\d" on external tables in gpdb5, but is not present in previous gpdb versions. When transferring from clusters on previous versions, we will ignore this syntax.

There are plenty of VACUUMs on append-only tables elsewhere in the
regular regression suite, e.g. in 'vacuum_gp', 'appendonly', 'aocs',
uao_compaction tests, and many more.

Subtransactions with append-only tables are covered at least by the
uao_dml/uao_dml_[row|column] tests in the main regression suite.

These ALTER TABLE commands are covered by the 'alter_table_ao' and
'alter_table_aocs' tests in the main regression suite.

Currently we see FTS failure in the CI pipeline due to the filerep resync
manager and workers are still in the recovery mode while relcache got
initialized.

This reverts commit f8f6587e and
0e9d10d2.

We have sufficient coverage for these debugging functions in
src/test/regress/sql/sql/gp_toolkit_ao_funcs.sql already.

Bad things happen otherwise.  One case in point is create database
followed by a crash.  Create database requests a checkpoint after
inserting new tuple into pg_database.  Crash happens right after
create database commits and before clog update is not flushed to disk.
Relcache initialization before xlog replay will set HEAP_XMIN_INVALID
hint bit in the newly created database's tuple because clog did not
report the xmin as committed.

The need for heap access methods before xlog replay is removed by commit
e2d6aa1481f6cdbd846d4b17b68eb4387dae9211.  This commit simply moves the
relcache initialization to pass4, where it is really needed.

Do not bother to remove relcache init file at the end of crash recovery pass2.

Error out if relation cache initialized at wrong time.

There are only a few tests left in "uao", most have now been moved to
installcheck-world. The "uao" job now runs in about 4 minutes, of which
2 minutes is spent on setting up the test cluster. That seems like
overkill, so simplify things by merging the remaining uao fault injection
tests into the larger "storage" job.

These are the same tests queries for column-oriented append-only tables,
as those moved by commit 11a5a807, for row-oriented append-only tables.
There were two additional tests that were never executed for row-oriented
tables though: phantom_reads_update_serializable and
phantom_reads_delete_serializable. I believe that was an oversight in the
original test suite; they are now also executed for row-oriented tables.

We use the UAO templating mechanism, to run the same test files against
row- and column-oriented tables. To make that work, fix a bug in the
templating mechanism pg_regress.c: if the --ao-dir argument was shorter
than 7 characters, the uao directory was not detected correctly.

We have sufficient test coverage for gpload in the Behave tests in
gpMgmt/test/behave/mgmt_utils. And we have plenty of tests for external
tables elsewhere.

The combination of using an external table or gpload as the source, and
append-only table as the target, is not particularly interesting. So remove
the tests.

Update pg_dump, cdb_dump_agent, psql to fix MU tests after ON MASTER patch

cURL automatically adds the CONTENT-LENGTH header when performs "PUT"
request, we also did it without this commit, AWS is OK with that, but
others would report duplicate CONTENT-LENGTH error.
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

palloc and repalloc will check request size is under MaxAllocSize,
but the implementation(AllocSetAllocImpl, AllocSetReallocImpl) allocs
ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ more bytes to store block
information. Later MemoryContextNoteAlloc is called with block size.
If palloc request size is between MaxAllocSize - (ALLOC_BLOCKHDRSZ +
ALLOC_CHUNKHDRSZ) and MaxAllocSize, an internal error occurs.

Fix: simply remove those assertions, the calculations in
MemoryContextNoteAlloc and MemoryContextNoteFree should work with larger
values.

This resolves https://github.com/greenplum-db/gpdb/issues/1090.
Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

AWS S3 returns internal errors (HTTP response code 500) serveral times,
unusual but better not to ignore, this commit re-sends requests when it
happens.
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>

* We were accessing a non-existent variable as it's being generated by
  the later function. Swapping the two methods.

* Fix on top of a86e4901

Signed-off-by: NMarbin Tan <mtan@pivotal.io>

Signed-off-by Gang Xiong <gxiong@pivotal.io>

Signed-off-by Gang Xiong <gxiong@pivotal.io>

The gppersistentrebuild tool was trying to use string_agg without
casting the oid to text. This used to work before but not anymore. The
behave tests only have two scenarios but the second scenario would
always fail because it would run when a primary/mirror segment pair
were still in resync mode.

* When mocking, we may setup patches that are not cleaned up properly
  and causes test pollutions. Ensure that we do tearDown properly and
  raise a condition if we are not.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

When running gpconfig -s with --file option, we can now view the
postgresql.conf guc settings instead of just viewing the user guc
settings in the database. This option also reports if there are any
inconsistencies with the postgresql.conf between the segments.

reformat gpconfig:
* enable unit testing

remove gpconfig white-space

Add gpconfig behave tests:
* When you run gpconfig with -s and --file, you can sometimes run into
a nonetype string format error. This is due to us not joining all the
threads properly and closing them out properly.

* Added gpconfig behave tests to ensure that we are not missing anything
 in the unittest tests. As we are mocking the pool, we need to make sure
 that we are not doing anything silly, so adding minimal behave tests.
 Single node is probably enough.
Signed-off-by: NChumki Roy <croy@pivotal.io>
Signed-off-by: NMarbin Tan <mtan@pivotal.io>