SET DISTRIBUTED BY and EXTEND TABLE subcommands worked differently from
all other ALTER TABLE subcommands in who's responsible for closing the
relcache reference. In all other subcommands, ATRewriteCatalogs opens and
closes the 'rel', but for these two commands, the ATExec*() function
closed it. I don't see any good reason for that. There were very old
comments about forcing the relcache entry to be forgotten, but that
explanation doesn't make sense to me, and everything seems to work without
the early closing. Maybe it was needed a long time ago, but the code has
changed a lot since it was written. Simplify, by closing the relation in
ATRewriteCatalogs(), like with all other ALTER TABLE subcommands.
Reviewed-by: NAsim R P <pasim@vmware.com>

The copydml test creates both BEFORE and AFTER triggers on a table and
checks the execution order of the client output.  The original output
order is "BEFORE -> RESULT -> AFTER", this is the order produced in
ic-tcp mode; in ic-udpifc mode the order has a chance to become "RESULT
-> BEFORE -> AFTER", it is not determined.  There are 2 variants of the
answer files for these 2 orders, so the test can pass on any order.

In ic-proxy mode, the order can also be "BEFORE -> AFTER -> RESULT", so
we also need a 3rd variant of the answer file for this order.  By
providing this we could pass the test in ic-proxy mode, and we could
re-enable the copydml test, it was previously disabled on ic-proxy
pipeline jobs.
Reviewed-by: NAsim R P <pasim@vmware.com>
Reviewed-by: NHubert Zhang <hzhang@pivotal.io>

walreceiver is quite sensitive to any WAL write. After create a table
and insert some tuples, it doesn't run a vacuum. There may be other
cases that cause some WAL traffic. One of the WAL records is
relevant to the hot-standby to record RUNNING_XACTS.

The last test_receive() only tests there should be no WAL traffic
after receiving the WAL records to the current xlog location. It still
has a gap that a new WAL record transmitted to the walreceiver.
It's not meaningful to run test_receive(), since the function
test_receive_and_verify() has verified that all the WAL traffic
to the latest xlog location is received. So, removed test_receive().
Reviewed-by: NPaul Guo <paulguo@gmail.com>
Reviewed-by: N(Jerome)Junfeng Yang <jeyang@pivotal.io>

gpsd was failing because the connectionString that we passed to pgdb.connect
had the parameters in the wrong order. It started failing after upgrading to a
higher version of PyGreSQL. So use a dictionary instead in order to avoid
sending in the parameters incorrectly.
Co-authored-by: NAshwin Agrawal <aashwin@vmware.com>

Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NHao Wu <gfphoenix78@gmail.com>

We've seen a panic case on gpdb 6 with stack as below,

3  markDirty (isXmin=0 '\000', tuple=0x7effe221b3c0, relation=0x0, buffer=16058) at tqual.c:105
4  SetHintBits (xid=<optimized out>, infomask=1024, rel=0x0, buffer=16058, tuple=0x7effe221b3c0) at tqual.c:199
5  HeapTupleSatisfiesMVCC (relation=0x0, htup=<optimized out>, snapshot=0x15f0dc0 <CatalogSnapshotData>, buffer=16058) at tqual.c:1200
6  0x00000000007080a8 in systable_recheck_tuple (sysscan=sysscan@entry=0x2e85940, tup=tup@entry=0x2e859e0) at genam.c:462
7  0x000000000078753b in findDependentObjects (object=0x2e856e0, flags=<optimized out>, stack=0x0, targetObjects=0x2e85b40, pendingObjects=0x2e856b0,
   depRel=0x7fff2608adc8) at dependency.c:793
8  0x00000000007883c7 in performMultipleDeletions (objects=objects@entry=0x2e856b0, behavior=DROP_RESTRICT, flags=flags@entry=0) at dependency.c:363
9  0x0000000000870b61 in RemoveRelations (drop=drop@entry=0x2e85000) at tablecmds.c:1313
10 0x0000000000a85e48 in ExecDropStmt (stmt=stmt@entry=0x2e85000, isTopLevel=isTopLevel@entry=0 '\000') at utility.c:1765
11 0x0000000000a87d03 in ProcessUtilitySlow (parsetree=parsetree@entry=0x2e85000,

The reason is that we pass a NULL relation to the visibility check code, which
might use the relation variable to determine if hint bit should be set or not.
Let's pass the correct relation variable even it might not be used finally.

I'm not able to reproduce the issue locally so I can not provide a test case
but that is surely a potential issue.
Reviewed-by: NAshwin Agrawal <aashwin@vmware.com>

When update or delete statement errors out because of the CTID is
not belong to the local segment, we should also print out the CTID
of the tuple so that it will be much easier to locate the wrong-
distributed data via:
  `select * from t where gp_segment_id = xxx and ctid='(aaa,bbb)'`.

Mainly fixed the file handler issues.

* docs - note how oss users get gpbackup

* small edit

Fixed the bug that the SIGHUP handler was installed for SIGINT by
mistake, so the ic-proxy bgworkers would die on SIGHUP.

By correcting the signal name, now we could let the ic-proxy bgworkers
reload the postgresql.conf by executing "gpstop -u".
Reviewed-by: NHubert Zhang <hzhang@pivotal.io>

Note that since PyGreSQL 5.0 this method will return the values of array
type columns as Python lists.

ref: https://pygresql.org/contents/pg/query.html

return value of strcmp is not checked in some branches.

conn = dbconn.connect() should be aligned with if statement, or it
will never be executed.

In a query that contains multiple init/sub plans, the packets of the
second subplan might be received while the first is still being
processed in the ic-proxy mode, this is because in ic-proxy mode a local
host handshake is used instead of the global one.

To distinguish the packets of different subplans, especially for the
early coming ones, we must stop handling on the BYE immediately, and
pass any unhandled early coming pkts to the successor or the
placeholder.

This fixes the random hanging during the ICW parallel group of
qp_functions_in_from.  No new test is added.
Co-authored-by: NHubert Zhang <hzhang@pivotal.io>
Co-authored-by: NNing Yu <nyu@pivotal.io>

sizeof(HeapTuple *) should be sizeof(HeapTuple)

Remove dead code.
insertDesc is alwasy NULL in ao_vacuum_rel_compact()

Remove unused varaible.
For tuplesort.h, we doesn't support mksort based cluster,
so we should just set is_mk_tuplesortstate to false

Clean identical code in heap.c and analyze.c

Also, the entry for ExtprotocolRelationid was in wrong place in
object_classes[]. It's a bit surprising that didn't cause any ill effects,
but let's fix it in any case.

It was added in commit 0138eed4, but was never used for anything.

The command execution framework shipped with a fault injection in
delivered code. See https://github.com/greenplum-db/gpdb/issues/10546
for execution details and implications.

It seems the fault injection framework was added in 2009, used
sparingly, and should be removed until it can be safely replaced.

Additionally, the "gppylib/test/regress" folder used fault injector, but
the "check-regress" target seems not to have been called - obvious
because pygresql regression checks are present, but pygresql has not
been in master for some time without causing any errors to these tests
Authored-by: NTyler Ramer <tramer@vmware.com>

Should check the len parameter of memcpy is not negative in
gp_hyperloglog.c
Should use errno instead of seekResult in cdbappendonlystorageread.c

Unlogged tables do not propagate to replica servers, skip them and their
initialization forks.

Fix some resource leak.

Adding pg_stat_clear_snapshot() in functions looping over
gp_stat_replication / pg_stat_replication to refresh result everytime
the query is run as part of same transaction. Without
pg_stat_clear_snapshot() query result is not refreshed for
pg_stat_activity neither for xx_stat_replication functions on multiple
invocations inside a transaction. So, in absence of it the tests
become flaky.

Also, tests commit_blocking_on_standby and dtx_recovery_wait_lsn were
initially committed with wrong expectations, hence were missing to
test the intended behavior. Now reflect the correct expectation.

This was missed in commit 96b332c0.

Orca's DP algorithms currently generate logical alternatives based only on cardinality; they do not take into account motions/partition selectors as these are physical properties handled later in the optimization process. Since DPv2 doesn't generate all possible alternatives for the optimization stage, we end up generating alternatives that do not support partition selection or can only place poor partition selectors.

This PR introduces partition knowledge into the DPv2 algorithm. If there is a possible partition selector, it will generate an alternative that considers it, in addition to the previous alternatives.

We introduce new properties, m_contain_PS  to indicate whether a SExpressionInfo contains a PS for a particular expression. We consider an expression to have a possible partition selector if the join expression columns and the partition table's partition key overlap. If they do, we mark this expression as containing a PS for a particular PT.

We consider a good PS one which is selective. Eg:
```
- DTS
- PS
   -TS
     - Pred
```

would be selective. However, if there is no selective predicate, we do not consider this as a promising PS.

For now, we add just a single alternative that satisfies this property and only consider linear trees.

This reverts commit ef887cfe.

This commit only affects cardinality estimation in ORCA when the user
sets `optimizer_damping_factor_join = 0`. It improves the square root
algorithm first introduced by commit ce453cf2.

In the original square root  algorithm, we assumed that distribution
column predicates would have some correlation with other predicates in
the join and therefore would be accordingly damped when calculating join
cardinality.

However, distribution columns are ideally unique in order to gain the
best performance for GPDB. Under this assumption, distribution columns
should not be correlated and thus needed to be treated as independent
when calculating join cardinality. This is a best guess since we do not
have a way to support correlated columns at this time.
Co-authored-by: NAshuka Xue <axue@vmware.com>
Co-authored-by: NChris Hajas <chajas@vmware.com>

We used to store them under /tmp/, we include the postmaster port number
in the file name in the hope that two clusters will not conflict with
each other on this file.

However the conflict still happen in the test src/bin/pg_basebackup.
And it can also happen if a second cluster is missed configured by
accident.  So to make things safe we also include the postmaster pid in
the domain socket path, there is no chance for two postmasters to share
the same pids.
Reviewed-by: NPaul Guo <pguo@pivotal.io>

This commit allows Orca to select plans that leverage IndexOnlyScan
node. A new GUC 'optimizer_enable_indexonlyscan' is used to enable or
disable this feature. Index only scan is disabled by default, until the
following issues are addressed:

  1) Implement cost comparison model for index only scans. Currently,
     cost is hard coded for testing purposes.
  2) Support index only scan using GiST and SP-GiST as allowed.
     Currently, code only supports index only scans on b-tree index.
Co-authored-by: NChris Hajas <chajas@vmware.com>

Add information about the compression parameters used when
a column is added to an AOCO table.