gpsd was failing because the connectionString that we passed to pgdb.connect
had the parameters in the wrong order. It started failing after upgrading to a
higher version of PyGreSQL. So use a dictionary instead in order to avoid
sending in the parameters incorrectly.
Co-authored-by: NAshwin Agrawal <aashwin@vmware.com>

Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NHao Wu <gfphoenix78@gmail.com>

We've seen a panic case on gpdb 6 with stack as below,

3  markDirty (isXmin=0 '\000', tuple=0x7effe221b3c0, relation=0x0, buffer=16058) at tqual.c:105
4  SetHintBits (xid=<optimized out>, infomask=1024, rel=0x0, buffer=16058, tuple=0x7effe221b3c0) at tqual.c:199
5  HeapTupleSatisfiesMVCC (relation=0x0, htup=<optimized out>, snapshot=0x15f0dc0 <CatalogSnapshotData>, buffer=16058) at tqual.c:1200
6  0x00000000007080a8 in systable_recheck_tuple (sysscan=sysscan@entry=0x2e85940, tup=tup@entry=0x2e859e0) at genam.c:462
7  0x000000000078753b in findDependentObjects (object=0x2e856e0, flags=<optimized out>, stack=0x0, targetObjects=0x2e85b40, pendingObjects=0x2e856b0,
   depRel=0x7fff2608adc8) at dependency.c:793
8  0x00000000007883c7 in performMultipleDeletions (objects=objects@entry=0x2e856b0, behavior=DROP_RESTRICT, flags=flags@entry=0) at dependency.c:363
9  0x0000000000870b61 in RemoveRelations (drop=drop@entry=0x2e85000) at tablecmds.c:1313
10 0x0000000000a85e48 in ExecDropStmt (stmt=stmt@entry=0x2e85000, isTopLevel=isTopLevel@entry=0 '\000') at utility.c:1765
11 0x0000000000a87d03 in ProcessUtilitySlow (parsetree=parsetree@entry=0x2e85000,

The reason is that we pass a NULL relation to the visibility check code, which
might use the relation variable to determine if hint bit should be set or not.
Let's pass the correct relation variable even it might not be used finally.

I'm not able to reproduce the issue locally so I can not provide a test case
but that is surely a potential issue.
Reviewed-by: NAshwin Agrawal <aashwin@vmware.com>

When update or delete statement errors out because of the CTID is
not belong to the local segment, we should also print out the CTID
of the tuple so that it will be much easier to locate the wrong-
distributed data via:
  `select * from t where gp_segment_id = xxx and ctid='(aaa,bbb)'`.

Mainly fixed the file handler issues.

* docs - note how oss users get gpbackup

* small edit

Fixed the bug that the SIGHUP handler was installed for SIGINT by
mistake, so the ic-proxy bgworkers would die on SIGHUP.

By correcting the signal name, now we could let the ic-proxy bgworkers
reload the postgresql.conf by executing "gpstop -u".
Reviewed-by: NHubert Zhang <hzhang@pivotal.io>

Note that since PyGreSQL 5.0 this method will return the values of array
type columns as Python lists.

ref: https://pygresql.org/contents/pg/query.html

return value of strcmp is not checked in some branches.

conn = dbconn.connect() should be aligned with if statement, or it
will never be executed.

In a query that contains multiple init/sub plans, the packets of the
second subplan might be received while the first is still being
processed in the ic-proxy mode, this is because in ic-proxy mode a local
host handshake is used instead of the global one.

To distinguish the packets of different subplans, especially for the
early coming ones, we must stop handling on the BYE immediately, and
pass any unhandled early coming pkts to the successor or the
placeholder.

This fixes the random hanging during the ICW parallel group of
qp_functions_in_from.  No new test is added.
Co-authored-by: NHubert Zhang <hzhang@pivotal.io>
Co-authored-by: NNing Yu <nyu@pivotal.io>

sizeof(HeapTuple *) should be sizeof(HeapTuple)

Remove dead code.
insertDesc is alwasy NULL in ao_vacuum_rel_compact()

Remove unused varaible.
For tuplesort.h, we doesn't support mksort based cluster,
so we should just set is_mk_tuplesortstate to false

Clean identical code in heap.c and analyze.c

Also, the entry for ExtprotocolRelationid was in wrong place in
object_classes[]. It's a bit surprising that didn't cause any ill effects,
but let's fix it in any case.

It was added in commit 0138eed4, but was never used for anything.

The command execution framework shipped with a fault injection in
delivered code. See https://github.com/greenplum-db/gpdb/issues/10546
for execution details and implications.

It seems the fault injection framework was added in 2009, used
sparingly, and should be removed until it can be safely replaced.

Additionally, the "gppylib/test/regress" folder used fault injector, but
the "check-regress" target seems not to have been called - obvious
because pygresql regression checks are present, but pygresql has not
been in master for some time without causing any errors to these tests
Authored-by: NTyler Ramer <tramer@vmware.com>

Should check the len parameter of memcpy is not negative in
gp_hyperloglog.c
Should use errno instead of seekResult in cdbappendonlystorageread.c

Unlogged tables do not propagate to replica servers, skip them and their
initialization forks.

Fix some resource leak.

Adding pg_stat_clear_snapshot() in functions looping over
gp_stat_replication / pg_stat_replication to refresh result everytime
the query is run as part of same transaction. Without
pg_stat_clear_snapshot() query result is not refreshed for
pg_stat_activity neither for xx_stat_replication functions on multiple
invocations inside a transaction. So, in absence of it the tests
become flaky.

Also, tests commit_blocking_on_standby and dtx_recovery_wait_lsn were
initially committed with wrong expectations, hence were missing to
test the intended behavior. Now reflect the correct expectation.

This was missed in commit 96b332c0.

Orca's DP algorithms currently generate logical alternatives based only on cardinality; they do not take into account motions/partition selectors as these are physical properties handled later in the optimization process. Since DPv2 doesn't generate all possible alternatives for the optimization stage, we end up generating alternatives that do not support partition selection or can only place poor partition selectors.

This PR introduces partition knowledge into the DPv2 algorithm. If there is a possible partition selector, it will generate an alternative that considers it, in addition to the previous alternatives.

We introduce new properties, m_contain_PS  to indicate whether a SExpressionInfo contains a PS for a particular expression. We consider an expression to have a possible partition selector if the join expression columns and the partition table's partition key overlap. If they do, we mark this expression as containing a PS for a particular PT.

We consider a good PS one which is selective. Eg:
```
- DTS
- PS
   -TS
     - Pred
```

would be selective. However, if there is no selective predicate, we do not consider this as a promising PS.

For now, we add just a single alternative that satisfies this property and only consider linear trees.

This reverts commit ef887cfe.

This commit only affects cardinality estimation in ORCA when the user
sets `optimizer_damping_factor_join = 0`. It improves the square root
algorithm first introduced by commit ce453cf2.

In the original square root  algorithm, we assumed that distribution
column predicates would have some correlation with other predicates in
the join and therefore would be accordingly damped when calculating join
cardinality.

However, distribution columns are ideally unique in order to gain the
best performance for GPDB. Under this assumption, distribution columns
should not be correlated and thus needed to be treated as independent
when calculating join cardinality. This is a best guess since we do not
have a way to support correlated columns at this time.
Co-authored-by: NAshuka Xue <axue@vmware.com>
Co-authored-by: NChris Hajas <chajas@vmware.com>

We used to store them under /tmp/, we include the postmaster port number
in the file name in the hope that two clusters will not conflict with
each other on this file.

However the conflict still happen in the test src/bin/pg_basebackup.
And it can also happen if a second cluster is missed configured by
accident.  So to make things safe we also include the postmaster pid in
the domain socket path, there is no chance for two postmasters to share
the same pids.
Reviewed-by: NPaul Guo <pguo@pivotal.io>

This commit allows Orca to select plans that leverage IndexOnlyScan
node. A new GUC 'optimizer_enable_indexonlyscan' is used to enable or
disable this feature. Index only scan is disabled by default, until the
following issues are addressed:

  1) Implement cost comparison model for index only scans. Currently,
     cost is hard coded for testing purposes.
  2) Support index only scan using GiST and SP-GiST as allowed.
     Currently, code only supports index only scans on b-tree index.
Co-authored-by: NChris Hajas <chajas@vmware.com>

Add information about the compression parameters used when
a column is added to an AOCO table.

The libuv source is in the build container, rather thank in a gcp
bucket. This commit is a partial revert from getting source from a
gcp bucket and instead uses the libuv available in the build
container image.
Co-authored-by: NTyler Ramer <tramer@vmware.com>
Co-authored-by: NKris Macoskey <kmacoskey@vmware.com>
Co-authored-by: NJamie McAtamney <jmcatamney@vmware.com>

Here is the diff output of the test result.

 drop database some_database_without_tablespace;
 -DROP
 +ERROR:  database "some_database_without_tablespace" is being accessed by other users
 +DETAIL:  There is 1 other session using the database.
 drop tablespace some_basebackup_tablespace;
 -DROP
 +ERROR:  tablespace "some_basebackup_tablespace" is not empty

The reason is that after client connection to the database exits, the server
needs some time (the process might be scheduled out soon, and the operation
needs to content for the ProcArrayLock lock) to release the PGPROC in
proc_exit()->ProcArrayRemove(). During dropdb() (for database drop), postgres
will call CountOtherDBBackends() to see if there are still sessions that are
using the database by checking proc->databaseId, and it will try at most 5 sec.
This test quits the db connection of some_database_without_tablespace and then
drops the database immediately. This should be mostly fine but if the system is
in slow or in heavy load, this still could lead to test flakiness.

This issue could be simulated using gdb. Let's poll until database drop
commands succeeds for the affected database.  It seems that drop database sql
command could not be in transaction block so I could not use plpgsql to
implement, instead I use dropdb utility and bash command to implement that.
Reviewed-by: NAsim R P <pasim@vmware.com>

None of the mock tests in the backend call into GPORCA functions, so
no need to generate mock objects for them. Saves some time and space when
running mock tests.
Reviewed-by: NAshuka Xue <axue@pivotal.io>

Co-authored-by: NTyler Ramer <tramer@vmware.com>
Co-authored-by: NJamie McAtamney <jmcatamney@vmware.com>