Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

* Update HA documentation

Mirrors must reside on different storage systems too.
Master fail-over must be triggered externally.
10-Gb Ethernet is optional.

Under former TCP interconnect, after declaring a cursor for an invalid
query like "declare c1 cursor for select c1/0 from foo", the following FETCH
command can still fetch an empty row instead of an error. This is
incorrect and does not consist with UDP interconnect.

The RCA is senders of TCP interconnect always send an EOF message to
their peers regardless of the errors on segments, so the receivers can
not tell the difference of EOF or an error.

The solution in this commit is do not send the EOF to the peers if senders is
encountering an error and let the QD to check the whole segments status when it
can not read data from interconnect for a long time.

This commit restore TCP interconnect and fix some hang issues.

* restore TCP interconnect code
* Add GUC called gp_interconnect_tcp_listener_backlog for tcp to control the backlog param of listen call
* use memmove instead of memcpy because the memory areas do overlap.
* call checkForCancelFromQD() for TCP interconnect if there are no data for a while, this can avoid QD from getting stuck.
* revert cancelUnfinished related modification in 8d251945, otherwise some queries will get stuck
* move and rename faultinjector "cursor_qe_reader_after_snapshot" to make test cases pass under TCP interconnect.

* Change the default level of gp_log_gang to off.
* Log the query plan size in level TERSE, it's useful for debugging.

It is insecurity to run priviledged containers on public pipelines, so
disable resource group jobs until a new priviledged VM mechanism can be
provided.

Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>

Adding tests to show that minimal functionality of gpcheck works.

gpcheck checks for specific system requirements. As such, we had to add
system configuration changes to control files at test runtime prior to
behave.

We explored running some tests as root to more realistically test how
gpcheck is used, but don't have that at this time.

Also adding back gpcheck_hostdump which is called by
gpcheck.
Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>

Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>

- had been failing for lack of a help string -- can come back and
  provide more detail if people are interested
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

This test class was named the same, "GpTestCase", as the name of the
superclass we use for all tests. Since the packaging was different, this
was not a python conflict, but it was odd and misleading. This class can
be named anything, so it seems easy to change it.

Also modernized the superclass, optimized imports, and added a main()
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

The compilation job for the binary swap test portion of the pipeline
used the same remote as our normal compilation job. If someone forked
the pipeline, they may encounter issues if that tag or branch is
missing or not updated in their forked remote of Greenplum. Adding an
additional separate remote variable fixes this.

Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>

- Remove iteration specific members of qexec packet
- Remove iterators_history table
- Remove measures used to populate iterators_history
- Remove iterator_aggregate flag
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>

This option is now the default behavior since we are removing code that
attempted to profile query executor ("iterator") processes.
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

This will speed-up the isolation2 sql tests without waiting for uao generation.
This is useful for iterating a test that does not involve uao templates.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

* document pl/perl extension

* fix typo

* add note about modules

* incorporate david s requested edits

This is needed to test that Greenplum HEAD is binary compatible with a
previous version of Greenplum. This is controlled by providing a
branch/tag to the gpdb_src_binary_swap git resource. We run the tests
in the same job where we run ICW to reuse the regression database
generated. To start this off, we'll be testing that HEAD can binary
swap over 5.0.0-alpha.5.

We need some tests to see if binaries can be swapped around. This
commit provides a simple framework and basic tests to check if
binaries can be upgraded/downgraded by just swapping the Greenplum
binary install paths. Running the tests requires the user to have
another Greenplum compiled which will be provided as an argument to
the test script. It also requires the regression database to be built
from running tests in src/test/regress.

Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>

Define a xlog format for AO operations. The xlog is generated when an AO block
is written.

A test is added to receive the AO log from the WAL Sender process after an
INSERT operation and verify that the the received xlog has the AO xlog record in
it.
Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>

* updating and conditionalizing PL/Java for gppackage install

* cleaning up CREATE LANGUAGE description of PL/Java

* cleanup PL/Java language in createlang

* including PL/Java, PL/R as gppkg-installed packages

* updating PL/R topic for latest package and version; conditionalizing gppkg install procedures

* remove duplicate sqlj.install_jar entry

Commit 5a2d9a5a pulled in free_readfile() from upstream but one
invocation was erroneously placed too early in the logic. Move down
to after all uses of optlines are done. With that, also allocate
memory for data pulled out of the config file, partially backporting
dd917bb7 from upstream which does the same thing.

At some point in the past it seems the keyword list was merged from
upstream 8.4 without merging the actual code, and instead keywords
which weren't yet supported where commented out. The XML whitespace
keywords for STRIP WHITESPACE and PRESERVE WHITESPACE were however
not uncommented when XML support was merged, resulting in the below
error when trying to restore the XML views in ICW back after a dump:

db=# SELECT XMLPARSE(CONTENT '<abc>x</abc>'::text PRESERVE WHITESPACE) AS "xmlparse";
ERROR:  syntax error at or near "WHITESPACE"
LINE 1: ...CT XMLPARSE(CONTENT '<abc>x</abc>'::text PRESERVE WHITESPACE...
                                                             ^
Put the keywords back, and also remove commented out keywords which
we dont support but will get when merging 8.4 to reduce the diff with
upstream. Also add a testcase for XML whitespace syntax parsing.

The aosegments table is in fact using the oid of the relation and
not the relfilenode. Also add a short note about the mapping in
pg_appendonly and reflow the section with the new content.

Record memory usage for resource group.

1. Update total memory usage for a resource group when
   a session belonging to this group allocates/frees memory.

2. Update total memory usage for related resource groups
   when a session enters into or leaves from a resource group.

3. Dispatch current resource group ID from QD to QEs to
   keep track of current resource group.

4. Show total memory usage of a resource group.

5. Add test case for memory usage recording of resource group.
Signed-off-by: Nxiong-gang <gxiong@pivotal.io>
Signed-off-by: NKenan Yao <kyao@pivotal.io>
Signed-off-by: NNing Yu <nyu@pivotal.io>

Because the g++ doesn't meet gpcloud's requirements.

This is a quick fix, we better upgrade GCC to building env's version.

g++ in travis doesn't support C++11 yet, which is needed by gpcloud.

This commit does things below to make developers' life, who are still
using old version C++ compilers, easier.

Add a flag to optionally disable gpcloud compilation.
Add a configure check for C++11 if gpcloud is enabled.
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>

Static variables when used inside function are initialized only once and stored
on static storage area. The original code without static initializes these
variables every time the function is called and these variables are stored in
stack. Since we don't change the array value, they can be static const.

There were tables that was planned on being worked on, however, it never
happened. As such, this is dead code and just adds lingering tables in
the user's perspective when running gpperfmon_install.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>