Previously, gpdbrestore would analyze all schemas in the database during
a schema-only restore. Now, only those schemas that were restored will
be analyzed.
Signed-off-by: NChris Hajas <chajas@pivotal.io>

With commit 387c485d winstar and winagg
fields were added in WindowRef Node, so this commit adds handling for
them in ORCA Translator.

I don't know what this was used for, but I don't see any code that would
send these messages anymore.

nMotionNodes tracks the number of Motion in a plan, and each
plan node maintains nMotionNodes. Counting number of Motions in a plan node by
traversing the tree and adding up nMotionNodes found in nested plans will give
incorrect number of Motion nodes. So instead of using nMotionNodes, use
a boolean flag to track if the subtree tree excluding the initplans
contains a motion node

Solves https://github.com/greenplum-db/gpdb/issues/3211

There are times where the flags/tags are not passed in the pipeline
configuration but the jobs shows as green anwyays (false positive).
This is due to the return code being 0 event though we don't do
anything.

Force an error when we don't specify flags/tags when running behave
with the makefile.

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NShoaib Lari <slari@pivotal.io>

gpcheckcloud will append new line to files who miss it, so that the
ending line won't concatenated with the first line of next file.

Add an option gpcheckcloud_newline to support configuring new line, will
report error with other new line than LF, CRLF or CR.
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

In commit d16710ca, I added an optimization to NOT IN subqueries, to
remove any DISTINCT and ORDER BY. But on second thoughts, we should use the
existing functions to do that. Also, the add_notin_subquery_rte() was not
a good place to do it. It's a surprising side-effect for that function.
Move the code to convert_IN_to_antijoin(), which is in line with the
similar calls in convert_EXPR_to_join() and convert_IN_to_join().

pg_config.h.in is generated by autoheader, but it had fallen out of date.
Has everyone just been adding stuff manually to it, or how did this happen?
In any case, let's run it now.

It occurred to me while looking at PR #1460 that when there's a DISTINCT or
an ORDER BY in a subselect NOT IN (...) subselect won't make any difference
to the overall result, so we can strip it off and safe the effort. In its
current form, PR #1460 would pessimize that case slightly more, by forcing
the subselect's resul to be gathered to a singled node for deduplication or
final ordering, while before, we would do only a local ordering /
deduplication on each segment. But it is a waste of effort to do that
even within each segment, and this PR gets rid of that.

Remove the assertion that WindowRef.winagg is set correctly in the
executor, because it won't be if the plan was generated by ORCA.

Add required steps before building GPDB on centos.
Fixes issue #2715, #2995 and #3163.

[ci skip]

Originally, if the REORGANIZE option is used, and the source and target
tables got same partition distribution policy, then the redistribute is
skipped. This is due to the nature of planner optimizer will skip
reshuffle if the source and target distribution policy match.

However, if both source and target distribution policies are random,
then planner will generate a redistribution motion to balance the tuples
across the cluster.

Leveraging that thought, we added new code path to temporarily set the
source table's distribution policy to random while executing the CTAS
query, and hence the optimizer can generate the proper query plan with
`redistribute motion`. We restore the policy after creating the
temporary table.

Test cases are added to show case the scenario where a table is loaded
with data inconsistent with its distribution policy when using
COPY ... ON SEGMENT. The second case catches a regression when using
SET WITH (REORGANIZE = TRUE) DISTRIBUTED RANDOMLY.
Signed-off-by: NJacob Champion <pchampion@pivotal.io>

* reorganizing / promoting external tables topic

* correcting graphics locations

* more reorg and consolidation of topics

* promoting shortdescs, consolidating web table topics

* removing duplicate, manual chapter toc

* promoting more shortdesc's, removing more manual tocs

* changing title of hdfs section

* promoting gphdfs section, removing oveverview section that mentions pxf

* adding shortdesc to gphdfs topic

This adds the 'winstar' field from the upstream. Also bring in the 'winagg'
field while we're at it, although it's only used for an assertion in
nodeWindow.c so far.

FaultInjector_UpdateHashEntry() was using FaultInjector_InsertHashEntry(), which
ends-up adding entry if not present without incrementing
`faultInjectorShmem->faultInjectorSlots`. This causes inconsistency, plus also
sometimes encounters "FailedAssertion(""!(faultInjectorShmem->faultInjectorSlots
== 0)""," during fault inject reset, as goes negative. Fixing the same by using
FaultInjector_LookupHashEntry() instead as that's what
FaultInjector_UpdateHashEntry() needs.

Scenario the Assertion was hitting:
gpfaultinjector -f all -m async -y resume -r primary -H ALL
gpfaultinjector -f all -m async -y reset -r primary -H ALL

This hack, to refrain from removing a trivial SubqueryScan if the subnode
was a Window node, was added back in 2009 along with the test case. I'm
not sure what the problem was, but this must've been just a quick band-aid
over whatever the real problem was. It doesn't seem to be needed anymore,
as everything works without it.

Expand the comment in the test case a little bit to explain what the point
of the query is. The original commit message said just "Fix MPP-4840", so
this is all the information I could find about it, unfortunately.

The README was still referencing gp-xerces even though it's no
longer required for ORCA, so this was removed. On top of that,
some Markdown formatting fixes, minor wordsmithing, typo fixes
as well as some whitespace cleanup and turned URLs into links.

Use the available macro for detecting FTP protocol and use pstrdup
instead of palloc+strncpy. Also fix a spacing issue with an ereport
call.

Mostly, move the responsibilities of the check_call() function to the
callers, transformAggregateCall() and transformWindowFuncCall().

This fixes one long-standing, albeit harmless, bug. Previously, you got an
"Unexpected internal error", if you tried to use a window function in the
WHERE clause of a DELETE statement, instead of a user-friendly syntax
error. Add a test case for that.

Move a few similar tests from 'olap_window_seq' to 'qp_olap_windowerr'.
Seems like a more appropriate place for them. Also, 'olap_window_seq' has
an alternative expected output file for ORCA, so it's nice to keep tests
that produce the same output with or without ORCA out of there. Also add a
test query for creating an index on an expression containing a window
function. There was a test for that already, but it was missing parens
around the expression, and therefore produced an error already in the
grammar.

This reverts commit f71e57c6, better to
keep the behavior.

The other one is <SEG_DATA_DIR>, they should keep the same style.

In GpMirrorListToBuild.buildMirrors() if there are no mirrors to
build, then return True, so that gprecoverseg returns a return code
of 0.
Signed-off-by: NShoaib Lari <slari@pivotal.io>
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

Deleting unused function CreateCurlHandler and make some other functions static
to avoid warnings.

With commit cedd89bf "Simplify tuple
serialization in Motion nodes.", the usage for this function was removed.

These functions as inline functions were producing warnings, based on discussion
https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/6fgKvN9QpV4/zjysjqIZAgAJ
converting them to macro as upstream. Adding explicit type casting wherever
needed now that DatumGetPointer() returns (char *) instead of (void *).

Compiling hstore was generating the warning for using deprecated function
heap_formtuple(). Fixing to use the newer one heap_form_tuple().

Back-patch commit 4e182361
(another thing that longfin's version of clang doesn't like).

This avoids a warning from clang 3.2 about an uninitialized variable
'dtype' in date_in().