Extracting column names from ShareInputScan in ORCA plans to support proper column name resolution using RTE_CTE (#992)

* Extracting column names from ShareInputScan in ORCA plans to support proper column name resolution using RTE_CTE.

* Code review on PR 992.

* PlannerGlobal has out-function support in the upstream, so removing it
  altogether doesn't seem like a good idea. I'm not sure if it get
  printed out with suitable verbose or debug flags, but I remember seeing
  it being printed out during debugging somehow, and it can be useful.

* Refactor the functions in cdbmutate.c, so that there's a separate function
  to do the DAG to Tree conversion, and a separate function for just
  collecting the producer nodes. It seems like a bad idea that a function
  called "apply_dag_to_tree" actually does something different, depending
  on a flag in a struct.

* Now that we have a separate array of producers, no need to hold the
  colnames etc. lists in ShareInputScan node itself. Since we can look up
  the producer node at will, we might as well look at the producer node's
  sub-tree directly every time we construct the CTE RTE.

* One complication from the previous change is that we can't call
  get_tle_name() in replace_shareinput_targetlists(), because that runs
  after the post-processing in setrefs.c, so all Vars have already been
  changed to use INNER/OUTER. get_tle_name() doesn't work with those.
  On closer inspection, I think this was a bit fiddly in the ORCA case
  before too, because in ORCA-generated plans, Vars always use the
  INNER/OUTER notation, so calling get_tle_name() on an ORCA-generated
  plan was always questionable. It happened to work, becuase ORCA also
  makes seems to always fill in TargetEntry.resname, so get_tle_name()
  always just picked that, rather than looking up the range table entry.
  This new structuring of the code avoids relying on that assumption.

* Refactored the code to create the fake CTE RTE to a separate function.
  replace_shareinput_targetlists_walker() had grown quite complex.

* Use the producers array in setrefs.c

* Get rid of separate sharedNodes list.

Now that we have an array of producers, conveniently indexed by share_id,
just use that. Mostly for sake of readability, although you might see
a performance gain in corner-cases involving a huge number of share input
scans.

Error was introduced by PG 8.3 merge. In Postgres there is only one
log, and its name pattern is set by global variable Log_filename.
But in GPDB there is a separate alert log used by gpperfmon, and its
file name is also generated by logfile_getname() function, and you
have to use the passed pattern instead of global variable as a file
name pattern, as alert filename pattern is "gpdb-alert-..."

* Fixing DXL Translator bug where we lose canSetTag during Query object mutation and the translator ends up using wrong canSetTag.

* Adding ICG test for verifying the the ORCA translator uses correct canSetTag.

We can avoid generating multiple versions of the slot_getattr. Once we
deform any of the attributes in the tuples, we make it a virtual tuple.
At code generation time, we know exactly how many need to be deformed
and can in fact go ahead deform all the way. This way we don't need to
worry about the case when slot_getattr is called on a virtual tuple with
attnum > nvalid - that is deformation is partially complete.

To enable this, we need to collect information from all the code
generators that depend on SlotGetAttrCodegen before it is generated. We
maintain a static map (keyed on the manager and the slot) to instances of
SlotGetAttrCodegen. We also introduce a InitDependencies phase that
happens before the GenerateCode phase, when dependants of
SlotGetAttrCodegen can retrieve instances from the static map.

This reverts commit 41857ff3.

Regression test find a deadlock issue, the test is as follow:

BEGIN;
CREATE TABLE dtm_plpg_foo (C_CUSTKEY INTEGER, C_NAME VARCHAR(25), C_ADDRESS VARCHAR(40))
partition by range (c_custkey) (partition p1 start(0) end(100000) every(1000));
INSERT INTO dtm_plpg_foo SELECT * FROM dtm_plpg_foo LIMIT 10000;
COMMIT;

The create statement leaked a ROW EXCLUSIVE lock on pg_class. If some other session request
and wait on ACCESS EXCLUSIVE lock before the insert statement, the insert statement will not
be able to get the ACCESS SHARE lock. So the entryDB reader gang will wait the lock holding
by QD process, while the QD process will wait the results from primary reader gangs.

This patch adds support for multi-dimensional arrays as both input
and output parameters for PL/Python functions. The number of dimensions
is limited by Postgres MAXDIM macrovariable, by default equal to 6.
Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d
arrays representing MxNxK cube, etc.
Patch includes regression tests for both correct multi-dimensional
array use cases and errorneous ones.

There was some code left over from the 8.3 merge, that prematurely reset
ActiveSnapshot. Remove the extraneous code.

Fixes github issue #1001. Thank you @clmyyclm for the report!

be mocked. Previously, only the last parameter of the mocked function can be
assigned a value by unit test.

into a separate function, and change the DispatchCommandQueryParms allocation to
palloc.

Commits a4fbb150 and 2818b911 removed the AppendOnlyEntry struct, and added
the same information into the relcache. I didn't notice there were cmocka
tests also using the AppendOnlyEntry struct.

There are quite a few places in the planner that need that information.
Save a few system table lookups by keeping it in RelOptInfo. There is
no syscache for pg_exttable. I don't see any measurable performance
improvement from this, but you can see the number of index scans drop in
pg_stat_sys_tables, and I think it makes the code slightly simpler anyway.

This shaves a couple of minutes off the total runtime on my laptop.

Keeping upstream test files as close to upstream as possible helps with
diffing and merging with upstream. There are more GPDB-specific changes
there still, but this is a step in the right direction. The 'alter_table'
test runs for quite a long time, so by running it more in parallel, this
also shortens a installcheck-good run slightly.

The test runs in a schema of its own, bfv_aggregate. It is created at
the beginning of the test, so we can safely assume that it's empty.
Also, rather than dropping created objects after each test, just drop
the whole schema at the end.

In the passing, also add a DROP TABLE for the boolean test table. It
was missing. Not that we necessarily need to drop every table created
in a test case, but let's be consistent within the test file.

Vacuuming the single table ought to be enough.

It's pretty much identical to Form_pg_appendonly, which is now available
in the relcache. No need to pass this additional struct around.

This way, you don't need to always fetch it from the system catalogs,
which makes things simpler, and is marginally faster too.

To make all the fields in pg_appendonly accessible by direct access to
the Form_pg_appendonly struct, change 'compresstype' field from text to
name. "No compression" is now represented by an empty string, rather than
NULL. I hope there are no applications out there that will get confused
by this.

The GetAppendOnlyEntry() function used to take a Snapshot as argument,
but that seems unnecessary. The data in pg_appendonly doesn't change for
a table after it's been created. Except when it's ALTERed, or rewritten
by TRUNCATE or CLUSTER, but those operations invalidate the relcache, and
we're never interested in the old version.

There's not much need for the AppendOnlyEntry struct and the
GetAppendOnlyEntry() function anymore; you can just as easily just access
the Form_pg_appendonly struct directly. I'll remove that as a separate
commit, though, to keep this one more readable.

Many headers in src/include/catalog have gotten this treatment in the
upstream. This makes life easier for the next commit, which adds a
reference to Form_pg_appendonly to RelationData. With this separation,
we can avoid pulling in a lot of other headers into rel.h, and avoid a
circular dependency,

In the previous coding, if a "name" field in a struct, that was used for
direct access to a catalog tuple, i.e. Form_pg_*, was not accidentally
aligned to 4-byte boundary, the struct would not match what's stored on
disk. I was bit by this, when I tried changing the compresstype field
in pg_appendonly from text to name.

This seems like a good idea from an upgrade point of view too. This change
broke pg_upgrade for any tables that use the name datatype. That's not
a big deal, because it shouldn't be used in user tables at all anyway, but
might as well take the hit in 5.0 when the upgrade is expected to be more
painful anyway.

Original commit:

commit 5f6f840e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Tue Jun 24 17:58:27 2008 +0000

    Reduce the alignment requirement of type "name" from int to char, and arrange
    to suppress zero-padding of "name" entries in indexes.

    The alignment change is unlikely to save any space, but it is really needed
    anyway to make the world safe for our widespread practice of passing plain
    old C strings to functions that are declared as taking Name.  In the previous
    coding, the C compiler was entitled to assume that a Name pointer was
    word-aligned; but we were failing to guarantee that.  I think the reason
    we'd not seen failures is that usually the only thing that gets done with
    such a pointer is strcmp(), which is hard to optimize in a way that exploits
    word-alignment.  Still, some enterprising compiler guy will probably think
    of a way eventually, or we might change our code in a way that exposes
    more-obvious optimization opportunities.

    The padding change is accomplished in one-liner fashion by declaring the
    "name" index opclasses to use storage type "cstring" in pg_opclass.h.
    Normally btree and hash don't allow a nondefault storage type, because they
    don't have any provisions for converting the input datum to another type.
    However, because name and cstring are effectively the same thing except for
    padding, no conversion is needed --- we only need index_form_tuple() to treat
    the datum as being cstring not name, and this is sufficient.  This seems to
    make for about a one-third reduction in the typical sizes of system catalog
    indexes that involve "name" columns, of which we have many.

    These two changes are only weakly related, but the alignment change makes
    me feel safer that the padding change won't introduce problems, so I'm
    committing them together.

They were not used for anything. Look at segrelid and segidxid in
pg_appendonly instead.

Spotted while debugging a buggy patch of mine, that the relation names
in the ReadBuffer() error messages currently come out as "(null)". Would
segfault on with a different libc implementation.

Running gpcheckcat after constraints installcheck test shows trigger metadata
inconsistencies between the master and segments. This issue was introduced
in the 8.3 merge. If CreateTrigger Statement has a trigger oid already, we
should reuse it like before the 8.3 merge.

This patch series adds the JSON datatype, with all the fixes that went
into PostgreSQL 9.2 after the initial commit. See individual commits for
more details.

Author: Álvaro Hernández Tortosa <aht@8kdata.com>
Reviewed-by: Daniel Gustafsson
Reviewed-by: Andreas Scherbaum

Note: We don't alien node elimination when there is an explain codegen
or explain analyze codegen query in master.

Signed-off-by: NShreedhar Hardikar <shardikar@gmail.com>