This closes #836
Signed-off-by: NMarc Spehlmann <cramja@gmail.com>

reorder header file

Currently gp_connections_per_thread=0 means using main thread to
do dispatch instead of creating new threads. It's not working if
segments report error before sending back any data.
For example:
create table foo(id int);
insert into foo values(0);
set gp_connections_per_thread=0;
select 100/id from foo;

We disable it for now and will reenable it when we have the new
implementation of non-threaded dispatcher.

This seems like a completely unnecessary change. Reduce the diff vs.
upstream.

This GUC will be used to control the MEMO size as well as optimization
time for large IN list or large array comparison expressions.

Only the Array with less number of elements than the GUC will be
expanded and participate in constraint derivation.

Trade-off of using this GUC is loss of potential benefits from the
constraint derivation (e.g. conflict detection, partition elimination)
with shorter optimization time and less memory utilization.

The checks for invalid TIDs are very cheap, a few CPU instructions. It's
better to catch bugs involving invalid TID early, so let's always check
for them.

The LOGs in storeAttrDefault() that were also tied to this GUC seemed
oddly specific. They were probably added long time ago to hunt for some
particular bug, and don't seem generally useful, so I just removed them.

Previously, previous free TID validation was done under the GUC
persistent_integrity_checks. This commit extracts the previous free TID
validation into another GUC validate_previous_free_tid and is enabled by
default. If the validation detects a corruption in the free TID list, we
will now switch to a new free TID list and leave the corrupted one detached
for cleanup during persistent table rebuild or during crash recovery.

Authors: Jimmy Yih and Abhijit Subramanya

Validation 1: look up new tuple's heap tid in unique index before insert.

The tuple being inserted is already inserted in heap.  Before its entry is
added to a unique index, we want to see if the index already has an entry with
this heap tid.  This should catch duplicate entries created in index but not in
the heap relation.  The validation is enabled by GUC "gp_indexcheck_insert".

Validation 2: index should point to all visible tuples after vacuum.

For each entry in index after it was vacuumed, fetch the heap tuple
and validate that it is visible.  For specific tables, validate that
the key is the same.  The validation is controlled by GUC
"gp_indexcheck_vacuum".

Closes #673.

This commit  generates code for code path: ExecVariableList > slot_getattr > _slot_getsomeattrs > slot_deform_tuple. This code path is executed during scan in simple select queries that do not have a where clause (e.g., select bar from foo;).

For each attribute A in the target list, regular implementation of ExecvariableList retrieves the slot that A comes from and calls slot_getattr. slot_get_attr() will eventually call slot_deform_tuple (through _slot_getsomeattrs), which fetches all yet unread attributes of the slot until the given attribute.

This commit generates the code for the case that all the attributes in target list use the same slot (created during a scan i.e, ecxt_scantuple). Moreover, instead of looping over the target list one at a time, it uses slot_getattr only once, with the largest attribute index from the target list.

If during code generation time, the completion is not possible (e.g., attributes use different slots), then function returns false and codegen manager will be responsible to manage the clean up.

This implementation does not support:
* Null attributes
* Variable length attributes
* Fixed length attributes passed by reference (e.g., uuid)
If at execution time, we see any of the above types of attributes, we fall back to the regular function.

Moreover, this commit:
* renames existing "codegen" guc, which is used for initiating llvm libraries, to "init_codegen" (Note: we set this guc in gpconfig),
* adds guc "codegen", which enables code generation and compilation at query execution time,
* enhances the existing code generation utilities by adding a function that creates all IR instructions for falling back to regular functions, and
* removes the existing code that generates the code of a naive slot_deform_tuple, which simply falls back to regular slot_deform_tuple.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

tuple.
This closes #649

Squashed commit of the following:

commit 96f415bc1744533e96a74bcc85016ff5df65d06e
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Thu Apr 21 15:08:49 2016 -0700

    Don't call prepare for execution if no generator is registered

commit 5318e9214a3bbc237c0d79017cdef02538c4e711
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Thu Apr 21 11:35:22 2016 -0700

    Adding if codegen protection in starting active manager and while
    deleting manager

commit 12b0045d51a5b056b32a6cc9ef370e6d9af315fd
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Fri Apr 15 14:33:55 2016 -0700

    Updating Readme for Codegen GPDB Function

commit d17119c5d8ce9a706b3321d5e7ad41f419fddbdc
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 13:25:14 2016 -0700

    Add documentation for how to codegen in README.md

commit d711be4bb044592e7f97bf31ceb5f32337ad6d03
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 11:19:03 2016 -0700

    Fixing doxygen namespace addtogroup

commit e34131bcc9bb3d36c5e8995e1a0efe11fbd55417
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 11:17:32 2016 -0700

    Changing CodeGen to Codegen

commit bedfcca80b5f1ff796783ca7c2035d920ae86c4e
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 10:51:39 2016 -0700

    Fix cpplint error

commit 32f89feedba655e1242529c78a54faf01ddf32bf
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 10:48:49 2016 -0700

    Change regular version of generated to regular version of target in comment

commit d1da2824c08e34eb7139b08cd9d2aff19bf008e7
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Wed Apr 13 10:42:09 2016 -0700

    Made GetFunctionPointer and CreateFunction to take typedef of function type as only public api

commit 9f48f77bd9fc50c7d29a4e8f7700452ef88dc79f
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 17:18:24 2016 -0700

    Rename doCodegeneration to GenerateCodeInternal

commit 9884d07f4c6ab08d9ccca8d07b80974081cda1fa
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 17:15:55 2016 -0700

    fix cpplint error

commit c21b3bee3e289675fc13017688269b40e26efa11
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 17:08:16 2016 -0700

    Update comments

commit f016bb582af65e84def1b55917017008e292d45b
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 17:01:19 2016 -0700

    Adding assert in static setToRegular function

commit 93a791a4ec415313933dd5b7c1ae17dbdd6193a6
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 16:49:15 2016 -0700

    Removing decltype in Slotdeformtype codegen since we know the functype

commit ed95193d0991cb24fd68f6d9a473de630f297b0c
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 16:45:49 2016 -0700

    Fixed compiler / unit test failure
    Changed InitCodeGen to InitCodegen

commit 3c95c9efdd4336e100a86748b0e87d79f07bcc11
Author: Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Date:   Tue Apr 12 16:28:24 2016 -0700

    Changing CodeGen back to Codegen while cherry-picking

commit 4e24ef22bcf54edc00c2a0d1f9f9703364cdf5ba
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Tue Apr 12 11:08:42 2016 -0700

    Changing auto to actual type

commit 1d92fc190dda3ec866edc83d56d9eca9fe90a63a
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Tue Apr 12 10:44:49 2016 -0700

    fix indent in codegen_utils

commit ea6fa787a2332f94b8d20d24825ba1f0a240d02e
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 17:25:53 2016 -0700

    Adding codegen guc to wrapper function so related function not executed when codegen guc is disabled

commit f1cd7d80d8d81f3565f3ab12cb94a3c4cc58c44a
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 17:13:24 2016 -0700

    Changing bool to unsigned int in codegen_wrapper.h

commit 7ef462540a312d077b332c732c125c55dedb6b14
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 16:59:08 2016 -0700

    Fix based on lint complain

commit ddc55cec0d579a03cb9e2303efa6d74f5183979f
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 16:19:48 2016 -0700

    Removing commented code from unittest

commit 28b8886ed2bfdc854aa38f871a41536c1e6cef29
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 16:17:45 2016 -0700

    Chaging auto in for to actual type

commit ad8d78e07e9b789f4d8ae3c6ccc3cea1a05089c0
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 16:13:21 2016 -0700

    * Handle return value of SetToGenerated from PrepareForExecution
    * Change size_t to unsigned integer

commit d46ee8be414b3ac2a9ea7ef899f29cef3df145c7
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 15:30:40 2016 -0700

    Introducing template function to do enroll for any type of generator

commit 83428605f906e28d0fc994dcc311e0cc3e6d9e12
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 14:41:11 2016 -0700

    Changing manager to take module name as argument

commit b88aa40f89860e7c15503365ea304d2392f71cda
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 14:27:31 2016 -0700

    adding more comments to SlotDeformTupleCodeGenInfo and unique counter

commit df1b704548fcbc1b43524064eb2f024995eea865
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 14:08:34 2016 -0700

    Change test target to have ext .t and adding gitignore for that extension

commit 9ee3521ca687c4ea7f13d15893f47146ecdb5277
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 13:59:57 2016 -0700

    Add target_include_directories for test library header file

commit effb7dc165b4c7094731dc39e0872741b28b2e6a
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Mon Apr 11 13:36:50 2016 -0700

    Move all possible ifdef from gpdb to codegen_wrapper based on pull request comments

commit 989b96a91c1e843577b5f24623d1e4a6b810394c
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Fri Apr 8 10:43:17 2016 -0700

    fixing comments for GetOrigFuncName in codegen_interface

commit 665de065c34ea13abcec8c5f3a3336558dae501f
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Fri Apr 8 10:27:31 2016 -0700

    Disabling codegen guc by default

commit ec34513e86dea95bd36ee528724273c1c19e7062
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Thu Apr 7 16:28:43 2016 -0700

    Fixed comments space/tab so it aligned in different editor

commit 35a60fe2fd053a5babd1820302afad95c5c2635f
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Thu Apr 7 16:12:08 2016 -0700

    Fixing coding style based on cpplint

commit 9477dcf6c0a7ffcca77b9f2c62c7340d13f474aa
Author: Karthikeyan Jambu Rajaraman <krajaraman@pivotal.io>
Date:   Thu Apr 7 13:37:01 2016 -0700

    Adding USE_CODEGEN ifdef

commit 142ffc1416341a3ac1d0fe37a83577168f2ec166
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Tue Apr 5 22:21:19 2016 +0000

    Make codegen work with GPDB unit test framework

commit 9dca9507dd933a5517bee3ca34615c09c23c1728
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Mon Apr 4 22:10:21 2016 +0000

    Add gpcodegen_mock.c

commit 41d515ff0342d3df3b0853ad91731f6d1d75843c
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Mon Apr 4 21:46:34 2016 +0000

    Remove unneccessary header file includes in codegen

commit 14c5e14b9834197693b1ef56efa96386cfe049c5
Author: Foyzur Rahman <frahman@gmail.com>
Date:   Tue Apr 12 14:49:51 2016 -0700

    Integrate codegen framework and make simple external call to slot deform tuple
Signed-off-by: NKarthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Signed-off-by: NFoyzur Rahman <frahman@gmail.com>

`set optimizer_join_arity_for_associativity_commutativity=7` will hint
at the optimizer to stop exploring join associativity and join
commutativity transformations when an n-ary join operator has more than
7 children during optimization, pruning quite a bit of the search space.

It didn't do anything useful AFAICS.

In some cases, the filerep processes get stuck waiting for the spin locks on an
in memory log and the postmaster PANICs if they can't acquire a lock after
timeout period. In order to avoid locking related issues we want to avoid
acquiring the locks in the first place. This patch introduces a new guc which
will cause filerep logs to be logged to server logs and will avoid logging to
the in memory log.

The gp_default_storage_options GUC had two scenarios in which the value between
master and segments could be conflicting which would cause tables to possibly
have different storage parameters between master and segments.

First Scenario:
The below gpconfig example would give newly created user tables a default storage
parameter of AO on segments while master sets the table as heap. Fixed by not
allowing the user to make them different values.
gpconfig -c gp_default_storage_options -v "'appendonly=true'" -m "'appendonly=false'"

Second Scenario:
During a psql session not in transaction, a session-level gp_default_storage_options
GUC value can be set which will also be dispatched to segment processes. Those
segment processes when idle for a period of time will be stopped and the next SQL
(e.g. CREATE TABLE) will spawn new segment processes which will not have the
session-level GUC value set. Fixed by setting the value at cdbgang creation.

Codes of ic_tcp and ic_udp are not maintained any more, ic_udpifc is
the only interconnect type for now. We keep the gp_interconnect_type
guc for backward compatibility.

Remove unnecessary #includes, add #includes that are actually needed by
some headers.

Renaming optimizer_release_mdcache to a more intuitive name: optimizer_metadata_caching. Before this commit, we were caching metadata in optimizer side by setting optimizer_release_mdcache = OFF. Now, we cache metadata in optimizer side by setting optimizer_metadata_caching=ON.

securitypolicy needs to be aware of classpath
added test to pljava

This reverts commit 6c31a3b4. Per
discussion, we will implement the same functionality in a simpler way.

Added a generation-based Metadata Versioning mechanism
which will be used by ORCA to cache and invalidate catalog
data in its Metadata Cache.
Versioning is disabled by default at this point, until the
Metadata Cache eviction policy is completed.

The current plan is to use something like pg_upgrade for future in-place
upgrades. The gpupgrade mechanism will not scale to the kind of drastic
catalog and other data directory layout changes that are coming as we
merge with later PostgreSQL releases.

Kept gpupgrademirror for now. Need to check if there's some logic that's
worth saving, for a possible pg_upgrade based solution later.

It was just syntax and catalogs, you couldn't actually do anything useful
with it. Remove it, so that we have less code to maintain, until it's time
to merge this stuff from upstream again when we merge with PostgreSQL 8.4.
It's probably easier to merge this back at that point than maintain this
backported version in the meanwhile. Less effort now, until we reach that
point, and once we get to the point in 8.4 that we merge this in, we'll
have all the preceding patches applied already, so it should merge quite
smoothly.

This makes merging and diffing with upstream easier. There were so many
GPDB-added GUCs in guc.c that "git merge" didn't handle it very well, and
you easily got merge conflicts that were error-prone to fix manually.

I kept a few GUCs in guc.c that are not present in REL8_2_0, because they
were backported from later PostgreSQL versions, rather than being
completely GPDB-specific.

Clean up the diff between guc.c and upstream, by removing unnecessary
includes, reordering a few GUCs to match upstream better, and similar
cosmetic changes.

I had to move MAX_KILOBYTES from guc.c to guc_tables.h, because guc_gp.c
needs it too. I also had to export assign_msglvl() function from guc.c, so
that it can be called from other assign_* functions from guc_gp.c. And I
had to add the code to process the ConfigureNames*_gp arrays in addition
to the upstream ones. These things increase the diff against upstream, but
not by too much, IMHO. I also copy-pasted constants like MS_PER_S and
KB_PER_MB, which seems safe as those are natural constants, and should
never change.