Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

While constructing the propagation expression in case of queries
involving default list partitions which have indexes, we ended
up with NULLs crashing the database.

Currently, we fallback to planner. But we need to handle it in a
better way in future.
Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

Signed-off-by: NBhunvesh Chaudhary <bchaudhary@pivotal.io>

With improvement in the expression de duplication algorithm, we now
also retain the order of expression due to which the test cases required
the order expression to be fixed.

We see a lot of plan diffs and plan size changes and the reason for that
is the we always preserve the order of deduped list.

Since we output different deduped list the plan changes are expected
specifically the way that we hash join and hash distribute.
Signed-off-by: NBhunvesh Chaudhary <bchaudhary@pivotal.io>

This commit introduces a performance improvement and stability for
deduplication process of passed expression list.

Earlier the running time complexity of this operation was O(n^2). This
commit reduces the running time to O(n) using a HashMap.

In addition, previously the order of expression is not preserved, as we
used to add the last entry for a expr in the output list, however now
we insert the first entry which keeps the order as is.

E.g. Let's assume everything in the below list is an expression:

A = `a,a,b,b,c,c`
If you run `dedup(A)` It will produce B=`a,b,c`.

Then if you add another `a` to the list B, it will B'=`a,b,c,a`
and when we run the `dedup(B')` it will generate `b,c,a` in the old
code.

With this change, it will always generate `a,b,c`.
Signed-off-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>

For some reason we didn't disable
InnerJoinWithInnerSelect2PartialDynamicIndexGetApply but we should.

While working on [#120303669] (greenplum-db/gporca#160) we found that
the abstraction in ORCA contains "the relation of an index". We don't
use this information anywhere, instead, we get indices from "table
descriptors", which contains "indices of a relation", not the other way
around. And retaining this dead abstraction actually introduces undue
complexity for (greenplum-db/gporca#160) and (greenplum-db/gpdb#2163) .
We are removing all mentions of "relation of an index".

This commit removes `RelationMdid` from the dxl:Index nodes from all our
DXL files.

As a consequence, we also removed:

0. dead member variable `CMDIndexGPDB::m_pmdidRel`
0. dead member variable `CParseHandlerMDIndex::m_pmdidRel`
0. serialization of the relation information of an index
0. deserialization of the relation information of an index
0. RelationMdid in related tests (minimally, more to follow)

This leaves the implementation method CMDIndexGPDB::PmdidRel()
non-override, which we'll remove next.

Signed-off-by: NJesse Zhang <sbjesse@gmail.com>

We do not need to call FSubsetPartCnstr() twice to check for equality.
Two part constraints can are equal if their lengths match and
one is subset of the other.
Signed-off-by: NOmer Arap <oarap@pivotal.io>

When I was trying to understand how does Orca generate plan for CTE using
shared input scan, I found that the share input scan is generated during CTE
producer & consumer DXL node to PlannedStmt translation stage, instead of Expr
to DXL  stage inside Orca. It turns out CDXLPhysicalSharedScan is not used
anywhere, so remove all the related dead code.

Previously when the final plan is translated into DXL, Orca uses range
predicates to represent the values of the list partition (range start and end
value equal a single list value), which is error-prone and redundant for the
query executor.

In this patch, we use the following way to translate the predicates of the list
partition selector in DXL (range based partition remain unchanged):
1. PK op Scalar -> Scalar reverse(op) Any(PartListValues)
For partition selector predicate `pk1 < 5`, will be translated to `ArrayComp(5 >
Any(PartListValues))`, which means as long as any value of list partition values
is less than 5, the partition will be selected.
2. PK is (not) NULL -> PartListNullTest
3. Propagation Expression ->
   Const1 = Any(PartListValues) or Const2 = Any(PartListValues) ...

[#140699737]
Closes #149

Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

We do not need to call FSubset() twice to check for equality.
Two CTE requirements will be considered equal if their lengths match and
one is subset of the other.
Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

Signed-off-by: NOmer Arap <oarap@pivotal.io>

Signed-off-by: NOmer Arap <oarap@pivotal.io>

Signed-off-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

This commit introduces caching of partition propagation spec for NL Join
as Orca is benefiting from caching in Hash Join. This commit introduces
significant performance in plan generation.
Signed-off-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

This commit refactors the partition propagation enforcement logics and
push the necessary classes and functions from CPhysicalHashJoin to
CPhysicalJoin.
Signed-off-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Signed-off-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Added a new FMatch method to allow comparison for Array of Column
Reference Arrays. This fix removes redundant Partition Selector operator in
Group Expression resulting in a smaller plan space.
Signed-off-by: NOmer Arap <oarap@pivotal.io>

Removing timing Scalar DXL To Scalar Expr Translation Time
as for certain queries it produces many lines of noise.

The equality parition filter of partition selector works very well for single
level partition table, but if the table has multilevel partitions, e.g. 2
levels with pk1, pk2 as the partition key for level 1 and 2, and there is a
equality predicate in the query, say pk1 = 2, then level 2 equality filter is null,
the function `FEqPartFiltersAllLevels` will return false, causing the equality
predicate put into PartFilters instead of PartEqFilters. This bug has been
fixed in the patch.

[#141826453]

In previous implementation of hash maps, all the bucket chains have been
traversed no matter if the bucket is pointing to a chain or not.

In this new version of `Clear()` we keep track of the indices for every
bucket where a chain is created and we simply deallocated these chains.
Therefore, no time is wasted on checking unallocated bucket chains.
Signed-off-by: NOmer Arap <oarap@pivotal.io>

Currently, the HashMapIter implementation scans through all hash map
buckets to get the next existing hash chain. This degrades performance
significantly.

This commit improves the iterator implementation by maintaining a
dynamic key array which holds the existing keys in HashMap. The
iteration is done using this array.
Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

Makes the code simpler for humans, and also allows the compiler to "see"
what the operations are, and optimize accordingly.

I hoped for the GPDB compiler warnings about failed inlining with -Winline
to go away with commit 5f774da5. It turns out that commit was not enough,
unfortunately, but this commit does the trick.