Rule based partition selection for list (sub)partitions (#2076)
GPDB supports range and list partitions. Range partitions are represented as a set of rules. Each rule defines the boundaries of a part. E.g., a rule might say that a part contains all values between (0, 5], where left bound is 0 exclusive, but the right bound is 5, inclusive. List partitions are defined by a list of values that the part will contain.
ORCA uses the above rule definition to generate expressions that determine which partitions need to be scanned. These expressions are of the following types:
1. Equality predicate as in PartitionSelectorState->levelEqExpressions: If we have a simple equality on partitioning key (e.g., part_key = 1).
2. General predicate as in PartitionSelectorState->levelExpressions: If we need more complex composition, including non-equality such as part_key > 1.
Note: We also have residual predicate, which the optimizer currently doesn't use. We are planning to remove this dead code soon.
Prior to this PR, ORCA was treating both range and list partitions as range partitions. This meant that each list part will be converted to a set of list values and each of these values will become a single point range partition.
E.g., consider the DDL:
```sql
CREATE TABLE DATE_PARTS (id int, year int, month int, day int, region text)
DISTRIBUTED BY (id)
PARTITION BY RANGE (year)
SUBPARTITION BY LIST (month)
SUBPARTITION TEMPLATE (
SUBPARTITION Q1 VALUES (1, 2, 3),
SUBPARTITION Q2 VALUES (4 ,5 ,6),
SUBPARTITION Q3 VALUES (7, 8, 9),
SUBPARTITION Q4 VALUES (10, 11, 12),
DEFAULT SUBPARTITION other_months )
( START (2002) END (2012) EVERY (1),
DEFAULT PARTITION outlying_years );
```
Here we partition the months as list partition using quarters. So, each of the list part will contain three months. Now consider a query on this table:
```sql
select * from DATE_PARTS where month between 1 and 3;
```
Prior to this ORCA generated plan would consider each value of the Q1 as a separate range part with just one point range. I.e., we will have 3 virtual parts to evaluate for just one Q1: [1], [2], [3]. This approach is inefficient. The problem is further exacerbated when we have multi-level partitioning. Consider the list part of the above example. We have only 4 rules for 4 different quarters, but we will have 12 different virtual rule (aka constraints). For each such constraint, we will then evaluate the entire subtree of partitions.
After this PR, we no longer decompose rules into constraints for list parts and then derive single point virtual range partitions based on those constraints. Rather, the new ORCA changes will use ScalarArrayOp to express selectivity on a list of values. So, the expression for the above SQL will look like 1 <= ANY {month_part} AND 3 >= ANY {month_part}, where month_part will be substituted at runtime with different list of values for each of quarterly partitions. We will end up evaluating that expressions 4 times with the following list of values:
Q1: 1 <= ANY {1,2,3} AND 3 >= ANY {1,2,3}
Q2: 1 <= ANY {4,5,6} AND 3 >= ANY {4,5,6}
...
Compare this to the previous approach, where we will end up evaluating 12 different expressions, each time for a single point value:
First constraint of Q1: 1 <= 1 AND 3 >= 1
Second constraint of Q1: 1 <= 2 AND 3 >= 2
Third constraint of Q1: 1 <= 3 AND 3 >= 3
First constraint of Q2: 1 <= 4 AND 3 >= 4
...
The ScalarArrayOp depends on a new type of expression PartListRuleExpr that can convert a list rule to an array of values. ORCA specific changes can be found here: https://github.com/greenplum-db/gporca/pull/149
Showing
想要评论请 注册 或 登录