1. 11 6月, 2020 2 次提交
  2. 10 6月, 2020 4 次提交
  3. 09 6月, 2020 2 次提交
  4. 08 6月, 2020 3 次提交
    • H
      Remove unused variable and rel_is_external_table() call. · 71e50c75
      Heikki Linnakangas 提交于
      Was left unused by commit e4b499aa that removed the pg_exttable catalog
      table.
      71e50c75
    • Z
      Fix lateral PANIC issue when subquery contain limit or groupby. · 8d1bb5a8
      Zhenghua Lyu 提交于
      Previous commit 62579728 fixes a lateral panic issue but does
      not handle all the bad cases because it only check if the query
      tree contains limit clause. Bad cases for example: if the subquery
      is like `q1 union all (q2 limit 1)` then the whole query tree
      does not contain limit clause.
      
      Another bad case is the lateral subquery may contain groupby.
      like:
      
          select * from t1_lateral_limit t1 cross join lateral
          (select (c).x+t2.a, sum(t2.a+t2.b) from t2_lateral_limit t2
           group by (c).x+t2.a)x;
      
      When planning the lateraled subquery we do not know where is
      the param in the subquery's query tree. Thus it is a bit complicated
      to precisely and efficiently resolve this issue.
      
      This commit adopts a simple method to fix panic issue: it justs
      check the subquery's query tree to see if there is any group-by
      or limit clause, if so, force gather each relation and materialize
      them. This is not the best plan we might get. But let's make it
      correct first and I think in future we should seriously consider
      how to fully and efficiently support lateral.
      8d1bb5a8
    • P
      Retire guc gp_session_role (#9396) · f6297b96
      Paul Guo 提交于
      Use guc gp_role only now and replace the functionality of guc gp_session_role with it
      also. Previously we have both gucs. The difference of the two gucs are (copied
      from code comment):
      
       * gp_session_role
       *
       * - does not affect the operation of the backend, and
       * - does not change during the lifetime of PostgreSQL session.
       *
       * gp_role
       *
       * - determines the operating role of the backend, and
       * - may be changed by a superuser via the SET command.
      
      This is not friendly for coding. For example, You could find Gp_role and
      Gp_session_role are set as GP_ROLE_DISPATCH on Postmaster & many aux processes
      on all nodes (even QE nodes) in a cluster, so you can see that to differ from
      QD postmaster and QE postmaster, current gpdb uses an additional -E option in
      postmaster arguments. These makes developers confusing when writing role branch
      related code given we have three related variables.  Also some related code is
      even buggy now (e.g. 'set gp_role' even FATAL quits).
      
      With this patch we just have gp_role now. Some changes which might be
      interesting in the patch are:
      
      1. For postmaster, we should specify '-c gp_role=' (e.g. via pg_ctl argument) to
         determine the role else we assume the utility role.
      
      2. For stand-alone backend, utility role is enforced (no need to specify by
         users).
      
      3. Could still connect QE/QD nodes using utility mode with PGOPTIONS, etc as
         before.
      
      4. Remove the '-E' gpdb hacking and align the '-E' usage with upstream.
      
      5. Move pm_launch_walreceiver out of the fts related shmem given the later is
         not used on QE.
      Reviewed-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
      Reviewed-by: NGang Xiong <gxiong@pivotal.io>
      Reviewed-by: NHao Wu <gfphoenix78@gmail.com>
      Reviewed-by: NYandong Yao <yyao@pivotal.io>
      f6297b96
  5. 06 6月, 2020 1 次提交
  6. 05 6月, 2020 7 次提交
    • L
      docs - update some xrefs and upgrade info (#10237) · 07c2029c
      Lisa Owen 提交于
      07c2029c
    • A
      Fix wrong data type introduced in bf36fb3b · a251127b
      Asim R P 提交于
      a251127b
    • B
      Fix user specify in matview_ao test · 24df496d
      bzhaoopenstack 提交于
      This patch will create a test role to exec the test.
      24df496d
    • H
      Fix flaky test in recoverseg_from_file · 1ee9998b
      Hubert Zhang 提交于
      1. after stop primary with content=1, we should check promotion
      status with 1U:
      2. after manually update dbid, we should trigger a fts probe and
      wait for the mirror promotion as well.
      Reviewed-by: NPaul Guo <pguo@pivotal.io>
      1ee9998b
    • H
      Support "NDV-preserving" function and op property (#10247) · a4362cba
      Hans Zeller 提交于
      Orca uses this property for cardinality estimation of joins.
      For example, a join predicate foo join bar on foo.a = upper(bar.b)
      will have a cardinality estimate similar to foo join bar on foo.a = bar.b.
      
      Other functions, like foo join bar on foo.a = substring(bar.b, 1, 1)
      won't be treated that way, since they are more likely to have a greater
      effect on join cardinalities.
      
      Since this is specific to ORCA, we use logic in the translator to determine
      whether a function or operator is NDV-preserving. Right now, we consider
      a very limited set of operators, we may add more at a later time.
      
      Let's assume that we join tables R and S and that f is a function or
      expression that refers to a single column and does not preserve
      NDVs. Let's also assume that p is a function or expression that also
      refers to a single column and that does preserve NDVs:
      
      join predicate       card. estimate                         comment
      -------------------  -------------------------------------  -----------------------------
      col1 = col2          |R| * |S| / max(NDV(col1), NDV(col2))  build an equi-join histogram
      f(col1) = p(col2)    |R| * |S| / NDV(col2)                  use NDV-based estimation
      f(col1) = col2       |R| * |S| / NDV(col2)                  use NDV-based estimation
      p(col1) = col2       |R| * |S| / max(NDV(col1), NDV(col2))  use NDV-based estimation
      p(col1) = p(col2)    |R| * |S| / max(NDV(col1), NDV(col2))  use NDV-based estimation
      otherwise            |R| * |S| * 0.4                        this is an unsupported pred
      Note that adding casts to these expressions is ok, as well as switching left and right side.
      
      Here is a list of expressions that we currently treat as NDV-preserving:
      
      coalesce(col, const)
      col || const
      lower(col)
      trim(col)
      upper(col)
      
      One more note: We need the NDVs of the inner side of Semi and
      Anti-joins for cardinality estimation, so only normal columns and
      NDV-preserving functions are allowed in that case.
      
      This is a port of these GPDB 5X and GPOrca PRs:
      https://github.com/greenplum-db/gporca/pull/585
      https://github.com/greenplum-db/gpdb/pull/10090
      
      This is take 2, after reverting the first attempt due to a merge conflict that
      caused a test to fail.
      a4362cba
    • D
      Docs - update gpcc compatibility info for 6.8 · 560ffcb1
      David Yozie 提交于
      560ffcb1
    • L
      docs - add pxf v5.12 to supported platforms (#10235) · fe3464af
      Lisa Owen 提交于
      fe3464af
  7. 04 6月, 2020 3 次提交
    • H
      Fix the logic in pg_lock_status() to keep track of which row to return. · 90634b6a
      Heikki Linnakangas 提交于
      The logic with 'whichrow' and 'whichresultset' introduced in commit
      991273b2 was slightly wrong. The last row (or first? not sure) of each
      result set was returned twice, and the corresponding number of rows at the
      end of the last result set were omitted.
      
      For example:
      
      postgres=# select gp_segment_id, * from pg_locks;
       gp_segment_id |  locktype  | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction |  pid  |      mode       | granted | fastpath | mppsessionid | mppiswriter | gp_segment_id
      ---------------+------------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+-----------------+---------+----------+--------------+-------------+---------------
                  -1 | relation   |    13200 |    11869 |      |       |            |               |         |       |          | 1/8                | 28748 | AccessShareLock | t       | t        |            6 | t           |            -1
                  -1 | virtualxid |          |          |      |       | 1/8        |               |         |       |          | 1/8                | 28748 | ExclusiveLock   | t       | t        |            6 | t           |            -1
                   0 | virtualxid |          |          |      |       | 1/7        |               |         |       |          | 1/7                | 28750 | ExclusiveLock   | t       | t        |            6 | t           |             0
                   1 | virtualxid |          |          |      |       | 1/7        |               |         |       |          | 1/7                | 28751 | ExclusiveLock   | t       | t        |            6 | t           |             1
                   1 | virtualxid |          |          |      |       | 1/7        |               |         |       |          | 1/7                | 28751 | ExclusiveLock   | t       | t        |            6 | t           |             1
      (5 rows)
      
      Note how the last row is duplicated. And the row for 'virtualxid' from
      segment 2 is omitted.
      
      I noticed this while working on the PostgreSQL v12 merge: the 'lock'
      regression test was failing because of this. I'm not entriely sure why we
      haven't seen failures on 'master'. I think it's pure chance that none of
      the lines that the test prints have been omitted on 'master'. But because
      that's been failing, I don't feel the need to add more tests for this.
      90634b6a
    • J
      Revert "Support "NDV-preserving" function and op property (#10225)" · 898e66b8
      Jesse Zhang 提交于
      Regression test "gporca" started failing after merging d565edac.
      
      This reverts commit d565edac.
      898e66b8
    • H
      Support "NDV-preserving" function and op property (#10225) · d565edac
      Hans Zeller 提交于
      Orca uses this property for cardinality estimation of joins.
      For example, a join predicate foo join bar on foo.a = upper(bar.b)
      will have a cardinality estimate similar to foo join bar on foo.a = bar.b.
      
      Other functions, like foo join bar on foo.a = substring(bar.b, 1, 1)
      won't be treated that way, since they are more likely to have a greater
      effect on join cardinalities.
      
      Since this is specific to ORCA, we use logic in the translator to determine
      whether a function or operator is NDV-preserving. Right now, we consider
      a very limited set of operators, we may add more at a later time.
      
      Let's assume that we join tables R and S and that f is a function or
      expression that refers to a single column and does not preserve
      NDVs. Let's also assume that p is a function or expression that also
      refers to a single column and that does preserve NDVs:
      
      join predicate       card. estimate                         comment
      -------------------  -------------------------------------  -----------------------------
      col1 = col2          |R| * |S| / max(NDV(col1), NDV(col2))  build an equi-join histogram
      f(col1) = p(col2)    |R| * |S| / NDV(col2)                  use NDV-based estimation
      f(col1) = col2       |R| * |S| / NDV(col2)                  use NDV-based estimation
      p(col1) = col2       |R| * |S| / max(NDV(col1), NDV(col2))  use NDV-based estimation
      p(col1) = p(col2)    |R| * |S| / max(NDV(col1), NDV(col2))  use NDV-based estimation
      otherwise            |R| * |S| * 0.4                        this is an unsupported pred
      Note that adding casts to these expressions is ok, as well as switching left and right side.
      
      Here is a list of expressions that we currently treat as NDV-preserving:
      
      coalesce(col, const)
      col || const
      lower(col)
      trim(col)
      upper(col)
      
      One more note: We need the NDVs of the inner side of Semi and
      Anti-joins for cardinality estimation, so only normal columns and
      NDV-preserving functions are allowed in that case.
      
      This is a port of these GPDB 5X and GPOrca PRs:
      https://github.com/greenplum-db/gporca/pull/585
      https://github.com/greenplum-db/gpdb/pull/10090
      d565edac
  8. 03 6月, 2020 6 次提交
    • S
      Remove unnecessary projections from duplicate sensitive Distribute(s) in ORCA · c02fd5a1
      Shreedhar Hardikar 提交于
      Duplicate sensitive HashDistribute Motions generated by ORCA get
      translated to Result nodes with hashFilter cols set. However, if the
      Motion needs to distribute based on a complex expression (rather than
      just a Var), the expression must be added into the targetlist of the
      Result node and then referenced in hashFilterColIdx.
      
      However, this can affect other operators above the Result node. For
      example, a Hash operator expects the targetlist of its child node to
      contain only elements that are to be hashed. Additional expressions here
      can cause issues with memtuple bindings that can lead to errors.
      
      (E.g The attached test case, when run without our fix, will give an
      error: "invalid input syntax for integer:")
      
      This PR fixes the issue by adding an additional Result node on top of
      the duplicate sensitive Result node to project only the elements from
      the original targetlist in such cases.
      c02fd5a1
    • A
      Squash me: address concerns in code review · bf36fb3b
      Asim R P 提交于
      Remember if the select call was interrupted.  Act on it after emitting
      debug logs and checking cancel requests from dispatcher.
      bf36fb3b
    • A
      Check errno as early as possible · 9fd138da
      Asim R P 提交于
      Previously, the result of select() system call and errno set by it was
      checked after performing several function calls, including checking
      for interrupts and checkForCancelFromQD.  That made it very likely for
      errno to change, losing the original value that was set by the
      select().
      
      This patch fixes it so that the errno is checked immediately after the
      system call.  This should address intermittent failures in CI with
      error message like this:
      
          ERROR","58M01","interconnect error: select: Success"
      9fd138da
    • W
      Add "FILL_MISSING_FIELDS" option for gpload. · cb76c301
      Wen Lin 提交于
      cb76c301
    • A
      Allow CLUSTER on append-optimized tables · 179feb77
      Andrey Borodin 提交于
      Cluster on AO tables is implemented by sorting the entire AO table using
      tuple sort framework, according to a btree index defined on the table.
      
      A faster way to cluster is to scan the tuples in index-order, but this
      requires index-scan support.  Append-optimized tables do not support
      index-scans currently, but when this support is added, the cluster
      operation can be enhanced accordingly.
      
      Author: Andrey Borodin <amborodin@acm.org>
      Reviewed and slightly edited by: Asim R P <pasim@vmare.com>
      
      Merges GitHub PR #9996
      179feb77
    • H
      Refactoring the DbgPrint and OsPrint methods (#10149) · b3fdede6
      Hans Zeller 提交于
      * Make DbgPrint and OsPrint methods on CRefCount
      
      Create a single DbgPrint() method on the CRefCount class. Also create
      a virtual OsPrint() method, making some objects derived from CRefCount
      easier to print from the debugger.
      
      Note that not all the OsPrint methods had the same signatures, some
      additional OsPrintxxx() methods have been generated for that.
      
      * Making print output easier to read, print some stuff on demand
      
      Required columns in required plan properties are always the same
      for a given group. Also, equivalent expressions in required distribution
      properties are important in certain cases, but in most cases they
      disrupt the display and make it harder to read.
      
      Added two traceflags, EopttracePrintRequiredColumns and
      EopttracePrintEquivDistrSpecs that have to be set to print this
      information. If you want to go back to the old display, use these
      options when running gporca_test: -T 101016 -T 101017
      
      * Add support for printing alternative plans
      
      A new method, CEngine::DbgPrintExpr() can be called from
      COptimizer::PexprOptimize, to allow printing of the best plan
      for different contexts. This is only enabled in debug builds.
      
      To use this:
      
      - run an MDP using gporca_test, using a debug build
      - print out memo after optimization (-T 101006 -T 101010)
      - set a breakpoint near the end of COptimizer::PexprOptimize()
      - if, after looking at the contents of memo, you want to see
        the optimal plan for context c of group g, do the following:
        p eng.DbgPrintExpr(g, c)
      
      You could also get the same info from the memo printout, but it
      would take a lot longer.
      b3fdede6
  9. 02 6月, 2020 2 次提交
    • H
      Clarify code that parses DISTRIBUTED BY, with new DistributionKeyElem node. · 506ebada
      Heikki Linnakangas 提交于
      Introduce a new DistributionKeyElem to hold each element in the list of
      columns (and optionally their opclasses) in DISTRIBUTED BY (<col> [opclass],
      ...) syntax. Previously, we have used IndexElem, which conveniently also
      holds a column name and its opclass, but it was not a very good fit because
      IndexElem also contains many other fields that are not needed. Using a
      new node type specifically for DISTRIBUTED BY makes the code dealing with
      distribution key lists more clear. To compare, PostgreSQL v10 uses a struct
      called PartitionElem for similar purposes in PARTITION BY clause. (But that
      is not to be confused with the PartitionElem struct in GPDB 6 and below,
      which is also related to partitioning sytnax but is quite different!)
      
      Unlike IndexElem, the new node type includes a 'location' field, to
      provide error position information in error messages. This can be seen in
      the error message changes in 'gp_create_table' test.
      
      While we're at it, remove the quotes around DISTRIBUTED BY in the
      "column <col> named in DISTRIBTED BY clause does not exist", for
      consistency with the same error message thrown with CREATE TABLE AS from
      setQryDistributionPolicy() function, and with the "duplicate column in
      DISTRIBUTED BY clause" error. The error thrown in CTAS case was not
      covered by existing tests, so also add a test for that.
      Reviewed-by: NZhenghua Lyu <zlv@pivotal.io>
      506ebada
    • R
      Fix up enable_hashagg when creating a UniquePath · 73e2c13f
      Richard Guo 提交于
      In PostgreSQL, semi-joins are implemented with JOIN_SEMI join types, or
      by first eliminating duplicates from the inner side, and then performing
      normal inner join (that's JOIN_UNIQUE_OUTER and JOIN_UNIQUE_INNER).
      
      GPDB has a third way to implement them: Perform an inner join first, and
      then eliminate duplicates from the result. This is performed by a
      UniquePath above the join. And there are two ways to implement the
      UniquePath: sorting and hashing. But if hashing is not enabled, we
      should not consider it.
      
      This patch fixes github issue #8437.
      Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
      Reviewed-by: NMelanie Plageman <mplageman@pivotal.io>
      Reviewed-by: NPaul Guo <pguo@pivotal.io>
      73e2c13f
  10. 01 6月, 2020 2 次提交
    • H
      Support multiple INITPLAN functions in one query. · 2589a33a
      Hubert Zhang 提交于
      We now use initplan id to differentiate the tuplestore used by
      different INITPLAN functions. INITPLAN will also write the function
      result into different tuplestores.
      
      Also fix the bug which appends initplan in the wrong place. It may
      generate wrong result in UNION ALL case.
      Reviewed-by: NZhenghua Lyu <zlv@pivotal.io>
      2589a33a
    • H
      Fix the module of the hash function of a motion (#10087) · f69978d2
      Hao Wu 提交于
      After some refactors for the path/planner, the code assumes that
      the module of the hash function of a motion equals to the Gang
      size of the parent slice. However, it isn't always true, if some
      tables are distributed on part of the segments. It could happen
      during gpexpand.
      Assume the following case, there is a GPDB cluster with 2 segments,
      and the cluster is running `gpexpand` to add 1 segment. Now t1
      and t2 are distributed on the first 2 segments, t3 has finished
      data transfer, i.e. t3 is distributed on three segments.
      See the following plan:
      
      gpadmin=# explain select t1.a, t2.b from t1 join t2 on t1.a = t2.b union all select * from t3;
                                                      QUERY PLAN
      -----------------------------------------------------------------------------------------------------------
       Gather Motion 3:1  (slice1; segments: 3)  (cost=2037.25..1230798.15 rows=7499310 width=8)
         ->  Append  (cost=2037.25..1080811.95 rows=2499770 width=8)
               ->  Hash Join  (cost=2037.25..1005718.85 rows=2471070 width=8)
                     Hash Cond: (t2.b = t1.a)
                     ->  Redistribute Motion 2:3  (slice2; segments: 2)  (cost=0.00..2683.00 rows=43050 width=4)
                           Hash Key: t2.b
                           ->  Seq Scan on t2  (cost=0.00..961.00 rows=43050 width=4)
                     ->  Hash  (cost=961.00..961.00 rows=28700 width=4)
                           ->  Seq Scan on t1  (cost=0.00..961.00 rows=28700 width=4)
               ->  Seq Scan on t3  (cost=0.00..961.00 rows=28700 width=8)
       Optimizer: Postgres query optimizer
      
      The slice2 shows that t2 will redistribute to all 3 segments and join
      with t1. Since t1 is distributed only on the first 2 segments, the data
      from t2 redistributed to the third segment couldn't have a match,
      which returns wrong results.
      
      The root cause is that the module of the cdb hash to redistribute t2 is
      3, i.e. the Gang size of the parent slice. To fix this issue,
      we add a field in Motion to record the number of receivers.
      With this patch, the plan generated is:
      
      gpadmin=# explain select * from t1  join t2 on t1.a = t2.b union all select a,a,b,b from t3;
                                                      QUERY PLAN
      ----------------------------------------------------------------------------------------------------------
       Gather Motion 3:1  (slice2; segments: 3)  (cost=2.26..155.16 rows=20 width=16)
         ->  Append  (cost=2.26..155.16 rows=7 width=16)
               ->  Hash Join  (cost=2.26..151.97 rows=7 width=16)
                     Hash Cond: (t1.a = t2.b)
                     ->  Seq Scan on t1  (cost=0.00..112.06 rows=5003 width=8)
                     ->  Hash  (cost=2.18..2.18 rows=2 width=8)
                           ->  Redistribute Motion 2:3  (slice1; segments: 2)  (cost=0.00..2.18 rows=3 width=8)
                                 Hash Key: t2.b
                                 Hash Module: 2
                                 ->  Seq Scan on t2  (cost=0.00..2.06 rows=3 width=8)
               ->  Seq Scan on t3  (cost=0.00..3.06 rows=2 width=16)
       Optimizer: Postgres query optimizer
      
      Note: the interconnect for the redistribute motion is still 2:3,
      but the data transfer only happens in 2:2.
      
      Co-authored-by: Zhenghua Lyu zlv@pivotal.io
      Reviewed-by: NPengzhou Tang <ptang@pivotal.io>
      Reviewed-by: NJesse Zhang <jzhang@pivotal.io>
      f69978d2
  11. 31 5月, 2020 4 次提交
    • H
      Allow AOCO tables to inherit a parent table. · 177abc76
      Heikki Linnakangas 提交于
      Because why not? This was forbidden back in 2012, with an old JIRA ticket
      that said:
      
          MPP-15735 - Inheritance is not supported with column oriented table
      
          This shouldn't be possible:
      
          Create table parent_tb2 (a1 int, a2 char(5), a3 text, a4 timestamp,
                 a5 date, column a1 ENCODING (compresstype=zlib,compresslevel=9,blocksize=32768))
            with (appendonly=true,orientation=column)
            distributed by (a1);
      
          Create table child_tb4 (c1 int, c2 char) inherits(parent_tb2)
            with (appendonly = true, orientation = column);
      
          The reason is, dealing with column compression and inheritance is tricky
          and inheritance shouldn't even be supported.
      
      That explanation didn't go into any details, but I can see the trickiness.
      There are many places you can specify column compression options even
      without inheritance: in gp_default_storage_options GUC, in an ENCODING
      directive in CREATE TABLE command, in a "COLUMN foo ENCODING ..."
      directive in the CREATE TABLE command, or in the datatype's default
      storage options. Inheritance adds another dimension to it: should the
      current gp_default_storage_options override the options inherited from the
      parent? What if there are multiple parents with conflicting options?
      
      The easiest solution is to never inherit the column encoding options from
      the parent. That's a bit lame, but there's some precedence for that with
      partitions: if you ALTER TABLE ADD PARTITION, the encoding options are
      also not copied from the parent. So that's what this patch does.
      
      One interesting corner case is to specify the options for an inherited
      column with "CREATE TABLE (COLUMN parentcol ENCODING ...) INHERITS (...)".
      Thanks to the previous refactoring, that works, even though 'parentcol'
      is not explicitly listed in the CREATE TABLE command but inherited from
      the parent. To make dump & restore of that work correctly, modify pg_dump
      so that it always uses the "COLUMN foo ENCODING ..." syntax to specify
      the options, instead of just tacking "ENCODING ..." after the column
      definition.
      
      One excuse for doing this right now is that even though we had forbidden
      "CREATE TABLE <tab> INHERITS (<parent>)" on AOCO tables, we missed "ALTER
      TABLE <tab> INHERIT <parent>". That was still allowed, and if you did
      that you got an inherited AOCO table that worked just fine, except that
      when it was dumped with pg_dump, the dump was unrestorable. It would have
      been trivial to add a check to forbid "ALTER TABLE INHERIT" on an AOCO
      table, instead, but it's even better to allow it.
      
      Fixes https://github.com/greenplum-db/gpdb/issues/10111Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
      177abc76
    • H
      Allow COLUMN ... ENCODING when some partitions are not AOCS. · 428bd390
      Heikki Linnakangas 提交于
      Before this commit, this worked:
      
          CREATE TABLE mpp17740 (
            a integer ENCODING (compresstype=none,blocksize=32768,compresslevel=0),
            b integer ENCODING (compresstype=zlib,blocksize=32768,compresslevel=2),
            e date    ENCODING (compresstype=none,blocksize=32768,compresslevel=0)
          )
          WITH (appendonly='true', orientation='column')
          DISTRIBUTED BY ("a") PARTITION BY RANGE("e")
          (
            PARTITION mpp17740_20120520 START ('2012-05-20'::date) END ('2012-05-21'::date) WITH (appendonly='true'),
            PARTITION mpp17740_20120523 START ('2012-05-23'::date) END ('2012-05-24'::date) WITH (appendonly='true'),
            PARTITION mpp17740_20120524 START ('2012-05-24'::date) END ('2012-05-25'::date) WITH (appendonly='true' )
          );
      
      But this errored out:
      
          CREATE TABLE mpp17740 (
            a integer,
            b integer,
            e date
          COLUMN "a" ENCODING (compresstype=none,blocksize=32768,compresslevel=0),
          COLUMN "b" ENCODING (compresstype=zlib,blocksize=32768,compresslevel=2),
          COLUMN "e" ENCODING (compresstype=none,blocksize=32768,compresslevel=0)
          )
          WITH (appendonly='true', orientation='column')
          WITH (appendonly='true', orientation='column')
          DISTRIBUTED BY ("a") PARTITION BY RANGE("e")
          (
            PARTITION mpp17740_20120520 START ('2012-05-20'::date) END ('2012-05-21'::date) WITH (appendonly='true'),
            PARTITION mpp17740_20120523 START ('2012-05-23'::date) END ('2012-05-24'::date) WITH (appendonly='true'),
            PARTITION mpp17740_20120524 START ('2012-05-24'::date) END ('2012-05-25'::date) WITH (appendonly='true' )
          );
          psql: NOTICE:  CREATE TABLE will create partition "mpp17740_1_prt_mpp17740_20120520" for table "mpp17740"
          psql: ERROR:  ENCODING clause only supported with column oriented partitioned tables
      
      That seems inconsistent. It's also problematic because in the next commit,
      I'm going to change pg_dump to use the latter syntax. Relax the checks so that
      the latter syntax doesn't throw an error.
      
      This includes a little refactoring to the way CREATE TABLE AS is
      dispatched. The internal CreateStmt that is constructed to create the
      table is now saved in the QD and dispatched to the QEs, instead of
      re-constructing it in each segment separately. A wholesale approach like
      that seems nicer than dispatching the looked-up table space name
      separately, and we would've needed an exception to the checks for ENCODING
      options, to allow the QE to parse them.
      Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
      428bd390
    • H
      Fix checks for non-existent or duplicated columns in COLUMN ENCODING. · 10d13f54
      Heikki Linnakangas 提交于
      These checks are supposed to run on the original user-supplied clauses,
      not the clauses derived from the user-supplied values and defaults. The
      derived clauses always pass the checks.
      
      Fixes https://github.com/greenplum-db/gpdb/issues/10115Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
      10d13f54
    • H
      Refactor the way ENCODING clauses are parsed. · 391b78eb
      Heikki Linnakangas 提交于
      Instead of parsing them in the parse analysis phase, delay it until
      DefineRelation(), after merging the definitions of inherited attributes.
      We don't support INHERIT clause on an AOCO table today, but if we lift
      that limitation (as I'm planning to do in a follow-up commit), we need to
      be able to apply "COLUMN <col> ENCODING ..." clauses to inherited columns
      too.
      
      Before this, all callers of create_ctas_internal() had to also call
      AddDefaultRelationAttributeOptions(), in case it was an AOCO table. Now
      that DefineRelation() handles the default storage options, that's no
      longer needed.
      Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
      391b78eb
  12. 30 5月, 2020 2 次提交
    • C
      Penalize cross products in Orca's DPv2 algorithm more accurately (#10029) · 457bb928
      Chris Hajas 提交于
      Previously in the DPv2 transform (exhaustive2) while we penalized
      cross joins for the remaining joins in greedy, we did
      not for the first join, which in some cases selected a cross join.
      This ended up selecting a poor join order in many cases and went against
      the intent of the alternative being generated, which is to minimize
      cross joins.
      
      We also increase the cost of the default penalty from 5 to 1024, which is the value we use in the cost model during the optimization stage.
      
      The greedy alternative also wasn't kept in the heap, so we include that now too.
      457bb928
    • C
      Ensure Material nodes generated by Orca always materialize · a65c97da
      Chris Hajas 提交于
      In cases where Orca generates a NLJ with a parameter on the inner side,
      the executor will not pass the EXEC_FLAG_REWIND flag down, as it assumed
      the inner side will always need to be rescanned. The material node will therefore
      not have its rewind flag set and can act as a no-op.
      
      This is not always correct. While the executor will set EXEC_FLAG_REWIND
      if the Materialize is directly above a motion, it does not recognize
      the case where the Materialize is on the inner side with other nodes
      between it and the motion, even though the Materialize serves to
      prevent a rescan of the underlying Motion node.
      
      This causes the execution to fail with:
      `Illegal rescan of motion node: invalid plan (nodeMotion.c:1623)` as it
      would attempt to rescan a motion.
      
      Since Orca only produces Materialize when necessary, either for
      performance reasons or to prevent rescan of an underlying Motion,
      EXEC_FLAG_REWIND should be set for any Materialize generated by Orca.
      
      Below is a valid plan generated by Orca:
      
      ```
       Result  (cost=0.00..3448.01 rows=1 width=4)
         ->  Nested Loop  (cost=0.00..3448.01 rows=1 width=1)
               Join Filter: true
               ->  Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..431.00 rows=2 width=4)
                     ->  Seq Scan on foo1  (cost=0.00..431.00 rows=1 width=4)
               ->  Result  (cost=0.00..431.00 rows=1 width=1)
                     Filter: (foo1.a = foo2.a)
                     ->  Materialize  (cost=0.00..431.00 rows=1 width=4)
                           ->  Hash Semi Join  (cost=0.00..431.00 rows=1 width=4)
                                 Hash Cond: (foo2.b = foo3.b)
                                 ->  Gather Motion 3:1  (slice2; segments: 3)  (cost=0.00..0.00 rows=1 width=8)
                                       ->  Bitmap Heap Scan on foo2  (cost=0.00..0.00 rows=1 width=8)
                                             Recheck Cond: (c = 3)
                                             ->  Bitmap Index Scan on f2c  (cost=0.00..0.00 rows=0 width=0)
                                                   Index Cond: (c = 3)
                                 ->  Hash  (cost=431.00..431.00 rows=1 width=4)
                                       ->  Gather Motion 3:1  (slice3; segments: 3)  (cost=0.00..431.00 rows=2 width=4)
                                             ->  Seq Scan on foo3  (cost=0.00..431.00 rows=1 width=4)
       Optimizer: Pivotal Optimizer (GPORCA)
       ```
      Co-authored-by: NChris Hajas <chajas@pivotal.io>
      Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
      a65c97da
  13. 29 5月, 2020 2 次提交
    • H
      Fix broken test for old gp_bloat_diag issue. · c9199036
      Heikki Linnakangas 提交于
      Commit 152783e1 added a test along with the fix, but I broke it soon
      after in commit c91e320c. I changed the name of the test table from
      'test' to 'wide_width_test', but forgot to adjust UPDATE command
      accordingly, so the UPDATE did nothing.
      
      Fix that, and also change the test to not rely on pg_attribute to create
      the test table. I noticed this as I was working on the PostgreSQL v12
      merge: the test was throwing an error because a new 'anyarray' column
      (attmissingval) was added to pg_attribute in upstream. That caused the
      CTAS to fail. Listing the table columns explicitly avoids that problem.
      
      I verified this by reverting commit 152783e1 locally, and running the
      test. It now throws an integer overflow error as intended.
      c9199036
    • G
      Fix possible build and install errors · d4efd3c9
      ggbq 提交于
      In most cases, the variable LN_S is 'ln -s', however, the LN_S can
      be changed to 'cp -pR' if the configure finds the file system does
      not support symbolic links. It would be incompatible when linking
      a subdir path to a relative path.
      
      cd to subdir first before linking a file.
      d4efd3c9