• P
    Support replicated table in GPDB · 7efe3204
    Pengzhou Tang 提交于
    * Support replicated table in GPDB
    
    Currently, tables are distributed across all segments by hash or random in GPDB. There
    are requirements to introduce a new table type that all segments have the duplicate
    and full table data called replicated table.
    
    To implement it, we added a new distribution policy named POLICYTYPE_REPLICATED to mark
    a replicated table and added a new locus type named CdbLocusType_SegmentGeneral to specify
    the distribution of tuples of a replicated table.  CdbLocusType_SegmentGeneral implies
    data is generally available on all segments but not available on qDisp, so plan node with
    this locus type can be flexibly planned to execute on either single QE or all QEs. it is
    similar with CdbLocusType_General, the only difference is that CdbLocusType_SegmentGeneral
    node can't be executed on qDisp. To guarantee this, we try our best to add a gather motion
    on the top of a CdbLocusType_SegmentGeneral node when planing motion for join even other
    rel has bottleneck locus type, a problem is such motion may be redundant if the single QE
    is not promoted to executed on qDisp finally, so we need to detect such case and omit the
    redundant motion at the end of apply_motion(). We don't reuse CdbLocusType_Replicated since
    it's always implies a broadcast motion bellow, it's not easy to plan such node as direct
    dispatch to avoid getting duplicate data.
    
    We don't support replicated table with inherit/partition by clause now, the main problem is
    that update/delete on multiple result relations can't work correctly now, we can fix this
    later.
    
    * Allow spi_* to access replicated table on QE
    
    Previously, GPDB didn't allow QE to access non-catalog table because the
    data is incomplete,
    we can remove this limitation now if it only accesses replicated table.
    
    One problem is QE need to know if a table is replicated table,
    previously, QE didn't maintain
    the gp_distribution_policy catalog, so we need to pass policy info to QE
    for replicated table.
    
    * Change schema of gp_distribution_policy to identify replicated table
    
    Previously, we used a magic number -128 in gp_distribution_policy table
    to identify replicated table which is quite a hack, so we add a new column
    in gp_distribution_policy to identify replicated table and partitioned
    table.
    
    This commit also abandon the old way that used 1-length-NULL list and
    2-length-NULL list to identify DISTRIBUTED RANDOMLY and DISTRIBUTED
    FULLY clause.
    
    Beside, this commit refactor the code to make the decision-making of
    distribution policy more clear.
    
    * support COPY for replicated table
    
    * Disable row ctid unique path for replicated table.
      Previously, GPDB use a special Unique path on rowid to address queries
      like "x IN (subquery)", For example:
      select * from t1 where t1.c2 in (select c2 from t3), the plan looks
      like:
       ->  HashAggregate
             Group By: t1.ctid, t1.gp_segment_id
                ->  Hash Join
                      Hash Cond: t2.c2 = t1.c2
                    ->  Seq Scan on t2
                    ->  Hash
                        ->  Seq Scan on t1
    
      Obviously, the plan is wrong if t1 is a replicated table because ctid
      + gp_segment_id can't identify a tuple, in replicated table, a logical
      row may have different ctid and gp_segment_id. So we disable such plan
      for replicated table temporarily, it's not the best way because rowid
      unique way maybe the cheapest plan than normal hash semi join, so
      we left a FIXME for later optimization.
    
    * ORCA related fix
      Reported and added by Bhuvnesh Chaudhary <bchaudhary@pivotal.io>
      Fallback to legacy query optimizer for queries over replicated table
    
    * Adapt pg_dump/gpcheckcat to replicated table
      gp_distribution_policy is no longer a master-only catalog, do
      same check as other catalogs.
    
    * Support gpexpand on replicated table && alter the dist policy of replicated table
    7efe3204
cdbcopy.c 19.7 KB