Skip to content

G
Gpdb

Greenplum / Gpdb

通知 7
Star 1
Fork 0
G
Gpdb

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 29 5月, 2019 8 次提交
  3. 28 5月, 2019 1 次提交
    • D
      Support AO/ENCODING in CREATE TABLE .. LIKE INCLUDING · 538059a8
      由 Daniel Gustafsson 提交于 
      INCLUDING STORAGE was introduced in PostgreSQL 9.0, but support for
setting the AO/AOCS/ENCODING storage options on the created table
was never supported as we merged upstream. This extends support for
copying over the table storage type as well as attribute encodings
when creating a table with INCLUDING STORAGE or ALL.

Backported from master commit 951065c4

Reported-by: Cyrille Lintz
Reviewed-by: Jimmy Yih
Reviewed-by: Asim R P
      538059a8
  5. 27 5月, 2019 5 次提交
  7. 25 5月, 2019 10 次提交
  9. 24 5月, 2019 8 次提交
    • H
      GUC with PGC_BACKEND context should also be dispatched to QE (#7741) (#7762) · 2a732f72
      由 Hubert Zhang 提交于 
      PGC_BACKEND means GUC could be set through PGOPTIONS in libpq
which is the same as how we pass GUC from QD to QE.
Reviewed-by: NAsim R P <apraveen@pivotal.io>
      2a732f72
    • S
      Modify pg_rewind to handle new tablespace layout. · 69312401
      由 Soumyadeep Chakraborty 提交于 
      * Add tests around pg_rewind's behavior with tablespaces.
* pg_rewind now lops off the source dbid from the symlink target before
populating the filemap.
* pg_rewind now appends the target dbid to the symlink target when
creating it and also creates the symlink target directory. Both of these
actions are performed as a separate action: `create_tablespace_layout`.
We distinguish tablespace symlinks with the `is_gp_tablespace` flag.
* pg_rewind now fetches the target dbid by invoking the postgres binary.
* Add test for pg_rewind with tablespaces which tests for objects
created after promotion of the replica. We introduced a new hook in the
pg_rewind suite: `before_master_restart_after_rewind`, which executes
right after pg_rewind completes and the master is restarted.
* Add test for pg_rewind with tablepace objects created before standby
promotion.
* Add pg_rewind test for objects removed after standby promotion
* Enable gprecoverseg tests around tablespaces that were waiting on
these changes.
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
(cherry picked from commit 55c55259)
      69312401
    • T
      pg_basebackup: WARN when running in TAR mode. · 85d62cf8
      由 Taylor Vesely 提交于 
      In GPDB when you take a pg_basebackup in tar mode, the segment will not
have the internal.auto.conf file, nor will any tablespaces have the dbid
appended to their symlinks in pg_tblspc.  The backups are still, in
theory, valid, but the tablespace mapping and internal.auto.conf files
will need to be added manually when extracting the backups.
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
(cherry picked from commit 0c639fa2)
      85d62cf8
    • T
      isolation2: Refactor tablespace related tests · e9a254d6
      由 Taylor Vesely 提交于 
      There was a race condition where replay of a DROP TABLESPACE command on
a replication mirror could occur after the tests had already physically
removed the directories. It should be safe if the tablespace directories
continue to exist after the test is complete, so don't clean them up.

(cherry picked from commit e363efdb1596d38f525b22991404d8aa2f6d30cf)
      e9a254d6
    • T
      isolation2: Fix race condition in mirror_promotion · 91fe1c3e
      由 Taylor Vesely 提交于 
      Force the mirror to create a restartpoint, and as a side-effect replay
the DROP TABLESPACE DDL before removing the tablespace directory.
Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io>
(cherry picked from commit 365d85075f6e15cf9e46aa8c1b8f9d436464d87f)
      91fe1c3e
    • S
      pg_basebackup: Fix incorrectly passing TAP test for tablespaces · d7614bb8
      由 Soumyadeep Chakraborty 提交于 
      The test was intended to confirm that pg_basebackup wouldn't overwrite
and existing tablespace directory (without --force-overwrite)
Before: The command was failing because -1 is an invalid dbid
Now: The command is failing as originally intended.
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
(cherry picked from commit 7956582fe51fc22475f81e0f8abebdf095dc1d35)
      d7614bb8
    • A
      pg_basebackup: Use new tablespace layout · 092ee750
      由 Alexandra Wang 提交于 
      The server now removes the source dbid from the end of any tablespace symlink
target found in its response to BASE_BACKUP: This means that it removes it from
the tablespace header as well as any tar directory entries for a given
tablespace.  The pg_basebackup client now adds the target dbid at the end of
the symlink target returned from the server response in order to create the
correct symlink:
<target_datadir>/pg_tblspc/<tablespace_oid> -> <tablespace_location>/<target_db_id>

Note: All of the above is applicable to user-defined tablespaces.

Also, we renamed some of the ddl objects in the isolation2 tests for
tablespaces as they were too long. Also, reducing the length of the LOCATION
clause of the tablespace object helped us avoid the tar header limit of 100
characters for symlinks.
Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
(cherry picked from commit e81dd542edc0a5c39b68964d6bc6548fab4ffd45)
      092ee750
    • T
      Introduce new tablespace directory layout. · 19829deb
      由 Taylor Vesely 提交于 
      This commit includes changes to the server to ensure that the utilities:
pg_rewind and pg_basebackup can be changed to support recovery in a
multi-segment-singular-host setting. We link pg_tblspc to a <dbid>
subdirectory of the tablespace, rather than to the path of the
tablespace directly, and we remove the <dbid> from the tablespace
version directory. At the same time, we have designed towards preserving
the response to pg_tablespace_location(<tablespace_oid>) such that it
does not return the dbid suffix. The design is such that it is the
responsibility of the utilities to append the dbid as and when required.

Before this commit:
  * the symlink to the tablespace directory looks like:
      pg_tblspc/spcoid/ -> /<tablespace_location>/
  * Under the symlink target, we would have the following:
    GPDB_MAJORVER_CATVER_db<dbid>/dboid/relfilenode
  * pg_tablespace_location(tsoid) returns: <tablespace_location>
e.g.
  * pg_tblspc/20981/ -> /data1/tsp1
  * Under /data1/tsp1: GPDB_6_201902061_db1/19849/192814
  * pg_tablespace_location(20981) returns: /data1/tsp1

After this commit:
  * the symlink to the tablespace directory looks like:
      pg_tblspc/spcoid/ -> /<tablespace_location>/<dbid>
  * Under the symlink target, we would have the following:
      GPDB_MAJORVER_CATVER/dboid/relfilenode
  * pg_tablespace_location(tsoid) returns: <tablespace_location>

e.g.
  * pg_tblspc/20981/ -> /data1/tsp1/1
  * Under /data1/tsp1/1: GPDB_6_201902061/19849/192814
  * pg_tablespace_location(20981) returns: /data1/tsp1

Motivation:

When tablespaces were aligned to upstream postgres, while removing
filespaces, we added the `tablespace_version_directory()` function to
supply each segment with a unique tablespace directory name. This was
accomplished by appending the 'magic' `GpIdentity.dbid` global variable
to the `GP_TABLESPACE_VERSION_DIRECTORY` in `tablespace_version_directory()`.

This is problematic for several reasons- but perhaps most severely is
the fact that in order to use any code in libpgcommon.so that references
this value, you need to first set the `GpIdentity.dbid` global,
otherwise any functions that deal with tablespaces will be broken in
unpredictable ways.

An example is pg_rewind- where `GetRelationPath()` will not return a valid
relation unless you repeatedly toggle the `GpIdentity.dbid` between the
value of the source or target segment dependant on the context of which
relfiles are being examined.

This commit bumps the catalog version here we have made breaking changes
in the tablespace filesystem layout.
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io>
(cherry picked from commit df36eabc33fec728a16a0e4d1979eb6ff889d789)
      19829deb
  11. 23 5月, 2019 7 次提交
    • A
      An infinite_loop fault should break when reset. · 17051070
      由 Asim R P 提交于 
      This commit fixes fault injector such that when a infinte_loop fault is reset,
the infinite loop is broken and the fault is removed, as expected.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit 44ff535c)
      17051070
    • A
      Test that commits block when standby has caughtup close enough · 3b70af3c
      由 Asim R P 提交于 
      Where close enough is defined in terms of number of WAL segments, using GUC
repl_catchup_within_range.  The test aims to validate this Greenplum specific
addition, only applicable to master/standby WAL replication.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit 46c74111)
      3b70af3c
    • A
      Test that commits block until standby flushes commit LSN · 58846100
      由 Asim R P 提交于 
      Master/standby WAL replication differs from primary/mirror, mainly in
the fact that there is no third party such as FTS to monitor
replication state.  This test validates that commits on master are
blocked until standby confirms that WAL upto commit LSN has been
received and flushed.  The test injects a fault on standby.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit a9414861)
      58846100
    • A
      Refactor fault injector to remove redundant code · faa7ddb0
      由 Asim R P 提交于 
      And some cosmetic changes to conform to PostgreSQL coding style.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit 032b0a30)
      faa7ddb0
    • A
      Inject faults by sending a libpq message · b3c3e909
      由 Asim R P 提交于 
      Faults can now be injected by sending a libpq message to postmaster, even if
the postmaster is acting as a mirror/standby.  The client must set
"gpconntype=fault" in libpq connection string in order to establish a fault
injection connection.  The fault details are sent as a separate message.  This
is similar to how FTS probe messages are sent.

A nice consequence is tests no longer have to rely on NOTICE messages emitted
by gp_inject_fault().  The new fault injection function returns a text datum,
which is easy to record and validate in answer files.

The old fault injection interface is left unchanged for now and the new
interface is added as "gp_inject_fault2()".  Replacing the old interface and
updating all tests and answer files can be done as a follow up patch.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit 89791f14)
      b3c3e909
    • A
      Avoid initializing StringInfo twice · 727c07a2
      由 Asim R P 提交于 
      Libpq messages are constructed using pq_beginmessage(), which
initializes StringInfo object passed to it and pq_endmessage() frees
it.  No need for explicit initialization, it must have been an
oversight.

Reviewed by: Jimmy Yih <jyih@pivotal.io>

(cherry picked from commit 39d0926e)
      727c07a2
    • J
      Add logic to handle ModifyTableState in slice dependency · ccc682d8
      由 Jimmy Yih 提交于 
      This was caught in the 9.5 merge iteration where reader/writer slices
were blocking each other because the ModifyTableState node type was
not being handled in the recursion. In master, this issue is somehow
being masked by planner by running an unneeded redistribute motion.

For example, on master:
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Insert on public.newfoo  (cost=0.85..4.62 rows=4 width=8)
   ->  Hash Join  (cost=0.85..4.53 rows=4 width=8)
         Output: subfoo.c, share0_ref1.a
         Hash Cond: (subfoo.c = share0_ref1.b)
         ->  Redistribute Motion 3:3  (slice1; segments: 3)  (cost=0.33..3.86 rows=4 width=4)
               Output: subfoo.c, subfoo.a
               Hash Key: subfoo.c
               ->  Subquery Scan on subfoo  (cost=0.33..3.66 rows=4 width=4)
                     Output: subfoo.c, subfoo.a
                     ->  Hash Join  (cost=0.33..3.56 rows=4 width=8)
                           Output: cte.a, bar.c
                           Hash Cond: (bar.c = cte.a)
                           ->  Seq Scan on public.bar  (cost=0.00..3.10 rows=4 width=4)
                                 Output: bar.c
                       ->  Hash  (cost=0.20..0.20 rows=4 width=4)
                                 Output: cte.a
                                 ->  Subquery Scan on cte  (cost=0.00..0.20 rows=4 width=4)
                                       Output: cte.a
                                       ->  Shared Scan (share slice:id 1:0)  (cost=3.15..3.36 rows=4 width=8)
                                             Output: share0_ref2.a, share0_ref2.b
         ->  Hash  (cost=0.40..0.40 rows=4 width=8)
               Output: share0_ref1.a, share0_ref1.b
               ->  Redistribute Motion 3:3  (slice2; segments: 3)  (cost=0.00..0.40 rows=4 width=8)
                     Output: share0_ref1.a, share0_ref1.b
                     Hash Key: share0_ref1.b
                     ->  Shared Scan (share slice:id 2:0)  (cost=3.15..3.36 rows=4 width=8)
                           Output: share0_ref1.a, share0_ref1.b
                           ->  Materialize  (cost=0.00..3.15 rows=4 width=8)
                                 Output: foo.a, foo.b
                                 ->  Seq Scan on public.foo  (cost=0.00..3.10 rows=4 width=8)
                                       Output: foo.a, foo.b
 Optimizer: Postgres query optimizer
 Settings: gp_cte_sharing=on
(33 rows)

vs. on 9.5 merge branch:
                                               QUERY PLAN
--------------------------------------------------------------------------------------------------------
 Insert on public.newfoo  (cost=0.85..4.43 rows=4 width=8)
   ->  Hash Join  (cost=0.85..4.33 rows=4 width=8)
         Output: subfoo.c, share0_ref1.a
         Hash Cond: (subfoo.c = share0_ref1.b)
         ->  Subquery Scan on subfoo  (cost=0.33..3.66 rows=4 width=4)
               Output: subfoo.c
               ->  Hash Join  (cost=0.33..3.56 rows=4 width=4)
                     Output: NULL::integer, bar.c
                     Hash Cond: (bar.c = cte.a)
                     ->  Seq Scan on public.bar  (cost=0.00..3.10 rows=4 width=4)
                           Output: bar.c
                     ->  Hash  (cost=0.20..0.20 rows=4 width=4)
                       Output: cte.a
                           ->  Subquery Scan on cte  (cost=0.00..0.20 rows=4 width=4)
                                 Output: cte.a
                                 ->  Shared Scan (share slice:id 0:0)  (cost=3.15..3.36 rows=4 width=8)
                                       Output: share0_ref2.a, share0_ref2.b
         ->  Hash  (cost=0.40..0.40 rows=4 width=8)
               Output: share0_ref1.a, share0_ref1.b
               ->  Redistribute Motion 3:3  (slice1; segments: 3)  (cost=0.00..0.40 rows=4 width=8)
                     Output: share0_ref1.a, share0_ref1.b
                     Hash Key: share0_ref1.b
                     ->  Shared Scan (share slice:id 1:0)  (cost=3.15..3.36 rows=4 width=8)
                           Output: share0_ref1.a, share0_ref1.b
                           ->  Materialize  (cost=0.00..3.15 rows=4 width=8)
                                 Output: foo.a, foo.b
                                 ->  Seq Scan on public.foo  (cost=0.00..3.10 rows=4 width=8)
                                       Output: foo.a, foo.b
 Optimizer: Postgres query optimizer
 Settings: gp_cte_sharing=on
(30 rows)

However, this issue could manifest depending on the plan costing so we
should fix this on master and 6X_STABLE.
Co-authored-by: NJesse Zhang <sbjesse@gmail.com>
      ccc682d8
  13. 22 5月, 2019 1 次提交
    • A
      Fix \dd psql command · 321211d8
      由 Asim R P 提交于 
      This was most likely due to some mismerge during 9.2 cycle.  See
598f4b0e and e0aa3ef2.

Fixes GitHub issue #7712.

Reviewed by Daniel Gustafsson, Heikki Linnakangas and Jimmy Yih.

(cherry picked from commit cbce9d3b)
      321211d8