- 29 5月, 2019 8 次提交
-
-
由 Jesse Zhang 提交于
- ubuntu jobs are using the systems compiler and the Docker build image does not contain the GCC 6.4.0 package; CI scripts are modified to conditionally source the gcc environment setup script - when running `make demo-cluster` on ubuntu, it looks like `su` does not preserve the environment set by "greenplum_path.sh" so we re-source "greenplum_path.sh" as gpadmin before starting the demo cluster Co-authored-by: NJesse Zhang <jzhang@pivotal.io> Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io> (cherry picked from commit 7708a43b)
-
由 Bradford D. Boyle 提交于
- add ubuntu18.04 as a choice for OS type - include ubuntu18.04 as an OS type for prod pipelines - python, sigar, and zstd build dependencies are included in the gpdb6-ubuntu18.04-build image - the ubuntu docker image has sigar and python installed already and there is no libquicklz-devel package for ubuntu so these inputs have been marked optional in the task - the configure flags were copied from the RHEL7 configure flags - update functions `include_quicklz` and `include_zstd` to handle differences in where the libraries are installed on RHEL7 and Ubuntu platforms - change mv to cp to make local iteration easier; this might also be slightly more performant on Concourse w/ the overlay volumes Co-authored-by: NGoutam Tadi <gtadi@pivotal.io> Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io> Co-authored-by: NJesse Zhang <jzhang@pivotal.io> (cherry picked from commit 6e92c42b)
-
由 Shaoqi Bai 提交于
* Remove supporting architectures for AIX aix5_ppc_64, aix5_ppc_32 is no longer in use, remove them https://groups.google.com/a/pivotal.io/forum/#!searchin/gp-releng/sync_tools%7Csort:date/gp-releng/-uu_bR3Sx2s/bk1nebiu * Another tiny clean up in gppkg This is a follow up clean up for https://github.com/greenplum-db/gpdb/pull/7727Reviewed-by: NLav Jain <ljain@pivotal.io> Reviewed-by: NBradford D. Boyle <bboyle@pivotal.io> (cherry picked from commit e2b8801d)
-
由 Daniel Gustafsson 提交于
There are no callers of this, and there seems little use fot it, so remove. Reviewed-by: NBen Christel <bchristel@pivotal.io> Reviewed-by: NBradford Boyle <bboyle@pivotal.io> (cherry picked from commit 2c275d1c)
-
由 Daniel Gustafsson 提交于
While it works for development and use, macOS is not a supported platform for Greenplum release engineering. The release engineering support for macOS was also stuck in 32-bit 10.6 land which is long since EOL'd. This removes all leftover macOS bits, while ensuring that building and running Greenplum on macOS still works. Reviewed-by: NBen Christel <bchristel@pivotal.io> Reviewed-by: NBradford Boyle <bboyle@pivotal.io> (cherry picked from commit 7be4dbb7)
-
由 Daniel Gustafsson 提交于
Commit 52c37372 removed leftovers from the no-longer maintained Solaris and HP-UX support for Greenplum. This commit removes a few more pieces found here and there. Reviewed-by: NBen Christel <bchristel@pivotal.io> Reviewed-by: NBradford Boyle <bboyle@pivotal.io> (cherry picked from commit 3417f72a)
-
由 Oliver Albertini 提交于
It's a Bourne Again script, Not a Bourne script. Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io> Co-authored-by: NOliver Albertini <oalbertini@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io> Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io> (cherry picked from commit ad1e76f2)
-
由 Oliver Albertini 提交于
The logic hasn't actually changed much (we are still validating the the postgres version and gpdb_src SHA match), but has become clearer and easier to follow: * avoids use of globals * separates assert function into two to have clearer scope separation Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io> Co-authored-by: NOliver Albertini <oalbertini@pivotal.io> Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io> Co-authored-by: NBen Christel <bchristel@pivotal.io> (cherry picked from commit cf181f38)
-
- 28 5月, 2019 1 次提交
-
-
由 Daniel Gustafsson 提交于
INCLUDING STORAGE was introduced in PostgreSQL 9.0, but support for setting the AO/AOCS/ENCODING storage options on the created table was never supported as we merged upstream. This extends support for copying over the table storage type as well as attribute encodings when creating a table with INCLUDING STORAGE or ALL. Backported from master commit 951065c4 Reported-by: Cyrille Lintz Reviewed-by: Jimmy Yih Reviewed-by: Asim R P
-
- 27 5月, 2019 5 次提交
-
-
由 Hubert Zhang 提交于
In gpexpand behave test there is a flaky failed test in "verify that the cluster has 8 new segments". We suspect some segments may be down after expansion. So we dump the gp_segment_configuration on that exception to confirm. Co-authored-by: NNing Yu <nyu@pivotal.io> Reviewed-by: NZhenghua Lyu <zlv@pivotal.io>
-
由 Ning Yu 提交于
The backend could set extra data on a vert, such as the raw data collected by the UDF. Also let GDD return each newly created vert so the backend has a chance to set the data.
-
由 Ning Yu 提交于
These dump functions are not part of the GDD algorithm, and they access the backend information, move them from gdddetector.c to gddbackend.c, so gdddetector.c is more focused on the algorithm.
-
由 Ning Yu 提交于
gdddetector.c contains the core GDD algorithm, we used to put the structs in it to hide the implementation details, the purpose was to simplify the detector's usage by providing a small set of carefully designed APIs. However some non-algorithm-related helpers, such as the dump functions, have to be put into this source file to access the struct internals, which increase the effort to understand and maintain the algorithm. So now we move the structs to gdddetectorpriv.h, the name still suggests that it is for private use, however when necessary the callers could access the struct internals by including this header.
-
由 Hubert Zhang 提交于
GPDB uses the high three bits of protocol version to indicate this is internal communication. Previous merge forgets to set these three bits when re-initialize connection. Co-authored-by: NNing Yu <nyu@pivotal.io> Reviewed-by: NZhenghua Lyu <zlv@pivotal.io> Reviewed-by: NAsim R P <apraveen@pivotal.io>
-
- 25 5月, 2019 10 次提交
-
-
由 Ashuka Xue 提交于
-
由 Jamie McAtamney 提交于
Check that the target passed in is valid by comparing it against secrets filenames. Co-authored-by: Jamie McAtamney <jmcatamney@pivotal.io Co-authored-by: Kalen Krempely <kkrempely@pivotal.io
-
由 Kalen Krempely 提交于
When branching gpdb simply update the gen_pipeline branch variable to freeze to the appropriate pipeline version.
-
由 Kalen Krempely 提交于
Rename pipeline_type to pipeline_target for clarity now that teams have their own concourse instance and the -t argument specifys the fly target. Also add two new flags: --pipeline_configuration, to make it easy for teams to make dev pipelines that match the prod pipeline, and --use_branch, to make it easier to have pipelines in flight for multiple branches without overlap. Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io> Co-authored-by: NShoaib Lari <slari@pivotal.io>
-
由 Kalen Krempely 提交于
Remove unneeded shell=True from subprocess command. This is safer to use as it does not rely on calling an unknown shell. Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
-
由 Kalen Krempely 提交于
Update gen_pipeline to take a team target for the pipeline_type parameter, which then uses the appropriate team's variable file. Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
-
由 Shoaib Lari 提交于
The master version of gen_pipeline.py was refactored for better formatting and clarity, and this commit backports that refactor. Co-authored-by: NShoaib Lari <slari@pivotal.io> Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
-
由 Adam Berlin 提交于
Cherry-pick of commit (b4bf1ed1)
-
由 Adam Berlin 提交于
We cannot ensure that these warnings will appear for all users because they depend on where their greenplum source directory is located. Therefore, ignore these errors. They are tested in the main regress suite also. (cherry picked from commit 55aeae18)
-
由 Adam Berlin 提交于
TAR has a limitation that symlink targets must be shorter than 100 characters. Base backup will truncate the symlink target path beyond 100 characters to match the TAR spec. This change introduces a warning to the user while creating a tablespace with some hints for how to avoid the problem. (cherry picked from commit 0990f748)
-
- 24 5月, 2019 8 次提交
-
-
由 Hubert Zhang 提交于
PGC_BACKEND means GUC could be set through PGOPTIONS in libpq which is the same as how we pass GUC from QD to QE. Reviewed-by: NAsim R P <apraveen@pivotal.io>
-
由 Soumyadeep Chakraborty 提交于
* Add tests around pg_rewind's behavior with tablespaces. * pg_rewind now lops off the source dbid from the symlink target before populating the filemap. * pg_rewind now appends the target dbid to the symlink target when creating it and also creates the symlink target directory. Both of these actions are performed as a separate action: `create_tablespace_layout`. We distinguish tablespace symlinks with the `is_gp_tablespace` flag. * pg_rewind now fetches the target dbid by invoking the postgres binary. * Add test for pg_rewind with tablespaces which tests for objects created after promotion of the replica. We introduced a new hook in the pg_rewind suite: `before_master_restart_after_rewind`, which executes right after pg_rewind completes and the master is restarted. * Add test for pg_rewind with tablepace objects created before standby promotion. * Add pg_rewind test for objects removed after standby promotion * Enable gprecoverseg tests around tablespaces that were waiting on these changes. Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> Co-authored-by: NAdam Berlin <aberlin@pivotal.io> Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> (cherry picked from commit 55c55259)
-
由 Taylor Vesely 提交于
In GPDB when you take a pg_basebackup in tar mode, the segment will not have the internal.auto.conf file, nor will any tablespaces have the dbid appended to their symlinks in pg_tblspc. The backups are still, in theory, valid, but the tablespace mapping and internal.auto.conf files will need to be added manually when extracting the backups. Co-authored-by: NAdam Berlin <aberlin@pivotal.io> (cherry picked from commit 0c639fa2)
-
由 Taylor Vesely 提交于
There was a race condition where replay of a DROP TABLESPACE command on a replication mirror could occur after the tests had already physically removed the directories. It should be safe if the tablespace directories continue to exist after the test is complete, so don't clean them up. (cherry picked from commit e363efdb1596d38f525b22991404d8aa2f6d30cf)
-
由 Taylor Vesely 提交于
Force the mirror to create a restartpoint, and as a side-effect replay the DROP TABLESPACE DDL before removing the tablespace directory. Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io> (cherry picked from commit 365d85075f6e15cf9e46aa8c1b8f9d436464d87f)
-
由 Soumyadeep Chakraborty 提交于
The test was intended to confirm that pg_basebackup wouldn't overwrite and existing tablespace directory (without --force-overwrite) Before: The command was failing because -1 is an invalid dbid Now: The command is failing as originally intended. Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> (cherry picked from commit 7956582fe51fc22475f81e0f8abebdf095dc1d35)
-
由 Alexandra Wang 提交于
The server now removes the source dbid from the end of any tablespace symlink target found in its response to BASE_BACKUP: This means that it removes it from the tablespace header as well as any tar directory entries for a given tablespace. The pg_basebackup client now adds the target dbid at the end of the symlink target returned from the server response in order to create the correct symlink: <target_datadir>/pg_tblspc/<tablespace_oid> -> <tablespace_location>/<target_db_id> Note: All of the above is applicable to user-defined tablespaces. Also, we renamed some of the ddl objects in the isolation2 tests for tablespaces as they were too long. Also, reducing the length of the LOCATION clause of the tablespace object helped us avoid the tar header limit of 100 characters for symlinks. Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io> Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> Co-authored-by: NAdam Berlin <aberlin@pivotal.io> (cherry picked from commit e81dd542edc0a5c39b68964d6bc6548fab4ffd45)
-
由 Taylor Vesely 提交于
This commit includes changes to the server to ensure that the utilities: pg_rewind and pg_basebackup can be changed to support recovery in a multi-segment-singular-host setting. We link pg_tblspc to a <dbid> subdirectory of the tablespace, rather than to the path of the tablespace directly, and we remove the <dbid> from the tablespace version directory. At the same time, we have designed towards preserving the response to pg_tablespace_location(<tablespace_oid>) such that it does not return the dbid suffix. The design is such that it is the responsibility of the utilities to append the dbid as and when required. Before this commit: * the symlink to the tablespace directory looks like: pg_tblspc/spcoid/ -> /<tablespace_location>/ * Under the symlink target, we would have the following: GPDB_MAJORVER_CATVER_db<dbid>/dboid/relfilenode * pg_tablespace_location(tsoid) returns: <tablespace_location> e.g. * pg_tblspc/20981/ -> /data1/tsp1 * Under /data1/tsp1: GPDB_6_201902061_db1/19849/192814 * pg_tablespace_location(20981) returns: /data1/tsp1 After this commit: * the symlink to the tablespace directory looks like: pg_tblspc/spcoid/ -> /<tablespace_location>/<dbid> * Under the symlink target, we would have the following: GPDB_MAJORVER_CATVER/dboid/relfilenode * pg_tablespace_location(tsoid) returns: <tablespace_location> e.g. * pg_tblspc/20981/ -> /data1/tsp1/1 * Under /data1/tsp1/1: GPDB_6_201902061/19849/192814 * pg_tablespace_location(20981) returns: /data1/tsp1 Motivation: When tablespaces were aligned to upstream postgres, while removing filespaces, we added the `tablespace_version_directory()` function to supply each segment with a unique tablespace directory name. This was accomplished by appending the 'magic' `GpIdentity.dbid` global variable to the `GP_TABLESPACE_VERSION_DIRECTORY` in `tablespace_version_directory()`. This is problematic for several reasons- but perhaps most severely is the fact that in order to use any code in libpgcommon.so that references this value, you need to first set the `GpIdentity.dbid` global, otherwise any functions that deal with tablespaces will be broken in unpredictable ways. An example is pg_rewind- where `GetRelationPath()` will not return a valid relation unless you repeatedly toggle the `GpIdentity.dbid` between the value of the source or target segment dependant on the context of which relfiles are being examined. This commit bumps the catalog version here we have made breaking changes in the tablespace filesystem layout. Co-authored-by: NAdam Berlin <aberlin@pivotal.io> Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> Co-authored-by: NSoumyadeep Chakraborty <sochakraborty@pivotal.io> (cherry picked from commit df36eabc33fec728a16a0e4d1979eb6ff889d789)
-
- 23 5月, 2019 7 次提交
-
-
由 Asim R P 提交于
Where close enough is defined in terms of number of WAL segments, using GUC repl_catchup_within_range. The test aims to validate this Greenplum specific addition, only applicable to master/standby WAL replication. Reviewed by: Jimmy Yih <jyih@pivotal.io> (cherry picked from commit 46c74111)
-
由 Asim R P 提交于
Master/standby WAL replication differs from primary/mirror, mainly in the fact that there is no third party such as FTS to monitor replication state. This test validates that commits on master are blocked until standby confirms that WAL upto commit LSN has been received and flushed. The test injects a fault on standby. Reviewed by: Jimmy Yih <jyih@pivotal.io> (cherry picked from commit a9414861)
-
由 Asim R P 提交于
Faults can now be injected by sending a libpq message to postmaster, even if the postmaster is acting as a mirror/standby. The client must set "gpconntype=fault" in libpq connection string in order to establish a fault injection connection. The fault details are sent as a separate message. This is similar to how FTS probe messages are sent. A nice consequence is tests no longer have to rely on NOTICE messages emitted by gp_inject_fault(). The new fault injection function returns a text datum, which is easy to record and validate in answer files. The old fault injection interface is left unchanged for now and the new interface is added as "gp_inject_fault2()". Replacing the old interface and updating all tests and answer files can be done as a follow up patch. Reviewed by: Jimmy Yih <jyih@pivotal.io> (cherry picked from commit 89791f14)
-
由 Jimmy Yih 提交于
This was caught in the 9.5 merge iteration where reader/writer slices were blocking each other because the ModifyTableState node type was not being handled in the recursion. In master, this issue is somehow being masked by planner by running an unneeded redistribute motion. For example, on master: QUERY PLAN -------------------------------------------------------------------------------------------------------------- Insert on public.newfoo (cost=0.85..4.62 rows=4 width=8) -> Hash Join (cost=0.85..4.53 rows=4 width=8) Output: subfoo.c, share0_ref1.a Hash Cond: (subfoo.c = share0_ref1.b) -> Redistribute Motion 3:3 (slice1; segments: 3) (cost=0.33..3.86 rows=4 width=4) Output: subfoo.c, subfoo.a Hash Key: subfoo.c -> Subquery Scan on subfoo (cost=0.33..3.66 rows=4 width=4) Output: subfoo.c, subfoo.a -> Hash Join (cost=0.33..3.56 rows=4 width=8) Output: cte.a, bar.c Hash Cond: (bar.c = cte.a) -> Seq Scan on public.bar (cost=0.00..3.10 rows=4 width=4) Output: bar.c -> Hash (cost=0.20..0.20 rows=4 width=4) Output: cte.a -> Subquery Scan on cte (cost=0.00..0.20 rows=4 width=4) Output: cte.a -> Shared Scan (share slice:id 1:0) (cost=3.15..3.36 rows=4 width=8) Output: share0_ref2.a, share0_ref2.b -> Hash (cost=0.40..0.40 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b -> Redistribute Motion 3:3 (slice2; segments: 3) (cost=0.00..0.40 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b Hash Key: share0_ref1.b -> Shared Scan (share slice:id 2:0) (cost=3.15..3.36 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b -> Materialize (cost=0.00..3.15 rows=4 width=8) Output: foo.a, foo.b -> Seq Scan on public.foo (cost=0.00..3.10 rows=4 width=8) Output: foo.a, foo.b Optimizer: Postgres query optimizer Settings: gp_cte_sharing=on (33 rows) vs. on 9.5 merge branch: QUERY PLAN -------------------------------------------------------------------------------------------------------- Insert on public.newfoo (cost=0.85..4.43 rows=4 width=8) -> Hash Join (cost=0.85..4.33 rows=4 width=8) Output: subfoo.c, share0_ref1.a Hash Cond: (subfoo.c = share0_ref1.b) -> Subquery Scan on subfoo (cost=0.33..3.66 rows=4 width=4) Output: subfoo.c -> Hash Join (cost=0.33..3.56 rows=4 width=4) Output: NULL::integer, bar.c Hash Cond: (bar.c = cte.a) -> Seq Scan on public.bar (cost=0.00..3.10 rows=4 width=4) Output: bar.c -> Hash (cost=0.20..0.20 rows=4 width=4) Output: cte.a -> Subquery Scan on cte (cost=0.00..0.20 rows=4 width=4) Output: cte.a -> Shared Scan (share slice:id 0:0) (cost=3.15..3.36 rows=4 width=8) Output: share0_ref2.a, share0_ref2.b -> Hash (cost=0.40..0.40 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b -> Redistribute Motion 3:3 (slice1; segments: 3) (cost=0.00..0.40 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b Hash Key: share0_ref1.b -> Shared Scan (share slice:id 1:0) (cost=3.15..3.36 rows=4 width=8) Output: share0_ref1.a, share0_ref1.b -> Materialize (cost=0.00..3.15 rows=4 width=8) Output: foo.a, foo.b -> Seq Scan on public.foo (cost=0.00..3.10 rows=4 width=8) Output: foo.a, foo.b Optimizer: Postgres query optimizer Settings: gp_cte_sharing=on (30 rows) However, this issue could manifest depending on the plan costing so we should fix this on master and 6X_STABLE. Co-authored-by: NJesse Zhang <sbjesse@gmail.com>
- 22 5月, 2019 1 次提交
-