- 16 2月, 2019 4 次提交
-
-
由 Jacob Champion 提交于
base.join_and_indicate_progress() waits for the pool to complete its work, printing indication dots to stdout once per second. If it takes less than a second for the pool to complete, we won't print anything (and we also won't hang for a second waiting for nothing to happen). The previous implementation required the caller to store a running tally of how many commands had been added to the pool; that requirement is now dropped. Unlike wait_and_printdots(), join_and_indicate_progress() *always* prints to file. Don't call it if you don't want to print; use WorkerPool.join() directly.
-
由 Jacob Champion 提交于
The current status reporting methods are difficult to test (they try to do a little too much, IMO). Introduce a simpler solution -- allow join() to accept a timeout. All status reporting can now be implemented using this primitive.
-
由 Jacob Champion 提交于
WorkerPool needs some help, and we need some test coverage before I can fix and refactor.
-
由 Adam Berlin 提交于
This reverts commit 848733b6.
-
- 15 2月, 2019 8 次提交
-
-
由 Ning Yu 提交于
A duration can be set on gpexpand phase2, then it can quit before redistributed all the tables. There is behave tests to verify this, however they should check whether gpexpand quit on time.
-
由 Daniel Gustafsson 提交于
The :else clause on the for loop is superfluous as the loop doesn't contain any break statement. Removing it will yield the same codepath but makes for improved readability. This also removes an unused import (time) as well as fixes a set of typos. Reviewed-by: NJimmy Yih <jyih@pivotal.io>
-
由 Paul Guo 提交于
Also refactor subquery_motionHazard_walker() to make it simpler.
-
由 Ning Yu 提交于
Unless cancelled with ctrl-c the gpexpand redistribution phase's return code is always 0, it does not indicate whether the redistribution succeed or not, to get this information we need to check the status from gpexpand.status.
-
由 Ning Yu 提交于
Most temp tables won't live for long, there is no need to redistribute them. On the other hand if they are populated in gpexpand.status_detail and disappeared before redistribution, an error is reported to the user, this just causes unnecessary panic.
-
由 Taylor Vesely 提交于
Pull from upstream Postgres to make DefineIndex recursively create partitioned indexes. Instead of creating an individual IndexStmt for every partition, create indexes by recursing on the partition children. This aligns index creation with upstream in preparation for adding INTERNAL_AUTO relationships between partition indexes. * The QD will now choose the same name for partition indexes as Postgres. * Update tests to reflect the partition index names changes. * The changes to DefineIndex are mostly cherry-picked from Postgres commit: 8b08f7d4 * transformIndexStmt and its callers have been aligned with Postgres REL9_4_STABLE Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
-
由 Jesse Zhang 提交于
This reverts the following commits: commit 0ee987e64 - "Don't dispatch index creations too eagerly in ALTER TABLE." commit 28dd0152 - "Enable alter table column with index (#6286)" The motivation of commit 0ee987e64 is to stop eager dispatch of index creation during ALTER TABLE, and instead perform a single dispatch. Doing so prevents index name already exists errors when altering data types on indexed columns such as: ALTER TABLE foo ALTER COLUMN test TYPE integer; ERROR: relation "foo_test_key" already exists Unfortunately, without eager dispatch of index creation the QEs can choose a different name for a relation than was chosen on the QD. Eager dispatch was the only mechanism we had to ensure a deterministic and consistent index name between the QE and QD in some scenarios. In the absence of another mechanism we must revert this commit. This commit also rolls back commit 28dd0125 to enable altering data types on indexed columns, which required commit 0ee987e64. Co-authored-by: NKalen Krempely <kkrempely@pivotal.io> Co-authored-by: NTaylor Vesely <tvesely@pivotal.io> Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>
-
由 Ning Yu 提交于
UDPIFC receives data in a rx thread, errors in this thread cannot be raised directly, they are recorded in memory and the main thread is responsible to raise it at some proper time. It is possible for the rx thread to record an error after last TEARDOWN, technically it should be counted for last query, but there was a bug that the main thread raised it in next SETUP, so the new query failed immediately due to out-of-date errors. This is fixed by discarding any rx thread errors at beginning of SETUP.
-
- 14 2月, 2019 16 次提交
-
-
由 Georgios Kokolatos 提交于
Upstream added support for multiple kinds of external toast datums (commit 36820250). In the heart of the implementation, the representation of the inline portion of a short varlena was modified to store a tag describing the location of the datum pointed to, namely memory or disk. The tag took the place of the member which was describing the length of the datum pointed to. Since in upstream the length, for on disk datums, was always set to (VARHDRSZ_EXTERNAL + sizeof(struct varatt_external)), the enum for the corresponding tag was chosen to be exactly that, i.e. 18. In Greenplum, the exact same thing happens. However, due to historic reasons, there is an additional two byte padding in the struct. That representation has (and still is) been reflected in VARHDRSZ_EXTERNAL, which means that Greenplum has been storing a value for length two bytes longer than upstream. This commit updates the value of VARTAG_ONDISK to match the value that Greenplum has been historically storing. The other solution would have been to re-write the data during upgrade, but it seems unreasonably risky and evasive to do so. This commit also removes a comment that stated that Greenplum did not always set the length of the datum pointed to. Extensive search in the latest 5 and 4 versions of Greenplum did not found this to be true anymore. If it were, then some data rewriting would have been required while upgrading Greenplum. Previous versions of Greenplum, (pre-2009) have not been checked and it should be considered that data from those versions will break if binary upgraded to the current version. Removes a GPDB_94_MERGE_FIXME. Co-authored-by: NDaniel Gustafsson <dgustafsson@pivotal.io> Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
-
由 Paul Guo 提交于
After we have parameterized path since pg 9.2 and lateral (since pg9.3 although we do not support the full functionality), merge join path and hash join path need to consider that. Besides, for nestloop path, the previous code is wrong. 1. It did not allow motion for paths include index (path_contains_inner_index()). That is wrong. Here are two examples of index paths which allow motion. -> Broadcast Motion 3:3 (slice1; segments: 3) (cost=0.17..24735.67 rows=86100 width=0) -> Index Only Scan using t2i on t2 (cost=0.17..21291.67 rows=28700 width=0) -> Broadcast Motion 1:3 (slice1; segments: 1) (cost=0.17..6205.12 rows=259 width=8) -> Index Scan using t2i on t2 (cost=0.17..6201.67 rows=29 width=8) Index Cond: (4 = a) 2. The inner path and outer path might require upper nodes for parameterized paths so current code code bms_overlap(inner_req_outer, outer_path->parent->relids) is definitely not sufficient, besides, outer_path could have paramterized paths also. For nestloop join, case 1 is covered by the test case added in join_gp. For case 2, the test case in join.sql (although ignored) in this patch actually partially tested. Note the change in this patch is conservative. In theory, we could refer subplan code to allow broadcast for base rel if needed (for this solution no motion is needed), but that needs much effort and does not seem to be deserved given we will probably refactor related code for the lateral support in the near future.
-
由 Richard Guo 提交于
This removes several places of GPDB_94_MERGE_FIXME.
-
由 Richard Guo 提交于
Currently we conduct non-equivalent class deduction only from quals of inner join. This patch avoids adding outer join quals to non_eq_clauses from the beginning. Reviewed-by: NMelanie Plageman <mplageman@pivotal.io>
-
由 Sambitesh Dash 提交于
Co-authored-by: NSambitesh Dash <sdash@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
-
由 Sambitesh Dash 提交于
We now pull down the libsigar artifacts (.targz) from GCS for Centos instead of ivy. After removing all the ivy dependencies for Centos{6,7}, `make sync_tools` no longer creates {GPDB_SRC}/gpAux/ext/rhel{6,7}_x86_64 in the concourse instance. This commit creates the necessary directories for Centos{6,7} platform. Co-authored-by: NNandish Jayaram <njayaram@pivotal.io> Co-authored-by: NSambitesh Dash <sdash@pivotal.io> Signed-off-by: NSambitesh Dash <sdash@pivotal.io>
-
由 Karen Huddleston 提交于
These should have been removed when the dependencies were shifted from Ivy to the operating system in commit 07175e06, but we missed these. Note that that the copylibs target was copying the libraries from the system into gpdb binaries, but that was wrong so we stopped it. Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
-
由 Lisa Owen 提交于
* docs - add some ADL/WASBS/GC pxf external table DDL examples * identify (bold/text) what the user provides, use consistent server names * referencing -> that references
-
由 Mel Kiyama 提交于
docs - exchange partition, add limitation exchanging with a partitioned table is not supported. (#6961)
-
由 Heikki Linnakangas 提交于
The 'y' option was removed in PostgreSQL 8.3 already, but we missed that in the merge. While we're at it, fix the comments and whitespace to match upstream, so that bootstrap.c is now 100% identical to the 9.4.20 version. Reviewed-by: NDaniel Gustafsson <dgustafsson@pivotal.io>
-
由 Bradford D. Boyle 提交于
Builds on the master pipeline were erroring because the libsigar-installer resource was not added to all jobs that ran compile_gpdb. This reverts commit 15b28462. Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io> Co-authored-by: NBen Christel <bchristel@pivotal.io>
-
由 Sambitesh Dash 提交于
* Pull libsigar from GCS for Centos{6,7} We now pull down the libsigar artifacts (.targz) from GCS for Centos instead of ivy. Co-authored-by: NNandish Jayaram <njayaram@pivotal.io> Co-authored-by: NSambitesh Dash <sdash@pivotal.io> * Create necessary folders when building on Centos{6,7} After removing all the ivy dependencies for Centos{6,7}, `make sync_tools` no longer creates {GPDB_SRC}/gpAux/ext/rhel{6,7}_x86_64 in the concourse instance. This commit creates the necessary directories for Centos{6,7} platform. Co-authored-by: NSambitesh Dash <sdash@pivotal.io> * Address code review comments * Missed updating the compile_gpdb.bash
-
由 Jacob Champion 提交于
Following in the last commit's footsteps -- when /usr/bin/id is executed to get the current username, we end up starting a Bash pipeline with three more processes (two tr + one awk) to parse the entire default output into a usable representation. `id -un` does that for us.
-
由 Jacob Champion 提交于
When constructing a DbURL without an explicit username, /usr/bin/id was executed regardless of whether we needed its output or not. That seems excessive; let's only use it if the environment doesn't already have what we need.
-
由 Jacob Champion 提交于
In a classic example of "too many layers", chk_local_db_running() used FileDirExists, which spins up a new Python interpreter, to check for the existence of a file. Just use os.path.exists() directly.
-
由 Jamie McAtamney 提交于
This commit checks whether there are any active pg_rewind connections when the mirror is down and displays an appropriate status message if so. To facilitate this, the pg_rewind call in gprecoverseg has been modified to specify an application_name when connecting to the database. Co-authored-by: NJacob Champion <pchampion@pivotal.io> Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io> Co-authored-by: NJimmy Yih <jyih@pivotal.io>
-
- 13 2月, 2019 12 次提交
-
-
由 Heikki Linnakangas 提交于
We had cherry-picked this from a later version of PostgreSQL in commit e43797f7, and got the second copy in the PostgreSQL 9.4.20 merge.
-
由 Heikki Linnakangas 提交于
And a stray reference to WORKFILE_HASHTABLE_NUM_PARTITIONS, which doesn't exist anymore, from a comment.
-
由 Heikki Linnakangas 提交于
It was only needed for EState. builtins.h is included from many different places, so it seems good to not pull in executor.h. Add explicit #includes into many places that were relying on the indirectly included executor.h.
-
由 Heikki Linnakangas 提交于
-
由 Heikki Linnakangas 提交于
-
由 Daniel Gustafsson 提交于
Commit 7fa85902 removed TINC from the tree, but commit ad6b920f accidentally re-introduced a few (dead) files. Remove them yet again. Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io> Reviewed-by: NJimmy Yih <jyih@pivotal.io> Reviewed-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>
-
由 Adam Lee 提交于
gpfdist tests have the same problem happened in sreh, increase the sleep. commit bb8575a9 Author: Heikki Linnakangas <hlinnakangas@pivotal.io> Date: Mon Nov 27 21:57:08 2017 +0200 Increase sleep between launching gpfdist and running test queries. We've seen a lot of failures in the 'sreh' test in the pipeline, like this: --- 263,269 ---- FORMAT 'text' (delimiter '|') SEGMENT REJECT LIMIT 10000; SELECT * FROM sreh_ext; ! ERROR: connection failed dummy_protocol://DUMMY_LOCATION INSERT INTO sreh_target SELECT * FROM sreh_ext; NOTICE: Found 10 data formatting errors (10 or more input rows). Rejected related input data. SELECT count(*) FROM sreh_target; I don't really know, but I'm guessing it could be because it sometimes takes more than one second for gpfdist to fully start up, if there's a lot of disk or other activity. Increase the sleep time from 1 to 3 seconds; we'll see if that helps.
-
由 Heikki Linnakangas 提交于
Most of the "masking" code was removed in 9.3 merge already, and the rest in commit c9dee15b. But these were left over. Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Richard Guo 提交于
Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
-
由 Tom Lane 提交于
We can allow this even without any specific knowledge of the semantics of the window function, so long as pushed-down quals will either accept every row in a given window partition, or reject every such row. Because window functions act only within a partition, such a case can't result in changing the window functions' outputs for any surviving row. Eliminating entire partitions in this way obviously can reduce the cost of the window-function computations substantially. The fly in the ointment is that it's hard to be entirely sure whether this is true for an arbitrary qual condition. This patch allows pushdown if (a) the qual references only partitioning columns, and (b) the qual contains no volatile functions. We are at risk of incorrect results if the qual can produce different answers for values that the partitioning equality operator sees as equal. While it's not hard to invent cases for which that can happen, it seems to seldom be a problem in practice, since no one has complained about a similar assumption that we've had for many years with respect to DISTINCT. The potential performance gains seem to be worth the risk. David Rowley, reviewed by Vik Fearing; some credit is due also to Thomas Mayer who did considerable preliminary investigation.
-
由 Richard Guo 提交于
PostgreSQL and GPDB both have this change. We prefer to keep the same with PostgreSQL, so remove this temporarily from GPDB and will cherry-pick d222585a from PostgreSQL later. Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
-
由 Richard Guo 提交于
Previously in qual_is_pushdown_safe(), we had this function qual_is_pushdown_safe_set_operation() to check if the qual contains references to WindowFunc node. This is unnecessary because if a qual contains references to WindowFunc node, its corresponding column would be marked as unsafe that prevents the qual from being pushed down. So remove the related functions. Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
-