- 27 9月, 2017 1 次提交
-
-
由 Lisa Owen 提交于
-
- 26 9月, 2017 13 次提交
-
-
由 Jacob Champion 提交于
Several GUCs are simply enumerated strings that are parsed into integer types behind the scenes. As of 8.4, the GUC system recognizes a new type, enum, which will do this for us. Move as many as we can to the new system. As part of this, - gp_idf_deduplicate was changed from a char* string to an int, and new IDF_DEDUPLICATE_* macros were added for each option - password_hash_algorithm was changed to an int - for codegen_optimization_level, "none" is the default now when codegen is not enabled during compilation (instead of the empty string). A couple of GUCs that *could* be represented as enums (optimizer_minidump, gp_workfile_compress_algorithm) have been purposefully kept with the prior system because they require the GUC variable to be something other than an integer anyway. Signed-off-by: NJacob Champion <pchampion@pivotal.io>
-
由 Heikki Linnakangas 提交于
We had a very simplistic implementation in parse-analysis already, which converted the FILTER WHERE clause into a CASE-WHEN expression. That did not work for non-strict aggregates, and didn't deparse back into a FILTER expression nicely, to name a few problems with it. Replace it with the PostgreSQL implementation. TODO: * ORCA support. It now falls back to the Postgres planner. * I disabled the three-stage DQA plan types if there are any FILTERs
-
由 Lav Jain 提交于
-
由 Xin Zhang 提交于
Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Ashwin Agrawal 提交于
Adding timestamp to better trace the command. Adding ClusterConfiguration to improve the reuse, and remove ColdStartMaster class. Signed-off-by: NXin Zhang <xzhang@pivotal.io>
-
由 Xin Zhang 提交于
Under SEGWALREP, the mode for gp_segment_configuration is 'n' for not-in-sync and 's' for in-sync. When new primary or mirror is added to the cluster, the initial state of the mode should be 'n' instead of 's' (in the original filerep mode.) If mirror exists and synchronized with the primary, then the mode will be updated to 's' to indicate the primary-mirror pair are in-sync. Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Ashwin Agrawal 提交于
Originally, gp_add_segment() is only used to add a primary segment. We created a new gp_add_segment_primary() to keep the original functionality, which automatically generate dbid and contentid. We generalized the gp_add_segment() to be able to directly update gp_segment_configuration and pg_filespace_entry for adding any type of segments with full specification of segment and filespace mappings. In this case, the new gp_add_segment() doesn't generate dbid and contentid automatically, and rely on input parameters. Originally there is separate code path in gp_add_segment_mirror() to figure out primary dbid, such logic is actually common even with gp_add_segment(), which can add both primary and mirror. In that case, we refactor the primary dbid detection logic in common function add_segment(), and refactor the gp_add_segment_mirror() to use the add_segment() instead of add_segment_config_entry(). We update gpinitsystem to use the function gp_add_segment() instead of update the gp_segment_configuration and pg_filespace_entry tables directly via SQL. Signed-off-by: NXin Zhang <xzhang@pivotal.io>
-
由 Xin Zhang 提交于
Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Ben Christel 提交于
- Move Conan instructions above manual orca setup instructions so that people don't do manual work first (like we did). - Inline recommendation to parallelize make with -j8. - Fix formatting of code blocks in README.md See issues: - https://github.com/greenplum-db/gpdb/issues/3178 - https://github.com/greenplum-db/gpdb/pull/3332
-
由 Kavinder Dhaliwal 提交于
This commit will display the contents of the Optimizer Mem Account when the optimizer GUC is on and explain_memory_verbosity is set to 'summary'. Signed-off-by: NSambitesh Dash <sdash@pivotal.io>
-
由 Heikki Linnakangas 提交于
When we merged with PostgreSQL 8.3, and got support for NULLS FIRST, we missed or stubbed out some code, that is required to honor NULLS FIRST in an aggregate's ORDER BY, e.g. "select array_agg(a order by b desc nulls last)"
-
由 Lisa Owen 提交于
-
由 Asim R P 提交于
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>
-
- 25 9月, 2017 8 次提交
-
-
由 Heikki Linnakangas 提交于
It wasn't very useful. ORCA and Postgres both just stack WindowAgg nodes on top of each other, and no-one's been unhappy about that, so we might as well do that, too. This reduces the difference between GPDB and the upstream implementation, and will hopefully make it smoother to switch. Rename the Window Plan node type to WindowAgg, to match upstream, now that it is fairly close to the upstream version.
-
由 Heikki Linnakangas 提交于
To match upstream.
-
由 Heikki Linnakangas 提交于
This test case could throw either "ROWS parameter cannot be negative", or "Division By Zero", depending on which gets evaluated first. Remove the division by zero error, to make it more predictable.
-
由 Richard Guo 提交于
This case is to test cancel/terminate queries concurrently while they are running or waiting in resource group.
-
由 Heikki Linnakangas 提交于
A Motion node often needs to "merge" the incoming streams, to preserve the overall sort order. Instead of carrying sort order information throughout the later stages of planning, in the Flow struct, pass it as argument directly to make_motion() and other functions, where a Motion node is created. This simplifies things. To make that work, we can no longer rely on apply_motion() to add the final Motion on top of the plan, when the (sub-)query contains an ORDER BY. That's because we no longer have that information available at apply_motion(). Add the Motion node in grouping_planner() instead, where we still have that information, as a path key. When I started to work on this, this also fixed a bug, where the sortColIdx of plan flow node may refer to wrong resno. A test case for that is included. However, that case was since fixed by other coincidental changes to partition elimination, so now this is just refactoring.
-
由 Peifeng Qiu 提交于
Concourse doesn't support AIX natively, we need to clone the repo with the correspond commit on remote machine, compile the packages, and download them back to concourse container as output. Testing client and loader for platform without gpdb server is another challenge. We setup GPDB server on concourse container just like most installcheck tests, and use SSH tunnel to forward ports from and to the remote host. This way both CL tools and GPDB server feel they are on the same machine, and the test can run normally.
-
由 Adam Lee 提交于
Replace popen() with popen_with_stderr() which is used in external web table also to collect the stderr output of program. Since popen_with_stderr() forks a `sh` process, it's almost always sucessful, this commit catches errors happen in fwrite(). Also passes variables as the same as what external web table does. Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>
-
由 Zhenghua Lyu 提交于
Previous code us python package psutil to get the mount information of the system which will read the content of /etc/mtab. In some environments, /etc/mtab does not contain the mount point information of cgroups. In this commit, we scan /proc/self/mounts to find out cgroup mount point.
-
- 23 9月, 2017 9 次提交
-
-
由 Kavinder Dhaliwal 提交于
-
由 Kavinder Dhaliwal 提交于
There are cases where during execution a Memory Intensive Operator (MI) may not use all the memory that is allocated to it. This means that this extra memory (quota - allocated) can be relinquished for other MI nodes to use during execution of a statement. For example -> Hash Join -> HashAggregate -> Hash In the above query fragment the HashJoin operator has a MI operator for both its inner and outer subtree. If there ever is the case that the Hash node used much less memory than was given as its quota it will now call MemoryAccounting_DeclareDone() and the difference between its quota and allocated amount will be added to the allocated amount of the RelinquishedPool. Doing this will enable HashAggregate to request memory from this RelinquishedPool if it exhausts its quota to prevent spilling. This PR adds two new API's to the MemoryAccounting Framework MemoryAccounting_DeclareDone(): Add the difference between a memory account's quota and its allocated amount to the long living RelinquishedPool MemoryAccounting_RequestQuotaIncrease(): Retrieve all relinquished memory by incrementing an operator's operatorMemKb and setting the RelinquishedPool to 0 Note: This PR introduces the facility for Hash to relinquish memory to the RelinquishedPool memory account and for the Agg operator (specifically HashAgg) to request an increase to its quota before it builds its hash table. This commit does not generally apply this paradigm to all MI operators Signed-off-by: NSambitesh Dash <sdash@pivotal.io> Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
-
由 sambitesh 提交于
Before this cherry-pick the below query would have errored out WITH outermost(x) AS ( SELECT 1 UNION (WITH innermost as (SELECT 2) SELECT * FROM innermost UNION SELECT 3) ) SELECT * FROM outermost; Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
-
由 Tom Meyer 提交于
To update 5.json, we ran: cat src/include/catalog/*.h | perl src/backend/catalog/process_foreign_keys.pl > gpMgmt/bin/gppylib/data/5.json Signed-off-by: NJacob Champion <pchampion@pivotal.io>
-
由 Todd Sedano 提交于
-
由 Taylor Vesely 提交于
In order to view the primary segments' replication stream data from their pg_stat_replication view, we currently need to connect to the primary segment individually via utility mode. To make life easier, we introduce a function that will fetch each primary segment's replication stream data and wrap it with a view named gp_stat_replication. It will now be possible to view all the cluster replication information from the master in a regular psql session. Authors: Taylor Vesely and Jimmy Yih
-
由 Bhuvnesh Chaudhary 提交于
Signed-off-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
-
由 Alexander Denissov 提交于
* run pxf_automation as part of pxf regression test Signed-off-by: NAlexander Denissov <adenissov@pivotal.io> * added missing input Signed-off-by: NLav Jain <ljain@pivotal.io> * Fix symbolic links * create extension pxf before running automation tests * Hack python psi module by copying it from system to gpdb python * remove if exists for extension * Run pxf_automation before regression tests * Change owner to gpadmin before running tests * Generalize copying of PSI package * Generalize install path using GPHOME
-
由 David Yozie 提交于
* Edits to make 'resource management' using resource queues, resources groups, consistent throughout. This is to distinguish between resource management and general workload profiles, as well as to avoid conusion with Workload Manager product. * Edits from Lisa's review
-
- 22 9月, 2017 6 次提交
-
-
由 Daniel Gustafsson 提交于
This merges and backports the upstream commits which replaces the amgetmulti AM function with amgetbitmap, which performs the whole indexscan in one call (for HashBitmap, StreamBitmaps are not affected by this). GPDB was more or less already doing this, as the upstream patch was from the beginning submitted from Greenplum. This commit refactors the AM function to mimick the upstream behavior, while keeping the GPDB API for the callsites. The below commits are included either in full, or in part: commit 4e82a954 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Thu Apr 10 22:25:26 2008 +0000 Replace "amgetmulti" AM functions with "amgetbitmap", in which the whole indexscan always occurs in one call, and the results are returned in a TIDBitmap instead of a limited-size array of TIDs. This should improve speed a little by reducing AM entry/exit overhead, and it is necessary infrastructure if we are ever to support bitmap indexes. In an only slightly related change, add support for TIDBitmaps to preserve (somewhat lossily) the knowledge that particular TIDs reported by an index need to have their quals rechecked when the heap is visited. This facility is not really used yet; we'll need to extend the forced-recheck feature to plain indexscans before it's useful, and that hasn't been coded yet. The intent is to use it to clean up 8.3's horrid @@@ kluge for text search with weighted queries. There might be other uses in future, but that one alone is sufficient reason. Heikki Linnakangas, with some adjustments by me. commit 1dcf6fdf Author: Teodor Sigaev <teodor@sigaev.ru> Date: Sat Aug 23 10:37:24 2008 +0000 Fix possible duplicate tuples while GiST scan. Now page is processed at once and ItemPointers are collected in memory. Remove tuple's killing by killtuple() if tuple was moved to another page - it could produce unaceptable overhead. Backpatch up to 8.1 because the bug was introduced by GiST's concurrency support. commit b9856b67 Author: Teodor Sigaev <teodor@sigaev.ru> Date: Wed Oct 22 12:53:56 2008 +0000 Fix GiST's killing tuple: GISTScanOpaque->curpos wasn't correctly set. As result, killtuple() marks as dead wrong tuple on page. Bug was introduced by me while fixing possible duplicates during GiST index scan.
-
由 Kavinder Dhaliwal 提交于
Before this commit all memory allocations made by ORCA/GPOS were a blackbox to GPDB. However the ground work had been in place to allow GPDB's Memory Accounting Framework to track memory consumption by ORCA. This commit introduces two new functions Ext_OptimizerAlloc and Ext_OptimizerFree which pass through their parameters to gp_malloc and gp_free and do some bookeeping against the Optimizer Memory Account. This introduces very little overhead to the GPOS memory management framework. Signed-off-by: NMelanie Plageman <mplageman@pivotal.io> Signed-off-by: NSambitesh Dash <sdash@pivotal.io>
-
由 Jacob Champion 提交于
8.4 seems to use more memory during this test. To get master green again, we're checking in these changes to the memory limits for the resource group tests. Follow-up should be on issue #3345; there's a good chance this will not be our final solution to this test failure. Signed-off-by: NTom Meyer <tmeyer@pivotal.io>
-
由 Heikki Linnakangas 提交于
We can encounter tuples that belong to later batches even after the first pass. Revert the comment to the way it is in upstream. I forgot to update
-
由 Heikki Linnakangas 提交于
Noteworthy changes that were not totally straightforward to merge: * Changes in the hash function. This replaces the contents of hashfunc.c directly with REL8_4_STABLE, not just changes otherwise included in the merge batch. That includes later changes to the hash algorithm used. I didn't feel like trying to fix it to an intermediate state that we would just rewrite again later. The hash function had been replaced in GPDB, too, but I couldn't quite figure out what the GPDB algorithm was, and whether it was better or how. In any case, I believe the new PostgreSQL algorithm is decent, so let's just use that. I'm not very impressed by the old code, there was weird stuff going on with the little and big endianess stuff. And at the top, WORDS_BIGENDIAN was misspelled as WORS_BIGENDIAN, so it never worked as intended on big endian systems. Note that GPDB uses a completely different set of hash functions for calculating the DISTRIBUTED BY key, so this doesn't affect pg_upgrade. This does invalidate hash indexes, but they're not supported on GPDB anyway. And we don't support hash partitioning either. * Pattern selectivity functions had been heavily modified in GPDB, but this replaces it with the upstream version. It was not clear to us what the purpose of the GPDB changes were. That ought to be revisited, and there's a GPDB_84_MERGE_FIXME comment about it. * Commit 95c238d9, to make COPY of CSV files faster, was not merged. The function had been heavily modified in GPDB, and it was not immediately clear how to resolve the conflicts. That commit was just a performance enhancement, so we can revisit that later. Added a GPDB_84_MERGE_FIXME comment about that too. * Resurrect the MyXactAccessedTempRel global variable. It's not used for anything in GPDB, as noted in the comment in PrepareTransaction. We had #ifdef'd out the variable, and all the places that set the variable. To reduce future merge conflicts, it seems better to have the variable and keep all the places where it's set unmodified from the upstream, and only comment out the place where it's checked in PrepareTransaction. * heap_release_fetch was removed in upstream, because it was unused. However, it was still used in one GPDB-specific function, in nbtree.c. Replace the call in nbtree.c with a ReleaseBuffer() + heap_fetch(), and add a GPDB_84_MERGE_FIXME to revisit. * This merge included an upstream change to add USE_SEGMENTED_FILES flag, but it was later later in the 8.4 dev cycle. Cherry-pick the change to remove it now, to avoid having to make it work just to remove it later. (commit 3c6248a8) * This adds support for enum-type GUCs, but we do not yet take advantage of that in the GPDB-specific GUCs, except for a few that shared code with client_min_messages and log_min_messages. * Reshuffle a few OIDs to avoid collision. We had reserved OID 1980 for int8_ops opclass. But that is now used for numeric_div_trunc function(), which we just merged in. In the upstream, we have reserved OID 3124 for the opclass, but only since version 9.2. Before that, we used whatever was free at initdb time. But we have been using OID 3124 for the GPDB-specific pg_proc_callback system table To resolve this mess, change the OID of pg_proc_callback from 3124 to 7176, to make 3124 available. And then use 3124 for int8_ops. That leaves 1980 for numeric_div_trunc function(), like in upstream. * TRUNCATE triggers now work, and to make that work, I made some changes to the way statement-level triggers are fired in general. The goal with statement-level triggers is to always execute them on the dispatcher, but they've been broken and unsupported before. At first, I thought these changes would be enough to do that for all statement-level triggers, but testing shows that not quite. So statement-level triggers are broken, like they were before, even though we pass the truncate-trigger tests now. This has been a joint effort between Heikki Linnakangas, Daniel Gustafsson, Jacob Champion and Tom Meyer.
-
由 Lisa Owen 提交于
* docs - add suse11 swapaccount req to resgroup cgroup cfg * must reboot after setting boot parameters
-
- 21 9月, 2017 3 次提交
-
-
由 Heikki Linnakangas 提交于
Ideally, we would use proper error codes, or find some other way to prevent the useless "(plperl.c:2118)" from appearing in PL/perl errors. Later versions of PostgreSQL do that, so we'll get that eventually. In the meanwhile, silence errors caused by code movement in that file. Same as we had done for plperl's own tests already.
-
由 Daniel Gustafsson 提交于
Leverage the core autoconf scaffolding for resolving the dependency on libcurl. Enabling PXF in autoconf now automatically adds libcurl as a dependency. Coupled with the recent commit which relaxes the curl version requirement on macOS, we can remove the library copying from the PXF makefile as well.
-
由 Heikki Linnakangas 提交于
The WITH RECURSIVE test case in 'join_gp' would miss some rows, if the hash algorithm (src/backend/access/hash/hashfunc.c) was replaced with the one from PostgreSQL 8.4, or if statement_mem was lowered from 1000 kB to 700 kB. This is what happened: 1. A tuple belongs to batch 0, and is kept in memory during processing batch 0. 2. The outer scan finishes, and we spill the inner batch 0 from memory to a file, with SpillFirstBatch, and start processing tuple 1 3. While processing batch 1, the number of batches is increased, and the tuple that belonged to batch 0, and was already written to the batch 0's file, is moved, to a later batch. 4. After the first scan is complete, the hash join is re-scanned 5. We reload the batch file 0 into memory. While reloading, we encounter the tuple that now doesn't seem to belong to batch 0, and throw it away. 6. We perform the rest of the re-scan. We have missed any matches to the tuple that was thrown away. It was not part of the later batch files, because in the first pass, it was handled as part of batch 0. But in the re-scan, it was not handled as part of batch 0, because nbatch was now larger, so it didn't belong there. To fix, when reloading a batch file we see a tuple that actually belongs to a later batch file, we write it to that later file. To avoid adding it there multiple times, if the hash join is re-scanned multiple times, if any tuples are moved when reloading a batch file, destroy the batch file and re-create it with just the remaining tuples. This is made a bit complicated by the fact that BFZ temp files don't support appending to a file that's already been rewinded for reading. So what we actually do, is always re-create the batch file, even if there has been no changes to it. I left comments about that, Ideally, we would either support re-appending to BFZ files, or stopped using BFZ workfiles for this altogether (I'm not convinced they're any better than plain BufFiles). But that can be done later. Fixes github issue #3284
-