- 24 11月, 2017 5 次提交
-
-
由 Heikki Linnakangas 提交于
This is functionality that was lost by the ripout & replace. commit 34d26872 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Tue Dec 15 17:57:48 2009 +0000 Support ORDER BY within aggregate function calls, at long last providing a non-kluge method for controlling the order in which values are fed to an aggregate function. At the same time eliminate the old implementation restriction that DISTINCT was only supported for single-argument aggregates. Possibly release-notable behavioral change: formerly, agg(DISTINCT x) dropped null values of x unconditionally. Now, it does so only if the agg transition function is strict; otherwise nulls are treated as DISTINCT normally would, ie, you get one copy. Andrew Gierth, reviewed by Hitoshi Harada
-
由 Heikki Linnakangas 提交于
This loses the functionality, and leaves all the regression tests that used those functions failing. The plan is to later backport the upstream implementation of those functions from PostgreSQL 9.4. The feature is called "ordered set aggregates" there.
-
由 Heikki Linnakangas 提交于
This adds some limitations, and removes some functionality that tte old implementation had. These limitations will be lifted, and missing functionality will be added back, in subsequent commits: * You can no longer have variables in start/end offsets * RANGE is not implemented (except for UNBOUNDED) * If you have multiple window functions that require a different sort ordering, the planner is not smart about placing them in a way that minimizes the number of sorts. This also lifts some limitations that the GPDB implementation had: * LEAD/LAG offset can now be negative. In the qp_olap_windowerr, a lot of queries that used to throw an "ROWS parameter cannot be negative" error are now passing. That error was an artifact of the eay LEAD/LAG were implemented. Those queries contain window function calls like "LEAD(col1, col2 - col3)", and sometimes with suitable values in col2 and col3, the second argument went negative. That caused the error. implementation of LEAD/LAG is OK with a negative argument. * Aggregate functions with no prelimfn or invprelimfn are now supported as window functions * Window functions, e.g. rank(), no longer require an ORDER BY. (The output will vary from one invocation to another, though, because the order is then not well defined. This is more annoying on GPDB than on PostgreSQL, because in GDPB the row order tends to vary because the rows are spread out across the cluster and will arrive in the master in unpredictable order) * NTILE doesn't require the argument expression to be in PARTITION BY * A window function's arguments may contain references to an outer query. This changes the OIDs of the built-in window functions to match upstream. Unfortunately, the OIDs had been hard-coded in ORCA, so to work around that until those hard-coded values are fixed in ORCA, the ORCA translator code contains a hack to map the old OID to the new ones.
-
由 Heikki Linnakangas 提交于
This is a cherry-pick from PosgreSQL 9.2. This will help to make some existing test cases pass, after the big window functions rewrite that we've been working on, where a NULL argument to percentile_cont() is otherwise ambiguous. While we could include this in the big window functions rewrite, that is large enough already, and this is a rather orthogonal patch, so seems better to do this as a separate commit. Add a test case, because there was none included in the upstream commit. Upstream commit: commit 1a8b9fb5 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Thu Nov 17 18:28:41 2011 -0500 Extend the unknowns-are-same-as-known-inputs type resolution heuristic. For a very long time, one of the parser's heuristics for resolving ambiguous operator calls has been to assume that unknown-type literals are of the same type as the other input (if it's known). However, this was only used in the first step of quickly checking for an exact-types match, and thus did not help in resolving matches that require coercion, such as matches to polymorphic operators. As we add more polymorphic operators, this becomes more of a problem. This patch adds another use of the same heuristic as a last-ditch check before failing to resolve an ambiguous operator or function call. In particular this will let us define the range inclusion operator in a less limited way (to come in a follow-on patch).
-
由 Heikki Linnakangas 提交于
It created a schema at the beginning, but the "set search_path" command was misspelled, so the schema was not actually used for anything. All the objects were created in public. Also, at the end, the "start/end_ignore" around the DROP commands were misspelled, so the output of the DROPs were not ignored, after all. Clean that up, by accepting the status quo. Continue to create the objects in public, but remove the useless schema. The objects might be useful for testing pg_dump/restore (particularly as part of pg_upgrade). And remove the misspelled ignore-comments, to avoid confusion.
-
- 23 11月, 2017 7 次提交
-
-
由 Heikki Linnakangas 提交于
The old logic was: 1. Decide if we need to put a Gather motion on top of the plan 2. Add nodes to handle DISTINCT 3. Add nodes to handle ORDER BY. 4. Add Gather node, if we decided so in step 1. If in step 1, if the result was already focused on a single segment, we would make note that no Gather is needed, and not add one in step 4. However, the DISTINCT processing might add a Redistribute Motion node, so that the final result is not focused on a single node. I couldn't come up with a query where that would happen, as the code stands, but we saw such a case on the "window functions rewrite" branch we've been working on. There, the sort order/distribution of the input can be changed to process window functions. But even if this isn't actively broken right now, it seems more robust to change the logic so that 'must_gather' means 'at the end, the result must end up on a single node', instead of 'we must add a Gather node'. The test that this adds exercises this issue after the the window functions rewrite, but right now it passes with or without these code changes. But might as well add it now.
-
由 Heikki Linnakangas 提交于
The last 8.4 merge commit introduced support for DISTINCT with hashing, and refactored the way grouping_planner() works with the path keys. That broke DISTINCT with window functions, because the new distinct_pathkeys field was not set correctly. In commit 474f1db0, I moved some GPDB-added tests from the 'aggregates' test, to a new 'gp_aggregates' test. But I forgot to add the new test file to the test schedule, so it was not run. Oops. Add it to the schedule now. The tests in 'gp_aggregates' cover this bug.
-
由 Abhijit Subramanya 提交于
The redo code for mmxlog records looks at the segment file number of the files to be removed in order to decide whether to call the remove method for heap or appendonly. If the segment file number is zero, it assumes a heap file and calls smgrdounlink(). This is incorrect since in case of appendonly files, since smgrdounlink() will end up deleting all the files including .1, .2 etc. This patch introduces two new mmxlog record types and removes the old record type for removing files so that the redo code can call the correct remove method for heap and appendonly files.
-
由 Heikki Linnakangas 提交于
This silences false regression failures caused by row order differences in some queries that use those constructs. While we don't have such false failures in the regression suite right now, the upcoming rewrite of the window functions implementation will change the plans for some queries, and would cause some.
-
由 Heikki Linnakangas 提交于
Currently, all the test queries on this data error out, which is why it hasn't mattered. But we're about to make the queries work, in the window functions rewrite.
-
由 Heikki Linnakangas 提交于
To avoid clashing with upstream functions with the same OIDs. We are about to merge the Window Functions patch from upstream, and the new window functions are clashing with these.
-
由 Dhanashree Kashid 提交于
The intent of these tests is to test fault injection for plans with SharedInputScan over a Sort. While the planner currnetly generates a SharedInputScan for the window functions, after the window functions rewrite we're working on, it will now. To fulfill the intent; change the query to use WITH clause and ORDER BY such that Planner generated SharedInputScan over a Sort node. Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>
-
- 22 11月, 2017 12 次提交
-
-
由 Ashwin Agrawal 提交于
Commit 33960006 seems to have introduced regression, need to add negation. For non-temp relations xloggging is needed, this was caught by new replic_check facility built for wal replication, as it flagged differenece between primary and mirror files, since primary was getting truncated to 0 but missing xlog record didn't truncate the file on mirror.
-
由 Ashwin Agrawal 提交于
-
由 Xin Zhang 提交于
Originally, we compute the offset of CONTRECORD, and compose the full xlaorecord, and then just extract the header information. During the composition of the xlaorecord, we point to a wrong location for the continous record, which results in incorrect parsing of XLOG records. Now, we only extract the header, and move offset to the proper page boundary, so that the common page header code can handle the CONTRECORD. Author: Xin Zhang <xzhang@pivotal.io> Author: Ashwin Agrawal <aagrawal@pivotal.io>
-
由 Ashwin Agrawal 提交于
ExpectedPrimaryAndMirrorConfiguration() can explicitly specify the primary and mirror status as well as the mode after given responses and cdb component databases. This will simplify the verification logic, and also easy to show the intend of each test, and verify the final segment configuration. Author: Ashwin Agrawal <aagrawal@pivotal.io> Author: Xin Zhang <xzhang@pivotal.io>
-
由 Ashwin Agrawal 提交于
-
由 Ashwin Agrawal 提交于
When primary and mirror are not in sync, cluster is running with no HA for that segment. If FTS fails to probe primary as well, currently nothing much can be done except retrying. So, in such case FTS should not mark both the segment as down as it would be cure is worse than the disease.
-
由 Ashwin Agrawal 提交于
If walsender state transitions to streaming, FTS must record transition of same on master in catalog. Recording the state to SYNC will enable failing over to mirror incase of need.
-
由 Ashwin Agrawal 提交于
-
由 Jimmy Yih 提交于
The strdups were removed in a recent 8.4 postgres merge commit fa287d01. Without them, pg_ctl restart will usually fail as the memory is freed for optline. Bring them back similar to how it was fixed in an earlier commit. Reference commit by Daniel Gustafsson: https://github.com/greenplum-db/gpdb/commit/627ba5d8d83caa133ed1d8c9e13570bd162933bf
-
由 Ashwin Agrawal 提交于
By setting up synchronized replication between primary and standby, commits on primary should be blocked by function `SyncRepWaitForLSN()` until its effects replicated to the standby. `SyncRepWaitForLSN()` was fast exiting if the `SyncStandbysDefined()` is false, which means GUC `synchronous_standby_names` is empty. There was a race condition where the GUC is set but not reflected by some backends. However, the `pg_stat_replication` `sync_state` is `sync`, and the `state` is `streaming`, which means the commit should block and get replicated to the mirror before it returns. The change is to NOT do the fast exit based on the GUC, but only rely on `WalSndCtl->sync_standbys_defined`. Issue was spotted by Asim Praveen during whiteboard discussion. Discussion: https://www.postgresql.org/message-id/flat/CABrsG8j3kPD+kbbsx_isEpFvAgaOBNGyGpsqSjQ6L8vwVUaZAQ@mail.gmail.com
-
由 Ashwin Agrawal 提交于
Fixing compilation error introduced due to commit 321c0529 with --enable-segwalrep.
-
由 Chris Hajas 提交于
Previously we used parname to reference partitions to exchange, but this doesn't work for range partitions that are not named. We now use partition rank to exchange partitions that do not have names. Solution derived from PR #1940 Signed-off-by: NKaren Huddleston <khuddleston@pivotal.io>
-
- 21 11月, 2017 10 次提交
-
-
由 Heikki Linnakangas 提交于
These had been added in GPDB a long time ago, presumably to make the test output repeatable. But they're not needed anymore. Remove them to match the upstream more closely, which helps when merging.
-
由 Venkatesh Raghavan 提交于
-
由 Heikki Linnakangas 提交于
If a DROP IF EXISTS is run concurrently with ALTER TABLE RENAME TO, it's possible that the table gets dropped on master, but not on the segments. This was fixed earlier already, by commit 4739cd22, but the fix got lost during the 8.4 merge. Fix it again, in the same fashion, and also add a test case so that it won't get lost again, at least not in exactly the same manner. Spotted by @kuien, github issue #3874. Thanks to @tvesely for the test case.
-
由 Bhuvnesh Chaudhary 提交于
The concept of optimization levels is known in many enterprise optimizers. It enables user to handle the degree of optimization that is being employed. The optimization levels allow the grouping of transformations into bags of rules (where each is assigned a particular level). By default all rules are applied, but if a user wants to apply fewer rules they are able to. This decision is made by them based on domain knowledge, and they know even with fewer rules being applied the plan generated satisfies their needs. The Cascade optimizer, on which GPORCA is based on, allows grouping of transformation rules into optimization levels. This concept of optimization levels has also been extended to join ordering allowing user to pick the join order via the query or, use greedy approach or use exhaustive approach. Postgres based planners use join_limit and from_limit to reduce the search space. While the objective of Optimization/Join is also to reduce search space, but the way it does it is different. It is requesting the optimizer to apply or not apply a subset of rules and providing more flexibility to the customer. This is one of the most frequently requested feature from our enterprise clients who have high degree of domain knowledge. This PR introduces this concept. In the immediate future we are planning to add different polynomial join ordering techniques with guaranteed bound as part of the "Greedy" search. Signed-off-by: NHaisheng Yuan <hyuan@pivotal.io>
-
由 Heikki Linnakangas 提交于
Much of the code and structs used by index scans and bitmap index scans had been fused together and refactored in GPDB, to share code between dynamic index scans and regular ones. However, it would be nice to keep upstream code unchanged as much as possible. To that end, refactor the exector code for dynamic index scans and dynamic bitmap index scans, to reduce the diff vs upstream. The Dynamic Index Scan executor node is now a thin wrapper around the regular Index Scan node, even thinner than before. When a new Dynamic Index Scan begins, we don't do much initialization at that point. When the scan begins, we initialize an Index Scan node for the first partition, and return rows from it until it's exhausted. On next call, the underlying Index Scan is destroyed, and a new Index Scan node is created, for the next partition, and so on. Creating and destroying the IndexScanState for every partition adds some overhead, but it's not significant compared to all the other overhead of opening and closing the relations, building scan keys etc. Similarly, a Dynamic Bitmap Index Scan executor node is just a thin wrapper for regular Bitmap Index Scan. When MultiExecDynamicBitmapIndexScan() is called, it initializes an BitmapIndexScanState for the current partition, and calls it. On ReScan, the BitmapIndexScan executor node for the old partiton is shut down. A Dynamic Bitmap Index Scan differs from Dynamic Index Scan in that a Dynamic Index Scan is responsible for iterating through all the active partitions, while a Dynamic Bitmap Index Scan works as a slave for the Dynamic Bitmap Heap Scan node above it. It'd be nice to do a similar refactoring for heap scans, but that's for another day.
-
由 Heikki Linnakangas 提交于
-
由 Heikki Linnakangas 提交于
The distributed clog, like normal clog, will not be consulted for subtransactions that are part of a still-in-progress transaction, so there is no need to update it until we're ready to commit the top transaction. This is basically the same changes that was done in the upstream for clog in commit 06da3c57. We're about to merge that change from the upstream as part of the PostgreSQL 8.4 merge, but we can make that change for the distributed log separately, to keep the actual merge commit smaller.
-
由 Heikki Linnakangas 提交于
By running pgindent, and tidying a bunch of things manually.
-
由 Heikki Linnakangas 提交于
For clarity, and to make merging easier. The code to manage the hash table of "pending resync EOFs" for append-only tables is moved to smgr_ao.c. One notable change here is that the pendingDeletesPerformed flag is removed. It was used to track whether there are any pending deletes, or any pending AO table resyncs, but we might as well check the pending delete list and the pending syncs hash table directly, it's hardly any slower than checking a separate boolean. There are still plenty of GPDB changes in smgr.c, but this is a good step forward.
-
由 Heikki Linnakangas 提交于
These were added in GPDB a long time ago, probably before gpdiff.pl was introduced to mask row order differences in regression test output. But now that gpdiff.pl can do that, these are unnecessary. Revert to match the upstream more closely. This includes updates to the 'rules' and 'inherit' tests, although they are disabled and still doesn't pass after these changes.
-
- 19 11月, 2017 1 次提交
-
-
由 Heikki Linnakangas 提交于
Long time ago, when updating our psql version to 8.3 (or something higher), we had decided to keep the old single-line style when displaying access privileges, to avoid having to update regression tests. It's time to move forward, update the tests, and use the nicer 8.3 style for displaying access privileges. Also, \d on a view no longer prints the View Definition. You need to use the verbose \d+ option for that. (I'm not a big fan of that change myself: when I want to look at a view I'm almost always interested in the View Definition. But let's not second-guess decisions made almost 10 years ago in the upstream.) Note: psql still defaults to the "old-ascii" style when printing multi-line fields. The new style was introduced only later, in 9.0, so to avoid changing all the expected output files, we should stick to the old style until we reach that point in the merge. This commit only changes the style for Access privileges, which is different from the multi-line style.
-
- 17 11月, 2017 3 次提交
-
-
由 Heikki Linnakangas 提交于
According to git history, this was added a very long time ago, before GPDB was open sourced, because it was needed by some migration tool back then. But the code that used it has been removed since.
-
由 Taylor Vesely 提交于
Initial work was done in collaboration with Kanaiya Kariya @kanhaiya7 and Jiangtian Nie @zhadan01 Signed-off-by: NXin Zhang <xzhang@pivotal.io> Signed-off-by: NAsim Praveen <apraveen@pivotal.io> Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Ashwin Agrawal 提交于
-
- 16 11月, 2017 2 次提交
-
-
由 Daniel Gustafsson 提交于
This was lost in commit 152d1223 which has the corresponding update for the postgres planner output.
-
由 Daniel Gustafsson 提交于
Avoid the extra string copy by using the StringInfo buffer and simplify the code a bit now that HASH partitions are removed.
-