- 01 4月, 2017 21 次提交
-
-
由 Christopher Hajas 提交于
-
由 Heikki Linnakangas 提交于
* Rewrite Kerberos test suite * Remove obsolete Kerberos test stuff from pipeline and TINC We now have a rewritten Kerberos test script in installcheck-world. * Update ICW kerberos test to run on concourse container This adds a whole new test script in src/test/regress, implemented in plain bash. It sets up temporary a KDC as part of the script, and doesn't therefore rely on a pre-existing Kerberos server, like the old MU_kerberos-smoke test job did. It does require MIT Kerberos server-side utilities to be installed, instead, but no server needs to be running, and no superuser privileges are required. This supersedes the MU_kerberos-smoke behave tests. The new rewritten bash script tests the same things: 1. You cannot connect to the server before running 'kinit' (to verify that the server doesn't just let anyone in, which could happen if the pg_hba.conf is misconfigured for the test, for example) 2. You can connect, after running 'kinit' 3. You can no longer connect, if the user account is expired The new test script is hooked up to the top-level installcheck-world target. There were also some Kerberos-related tests in TINC. Remove all that, too. They didn't seem interesting in the first place, looks like they were just copies of a few random other tests, intended to be run as a smoke test, after a connection had been authenticated with Kerberos, but there was nothing in there to actually set up the Kerberos environment in TINC. Other minor patches added: * Remove absolute path when calling kerberos utilities -- assume they are on path, so that they can be accessed from various installs -- add clarification message if sample kerberos utility is not found with 'which' * Specify empty load library for kerberos tools * Move kerberos test to its own script file -- this allows a failure to be recorded without exiting Make, and therefore the server can be turned off always * Add trap for stopping kerberos server in all cases * Use localhost for kerberos connection Signed-off-by: NMarbin Tan <mtan@pivotal.io> Signed-off-by: NChumki Roy <croy@pivotal.io> Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
-
由 Heikki Linnakangas 提交于
The loop to print each constraint's name was broken: it printed the name of the first constraint multiple times. A test case, as matter of principle. In the passing, change the set of tests around this error to all use the same partitioned table, rather than drop and recreate it for each command. And reduce the number of partitions from 10 to 5. Shaves some milliseconds from the time to run the test.
-
由 Tom Meyer 提交于
Signed-off-by: NJingyi Mei <jmei@pivotal.io>
-
由 Tom Meyer 提交于
Signed-off-by: NJingyi Mei <jmei@pivotal.io>
-
由 Jingyi Mei 提交于
This comes from 4.3_STABLE repo Signed-off-by: NTom Meyer <tmeyer@pivotal.io>
-
由 Tom Meyer 提交于
Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>
-
由 Toolsmiths Team 提交于
This reverts commit 3531b6f681a05352f46f2f57861837f1ced2c6a0.
-
由 Tushar Dadlani 提交于
Signed-off-by: NTom Meyer <tmeyer@pivotal.io>
-
由 Tom Meyer 提交于
- This only includes ICW and packaging [#139228021] Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>
-
由 David Sharp 提交于
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>
-
由 foyzur 提交于
GPDB supports range and list partitions. Range partitions are represented as a set of rules. Each rule defines the boundaries of a part. E.g., a rule might say that a part contains all values between (0, 5], where left bound is 0 exclusive, but the right bound is 5, inclusive. List partitions are defined by a list of values that the part will contain. ORCA uses the above rule definition to generate expressions that determine which partitions need to be scanned. These expressions are of the following types: 1. Equality predicate as in PartitionSelectorState->levelEqExpressions: If we have a simple equality on partitioning key (e.g., part_key = 1). 2. General predicate as in PartitionSelectorState->levelExpressions: If we need more complex composition, including non-equality such as part_key > 1. Note: We also have residual predicate, which the optimizer currently doesn't use. We are planning to remove this dead code soon. Prior to this PR, ORCA was treating both range and list partitions as range partitions. This meant that each list part will be converted to a set of list values and each of these values will become a single point range partition. E.g., consider the DDL: ```sql CREATE TABLE DATE_PARTS (id int, year int, month int, day int, region text) DISTRIBUTED BY (id) PARTITION BY RANGE (year) SUBPARTITION BY LIST (month) SUBPARTITION TEMPLATE ( SUBPARTITION Q1 VALUES (1, 2, 3), SUBPARTITION Q2 VALUES (4 ,5 ,6), SUBPARTITION Q3 VALUES (7, 8, 9), SUBPARTITION Q4 VALUES (10, 11, 12), DEFAULT SUBPARTITION other_months ) ( START (2002) END (2012) EVERY (1), DEFAULT PARTITION outlying_years ); ``` Here we partition the months as list partition using quarters. So, each of the list part will contain three months. Now consider a query on this table: ```sql select * from DATE_PARTS where month between 1 and 3; ``` Prior to this ORCA generated plan would consider each value of the Q1 as a separate range part with just one point range. I.e., we will have 3 virtual parts to evaluate for just one Q1: [1], [2], [3]. This approach is inefficient. The problem is further exacerbated when we have multi-level partitioning. Consider the list part of the above example. We have only 4 rules for 4 different quarters, but we will have 12 different virtual rule (aka constraints). For each such constraint, we will then evaluate the entire subtree of partitions. After this PR, we no longer decompose rules into constraints for list parts and then derive single point virtual range partitions based on those constraints. Rather, the new ORCA changes will use ScalarArrayOp to express selectivity on a list of values. So, the expression for the above SQL will look like 1 <= ANY {month_part} AND 3 >= ANY {month_part}, where month_part will be substituted at runtime with different list of values for each of quarterly partitions. We will end up evaluating that expressions 4 times with the following list of values: Q1: 1 <= ANY {1,2,3} AND 3 >= ANY {1,2,3} Q2: 1 <= ANY {4,5,6} AND 3 >= ANY {4,5,6} ... Compare this to the previous approach, where we will end up evaluating 12 different expressions, each time for a single point value: First constraint of Q1: 1 <= 1 AND 3 >= 1 Second constraint of Q1: 1 <= 2 AND 3 >= 2 Third constraint of Q1: 1 <= 3 AND 3 >= 3 First constraint of Q2: 1 <= 4 AND 3 >= 4 ... The ScalarArrayOp depends on a new type of expression PartListRuleExpr that can convert a list rule to an array of values. ORCA specific changes can be found here: https://github.com/greenplum-db/gporca/pull/149
-
由 Ashwin Agrawal 提交于
auto-vacuum limit would be reached first and then warn limit, followed by other limits. So, there is no reason to rollback after bumping the xid to auto-vacuum limit. Can land in all kinds of wierd issues by doing the same. Practically, these tests need to be fully re-written maybe by modifying the GUCs and then actually generating XID to reach the same instead of similating by bumping the counter, but will attempt that in another commit.
-
由 Ashwin Agrawal 提交于
In some of persistent table functions, GetTopTransactionId() was called inside critical section. With laxy xid allocation if at DDL time transactionId stop limit has reached, this causes simple ERROR upgraded to PANIC. Hence modify code to call GetTopTransactionId() before entering critical section to avoid PANIC but just have ERROR as before.
-
由 Ashwin Agrawal 提交于
Commit fb86c90d "Simplify management of distributed transactions." cleanedup lot of code for LocalDistribXactData and introduced LocalDistribXactData in PROC for debugging purpose. But it's only correctly maintained for QE's, QD never populated LocalDistribXactData in MyProc. Instead TMGXACT also had LocalDistribXactData which was just set initially for QD but never updated later and confused more than serving the purpose. Hence removing LocalDistribXactData from TMGXACT, as it already has other fields which provide required information. Also, cleaned-up QD related states as even in PROC only QE uses LocalDistribXactData.
-
由 Ashwin Agrawal 提交于
As part of 8.3 merge, upstream commit 295e6398 "Implement lazy XID allocation" was merged. But transactionIds were still allocated in StartTransaction as code changes required to make it work for GPDB with distrbuted transaction was pending, thereby feature remained as disabled. Some progress was made by commit a54d84a3 "Avoid assigning an XID to DTX_CONTEXT_QE_AUTO_COMMIT_IMPLICIT queries." Now this commit addresses the pending work needed for handling deferred xid allocation correctly with distributed transactions and fully enables the feature. Important highlights of changes: 1] Modify xlog write and xlog replay record for DISTRIBUTED_COMMIT. Even if transacion is read-only for master and no xid is allocated to it, it can still be distributed transaction and hence needs to persist itself in such a case. So, write xlog record even if no local xid is assigned but transaction is prepared. Similarly during xlog replay of the XLOG_XACT_DISTRIBUTED_COMMIT type, perform distributed commit recovery ignoring local commit. Which also means for this case don't commit to distrbuted log, as its only used to perform reverse map of localxid to distributed xid. 2] Remove localXID from gxact, as its no more needed to be maintained and used. 3] Refactor code for QE Reader StartTransaction. There used to be wait-loop with sleep checking to see if SharedLocalSnapshotSlot has distributed XID same as that of READER to assign reader some xid as that of writer, for SET type commands till READER actually performs GetSnapShotData(). Since now a) writer is not going to have valid xid till it performs some write, writers transactionId turns out InvalidTransaction always here and b) read operations like SET doesn't need xid, any more hence need for this wait is gone. 4] Thow error if using distributed transaction without distributed xid. Earlier AssignTransactionId() was called for this case in StartTransaction() but such scenario doesn't exist hence convert it to ERROR. 5] QD earlier during snapshot creation in createDtxSnapshot() was able to assign localXid in inProgressEntryArray corresponding to distribXid, as localXid was known by that time. That's no more the case and localXid mostly will get assigned after snapshot is taken. Hence now even for QD similar to QE's snapshot creation time localXid is not populated but later found in DistributedSnapshotWithLocalMapping_CommittedTest(). There is chance to optimize and try to match earlier behavior somewhat by populating gxact in AssignTransactionId() once locakXid is known but currently seems not so much worth it as QE's anyways have to perform the lookups.
-
由 Ashwin Agrawal 提交于
-
由 Ashwin Agrawal 提交于
-
由 Ashwin Agrawal 提交于
-
由 Ashwin Agrawal 提交于
Leverage the fact that inProgressEntryArray is sorted based on distribXid while creating the snapshot in createDtxSnapshot. So, can break out fast in function DistributedSnapshotWithLocalMapping_CommittedTest().
-
由 Ashwin Agrawal 提交于
GetTopTransactionId() bumps the xid counter, doesn't seem required for this function. For GPDB this function may be called by QE READER, causing XID assignment in reader which is a voilation. Hence instead use ReadNewTransactionId() to just read current value instead of bumping the value.
-
- 31 3月, 2017 17 次提交
-
-
由 Omer Arap 提交于
-
由 Heikki Linnakangas 提交于
The old comment was unsure if toast tables and indexes have their Oids in sync across the cluster. They do, so let's rely on that, which is simpler.
-
由 Daniel Gustafsson 提交于
Rearranges ereport() calls and variable declarations to match the upstream style a little closer, and reformats some comments. The printing of the distribution policy was using strcat() safely, but it takes some thinking to be sure so add a comment explaining the code.
-
由 Daniel Gustafsson 提交于
When creating a table using the CTAS construction with an explicit distribution clause, the duplicate check for distribution columns wasn't applied: CREATE TABLE tt AS SELECT * FROM t DISTRIBUTED BY (c,c); Move the duplicate distribution check to the parsing stage since it is a syntax error to supply duplicate column references. This alters the behavior such that supplying a duplicate non-existing column will error out on the duplication rather than the column being non-existing. Since syntax errors should have precedence over semantic errors I however find that this is the correct order. Remove the existing duplicate checks which only applied to CREATE TABLE and ALTER TABLE, and add test cases to cover CTAS. The ticket reference was removed since it's not public and also clearly didn't cover all the cases. The DROP TABLE is also moved to not using IF EXISTS since it would be an error if the table didn't exist at that point.
-
由 Heikki Linnakangas 提交于
Digging into the git history, it was removed sometime during the 8.2 (sic) merge, back in 2006. I don't think it was intentional.
-
由 Heikki Linnakangas 提交于
It was added back in 2009. Seems quite clear at this point that we're not going to change how this works.
-
由 Heikki Linnakangas 提交于
It was added back in 2010, as part of a patch to: commit d0ca3d8a4333db510ac68145c30ff917626d2037 Author: kentj <a@b> Date: Mon Jan 25 13:50:14 2010 -0800 MPP-7734: initialize executor nodes for the active slice instead of the whole tree. Postponding the initialization of the subplans of an Append node to the time when it is processed. Send init gpmon packages for the subnodes of the Append node even though we don't initialize them. [git-p4: depot-paths = "//cdb2/main/": change = 43428] However, it was reverted only a few weeks later: commit 5cb7e64a093dfcc1bcbdca5ed74c261e9c56d3a3 Author: kentj <a@b> Date: Tue Feb 16 16:40:49 2010 -0800 MPP-8031, MPP-7734: revert back the Append changes, since it breaks the explain analyze. It is hard to fix the explain analyze with the existing Append changes. It needs some more thinking. [git-p4: depot-paths = "//cdb2/main/": change = 45078] The revert removed all use of the flag, but left the flag behind. Remove it.
-
由 Heikki Linnakangas 提交于
-
由 Heikki Linnakangas 提交于
A HashJoin would always have an outer_plan, hence the if-statement was pointless. It was added a long time ago in GPDB, due to a bug that was related to Adaptive Joins. We haven't had Adaptive Joins for ages, so it was dead code.
-
由 Heikki Linnakangas 提交于
-
由 Christopher Hajas 提交于
This reverts commit b20a3f92 and 37cdd250 since the gptransfer suite is failing.
-
由 Peifeng Qiu 提交于
S3QueryAbort's message was hardcoded to "Query is aborted". This commit initializes it via constructor.
-
由 Adam Lee 提交于
-
由 Christopher Hajas 提交于
This was missed in b20a3f92
-
由 Larry Hamel 提交于
Signed-off-by: NChumki Roy <croy@pivotal.io>
-
由 Chris Hajas 提交于
Instead of simply printing which files are different, print the contents of the diff files.
-
由 Todd Sedano 提交于
-
- 30 3月, 2017 2 次提交
-
-
由 Todd Sedano 提交于
-
由 Daniel Gustafsson 提交于
The gp_create_table suite tests that the maximum number of columns can be specified as a distribution clause (well, it really tests that CREATE TABLE allows the maximum number of columns since that check applies first), and then tests to CTAS from the resulting table. There is no reason to believe that there is a difference between CTAS with 1600 columns and CTAS with fewer columns. Remove this case to speed up the test significantly and also adjust the DROP TABLE clauses to match reality.
-