- 19 4月, 2018 6 次提交
-
-
由 David Kimura 提交于
Dispatcher has DISPATCH_WAIT_TIMEOUT_MSEC (current value is 2000) as poll timeout. It waited for 30 iterations of poll to timeout before checking the segment status. And then initiated fts probe before checking the segment status. As a result it took ~minute for query to fail in case of segment failures. This commit updates to check segment status on every poll timeout. It also leverages fts version to optimize whether to check segments. It avoids performing fts probe, instead it relies on fts to be called on regular intervals and provide cached results. With this change test time for twophase_tolerance_with_mirror_promotion was cut down by ~2 minutes. Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>
-
由 Ashwin Agrawal 提交于
Sometimes test need to use this fault to suspend and then stop the database. If this fault is inside the HOLD_INTERRUPTS (which is where its before this change), CHECK_FOR_INTERRUPTS doesn't do anything. Hence stopping of segment takes super long time as kill -9 needs to be enforced by gpstop after timeout. Co-authored-by: NDavid Kimura <dkimura@pivotal.io>
-
由 Ning Yu 提交于
In ResGroupDropFinish() uninitialized memory address can be accessed due to some reasons: 1. the group pointer is not initialized on segments; 2. the hash table node pointed by group is recycled in removeGroup(); This invalid access can cause crash issue on segments. Also move some global vars to resgroup.c, They were put in resgroup-ops-linux.c, which was only compiled and linked on linux, so on other OS like macos the vars can not be found.
-
由 Jialun 提交于
User query can use global shared memory across resource group when available(Same as the PR 4843, just made the test cases stable) (#4866) 1) Global shared memory will be used if the query has run out of the group shared memory. 2) The limit of memory_spill_ratio changes to [0, INT_MAX], because global shared memory can be allocated, 100% limitation is not make sense. 3) Using Atomic Operation & "Compare And Save" instead of lock to get high performance. 4) Modify the test cases according to the new rules.
-
由 Ning Yu 提交于
Cgroup mount point is detected during startup, but due to a bug the mount point was used even if cgroup is not mounted at all. Fixed by correcting the checking logic.
-
由 Ning Yu 提交于
Memory auditor was a new feature introduced to allow external components (e.g. pl/container) managed by resource group. This feature requires a new gpdb dir to be created in cgroup memory controller, however on 5X branch unless the users created this new dir manually then the upgrade from a previous version would fail. In this commit we provide backward compatibility by checking the release version: - on 6X and master branches the memory auditor feature is always enabled so the new gpdb dir is mandatory; - on 5X branch only if the new gpdb dir is created with proper permissions the memory auditor feature could be enabled, when it's disabled `CREATE RESOURCE GROUP WITH (memory_auditor='cgroup') will fail with guide information on how to enable it; Binary swap tests are also provided to verify backward compatibility in future releases. As cgroup need to be configured to enable resgroup we split the resgroup binary swap tests into two parts: - resqueue mode only tests which can be triggered in the icw_gporca_centos6 pipeline job after the ICW tests, these parts have no requirements on cgroup; - complete resqueue & resgroup modes tests which can be triggered in the mpp_resource_group_centos{6,7} pipeline jobs after the resgroup tests, these parts need cgroup to be properly configured;
-
- 18 4月, 2018 1 次提交
-
-
由 Tom Lane 提交于
We are developing on a machine with flex 2.6.4 and we couldn't build. This was fixed in 72b1e3a2 in upstream (9.6 at the time) in 2016, but wasn't backported to 9.1 or older. Original commit message: > Now that we know about the %top{} trick, we can revert to building flex > lexers as separate .o files. This is worth doing for a couple of reasons > besides sheer cleanliness. We can narrow the scope of the -Wno-error flag > that's forced on scan.c. Also, since these grammar and lexer files are > so large, splitting them into separate build targets should have some > advantages in build speed, particularly in parallel or ccache'd builds. > > We have quite a few other .l files that could be changed likewise, but the > above arguments don't apply to them, so the benefit of fixing them seems > pretty minimal. Leave the rest for some other day. (cherry picked from commit 72b1e3a2) Fixes greenplum-db/gpdb#4863 Co-authored-by: NJesse Zhang <sbjesse@gmail.com> Co-authored-by: NAsim R P <apraveen@pivotal.io>
-
- 17 4月, 2018 3 次提交
-
-
由 Jialun 提交于
1) Global shared memory will be used if the query has run out of the group shared memory. 2) The limit of memory_spill_ratio changes to [0, INT_MAX], because global shared memory can be allocated, 100% limitation is not make sense. 3) Using Atomic Operation & "Compare And Save" instead of lock to get high performance. 4) Modify the test cases according to the new rules.
-
由 David Kimura 提交于
Large objects are currently not supported in Greenplum. Rather than deceive the user with a non-functional large object api, we disable them for now. We disable the large object tests in privileges regress test by using ignore blocks instead of commenting them out or deleting them to reduce merge conflicts in future postgres merges. Co-authored-by: NJimmy Yih <jyih@pivotal.io>
- 14 4月, 2018 2 次提交
-
-
由 Ashwin Agrawal 提交于
When the blocksize is 2MB, the function AppendOnlyStorage_GetUsableBlockSize would give out the wrong usable block size. The expected result is 2MB. But the return value of the function call would give out (2M -4). This is because the macro AOSmallContentHeader_MaxLength is defined as (2M -1). After rounding down to 4 byte aligned, the result is (2M - 4). Without the fix can encounter errors as follows: "ERROR: Used length 2097152 greater than bufferLen 2097148 at position 8388592 in table 'xxxx'". Also removed some related, but unused macro variables, just for cleaning up codes related to AO storage. Co-authored-by: NLirong Jian <jian@hashdata.cn>
-
由 Venkatesh Raghavan 提交于
Changes in 8.4 for statistics computation is comprehensive. Will fix any fallouts of this change as we observe them. Current tests show that things are kosher so removing the fix me.
-
- 13 4月, 2018 1 次提交
-
-
由 Asim R P 提交于
Commit b3f300b9 introduced the novel idea tracking oldest xmin among all distributed snapshots on QEs. However, the idea is not applicable to QD because all distributed transactions can be found in ProcArray on QD. Local oldest xmin is therefore the oldest xmin among all distributed snapshots on QD. This patch fixes the maintenance of oldest xmin on QD by avoiding DistributedLog_AdvanceOldestXmin() and all the heavy-lifting that it performs. Calling this on QD was also hitting the "local snapshot's xmin is older than recorded distributed oldestxmin" error occasionally in CI.
-
- 12 4月, 2018 3 次提交
-
-
由 Jacob Champion 提交于
8.2->8.3 upgrade of NUMERIC types was implemented for row-oriented AO tables, but not column-oriented. Correct that here. Store upgraded Datum data in a per-DatumStream buffer, to avoid "upgrading" the same data multiple times (multiple tuples may be pointing at the same data buffer, for example with RLE compression). Cache the column's base type in the DatumStreamRead struct. Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
-
由 Bhuvnesh Chaudhary 提交于
This reverts commit 8b0a7fed. Due to this commit, full join queries with condition on varchar columns started failing due to the below error. It is expected that there is a relabelnode on top of varchar columns while looking up the sort operator, however because of the said commit we removed the relabelnode. ```sql create table foo(a varchar(30), b varchar(30)); postgres=# select X.a from foo X full join (select a from foo group by 1) Y ON X.a = Y.a; ERROR: could not find member 1(1043,1043) of opfamily 1994 (createplan.c:4664) ``` Will reopen the issue #4175 which brought this patch.
-
由 Andreas Scherbaum 提交于
-
- 11 4月, 2018 4 次提交
-
-
由 Pengzhou Tang 提交于
Executing a query plan containing a large number of slices may slow down the entire Greenplum cluster: each "n-gang" slice corresponds to a separate process per segment. An example of such queries is a UNION ALL atop several complex views. To prevent such a situation, add a GUC gp_max_slices and refuse to execute plans of which the number of slices exceed that limit. Signed-off-by: NJesse Zhang <sbjesse@gmail.com>
-
由 xiong-gang 提交于
This GUC determines whether curl verifies the authenticity of the gpfdist's certificate
-
由 Bhuvnesh Chaudhary 提交于
The patch 62aba765 from upstream fixed the CVE-2009-4136 (security vulnerability) with the intent to properly manage session-local state during execution of an index function by a database superuser, which in some cases allowed remote authenticated users to gain privileges via a table with crafted index functions. Looking into the details of the CVE-2009-4136 and related CVE-2007-6600, the patch should ideally have limited the scope while we calculate the stats on the index expressions, where we run functions to evaluate the expression and could potentially present a security threat. However, the patch changed the user to table owner before collecting the sample, due to which even if analyze was run by superuser the sample couldn't be collected as the table owner did not had sufficient privileges to access the table. With this commit, we switch back to the original user while collecting the sample as it does not deal with indexes, or function call which was the original intention of the patch. Upstream did not face the privilege issue, as it does block sampling instead of issuing a query. Signed-off-by: NSambitesh Dash <sdash@pivotal.io>
-
由 Abhijit Subramanya 提交于
This FIXME is two-fold: - Handling LIMIT 0 The LIMIT is already handled in the caller, convert_EXISTS_sublink_to_join(): When an existential sublink contains an aggregate without GROUP BY or HAVING, we can safely replace it by a one-time TRUE/FALSE filter based on the type of sublink since the result of aggregate is always going to be one row even if it's input rows are 0. However this assumption is incorrect when sublink contains LIMIT/OFFSET, such as, if the final limit count after applying the offset is 0. - Rules for demoting HAVING to WHERE previously, simplify_EXISTS_query() only disallowed demoting HAVING quals to WHERE, if it did not contain any aggregates. To determine the same, previously it used query->hasAggs, which is incorrect since hasAggs indicates that aggregate is present either in targetlist or HAVING. This penalized the queries, wherein HAVING did not contain the agg but targetlist did (as demonstrated in the newly added test). This check is now replaced by contain_aggs_of_level(). Also, do not demote if HAVING contains volatile functions since they need to be evaluated once per group. Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>
-
- 10 4月, 2018 5 次提交
-
-
由 Richard Guo 提交于
Relations with external storage, as well as AO and CO should have InvalidTransactionId in relfrozenxid during upgrade. The idea here is to keep the same logic with function should_have_valid_relfrozenxid().
-
由 Bhuvnesh Chaudhary 提交于
This reverts commit 54ee5b5c. In qual_is_pushdown_safe_set_operation it crashed while Assert(subquery)
-
由 Bhuvnesh Chaudhary 提交于
Previously, if there was a subquery contaning window functions, pushing down of the filters was banned. This commit fixes the issue, by pushing downs filters which are not on the columns projected using window functions. Adding relevant tests. Test case: After porting the fix to gpdb master, in the below case the filter `b = 1` is pushed down on ``` explain select b from (select b, row_number() over (partition by b) from foo) f where b = 1; QUERY PLAN ---------------------------------------------------------------------------------------------------- Gather Motion 3:1 (slice2; segments: 3) (cost=0.00..1.05 rows=1 width=4) -> Subquery Scan on f (cost=0.00..1.05 rows=1 width=4) -> WindowAgg (cost=0.00..1.04 rows=1 width=4) -> Redistribute Motion 3:3 (slice1; segments: 3) (cost=0.00..1.03 rows=1 width=4) Hash Key: foo.b -> Seq Scan on foo (cost=0.00..1.01 rows=1 width=4) Filter: b = 1 Optimizer: legacy query optimizer ``` Currently on master the plan is as below where the filter is not pushed down. ``` explain select b from (select b, row_number() over (partition by b) from foo) f where b = 1; QUERY PLAN ---------------------------------------------------------------------------------------------------- Gather Motion 3:1 (slice2; segments: 3) (cost=1.04..1.07 rows=1 width=4) -> Subquery Scan on f (cost=1.04..1.07 rows=1 width=4) Filter: f.b = 1 -> WindowAgg (cost=1.04..1.06 rows=1 width=4) Partition By: foo.b -> Sort (cost=1.04..1.04 rows=1 width=4) Sort Key: foo.b -> Redistribute Motion 3:3 (slice1; segments: 3) (cost=0.00..1.03 rows=1 width=4) Hash Key: foo.b -> Seq Scan on foo (cost=0.00..1.01 rows=1 width=4) Optimizer: legacy query optimizer
-
由 Ben Christel 提交于
This is just refactoring. Related to (#4690). Co-authored-by: NBen Christel <bchristel@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
-
由 Abhijit Subramanya 提交于
This commit replaces the usage of CdbOpenRelationRv() with parserOpenTable() for updates and deletes in setTargetList(). We cannot use the same function for inserts since it upgrades the lock to Exclusive mode so leave the heap_openrv() call intact there.
-
- 09 4月, 2018 2 次提交
-
-
由 Jacob Champion 提交于
Partial backport of functionality from commit 2ef085d0 in Postgres 9.4, which checks to make sure that a toast table has a valid index. Prior to this change, toast tables with invalid indices (for example, after a pg_upgrade) caused confusing error messages of the form missing chunk number 0 for toast value <> in pg_toast_<> A regression test case has been added using isolation2.
-
由 Pengzhou Tang 提交于
GUC "is_supersuer" only provide value for SHOW to display, it's useless on segments. Those two flags are all designed to determine the value of is_superuser, so it's unnecessary to pass them to the segments. With this commit, another problem is resolved, GPDB used to dispatch a command within empty transaction and resource owner to define an index concurrently. However, a syscache access in superuser_arg() may report a SIGSEGV because CurrentResourceOwner is NULL, this commit use SessionUserIsSuperuser instead of superuser_arg() to avoid such error.
-
- 08 4月, 2018 1 次提交
-
-
由 Paul Guo 提交于
For pg_upgrade, gpdb runs in GP_ROLE_UTILITY mode and with IsBinaryUpgrade set as true. In this patch we allow some partition related code run if IsBinaryUpgrade is true so that those partition related sql clauses, which are generated by pgdump, could recover previous partition schemas. Co-authored-by: NMax Yang <myang@pivotal.io>
-
- 07 4月, 2018 3 次提交
-
-
由 Asim R P 提交于
To enforce consistent isolation level within a distributed transaction, local transactions on QEs should assume the same isolation level as the transaction on QD. This was previously achieved by dispatching isolation level and read-only property from QD to QE as command line options. In case of explicit BEGIN, the isolation level was dispatched as flags in DtxContextInfo. This patch makes it consistent such that QEs (readers as well as writers) read the transaction options from DtxContextInfo. The poblem with setting transaction isolation level as command line opiton is command line options are processed during process initialization, when a transaction is already open and a snapshot is taken. Changing isolation level after taking a snapshot is not correct. This patch allows merging with the check for transaction_isolation GUC as it stands in 9.1, without any Greenplum-specific changes. Co-authored-by: NJacob Champion <pchampion@pivotal.io>
-
由 Asim R P 提交于
This was probably an oversight, there is no need to update "ready" field of SharedLocalSnapshotSlot object separately.
-
由 Asim R P 提交于
No change in functionality here. The ereportif() macro avoids ereport() invocation if the preficate is not true. Fixed indentation in a couple of places on the way.
-
- 05 4月, 2018 2 次提交
-
-
由 Jimmy Yih 提交于
This was found when server encoding was WIN1251 but the client encoding was UTF8. The conversion is valid for these character sets but the client would receive an "invalid byte sequence for encoding" error. We fix the issue by calling the correct conversion function. It was switched up during a postgres backport last year. Thanks to Heikki for debugging the issue.
-
由 Jimmy Yih 提交于
There was a GPDB_90_MERGE_FIXME regarding what we should report for ALTER ROLE IN DATABASE in pg_stat_last_shoperation table for metadata tracking. We should always report that the role object was altered if coming from ALTER ROLE code path. To do this, we just check if the role oid is valid because the call to AlterSetting from ALTER DATABASE uses InvalidOid for role oid. While we're at it, we might as well add some tests for pg_stat_last_shoperation.
-
- 03 4月, 2018 3 次提交
-
-
由 Adam Lee 提交于
The pg_exttable.fmterrtbl column stored the OID of the error table, but without an error table it is just set to the OID of the external table. That is not necessary, there are other columns which indicate if error logging is enabled. Therefore this column can be removed.
-
由 Amil Khanzada 提交于
With this framework, extension developers can set idle_session_timeout_action_hook and get_idle_session_timeout_hook to implement actions to take after a session has gone idle for a specified time period (e.g. killing it). The code assumes that the (idle gang timeout) < (idle session timeout); if the reverse happens the session is killed at (idle gang timeout). Co-authored-by: NDavid Sharp <dsharp@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io> Co-authored-by: NBen Christel <bchristel@pivotal.io>
-
由 Ben Christel 提交于
The functionality for gp_vmem_idle_resource_timeout is implemented by these files. Refactor to make it TDDable and easier to understand. We moved DisconnectAndDestroyUnusedGangs to cdbgang because it will be a natural seam for our mocks when we write more unit tests that expect it to be called. Also, it seems to belong to that file given all of the functions it calls are already in cdbgang. Co-authored-by: NBen Christel <bchristel@pivotal.io> Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
-
- 02 4月, 2018 2 次提交
-
-
由 Pengzhou Tang 提交于
-
由 Pengzhou Tang 提交于
This can make GpPolicyFetch() a little bit faster.
-
- 31 3月, 2018 1 次提交
-
-
由 Ashwin Agrawal 提交于
Currently, in CI autovacuum worker is randmly hitting faults intended for other backends by test and causing failures. Since there exists no test now to test auto-vacuum processes with fault injector framework, lets skip them. Other alternative would be to vacuum all existing databases at end of auto-vacuum test to drop age for every database and avoid triggering auto-vacuum after that test but that seems too heavy and unnecessary.
-
- 30 3月, 2018 1 次提交
-
-
由 Jimmy Yih 提交于
The pg_tablespace_location function only returns the tablespace location of the connected segment. For Greenplum, we need to be able to easily see the tablespace locations for all segments. This is especially needed because we support defining tablespace locations per-segment.
-