- 15 6月, 2016 3 次提交
-
-
由 Heikki Linnakangas 提交于
The printablePredicate of a static PartitionSelector node contains Var nodes with varno=INNER. That's bogus, because the PartitionSelector node doesn't actually have any child nodes, but works at execution time because the printablePredicate is not only used by EXPLAIN. In most cases, it still worked, because most Var nodes carry a varnoold field, which is used by EXPLAIN for the lookup, but there was one case of "bogus varno" error even memorized in the expected output of the regression suite. (PostgreSQL 8.3 changed the way EXPLAIN resolves the printable name so that varnoold no longer saves the bacon, and you would get a lot more of those errors) To fix, teach the EXPLAIN of a Sequence node to also reach into the static PartitionSelector node, and print the printablePredicate as if that qual was part of the Sequence node directly. The user-visible effect of this is that the static Partition Selector expression now appears in EXPLAIN output as a direct attribute of the Sequence node, not as a separate child node. Also, if a static Partition Selector doesn't have a "printablePredicate", i.e. it doesn't actually do any selection, it's not printed at all.
-
由 Heikki Linnakangas 提交于
This is mostly copy-pasted from CoerceToDomain. Note: This requires an up-to-date version of ORCA to compile, older versions of the ORCA library itself don't know about CoerceViaIO nodes either.
-
由 Nikos Armenatzoglou 提交于
When SharedScan requests a tuple from the underlying Sort, and the Sort has spilled to disk, the mk_sort code (also the regular sort) makes a copy of the memtuple, but places it in the sortcontext. When hitting a QueryFinishPending (e.g., master received enough tuples, so stop the execution), EagerFree is called for the entire tree. When reaching the sort node, the sortcontext is deleted, even though the SharedScan above it still has a pointer to a memtuple allocated in this context. The solution is: pass a memory context for the mk_sort and regular sort to use (change the API and implementation of the following two functions: tuplesort_gettupleslot_pos_mk and tuplesort_gettupleslot_pos). Those functions should use the context provided by the caller to make a copy of the memtuple. Signed-off-by: NGeorge Caragea <gcaragea@pivotal.io>
-
- 14 6月, 2016 1 次提交
-
-
由 Daniel Gustafsson 提交于
Fixed a few spelling errors, and minor whitespace issues, in the Resource Scheduling README while reading it. Also updated the file markers on the README, Makefile and code to match upstream CVS to Git conversion. No semantic changes made to the text.
-
- 13 6月, 2016 2 次提交
-
-
由 Kenan Yao 提交于
Include a map from sliceIndex to gang_id in the dispatched string, and remove the localSlice field, hence QE should get the localSlice from the map now. By this way, we avoid duplicating and modifying the dispatch text string slice by slice, and each QE of a sliced dispatch would get same contents now. The extra space cost is sizeof(int) * SliceNumber bytes, and the extra computing cost is iterating the SliceNumber-size array. Compared with memcpy of text string for each slice in previous implementation, this way is much cheaper, because SliceNumber is much smaller than the size of dispatch text string. Also, since SliceNumber is so small, we just use an array for the map instead of a hash table. Also, clean up some dead code in dispatcher, including: (1) Remove primary_gang_id field of Slice struct and DispatchCommandDtxProtocolParms struct, since dispatch agent is deprecated now; (2) Remove redundant logic in cdbdisp_dispatchX; (3) Clean up buildGpDtxProtocolCommand;
-
由 Pengzhou Tang 提交于
In commit d2725929, GPDB marked all allocatedReaderGangs with noReuse flag. When plan contains init plan and a SET command executed within it, GPDB will mark pre-assigned gangs to noReuse and destroy them which make query crash
-
- 11 6月, 2016 4 次提交
-
-
由 Foyzur Rahman 提交于
The GPDB Vmem is the lowest layer of memory allocator that supports higher allocators such as AllocSet. This layer (mostly defined in memprot.c) is in charge of actually calling malloc/realloc/free to allocate/reallocate/free memory. In this process this layer is also in charge of reserving "virtual" memory or Vmem, which is a GPDB specific shared memory counter to track per-segment combined allocations across all the GPDB processes under Vmem umbrella. The Vmem counter is managed by a separate module Vmem_Tracker, and the memprot functions (such as gp_malloc, gp_free2 and gp_realloc) call the APIs provided by VmemTracker. Previously the memprot allocators (gp_alloc/gp_realloc/gp_free) were only allocating/freeing memory but were not adding any additional metadata in the header (and there was no header) to track the size of allocations. Therefore, there was no gp_free as freeing memory requires the size of the free to adjust Vmem counter inside VmemTracker. This was patched by explicitly passing size info in gp_free2. In this PR we do the following: * We add allocation size in Vmem header (along with checksums which are only available in debug build to detect header and footer boundary, and buffer overruns). * We remove size information from the block header of AllocSet. * We rename gp_free2 to gp_free as the second parameter (size information) is now obtained from the header and therefore no longer necessary * We modify all the consumers of memprot.c APIs to use the new APIs * We add unit tests to test the metadata and the correctness of the new Vmem allocators This is the first step to integrate external modules and third party allocations with Vmem. A long running issue in GPDB is its inability to track allocations by external components including libraries such as ORCA. Therefore, the central Vmem counter is often way off from the underlying allocations, and this may run the system out of memory. By maintaining the size information in the Vmem header, we now have a self-contained allocator that can be exposed to external allocators such as GPOS allocators, without forcing them to manage size information separately. This fixes #117269929. Signed-off-by: NMarc Spehlmann <marc.spehlmann@gmail.com>
-
This GUC will be used to control the MEMO size as well as optimization time for large IN list or large array comparison expressions. Only the Array with less number of elements than the GUC will be expanded and participate in constraint derivation. Trade-off of using this GUC is loss of potential benefits from the constraint derivation (e.g. conflict detection, partition elimination) with shorter optimization time and less memory utilization.
-
由 Nikos Armenatzoglou 提交于
-
由 Nikos Armenatzoglou 提交于
This reverts commit d9edb869. g_dataSourceCtx should not be reset in AtAbort_ExtTables, since external tables is not the only component that uses it.
-
- 09 6月, 2016 2 次提交
-
-
由 Heikki Linnakangas 提交于
Backport this patch from PostgreSQL 9.0, which replaces the tuple table array with a linked list of individually palloc'd slots. With that, we don't need to know the size of the array beforehand, and don't need to count the slots. The counting was especially funky for subplans in GPDB, and it was about to change with the upcoming PostgreSQL 8.3 merge again. This makes it a lot simpler. I don't plan to backport the follow-up patch to remove the ExecCountSlots infrastructure. We'll get that later, when we merge with PostgreSQL 9.0. commit f92e8a4b Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Sun Sep 27 20:09:58 2009 +0000 Replace the array-style TupleTable data structure with a simple List of TupleTableSlot nodes. This eliminates the need to count in advance how many Slots will be needed, which seems more than worth the small increase in the amount of palloc traffic during executor startup. The ExecCountSlots infrastructure is now all dead code, but I'll remove it in a separate commit for clarity. Per a comment from Robert Haas.
-
由 Heikki Linnakangas 提交于
This reverts much of the changes vs. upstream, related to temp schema creation. Instead of using the normal CREATE SCHEMA processing to also create the temporary schema, let InitTempTableNameSpace() to do that like in the upstream. But in addition to creating the the temp schema locally, it dispatches a special CreateSchemaStmt command to the executor nodes, which instructs the executor nodes to also call InitTempTableNameSpace().
-
- 08 6月, 2016 4 次提交
-
-
由 Asim R P 提交于
If found inconsistent, the free tid list will be rebuilt automatically during recovery. During normal operation, super user may invoke the function gp_persistent_freelist_rebuild(OID) to rebuild the free list. A basic test case is added to verify sanity of a free tid list rebuilt using the function.
-
由 Asim R P 提交于
Break long elog messages into multiple lines, remove trailing whitespace and start elog messages with lower case.
-
由 Nikos Armenatzoglou 提交于
-
This closes #815
-
- 07 6月, 2016 12 次提交
-
-
由 Heikki Linnakangas 提交于
This allows removing the weird pfree() of the resultTupleSlot's tuple descriptor. What would've happened without the pfree() is that the old slot was allocated in the first ExecAssignScanProjectionInfo() call, in partitionMemoryContext, and then immediately destroyed when the memory context was reset. The second call to ExecAssignScanProjectionInfo() tries to free the slot, again, causing the segfault. But we can avoid that by this rearrangement of the calls in a cleaner way. In the passing, clean up the code a bit. I found having separate variables, indexState and scanState, which point to the same struct, to be confusing.
-
由 Heikki Linnakangas 提交于
ss_ScanTupleSlot is not an array, it's a single slot. The slot is allocated from a bigger array, however, so this trampled over some other slot that was allocated right after the scan slot. This has apparently been harmless, as no-one's noticed, but it's surely wrong. I bumped into this in the PostgreSQL 8.3 merge branch, where I had changed the way the slots are allocated so that they're not stored in one big array anymore. This bug led to segfaults in that case.
-
由 Heikki Linnakangas 提交于
We had later added code that allows "DEFAULT NULL", by doing a table rewrite. DEFAULT NULL is really the same as no default, so we might as well do a table rewrite for that case too, and save the code needed to handle them differently.
-
由 Heikki Linnakangas 提交于
A function might legitimately assume that an argument's varlen datum always has a 4-byte header, if the datatype is marked as 'plain', and crash if we then pass it a datum with 1-byte header, because it was packed in a memtuple. I bumped into this while working on the PostgreSQL 8.3 merge, because the merge brought us one such function: ts_rewrite().
-
由 Heikki Linnakangas 提交于
The one in nodeBitmapHeapScan.c is what the upstream has. The one in execBitmapHeapScan.c was copied from it in GPDB. No need for the duplication. It's not clear to me why we have the execBitmapHeapScan.c file at all, why not just use all the functions in nodeBitmapHeapScan.c. But I'll leave investigating that for another day.
-
由 Heikki Linnakangas 提交于
This has the effect that gpmapreduce is exempt from logging also when it uses the extended query protocol (we only to only perform the checks in exec_simple_query()). That makes no difference in practice, though, because gpmapreduce doesn't actually use the extended query protocol.
-
由 Heikki Linnakangas 提交于
This isn't strictly necessary, but makes the code easier to read, IMHO. And it seems like a very useful property that the index can only exist in the segments, if it exists in the master.
-
由 Heikki Linnakangas 提交于
-
由 Heikki Linnakangas 提交于
It seems better to send in compressed form, and decompress in the receiver, to reduce network I/O. The amount of CPU work required for the decompression is the same whether its done in the sender or the receiver. There was even a comment implying that, but for some reason, we didn't do it that way. This is in preparation for the PostgreSQL 8.3 merge: The HEAP_COMPRESSED (and HEAP_HASEXTENDED which included it) flag was removed in PostgreSQL 8.3.
-
由 Heikki Linnakangas 提交于
Commit dc87e717 moved DataSourceContext to TopTransactionContext, so it doesn't need to be explicitly destroyed at end of transaction anymore. But the pointer to it still needs to be reset. Fixes failures in the external_table regression test.
-
由 Heikki Linnakangas 提交于
Per report and analysis by Asim R P.
-
由 Foyzur Rahman 提交于
Signed-off-by: NMarc Spehlmann <marc.spehlmann@gmail.com>
-
- 06 6月, 2016 3 次提交
-
-
由 Heikki Linnakangas 提交于
This is a partial backport of a larger body of work which also already have been partially backported. Remove the GPDB-specific "breadcrumbs" mechanism from the parser. It is made obsolete by the upstream mechanism. We lose context information from a few errors, which is unfortunate, but seems acceptable. Upstream doesn't have context information for those errors either. The backport was originally done by Daniel Gustafsson, on top of the PostgreSQL 8.3 merge. I tweaked it to apply it to master, before the merge. Upstream commit: commit b153c092 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Mon Sep 1 20:42:46 2008 +0000 Add a bunch of new error location reports to parse-analysis error messages. There are still some weak spots around JOIN USING and relation alias lists, but most errors reported within backend/parser/ now have locations.
-
由 Heikki Linnakangas 提交于
The function was rewritten in GPDB, and its behaviour was changed to also return 'true' if the expression contains an Aggref of the given level. That change in behaviour was made back in 2006, as part of a commit containing a lot of subquery optimization changes. I could not find an explanation for that particular change, and all the regression tests pass without so I assume that it has become obsolete at some point over they years. This smoothens the way for future merges with upstream, by reducing the diff in both code and behaviour. Also, you get a more accurate error message in a few cases, as seen by the changes to expected output.
-
由 Tom Lane 提交于
This will support fixing contrib/intarray (and probably other places) so that they don't have to fail on arrays that contain a null bitmap but no live null entries.
-
- 05 6月, 2016 1 次提交
-
-
由 Daniel Gustafsson 提交于
Fixes compiler warning for implicit function declaration in proc.c
-
- 04 6月, 2016 2 次提交
-
-
由 Nikos Armenatzoglou 提交于
Signed-off-by: NGeorge Caragea <gcaragea@pivotal.io>
-
由 Heikki Linnakangas 提交于
If log_min_messages = 'debug5', we call the text output function for everything that's serialized to be sent from QD to QEs. I added a new node type, QueryDispatchDesc, in commit 41478b89, but I only gave it the "fast" output function, not the text version. That produced a warning about missing output function, with log_min_messages='debug5'. This fixes the failure of qp_executor test, reported by Omar Arap.
-
- 03 6月, 2016 6 次提交
-
-
由 Heikki Linnakangas 提交于
The readRangeTblEntry() function was missing the line for 'pseudocols' field. I'm surprised the function worked at all, I thought the read functions don't work if there are any extra fields. Maybe they're more forgiving if it's last field that's missing. In any case, seems like an oversight. It doesn't matter in practice, as the pseudocols field is only used during planning, and we don't serialize nodes at that stage. Rule definitions are serialized before planning, and for the transfer between QD and QEs, we use the 'fast' versions of these functions. In the 'fast' version, the 'pseudocols' is missing from both the out function and the read function. That seems intentional, so add a comment about it.
-
由 Heikki Linnakangas 提交于
The copy/out/read functions for it were wrong: a Bitmapset is not a Node, so one should use e.g. COPY_BITMAPSET_FIELD() instead of COPY_NODE_FIELD() for them. But since the fields are currently unused, let's just remove them. These fields will be resurrected soon, by the PostgreSQL 8.3 merge, as they were introduced in PostgreSQL 8.3. Then they will actually be used, too.
-
由 Heikki Linnakangas 提交于
Seems more straightforward. Firstly, modifying a PlannedStmt at execution is dubious, if the PlannedStmt is reused for executing the same query again. Secondly, if the original Query objects are indeed not needed after planning, we can save a little bit of memory, and avoid the overhead of stripping the plan on every execution. This also makes any breakage more obvious, if it turns out that the Query is actually still needed at execution for some reason, as that issue would then show up on the first execution already, and not only on reuse of a PlannedStmt. Per Kenan Yao's observation that stripPlanBeforeDispatch() also scribbles on the PlannedStmt.
-
由 Heikki Linnakangas 提交于
This is also transient information, only valid during execution, and should not be cached with the plan.
-
由 Heikki Linnakangas 提交于
That includes the slice table, transientRecordTypes, and IntoClause's oidInfo. These are transient information, created in ExecutorStart, not something that should be cached along with the plan. transientRecordTypes and oidInfo in particular were stored in PlannedStmt only so that they can be conveniently dispatched to QEs along with the plan. That's not a problem at the moment, but with the upcoming PostgreSQL 8.3 merge, we'll start keeping the PlannedStmt struct around for many executions, so let's create a new struct to hold that kind of information, which is transmitted from QD to QEs along with the plan (that new struct is called QueryDispatchDesc).
-
由 Heikki Linnakangas 提交于
This isn't a problem right now, but with PostgreSQL 8.3, PlannedStmt is supposed to be immutable, because plan caching works by saving the PlannedStmt. So let's tidy this up, before we merge that from the upstream. There are more GPDB-added fields in there that are modified, but this is a start.
-