- 22 2月, 2017 1 次提交
-
-
由 Peifeng Qiu 提交于
gp_dump_agent is used by management utils for backup/restore operation. Apply similar patch for pg_dump to cdb_dump_agent. Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>
-
- 21 2月, 2017 1 次提交
-
-
由 Haozhou Wang 提交于
We will first support external table query execution on master segment before we fully support task dispatch on master segment. 1. Modify grammar to enable ON clause for external table. 2. Modify ExtTableEntry catalog to record on clause. 3. Update legacy planner to support on clause for external table. 4. Update pg_dump. 5. Update regress tests. Signed-off-by: NHaozhou Wang <hawang@pivotal.io> Signed-off-by: NAdam Lee <ali@pivotal.io> Signed-off-by: NYuan Zhao <yuzhao@pivotal.io> * Update catalog version For "Support ON MASTER clause for external table" * Add on master regression tests for gpcloud
-
- 15 2月, 2017 1 次提交
-
-
由 Heikki Linnakangas 提交于
There was a rather ugly mechanism in fileam.c, to ensure that the utl_fclose() function was called for any still-open external table, on abort. That relied on a global variable and assumed that there can be only one external table open at a time, which seems like a shaky assumption. With this patch, utl_file() is no longer guaranteed to be called on abort. Instead, make the UTL_FILE implementations responsible for cleaning up any low-level resources, like file descriptors or handles to external libraries (like libcurl), themselves. Use the ResourceOwner mechanism for that. For curl- and EXECUTE-style external tables, register a new ResourceOwner hook to close with the libcurl handles, and launched subprocesses, on abort. For file external tables, modify fstream.c to open the underlying file with OpenTransientFile(), so that it gets closed automatically on abort. This changes the url_fopen() API, so that it never returns a NULL. Instead of returning NULL, and reporting an error in the *response_code/string arguments, make url_fopen() always report errors directly with ereport(). That makes the callers simpler, and allows the implementations to flexibly report whatever details of the error they have available.
-
- 13 2月, 2017 2 次提交
-
-
由 Daniel Gustafsson 提交于
-pthread is coming as part of PTHREAD_CFLAGS, no need to add it separately.
-
由 Daniel Gustafsson 提交于
cdb_dumpall_agent.c is neither compiled, not does it contain anything of interest. Kill it for now, if we want cdb_dumpall_agent it can be resurrected or started on from scratch.
-
- 07 2月, 2017 2 次提交
-
-
由 Daniel Gustafsson 提交于
-
由 Daniel Gustafsson 提交于
External tables cannot inherit attributes when acting as a partition member in a partititioned table. Attribute inheritance was cleaned up in upstream commit 02e64181 which was part of the 8.3 merge, and as an effect of this external tables were dumped as inheriting the partition parent attributes. Fix by always setting the attributes as local for external tables and make it even more explicit by altering the attribute check in dumpExternal to only consider dropped attributes. Apply the fix to both pg_dump and cdb_dump_agent albeit a tad differently as pg_dump has abstracted the logic for printing column names. Report and debugging by Chris Hajas, Lawrence Hamel and Karen Huddleston
-
- 28 1月, 2017 1 次提交
-
-
由 Christopher Hajas 提交于
-
- 18 1月, 2017 4 次提交
-
-
由 Daniel Gustafsson 提交于
There was previously no support added for CREATE OPERATOR in the binary upgrade code, add Oid pre-assignment support for operators.
-
由 Daniel Gustafsson 提交于
-
由 Daniel Gustafsson 提交于
-
由 Daniel Gustafsson 提交于
The binary-upgrade support was added with the pg_upgrade patch and allows for the Oids in the old cluster to be pre-assigned to their respective objects during pg_upgrade. The main motivation for this refactoring is that we need to pre-assign all Oids before object creation during restore into the new cluster since a restore can otherwise allocate an Oid which is later preassigned. This brings all pre-assignments into the head of the dumpfile such that they happen before most object creations (pg_dumpall objects are still written to the dumpfile before pg_dump has a chance). The main contributions of this patch include: * All binary-upgrade methods are moved to binary_upgrade.{c|h} for pg_dump and binary_upgradeall.{c[h} for pg_dumpall. This greatly reduce the diff wrt upstream PostgreSQL * Oid preassign calls are now loaded into the backup archive as TOC entries which allows for them to be sorted to be output before object creation * Avoid usage of PQExpBuffers for trivial string construction where a fixed buffer on the stack is sufficient, or simply outputting the string directly * Binary upgrade dumping in pg_dump is moved to a separate function which loops over the dumpable objects instead of being mixed in with the main code * Various simplifications of the code and cleanups where possible
-
- 16 1月, 2017 3 次提交
-
-
由 Tom Lane 提交于
Cherry-picked into GPDB ahead of merge to fix compiler warning from backporting of bytea HEX encode/decode.
-
由 Daniel Gustafsson 提交于
Coverity complained that the aoseg_namespace variable potentially was uninitialized at usage, and while technically correct it would require the pg_aoseg namespace to be missing in the source cluster which seems far fatched. Fix by forcing initialization to invalid and ensuring that the Oid has been changed to a valid Oid before using. While looking at the code I also realized it had been blindly cargoculted to the same design pattern as the remaining queries which here was pointless. Simplify the logic and avoid allocating an extra PQExpBuffer.
-
由 Daniel Gustafsson 提交于
While the type cache used in binary upgrades isn't a dumpable object, it has a DO_ identifier associated to be able to use the indexing code. Skip it when looking through the DO objects to keep compilers and static analyzers happy. This was already done in pg_dump, do so for cdb_dump_agent too and expand the documentation on why.
-
- 13 1月, 2017 4 次提交
-
-
由 Jesse Zhang 提交于
This doesn't save too much time, it is just the pedant in me cringing when I looked at why we had a mandatory proprietary dependency. Alas this probably should have been done when we first introduced the `--enable-netbackup` configure flag, but better late than never.
-
由 Jesse Zhang 提交于
The four BSA agents seem to (inherently, as opposed to incidentally) share the same dependencies. If that's true, we might as well declare them together in one place.
-
由 Jesse Zhang 提交于
Those binaries need the gpbsa shared object to build. This wasn't exposed until EITHER 0. We changed the order we build them in the previous commit; OR 0. We ran a parallel build This commit simply adds the proper dependency declaration to all the offending targets.
-
由 Jesse Zhang 提交于
This should not break anything given the declarative nature of Make. But it does (especially noticeable if you are not running a parallel `make`). Because we are not declaring a dependency on gpbsa libraries on the the dump agents executables.
-
- 12 1月, 2017 1 次提交
-
-
由 Dave Cramer 提交于
* Implement bytea HEX encode/decode this is from upstream commit a2a8c7a6 The primary reason for back patching is to allow pgadmin4 to work Unfortunately there are a few hacks to make this work without Enum Config GUCs mostly around setting the GUC as the target is an integer but the guc value is a string * fixed regression tests \ removed unused code
-
- 11 1月, 2017 2 次提交
-
-
由 Heikki Linnakangas 提交于
Constraints are not uniquely identified by namespace and name alone. It's possible to have multiple constraints with the same name, but attached to different tables or domains. So include parent relation and domain OIDs in the key. This was uncovered by a regression test in the TINC test suite. Move the OID consistency tests from TINC into the normal regression test suite, to have test coverage for this.
-
由 Haozhou Wang 提交于
1. Add custom external table dependency with protocol to keep the dumped SQL ordered correctly. (create protocol must execute before creating custom external table) 2. Remove \" from dumped location URI 3. Support dump multiple locations Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io> Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>
-
- 10 1月, 2017 1 次提交
-
-
由 Daniel Gustafsson 提交于
Creating an external table with duplicate location URIs is a syntax error so move the duplicate check to the parsing step. This simplifies the external table command code as well as provides an error message with context marker to the user. Also fix up test sources to match the new error messages and remove the duplicate test from gpfdist since we already test that in ICW.
-
- 05 1月, 2017 2 次提交
-
-
由 Haozhou Wang 提交于
In case user wanna specify parameters with OPTIONS other than config file in LOCATION, it provides flexible and friendly grammar. Usage: CREATE EXTERNAL TABLE xxx LOCATION xxx FORMAT xxx [OPTIONS ( [option_key 'option_value' [, ...]] )] xxx; Signed-off-by: NKuien Liu <kliu@pivotal.io> Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io> Signed-off-by: NAdam Lee <ali@pivotal.io> Signed-off-by: NYuan Zhao <yuzhao@pivotal.io> Signed-off-by: NJesse Zhang <jzhang@pivotal.io>
-
由 Chris Hajas 提交于
* Backs up search path, optimizer settings, and gp_default_storage_options * These settings will only be restored during database restores (`-e` option)
-
- 03 1月, 2017 1 次提交
-
-
由 Daniel Gustafsson 提交于
Commit 08d85beb introduced a doublefree on the PQExpBuffer in pg_dump when populating the typecache for binary-upgrade mode. Fix by moving all buffer management into the scope where it's used.
-
- 31 12月, 2016 1 次提交
-
-
由 Daniel Gustafsson 提交于
Ensure that we close all PQExpBuffers before closing scope. Spotted by Coverity
-
- 27 12月, 2016 1 次提交
-
-
由 Daniel Gustafsson 提交于
Fixing incorret filename identifiers in the header comment blocks around the code, mostly stemming from copy-paste it would seem.
-
- 23 12月, 2016 1 次提交
-
-
由 Larry Hamel 提交于
-- previously, within a 60-second loop to test if a pqPing was successful, there was a 2-second fail-fast test to make sure that the postmaster.pid file was written to disk. -- However, in extreme loading conditions, the underlying OS may be too busy to accomplish that within 2 seconds. Removing the fail-fast means that in that extreme loading case, pg_ctl will still be able to start, assuming that the OS can write a file within 60 seconds. -- This does mean that in cases where the pidfile cannot be written at all (for instance, if the disk is full) the failure will take the full 60 seconds to be reported.
-
- 20 12月, 2016 4 次提交
-
-
由 Daniel Gustafsson 提交于
This works by building an indexarray containing all the pg_type entries on the first lookup, once built the indexarray can be queried and further SQL queries avoided. The corresponding array type is cached together with the base type.
-
由 Heikki Linnakangas 提交于
Backport another pg_upgrade-related change from upstream. Distinguishing between binary-upgrade and "normal" utility mode turns out to be rather handy. CAUTION: This removes the GPDB-specific postmaster command-line options -b and -C, becuase they clashed with the newly introduced -b flag, used to choose binary-upgrade mode! In order to invoke the GPDB functionality previously reached via -b -C, the long option forms need to be used. This commit switches pg_ctl over to using long options. This commit adds a substantial amount of Greenplum specific code to handle binary-upgrade in the database backend. Below is a list of the changes: - When restoring a schema dump on a master node, in binary-upgrade mode, we want to populate gp_distribution_policy, while in "normal" utility mode, we don't. In the passing, simplify the query used to fetch the entries from gp_distribution_policy in pg_dump. - pg_dump will in binary upgrade mode preassign Oids for the Oid dispatcher to ensure that object Oids are stable across dump/restore. This requires the Oid dispatcher to handle binary upgrade mode. Dump files created with binary-upgrade mode will assign multiple Oids into the preassigned Oids list, sometimes for objects not created within the current transaction. Allow the preassigned_oids list to preserve the Oids across end-of-xact when in binary upgrade mode. - The binary upgrade functions from pg_upgrade_support are used to preassign Oids during pg_dump in binary mode and are installed into a separate binary_upgrade schema. Due to a chicken-and-egg problem, installing these functions by themselves will however fail Oid assignment as they are required for Oid assignment. This reserves a block of OIDs for use with the binary_upgrade schema to ensure that the functions can be installed. The original commit from upstream which has been heavily amended to: commit 76dd09bb Author: Bruce Momjian <bruce@momjian.us> Date: Mon Apr 25 12:00:21 2011 -0400 Add postmaster/postgres undocumented -b option for binary upgrades. This option turns off autovacuum, prevents non-super-user connections, and enables oid setting hooks in the backend. The code continues to use the old autoavacuum disable settings for servers with earlier catalog versions. This includes a catalog version bump to identify servers that support the -b option. Heikki Linnakangas, Daniel Gustafsson and Dave Cramer
-
由 Heikki Linnakangas 提交于
The binary-upgrade mode in pg_dump which was backported in a previous commit is here extended to handle Greenplum clusters. The gist of the changes are to extend the Oid dumping to cover all object types as upstream PostgreSQL only covers types. The changes include: Make "pg_dump --binary-upgrade" connect in utility mode, it's supposed to be run against a single segment server. Also make pg_dump work against 4.3 clusters as the binary-upgrade mode will be used with pg_upgrade on the old cluster in the upgrade. Dump Oids for all object types in binary upgrade mode. In PostgreSQL only type Oids are dumped in binary upgrade mode, for Greenplum however we need to dump the Oids for all objects that require Oid dispatching (see oid_dispatch.c for details). Inject calls in the dump file for the Oid dispatcher to preassign Oids during restore. For partitioned tables, the Oids for each child table as well as their constraints are added to the dump file. The distributioncheck for partitions is skipped when restoring during binary upgrade. The distribution info is not available in the segments so during binary upgrade we cant enforce this check. Avoid creating array type for AO relation types, it's not useful for anything and only grows the catalog for no use. Heikki Linnakangas, Daniel Gustafsson and Dave Cramer
-
由 Heikki Linnakangas 提交于
It is needed by pg_upgrade. Note that pg_upgrade runs the *new* version's pg_dump binary, so there is no need to backport this to GPDB 4.3. Original commit: Author: Bruce Momjian <bruce@momjian.us> Date: Tue Feb 17 15:41:50 2009 +0000 Add pg_dump --binary-upgrade flag to be used by binary upgrade utilities. The new code allows transfer of dropped column information to the upgraded server.
-
- 19 12月, 2016 1 次提交
-
-
由 Daniel Gustafsson 提交于
The different kinds of NOTICE messages regarding table distribution were using a mix of upper and lower case for 'DISTRIBUTED BY'. Make them consistent by using upper case for all messages and update the test files, and atmsort regexes, to match.
-
- 17 12月, 2016 4 次提交
-
-
由 Daniel Gustafsson 提交于
Ensure to free any allocated content on premature exit. These potential leaks are small but we might as well clean up to make static analysis happy. Per defect in Coverity
-
由 Daniel Gustafsson 提交于
transformPassThroughParms() is hardcoded to always return true since the code that could render false has been killed. No single callsite further inspect the returnvalue (for good reasons). Remove return value altogether and make function return void.
-
由 Daniel Gustafsson 提交于
At this point we already know that pQry is non-NULL as we have dereferenced it previous in the function. Remove useless check to make the code clearer. Per gripe by Coverity.
-
由 Daniel Gustafsson 提交于
Remove commented out code which clearly hasn't been in use for some time. Anyone interested in this code have the VCS archives at their disposal.
-
- 08 12月, 2016 2 次提交
-
-
由 Daniel Gustafsson 提交于
This patch started out as a small refactor of executable discovery due to the function having multiple leaks, but turned into a revamping of the way the commandlines are constructed (since that's where the executable paths were used). The changes introduced in this diff are: - Build the commandlines in PQExpBuffers instead of fixed size buffer believed to be large enough - Move commandline functions from cdb_dump_util into cdb_restore_agent as that's the only callsite, no need to link that code into other binaries. Also refactor the functions significantly as there were many functions doing essentially the same thing. - Remove code to find executables and just use find_other_exec() since all binaries are in the same folder - Change shellEscape() to not always resetting the PQExpBuffer, also use the same code as in src/backend/cdb/cdbbackup.c which handles quoting as well - Remove various small functions that only returned a static string and variables holding a static string - Use MAXPGPATH instead of locally defined macro for the same thing - Remove the highly uninteresting unittests for string concatenation
-
由 Daniel Gustafsson 提交于
The code is commented out without any commented out callsites so remove. This is what the vcs history is for
-