- 25 10月, 2016 30 次提交
-
-
由 Foyzur Rahman 提交于
Previously accounting tree and reference to accounts were pointer based. This prevents us from freeing accounts at the end of each statement as later statements of that transaction can potentially access memory and the corresponding accounts. This resulted in a perceived memory leak as the accounts were not freed per-statement. In this implementation we convert pointers to index that can refer to an array of accounts. This allows us to partition indexes based on "generation" to define which accounts are live and which are already dead. This is also a soft-pointer so we don't crash accessing it. This also allows us to optimize the previous heavyweight "rollover" at the time of generation overflow as we no longer need to detect the validity of accounts based on generation, and rather we can just refer to logical range of indexes to identify the liveness. [#120028219]
-
由 Omer Arap 提交于
-
由 Adam Lee 提交于
-
由 Adam Lee 提交于
Two sub query cases and a stress case which could trigger segment fault with former version. Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
-
由 Peifeng Qiu 提交于
1. Add PGAllocator class to delegate palloc to std::vector. 2. Replace all significant buffer with S3Vector, which uses PGAllocator to allocate memory. 3. Fix header problems, retire s3common.h and s3common.cpp 4. Fix unit tests and regression tests. Signed-off-by: NKuien Liu <kliu@pivotal.io> Signed-off-by: NAdam Lee <ali@pivotal.io> Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Peifeng Qiu 提交于
We assume that only the main thread should handle signals, so we need to make child threads be quiet with a set of signals. Signed-off-by: NKuien Liu <kliu@pivotal.io>
-
由 Kuien Liu 提交于
The implementation of multi-thread in S3 writers is different from that of S3 reader because it creates threads dynamically (with benefits when data size to upload is small). But when exception is catched in thread, it needs to notify the main thread again, otherwise it may hang for waiting to create a new thread. Signed-off-by: NKuien Liu <kliu@pivotal.io>
-
由 Adam Lee 提交于
Some production systems, especially docker images, have no system-wide third-party header files. This commit could also reads header files from GPDB directory to compile gpcheckcloud. Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Peifeng Qiu 提交于
If user specifies bad config value, for example, chunksize > INT_MAX, or threadnum < 0 etc, we add range check to prevent undefined value. Signed-off-by: NKuien Liu <kliu@pivotal.io>
-
由 Kuien Liu 提交于
When config file does not exist or is failed to parse, we write log to pg_log by default. In previous, related logs are lost. Besides, we initialize segid and segnum before parsing config file to print correct message to users. Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Kuien Liu 提交于
strcmpci() is a function, served in built-in lib/ini, which compares strings while ignoring case. With it we can set the log_level as 'DEBUG', 'debug', or 'Debug', or log_type as 'Internal' or 'internal'. It affects only the log related configures in gps3ext extension. Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Adam Lee 提交于
To ignore line numbers and so on in expected and results outputs while pg_regress detecting failures.
-
由 Kuien Liu 提交于
When SELECT FROM an external table with constraint 'LIMIT 0', the transaction will abort. We add this regression test to verify gps3ext can handle this case. Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
-
由 Peifeng Qiu 提交于
In some cases QEs may be reset, then S3 extension stays on undefined behaviors. It may or may not catch the transaction abort signal. For example, when GPDB cannot parse imported data from S3, query and transaction are aborted and reset as new QE context starts. To solve it, we keep the signal of transaction cancel once it is captured until we stop all running threads and exit gracefully. Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Peifeng Qiu 提交于
1. add s3ext specific exception types and handlers 2. enchance the error messages 3. refactor CHECK_OR_DIE marcos 4. refactor S3RestfulService and S3Interface classes 5. refactor S3BucketReader's function getNextKey() 6. retire two kinds of "ResponseStatus": RESPONSE_FAIL, RESPONSE_ABORT 7. update corresponding unit tests Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Peifeng Qiu 提交于
1. Handle multiple config sections. 2. Able to check if the specified section exists. 3. Update regression test cases to verify different config sections in one query. Signed-off-by: NAdam Lee <ali@pivotal.io>
-
由 Adam Lee 提交于
Also update some methods and variables' names. Signed-off-by: NKuien Liu <kliu@pivotal.io>
-
由 Adam Lee 提交于
Avoid cycle including, keep codes simple and clear. Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Adam Lee 提交于
Support different configurations, except logging related ones, for different s3 external tables. Also retired most of the global variables. Signed-off-by: Haozhou Wang, Kuien Liu
-
由 Kuien Liu 提交于
Refactor compress_writer and s3key_writer to support row larger than chunk buffer, fix a corner case when the size of compression output is bigger than input. Add two regression tests to generate rows with mixed length, some of rows are larger than the size of chunkbuffer. Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
-
由 Kuien Liu 提交于
Support to interrupt gpcheckcloud by signals, including SIGTERM, SIGINT SIGABRT, SIGTSTP. For example, user can press CTRL-C to stop the program. And, we fix a deadlock in multi-thread upload during canceling. Besides, we further improve the message of gpreader for ABORT event. Signed-off-by: Haozhou Wang, Peifeng Qiu Pivotal tracker ID: [#129425589]
-
由 Peifeng Qiu 提交于
Signed-off-by: NAdam Lee <ali@pivotal.io>
-
由 Peifeng Qiu 提交于
GCC 4.4.2 doesn't fully implement move semantics, use swap() instead of std::move() to move vectors as a workaround. Signed-off-by: NAdam Lee <ali@pivotal.io>
-
由 Adam Lee 提交于
Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
-
由 Kuien Liu 提交于
To detect the query interruption event (e.g., triggered by user), we should consider both QueryCancelPending variable and transaction status. Here QueryCancelPending is not sufficient because it will be reset before the extprotocol last call, then we cannot distinguish normal exit/finish from abnormal transaction abort. Signed-off-by: Peifeng Qiu, Haozhou Wang
-
由 Peifeng Qiu 提交于
1. modify s3key_writer, support concurrent multipart uploading. 2. update upload cancel logic. 3. add unit tests. Signed-off-by: NKuien Liu <kliu@pivotal.io>
-
由 Kuien Liu 提交于
1. remove several tests which are duplicate to s3 regression. 2. add timer to record elapsed time of each test. 3. fix an issue that upload failure will not be captured. Signed-off-by: Haozhou Wang, Peifeng Qiu, Adam Lee
-
由 Kuien Liu 提交于
-
由 Adam Lee 提交于
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
-
由 Andreas Scherbaum 提交于
Update copyright statement Adresses #1056
-
- 24 10月, 2016 1 次提交
-
-
由 Adam Lee 提交于
Four characters "'", "\", "<", ">" could cause shell injection in gphdfs. This quick fix filters out them to avoid the security issue. Signed-off-by: NHaozhou Wang <hawang@pivotal.io> Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>
-
- 22 10月, 2016 6 次提交
-
-
由 Daniel Gustafsson 提交于
gpcheck was deprecated in 4.3.6.0 but the configuration file was never removed. Clean out together with install rule. Reported by zhangyuvip
-
由 Ashwin Agrawal 提交于
CI for some reason is getting hung, waiting for someone to hit enter in probe_config. So, disabling pagination for psql for the same.
-
由 foyzur 提交于
If there is an unknown configuration parameter in postgresql.conf, currently we silently print a log message and skip reading any configuration parameter from postgresql.conf. This does not prevent database startup. As some of our gucs depend on explicitly mentioned values on certain platforms (e.g., gp_vmem_protect_limit is set to 8GB explicitly in postgresql.conf, and without explicit mention, it defaults to 0 on darwin platforms), this can result in assertion failure and confusing dtm errors as QEs fail to start with 0 vmem protect limit (and potentially other invalid guc values). This change prevent database startup if there is any unknown configuration parameter. This is relevant to Github issue #1201. However, for issue #1201 we will need another fix to correct bugbuster test so that it doesn't put unrecognized parameter in the postgresql.conf. * Revert "Preventing gpstart if there is an unknown configuration parameter in postgresql.conf." This reverts commit 6fa636cd924aea83e22964e0d88d0e9736369cc2. * Error out GPDB start if guc context is PGC_POSTMASTER.
-
由 Nikos Armenatzoglou 提交于
-
由 Venkatesh Raghavan 提交于
-
由 Venkatesh Raghavan 提交于
-
- 21 10月, 2016 3 次提交
-
-
由 Kenan Yao 提交于
-
由 Kenan Yao 提交于
designed to verify no QE left alive when QD has exited, but QE exit in this situation is an asynchronous procedure, so we have to wait some time before we do verification.
-
由 Shoaib Lari 提交于
In certain rare cases, crash recovery may prevent start up due to stale relfilenodes in relcache init file. E.g. database crashes right after reindex operation on critical system tables in "template1" database is committed. This can happen only in case of "template1" database because MyDatabaseId is set to "template1" during crash recovery. Relcache needs to be initialized in pass 2 of recovery so that heap access methods can be used to update persistent tables. Relcache init file written in pass 2 contains critical relations and indexes from "template1" database. Pass 3 applies redo records from xlog to all relations other than persistent tables. Relfilenodes in the relcache init file may become out of date during pass 3. Note that pass 3 does not need relcache and therefore does not bother keeping it up to date. Pass 4 should initialize relcache from scratch rather than using a stale init file created by pass 2. Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>
-