Skip to content

G
Gpdb

Greenplum / Gpdb

通知 7
Star 1
Fork 0
G
Gpdb

体验新版 GitCode，发现更多精彩内容 >>

提交 f8734d9d 编写于 作者: H Heikki Linnakangas
浏览文件
操作

Move tests on 'demoprot' from TINC to contrib, where the source is. 

I had to change some of the tests that used 4 tests rows, to have more (9),
to make them pass on a 3 segment cluster. If you only insert 4 rows to an
external table, some segments might not receive any rows, and reading from
the same file later will fail because the file wasn't created. I also
changed the queries that tested that rows are distributed evenly, to work
with a different number of segments.

Hook up the new tests to installcheck-world.
上级 6133a493
展开全部 隐藏空白更改
内联 并排
Showing with 398 addition and 861 deletion
GNUmakefile.in
浏览文件 @ f8734d9d
......@@ -121,6 +121,7 @@ installcheck-world:
$(MAKE) -C src/pl installcheck
#$(MAKE) -C src/interfaces/ecpg installcheck
#$(MAKE) -C contrib installcheck
$(MAKE) -C contrib/extprotocol installcheck
$(MAKE) -C gpAux/extensions installcheck
$(MAKE) -C src/bin/gpfdist installcheck
......
contrib/extprotocol/Makefile
浏览文件 @ f8734d9d
MODULE_big = gpextprotocol
OBJS = gpextprotocol.o
REGRESS = setup exttableext
PG_CPPFLAGS = -I$(libpq_srcdir)
PG_LIBS = $(libpq_pgport)
......
contrib/extprotocol/expected/setup.out 0 → 100644
浏览文件 @ f8734d9d
CREATE SCHEMA exttableext;
GRANT ALL ON SCHEMA exttableext TO PUBLIC;
SET search_path TO 'exttableext';
-- Create an example table exttabtest that will be used as source table
CREATE TABLE exttabtest(
id int,
name varchar(20),
value1 int,
value2 int
)
DISTRIBUTED BY (id);
-- Loading 100 records
-- Use only 100 rows for easy to verify the results
-- In order to test multiple data buffers and related edge cases, more data (at least several MBs or more)
-- will be used, as in Performance tests 1M and 100M test cases
\echo 'loading data...'
loading data...
INSERT INTO exttabtest SELECT i, 'name'||i, i*2, i*3 FROM generate_series(1,100) i;
-- Test 1: create read and write functions based on example gpextprotocol.so
-- Note: Only STABLE is supported for protocol, though this has not been enforced at the time of testing
CREATE OR REPLACE FUNCTION write_to_file_stable() RETURNS integer AS
'$libdir/gpextprotocol.so', 'demoprot_export' LANGUAGE C STABLE;
CREATE OR REPLACE FUNCTION read_from_file_stable() RETURNS integer AS
'$libdir/gpextprotocol.so', 'demoprot_import' LANGUAGE C STABLE;
-- Check pg_proc catalog table for new created functions
SELECT proname, prolang,proisstrict,provolatile,pronargs,prorettype,prosrc,proacl FROM pg_proc
WHERE proname like 'write_to_file%'
or proname like 'read_from_file%'
ORDER BY proname;
proname | prolang | proisstrict | provolatile | pronargs | prorettype | prosrc | proacl
-----------------------+---------+-------------+-------------+----------+------------+-----------------+--------
read_from_file_stable | 13 | f | s | 0 | 23 | demoprot_import |
write_to_file_stable | 13 | f | s | 0 | 23 | demoprot_export |
(2 rows)
-- Test 2: create bi-directional protocol (read and write) using STABLE functions
DROP PROTOCOL IF EXISTS demoprot;
NOTICE: protocol "demoprot" does not exist, skipping
CREATE PROTOCOL demoprot (
readfunc = read_from_file_stable,
writefunc = write_to_file_stable
);
-- Check dependency: pg_depend table
select count(*) from pg_depend
where objid in (
select oid from pg_extprotocol where ptcname='demoprot');
count
-------
2
(1 row)
-- Check pg_extprotocol for new created protocol
select extprot.ptcname, proc1.proname readfunc, proc2.proname writefunc
from pg_extprotocol extprot, pg_proc proc1, pg_proc proc2
where extprot.ptcname='demoprot'
and extprot.ptcreadfn=proc1.oid
and extprot.ptcwritefn=proc2.oid;
ptcname | readfunc | writefunc
----------+-----------------------+----------------------
demoprot | read_from_file_stable | write_to_file_stable
(1 row)
DROP EXTERNAL TABLE IF EXISTS clean_exttabtest_files;
NOTICE: table "clean_exttabtest_files" does not exist, skipping
CREATE EXTERNAL WEB TABLE clean_exttabtest_files(stdout text) EXECUTE 'rm -f exttabtest*' ON ALL FORMAT 'text';
GRANT ALL ON clean_exttabtest_files TO PUBLIC;
SELECT * FROM clean_exttabtest_files;
stdout
--------
(0 rows)
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/sql/exttableext.sql → contrib/extprotocol/sql/exttableext.sql
浏览文件 @ f8734d9d
......@@ -27,9 +27,11 @@ SET search_path TO 'exttableext';
EXCEPT ALL
SELECT * FROM exttabtest;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from exttabtest_r
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from exttabtest_r group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Test 4.1: create uni-directional write protocol
-- create WET using created protocol
......@@ -101,9 +103,11 @@ SET search_path TO 'exttableext';
EXCEPT ALL
SELECT * FROM exttabtest;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from exttabtest_r_dist
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from exttabtest_r_dist group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Test 6: using two urls and using CSV format
......@@ -128,9 +132,11 @@ SET search_path TO 'exttableext';
EXCEPT ALL
SELECT * FROM exttabtest;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from exttabtest_r_2url
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from exttabtest_r_2url group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Check the output file at each segments
-- ! gpssh -f allsegs ls -l /data/hhuang/MAIN/main_debug/primary/gpseg*/exttabtest_2url*.csv
......@@ -157,9 +163,11 @@ SET search_path TO 'exttableext';
EXCEPT ALL
SELECT * FROM exttabtest;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from exttabtest_r_2url
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from exttabtest_r_2url group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Checking the output files on segments
-- ! gpssh -f allsegs ls -l /data/hhuang/MAIN/main_debug/primary/gpseg*/exttabtest_2url*.txt
......@@ -295,32 +303,34 @@ SET search_path TO 'exttableext';
LOCATION('badprotocol://exttabtest.txt')
FORMAT 'text';
-- Test 20: Small dataset - 4 records
-- Test 20: Small dataset - 20 records
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_4records;
CREATE WRITABLE EXTERNAL TABLE exttabtest_w_4records (like exttabtest)
LOCATION('demoprot://exttabtest_4records.txt')
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_20records;
CREATE WRITABLE EXTERNAL TABLE exttabtest_w_20records (like exttabtest)
LOCATION('demoprot://exttabtest_20records.txt')
FORMAT 'text'
DISTRIBUTED BY (id);
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_4records;
CREATE READABLE EXTERNAL TABLE exttabtest_r_4records (like exttabtest)
LOCATION('demoprot://exttabtest_4records.txt')
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_20records;
CREATE READABLE EXTERNAL TABLE exttabtest_r_20records (like exttabtest)
LOCATION('demoprot://exttabtest_20records.txt')
FORMAT 'text';
-- write to WET
INSERT INTO exttabtest_w_4records (SELECT * FROM exttabtest where id<=4);
INSERT INTO exttabtest_w_20records (SELECT * FROM exttabtest where id<=20);
-- read from RET
SELECT * FROM exttabtest_r_4records order by id;
SELECT * FROM exttabtest_r_20records order by id;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from exttabtest_r_4records
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from exttabtest_r_20records group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Drop External Tables
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_4records;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_4records;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_20records;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_20records;
-- Test 21: Small dataset - 1 record
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_1record;
......@@ -597,7 +607,10 @@ DISTRIBUTED BY (id);
INSERT INTO formatsource SELECT 'name'||i, i, i*2, i*3 FROM generate_series(1,100) i;
-- Check data distribution
SELECT gp_segment_id, count(*) FROM formatsource GROUP BY 1 ORDER BY 1;
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from formatsource group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Test 61: create STABLE read and write functions based on example gpformatter.so
-- Note: Only STABLE is supported for formatter.
......@@ -706,9 +719,12 @@ DROP FUNCTION formatter_import_todrop();
-- Read from RET
SELECT count(*) FROM format_r;
-- verify data should be evenly distributed
SELECT gp_segment_id, count(*) from format_r
GROUP BY 1 ORDER BY 1;
-- verify data should be evenly distributed
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from format_r group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Test 64: Drop format function with external table using the function
DROP FUNCTION formatter_export_i(record);
......@@ -855,16 +871,21 @@ DROP FUNCTION formatter_import_todrop();
insert into format_long select 2.0,i from repeat('oxo2', 1000) i;
insert into format_long select 3.0,i from repeat('oxo3', 1000) i;
insert into format_long select 4.0,i from repeat('oxo4', 1000) i;
insert into format_long select 5.0,i from repeat('oxo5', 1000) i;
insert into format_long select 6.0,i from repeat('oxo6', 1000) i;
insert into format_long select 7.0,i from repeat('oxo7', 1000) i;
insert into format_long select 8.0,i from repeat('oxo8', 1000) i;
insert into format_long select 9.0,i from repeat('oxo9', 1000) i;
-- Check distribution is even
select gp_segment_id,count(*) from format_long
group by gp_segment_id
order by gp_segment_id;
with t as (
SELECT gp_segment_id as segid, count(*) as cnt from format_long group by gp_segment_id
)
select max(cnt) - min(cnt) > 20 from t;
-- Write to WET
-- insert should be successful
INSERT INTO format_long_w (SELECT * FROM format_long);
-- INSERT 0 4
-- Read from RET
select count(*) from format_long_r;
......@@ -915,15 +936,19 @@ DROP FUNCTION formatter_import_todrop();
insert into format_long select 2.0,i from repeat('oxox2', 1000) i;
insert into format_long select 3.0,i from repeat('oxox3', 1000) i;
insert into format_long select 4.0,i from repeat('oxox4', 1000) i;
insert into format_long select 5.0,i from repeat('oxox5', 1000) i;
insert into format_long select 6.0,i from repeat('oxox6', 1000) i;
insert into format_long select 7.0,i from repeat('oxox7', 1000) i;
insert into format_long select 8.0,i from repeat('oxox8', 1000) i;
insert into format_long select 9.0,i from repeat('oxox9', 1000) i;
-- Write to WET
-- insert should be successful
INSERT INTO format_long_w (SELECT * FROM format_long);
-- INSERT 0 4
-- Read from RET
select count(*) from format_long_r;
-- returns count = 4
-- returns count = 9
-- Test 73: Interlacing short and long record, testing FMT_NEED_MORE_DATA
-- When loading data from RET, long record may trigger FORMATTER_RETURN_NOTIFICATION(fcinfo, FMT_NEED_MORE_DATA).
-- Verify the data can be loaded successfully and should exactly match the source records.
......@@ -960,15 +985,19 @@ DROP FUNCTION formatter_import_todrop();
insert into format_long select 2.0,i from repeat('oxo2', 1000) i;
insert into format_long select 3.0,'oxo3';
insert into format_long select 4.0,i from repeat('oxo4', 1000) i;
insert into format_long select 5.0,'oxo5';
insert into format_long select 6.0,i from repeat('oxo6', 1000) i;
insert into format_long select 7.0,'oxo7';
insert into format_long select 8.0,i from repeat('oxo8', 1000) i;
insert into format_long select 9.0,'oxo9';
-- Write to WET
-- insert should be successful
INSERT INTO format_long_w (SELECT * FROM format_long);
-- INSERT 0 4
-- Read from RET
select count(*) from format_long_r;
-- returns count = 4
-- returns count = 9
-- read from RET, both should return 0
SELECT * FROM format_long_r
......@@ -1091,11 +1120,15 @@ DROP FUNCTION formatter_import_todrop();
insert into formatsource select 2,'oxo2';
insert into formatsource select 3,'oxo3';
insert into formatsource select 4,'oxo4';
insert into formatsource select 5,'oxo5';
insert into formatsource select 6,'oxo6';
insert into formatsource select 7,'oxo7';
insert into formatsource select 8,'oxo8';
insert into formatsource select 9,'oxo9';
-- Write to WET
-- insert should be successful
INSERT INTO format_w (SELECT * FROM formatsource);
-- INSERT 0 4
-- Read from RET
select id, value1 from format_r order by id;
......@@ -1130,11 +1163,15 @@ DROP FUNCTION formatter_import_todrop();
insert into formatsource values (2,null,null);
insert into formatsource values (3,null,null);
insert into formatsource values (4,null,null);
insert into formatsource values (5,null,null);
insert into formatsource values (6,null,null);
insert into formatsource values (7,null,null);
insert into formatsource values (8,null,null);
insert into formatsource values (9,null,null);
-- Write to WET
-- insert should be successful
INSERT INTO format_w (SELECT * FROM formatsource);
-- INSERT 0 4
-- Read from RET
select * from format_r where name is null order by id;
......
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/sql/setup/query00.sql → contrib/extprotocol/sql/setup.sql
浏览文件 @ f8734d9d
DROP SCHEMA IF EXISTS exttableext CASCADE;
CREATE SCHEMA exttableext;
GRANT ALL ON SCHEMA exttableext TO PUBLIC;
SET search_path TO 'exttableext';
-- Create an example table exttabtest that will be used as source table
-- Clean up existing external tables, functions
DROP TABLE IF EXISTS exttabtest;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_1m;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_1m_null;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_2url;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_circle;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_dist;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_invalid;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_new;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_null;
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_uni;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_1m;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_1m_null;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_2url;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_5url;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_circle;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_dist;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_invalid;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_misspath;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_new;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_null;
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_uni;
DROP EXTERNAL TABLE IF EXISTS format_long_r;
DROP EXTERNAL TABLE IF EXISTS format_long_w;
DROP EXTERNAL TABLE IF EXISTS format_r;
DROP EXTERNAL TABLE IF EXISTS format_r_s1;
DROP EXTERNAL TABLE IF EXISTS format_r_s2;
DROP EXTERNAL TABLE IF EXISTS format_w;
DROP EXTERNAL TABLE IF EXISTS format_w_s1;
DROP EXTERNAL TABLE IF EXISTS format_w_s2;
DROP FUNCTION IF EXISTS formatter_export_s(record) CASCADE;
DROP FUNCTION IF EXISTS formatter_export_v(record) CASCADE;
DROP FUNCTION IF EXISTS formatter_import_s() CASCADE;
DROP FUNCTION IF EXISTS formatter_import_v() CASCADE;
DROP FUNCTION IF EXISTS read_from_file() CASCADE;
DROP FUNCTION IF EXISTS read_from_file_stable() CASCADE;
DROP FUNCTION IF EXISTS write_to_file_stable() CASCADE;
DROP FUNCTION IF EXISTS url_validator() CASCADE;
DROP FUNCTION IF EXISTS write_to_file() CASCADE;
DROP FUNCTION IF EXISTS write_to_file_stable() CASCADE;
CREATE TABLE exttabtest(
id int,
name varchar(20),
......
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/expected/setup/query00.ans 已删除 100644 → 0
浏览文件 @ 6133a493
DROP SCHEMA IF EXISTS exttableext;
DROP SCHEMA
CREATE SCHEMA exttableext;
CREATE SCHEMA
GRANT ALL ON SCHEMA exttableext TO PUBLIC;
GRANT
SET search_path TO 'exttablext';
SET
-- Create an example table exttabtest that will be used as source table
-- Clean up existing external tables, functions
DROP TABLE IF EXISTS exttabtest;
psql:/path/sql_file:1: NOTICE: table "exttabtest" does not exist, skipping
DROP TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_1m;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_1m" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_1m_null;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_1m_null" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_2url;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_2url" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_circle;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_circle" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_dist;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_dist" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_invalid;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_invalid" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_new;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_new" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_null;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_null" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_r_uni;
psql:/path/sql_file:1: NOTICE: table "exttabtest_r_uni" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_1m;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_1m" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_1m_null;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_1m_null" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_2url;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_2url" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_5url;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_5url" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_circle;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_circle" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_dist;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_dist" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_invalid;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_invalid" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_misspath;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_misspath" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_new;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_new" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_null;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_null" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS exttabtest_w_uni;
psql:/path/sql_file:1: NOTICE: table "exttabtest_w_uni" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_long_r;
psql:/path/sql_file:1: NOTICE: table "format_long_r" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_long_w;
psql:/path/sql_file:1: NOTICE: table "format_long_w" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_r;
psql:/path/sql_file:1: NOTICE: table "format_r" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_r_s1;
psql:/path/sql_file:1: NOTICE: table "format_r_s1" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_r_s2;
psql:/path/sql_file:1: NOTICE: table "format_r_s2" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_w;
psql:/path/sql_file:1: NOTICE: table "format_w" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_w_s1;
psql:/path/sql_file:1: NOTICE: table "format_w_s1" does not exist, skipping
DROP EXTERNAL TABLE
DROP EXTERNAL TABLE IF EXISTS format_w_s2;
psql:/path/sql_file:1: NOTICE: table "format_w_s2" does not exist, skipping
DROP EXTERNAL TABLE
DROP FUNCTION IF EXISTS formatter_export_s(record) CASCADE;
psql:/path/sql_file:1: NOTICE: function formatter_export_s(record) does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS formatter_export_v(record) CASCADE;
psql:/path/sql_file:1: NOTICE: function formatter_export_v(record) does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS formatter_import_s() CASCADE;
psql:/path/sql_file:1: NOTICE: function formatter_import_s() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS formatter_import_v() CASCADE;
psql:/path/sql_file:1: NOTICE: function formatter_import_v() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS read_from_file() CASCADE;
psql:/path/sql_file:1: NOTICE: function read_from_file() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS read_from_file_stable() CASCADE;
psql:/path/sql_file:1: NOTICE: function read_from_file_stable() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS write_to_file_stable() CASCADE;
psql:/path/sql_file:1: NOTICE: function write_to_file_stable() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS url_validator() CASCADE;
psql:/path/sql_file:1: NOTICE: function url_validator() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS write_to_file() CASCADE;
psql:/path/sql_file:1: NOTICE: function write_to_file() does not exist, skipping
DROP FUNCTION
DROP FUNCTION IF EXISTS write_to_file_stable() CASCADE;
psql:/path/sql_file:1: NOTICE: function write_to_file_stable() does not exist, skipping
DROP FUNCTION
CREATE TABLE exttabtest(
id int,
name varchar(20),
value1 int,
value2 int
)
DISTRIBUTED BY (id);
CREATE TABLE
-- Loading 100 records
-- Use only 100 rows for easy to verify the results
-- In order to test multiple data buffers and related edge cases, more data (at least several MBs or more)
-- will be used, as in Performance tests 1M and 100M test cases
\echo 'loading data...'
loading data...
INSERT INTO exttabtest SELECT i, 'name'||i, i*2, i*3 FROM generate_series(1,100) i;
INSERT 0 100
DROP EXTERNAL TABLE IF EXISTS clean_exttabtest_files;
DROP TABLE
CREATE EXTERNAL WEB TABLE clean_exttabtest_files(stdout text) EXECUTE 'rm -f exttabtest*' ON ALL FORMAT 'text';
CREATE TABLE
GRANT ALL ON clean_exttabtest_files TO PUBLIC;
GRANT
SELECT * FROM clean_exttabtest_files;
stdout
--------
(0 rows)
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/init_file 已删除 100644 → 0
浏览文件 @ 6133a493
-- start_matchsubs
# Change things like "psql: /Users/mglkey/cdbfast/main/foo/bar.sql:123:" to "PATH"
m/psql:\/.*:\d+:/
s/psql:\/.*:\d+:/psql:PATH:/
-- end_matchsubs
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/sql/setup/removeOutputFile.sql 已删除 100644 → 0
浏览文件 @ 6133a493
-- WET using example protocol demoprot will export data files to data directory on primary segments
-- If NOT cleaned, these files will break data replication (between primary and mirror).
-- Therefore it is necessary to clean up generated data files after each test case run.
--
-- query to construct command to remove the generated data file from data directory
SELECT 'gpssh -h '|| gpsc.hostname ||' rm -f '|| fse.fselocation || '/exttabtest*'
FROM pg_filespace_entry fse, gp_segment_configuration gpsc
WHERE gpsc.dbid = fse.fsedbid and gpsc.role = 'p' and gpsc.content > -1
ORDER BY gpsc.hostname, fse.fselocation;
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/basic/exttab/exttableext/test_exttable.py 已删除 100755 → 0
浏览文件 @ 6133a493
"""
Copyright (C) 2004-2015 Pivotal Software, Inc. All rights reserved.
This program and the accompanying materials are made available under
the terms of the under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import os
from mpp.models import SQLTestCase
from tinctest.lib import local_path, run_shell_command
class ExternalTableExtensionTests(SQLTestCase):
"""
@tags list_mgmt_expand
@product_version gpdb: [4.3-]
"""
sql_dir = 'sql/'
ans_dir = 'expected/'
def tearDown(self):
# execute commands to remove output files created in primary segment's data directories
# This cleans up all the files exported by demoprot on the primary segments.
# expects removeOutputFile.sql in sql/setup to be run and its corredponfing out file available in
# out_dir/setup/removeOutputFile.out
out_file = os.path.join(self.get_out_dir(), 'setup', 'removeOutputFile.out')
with open(out_file) as f:
for line in f:
if (line.find('gpssh ')>=0 and line.find('SELECT')<0):
ok = run_shell_command(line)
if not ok:
raise Exception('Output file remove operation error: %s' %line)
super(ExternalTableExtensionTests, self).tearDown()
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或