Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Greenplum
Gpdb
提交
df579765
G
Gpdb
项目概览
Greenplum
/
Gpdb
通知
7
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
G
Gpdb
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
df579765
编写于
6月 27, 2017
作者:
L
Lisa Owen
提交者:
David Yozie
6月 27, 2017
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
docs - s3 protocol s3_endpoint must include hostname (#2707)
上级
4bc69dc2
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
28 addition
and
27 deletion
+28
-27
gpdb-doc/dita/admin_guide/load/topics/g-s3-protocol.xml
gpdb-doc/dita/admin_guide/load/topics/g-s3-protocol.xml
+28
-27
未找到文件。
gpdb-doc/dita/admin_guide/load/topics/g-s3-protocol.xml
浏览文件 @
df579765
...
...
@@ -112,10 +112,11 @@
configuration file location in the
<codeph>
LOCATION
</codeph>
clause of the
<codeph>
CREATE EXTERNAL TABLE
</codeph>
command. This is the syntax:
</p>
<codeblock>
's3://
<varname>
S3_endpoint
</varname>
[:
<varname>
port
</varname>
]/
<varname>
bucket_name
</varname>
/[
<varname>
S3_prefix
</varname>
] [region=
<varname>
S3_region
</varname>
] [config=
<varname>
config_file_location
</varname>
]'
</codeblock>
<p>
For the
<codeph>
s3
</codeph>
protocol you must
specify the S3 endpoint and S3 bucket
<p>
The
<codeph>
s3
</codeph>
protocol requires that you
specify the S3 endpoint and S3 bucket
name. Each Greenplum Database segment instance must have access to the S3 location. The
optional
<varname>
S3_prefix
</varname>
value is used to select files for read-only S3
tables, or as a filename prefix to use when uploading files for S3 writable tables.
</p>
<note>
The Greenplum Database
<codeph>
s3
</codeph>
protocol URL must include the S3 endpoint hostname.
</note>
<p>
To specify an ECS endpoint (an Amazon S3 compatible service) in the
<codeph>
LOCATION
</codeph>
clause, you must set the
<codeph>
s3
</codeph>
configuration
file parameter
<codeph>
version
</codeph>
to 2. The
<codeph>
version
</codeph>
parameter
...
...
@@ -137,32 +138,32 @@
result of inserting data to the table. See
<xref
href=
"#amazon-emr/section_c2f_zvs_3x"
format=
"dita"
/>
.
</p>
<p>
For read-only S3 tables, the S3 file prefix is optional. If you specify an
<varname>
S3_prefix
</varname>
, then
<codeph>
s3
</codeph>
protocol selects all files
<varname>
S3_prefix
</varname>
, then
the
<codeph>
s3
</codeph>
protocol selects all files
that start with the specified prefix as data files for the external table. The
<codeph>
s3
</codeph>
protocol does not use the slash character (
<codeph>
/
</codeph>
)
as
delimiter, so a slash character follows a prefix is treated as part of the prefix
itself.
</p>
<p>
For example, consider the following 5 files that each have
<codeph>
domain
</codeph>
as
the
<varname>
S3_endpoint
</varname>
, and
<codeph>
test1
</codeph>
as the
<varname>
bucket_name
</varname
>
:
</p>
<codeblock>
s3://
domain
/test1/abc
s3://
domain
/test1/abc/
s3://
domain
/test1/abc/xx
s3://
domain
/test1/abcdef
s3://
domain
/test1/abcdefff
</codeblock>
<codeph>
s3
</codeph>
protocol does not use the slash character (
<codeph>
/
</codeph>
)
as a delimiter, so a slash character following a prefix is treated as part of the
prefix
itself.
</p>
<p>
For example, consider the following 5 files that each have
the
<varname>
S3_endpoint
</varname>
named
<codeph>
s3-us-west-2.amazonaws.com
</codeph>
and the
<varname>
bucket_name
</varname>
<codeph>
test1
</codeph
>
:
</p>
<codeblock>
s3://
s3-us-west-2.amazonaws.com
/test1/abc
s3://
s3-us-west-2.amazonaws.com
/test1/abc/
s3://
s3-us-west-2.amazonaws.com
/test1/abc/xx
s3://
s3-us-west-2.amazonaws.com
/test1/abcdef
s3://
s3-us-west-2.amazonaws.com
/test1/abcdefff
</codeblock>
<ul
id=
"ul_yll_xjm_qv"
>
<li>
If the S3 URL is provided as
<codeph>
s3://
domain
/test1/abc
</codeph>
, then the
<li>
If the S3 URL is provided as
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abc
</codeph>
, then the
<codeph>
abc
</codeph>
prefix selects all 5 files.
</li>
<li>
If the S3 URL is provided as
<codeph>
s3://
domain
/test1/abc/
</codeph>
, then the
<li>
If the S3 URL is provided as
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abc/
</codeph>
, then the
<codeph>
abc/
</codeph>
prefix selects the files
<codeph>
s3://
domain
/test1/abc/
</codeph>
and
<codeph>
s3://
domain
/test1/abc/xx
</codeph>
.
</li>
<li>
If the S3 URL is provided as
<codeph>
s3://
domain
/test1/abcd
</codeph>
, then the
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abc/
</codeph>
and
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abc/xx
</codeph>
.
</li>
<li>
If the S3 URL is provided as
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abcd
</codeph>
, then the
<codeph>
abcd
</codeph>
prefix selects the files
<codeph>
s3://
domain
/test1/abcdef
</codeph>
and
<codeph>
s3://
domain
/test1/abcdefff
</codeph></li>
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abcdef
</codeph>
and
<codeph>
s3://
s3-us-west-2.amazonaws.com
/test1/abcdefff
</codeph></li>
</ul>
<p>
Wildcard characters are not supported in a
<varname>
S3_prefix
</varname>
; however, the S3
<p>
Wildcard characters are not supported in a
n
<varname>
S3_prefix
</varname>
; however, the S3
prefix functions as if a wildcard character immediately followed the prefix itself.
</p>
<p>
All of the files selected by the S3 URL
(
<varname>
S3_endpoint
</varname>
/
<varname>
bucket_name
</varname>
/
<varname>
S3_prefix
</varname>
)
...
...
@@ -286,7 +287,7 @@ server_side_encryption = sse-s3
absolute path to the location with the
<codeph>
config
</codeph>
parameter in the
<codeph>
s3
</codeph>
protocol
<codeph>
LOCATION
</codeph>
clause. This example specifies
a location in the
<codeph>
gpadmin
</codeph>
home directory.
</p>
<codeblock>
LOCATION ('s3://s3.amazonaws.com/test/my_data config=/home/gpadmin/s3.conf')
</codeblock>
<codeblock>
LOCATION ('s3://s3
-us-west-2
.amazonaws.com/test/my_data config=/home/gpadmin/s3.conf')
</codeblock>
<p>
All segment instances on the hosts use the file
<codeph>
/home/gpadmin/s3.conf
</codeph>
.
</p>
</section>
...
...
@@ -513,9 +514,9 @@ chunksize = 67108864</codeblock></p>
capture the output and create an
<codeph>
s3
</codeph>
configuration file to connect to
Amazon S3.
</p><p>
The utility is installed in the Greenplum Database
<codeph>
$GPHOME/bin
</codeph>
directory.
</p><b>
Syntax
</b>
<codeblock>
gpcheckcloud {
<b>
-c
</b>
|
<b>
-d
</b>
} "
<b>
s3://
</b><varname>
S3_endpoint
</varname>
/
<varname>
bucketname
</varname>
/[
<varname>
S3_prefix
</varname>
] [config=
=
<varname>
path_to_config_file
</varname>
]"
<codeblock>
gpcheckcloud {
<b>
-c
</b>
|
<b>
-d
</b>
} "
<b>
s3://
</b><varname>
S3_endpoint
</varname>
/
<varname>
bucketname
</varname>
/[
<varname>
S3_prefix
</varname>
] [config=
<varname>
path_to_config_file
</varname>
]"
gpcheckcloud
<b>
-u
</b>
<
file_to_upload> "
<b>
s3://
</b><varname>
S3_endpoint
</varname>
/
<varname>
bucketname
</varname>
/[
<varname>
S3_prefix
</varname>
] [config=
=
<varname>
path_to_config_file
</varname>
]"
gpcheckcloud
<b>
-u
</b>
<
file_to_upload> "
<b>
s3://
</b><varname>
S3_endpoint
</varname>
/
<varname>
bucketname
</varname>
/[
<varname>
S3_prefix
</varname>
] [config=
<varname>
path_to_config_file
</varname>
]"
gpcheckcloud
<b>
-t
</b>
gpcheckcloud
<b>
-h
</b></codeblock>
...
...
@@ -561,13 +562,13 @@ gpcheckcloud <b>-h</b></codeblock>
current directory.
<codeblock>
gpcheckcloud -t > ./mytest_s3.config
</codeblock></p><p>
This
example attempts to upload a local file,
<filepath>
test-data.csv
</filepath>
to an S3
bucket location using the
<codeph>
s3
</codeph>
configuration file
<codeph>
s3.mytestconf
</codeph>
:
<codeblock>
gpcheckcloud -u ./test-data.csv "s3://
domain
/test1/abc config=s3.mytestconf"
</codeblock></p><p>
A
<codeph>
s3.mytestconf
</codeph>
:
<codeblock>
gpcheckcloud -u ./test-data.csv "s3://
s3-us-west-2.amazonaws.com
/test1/abc config=s3.mytestconf"
</codeblock></p><p>
A
successful upload results in one or more files placed in the S3 bucket using the
filename format
<codeph>
abc
<
segment_id>
<
random>.data[.gz]
</codeph>
. See
<xref
href=
"#amazon-emr/section_c2f_zvs_3x"
format=
"dita"
/>
.
</p><p>
This example attempts to
connect to an S3 bucket location with the
<codeph>
s3
</codeph>
configuration file
<codeph>
s3.mytestconf
</codeph>
.
<codeblock>
gpcheckcloud -c "s3://
domain
/test1/abc config=s3.mytestconf"
</codeblock></p><p>
Download
<codeph>
s3.mytestconf
</codeph>
.
<codeblock>
gpcheckcloud -c "s3://
s3-us-west-2.amazonaws.com
/test1/abc config=s3.mytestconf"
</codeblock></p><p>
Download
all files from the S3 bucket location and send the output to
<codeph>
STDOUT
</codeph>
.
<codeblock>
gpcheckcloud -d "s3://
domain
/test1/abc config=s3.mytestconf"
</codeblock></p></section>
<codeblock>
gpcheckcloud -d "s3://
s3-us-west-2.amazonaws.com
/test1/abc config=s3.mytestconf"
</codeblock></p></section>
</body>
</topic>
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录