is secured with Kerberos ("Kerberized"), you must configure Greenplum Database
and PXF to allow users accessing external tables to authenticate with Kerberos.
Refer to <xref
href="../../../pxf/5-14/pxf_kerbhdfs.html"scope="external"format="html">Configuring PXF for Secure HDFS</xref> for the procedure to perform this setup. </p>
href="../../../pxf/5-15/pxf_kerbhdfs.html"scope="external"format="html">Configuring PXF for Secure HDFS</xref> for the procedure to perform this setup. </p>
@@ -92,17 +92,17 @@ Configuration properties required by PXF, and the `gphdfs` equivalent, if applic
After you determine the equivalent PXF configuration properties, you will:
1. Update the Java version installed on each Greenplum Database host, if necessary. PXF supports Java version 8. If your Greenplum Database cluster hosts are running Java 7, upgrade to Java version 8 as described in [Installing Java for PXF](../../pxf/5-14/install_java.html).
1. Update the Java version installed on each Greenplum Database host, if necessary. PXF supports Java version 8. If your Greenplum Database cluster hosts are running Java 7, upgrade to Java version 8 as described in [Installing Java for PXF](../../pxf/5-15/install_java.html).
2. Initialize PXF as described in [Initializing PXF](../../pxf/5-14/init_pxf.html).
2. Initialize PXF as described in [Initializing PXF](../../pxf/5-15/init_pxf.html).
You must initialize PXF before you can access Hadoop.
3. [Configure the PXF Hadoop Connectors](../../pxf/5-14/client_instcfg.html). This procedure creates a PXF server configuration that provides PXF the information that it requires to access Hadoop.
3. [Configure the PXF Hadoop Connectors](../../pxf/5-15/client_instcfg.html). This procedure creates a PXF server configuration that provides PXF the information that it requires to access Hadoop.
4. Start PXF in your Greenplum Database cluster as described in [Starting PXF](../../pxf/5-14/cfginitstart_pxf.html#start_pxf).
4. Start PXF in your Greenplum Database cluster as described in [Starting PXF](../../pxf/5-15/cfginitstart_pxf.html#start_pxf).
5. Register the PXF extension and [grant users access to PXF](../../pxf/5-14/using_pxf.html).
5. Register the PXF extension and [grant users access to PXF](../../pxf/5-15/using_pxf.html).
## <aid="create_pxf_ext_table"></a>Creating a PXF External Table
You are not required to specify the HDFS host and port number when you create a PXF external table. PXF obtains this information from the `default` server configuration, or from the server configuration name that you specify in `<server_name>`.
Refer to [Creating an External Table](../../pxf/5-14/intro_pxf.html#create_external_table) in the PXF documentation for more information about the PXF `CREATE EXTERNAL TABLE` syntax and keywords.
Refer to [Creating an External Table](../../pxf/5-15/intro_pxf.html#create_external_table) in the PXF documentation for more information about the PXF `CREATE EXTERNAL TABLE` syntax and keywords.
When you create an external table specifying the `gphdfs` protocol, you identify the format of the external data in the `FORMAT` clause (discussed in the next section). PXF uses a `PROFILE` option in the `LOCATION` clause to identify the source and type of the external data.
...
...
@@ -138,7 +138,7 @@ When you create an external table specifying the `gphdfs` protocol, you identify
| Parquet | hdfs:parquet |
| Text | hdfs:text |
Refer to [Connectors, Data Formats, and Profiles](../../pxf/5-14/access_hdfs.html#hadoop_connectors) in the PXF documentation for more information about the PXF profiles supported for Hadoop.
Refer to [Connectors, Data Formats, and Profiles](../../pxf/5-15/access_hdfs.html#hadoop_connectors) in the PXF documentation for more information about the PXF profiles supported for Hadoop.
Both `gphdfs` and `pxf` utilize custom options in the `LOCATION` clause to identify data-format-, operation-, or profile-specific options supported by the protocol. For example, both `gphdfs` and `pxf` support parquet and compression options on `INSERT` operations.
@@ -53,7 +53,7 @@ Perform this procedure in your Greenplum Database 5 installation:
3. Greenplum 6 removes the `gphdfs` external table protocol. If you have `gphdfs` external tables defined in your Greenplum 5 installation, you must delete or migrate them to `pxf` as described in [Migrating gphdfs External Tables to PXF](gphdfs-pxf-migrate.html).
3. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-14/cfginitstart_pxf.html#stop_pxf).
3. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-15/cfginitstart_pxf.html#stop_pxf).
4. If you plan to install Greenplum Database 6 on a new set of hosts, be sure to save a copy of the `$PXF_CONF` directory in your Greenplum 5 installation.
...
...
@@ -78,7 +78,7 @@ After you install and configure Greenplum Database 6 and migrate table definitio
gpadmin@gp6master$ pxf version
```
3. If the version of PXF installed in your Greenplum 5 installation is newer than the version of PXF installed in your Greenplum 6 cluster, install the PXF distribution on your Greenplum 6 hosts as described in [Installing PXF](../../pxf/5-14/installing_pxf.html). Be sure to install a version of PXF that is the same, or newer, than the version of PXF installed in your Greenplum 5 cluster.
3. If the version of PXF installed in your Greenplum 5 installation is newer than the version of PXF installed in your Greenplum 6 cluster, install the PXF distribution on your Greenplum 6 hosts as described in [Installing PXF](../../pxf/5-15/installing_pxf.html). Be sure to install a version of PXF that is the same, or newer, than the version of PXF installed in your Greenplum 5 cluster.
2. If you installed Greenplum Database 6 on a new set of hosts, copy the `$PXF_CONF` directory from your Greenplum 5 installation to the master node. Consider copying the directory to the same file system location at which it resided in the 5 cluster. For example, if `PXF_CONF=/usr/local/greenplum-pxf`:
...
...
@@ -86,7 +86,7 @@ After you install and configure Greenplum Database 6 and migrate table definitio
3. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-14/init_pxf.html), specifying the `PXF_CONF` directory that you copied in the step above.
3. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-15/init_pxf.html), specifying the `PXF_CONF` directory that you copied in the step above.
4. Perform any version-applicable upgrade actions identified in [Step 2](https://gpdb.docs.pivotal.io/5280/pxf/upgrade_pxf.html#pxfup) of the PXF 5.x upgrade documentation in your Greenplum Database 6 installation. **Start with step 6 in the procedure**.
...
...
@@ -96,7 +96,7 @@ After you install and configure Greenplum Database 6 and migrate table definitio
gpadmin@gp6master$ pxf cluster sync
```
6. Start PXF on each Greenplum Database 6 segment host as described in [Starting PXF](../../pxf/5-14/cfginitstart_pxf.html#start_pxf).
6. Start PXF on each Greenplum Database 6 segment host as described in [Starting PXF](../../pxf/5-15/cfginitstart_pxf.html#start_pxf).
7. Verify the migration by testing that each PXF external table can access the referenced data store.
@@ -33,7 +33,7 @@ Perform this procedure before you upgrade to a new version of Greenplum Database
gpadmin@gpmaster$ pxf version
```
3. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-14/cfginitstart_pxf.html#stop_pxf).
3. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-15/cfginitstart_pxf.html#stop_pxf).
4. **If you are upgrading from Greenplum Database version 5.14 or earlier**:
...
...
@@ -67,13 +67,13 @@ After you upgrade to the new version of Greenplum Database, perform the followin
gpadmin@gpmaster pxf cluster register
```
2. Start PXF on each segment host as described in [Starting PXF](../../pxf/5-14/cfginitstart_pxf.html#start_pxf).
2. Start PXF on each segment host as described in [Starting PXF](../../pxf/5-15/cfginitstart_pxf.html#start_pxf).
3. Exit this procedure.
2. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-14/init_pxf.html).
2. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-15/init_pxf.html).
3. PXF user impersonation is on by default in Greenplum Database version 5.5.0 and later. If you are upgrading from an older *PXF.from* version, you must configure user impersonation for the underlying Hadoop services. Refer to [Configuring User Impersonation and Proxying](../../pxf/5-14/pxfuserimpers.html) for instructions, including the configuration procedure to turn off PXF user impersonation.
3. PXF user impersonation is on by default in Greenplum Database version 5.5.0 and later. If you are upgrading from an older *PXF.from* version, you must configure user impersonation for the underlying Hadoop services. Refer to [Configuring User Impersonation and Proxying](../../pxf/5-15/pxfuserimpers.html) for instructions, including the configuration procedure to turn off PXF user impersonation.
4. **If you are upgrading from Greenplum Database version 5.14 or earlier**:
...
...
@@ -96,11 +96,11 @@ After you upgrade to the new version of Greenplum Database, perform the followin
1. PXF now bundles Hadoop version 2.9.2 dependent JAR files. If you registered additional Hadoop-related JAR files in `$PXF_CONF/lib`, ensure that these libraries are compatible with Hadoop version 2.9.2.
2. The PXF JDBC connector now supports file-based server configuration. If you choose to utilize this new feature with existing external tables that reference an external SQL database, refer to the configuration instructions in [Configuring the JDBC Connector](../../pxf/5-14/jdbc_cfg.html) for additional information.
2. The PXF JDBC connector now supports file-based server configuration. If you choose to utilize this new feature with existing external tables that reference an external SQL database, refer to the configuration instructions in [Configuring the JDBC Connector](../../pxf/5-15/jdbc_cfg.html) for additional information.
2. The PXF JDBC connector now supports statement query timeouts. This feature requires explicit support from the JDBC driver. The default query timeout is `0`, wait forever. Some JDBC drivers support a `0` timeout value without fully supporting the statement query timeout feature. Ensure that any JDBC driver that you have registered supports the default timeout, or better yet, fully supports this feature. You may need to update your JDBC driver version to obtain this support. Refer to the configuration instructions in [JDBC Driver JAR Registration](../../pxf/5-14/jdbc_cfg.html#cfg_jar) for information about registering JAR files with PXF.
2. The PXF JDBC connector now supports statement query timeouts. This feature requires explicit support from the JDBC driver. The default query timeout is `0`, wait forever. Some JDBC drivers support a `0` timeout value without fully supporting the statement query timeout feature. Ensure that any JDBC driver that you have registered supports the default timeout, or better yet, fully supports this feature. You may need to update your JDBC driver version to obtain this support. Refer to the configuration instructions in [JDBC Driver JAR Registration](../../pxf/5-15/jdbc_cfg.html#cfg_jar) for information about registering JAR files with PXF.
3. If you plan to use Hive partition filtering with integral types, you must set the `hive.metastore.integral.jdo.pushdown` Hive property in your Hadoop cluster's `hive-site.xml` and in your PXF user configuration `$PXF_CONF/servers/default/hive-site.xml` file. See [Updating Hadoop Configuration](../../pxf/5-14/client_instcfg.html#client-cfg-update).
3. If you plan to use Hive partition filtering with integral types, you must set the `hive.metastore.integral.jdo.pushdown` Hive property in your Hadoop cluster's `hive-site.xml` and in your PXF user configuration `$PXF_CONF/servers/default/hive-site.xml` file. See [Updating Hadoop Configuration](../../pxf/5-15/client_instcfg.html#client-cfg-update).
5. **If you are upgrading from Greenplum Database version 5.21.1 or earlier**: The PXF Hive connector no longer supports providing a `DELIMITER=<delim>` option in the `LOCATION` URI when you create an external table that specifies the `HiveText` or `HiveRC` profiles. If you have previously created an external table that specified a `DELIMITER` in the `LOCATION` URI, you must drop the table, and recreate it omitting the `DELIMITER` from the `LOCATION`. You are still required to provide a non-default delimiter in the external table formatting options.
...
...
@@ -147,5 +147,5 @@ After you upgrade to the new version of Greenplum Database, perform the followin
@@ -29,7 +29,7 @@ Perform this procedure before you upgrade to a new version of Greenplum Database
gpadmin@gpmaster$ pxf version
```
2. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-14/cfginitstart_pxf.html#stop_pxf).
2. Stop PXF on each segment host as described in [Stopping PXF](../../pxf/5-15/cfginitstart_pxf.html#stop_pxf).
3. Upgrade to the new version of Greenplum Database and then continue your PXF upgrade with [Step 2: Upgrading PXF](#pxfup).
...
...
@@ -52,11 +52,11 @@ After you upgrade to the new version of Greenplum Database, perform the followin
gpadmin@gpmaster pxf cluster register
```
2. Start PXF on each segment host as described in [Starting PXF](../../pxf/5-14/cfginitstart_pxf.html#start_pxf).
2. Start PXF on each segment host as described in [Starting PXF](../../pxf/5-15/cfginitstart_pxf.html#start_pxf).
3. Exit this procedure.
2. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-14/init_pxf.html). You may choose to use your existing `$PXF_CONF` for the initialization.
2. Initialize PXF on each segment host as described in [Initializing PXF](../../pxf/5-15/init_pxf.html). You may choose to use your existing `$PXF_CONF` for the initialization.
3. **If you are upgrading from Greenplum Database version 6.1.x or earlier** and you have configured any JDBC servers that access Kerberos-secured Hive, you must now set the `hadoop.security.authentication` property to the `jdbc-site.xml` file to explicitly identify use of the Kerberos authentication method. Perform the following for each of these server configs:
...
...
@@ -99,5 +99,5 @@ After you upgrade to the new version of Greenplum Database, perform the followin
