diff --git a/gpdb-doc/dita/analytics/analytics.ditamap b/gpdb-doc/dita/analytics/analytics.ditamap
index 110788ac78718a142d885b2d08368fcef379f1b7..a48f78574aa6534a7ef636c10b8e841c831de8a8 100644
--- a/gpdb-doc/dita/analytics/analytics.ditamap
+++ b/gpdb-doc/dita/analytics/analytics.ditamap
@@ -12,7 +12,13 @@
This section includes the following information about PL/Container 1.1 and later: User defined functions (UDFs), using the Greenplum supported procedural languages, PL/Python
+ and PL/R, should only be created and used by trusted Greenplum Database administrators. This
+ restricts the data scientists using their own functions for training and debugging models.
+ PL/Container overcomes this limitation by allowing users to run UDFs inside a restricted
+ environment with no security risks. This topic covers information about the architecture, installation, and setup of
+ PL/Container: For detailed information about using PL/Container, refer to:
+
-
+
+
+
The PL/Container language extension is available as an open source module. For information
+ about the module, see the README file in the GitHub repository at
The Greenplum Database PL/Container language extension (PL/Container) is an interface that
- allows Greenplum Database to interact with a Docker container to execute a user-defined
- function (UDF) in the container. Docker containers ensure the user code cannot access the
- file system of the source host. Also, containers are started without network access or with
- limited network access and cannot connect back to Greenplum Database or open any other
- external connections. For information about available UDF languages, see
Generally speaking, a Docker container is a Linux process that runs in a managed way
- by using Linux kernel features such as cgroups, namespaces and union file systems. A Docker
- image is the basis of a container. A Docker container is a running instance of a
- Docker image. When you start a Docker container you specify a Docker image. A Docker image
- is the collection of root filesystem changes and execution parameters that are used when you
- run a Docker container on the host system. An image does not have state and never changes.
- For information about Docker, see the Docker web site
Greenplum Database starts a container only on the first call to a function in that - container. For example, consider a query that selects table data using all available - segments, and applies a transformation to the data using a PL/Container function. In this - case, Greenplum Database would start the Docker container only once on each segment, and - then contact the running container to obtain the results.
-After starting a full cycle of a query execution, the executor sends a call to the - container. The container might respond with an SPI - SQL query executed by the container to - get some data back from the database, returning the result to the query executor.
-The container shuts down when the connection to it is closed. This occurs when you close - the Greenplum Database session that started the container. A container running in standby - mode has almost no consumption of CPU resources as it is waiting on the socket. PL/Container - memory consumption depends on the amount of data you cache in global dictionaries.
-The PL/Container language extension is available as an open source module. For information
- about the module, see the README file in the GitHub repository at
These are requirements for running PL/Container with Greenplum Database:
-You can check your kernel version with the command
-
For PL/Container 2.1.x on CentOS or RHEL 7.x, or Ubuntu 18.04 - Docker 19.03
-See
These are PL/Container limitations:
-The Greenplum Database PL/Container language extension allows users to create and run
+ PL/Python or PL/R user-defined functions (UDFs) securely, inside a Docker container. A
+ Docker container is a Linux process that runs in a managed way by using Linux kernel
+ features such as cgroups, namespaces and union file systems. For information about Docker,
+ see the
Running UDFs inside the Docker container has several benefits:
+Example of the process flow:
+Consider a query that selects table data using all available segments, and transforms the + data using a PL/Container function. On the first call to a function in a segment + container, the query executor on master host starts the container on that segment host. It + then contacts the running container to obtain the results. The container might respond + with an SPI - a SQL query executed by the container to get some data back from the + database - returning the result to the query executor.
+A container running in standby mode waits on the socket and does not consume any CPU + resources. PL/Container memory consumption depends on the amount of data cached in global + dictionaries.
+The container connection is closed by closing the Greenplum Database session that started + the container, and the container shuts down.
+Greenplum Database runs PL/Container user-defined functions in Docker containers. The - Docker containers and the Greenplum Database server share CPU and memory resources on the - same hosts. In the default case, Greenplum Database is unaware of the resources consumed by - running PL/Container instances. Using PL/Container 1.2 and later with Greenplum Database - 5.8.0 and later, you can use Greenplum Database resource groups to control overall CPU and - memory resource usage for running PL/Container instances, as described in the following - section.
-PL/Container manages resource usage at two levels - the container level and the runtime
- level. You can control container-level CPU and memory resources with the
-
You cannot, by default, restrict the number of executing PL/Container container instances, - nor can you restrict the total amount of memory or CPU resources that they consume.
- -With PL/Container 1.2.0 and later, you can use Greenplum Database resource groups to
- manage and limit the total CPU and memory resources of containers in PL/Container
- runtimes. For more information about enabling, configuring, and using Greenplum Database
- resource groups, refer to
Resource groups for external components such as PL/Container use Linux control groups - (cgroups) to manage component-level use of memory and CPU resources. When you manage - PL/Container resources with resource groups, you configure both a memory limit and a CPU - limit that Greenplum Database applies to all container instances that share the same - PL/Container runtime configuration.
-When you create a resource group to manage the resources of a PL/Container runtime, you
- must specify
PL/Container does not use the
You can create one or more resource groups to manage your running PL/Container instances.
- After you create a resource group for PL/Container, you assign the resource group to one
- or more PL/Container runtimes. You make this assignment using the
You assign a resource group to a PL/Container runtime configuration by specifying the
-
You can also assign a resource group to a PL/Container runtime using the
-
After you assign a resource group to a PL/Container runtime, all container instances that - share the same runtime configuration are subject to the memory limit and the CPU limit - that you configured for the group. If you decrease the memory limit of a PL/Container - resource group, queries executing in running containers in the group may fail with an out - of memory error. If you drop a PL/Container resource group while there are running - container instances, Greenplum Database kills the running containers.
- -To use Greenplum Database resource groups to manage PL/Container resources, you must - explicitly configure both resource groups and PL/Container.
-Perform the following procedure to configure PL/Container to use Greenplum Database - resource groups for CPU and memory resource management:
-For
- information about the
PL/Python and PL/R image are available from the Greenplum Database product download site of
- Pivotal Network at
The Data Science packages contain a set of Python modules or R functions and data sets
- related to data science.
The Docker image tag represents the PL/Container release version (for example, 2.1.0). For
- example, the full Docker image name for the PL/Container for Python Docker image is similar
- to
To use PL/Container, install the PL/Container language extension, install Docker images, - and configure PL/Container to use the images.
If you are upgrading from PL/Container
- 1.0, see
Install the PL/Container language extension with the Greenplum Database
-
The following sections describe these tasks in detail.
+To use PL/Container you need to install Docker on all Greenplum Database host systems. + The these instructions show how to set up the Docker service on CentOS 7 but RHEL 7 is a + similar process.
+These steps install the docker package and start the Docker service as a user with sudo + privileges.
+For a list of observations while using Docker and PL/Container, see the
Install the PL/Container language extension using the
The command - registers PL/Container and creates PL/Container-specific functions and - views.
The
- SQL script registers the language
You'll need to + register the utility separately on each database that might need the PL/Container + functionality.
After installing PL/Container, you can manage Docker images and manage the PL/Container
- configuration with the Greenplum Database
To upgrade to PL/Container 1.1 or later, uninstall version 1.0 and install the new
- version. You cannot use the
As part of the upgrade process, you must drop PL/Container from all databases that are - configured with PL/Container.
-For
- information about the catalog tables,
These steps upgrade from PL/Container 1.0 to PL/Container 1.1 or later in a database. The - steps save the PL/Container 1.0 configuration and restore the configuration for use with - PL/Container 1.1 or later.
The
- script drops the
The - command registers PL/Container as an extension and creates PL/Container-specific - functions and views.
After upgrading PL/Container for a database, re-install any user-created PL/Container - UDFs that are required.
- -Install the Docker images that PL/Container will use to create language specific + containers to run the UDFs.
+ +The PL/Container open source module contains dockerfiles to build
+ Docker images that can be used with PL/Container. You can build a Docker image to run
+ PL/Python UDFs and a Docker image to run PL/R UDFs. See the dockerfiles in the GitHub
+ repository at
If + you require different images from the ones provided by Pivotal, you can create custom + Docker images, install the image and add the image to the PL/ Container configuration. +
Use the
The utility displays progress information, similar to:
+For more information on
Add the image information to the PL/Container configuration file using
+
Use the
For details on other
+
For
+ more information about enabling, configuring, and using Greenplum Database resource
+ groups with PL/Container, see
You can now create a simple function to test your PL/Container installation.
+List the names of the runtimes your created and added to the PL/Container XML
+ file:
Which will show a list of all installed
+ runtimes:
You can also view the PL/Container configuration information with the
Use the
If PL/Container extension is not registered with the selected database, first enable the + extension:
+Create a simple function to test your installation, where the function will use the
+ runtime
And test the function using:
+For further details and examples about using PL/Container functions, see