提交 7928421c 编写于 作者: L liqingping

docs: update architecture image and user guide

上级 d0e055a1
......@@ -24,19 +24,19 @@ di-server-7b86ff8df4-jfgmp 1/1 Running 0 59s
Install global components of DIJob defined in AggregatorConfig:
```bash
kubectl create -f examples/di_v1alpha1_agconfig.yaml -n di-system
kubectl create -f config/samples/agconfig.yaml -n di-system
```
### Submit DIJob
```bash
# submit DIJob
$ kubectl create -f examples/di_v1alpha1_dijob.yaml
$ kubectl create -f config/samples/dijob-cartpole.yaml
# get pod and you will see coordinator is created by di-operator
# a few seconds later, you will see collectors and learners created by di-server
$ kubectl get pod
# get logs of coordinator
$ kubectl logs dijob-example-coordinator
$ kubectl logs cartpole-dqn-coordinator
```
## User Guide
......
......@@ -8,7 +8,7 @@ DI框架分为3个重要的模块,分别是coordinator、collector和learner
为了提供DI在Kubernetes(K8s)中运行的支持,我们设计了DI Orchestrator,本文将说明利用DI Orchestrator,DI各个模块在K8s系统上如何被创建、如何相互发现、如何开始训练等。DI Orchestrator的架构如下图所示:
![](images/di-arch.png)
![](images/di-arch.svg)
整体分为两大模块:`di-server``di-operator``DDPL`指ddp learner,`Lm`指Learner,`Cn`指Collector,`Aggregator+DDPL`构成一个logic learner。接下来将首先介绍一个DI任务提交到K8s之后DI Orchestrator如何将DI的各个模块(在K8s中就是一个[pod](https://kubernetes.io/docs/concepts/workloads/pods/))创建并启动,然后将对di-server和di-operator进行介绍。
......
......@@ -8,7 +8,7 @@ For the introduction of DI, please refer to [DI developer tutorial](https://open
In order to provide running support for DI in Kubernetes (K8s), we designed `DI Orchestrator`. This article will explain how to use DI Orchestrator, how each module of DI is created on K8s and discovers each other, how to start training, etc. The architecture of DI Orchestrator is shown in the figure below:
![](images/di-arch.png)
![](images/di-arch.svg)
There are two main modules that is `di-server` and `di-operator`.
`DDPL` represents ddp learner, `Lm` represents logic learner, `Cn` represents collector, and `Aggregator+DDPL` constructs a logic learner. In the following pages, we will first introduce how `DI Orchestrator` creates and starts each module of DI after a DI job is submitted to K8s, and then introduces the architecture of `di-server` and `di-operator`.
......
# developer guide
# Developer Guide
## prerequisites
## Prerequisites
- a well prepared kubernetes cluster. Follow the [instructions](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) to create a kubernetes cluster, or create a local kubernetes node referring to [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) or [minikube](https://minikube.sigs.k8s.io/docs/start/)
- kustomize. Installed by the following command
```bash
......@@ -11,7 +11,7 @@ kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
```bash
kubectl create -f ./config/certmanager/cert-manager.yaml
```
## project initialization
## Project Initialization
This project is based on [kubebuilder v3](https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.0.0/kubebuilder_linux_amd64), since CRDs generated by kubebuilder v2 is not compatible in kubernetes v1.20.
```bash
kubebuilder init --domain opendilab.org --license apache2 --owner "The OpenDILab authors"
......@@ -32,26 +32,28 @@ make manifests
```
New CRD files will be generated in [./config/crd/bases](./config/crd/bases)
## controller logic
## Controller Logic
Referenced to [controllers](./controllers)
## di-server logic
## DI Server Logic
Referenced to [server](./server)
## Installation
Run the following command in the project root directory.
```bash
# build images. If you are not working in Linux, here you should use `make docker-build`
make dev-images
# build images.
make docker-build
make docker-push
# deploy di-operator and server to cluster
make dev-deploy
```
Since the CustomResourceDefinitions are too long, you will probably find the following error:
![](docs/images/deploy-failed.png)
```bash
The CustomResourceDefinition "dijobs.diengine.opendilab.org" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
```
Then run the following command will solve the problem:
Then running the following command will solve the problem:
```bash
kustomize build config/crd | kubectl create -f -
```
......@@ -66,5 +68,5 @@ di-server-7b86ff8df4-jfgmp 1/1 Running 0 59s
Install global components of DIJob defined in AggregatorConfig:
```bash
kubectl create -f examples/di-mock-agconfig.yaml -n di-system
kubectl create -f config/samples/agconfig.yaml -n di-system
```
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册