未验证 提交 d1b2d40d 编写于 作者: C Changjian Gao 提交者: GitHub

Update "How to use on K8s" doc: more details and troubleshooting guide (#326)

* Fix typo

* Update "How to Use on K8s" doc: more details and troubleshooting guide

* Update

* Update

* Revert P8s metrics port

* Revert "Revert P8s metrics port"

This reverts commit 2327f56006f3897aedca356176a1a71fd62e9566.
上级 02d8b408
...@@ -3,26 +3,24 @@ ...@@ -3,26 +3,24 @@
JuiceFS provides the [CSI driver](https://github.com/juicedata/juicefs-csi-driver) for Kubernetes. JuiceFS provides the [CSI driver](https://github.com/juicedata/juicefs-csi-driver) for Kubernetes.
## Prerequisites ## Prerequisites
- Kubernetes 1.14+ - Kubernetes 1.14+
## Installation ## Installation
### Install with Helm ### Install with Helm
To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install), Helm 3 is required. To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install), Helm 3 is required.
1. Prepare a file `values.yaml` with access infomation about Redis and object storage (take Amazon S3 `us-east-1` as an example) 1. Prepare a file `values.yaml` with access information about Redis and object storage (take Amazon S3 `us-east-1` as an example)
```yaml ```yaml
storageClasses: storageClasses:
- name: juicefs-sc - name: juicefs-sc
enabled: true enabled: true
reclaimPolicy: Delete reclaimPolicy: Retain
backend: backend:
name: "test" name: "test"
metaurl: "redis://juicefs.afyq4z.0001.use1.cache.amazonaws.com/3" metaurl: "redis://juicefs.afyq4z.0001.use1.cache.amazonaws.com/3"
...@@ -44,8 +42,16 @@ helm upgrade juicefs-csi-driver juicefs-csi-driver/juicefs-csi-driver --install ...@@ -44,8 +42,16 @@ helm upgrade juicefs-csi-driver juicefs-csi-driver/juicefs-csi-driver --install
3. Check the deployment 3. Check the deployment
- Check running pods: the deployment will launch a `StatefulSet` with replica `1` for the `juicefs-csi-controller` and a `DaemonSet` for `juicefs-csi-node`, so run `kubectl -n kube-system get pods | grep juicefs-csi` should see `n+1` (where `n` is the number of worker node of the kubernetes cluster) pods is running. - Check pods are running: the deployment will launch a `StatefulSet` named `juicefs-csi-controller` with replica `1` and a `DaemonSet` named `juicefs-csi-node`, so run `kubectl -n kube-system get pods -l app.kubernetes.io/name=juicefs-csi-driver` should see `n+1` (where `n` is the number of worker nodes of the Kubernetes cluster) pods is running. For example:
- Check secret: `kubectl -n kube-system describe secret juicefs-sc-secret` will show the secret with above `backend` fields:
```sh
$ kubectl -n kube-system get pods -l app.kubernetes.io/name=juicefs-csi-driver
NAME READY STATUS RESTARTS AGE
juicefs-csi-controller-0 3/3 Running 0 22m
juicefs-csi-node-v9tzb 3/3 Running 0 14m
```
- Check secret: `kubectl -n kube-system describe secret juicefs-sc-secret` will show the secret with above `backend` fields in `values.yaml`:
``` ```
Name: juicefs-sc-secret Name: juicefs-sc-secret
...@@ -70,15 +76,13 @@ secret-key: 0 bytes ...@@ -70,15 +76,13 @@ secret-key: 0 bytes
storage: 2 bytes storage: 2 bytes
``` ```
- Check storage class: `kubectl get sc juicefs-sc` will show the storage class like this:
- Check storage class: `kubectl get sc` will show the storage class like this:
``` ```
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
juicefs-sc csi.juicefs.com Delete Immediate false 21m juicefs-sc csi.juicefs.com Retain Immediate false 69m
``` ```
### Install with kubectl ### Install with kubectl
1. Deploy the driver: 1. Deploy the driver:
...@@ -95,7 +99,7 @@ curl -sSL https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/ ...@@ -95,7 +99,7 @@ curl -sSL https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/
2. Create storage class 2. Create storage class
- Add secret `juicefs-sc-secret` : - Create secret `juicefs-sc-secret`:
```bash ```bash
kubectl -n kube-system create secret generic juicefs-sc-secret \ kubectl -n kube-system create secret generic juicefs-sc-secret \
...@@ -121,26 +125,26 @@ parameters: ...@@ -121,26 +125,26 @@ parameters:
csi.storage.k8s.io/node-publish-secret-namespace: kube-system csi.storage.k8s.io/node-publish-secret-namespace: kube-system
csi.storage.k8s.io/provisioner-secret-name: juicefs-sc-secret csi.storage.k8s.io/provisioner-secret-name: juicefs-sc-secret
csi.storage.k8s.io/provisioner-secret-namespace: kube-system csi.storage.k8s.io/provisioner-secret-namespace: kube-system
reclaimPolicy: Delete reclaimPolicy: Retain
volumeBindingMode: Immediate volumeBindingMode: Immediate
``` ```
## Use JuiceFS ## Use JuiceFS
Now we can use JuiceFS in our pods. Here we create a `PVC` and refer it in a pod as an example: Now we can use JuiceFS in our pods. Here we create a `PVC` and refer to it in a pod as an example:
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: PersistentVolumeClaim kind: PersistentVolumeClaim
metadata: metadata:
name: jtext-pvc name: juicefs-pvc
spec: spec:
accessModes: accessModes:
- ReadWriteMany - ReadWriteMany
resources: resources:
requests: requests:
storage: 10Gi storage: 10Pi
storageClassName: juicefs-sc storageClassName: juicefs-sc
--- ---
apiVersion: v1 apiVersion: v1
...@@ -162,25 +166,29 @@ spec: ...@@ -162,25 +166,29 @@ spec:
volumes: volumes:
- name: juicefs-pv - name: juicefs-pv
persistentVolumeClaim: persistentVolumeClaim:
claimName: jtext-pvc claimName: juicefs-pvc
``` ```
Save above content to a file named like `juicefs-app.yaml` ,then use command `kubectl apply -f juicefs-app.yaml` to bootstrap the pod. Save above content to a file named like `juicefs-app.yaml`, then use command `kubectl apply -f juicefs-app.yaml` to bootstrap the pod. After that, you can check the status of pod:
```sh
$ kubectl get pod juicefs-app
NAME READY STATUS RESTARTS AGE
juicefs-app 1/1 Running 0 10m
```
For more details about JuiceFS CSI driver please refer [JuiceFS CSI driver](https://github.com/juicedata/juicefs-csi-driver). If the status of pod is not `Running` (e.g. `ContainerCreating`), there may have some issues. Please refer to the [troubleshooting](https://github.com/juicedata/juicefs-csi-driver/blob/master/docs/troubleshooting.md) document.
For more details about JuiceFS CSI driver please refer to [JuiceFS CSI Driver](https://github.com/juicedata/juicefs-csi-driver).
## Monitoring ## Monitoring
JuiceFS CSI driver can export [prometheus](https://prometheus.io) metrics at port `:9560` . JuiceFS CSI driver can export [Prometheus](https://prometheus.io) metrics at port `9567`.
### Configurate Prometheus server ### Configure Prometheus server
Add a job to `prometheus.yml` : Add a job to `prometheus.yml`:
```yaml ```yaml
scrape_configs: scrape_configs:
...@@ -194,14 +202,14 @@ scrape_configs: ...@@ -194,14 +202,14 @@ scrape_configs:
- source_labels: [__address__] - source_labels: [__address__]
action: replace action: replace
regex: ([^:]+)(:\d+)? regex: ([^:]+)(:\d+)?
replacement: $1:9560 replacement: $1:9567
target_label: __address__ target_label: __address__
- source_labels: [__meta_kubernetes_pod_node_name] - source_labels: [__meta_kubernetes_pod_node_name]
target_label: node target_label: node
action: replace action: replace
``` ```
Here we assume the prometheus server is running inside Kubernetes cluster, if your prometheus server is running outside Kubernetes cluster, make sure Kubernetes cluster nodes are reachable from prometheus server, refer [this issue](https://github.com/prometheus/prometheus/issues/4633) to add the `api_server` and `tls_config` client auth to the above configuration like this: Here we assume the Prometheus server is running inside Kubernetes cluster, if your prometheus server is running outside Kubernetes cluster, make sure Kubernetes cluster nodes are reachable from Prometheus server, refer to [this issue](https://github.com/prometheus/prometheus/issues/4633) to add the `api_server` and `tls_config` client auth to the above configuration like this:
```yaml ```yaml
scrape_configs: scrape_configs:
...@@ -219,9 +227,6 @@ scrape_configs: ...@@ -219,9 +227,6 @@ scrape_configs:
... ...
``` ```
### Configure Grafana dashboard
We provide a [dashboard template](./k8s_grafana_template.json) for [Grafana](https://grafana.com), which can be imported to show the collected metrics in Prometheus.
### Configurate Grafana dashboard
We provide a [dashboard template](./k8s_grafana_template.json) for [Grafana](https://grafana.com/) , which can be imported to show the collected metrics in Prometheus.
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册