Add helm chart proto (#3102)

.gitignore

@@ -34,3 +34,10 @@ yarn-error.log*
 #Ignore Cypress tests temp files
 /tests/cypress/fixtures
 /tests/cypress/screenshots
+.idea/
+
+#Ignore helm-related files
+/helm-chart/Chart.lock
+/helm-chart/values.*.yaml
+/helm-chart/*.values.yaml
+/helm-chart/charts/*

CHANGELOG.md

@@ -12,6 +12,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Documentation on mask annotation (<https://github.com/openvinotoolkit/cvat/pull/3044>)
 - Hotkeys to switch a label of existing object or to change default label (for objects created with N) (<https://github.com/openvinotoolkit/cvat/pull/3070>)
 - A script to convert some kinds of DICOM files to regular images (<https://github.com/openvinotoolkit/cvat/pull/3095>)
+- Helm chart prototype (<https://github.com/openvinotoolkit/cvat/pull/3102>)
+
 
 ### Changed
 

helm-chart/.helmignore

+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

helm-chart/Chart.yaml

+apiVersion: v2
+name: cvat
+description: A Helm chart for Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.2.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: latest
+# We dont use it, so you can override it using values.override.yaml
+
+dependencies:
+  - name: redis
+    version: "12.9.*"
+    repository: https://charts.bitnami.com/bitnami
+    condition: redis.enabled
+  - name: postgresql
+    version: "10.3.*"
+    repository: https://charts.bitnami.com/bitnami
+    condition: postgresql.enabled

helm-chart/README.md

+# FAQ
+## What should be configured before installation?
+1. You should have configured connection to existed k8s cluster
+2. Helm must be installed
+3. You should download chart external dependencies, using following commands:
+```
+   helm repo add bitnami https://charts.bitnami.com/bitnami
+   helm repo update
+   helm dependency update
+```
+4. (Optional) Install ingress of your choice (for example: https://github.com/kubernetes/ingress-nginx)
+5. (Optional) Create certificates for https  (for example: https://github.com/jetstack/cert-manager/ )
+6. (Optional) Create values.override.yaml and override there parameters you want
+7. Change postgresql password as described below
+8. Add ingress to values.override.yaml(example also below)
+7. Deploy cvat using command below
+## How to deploy new version of chart to cluster?
+Execute following command:
+```helm upgrade <release_name> --install ./helm-chart -f ./helm-chart/values.yaml  -f values.override.yaml(if exists) --namespace <desired namespace>```
+## How to create superuser?
+```
+HELM_RELEASE_NAMESPACE="<insert>" &&\
+HELM_RELEASE_NAME="<insert>" &&\
+BACKEND_POD_NAME=$(kubectl get pod --namespace $HELM_RELEASE_NAMESPACE -l tier=backend,app.kubernetes.io/instance=$HELM_RELEASE_NAME -o jsonpath='{.items[0].metadata.name}') &&\
+kubectl exec -it --namespace $HELM_RELEASE_NAMESPACE $BACKEND_POD_NAME -c cvat-backend-app-container -- python manage.py createsuperuser
+```
+## How to change embedded postgresql password?
+There are several passwords used here, for security reasons - better change them all.
+```
+postgresql:
+  secret:
+    password: cvat_postgresql
+    postgres_password: cvat_postgresql_postgres
+    replication_password: cvat_postgresql_replica
+```
+Or, if you know how to work with k8s - you could create your own secret and use it here:
+```
+postgresql:
+   global:
+     postgresql:
+       existingSecret: cvat-postgres-secret
+```
+## How to describe ingress:
+  Just set `ingress.enabled:` to `true`, then copy example, uncomment it and change values there
+## How to understand what diff will be inflicted by 'helm upgrade'?
+You can use https://github.com/databus23/helm-diff#install for that
+## I want to use my own postgresql/redis with your chart.
+Just set `postgresql.enabled` or `redis.enabled` to `false`, as described below.
+Then - put your instance params to "external" field
+## I want to override some settings in values.yaml.
+Just create file `values.override.yaml` and place your changes here, using same structure as in `values.yaml`.
+Then reference it in helm update/install command using `-f` flag
+## Why you used external charts to provide redis and postgres?
+Because they definitely know what they do better then we are, so we are getting more quality and less support
+## What is kubernetes and how it is working?
+See https://kubernetes.io/
+## What is helm and how it is working?
+See https://helm.sh/

helm-chart/templates/_helpers.tpl

+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "cvat.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "cvat.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "cvat.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "cvat.labels" -}}
+helm.sh/chart: {{ include "cvat.chart" . }}
+{{ include "cvat.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "cvat.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "cvat.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "cvat.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "cvat.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}

helm-chart/templates/cvat-postgres-secret.yml

+{{- if .Values.postgresql.secret.create }}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+type: generic
+stringData:
+  postgresql-hostname: "{{ .Release.Name }}-postgresql"
+  postgresql-database: {{ .Values.postgresql.postgresqlDatabase }}
+  postgresql-username: {{ .Values.postgresql.postgresqlUsername }}
+  postgresql-password: {{ .Values.postgresql.secret.password }}
+  postgresql-postgres-password: {{ .Values.postgresql.secret.postgres_password }}
+  postgresql-replication-password: {{ .Values.postgresql.secret.replication_password }}
+{{- end }}

helm-chart/templates/cvat_backend/deployment.yml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ .Release.Name }}-backend
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: cvat-app
+    tier: backend
+    {{- include "cvat.labels" . | nindent 4 }}
+    {{- with .Values.cvat.backend.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+  {{- with .Values.cvat.backend.annotations }}
+  annotations:
+  {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+  replicas: {{ .Values.cvat.backend.replicas }}
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      {{- include "cvat.labels" . | nindent 6 }}
+      {{- with .Values.cvat.backend.labels }}
+      {{- toYaml . | nindent 6 }}
+      {{- end }}
+      app: cvat-app
+      tier: backend
+  template:
+    metadata:
+      labels:
+        app: cvat-app
+        tier: backend
+        {{- include "cvat.labels" . | nindent 8 }}
+        {{- with .Values.cvat.backend.labels }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+      {{- with .Values.cvat.backend.annotations }}
+      annotations:
+      {{- toYaml . | nindent 8 }}
+      {{- end }}
+    spec:
+      containers:
+        - name: cvat-backend-app-container
+          image: {{ .Values.cvat.backend.image }}:{{ .Values.cvat.backend.tag }}
+          {{- with .Values.cvat.backend.resources }}
+          resources:
+          {{- toYaml . | nindent 12 }}
+          {{- end }}
+          env:
+          - name: DJANGO_MODWSGI_EXTRA_ARGS
+            value: {{ .Values.cvat.backend.envs.DJANGO_MODWSGI_EXTRA_ARGS}}
+          - name: ALLOWED_HOSTS
+            value: {{ .Values.cvat.backend.envs.ALLOWED_HOSTS | squote}}
+          - name: UI_HOST
+            value: "{{ .Release.Name }}-frontend-service"
+          - name: UI_PORT
+            value: "{{ .Values.cvat.frontend.service.ports }}"
+          {{- if .Values.redis.enabled }}
+          - name: CVAT_REDIS_HOST
+            value: "{{ .Release.Name }}-redis-master"
+          {{- else }}
+          - name: CVAT_REDIS_HOST
+            value: "{{ .Values.redis.external.host }}"
+          {{- end }}
+          - name: CVAT_POSTGRES_HOST
+            valueFrom:
+              secretKeyRef:
+                name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
+                key: postgresql-hostname
+          - name: CVAT_POSTGRES_USER
+            valueFrom:
+              secretKeyRef:
+                name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
+                key: postgresql-username
+          - name: CVAT_POSTGRES_DBNAME
+            valueFrom:
+              secretKeyRef:
+                name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
+                key: postgresql-database
+          - name: CVAT_POSTGRES_PASSWORD
+            valueFrom:
+              secretKeyRef:
+                name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
+                key: postgresql-password
+          {{- with .Values.cvat.backend.additionalEnv }}
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+          ports:
+          - containerPort: 8080
+          volumeMounts:
+          - mountPath: /home/django/data
+            name: cvat-backend-data
+            subPath: data
+          - mountPath: /home/django/keys
+            name: cvat-backend-data
+            subPath: keys
+          - mountPath: /home/django/logs
+            name: cvat-backend-data
+            subPath: logs
+          - mountPath: /home/django/models
+            name: cvat-backend-data
+            subPath: models
+          {{- with .Values.cvat.backend.additionalVolumeMounts }}
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+      initContainers:
+        - name: user-data-permission-fix
+          image: busybox
+          command: ["/bin/chmod", "-R", "777", "/home/django"]
+          {{- with .Values.cvat.backend.resources }}
+          resources:
+          {{- toYaml . | nindent 12 }}
+          {{- end }}
+          volumeMounts:
+          {{- if .Values.cvat.backend.defaultStorage.enabled }}
+          - mountPath: /home/django/data
+            name: cvat-backend-data
+            subPath: data
+          - mountPath: /home/django/keys
+            name: cvat-backend-data
+            subPath: keys
+          - mountPath: /home/django/logs
+            name: cvat-backend-data
+            subPath: logs
+          - mountPath: /home/django/models
+            name: cvat-backend-data
+            subPath: models
+          {{- end }}
+          {{- with .Values.cvat.backend.additionalVolumeMounts }}
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+      {{- with .Values.cvat.frontend.affinity }}
+      affinity:
+      {{- toYaml . | nindent 8 }}
+      {{- end }}
+      volumes:
+        {{- if .Values.cvat.backend.defaultStorage.enabled }}
+        - name: cvat-backend-data
+          persistentVolumeClaim:
+            claimName: "{{ .Release.Name }}-backend-data"
+        {{- end }}
+        {{- with .Values.cvat.backend.additionalVolumes }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+      {{- with .Values.imagePullSecrets }}
+      imagePullSecrets:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}

helm-chart/templates/cvat_backend/service.yml

+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Release.Name }}-backend-service
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+    app: cvat-app
+    tier: backend
+spec:
+  selector:
+    app: cvat-app
+    tier: backend
+    {{- include "cvat.labels" . | nindent 4 }}
+  {{- with .Values.cvat.backend.service }}
+  {{- toYaml . | nindent 2 }}
+  {{- end }}

helm-chart/templates/cvat_backend/storage.yml

+{{- if .Values.cvat.backend.defaultStorage.enabled }}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ .Release.Name }}-backend-data
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+    app: cvat-app
+    tier: backend
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: {{ .Values.cvat.backend.defaultStorage.size }}
+{{- end}}

helm-chart/templates/cvat_frontend/deployment.yml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ .Release.Name }}-frontend
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+    app: cvat-app
+    tier: frontend
+spec:
+  replicas: {{ .Values.cvat.frontend.replicas }}
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      {{- include "cvat.labels" . | nindent 6 }}
+      app: cvat-app
+      tier: frontend
+  template:
+    metadata:
+      labels:
+        {{- include "cvat.labels" . | nindent 8 }}
+        app: cvat-app
+        tier: frontend
+        {{- with .Values.cvat.backend.labels }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+        {{- with .Values.cvat.backend.annotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+    spec:
+      containers:
+        - name: cvat-frontend-app-container
+          image: {{ .Values.cvat.frontend.image }}:{{ .Values.cvat.frontend.tag }}
+          {{- with .Values.cvat.frontend.resources }}
+          resources:
+          {{- toYaml . | nindent 12 }}
+          {{- end }}
+          ports:
+          - containerPort: 80
+          {{- with .Values.cvat.frontend.additionalEnv }}
+          env:
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+          {{- with .Values.cvat.frontend.additionalVolumeMounts }}
+          volumeMounts:
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+        {{- with .Values.cvat.frontend.additionalVolumes }}
+      volumes:
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+        {{- with .Values.cvat.frontend.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.imagePullSecrets }}
+      imagePullSecrets:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}

helm-chart/templates/cvat_frontend/service.yml

+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Release.Name }}-frontend-service
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+    app: cvat-app
+    tier: frontend
+spec:
+  selector:
+    app: cvat-app
+    tier: frontend
+    {{- include "cvat.labels" . | nindent 4 }}
+  {{- with .Values.cvat.frontend.service }}
+  {{- toYaml . | nindent 2 }}
+  {{- end }}

helm-chart/templates/ingress.yaml

+{{- if .Values.ingress.enabled -}}
+{{- $fullName := include "cvat.fullname" . -}}
+{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullName }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "cvat.labels" . | nindent 4 }}
+  {{- with .Values.ingress.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+{{- if .Values.ingress.tls }}
+  tls:
+  {{- range .Values.ingress.tls }}
+    - hosts:
+      {{- range .hosts }}
+        - {{ . | quote }}
+      {{- end }}
+      secretName: {{ .secretName }}
+  {{- end }}
+{{- end }}
+  rules:
+  {{- range .Values.ingress.hosts }}
+    - host: {{ .host | quote }}
+      http:
+        paths:
+        {{- range .paths }}
+          - path: {{ .path }}
+            {{- if .pathType }}
+            pathType: {{ .pathType }}
+            {{- end }}
+            backend:
+              serviceName: {{ .service.name }}
+              servicePort: {{ .service.port }}
+        {{- end }}
+  {{- end }}
+{{- end }}

helm-chart/values.yaml

+# Default values for cvat.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+
+imagePullSecrets: []
+nameOverride: ""
+fullnameOverride: ""
+
+cvat:
+  backend:
+    replicas: 1
+    image: openvino/cvat_server
+    tag: latest
+    labels: {}
+    #  test: test
+    annotations: {}
+    # test.io/test: test
+    resources: {}
+    affinity: {}
+    # nodeAffinity:
+    #   requiredDuringSchedulingIgnoredDuringExecution:
+    #     nodeSelectorTerms:
+    #     - matchExpressions:
+    #       - key: kubernetes.io/e2e-az-name
+    #         operator: In
+    #         values:
+    #         - e2e-az1
+    #         - e2e-az2
+    envs:
+        ALLOWED_HOSTS: "*"
+        DJANGO_MODWSGI_EXTRA_ARGS: ""
+    additionalEnv: []
+    # Example:
+    #  - name: volume-from-secret
+    # - name: TEST
+    #  value: "test"
+    additionalVolumes: []
+    # Example(assumes that pvc was already created):
+    # - name: tmp
+    #   persistentVolumeClaim:
+    #       claimName: tmp
+    additionalVolumeMounts: []
+    # Example:
+    # -   mountPath: /tmp
+    #     name: tmp
+    #     subPath: test
+    service:
+      type: ClusterIP
+      ports:
+        - port: 8080
+          targetPort: 8080
+          protocol: TCP
+          name: http
+    defaultStorage:
+        enabled: true
+        size: 20Gi
+  frontend:
+    replicas: 1
+    image: openvino/cvat_ui
+    tag: latest
+    labels: {}
+    #  test: test
+    annotations: {}
+    # test.io/test: test
+    resources: {}
+    affinity: {}
+    # nodeAffinity:
+    #   requiredDuringSchedulingIgnoredDuringExecution:
+    #     nodeSelectorTerms:
+    #     - matchExpressions:
+    #       - key: kubernetes.io/e2e-az-name
+    #         operator: In
+    #         values:
+    #         - e2e-az1
+    #         - e2e-az2
+    additionalEnv: []
+    # Example:
+    #  - name: volume-from-secret
+    # - name: TEST
+    #  value: "test"
+    additionalVolumes: []
+    # Example(assumes that pvc was already created):
+    # - name: tmp
+    #   persistentVolumeClaim:
+    #       claimName: tmp
+    additionalVolumeMounts: []
+    # Example:
+    # -   mountPath: /tmp
+    #     name: tmp
+    #     subPath: test
+    service:
+      type: ClusterIP
+      ports:
+        - port: 80
+          targetPort: 80
+          protocol: TCP
+          name: http
+
+postgresql:
+  #See https://github.com/bitnami/charts/blob/master/bitnami/postgresql/ for more info
+  enabled: true # false for external db
+  external:
+    host: 127.0.0.1
+    port: 5432
+    user: postgres
+    password: postgres
+    dbname: cvat
+ # If not external following config will be applied by default
+  global:
+    postgresql:
+      existingSecret: cvat-postgres-secret
+  secret:
+    create: true
+    name: postgres-secret
+    password: cvat_postgresql
+    postgres_password: cvat_postgresql_postgres
+    replication_password: cvat_postgresql_replica
+  postgresqlDatabase: cvat
+  postgresqlUsername: cvat
+  service:
+    port: 5432
+
+redis:
+  #See https://github.com/bitnami/charts/blob/master/bitnami/redis/ for more info
+  enabled: true
+  external:
+      host: 127.0.0.1
+  usePassword: false
+  cluster:
+    enabled: false
+
+ingress:
+  enabled: false
+# Example for nginx ingress and cert manager
+#    annotations:
+#        kubernetes.io/ingress.class: nginx
+#        kubernetes.io/tls-acme: "true"
+#        ingress.kubernetes.io/ssl-redirect: "true"
+#        nginx.ingress.kubernetes.io/use-regex: "true"
+#        nginx.ingress.kubernetes.io/secure-backends: "true"
+#        nginx.ingress.kubernetes.io/proxy-body-size: "0"
+#        nginx.ingress.kubernetes.io/proxy-send-timeout: "120"
+#        nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
+#        cert-manager.io/cluster-issuer: example.issuer.name
+#    hosts:
+#        - host: cvat.example.com
+#          paths:
+#              - path: "/api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.*"
+#                service:
+#                    name: cvt-test-backend-service
+#                    port: 8080
+#              - path : "/"
+#                pathType: "Prefix"
+#                service:
+#                    name: cvt-test-frontend-service
+#                    port: 80
+#
+#    tls:
+#        - hosts:
+#              - cvat.example.com
+#          secretName: ingress-tls-cvat