Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
DolphinScheduler
提交
f166de17
DolphinScheduler
项目概览
apache
/
DolphinScheduler
上一次同步 接近 2 年
通知
707
Star
9572
Fork
3514
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
f166de17
编写于
2月 21, 2021
作者:
S
Shiwen Cheng
提交者:
GitHub
2月 21, 2021
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Improvement][K8s] Support resource storage when fs.defaultFS=
file:///
(#4777)
上级
4fd3cf9e
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
68 addition
and
14 deletion
+68
-14
docker/kubernetes/dolphinscheduler/templates/_helpers.tpl
docker/kubernetes/dolphinscheduler/templates/_helpers.tpl
+0
-9
docker/kubernetes/dolphinscheduler/templates/configmap-dolphinscheduler-common.yaml
...cheduler/templates/configmap-dolphinscheduler-common.yaml
+1
-1
docker/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
...nscheduler/templates/deployment-dolphinscheduler-api.yaml
+10
-1
docker/kubernetes/dolphinscheduler/templates/pvc-dolphinscheduler-fs-file.yaml
...phinscheduler/templates/pvc-dolphinscheduler-fs-file.yaml
+37
-0
docker/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-master.yaml
...eduler/templates/statefulset-dolphinscheduler-master.yaml
+1
-1
docker/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-worker.yaml
...eduler/templates/statefulset-dolphinscheduler-worker.yaml
+11
-2
docker/kubernetes/dolphinscheduler/values.yaml
docker/kubernetes/dolphinscheduler/values.yaml
+8
-0
未找到文件。
docker/kubernetes/dolphinscheduler/templates/_helpers.tpl
浏览文件 @
f166de17
...
...
@@ -129,13 +129,4 @@ We truncate at 63 chars because some Kubernetes name fields are limited to this
{{- define "dolphinscheduler.zookeeper.quorum" -}}
{{- $port := default "2181" (.Values.zookeeper.service.port | toString) -}}
{{- printf "%s:%s" (include "dolphinscheduler.zookeeper.fullname" .) $port | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{
{
/*
Create
a
default
dolphinscheduler
worker
base
dir
.
We
truncate
at
63
chars
because
some
Kubernetes
name
fields
are
limited
to
this
(
by
the
DNS
naming
spec
).
*/
}
}
{{- define "dolphinscheduler.data.basedir.path" -}}
{{- $name := default "/tmp/dolphinscheduler" .Values.common.configmap.DOLPHINSCHEDULER_DATA_BASEDIR_PATH -}}
{{- printf "%s" $name | trunc 63 | trimSuffix "/" -}}
{{- end -}}
\ No newline at end of file
docker/kubernetes/dolphinscheduler/templates/configmap-dolphinscheduler-common.yaml
浏览文件 @
f166de17
...
...
@@ -28,7 +28,7 @@ data:
{{- range .Values.common.configmap.DOLPHINSCHEDULER_ENV }}
{{ . }}
{{- end }}
DOLPHINSCHEDULER_DATA_BASEDIR_PATH: {{
include "dolphinscheduler.data.basedir.path" .
| quote }}
DOLPHINSCHEDULER_DATA_BASEDIR_PATH: {{
.Values.common.configmap.DOLPHINSCHEDULER_DATA_BASEDIR_PATH
| quote }}
RESOURCE_STORAGE_TYPE: {{ .Values.common.configmap.RESOURCE_STORAGE_TYPE | quote }}
RESOURCE_UPLOAD_PATH: {{ .Values.common.configmap.RESOURCE_UPLOAD_PATH | quote }}
FS_DEFAULT_FS: {{ .Values.common.configmap.FS_DEFAULT_FS | quote }}
...
...
docker/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
浏览文件 @
f166de17
...
...
@@ -206,6 +206,10 @@ spec:
volumeMounts
:
-
mountPath
:
"
/opt/dolphinscheduler/logs"
name
:
{{
include "dolphinscheduler.fullname" .
}}
-api
{{
- if .Values.common.fsFileResourcePersistence.enabled
}}
-
mountPath
:
{{
.Values.common.configmap.RESOURCE_UPLOAD_PATH | quote
}}
name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
{{
- end
}}
volumes
:
-
name
:
{{
include "dolphinscheduler.fullname" .
}}
-api
{{
- if .Values.api.persistentVolumeClaim.enabled
}}
...
...
@@ -213,4 +217,9 @@ spec:
claimName
:
{{
include "dolphinscheduler.fullname" .
}}
-api
{{
- else
}}
emptyDir
:
{}
{{
- end
}}
\ No newline at end of file
{{
- end
}}
{{
- if .Values.common.fsFileResourcePersistence.enabled
}}
-
name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
persistentVolumeClaim
:
claimName
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
{{
- end
}}
\ No newline at end of file
docker/kubernetes/dolphinscheduler/templates/pvc-dolphinscheduler-fs-file.yaml
0 → 100644
浏览文件 @
f166de17
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
{{
- if .Values.common.fsFileResourcePersistence.enabled
}}
apiVersion
:
v1
kind
:
PersistentVolumeClaim
metadata
:
name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
labels
:
app.kubernetes.io/name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
app.kubernetes.io/instance
:
{{
.Release.Name
}}
app.kubernetes.io/managed-by
:
{{
.Release.Service
}}
annotations
:
"
helm.sh/resource-policy"
:
keep
spec
:
accessModes
:
{{
- range .Values.common.fsFileResourcePersistence.accessModes
}}
-
{{
. | quote
}}
{{
- end
}}
storageClassName
:
{{
.Values.common.fsFileResourcePersistence.storageClassName | quote
}}
resources
:
requests
:
storage
:
{{
.Values.common.fsFileResourcePersistence.storage | quote
}}
{{
- end
}}
\ No newline at end of file
docker/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-master.yaml
浏览文件 @
f166de17
...
...
@@ -247,4 +247,4 @@ spec:
resources
:
requests
:
storage
:
{{
.Values.master.persistentVolumeClaim.storage | quote
}}
{{
- end
}}
{{
- end
}}
\ No newline at end of file
docker/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-worker.yaml
浏览文件 @
f166de17
...
...
@@ -324,13 +324,17 @@ spec:
failureThreshold
:
{{
.Values.worker.readinessProbe.failureThreshold
}}
{{
- end
}}
volumeMounts
:
-
mountPath
:
{{
include "dolphinscheduler.data.basedir.path" .
| quote
}}
-
mountPath
:
{{
.Values.common.configmap.DOLPHINSCHEDULER_DATA_BASEDIR_PATH
| quote
}}
name
:
{{
include "dolphinscheduler.fullname" .
}}
-worker-data
-
mountPath
:
"
/opt/dolphinscheduler/logs"
name
:
{{
include "dolphinscheduler.fullname" .
}}
-worker-logs
-
mountPath
:
"
/opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh"
subPath
:
"
dolphinscheduler_env.sh"
name
:
{{
include "dolphinscheduler.fullname" .
}}
-common-env
{{
- if .Values.common.fsFileResourcePersistence.enabled
}}
-
mountPath
:
{{
.Values.common.configmap.RESOURCE_UPLOAD_PATH | quote
}}
name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
{{
- end
}}
volumes
:
-
name
:
{{
include "dolphinscheduler.fullname" .
}}
-worker-data
{{
- if .Values.worker.persistentVolumeClaim.dataPersistentVolume.enabled
}}
...
...
@@ -353,6 +357,11 @@ spec:
items
:
-
key
:
DOLPHINSCHEDULER_ENV
path
:
dolphinscheduler_env.sh
{{
- if .Values.common.fsFileResourcePersistence.enabled
}}
-
name
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
persistentVolumeClaim
:
claimName
:
{{
include "dolphinscheduler.fullname" .
}}
-fs-file
{{
- end
}}
{{
- if .Values.worker.persistentVolumeClaim.enabled
}}
volumeClaimTemplates
:
{{
- if .Values.worker.persistentVolumeClaim.dataPersistentVolume.enabled
}}
...
...
@@ -389,4 +398,4 @@ spec:
requests
:
storage
:
{{
.Values.worker.persistentVolumeClaim.logsPersistentVolume.storage | quote
}}
{{
- end
}}
{{
- end
}}
{{
- end
}}
\ No newline at end of file
docker/kubernetes/dolphinscheduler/values.yaml
浏览文件 @
f166de17
...
...
@@ -94,6 +94,14 @@ common:
FS_S3A_ENDPOINT
:
"
s3.xxx.amazonaws.com"
FS_S3A_ACCESS_KEY
:
"
xxxxxxx"
FS_S3A_SECRET_KEY
:
"
xxxxxxx"
## If RESOURCE_STORAGE_TYPE is HDFS and FS_DEFAULT_FS is file:///, fsFileResourcePersistence should be enabled for resource storage
fsFileResourcePersistence
:
enabled
:
false
accessModes
:
-
"
ReadWriteMany"
## storageClassName must support the access mode: ReadWriteMany
storageClassName
:
"
-"
storage
:
"
20Gi"
master
:
## PodManagementPolicy controls how pods are created during initial scale up, when replacing pods on nodes, or when scaling down.
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录