Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
milvus
milvus
提交
9f5a0d60
M
milvus
项目概览
milvus
/
milvus
12 个月 前同步成功
通知
261
Star
22476
Fork
2472
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
milvus
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
9f5a0d60
编写于
4月 18, 2022
作者:
J
john-h-luo
提交者:
GitHub
4月 18, 2022
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
correct typo modify comment (#16492)
Signed-off-by:
N
John Luo
<
jiehua.luo@zilliz.com
>
上级
fce8f6cf
变更
18
隐藏空白更改
内联
并排
Showing
18 changed file
with
299 addition
and
0 deletion
+299
-0
deployments/docker/cluster-distributed-deployment/ansible.cfg
...oyments/docker/cluster-distributed-deployment/ansible.cfg
+4
-0
deployments/docker/cluster-distributed-deployment/deploy-docker.yml
...s/docker/cluster-distributed-deployment/deploy-docker.yml
+16
-0
deployments/docker/cluster-distributed-deployment/deploy-milvus.yml
...s/docker/cluster-distributed-deployment/deploy-milvus.yml
+37
-0
deployments/docker/cluster-distributed-deployment/inventory.ini
...ments/docker/cluster-distributed-deployment/inventory.ini
+53
-0
deployments/docker/cluster-distributed-deployment/roles/configure-hosts-file/tasks/main.yml
...uted-deployment/roles/configure-hosts-file/tasks/main.yml
+5
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-datacoord/tasks/main.yml
...tributed-deployment/roles/deploy-datacoord/tasks/main.yml
+12
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-datanode/tasks/main.yml
...stributed-deployment/roles/deploy-datanode/tasks/main.yml
+11
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-etcd/tasks/main.yml
...r-distributed-deployment/roles/deploy-etcd/tasks/main.yml
+17
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-indexcoord/tasks/main.yml
...ributed-deployment/roles/deploy-indexcoord/tasks/main.yml
+12
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-indexnode/tasks/main.yml
...tributed-deployment/roles/deploy-indexnode/tasks/main.yml
+12
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-minio/tasks/main.yml
...-distributed-deployment/roles/deploy-minio/tasks/main.yml
+16
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-proxy/tasks/main.yml
...-distributed-deployment/roles/deploy-proxy/tasks/main.yml
+11
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-pulsar/tasks/main.yml
...distributed-deployment/roles/deploy-pulsar/tasks/main.yml
+15
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-querycoord/tasks/main.yml
...ributed-deployment/roles/deploy-querycoord/tasks/main.yml
+12
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-querynode/tasks/main.yml
...tributed-deployment/roles/deploy-querynode/tasks/main.yml
+11
-0
deployments/docker/cluster-distributed-deployment/roles/deploy-rootcoord/tasks/main.yml
...tributed-deployment/roles/deploy-rootcoord/tasks/main.yml
+12
-0
deployments/docker/cluster-distributed-deployment/roles/docker-installation/tasks/main.yml
...buted-deployment/roles/docker-installation/tasks/main.yml
+30
-0
deployments/docker/cluster-distributed-deployment/roles/install-modules/tasks/main.yml
...stributed-deployment/roles/install-modules/tasks/main.yml
+13
-0
未找到文件。
deployments/docker/cluster-distributed-deployment/ansible.cfg
0 → 100644
浏览文件 @
9f5a0d60
[defaults]
host_key_checking = False
inventory = inventory.ini
private_key_file=~/.my_ssh_keys/gpc_sshkey
deployments/docker/cluster-distributed-deployment/deploy-docker.yml
0 → 100644
浏览文件 @
9f5a0d60
---
-
name
:
setup pre-requisites
#Install prerequisite
hosts
:
all
become
:
yes
become_user
:
root
roles
:
-
install-modules
-
configure-hosts-file
-
name
:
install docker
become
:
yes
become_user
:
root
hosts
:
dockernodes
roles
:
-
docker-installation
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/deploy-milvus.yml
0 → 100644
浏览文件 @
9f5a0d60
---
# Milvus dependent, before running this playbook, modify the Inventory to make sure
# correct IP address and Host name are in place.
-
name
:
Create milvus-etcd, minio, pulsar
hosts
:
dependencies
become
:
yes
become_user
:
root
tags
:
docker
roles
:
-
deploy-etcd
-
deploy-minio
-
deploy-pulsar
# Deploy all 3 nodes on the same host can be benefitial from I/O optimized disk, and it's easier
# to have additional host running a set of milvus nodes.
-
name
:
Create milvus nodes
hosts
:
nodes
become
:
yes
become_user
:
root
tags
:
docker
roles
:
-
deploy-datanode
-
deploy-indexnode
-
deploy-querynode
#Deploy all Coordinators and Proxy on the same host will reduce time to traffic on network.
-
name
:
Create milvus coords
hosts
:
coords
become
:
yes
become_user
:
root
tags
:
docker
roles
:
-
deploy-rootcoord
-
deploy-indexcoord
-
deploy-datacoord
-
deploy-querycoord
-
deploy-proxy
deployments/docker/cluster-distributed-deployment/inventory.ini
0 → 100644
浏览文件 @
9f5a0d60
[dockernodes]
dockernode01
dockernode02
dockernode03
[admin]
ansible-controller
[coords]
; Take note the IP of this host VM, and replace 10.170.0.17 with it.
dockernode01
[nodes]
dockernode02
[dependencies]
; dependencies node will host etcd, minio, pulsar, these 3 roles are the foundation of Milvus.
; Take note the IP of this host VM, and replace 10.170.0.19 with it.
dockernode03
[docker:children]
dockernodes
coords
nodes
dependencies
[docker:vars]
ansible_python_interpreter
=
/usr/bin/python3
StrictHostKeyChecking
=
no
; Setup variables to controll what type of network to use when creating containers.
dependencies_network
=
host
nodes_network
=
host
; Setup varibale to controll what version of Milvus image to use.
image
=
milvusdb/milvus-dev:master-20220412-4781db8a
; Setup static IP addresses of the docker hosts as variable for container environment variable config.
; Before running the playbook, below 4 IP addresses need to be replaced with the IP of your host VM
; on which the etcd, minio, pulsar, coordinators will be hosted.
etcd_ip
=
10.170.0.19
minio_ip
=
10.170.0.19
pulsar_ip
=
10.170.0.19
coords_ip
=
10.170.0.17
; Setup container environment which later will be used in container creation.
ETCD_ENDPOINTS
=
{{etcd_ip}}:2379
MINIO_ADDRESS
=
{{minio_ip}}:9000
PULSAR_ADDRESS
=
pulsar://{{pulsar_ip}}:6650
QUERY_COORD_ADDRESS
=
{{coords_ip}}:19531
DATA_COORD_ADDRESS
=
{{coords_ip}}:13333
ROOT_COORD_ADDRESS
=
{{coords_ip}}:53100
INDEX_COORD_ADDRESS
=
{{coords_ip}}:31000
deployments/docker/cluster-distributed-deployment/roles/configure-hosts-file/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
---
-
name
:
Configure Hosts File
lineinfile
:
path=/etc/hosts regexp='.*{{ item }}$' line="{{ hostvars[item].ansible_default_ipv4.address }} {{item}}" state=present
when
:
hostvars[item].ansible_default_ipv4.address is defined
with_items
:
"
{{
groups['docker']
}}"
deployments/docker/cluster-distributed-deployment/roles/deploy-datacoord/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
datacoord
docker_container
:
name
:
datacoord
image
:
"
{{image}}"
command
:
[
"
milvus"
,
"
run"
,
"
datacoord"
]
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
DATA_COORD_ADDRESS
:
"
{{DATA_COORD_ADDRESS}}"
METRICS_PORT
:
"
9095"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-datanode/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
datanode
docker_container
:
name
:
datanode
image
:
"
{{image}}"
command
:
"
milvus
run
datanode"
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
METRICS_PORT
:
"
9092"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-etcd/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
etcd
docker_container
:
name
:
etcd
image
:
quay.io/coreos/etcd:v3.5.0
command
:
"
etcd
-listen-peer-urls=http://{{etcd_ip}}:2380
-advertise-client-urls=http://{{etcd_ip}}:2379
-listen-client-urls
http://0.0.0.0:2379,http://0.0.0.0:4001
-initial-advertise-peer-urls=http://{{etcd_ip}}:2380
--listen-metrics-urls=http://{{etcd_ip}}:2381
--initial-cluster
default=http://{{etcd_ip}}:2380
--data-dir
/etcd"
healthcheck
:
test
:
[
"
CMD"
,
"
wget"
,
"
-q"
,
"
--tries=1"
,
"
--spider"
,
"
http://etcd_ip:/health"
]
interval
:
30s
timeout
:
20s
retries
:
3
env
:
ETCD_AUTO_COMPACTION_MODE
:
revision
ETCD_AUTO_COMPACTION_RETENTION
:
"
1000"
ETCD_QUOTA_BACKEND_BYTES
:
"
4294967296"
# volumes:
# - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
network_mode
:
"
{{dependencies_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-indexcoord/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
indexcoord
docker_container
:
name
:
indexcoord
image
:
"
{{image}}"
command
:
[
"
milvus"
,
"
run"
,
"
indexcoord"
]
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
INDEX_COORD_ADDRESS
:
"
{{INDEX_COORD_ADDRESS}}"
METRICS_PORT
:
"
9097"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-indexnode/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
indexnode
docker_container
:
name
:
indexnode
image
:
"
{{image}}"
command
:
"
milvus
run
indexnode"
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
INDEX_COORD_ADDRESS
:
"
{{INDEX_COORD_ADDRESS}}"
METRICS_PORT
:
"
9093"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-minio/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
"
minio"
docker_container
:
name
:
minio
image
:
minio/minio:RELEASE.2022-03-17T06-34-49Z
env
:
MINIO_ACCESS_KEY
:
minioadmin
MINIO_SECRET_KEY
:
minioadmin
command
:
"
minio
server
/minio_data"
# volumes:
# - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
healthcheck
:
test
:
[
"
CMD"
,
"
curl"
,
"
-f"
,
"
http://127.0.0.1:9000/minio/health/live"
]
interval
:
30s
timeout
:
20s
retries
:
3
network_mode
:
"
{{dependencies_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-proxy/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
"
proxy"
docker_container
:
name
:
proxy
image
:
"
{{image}}"
command
:
"
milvus
run
proxy"
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
METRICS_PORT
:
"
9098"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
deployments/docker/cluster-distributed-deployment/roles/deploy-pulsar/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
"
pulsar"
docker_container
:
name
:
pulsar
image
:
apachepulsar/pulsar:2.8.2
env
:
# bin/apply-config-from-env.py script will modify the configuration file based on the environment variables
# nettyMaxFrameSizeBytes must be calculated from maxMessageSize + 10240 (padding)
nettyMaxFrameSizeBytes
:
"
104867840"
# this is 104857600 + 10240 (padding)
defaultRetentionTimeInMinutes
:
"
10080"
defaultRetentionSizeInMB
:
"
8192"
# maxMessageSize is missing from standalone.conf, must use PULSAR_PREFIX_ to get it configured
PULSAR_PREFIX_maxMessageSize
:
"
104857600"
PULSAR_GC
:
-XX:+UseG1GC
command
:
"
bin/pulsar
standalone
--no-functions-worker
--no-stream-storage"
#/bin/bash -c bin/apply-config-from-env.py conf/standalone.conf &&
network_mode
:
"
{{dependencies_network}}"
#Use the network which defined as variable in Inventory.
deployments/docker/cluster-distributed-deployment/roles/deploy-querycoord/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
querycoord
docker_container
:
name
:
querycoord
image
:
"
{{image}}"
command
:
"
milvus
run
querycoord"
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
QUERY_COORD_ADDRESS
:
"
{{QUERY_COORD_ADDRESS}}"
METRICS_PORT
:
"
9096"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-querynode/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
querynode
docker_container
:
name
:
querynode
image
:
"
{{image}}"
command
:
"
milvus
run
querynode"
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
METRICS_PORT
:
"
9091"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
\ No newline at end of file
deployments/docker/cluster-distributed-deployment/roles/deploy-rootcoord/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
-
name
:
rootcoord
docker_container
:
name
:
rootcoord
image
:
"
{{image}}"
command
:
[
"
milvus"
,
"
run"
,
"
rootcoord"
]
env
:
ETCD_ENDPOINTS
:
"
{{ETCD_ENDPOINTS}}"
MINIO_ADDRESS
:
"
{{MINIO_ADDRESS}}"
PULSAR_ADDRESS
:
"
{{PULSAR_ADDRESS}}"
QUERY_COORD_ADDRESS
:
"
{{QUERY_COORD_ADDRESS}}"
METRICS_PORT
:
"
9094"
network_mode
:
"
{{nodes_network}}"
#Use the network which defined as variable in Inventory.
deployments/docker/cluster-distributed-deployment/roles/docker-installation/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
---
-
name
:
Install Docker Dependencies
apt
:
name={{ item }} state=latest update_cache=yes
with_items
:
-
apt-transport-https
-
ca-certificates
-
software-properties-common
tags
:
docker
-
name
:
Get Docker key
apt_key
:
url
:
https://download.docker.com/linux/ubuntu/gpg
state
:
present
tags
:
docker
-
name
:
Add Docker packages to Repository
apt_repository
:
repo
:
deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_lsb.codename|lower }} stable
tags
:
docker
-
name
:
Install Docker-CE
apt
:
name="docker-ce" state=latest update_cache=yes
tags
:
docker
-
name
:
Install python3-docker
apt
:
name="python3-docker" state=latest update_cache=yes
tags
:
docker
-
name
:
Install docker-compose python3 library
command
:
"
pip3
install
docker-compose"
deployments/docker/cluster-distributed-deployment/roles/install-modules/tasks/main.yml
0 → 100644
浏览文件 @
9f5a0d60
---
-
name
:
Install Packages
apt
:
name={{ item }} state=latest update_cache=yes
with_items
:
-
ntp
-
python3
-
tcpdump
-
wget
-
openssl
-
curl
-
python3-pip
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录