提交 bc80ef54 编写于 作者: B bao liang 提交者: easyscheduler

refactor zkClient; update documents (#683)

* update english documents

* refactor zk client

* update documents

* update zkclient
上级 7e8b5c69
......@@ -42,20 +42,13 @@ escheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
#### Preparations 3: SSH Secret-Free Configuration
Configure SSH secret-free login on deployment machines and other installation machines. If you want to install easyscheduler on deployment machines, you need to configure native password-free login itself.
- [Connect the host and other machines SSH](http://geek.analysys.cn/topic/113)
#### Preparations 4: database initialization
* Create databases and accounts
Enter the mysql command line service by following MySQL commands:
> mysql -h {host} -u {user} -p{password}
Then execute the following command to create database and account
Execute the following command to create database and account
```sql
CREATE DATABASE escheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
......@@ -64,17 +57,8 @@ Configure SSH secret-free login on deployment machines and other installation ma
flush privileges;
```
* Versions 1.0.0 and 1.0.1 create tables and import basic data
Instructions:在escheduler-backend/sql/escheduler.sql和quartz.sql
```sql
mysql -h {host} -u {user} -p{password} -D {db} < escheduler.sql
mysql -h {host} -u {user} -p{password} -D {db} < quartz.sql
```
* Version 1.0.2 later (including 1.0.2) creates tables and imports basic data
Modify the following attributes in conf/dao/data_source.properties
* creates tables and imports basic data
Modify the following attributes in ./conf/dao/data_source.properties
```
spring.datasource.url
......@@ -88,7 +72,7 @@ Configure SSH secret-free login on deployment machines and other installation ma
#### Preparations 5: Modify the deployment directory permissions and operation parameters
Let's first get a general idea of the role of files (folders) in the escheduler-backend directory after decompression.
instruction of escheduler-backend directory
```directory
bin : Basic service startup script
......@@ -99,7 +83,7 @@ sql : The project relies on SQL files
install.sh : One-click deployment script
```
- Modify permissions (please modify the deployUser to the corresponding deployment user) so that the deployment user has operational privileges on the escheduler-backend directory
- Modify permissions (please modify the 'deployUser' to the corresponding deployment user) so that the deployment user has operational privileges on the escheduler-backend directory
`sudo chown -R deployUser:deployUser escheduler-backend`
......@@ -109,9 +93,9 @@ install.sh : One-click deployment script
- Modify the parameters in **install.sh** to replace the values required by your business
- MonitorServerState switch variable, added in version 1.0.3, controls whether to start the self-start script (monitor master, worker status, if off-line will start automatically). The default value of "false" means that the self-start script is not started, and if it needs to start, it is changed to "true".
- hdfsStartupSate switch variable controls whether to starthdfs
- 'hdfsStartupSate' switch variable controls whether to start hdfs
The default value of "false" means not to start hdfs
If you need to start hdfs instead of "true", you need to create the hdfs root path by yourself, that is, hdfsPath in install.sh.
Change the variable to 'true' if you want to use hdfs, you also need to create the hdfs root path by yourself, that 'hdfsPath' in install.sh.
- If you use hdfs-related functions, you need to copy**hdfs-site.xml** and **core-site.xml** to the conf directory
......@@ -129,7 +113,7 @@ Automated deployment is recommended, and experienced partners can use source dep
`sh install.sh`
- Use the jps command to see if the service is started (jps comes with Java JDK)
- Use the `jps` command to check if the services are started (`jps` comes from `Java JDK`)
```aidl
MasterServer ----- Master Service
......@@ -138,7 +122,8 @@ Automated deployment is recommended, and experienced partners can use source dep
ApiApplicationServer ----- API Service
AlertServer ----- Alert Service
```
If there are more than five services, the automatic deployment is successful
If all services are normal, the automatic deployment is successful
After successful deployment, the log can be viewed and stored in a specified folder.
......@@ -164,49 +149,46 @@ After downloading the release version of the source package, unzip it into the r
* View directory
After normal compilation, target/escheduler-{version}/ is generated in the current directory
After normal compilation, ./target/escheduler-{version}/ is generated in the current directory
### 2.3 Start-and-stop services commonly used in systems (for service purposes, please refer to System Architecture Design for details)
* stop all services in the cluster at one click
* stop all services in the cluster
` sh ./bin/stop_all.sh`
* one click to open all services in the cluster
* start all services in the cluster
` sh ./bin/start_all.sh`
* start and stop Master
* start and stop one master server
```start master
sh ./bin/escheduler-daemon.sh start master-server
sh ./bin/escheduler-daemon.sh stop master-server
```
* start and stop Worker
* start and stop one worker server
```start worker
sh ./bin/escheduler-daemon.sh start worker-server
sh ./bin/escheduler-daemon.sh stop worker-server
```
* start and stop Api
* start and stop api server
```start Api
sh ./bin/escheduler-daemon.sh start api-server
sh ./bin/escheduler-daemon.sh stop api-server
```
* start and stop Logger
* start and stop logger server
```start Logger
sh ./bin/escheduler-daemon.sh start logger-server
sh ./bin/escheduler-daemon.sh stop logger-server
```
* start and stop Alert
* start and stop alert server
```start Alert
sh ./bin/escheduler-daemon.sh start alert-server
......@@ -214,7 +196,7 @@ sh ./bin/escheduler-daemon.sh stop alert-server
```
## 3、Database Upgrade
Database upgrade is a function added in version 1.0.2. The database can be upgraded automatically by executing the following commands
Database upgrade is a function added in version 1.0.2. The database can be upgraded automatically by executing the following command:
```upgrade
sh ./script/upgrade_escheduler.sh
......
......@@ -8,7 +8,7 @@
* [Maven](http://maven.apache.org/download.cgi)(3.3+) :Must be installed
Because the escheduler-rpc module in EasyScheduler uses Grpc, you need to use Maven to compile the generated classes.
For those who are not familiar with Maven, please refer to: [maven in five minutes](http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)(3.3+)
For those who are not familiar with maven, please refer to: [maven in five minutes](http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)(3.3+)
http://maven.apache.org/install.html
......@@ -23,7 +23,7 @@ After importing the EasyScheduler source code into the development tools such as
* View directory
After normal compilation, it will generate target/escheduler-{version}/ in the current directory.
After normal compilation, it will generate ./target/escheduler-{version}/ in the current directory.
```
bin
......
......@@ -31,7 +31,7 @@ esc_proxy="8888"
esc_proxy_port="http://192.168.xx.xx:12345"
```
>Front-end automatic deployment based on Linux system`yum`operation, before deployment, please install and update`yum`
>Front-end automatic deployment based on Linux system `yum` operation, before deployment, please install and update`yum`
under this directory, execute`./install-escheduler-ui.sh`
......
......@@ -7,9 +7,6 @@
## Operational Guidelines
- Administrator accounts can only be managed in terms of authority, do not participate in specific business, can not create projects, and can not perform related operations on process definition.
- The following operations can only be performed by using ordinary user login system.
### Create a project
- Click "Project - > Create Project", enter project name, description, and click "Submit" to create a new project.
......@@ -18,7 +15,7 @@
<img src="https://user-images.githubusercontent.com/53217792/61776719-2ee50380-ae2e-11e9-9d11-41de8907efb5.png" width="60%" />
</p>
> Project Home Page contains task status statistics, process status statistics, process definition statistics, queue statistics, command statistics.
> Project Home Page contains task status statistics, process status statistics.
- Task State Statistics: It refers to the statistics of the number of tasks to be run, failed, running, completed and succeeded in a given time frame.
- Process State Statistics: It refers to the statistics of the number of waiting, failing, running, completing and succeeding process instances in a specified time range.
......
# EasyScheduler upgrade documentation
## 1. Back up the previous version of the file and database
## 1. Back up the previous version of the files and database
## 2. Stop all services of escheduler
......@@ -9,7 +9,7 @@
## 3. Download the new version of the installation package
- [gitee](https://gitee.com/easyscheduler/EasyScheduler/attach_files), download the latest version of the front and rear installation package (backend referred to as escheduler-backend, front end referred to as escheduler-ui)
- [gitee](https://gitee.com/easyscheduler/EasyScheduler/attach_files), download the latest version of the front and back installation packages (backend referred to as escheduler-backend, front end referred to as escheduler-ui)
- The following upgrade operations need to be performed in the new version of the directory
## 4. Database upgrade
......@@ -27,11 +27,12 @@
## 5. Backend service upgrade
- Modify the contents of the install.sh configuration and execute the upgrade script
- Modify the content of the install.sh configuration and execute the upgrade script
`sh install.sh`
## 6. Frontend service upgrade
- Overwrite the previous version of the dist directory
- Restart the nginx service
......
......@@ -106,7 +106,7 @@ npm install node-sass --unsafe-perm //单独安装node-sass依赖
`src => combo` 项目第三方资源本地化 `npm run combo`具体查看`build/combo.js`
`src => font` 字体图标库可访问 https://www.iconfont.cn 进行添加 注意:字体库用的自己的 二次开发需要重新引入自己的库 `src/sass/common/_font.scss`
`src => font` 字体图标库可访问 `https://www.iconfont.cn` 进行添加 注意:字体库用的自己的 二次开发需要重新引入自己的库 `src/sass/common/_font.scss`
`src => images` 公共图片存放
......
......@@ -8,12 +8,12 @@
* [Maven](http://maven.apache.org/download.cgi)(3.3+) :必装
因EasyScheduler中escheduler-rpc模块使用到Grpc,需要用到Maven编译生成所需要的类
Maven不熟的伙伴请参考: [maven in five minutes](http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)(3.3+)
maven不熟的伙伴请参考: [maven in five minutes](http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)(3.3+)
http://maven.apache.org/install.html
## 项目编译
将EasyScheduler源码下载导入Idea开发工具后,首先转为Maven项目(右键点击后选择"Add Framework Support")
将EasyScheduler源码下载导入Idea开发工具后,首先转为Maven项目(右键点击后选择"Add Framework Support")
* 执行编译命令:
......@@ -23,7 +23,7 @@ http://maven.apache.org/install.html
* 查看目录
正常编译完后,会在当前目录生成 target/escheduler-{version}/
正常编译完后,会在当前目录生成 ./target/escheduler-{version}/
```
bin
......
......@@ -43,12 +43,8 @@ escheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
#### 准备四:数据库初始化
* 创建database和账号
通过以下mysql命令进入mysql命令行服务:
> mysql -h {host} -u {user} -p{password}
然后执行以下命令创建database和账号
执行以下命令创建database和账号
```sql
CREATE DATABASE escheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
......@@ -57,17 +53,8 @@ escheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
flush privileges;
```
* 1.0.0和1.0.1版本创建表和导入基础数据
说明:在escheduler-backend/sql/escheduler.sql和quartz.sql
```sql
mysql -h {host} -u {user} -p{password} -D {db} < escheduler.sql
mysql -h {host} -u {user} -p{password} -D {db} < quartz.sql
```
* 1.0.2之后(含1.0.2)版本创建表和导入基础数据
修改conf/dao/data_source.properties中的下列属性
* 创建表和导入基础数据
修改./conf/dao/data_source.properties中的下列属性
```
spring.datasource.url
......@@ -81,7 +68,7 @@ escheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
#### 准备五: 修改部署目录权限及运行参数
我们先来大体了解下解压后escheduler-backend目录下的文件(夹)的作用
escheduler-backend目录介绍
```
bin : 基础服务启动脚本
......@@ -92,7 +79,7 @@ sql : 项目依赖sql文件
install.sh : 一键部署脚本
```
- 修改权限(请将deployUser自行修改为对应部署用户),使得部署用户对escheduler-backend目录有操作权限
- 修改权限(请将'deployUser'字段修改为对应部署用户),使得部署用户对escheduler-backend目录有操作权限
`sudo chown -R deployUser:deployUser escheduler-backend`
......@@ -124,7 +111,7 @@ install.sh : 一键部署脚本
`sh install.sh`
- 使用jps命令查看服务是否启动(jps为java JDK自带)
- 使用`jps`命令查看服务是否启动(`jps``java JDK`自带)
```aidl
MasterServer ----- master服务
......@@ -133,7 +120,7 @@ install.sh : 一键部署脚本
ApiApplicationServer ----- api服务
AlertServer ----- alert服务
```
如果有以上5个服务,说明自动部署成功
如果以上服务都正常启动,说明自动部署成功
部署成功后,可以进行日志查看,日志统一存放于指定文件夹内
......@@ -159,7 +146,7 @@ install.sh : 一键部署脚本
* 查看目录
正常编译完后,会在当前目录生成 `target/escheduler-{version}/`
正常编译完后,会在当前目录生成 `./target/escheduler-{version}/`
```查看目录
../
......
......@@ -7,9 +7,6 @@
## 操作指南
- 管理员账号只能在权限上进行管理,不参与具体的业务,不能创建项目,不能对流程定义执行相关操作
- 以下操作需要使用普通用户登录系统才能进行。
### 创建项目
- 点击“项目管理->创建项目”,输入项目名称,项目描述,点击“提交”,创建新的项目。
......
......@@ -64,16 +64,6 @@ public abstract class AbstractZKClient {
*/
protected String deadServerZNodeParentPath = null;
/**
* master node parent path
*/
protected String masterZNodeParentPath = null;
/**
* worker node parent path
*/
protected String workerZNodeParentPath = null;
/**
* server stop or not
*/
......@@ -318,18 +308,16 @@ public abstract class AbstractZKClient {
List<String> childrenList = new ArrayList<>();
try {
// read master node parent path from conf
masterZNodeParentPath = conf.getString(Constants.ZOOKEEPER_ESCHEDULER_MASTERS);
if(zkClient.checkExists().forPath(masterZNodeParentPath) != null){
childrenList = zkClient.getChildren().forPath(masterZNodeParentPath);
if(zkClient.checkExists().forPath(getZNodeParentPath(ZKNodeType.MASTER)) != null){
childrenList = zkClient.getChildren().forPath(getZNodeParentPath(ZKNodeType.MASTER));
}
} catch (Exception e) {
if(!e.getMessage().contains("java.lang.IllegalStateException: instance must be started")){
logger.warn(e.getMessage(),e);
}
}finally {
return childrenList.size();
}
return childrenList.size();
}
/**
......@@ -585,9 +573,9 @@ public abstract class AbstractZKClient {
public String toString() {
return "AbstractZKClient{" +
"zkClient=" + zkClient +
", deadServerZNodeParentPath='" + deadServerZNodeParentPath + '\'' +
", masterZNodeParentPath='" + masterZNodeParentPath + '\'' +
", workerZNodeParentPath='" + workerZNodeParentPath + '\'' +
", deadServerZNodeParentPath='" + getZNodeParentPath(ZKNodeType.DEAD_SERVER) + '\'' +
", masterZNodeParentPath='" + getZNodeParentPath(ZKNodeType.MASTER) + '\'' +
", workerZNodeParentPath='" + getZNodeParentPath(ZKNodeType.WORKER) + '\'' +
", stoppable=" + stoppable +
'}';
}
......
......@@ -182,7 +182,8 @@ public class ZKMasterClient extends AbstractZKClient {
* monitor master
*/
public void listenerMaster(){
PathChildrenCache masterPc = new PathChildrenCache(zkClient, masterZNodeParentPath, true ,defaultThreadFactory);
PathChildrenCache masterPc = new PathChildrenCache(zkClient,
getZNodeParentPath(ZKNodeType.MASTER), true ,defaultThreadFactory);
try {
masterPc.start();
......@@ -280,7 +281,8 @@ public class ZKMasterClient extends AbstractZKClient {
*/
public void listenerWorker(){
PathChildrenCache workerPc = new PathChildrenCache(zkClient,workerZNodeParentPath,true ,defaultThreadFactory);
PathChildrenCache workerPc = new PathChildrenCache(zkClient,
getZNodeParentPath(ZKNodeType.WORKER),true ,defaultThreadFactory);
try {
workerPc.start();
workerPc.getListenable().addListener(new PathChildrenCacheListener() {
......
......@@ -117,7 +117,7 @@ public class ZKWorkerClient extends AbstractZKClient {
String heartbeatZKInfo = ResInfo.getHeartBeatInfo(new Date());
workerZNode = workerZNodeParentPath + "/" + OSUtils.getHost() + "_";
workerZNode = getZNodeParentPath(ZKNodeType.WORKER) + "/" + OSUtils.getHost() + "_";
workerZNode = zkClient.create().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(workerZNode,
heartbeatZKInfo.getBytes());
......@@ -144,7 +144,7 @@ public class ZKWorkerClient extends AbstractZKClient {
* monitor worker
*/
private void listenerWorker(){
PathChildrenCache workerPc = new PathChildrenCache(zkClient, workerZNodeParentPath, true, defaultThreadFactory);
PathChildrenCache workerPc = new PathChildrenCache(zkClient, getZNodeParentPath(ZKNodeType.WORKER), true, defaultThreadFactory);
try {
Date now = new Date();
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册