1. 29 7月, 2022 1 次提交
    • J
      [ci] Make cluster test work on release branch (#11199) · 93a73b12
      Jiajie Zhong 提交于
      When I try to release the 3.0.0 on branch `3.0.0-prepare`,
      I find out cluster-test error due to version change of
      tarball, I think we should make cluster test work on both
      dev and prepare release branch, so I add this patch
      93a73b12
  2. 14 5月, 2022 1 次提交
    • X
      [Feature-9474] [CI] Add cluster test script verify on shell script (#9997) · baf654cb
      xiangzihao 提交于
      * cluster test
      
      * fix init db failed
      
      * fix init db failed
      
      * fix init db failed
      
      * fix init db failed
      
      * fix init db failed
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add sudo
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * remove cluster-test workflows
      
      * add github actions
      
      * add github actions
      
      * refactor test to docker mode
      
      * refactor test to docker mode
      
      * refactor test to docker mode
      
      * refactor test to docker mode
      
      * remove create schema logic
      
      * remove create schema logic
      
      * remove create schema logic
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * fix runnint cluster test error
      
      * add github actions
      
      * add github actions
      
      * add github actions
      
      * add cluster test timeout
      
      * add cluster start test loop check
      
      * add cluster start test loop check
      
      * optimize cluster start test loop check
      baf654cb
  3. 07 4月, 2022 1 次提交
    • J
      [python] Migrate pythonGatewayServer into api server (#9372) · 3457cee9
      Jiajie Zhong 提交于
      Currently the size of our distribute package is up to
      800MB, this patch is migrate python gateway server into
      api server
      
      The distribute package size before and after this patch is:
      
      ```sh
      # before
      796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
      
      # after
      647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
      ```
      3457cee9
  4. 07 1月, 2022 1 次提交
  5. 10 12月, 2021 1 次提交
    • K
      Split the components into individual package, refactor configurations to use... · 67cc260d
      kezhenxu94 提交于
      Split the components into individual package, refactor configurations to use Spring properties (#7160)
      
      * Split the components into individual package
      
      A follow-up PR will be made to build dedicated Docker images for each
      component, so that every component Docker image has minimal jars, which
      is easy to maintain and good for security fixes.
      
      * Split the components into individual package
      
      A follow-up PR will be made to build dedicated Docker images for each
      component, so that every component Docker image has minimal jars, which
      is easy to maintain and good for security fixes.
      
      * Split the components into individual package
      
      A follow-up PR will be made to build dedicated Docker images for each
      component, so that every component Docker image has minimal jars, which
      is easy to maintain and good for security fixes.
      67cc260d
  6. 14 11月, 2021 1 次提交
  7. 09 11月, 2021 1 次提交
  8. 07 11月, 2021 1 次提交
  9. 05 11月, 2021 1 次提交
  10. 14 10月, 2021 1 次提交
    • J
      Fix install.sh for DS 2.0 and add comment to install_config.conf (#6506) · 6495a204
      Jiajie Zhong 提交于
      * Fix install.sh for DS 2.0
      
      * Use #* make it work in macos
      
      * Add comment for properties change
      
      * Add JAVA HOME for only one file change to deploy
      
      * Reduce config change by using default value of: plugin registry, alert, task as
      
      * Change javaHome default value
      
      * Delete config param zkQuorum
      
      * Reorder param
      
      * Add comment for install_config.conf
      6495a204
  11. 29 9月, 2021 1 次提交
  12. 23 9月, 2021 1 次提交
  13. 21 9月, 2021 1 次提交
  14. 15 6月, 2021 1 次提交
  15. 02 6月, 2021 1 次提交
  16. 24 5月, 2021 1 次提交
    • S
      [Improvement][Docker/K8s] Support more configurations and service accesses,... · b537f4eb
      Shiwen Cheng 提交于
      [Improvement][Docker/K8s] Support more configurations and service accesses, reduce the duplication configurations, improve image for python and the compatibility of python home (#5393)
      
      * [Improvement][Docker] Support more configurations and improve image for python
      
      * [Improvement][K8s] Support more configurations, more service accesses, and reduce the duplication configurations
      
      * [Improvement][Python] Improve the compatibility of python home
      
      * [Improvement][Install] Fix install config and install.sh
      
      * [Improvement][Install] Fix workflow ut
      
      * [Improvement][Docker] Optimize docker-swarm/check script
      
      * [Improvement][DB] Update default username and password of database
      
      * Update comments in master/worker.properties
      
      * Specify version for all images
      
      * [Improvement][Docker] Optimize PS1 and WORKDIR
      
      * [Improvement][SQL] Reduce heap size to 64m in create-dolphinscheduler.sh and upgrade-dolphinscheduler.sh
      
      * [Fix-5431][K8s] Fix master and worker cannot get the right address with custom DNS
      b537f4eb
  17. 06 2月, 2021 1 次提交
  18. 31 7月, 2020 1 次提交
    • D
      [Feature][server] add embeded zookeeper server in dolphinscheduler-daemon.sh (#3293) · 6d903e26
      dailidong 提交于
      * update README about DolphinScheduler
      
      * Update issue templates
      
      * update
      
      * regularize api pom xml
      update rpc maven compile to 1.8
      
      * regularize api pom xml
      
      * change commons.lang3.StringUtils to common.utils.StringUtils
      
      * update pom.xml
      
      * update
      
      * correct equals method
      
      * jasper-runtime is needed when api server start
      
      * jasper-runtime jar is needed when api server start
      
      * combine logback config of master/worker/alert/api server to one logback.xml
      
      * remove tomcat runtime jar
      
      * add UT
      
      * add license
      
      * remove jasper-runtime jar, not need anymore
      
      * sovle jar conflict, remove servlet-api 2.5 jar
      
      * remove servlet 2.5
      
      * add embedded database h2 and update embedded zookeeper
      
      * delete logs
      
      * add license
      
      * change log level to INFO
      
      * simplify and optimize config
      
      * add development state
      
      * simplify common config
      
      * add worker group config
      
      * add worker group config
      
      * add worker group config
      
      * this command just for getting a quick experience,not recommended for production. this operation will start a standalone zookeeper server
      
      * Merge remote-tracking branch 'upstream/dev' into dev
      
      * add logback for zookeeper
      
      * Update logback-zookeeper.xml
      Co-authored-by: Nlidongdai <escheduler@outlook.com>
      Co-authored-by: Ndailidong <dolphinschedule@gmail.com>
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@outlook.com>
      6d903e26
  19. 30 7月, 2020 1 次提交
  20. 24 7月, 2020 1 次提交
  21. 17 7月, 2020 1 次提交
  22. 30 6月, 2020 3 次提交
    • B
      Revert "branch dev merge from dev-1.3.0 (#3087)" (#3094) · 5bac559d
      bao liang 提交于
      This reverts commit ab6e4310.
      Co-authored-by: Ndailidong <dailidong66@gmail.com>
      5bac559d
    • D
      branch dev merge from dev-1.3.0 (#3087) · ab6e4310
      dailidong 提交于
      * add DolphinScheduler slack channel
      
      * Update README.md
      Co-Authored-By: NJiajie Zhong <zhongjiajie955@hotmail.com>
      
      * modify soft_version 1.2.2 to 1.3.0
      
      * Repair worklist cannot be deleted
      
      * update schema version (#2657)
      
      * change schema version from 1.2.2 to 1.3.0
      
      * change version value of table t_ds_version from 2.0.0 to 1.3.0
      
      * fix bug: sub worker flow in complement data would run start date (#2581) (#2656)
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * Fix conditions node save exception
      
      * add unique key for t_ds_process_definition
      
      * add unique key for t_ds_process_definition
      
      * script variable has "processDefinitionId" is error #2664 (#2666)
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * update filename
      
      * Update dolphinscheduler_ddl.sql
      
      * Delete dolphinscheduler_ddl.sql
      
      * Delete dolphinscheduler_ddl.sql
      
      * add unique key for t_ds_process_definition
      
      add unique key for t_ds_process_definition
      
      * add unique key for t_ds_process_definition
      
      add unique key for t_ds_process_definition
      
      * Modify the alert log file
       
      
      * blank in eamil and left font align #2648 (#2673)
      
      * script variable has "processDefinitionId" is error #2664
      
      * blank in eamil and left font align  #2648
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * fix zookeeper health check and merge volumes into logs volume (#2674)
      
      * fix zookeeper health check and merge volumes into logs volume
      
      1. Fix zookeeper health check
      2. Merge volumes into logs volume
      3. Modify `dockerfile/hooks/build` script
      
      * Modify ci_e2e and docker image registry
      
      * Modify ci_e2e
      
      * Modify ci_e2e
      
      * Modify ci_e2e and move ./dockerfile/hooks/check to ./docker/docker-swarm/
      
      * migrate docker composer and move dockerfile dir to docker/build
      
      * ut docker
      
      * alter dockerfile to docker/build dir
      
      * delete docker-compose old
      
      * delete dockerfile zk
      
      * delete dockerfile zk
      
      * delete dockerfile zk
      
      * [bug fix] some bugs from 1.2.0 to 1.3.0 (#2676)
      
      * fix bug: some bugs about worker group from 1.2.0 to 1.3.0
      
      * fix bug: scheduler from 1.2.0 to 1.3.0 do not work
      
      * fix bug: scheduler from 1.2.0 to 1.3.0 do not work
      
      * fix ut bug
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * add worker group config (#2681)
      
      * update README about DolphinScheduler
      
      * Update issue templates
      
      * update
      
      * regularize api pom xml
      update rpc maven compile to 1.8
      
      * regularize api pom xml
      
      * change commons.lang3.StringUtils to common.utils.StringUtils
      
      * update pom.xml
      
      * update
      
      * correct equals method
      
      * jasper-runtime is needed when api server start
      
      * jasper-runtime jar is needed when api server start
      
      * combine logback config of master/worker/alert/api server to one logback.xml
      
      * remove tomcat runtime jar
      
      * add UT
      
      * add license
      
      * remove jasper-runtime jar, not need anymore
      
      * sovle jar conflict, remove servlet-api 2.5 jar
      
      * remove servlet 2.5
      
      * add embedded database h2 and update embedded zookeeper
      
      * delete logs
      
      * add license
      
      * change log level to INFO
      
      * simplify and optimize config
      
      * add development state
      
      * simplify common config
      
      * add worker group config
      
      * add worker group config
      Co-authored-by: Nlidongdai <escheduler@outlook.com>
      Co-authored-by: Ndailidong <dolphinschedule@gmail.com>
      
      * delete log files while deleting process instances #2463 (#2693)
      
      * script variable has "processDefinitionId" is error #2664
      
      * blank in eamil and left font align  #2648
      
      * delete log files while deleting process instances #2463
      
      * delete log files while deleting process instances #2463
      
      * delete log files while deleting process instances #2463
      
      * delete log files while deleting process instances #2463
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * Update dolphinscheduler_ddl.sql
      
      * Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml (#2692)
      
      * Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml
      
      1. add initdb volume for pg service in docker-compose.yml
      2. modify ci_ut.yml
      
      * add sudo in ci_ut.yml
      
      * Query the metadata of indexes is not correct in postgresql/dolphinscheduler_ddl.sql
      
      1. replace information_schema.STATISTICS to pg_stat_all_indexes
      2. modify ProcessDefinitionMapperTest.java
      
      * Remove inner zk and pg
      
      * add inner pg
      
      * conditions The task node adds a background color to distinguish successful and failed connections
      
      * add LoggerServerTest UT (#2713)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * add UT (#2717)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * fix worker group config no effect (#2719)
      
      * Conditional judgment adds text
      
      * fix
      
      * fix #2449:It's should successful to cancel authorize a resource of a user if all the online process definitions created by this user didn't use this resource. (#2725)
      
      * Underline the list of workflow operations
      
      * Modify HDFS/S3 path (#2736)
      
      * Modify HDFS/S3 path
       
      
      * Modify HDFS/S3 path
       
      
      * Update common.properties
      Co-authored-by: weixin_41213428's avatarchenxingchun <438044805@qq.com>
      Co-authored-by: Ndailidong <dailidong66@gmail.com>
      
      * Modify ip configuration
      
      * set worker group  (#2743)
      
      * simple config
      
      * fix worker group config no effect
      
      * fix #2598::allow to update resource suffix,but if it is authorized to other users,it is not allowed (#2732)
      
      * fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.
      
      * add RESOURCE_IS_AUTHORIZED status
      
      * verify whether the suffix is empty
      
      * remove extra variables
      
      * fix code smell
      
      * Update known-dependencies.txt
      
      * update worker group config (#2746)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * Update datasource.properties
      
      * remove codehaus janino jar (#2742)
      
      the license about janino maybe not compatiable with Apache v2
      
      * [bug fix] fix bug:2634:Process shows success,when the task of the process is killed in the terminal (#2731)
      
      * fix bug: some bugs about worker group from 1.2.0 to 1.3.0
      
      * fix bug: scheduler from 1.2.0 to 1.3.0 do not work
      
      * fix bug: scheduler from 1.2.0 to 1.3.0 do not work
      
      * fix ut bug
      
      * fix bug: Process shows success,when the task of the process is killed in the termina
      
      * fix bug #2634: Process shows success,when the task of the process is killed in the terminal
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * Update known-dependencies.txt (#2748)
      
      * fix getWorkers bug on RunConfig (#2747)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * master select worker filter high load worker #2704 (#2733)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * Fix rename check parameter
      
      * fix
      
      * fix 2598:change getId to getUserId (#2757)
      
      * fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.
      
      * add RESOURCE_IS_AUTHORIZED status
      
      * verify whether the suffix is empty
      
      * remove extra variables
      
      * fix code smell
      
      * change getId to getUserId
      
      * Fix MySQL autoDeserialize bug (#2728)
      
      * Fix MySQL autoDeserialize bug
      
      * add MySQLDataSourceTest
      
      * Fix workflow definition history node data cannot be opened
      
      * fix
      
      * delete normalize
      
      * update e2e chrome version (#2775)
      Co-authored-by: weixin_41213428's avatarchenxingchun <438044805@qq.com>
      
      * Modify the background color of conditions task node connection
      
      * [BUF FIX]fix: the master would be blocked when worker group not exists (#2764)
      
      * feature: add number configuration for master dispatch tasks
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix ut
      
      * fix ut
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * [BUG FIX] fix bug: dependent task failed when conditions task exists (#2768)
      
      * fix bug 2464: change dependent task for process.
      
      * remove unused code
      
      * add ut
      
      * add ut
      
      * update comments
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * Hourly dependency configuration, 'currentHour' and 'last24Hours' (#2696)
      
      * [feature] Add hourly dependency scenarios
      
      * [Feature] #2680
      
      add DependentUtilsTest UT
      Co-authored-by: Nchangqun.xu <changqun.xu@17zuoye.com>
      Co-authored-by: Nxuchangqun <changqun.x@gmail.com>
      
      * fix sql task bug #2739 (#2741)
      
      * Solve the failure to execute non-query sql
      
      * fix sql task bug
      Co-authored-by: Nsunchaohe <sunzhaohe@linklogis.com>
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@outlook.com>
      Co-authored-by: Nlgcareer <18610854716@163.com>
      
      * Worker group adds check
      
      * fix
      
      * Worker group adds check (#2787)
      
      * Worker group adds check
      
      * fix
      
      * add not worker log and remove worker invalid property (#2784)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * add not worker log and remove worker invalid property
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory (#2786)
      
      * fixed #2788  resolve the bug #2788  (#2789)
      
      * Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory
      
      * fixed #2788
      
      * [BUG FIX] fix bug:cannot pause work flow when task state is "submit success" (#2783)
      
      * feature: add number configuration for master dispatch tasks
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix ut
      
      * fix ut
      
      * fix bug(2781): cannot pause work flow when task state is "submit success"
      
      * fix code smell
      
      * add mysql other param blank judge
      
      * test
      
      * update comments
      
      * update comments
      
      * add ut
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * fix bug:#2735 (#2770)
      
      1. store process id in zk.
      2. resolve host and port through zk path.
      3. modify ui key.
      
      * Modify startup parameter worker group value
      
      * Fix [Bug] process definition json worker group convert #2794 (#2795)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * add not worker log and remove worker invalid property
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * [BUG FIX]fix bug: Restart the worker service again, the previously submitted successful tasks are not executed bug (#2800)
      
      * feature: add number configuration for master dispatch tasks
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix ut
      
      * fix ut
      
      * fix bug(2781): cannot pause work flow when task state is "submit success"
      
      * fix code smell
      
      * add mysql other param blank judge
      
      * test
      
      * update comments
      
      * update comments
      
      * add ut
      
      * fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
      
      * update comments
      
      * add sleep
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * delete unnecessary configuration items (#2803)
      
      * rename worker_group_id to worker_group and set the value of worker_group_name (#2801)
      
      * process definition json worker group convert #2794 (#2809)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * add not worker log and remove worker invalid property
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * compatible old version host not port delete log file (#2811)
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      
      * add LoggerServerTest UT
      add RemoveTaskLogRequestCommandTest UT
      add RemoveTaskLogResponseCommandTest
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * master select worker filter high load worker #2704
      
      * add not worker log and remove worker invalid property
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * process definition json worker group convert #2794
      
      * compatible old version host not port delete log file
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      
      * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes (#2806)
      
      * delete unnecessary configuration items
      
      * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes
      
      * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes
      
      * fix oracle connect failure (#2812)
      
      * use method getJdbcUrl of parent
      
      * Remove the incorrect print message
      
      * Modify process group workerGroup value display and filter child node duplicate values
      
      * Repair process definition bulk delete function
      
      * when init workflow, the tenantId linked by the tenantId of user
      
      * dag download does not refresh, modify the task instance operation width value
      
      * fix
      
      * [bug fix]fix bug: After the master is fault-tolerant, it cannot resume operation  (#2813)
      
      * feature: add number configuration for master dispatch tasks
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix ut
      
      * fix ut
      
      * fix bug(2781): cannot pause work flow when task state is "submit success"
      
      * fix code smell
      
      * add mysql other param blank judge
      
      * test
      
      * update comments
      
      * update comments
      
      * add ut
      
      * fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
      
      * update comments
      
      * add sleep
      
      * add null point check
      
      * fix bug:After the master is fault-tolerant, it cannot resume operation
      
      * fix bug: do not failover the host is 'NULL' process
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * When update the data source,the host and port show error (#2821)
      
      * When update the data source,the host and port show error
      
      * update DataSourceServiceTest
      
      * update DataSourceServiceTest
      
      * update DataSourceServiceTest
      
      * Fix ORACLE service name value display
      
      * Add tips for download workflow
      
      * fix
      
      * [bug fix] fix: The workflow is fault-tolerant and 2 task instances are generated (#2833)
      
      * feature: add number configuration for master dispatch tasks
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix bug(#2762) the master would be blocked when worker group not exists
      
      * fix ut
      
      * fix ut
      
      * fix bug(2781): cannot pause work flow when task state is "submit success"
      
      * fix code smell
      
      * add mysql other param blank judge
      
      * test
      
      * update comments
      
      * update comments
      
      * add ut
      
      * fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
      
      * update comments
      
      * add sleep
      
      * add null point check
      
      * fix bug:After the master is fault-tolerant, it cannot resume operation
      
      * fix bug: do not failover the host is 'NULL' process
      
      * fix bug:worker failover error.
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * add maven Xmx to 5g
      
      * fix bug: send task result to master until success. (#2839)
      
      * fix bug: send task result to master until success.
      
      * fix bug: send task result to master until success.
      
      * add sleep
      
      * set null if send result failed
      
      * set null if send result failed
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * Fix errors when running on win10 (#2842)
      
      1. DS cant run bat script
      2. DS cant run python script
      3. DS cant run datax script
      
      * Fix abnormal display of regular worker parameters
      
      * add source license full text into source license (#2840)
      
      * add source license full text into source license
      
      * exclude ScriptRunner.java on rat check
      
      * fix table not exist when init db metadata (#2838)
      
      * fix table not exist when init db metadata
      
      * fix db init error
      
      * fix ds-1.3.0 init db
      
      * init db modify process instance json
      Co-authored-by: Neights <yelli.hl@foxmail.com>
      
      * change notice copyright date from 2019 to 2019-2020 (#2847)
      
      * add source license full text into source license
      
      * exclude ScriptRunner.java on rat check
      
      * change notice copyright date from 2019 to 2019-2020
      
      * fix [BUG] TaskExecutionContextCacheManagerImpl Do not execute removeByTaskInstanceId #2745 (#2754)
      
      * mr resource display and create user username length cannot be less than 3
      
      * fix
      
      * fix
      
      * fix
      
      * fix table T_DS_ALERT (#2854)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * fix
      
      * fix
      
      * modify t_ds_task_instance app_link change sql (#2861)
      
      * change max length of user name to 39 (#2856)
      
      * fix bug: send task result to master until success.
      
      * fix bug: send task result to master until success.
      
      * add sleep
      
      * set null if send result failed
      
      * set null if send result failed
      
      * change max length of username to 39
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * copy resources need get top directory first (#2866)
      
      * copy resources need get top directory first
      
      * copy resources need get top directory first
      
      * fix #2860:copy files and create empty directory (#2876)
      
      * copy resources need get top directory first
      
      * copy resources need get top directory first
      
      * fix #2860:copy files and create empty directory
      
      * Fix bug:If user didn't set the value of tenant,release process definition fail (#2885)
      
      * copy resources need get top directory first
      
      * copy resources need get top directory first
      
      * fix #2860:copy files and create empty directory
      
      * Fix bug:If user didn't set the value of tenant,release process definition online fail
      
      * merge method updateuser if and fix bug#2892
      
      fix [BUG] When editing user information, clear the phone number, the database table t_ds_user.phone is not cleared #2892
      
      * fix bug: sqoop mysql import params error (#2896)
      
      * fix bug: sqoop mysql import params error
      
      * fix bug: sqoop mysql import params error
      
      * Fixed selection of tenant value and defalut changed to lower case
      
      * update how to build and remove docs link (#2906)
      
      * Fix tasks not execute in sub-process when rerun scheduled process. #2903 (#2908)
      
      * fix create table sql exception (#2912)
      
      fix create table sql exception
      
      * fix #2910 master server will show exception for some time when it restart (#2913)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * update h2 database
      
      * fix #2910 master server will show exception for some time when it restart
      
      * Fixbug datax task (#2909)
      
      * fix:  local param bug
      
      * fix: UT bug
      Co-authored-by: NRubik-W <whm_777@163.com>
      
      * [license] add license and notice (#2916)
      
      * fix bug: send task result to master until success.
      
      * fix bug: send task result to master until success.
      
      * add sleep
      
      * set null if send result failed
      
      * set null if send result failed
      
      * change max length of username to 39
      
      * test for gc
      
      * change version to 1.3.0
      
      * check and add some license files
      
      * check and add some license files
      
      * revert
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      
      * fix oom when no master is active in dev-1.3.0 (#2918)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * update h2 database
      
      * fix #2910 master server will show exception for some time when it restart
      
      * fix oom when no master is active
      
      * fix worker oom when master server restart
      
      * fix oom
      
      * fix
      
      * add UT
      
      * fix worker group config no effect
      
      * Python, sql, sqoop,datax task node editor adds zoom function #2929 (#2933)
      
      * Python, sql, sqoop task node editor adds zoom function
      
      * fix
      
      * change 1.3.0 to 1.3.0-snapshot
      
      * add janino-3.0.12.jar
      
      * delete duplicate codes
      
      * add commons-compiler-3.0.12.jar
      
      * change delete status according to delete action result
      
      * Delete the pom files animation.scss, _normalize.scss and _animation.scss under the common folder
      
      * Change email to default (#2981)
      
      * add mybatis-3 notice in the top-level notice (#3003)
      
      * fix download resource file that under the directory error (#3007)
      
      * add mybatis-3 notice in the top-level notice
      
      * fix download resource file that under the directory error
      
      * Fix unfiltered jar files in resource directory and UDF upload resource parameters (#3008)
      
      * Change email to default
      
      * Fix unfiltered jar files in resource directory and UDF upload resource parameters
      
      * fix singe resource manager will NPE (#3013)
      
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * update h2 database
      
      * fix #2910 master server will show exception for some time when it restart
      
      * fix oom when no master is active
      
      * fix worker oom when master server restart
      
      * fix oom
      
      * fix
      
      * add UT
      
      * fix worker group config no effect
      
      * fix singe resource manager will NPE
      
      * Update HadoopUtilsTest.java
      
      * merge from dev-1.3.0
      
      * update ut
      
      * remove duplicate code
      
      * remove exception ut
      
      * remove exception ut
      
      * remove exception ut
      
      * remove exception ut
      
      * remove exception ut
      
      * [bugfix] remove Conflicting configuration sudo and user (#3038)
      Co-authored-by: Nzhangchunyang1024 <zhangchunyang0123@gamil.com>
      
      * [bugfix] 1. fix alert.json includes repeated tag 'DOLPHIN_ALERT' 2. metainfo.xml The version number of the RPM package is no longer specified in the metainfo.xml (#3032)
      Co-authored-by: Nzhangchunyang1024 <zhangchunyang0123@gamil.com>
      
      * [feature] readme.md adds instructions for changing environment variables (#3031)
      Co-authored-by: Nzhangchunyang1024 <zhangchunyang0123@gamil.com>
      
      * merge from dev
      
      * Change favicon.ico icon introduction method and package compilation to increase version number (#3030)
      Co-authored-by: NJiajie Zhong <zhongjiajie955@hotmail.com>
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@outlook.com>
      Co-authored-by: weixin_41213428's avatarchenxingchun <438044805@qq.com>
      Co-authored-by: Nbreak60 <790061044@qq.com>
      Co-authored-by: Nlgcareer <18610854716@163.com>
      Co-authored-by: Nbao liang <29528966+lenboo@users.noreply.github.com>
      Co-authored-by: Nbaoliang <baoliang@analysys.com.cn>
      Co-authored-by: NSimon <3656562@qq.com>
      Co-authored-by: Nxingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
      Co-authored-by: N张世鸣 <zhangshiming@cvte.com>
      Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>
      Co-authored-by: Nliwenhe1993 <liwenhe@apache.org>
      Co-authored-by: Nxiaochun.liu <xiaochun.liu@centurygame.com>
      Co-authored-by: Nlidongdai <escheduler@outlook.com>
      Co-authored-by: Ndailidong <dolphinschedule@gmail.com>
      Co-authored-by: NTboy <guo.jiwei@immomo.com>
      Co-authored-by: Nxuchangqun <58328021+xuchangqun@users.noreply.github.com>
      Co-authored-by: Nchangqun.xu <changqun.xu@17zuoye.com>
      Co-authored-by: Nxuchangqun <changqun.x@gmail.com>
      Co-authored-by: _和's avatarzixi0825 <649790970@qq.com>
      Co-authored-by: Nsunchaohe <sunzhaohe@linklogis.com>
      Co-authored-by: Nzhangchunyang <zhangchunyang0123@gmail.com>
      Co-authored-by: Ntlhhup <137273278@qq.com>
      Co-authored-by: Nyelli.hl@foxmail.com <amarantine@my.com>
      Co-authored-by: Neights <yelli.hl@foxmail.com>
      Co-authored-by: Nyaoyao <42244568+1445103710@users.noreply.github.com>
      Co-authored-by: N小清 <62982788+597365581@users.noreply.github.com>
      Co-authored-by: Ngabry.wu <gabrywu@apache.org>
      Co-authored-by: Nt1mon <178317391@qq.com>
      Co-authored-by: NRubik-W <39549317+Rubik-W@users.noreply.github.com>
      Co-authored-by: NRubik-W <whm_777@163.com>
      Co-authored-by: Nzhangchunyang1024 <zhangchunyang0123@gamil.com>
      ab6e4310
    • D
      fix singe resource manager will NPE (#3013) · 7445fa94
      dailidong 提交于
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * update h2 database
      
      * fix #2910 master server will show exception for some time when it restart
      
      * fix oom when no master is active
      
      * fix worker oom when master server restart
      
      * fix oom
      
      * fix
      
      * add UT
      
      * fix worker group config no effect
      
      * fix singe resource manager will NPE
      
      * Update HadoopUtilsTest.java
      7445fa94
  23. 29 6月, 2020 1 次提交
  24. 19 6月, 2020 1 次提交
    • D
      fix singe resource manager will NPE (#3013) · 620ecffa
      dailidong 提交于
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      
      * update h2 database
      
      * fix #2910 master server will show exception for some time when it restart
      
      * fix oom when no master is active
      
      * fix worker oom when master server restart
      
      * fix oom
      
      * fix
      
      * add UT
      
      * fix worker group config no effect
      
      * fix singe resource manager will NPE
      
      * Update HadoopUtilsTest.java
      620ecffa
  25. 01 6月, 2020 1 次提交
    • D
      fix table T_DS_ALERT (#2854) · 6886ed51
      dailidong 提交于
      * fix worker group config no effect
      
      * remove codehaus janino jar
      the license about janino maybe not compatiable with Apache v2
      
      * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
      
      # Conflicts:
      #	dolphinscheduler-server/src/main/resources/config/install_config.conf
      
      * datasource config
      
      * Update datasource.properties
      
      * fix RunConfig bug
      
      * remove param monitor server state
      
      * fix table T_DS_ALERT
      6886ed51
  26. 19 5月, 2020 1 次提交
    • B
      fix · 0e649456
      break60 提交于
      0e649456
  27. 18 5月, 2020 2 次提交
  28. 14 5月, 2020 1 次提交
  29. 13 5月, 2020 1 次提交
  30. 12 5月, 2020 1 次提交
    • D
      add worker group config (#2681) · 06c4ddf8
      dailidong 提交于
      * update README about DolphinScheduler
      
      * Update issue templates
      
      * update
      
      * regularize api pom xml
      update rpc maven compile to 1.8
      
      * regularize api pom xml
      
      * change commons.lang3.StringUtils to common.utils.StringUtils
      
      * update pom.xml
      
      * update
      
      * correct equals method
      
      * jasper-runtime is needed when api server start
      
      * jasper-runtime jar is needed when api server start
      
      * combine logback config of master/worker/alert/api server to one logback.xml
      
      * remove tomcat runtime jar
      
      * add UT
      
      * add license
      
      * remove jasper-runtime jar, not need anymore
      
      * sovle jar conflict, remove servlet-api 2.5 jar
      
      * remove servlet 2.5
      
      * add embedded database h2 and update embedded zookeeper
      
      * delete logs
      
      * add license
      
      * change log level to INFO
      
      * simplify and optimize config
      
      * add development state
      
      * simplify common config
      
      * add worker group config
      
      * add worker group config
      Co-authored-by: Nlidongdai <escheduler@outlook.com>
      Co-authored-by: Ndailidong <dolphinschedule@gmail.com>
      06c4ddf8
  31. 08 5月, 2020 1 次提交
  32. 29 4月, 2020 1 次提交
  33. 28 4月, 2020 1 次提交
    • X
      Increase dataX environment variable, sslTrust default value (#2555) · 7a100086
      xingchun-chen 提交于
      * add LoginTest license
      
      * Delete useless packages
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * e2e add project and workflow case
      
      * Update worker_group_id to worker_group in init.sql
      
      * Update worker_group_id to worker_group in init.sql
      
      * Update worker_group_id to worker_group
      
      * Increase dataX environment variable, sslTrust default value
      Co-authored-by: weixin_41213428's avatarchenxingchun <438044805@qq.com>
      7a100086
  34. 21 4月, 2020 1 次提交
    • D
      simplify and optimize config (#2469) · eec92ed1
      dailidong 提交于
      * [Refactor worker] simplify and optimize config (#2386)
      
      * simplify config
      
      * simplify config
      
      * simplify and optimize config
      
      * Update HadoopUtils.java
      
      optimize HadoopUtils
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HttpUtils.java
      
      * Update pom.xml
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      
      * Update HadoopUtilsTest.java
      eec92ed1
  35. 10 4月, 2020 1 次提交
  36. 09 1月, 2020 1 次提交
    • B
      add out put log when master/worker server start (#1769) · e0e0696b
      bao liang 提交于
      * merge hadoop.properties into common.properties
      
      * merge hadoop,zookeeper.properties into common.properties
      remove combined.properties/master.properties/worker.properties
      
      * change db user/pwd to test/test
      
      * rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
      remove some unused in install.sh
      
      * add out put log when master/worker server start...
      
      * add start log when servers start
      e0e0696b
  37. 26 12月, 2019 1 次提交
    • Q
      remove kazoo (#1575) · 18e8703b
      qiaozhanwei 提交于
      * remove LogViewServiceGrpc.java file and pom modify
      
      * remove kazoo
      
      * remove kazoo
      
      * remove kazoo
      
      * remove common monitor package
      
      * add license
      
      * remove kazoo modify
      
      * remove kazoo modify
      
      * remove kazoo modify
      
      * remove kazoo modify
      
      * remove kazoo modify
      
      * remove kazoo modify
      
      * install.sh remove python kazoo
      
      * add system param whether repeat running
      
      * remove kazoo modify
      
      * BusinessTimeUtils remove whther repeat running inner param
      18e8703b