提交 · f112415b13d7fdd18cf668c2cc639e79cdf08458 · yanghaoyuan.sh / DolphinScheduler

24 3月, 2020 1 次提交

由 qiaozhanwei 提交于 3月 24, 2020

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* add license

* modify javadoc error

* TaskExecutionContext create modify

* buildAckCommand taskInstanceId not set modify

* java doc error modify

* add comment

* ExecutorManager interface add generic type

* add TaskInstanceCacheManager receive Worker report result

* TaskInstance setExecutePath

* add TaskInstanceCacheManager to receive Worker Task result report

* TaskInstanceCacheManager add remove method

* add license

* add dispatcht task method

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* 1，worker TaskPros use TaskExecutionContext replase
2，Master kill Task , KillTaskProcessor modify

* worker remove db

* ShellTask modify

* master persistence processId and appIds

* master persistence processId and appIds

* master add kill task logic

* master add kill task logic

* master add kill task logic

* javadoc error modify

* remove chinese log

* executeDirectly method add Override

* remote module modify

* TaskKillResponseProcessor command type modify

* create buildKillCommand

* host add host:port format

* host add host:port format

* TaskAckProcessor modify

* TaskAckProcessor modify

* task prioriry refator

* remove ITaskQueue

* task prioriry refator

* remove ITaskQueue

* TaskPriority refactor

* remove logs

* WorkerServer refactor

* MasterSchedulerService modify

* WorkerConfig listen port modify

* modify master and worker listen port

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* Encapsulate the parameters required by sqltask

* 1，Encapsulate the parameters required by sqltask
2，SQLTask optimization

* AbstractTask modify

* ProcedureTask optimization

* MasterSchedulerService modify

* TaskUpdateQueueConsumer modify

* test

* DataxTask process run debug

* DataxTask process run debug

* add protobuf dependency，MR、Spark task etc need this

* TaskUpdateQueueConsumer modify

* TaskExecutionContextBuilder set TaskInstance workgroup

* WorkerGroupService queryAllGroup modify
query available work group

* 1，get workergroup from zk modify
2，SpringConnectionFactory repeat load modify

* master and worker register ip  use OSUtils.getHost()

* ProcessInstance host set ip:port format

* worker fault tolerance modify

* Constants and .env modify

* master fault tolerant bug modify

* UT add pom.xml

* timing online  modify

* when taskResponse is faster than taskAck to db，task state will error
add async queue and new a thread reslove this problem

* TaskExecutionContext set host

* 1，TaskManager refactor
2， api start load server dolphinschedule-daemon.sh modify

* 1，TaskManager refactor
2， api start load server dolphinschedule-daemon.sh modify

* add UT in pom.xml

* revert dolphinscheduler-daemon.sh
Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>

f112415b

23 3月, 2020 1 次提交

UT Coverage rate test (#2276) · 052f9d10

由 qiaozhanwei 提交于 3月 23, 2020

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* add license

* modify javadoc error

* TaskExecutionContext create modify

* buildAckCommand taskInstanceId not set modify

* java doc error modify

* add comment

* ExecutorManager interface add generic type

* add TaskInstanceCacheManager receive Worker report result

* TaskInstance setExecutePath

* add TaskInstanceCacheManager to receive Worker Task result report

* TaskInstanceCacheManager add remove method

* add license

* add dispatcht task method

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* 1，worker TaskPros use TaskExecutionContext replase
2，Master kill Task , KillTaskProcessor modify

* worker remove db

* ShellTask modify

* master persistence processId and appIds

* master persistence processId and appIds

* master add kill task logic

* master add kill task logic

* master add kill task logic

* javadoc error modify

* remove chinese log

* executeDirectly method add Override

* remote module modify

* TaskKillResponseProcessor command type modify

* create buildKillCommand

* host add host:port format

* host add host:port format

* TaskAckProcessor modify

* TaskAckProcessor modify

* task prioriry refator

* remove ITaskQueue

* task prioriry refator

* remove ITaskQueue

* TaskPriority refactor

* remove logs

* WorkerServer refactor

* MasterSchedulerService modify

* WorkerConfig listen port modify

* modify master and worker listen port

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* Encapsulate the parameters required by sqltask

* 1，Encapsulate the parameters required by sqltask
2，SQLTask optimization

* AbstractTask modify

* ProcedureTask optimization

* MasterSchedulerService modify

* TaskUpdateQueueConsumer modify

* test

* DataxTask process run debug

* DataxTask process run debug

* add protobuf dependency，MR、Spark task etc need this

* TaskUpdateQueueConsumer modify

* TaskExecutionContextBuilder set TaskInstance workgroup

* WorkerGroupService queryAllGroup modify
query available work group

* 1，get workergroup from zk modify
2，SpringConnectionFactory repeat load modify

* master and worker register ip  use OSUtils.getHost()

* ProcessInstance host set ip:port format

* worker fault tolerance modify

* Constants and .env modify

* master fault tolerant bug modify

* UT add pom.xml
Co-authored-by: Nqiaozhanwei <qiaozhanwei@analysys.com.cn>

052f9d10

12 3月, 2020 1 次提交

Fix javadoc and DataxTaskTest UT error (#2147) · 16eeaf50

由 t1mon 提交于 3月 12, 2020

* Fix javadoc and DataxTaskTest UT error

* Fix UT error
* LoggerServiceTest UT error
* WorkerGroupServiceTest UT error
Modify Host.of()
* Add address NPE check

* Trigger actions rerun

* Try to solve maven VM crash

* Try to solve maven VM crash 1

* Try to solve maven VM crash 2

* Try to solve maven VM crash 3

16eeaf50

14 2月, 2020 2 次提交

Refactor Architecture Basic modification #1658 (#1946) · c1ee1333

由 qiaozhanwei 提交于 2月 14, 2020

* 1，remove dolphinscheduler-rpc module  2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925)

* 1，remove dolphinscheduler-rpc module
2，add dolphinscheduler-remote module
3，add dolphinscheduler-service module
4，refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates
Co-authored-by: Nkhadgarmage <khadgar.mage@outlook.com>
Co-authored-by: Nzhukai <boness@qq.com>
Co-authored-by: NYelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel
Co-authored-by: NTboy <technoboy@yeah.net>
Co-authored-by: Nkhadgarmage <khadgar.mage@outlook.com>
Co-authored-by: Nzhukai <boness@qq.com>
Co-authored-by: NYelli <amarantine@my.com>

c1ee1333

Fix socket bug: read before sock connect. Add UT for FourLetterWordMain. (#1954) · cdd53a76

由 Jave-Chen 提交于 2月 14, 2020

* Fix socket bug: read before sock connect. Add UT for FourLetterWordMain.

* Fix code smell

   Remove  @Ignore and  @test from testCmd().

* correct comment

cdd53a76

13 2月, 2020 1 次提交
- Y
  
  add DagHelperTest to pom · 77023aee
  由 Yeleights 提交于 2月 13, 2020
  
  77023aee
12 2月, 2020 1 次提交
- modify DataAnalysisServiceTest by mock · 735034c7
  由一子三木提交于 2月 12, 2020
  
  735034c7
10 2月, 2020 1 次提交

Add modify user name for process definition (#1919) · d937a6ae

由 Yelli 提交于 2月 10, 2020

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

d937a6ae

09 2月, 2020 1 次提交
- modify loggerService UT by Mock · e00ef491
  由一子三木提交于 2月 09, 2020
  
  e00ef491
07 2月, 2020 2 次提交
- Z
  
  pom add ut， add cronutils ut · 6b586f31
  由 zhangzy 提交于 2月 07, 2020
  
  6b586f31
- Z
  Remove ScheduleUtil，use the existing CronUtils · 662233e0
  由 zhangzy 提交于 2月 07, 2020
```
CronUtils is a public quartz tools, then the project dolphinscheduler-api can remove dolphinscheduler-server
```
  662233e0
06 2月, 2020 2 次提交

D
combine logback config of master/worker/alert/api server to one (#1898) · c63cd3e2
由 dailidong 提交于 2月 06, 2020
```
* combine logback config of master/worker/alert/api server to one logback.xml
```
c63cd3e2

魔

[feature] data synchronization function (#1881) · f942e5ce

由魔方不在手提交于 2月 06, 2020

* feature：data synchronization function

* feature：data synchronization function

* feature：data synchronization function

* fix：add license

* fix：resource close bug

* fix：remove class author

* fix：Improve test coverage

* fix：add UT

* fix：add UT

* fix：data sync node renamed to datax

* fix：add UT

* fix: UT bug

* fix：Optimize variable name

* fix：pom

* fix：Variable name error

* fix：optimize the code

f942e5ce

03 2月, 2020 1 次提交

fix api server startup failure (#1886) · 42b3e5bd

由 Tboy 提交于 2月 03, 2020

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

42b3e5bd

01 2月, 2020 1 次提交

some updates for pom (#1879) · 6b0debec

由 Tboy 提交于 2月 01, 2020

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

6b0debec

30 1月, 2020 1 次提交
- T
  remove unused properties in pom (#1876) · a5536617
  由 Tboy 提交于 1月 30, 2020
```
* #1875 remove unused properties in pom
```
  a5536617
16 1月, 2020 2 次提交

老

Supplementary data by schedule (#1830) · 1c77708a

由老佛爷提交于 1月 16, 2020

* Supplementary data by schedule

* fix sonar check bug

* fix code duplicated blocks

* ut

* loop by day

* MasterExecThread test

* test add licene
Co-authored-by: Ndailidong <dailidong66@gmail.com>

1c77708a

add BaseServiceTest BaseDAGServiceTest UT (#1835) · c558da0b

由 samz406 提交于 1月 16, 2020

* modify UdfFuncServiceTest UT

* fix get tenantCode my NPE

* add  ResourcesServiceTest

* add error log

* add BaseServiceTest BaseServiceTest UT

* update

c558da0b

15 1月, 2020 1 次提交
- add BaseServiceTest BaseServiceTest UT · b4a6d544
  由一子三木提交于 1月 15, 2020
  
  b4a6d544
09 1月, 2020 3 次提交
- Y
  
  add pom mysql test (#1784) · e10810a5
  由 Yelli 提交于 1月 09, 2020
  
  e10810a5
- D
  
  remove lombok dependency(last) (#1771) · aa715e17
  由 DK.Pino 提交于 1月 09, 2020
  
  aa715e17
- Y
  refactor export process (#1772) · 7dfa9976
  由 Yelli 提交于 1月 09, 2020
```
* refactor process export

* comment & null check for export process

* add License

* add javadoc

* add DataSourceParamTest and DependentParamTest

* refactor process export & add exportProcessMetaDataStr UT
```
  7dfa9976
07 1月, 2020 3 次提交

T
remove freemarker dependency (#1757) · 074fd41a
由 Tboy 提交于 1月 07, 2020
```
* remove freemarker dependency
```
074fd41a
T
move version properties to parent pom for easy management of api module (#1736) · 4cc058c9
由 Tboy 提交于 1月 07, 2020
```
* move version to parent pom

* move version properties to parent pom for easy management
```
4cc058c9

Refactor alert template (#1713) · 9eb823c0

由 DK.Pino 提交于 1月 07, 2020

* refactor Alert Template

* refactor alert template(remove freemarker)

* add license

* update abstract class to interface

* make test into root pom

* fix powermock static

* fix powermock static

* remove custom exception

* remove custom exception

9eb823c0

06 1月, 2020 3 次提交
- S
  fix get tenantCode my NPE (#1684) · 15ae4dfa
  由 samz406 提交于 1月 06, 2020
```
* modify UdfFuncServiceTest UT

* fix get tenantCode my NPE

* add  ResourcesServiceTest

* add error log
```
  15ae4dfa
- T
  
  move version to parent pom (#1722) · 94fa1adf
  由 Tboy 提交于 1月 06, 2020
  
  94fa1adf
- J
  Fix sonarcloud analysis on merge PR (#1707) · d3d96197
  由 Jave-Chen 提交于 1月 06, 2020
```
* #1706 fix sonarcloud analysis on merge PR

* sonarcloud use cobertura report and don't run UT again; include ui source file; merge to one workflow
```
  d3d96197
04 1月, 2020 2 次提交

Q
Revert "move version to parent pom (#1712)" (#1715) · 1f7db8f7
由 qiaozhanwei 提交于 1月 04, 2020
```
This reverts commit 87e8f0d8.
```
1f7db8f7

move version to parent pom (#1712) · 87e8f0d8

由 Tboy 提交于 1月 04, 2020

* remove commons-lang3 dependancy

* remove commons-lang3 dependancy

* remove unused import

* all the submodule should not contain version, so move version to parent,

87e8f0d8

03 1月, 2020 1 次提交

Feature ut common thread (#1674) · b26dcba6

由 AmoryWang 提交于 1月 03, 2020

* add common thread ut

* Update ThreadUtilsTest.java

add license header

* delete useless file
Co-authored-by: Ndailidong <dailidong66@gmail.com>

b26dcba6

02 1月, 2020 2 次提交

#1544 fix bug: workflow import (#1676) · 26ed786c

由 Yelli 提交于 1月 02, 2020

* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

26ed786c

add ResourcesServiceTest · 32604123
由一子三木提交于 1月 02, 2020

32604123

31 12月, 2019 2 次提交
- Z
  Add ParamUtilsTest which is the UT of ParamUtils (#1634) · b4711194
  由 zhukai 提交于 12月 31, 2019
```
1. Add ParamUtilsTest.
2. Add a null check in the method convert.
3. Add the UT path to the root pom.
```
  b4711194
- S
  
  modify UdfFuncServiceTest UT (#1644) · dc6c1890
  由 samz406 提交于 12月 31, 2019
  
  dc6c1890
30 12月, 2019 2 次提交
- modify UdfFuncServiceTest UT · d5fe55ad
  由一子三木提交于 12月 30, 2019
  
  d5fe55ad
- S
  add service UT (#1637) · 503be5f4
  由 samz406 提交于 12月 30, 2019
```
* User update  not check params

* user phone update when noteEmpty

* modify saveWorkerGroup may NPE

* add some service UT

* add  service ut include
```
  503be5f4
29 12月, 2019 2 次提交

Q
CommandMapperTest UT modify #1465 (#1625) · e61586cd
由 qiaozhanwei 提交于 12月 29, 2019
```
* CommandMapperTest UT modify
```
e61586cd

AlertMappert UT modify #1465 (#1616) · e9fcf612

由 qiaozhanwei 提交于 12月 29, 2019

* remove LogViewServiceGrpc.java file and pom modify

* remove kazoo

* remove kazoo

* remove kazoo

* remove common monitor package

* add license

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* install.sh remove python kazoo

* add system param whether repeat running

* remove kazoo modify

* BusinessTimeUtils remove whther repeat running inner param

* add AccessTokenMapperTest UT

* CI UT yml modify，start postgresql and zookeeper by default

* add AlertGroupMapperTest UT in github action

* Conflicts reslove

* AlertMappert UT modify

* AlertMappert UT modify

* AlertMappert UT modify

e9fcf612

27 12月, 2019 1 次提交

add AlertGroupMapperTest UT in github action (#1590) · ef401b98

由 qiaozhanwei 提交于 12月 27, 2019

* remove LogViewServiceGrpc.java file and pom modify

* remove kazoo

* remove kazoo

* remove kazoo

* remove common monitor package

* add license

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* install.sh remove python kazoo

* add system param whether repeat running

* remove kazoo modify

* BusinessTimeUtils remove whther repeat running inner param

* add AccessTokenMapperTest UT

* CI UT yml modify，start postgresql and zookeeper by default

* add AlertGroupMapperTest UT in github action

* Conflicts reslove

ef401b98

yanghaoyuan.sh / DolphinScheduler 与 Fork 源项目一致

yanghaoyuan.sh / DolphinScheduler
与 Fork 源项目一致