Co-authored-by: Ncaishunfeng <534328519@qq.com>

* cherry-pick [DS-6963][MasterServer]add cache manager for tenant and user

* [cherry-pick][Feature-6988][MasterServer] add cache manager for workflow

* [Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* [cherry-pick][Feature-6963][MasterServer] unified cache manager

* [cherry-pick][Bug-7292][ApiServer] fix cache error when standalone

* [cherry-pick][Fix-7245] [API] Fix spelEvaluationException: EL1008E: Property or field 'all' cannot be found

* bugfix
Co-authored-by: NzwZjut <zw15068835817@163.com>
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKerwin <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Ncaishunfeng <534328519@qq.com>

* cherry-pick

* fix bug_7629 (#7630)

* fix bug_7564 (#7625)

* fix bug_7512 (#7716)

* [Fix-7459][Sqoop] running error in parallel (#7752)

* fix bug_7459

* fix bug_7459

* fix bug_7459

* fix bug_7459

* fix bug_7459
Co-authored-by: NSbloodyS <sbloodys@qq.com>

* [Feature-7576][Master] Optimize complement task's date (#7585)

* feature_7576

* feature_7576

* feature 7576

* feature 7576

* [Fix-7767]Task instance is not deleted while deleting the process instance (#7768)

* fix bug_7767

* fix bug_7767

* fix bug_7767

* fix bug_7767

* fix bug_7767

* [Feature-5951][Improvement][UI] When creating an alarm instance, the alarm instance name input box prompts optimization (#7770)

* cherry-pick

* [Fix-7773][Api] malform batchDeleteProcessInstanceByIds log output (#7774)

* bug_7773

* bug_7773

* bug_7773

* bug_7773

* bug_7773

* [Feature-7777][UI] process definition page add create username (#7778)

* feature_7777

* fix bug_7499 (#7739)
Co-authored-by: NSbloodyS <sbloodys@qq.com>

* fix conflict

* fix conflict

* fix mapState never used error

* fix bug_7789 (#7790)
Co-authored-by: NSbloodyS <sbloodys@qq.com>
Co-authored-by: NSbloodyS <sbloodys@qq.com>
Co-authored-by: Nwangyizhi <wangyizhi1_yewu@cmss.chinamobile.com>

* [cherry-pick][python] Make it work to 2.0.2

* Remove unused ProcessExecutionTypeEnum

* Add queryByName to project

* Add checkTenantExists to tenant

* Add queryByTenantCode to tenant

* Add queryQueueName to queue

* Add all content from dev branch

* Add gitignore

* Add pydolphinscheduler content

* Add ds-py to bin test

* Py merge to 202

* Fix version

* Fix missing variable

* Add py4j as known deps

* Fix core database bug

* [cherry-pick][BUG-7554] Fix database lose data in standalone

* Fix thread starts too early
Co-authored-by: Ncaishunfeng <534328519@qq.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

* [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238)

* to #7237

* rerun test
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 05aef279 and handle conflicts

* to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081  (#7082)

* to #7081

* fix #7081

* to #7081
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 8ebe0606 and handle conflicts

* cherry-pick 1f184440 and handle conflicts

* fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

* add default constructor (#6780)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* to #7108 (#7109)

* to #7450

* to #7450: fix parallel bug

* add index

* expose config to user

* fix bug

* fix bug

* add delay delete

* fix bug

* add License

* fix ut

* fix ut

* fix name
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

* [Improvement-#7529][tools] Init DB schema from the full sql file. (#7530)

* [DS-7227][refactor] optimize alert plugin instance list order (#7506)

[DS-7227][refactor] optimize alert plugin instance list order
query all alert plugin instance list order by update time desc
optimize `select *` sql in the AlterPluginInstanceMapper.xml
Co-authored-by: Nlgcareer <18610854716@163.com>
Co-authored-by: NTom <532066967@qq.com>

Co-authored-by: Ncaishunfeng <534328519@qq.com>

* [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238)

* to #7237

* rerun test
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 05aef279 and handle conflicts

* to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081  (#7082)

* to #7081

* fix #7081

* to #7081
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 8ebe0606 and handle conflicts

* cherry-pick 1f184440 and handle conflicts

* fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

* add default constructor (#6780)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* to #7108 (#7109)

* to #7511

* to #7511

* to #7511

* to #7511
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

* [Feature-7110][dolphinscheduler-api] support grant project by code

* [Feature-7180][dolphinscheduler-api] Revoke project permission for specified user

* [Feature-7191][dolphinscheduler-api] support query authorized user list by project code

* [Feature-7301][dolphinscheduler-api] Support query access token for specified user
Co-authored-by: Nouyangyewei <yewei.oyyw@alibaba-inc.com>

* [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238)

* to #7237

* rerun test
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 05aef279 and handle conflicts

* to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081  (#7082)

* to #7081

* fix #7081

* to #7081
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* chery-pick 8ebe0606 and handle conflicts

* cherry-pick 1f184440 and handle conflicts

* fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

* add default constructor (#6780)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* to #7108 (#7109)
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: NKirs <acm_master@163.com>

[2.0.1][Fix-7284] [api] Fix the missing project information in the process instance timeout alarm  (#7309)

* cherry pick

* Fix the missing project information in the process instance timeout alarm

* fix taskNode NPE

* fix switch node NPE when no else branch
Co-authored-by: Ncaishunfeng <534328519@qq.com>

* [Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)

1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.

* [BUG][SQL] fix the process list page load slowly. (#7166)

* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1

* [2.0.1] remove unused foreign_key

* [2.0.1] add version in sql

* [2.0.1] change serial for postgre
Co-authored-by: NGaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>

Co-authored-by: Ncaishunfeng <534328519@qq.com>

alert plugin could not filter by given
pattern, cause api server do not handle
parameter searchVal, This patch add parameter
`searchVal` for alert plugin.

[Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107) (#7150)
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>

* [deploy] Use install_config.conf export datasource properties

Directly user install_config.conf to set spring datasource
properties, delete datasource.properties file for more clearly

* Remove unused properties

* pick-6863

* [Improvement][API][num-7] add task delete check (#6887)

* upgrade add sub_process

* add interface method

* add interface method

* fix ut

* add task delete check

* add task delete check

* [Improvement][API][num-5] The workflow of the main table version is not allowed to be deleted (#6893)

* feature:deleteProcessDefinitionVersion

* pref:delete duplicate enum definition
Co-authored-by: N北笙 <“zhanqian@cai-inc.com”>

* [DS-6931][fix] swagger failed to start due to improper use of request param annotation (#6933)
Co-authored-by: Neye <eye.gu@aloudata.com>

* [fix #6878]  implement interface queryUpstreamRelation queryDownstreamRelation and add UT (#6888)

* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, add license

* fix: #6878, code style

* fix: #6878, code style

* fix: #6878, fix Duplication

* fix: #6878

* fix: #6878 , fix style

* to #6878: fix style

* to #6878: fix style

* to #6878: add buildTaskDefinition to overwrite equals and hashCode
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>

* [Improvement][API][num-6] TaskDefinition  releaseTaskDefinition (#6965)

* finished releaseTaskDefinition

* add releaseTaskDefinition unit test

* change import

* add import

* bug fix

* change import
Co-authored-by: N时光 <caizhedong@cai-inc.com>

* pick-6966

* update task_definition_log field typ (#6967)

* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* pick-6970

* pick-6977

* pick-6984

* add processDefinition releaseWorkflowAndSchedule (#7044)

* [Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (#7047)

* [Improvement] save schedule when workflow is offline

* [Improvement] rollback transactional when creating/updating schedule failed.

* [Improvement] resolve merge conflict

* [Improvement] do not set projectName and processDefinitionName when saving schedule

* [Improvement] do not set projectName and processDefinitionName when saving schedule
Co-authored-by: Nedward-yang <yangjianh210@gmail.com>

* pick-7053

* pick-7098

* pick add 6772 relation code

* fix ut
Co-authored-by: NJeff Zhan <360400325@qq.com>
Co-authored-by: N北笙 <“zhanqian@cai-inc.com”>
Co-authored-by: ououtt <734164350@qq.com>
Co-authored-by: Neye <eye.gu@aloudata.com>
Co-authored-by: NzwZjut <zw15068835817@163.com>
Co-authored-by: Nhonghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: N时光 <41109695+yikeshiguang@users.noreply.github.com>
Co-authored-by: N时光 <caizhedong@cai-inc.com>
Co-authored-by: Nzhang <60912722+boy-xiaozhang@users.noreply.github.com>
Co-authored-by: NEdwardYang <yangjianh210@126.com>
Co-authored-by: Nedward-yang <yangjianh210@gmail.com>

PR #7003
ISSUE #6998

* upgrade compatibility exception

* upgrade compatibility exception
Co-authored-by: NJinYong Li <42576980+JinyLeeChina@users.noreply.github.com>

* Fix the failure of querying details of alert group

* code cleanup

PR #6942
ISSUE #6941
Co-authored-by: NKerwin <37063904+zhuangchong@users.noreply.github.com>

[Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046) (#7059)
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>

* data source spi

* Fix datasource not work in tests and standalone server, and run all tests by default (#6743)

* Refactor alert plugins and simplify its usage (#6723)

* [Fix-6792] resolve API and alert port conflicts (#6792) (#6815)

* [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792)

* [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) (#6828)

* [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490)

* fix pom dep scope

* filter alert

* filter alert

* filter alert

* fix checkstyle

* remove hikaricp version

* update pom

* add properties

* add properties

* rename properties

* fix e2e fail

* rename properties

* [Fix-6792] resolve API and alert port conflicts (#6792) (#6815)

[Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792)

* fix ut fail

* fix license

* fix license

* update package

* update package

* remove unused license
Co-authored-by: Nmask <39329477+narcasserun@users.noreply.github.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

* [2.0.1-prepare#6712]Refactor registry plugin and simplify its usage (#6712)

* upgrade version and fix dep err

* Clean up the utility codes (#6732)

* remove code style check

* upgrade docker version
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

* [Fix][Common] rewrite code generate,fix bit shift (#6914)

* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/EnvironmentServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/CodeGenerateUtilsTest.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/upgrade/UpgradeDao.java
#	dolphinscheduler-dist/release-docs/LICENSE
#	dolphinscheduler-python/src/main/java/org/apache/dolphinscheduler/server/PythonGatewayServer.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java

* [Fix][Common] rewrite code generate,fix bit shift (#6914)

* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/EnvironmentServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/CodeGenerateUtilsTest.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/upgrade/UpgradeDao.java
#	dolphinscheduler-dist/release-docs/LICENSE
#	dolphinscheduler-python/src/main/java/org/apache/dolphinscheduler/server/PythonGatewayServer.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java

* [Fix][Common] rewrite code generate,fix bit shift (#6914)

* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/EnvironmentServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/CodeGenerateUtilsTest.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/upgrade/UpgradeDao.java
#	dolphinscheduler-dist/release-docs/LICENSE
#	dolphinscheduler-python/src/main/java/org/apache/dolphinscheduler/server/PythonGatewayServer.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java

* [Fix][Common] rewrite code generate,fix bit shift (#6914)

* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/EnvironmentServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/CodeGenerateUtilsTest.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/upgrade/UpgradeDao.java
#	dolphinscheduler-dist/release-docs/LICENSE
#	dolphinscheduler-python/src/main/java/org/apache/dolphinscheduler/server/PythonGatewayServer.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
Co-authored-by: NJinYong Li <42576980+JinyLeeChina@users.noreply.github.com>

Co-authored-by: Nding <dingpf@helowin>

* upgrade add sub_process (#6845)

* cherry-pick 6846 change sub process id to code

* cherry-pick 6846 change sub process id to code

* ut fix
Co-authored-by: NJinYong Li <42576980+JinyLeeChina@users.noreply.github.com>

* add convert dependent/conditions

* fix upgrade 2.0 bug

* fix upgrade 2.0 bug

* Optimizing SQL scripts

* add upgrade 2.0 ddl
# Conflicts:
#	sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql

Co-authored-by: N张柏强 <zhangbaiqiang@lvshou.com>

* Optimizing SQL scripts

* add upgrade 2.0 ddl

* add upgrade 2.0.0 code

* fix valid license header

* fix valid license header

* fix valid license header

* fix ut

* fix code style

* fix code style

* Specify the catalog and schema when check table and column exsits

* Optimizing SQL scripts

* add upgrade 2.0 ddl
# Conflicts:
#	sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql