* Update instructions on worker groups

[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)

* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted

* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc
Co-authored-by: NAmy <amywang0104@163.com>
Co-authored-by: Ndevosend <devosend@gmail.com>

* [Improvement][Metrics] Add metrics for alert server (#11131)

* Update related docs of metrics

* Add grafana demo dashboards for alert server metrics

* Refactor metric classes with UtilityClass annotation

* Refactor meter names in camelCase for checkstyle

* use execution status instead of status code

* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml

* Fix network error cause worker cannot send message to master

* [Feature][Metrics] Add resource download related metrics for workers (#9324)

* [Feature][Metrics] Fix bugs and add grafana demos for worker resource download metrics (#9324)

* [Feature][Metrics] Add docs to resource related metrics (#9324)

* [Feature][Metrics] Use tags to indicate status in metrics (#9324)

* [Feature][Metrics] Fix demos, docs and remove redundant code (#9324)

* [Feature][Metrics] Remove .pnpm-debug.log (#9324)

* [Feature][Metrics] Fix style check (#9324)

* [Feature][Metrics] Replace KB with bytes for the unit of resource file size in metrics (#9324)

* [Feature][Metrics] Make code neat (#9324)

* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status

* Fix state event handle error will not retry

* Use state event handler to deal with the event

* Global parameter and local parameter calculation external expansion.

* k8s task ut fix.

* TimePlaceholderUtils import DateUtils fix

* follow the review comments to fix.

* follow the review comments to fix.

* e2e rerun

* fix worker failover may lose event

* Optimize master log, add workflow instance id and task instance id in log

* Use MDC to set the workflow info in log4j

* Add workflowInstanceId and taskInstanceId in MDC

* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards

* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe

* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT

* Add dolphinscheduler-scheduler module

* Remove quartz at WorkerServer

* move k8s and permission from dolphinscheduler-service to dolphinscheduler-api

* fix 9584

* master recall

* fix ut

* update logger

* update delay queue

* fix ut

* remove sleep
Co-authored-by: N进勇 <lijinyong@cai-inc.com>
Co-authored-by: NJinyLeeChina <jiny.li@foxmail.com>

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac
Co-authored-by: Nouyangl <ouyangl@tebon.com.cn>

* [common] Make dolphinscheduler_env.sh work

* Change dist tarball `dolphinscheduler_env.sh` location
  from `bin/` to `conf/`, which users could finish their
  change configuration operation in one single directory.
  and we only need to add `$DOLPHINSCHEDULER_HOME/conf`
  when we start our sever instead of adding both
  `$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin`
* Change the `start.sh`'s path of `dolphinscheduler_env.sh`
* Change the setting order of `dolphinscheduler_env.sh`
* `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh`
when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh`
* Change the related docs

[Improvement-9609][Worker]The resource download method is selected according to the configurati… (#9636)

* The resource download method is selected according to the configuration and the service startup verification is added.

* common check CI fix

* Startup check changed to running check

* code smell

* Coordinate resources to increase test coverage.

* Split resource download method.

* Unit Test Coverage
Co-authored-by: NWangJPLeo <wangjipeng@whaleops.com>

* cancel application when kill task

* add warn log

* add cancel application

* Storage Management StorageOperate No Instance

* Add StorageOperateManager unit test

* Add license header

* Fix issues in SonarCloud code analysis
Co-authored-by: NWangJPLeo <wangjipeng@whaleops.com>

* rebase dev

* change task state to dispatch if dispatch success

* update task host when dispatch

* add dispatch task event

* test

* check tenant after enable auto create

* handle dispatch state
Co-authored-by: Ncaishunfeng <534328519@qq.com>