* make sync metrics concurrency

* add changelog

* add changelog

* polish codes

* change default value

* remove unnecessary codes
Co-authored-by: NEvan <evanljp@outlook.com>
Co-authored-by: NZhenxu Ke <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

* fix transaction too large error when use TiDB as storage

* add CHANGES.md

* fix typo

* add tidb storage provider module.

* add e2e.

* revert ui submodule.

* fix license.

* Fix wrong health check test

* add tidb ttl e2e.
Co-authored-by: Nmoonsphere <moonsphere@m.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* Add FAQ about VNode on the UI.
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

Add syncBulkActions config to avoid the large amount of metrics data written ES in a single sync bulk request. (#5699)

* distinct the number of metrics or common record index with super dataset index.

Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.

### Motivation

 Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.

 ### Modification

 Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.

 ### Result

 - We can switch a configuration by environment variables or system property

 - Closes #4511

* Temp commit

* Support secretsManagementFile file.

* Update doc.

* 1. Support JKS/pass runtime change too.
2. Follow review.

* Fix format.

* Fix username/password/trustPass haven't been updated in the es client.

* Fix doc issue.
Co-authored-by: NJared Tan <jian.tan@daocloud.io>

* Revert "[BugFix] Docker InfluxDB cannot be enabled (#4483)"

This reverts commit 73d0802c.

* Fix unclear doc and add missing variable

* provide influxdb as a new storage plugin
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

* Support Daily Index Step.

* Fix format.

* Fix mischanged ES.

* Support day/hour/minute metrics merging into one index. Reduce the number of index 50%.

* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* [Feature] Officially support ElasticSearch 7 as backend storage

* Extract bootstrap module to be shared by multiple starters

* Add missing configuration file and distinguish different version in test

* Update known-oap-backend-dependencies-es7.txt

Update the dependency lib

* Fix missing adaptation to ElasticSearch 7

* Rename ElasticSearch 7 specific class with Es7 infix

* Fix miss-adapted ElasticSearch DAO API

* Add ES7 specific configuration and polish documentations

* Polish documentations

* Polish documentations

* Is it a typo or not?

* delete datasource-settings.properties

* handle 'metadataQueryMaxSize'

* delete DynamicModuleConfig.java

* modify mysql config

* parsing yaml to support the configuration for multiple properties

* delete datasource-settings.properties

* modify doc

* add properties for docker-entrypoint.sh

* add testLoadConfig()

* support es https ssl config.

* update

* update docs.

* refactoring var name.

* fix

* update docs.

* add config in es section.

* fix logical.

* support elasticsearch address configuration with http scheme.

* update logical.

* Update backend-storage.md

* Update backend-storage.md

* Update ES-Server-FAQ.md

* The new feature of Elasticsearch storage with the time series implementation.

* Time bucket util improve.

* Fixed some Gramma error.

* Update ttl.md

Polish English a little.

* Update ttl.md

Add more explicit description of TTL

* Update ttl.md

* Update StorageModuleElasticsearchConfig.java

Set otherMetricsDataTTL = 0 as default, make user more clear.

* Rename the timeSeriesAble to capableOfTimeSeries

* Fixed a test case failure.

* Make Jaeger receiver works in 12450 as default.

* Finish compile and codebase.

* Make codes runnable.

* Finish Jaeger query tests.

* Add document for jaeger receiver.

* Add readme.

* Fix a break link.

* Finish doc.

* Codebase for zipkin span persistence.

* Fix missing fields in storage.

* Miss the latency field.

* Finish some tests.

* Fix wrong latency.

* Finish doc and reset application.yml

* Make the description more clear.

* Added elasticsearch warehouse certification support

* Add the elasticsearch service for Http Basic instructions

* Update backend-storage.md

Change document.

2. Zookeeper implementation of the cluster module using name space to support using same zookeeper cluster by different OAP clusters.

* Update document and user wall.

* Change format.

* user wall updated.

* Fix typo

* Correct namespace description

* Support sampling trace at server side and keep metric right.

* Add a trace sampling document

* Fix wrong default value.

* Fix document issue.

* Fix assemble issue.

* Fix wrong settings and doc.

* add TiDB configuration description.

* add tested mysql client driver version