* go2sky e2e
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.

* Finish step one, source and entity changed.

* Step 2, finish the basic and core DAO changes.

* Step 3. Change all source codes of backend, and make project compiling successfully.

* Make startup successful and persistence works.

* Fix dead codes in the MySQLTableInstaller.

* Fix mischange.

* Fix MySQL storage bootstrap error.

* Rollback dependency change.

* Simply enable tailing logs from container no mater it's in CI or not

* Fix segment table issue.

* Fix MySQL Instanller.

* Support Influxdb implementation of new endpoint traffic.

* Set and modify timeout to 90mins

* Remove unnecessary entity in the EndpointTraffic.

* Fix wrong entity builder and make name more clear.

* Fix 2 equal methods bug.

* Fix InfluxDB identifier(name) conflict issue.

* Fix endpoint topology query, endpoint id should be changed as a string.

* Fix a missing change.

* Fix a wrong commit.

* Column name can't be changed, due to it affect the hybrid metadb(h2/mysql) query, change it explicitly with double quotation marks.

* Fix formats.

* Remove endpoint/operation register from the agent. and change the plugin tests(without local tests). operationId relation codes have been removed from agent too.

* Try to fix the hytrix case.

* Fix 3 cases.

* Try to fix cases.

* Fix 2.

* Fix one.

* Fix the serviceId type mis-change.

* Fix int query

* Fix hystrix case.

* Fix a case.

* Fix EndpointCallRelationDispatcher set the source/dest endpoint without id encoding.

* Update documentation and add 2 test cases.
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* fix #4539, add PHP to the query protocol

* Add PHP module support to e2e test

* Add PHP e2e test into the CI control file

* Remove networks aliases

* Add PHP module support to e2e test

* Add e2e.php.yaml

* Locked version

* Add 200000 microsecond sleep in E2E-PHP
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

* Fix license headers and rename E2E job name

* Fix RAT plugin and enable both RAT and checkstyle plugin

* Provide active HTTP parameter collection automatically in the profiling context

* Fix style check

* fix style error

* Add profile status, storage it into TracingContext. simplify to get profiling method

* Change the profile status field name in TracingContext

* Replace gone images

* resolve issues

* resole description issues
Co-authored-by: NMrproliu <mrproliu@lagou.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* Remove unnecessary parameter in the profile log query

* Replace gone images
Co-authored-by: NMrproliu <mrproliu@lagou.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

Motivation:

According the feedback from the community, the e2e tests are hard for
them to understand and get them started, so this pull request aims to
rewrite the e2e tests with some improvements:

- debuggable locally, now you can run every individual case directly
inside the IDE, just like running a plain unit test.

- same functionalities of multiple counterparts share the same codes,
cases like different storages and cluster coordinators share the same
codes, easy to add more similar cases.

- the "end" now extends to the Doccker images, meaning that some bugs
like #4483 and #4471 can be more easily and early to be discovered.

- codes style and license headers are checked

Modifications:

- adopt jupeter (a.k.a junit 5), testcontainers as the new test
framework, all containers are orchestrated by docker-compose.yml,
well-known technology that developers should be familiar with.

- rewrite all test cases and verify them.

- reorganize the workflows in GitHub Actions.

Result:

- new e2e test should be easy to add.

- closes https://github.com/apache/skywalking/issues/4472

### Motivation

 Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.

 ### Modification

 Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.

 ### Result

 - We can switch a configuration by environment variables or system property

 - Closes #4511

* Enable OAP gRPC SSL transportation

Porting to OpenSSL to enable SSL transportation. The server private
key is in the format of PCKS#8, certificates is x509 though.
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Add netty-tcnative-boringssl-static 2.0.26 licnese
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Update docs/en/setup/backend/grpc-ssl.md
Co-Authored-By: Nkezhenxu94 <kezhenxu94@apache.org>

* Update docs/en/setup/backend/grpc-ssl.md
Co-Authored-By: Nkezhenxu94 <kezhenxu94@apache.org>

* Add SSL to grpc remote client and sharing server
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Update documents
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Fix nits
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Fix test case
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Add e2e-ssl and remove other test cases temporarily
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Add TLS relevant files
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Decrease timeout
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Test config works
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* triger CI on oap-ssl branch

* Disable agent ca
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Switch to sharing server
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Add agent log to console
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Upload log files

* Change backend
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Revert "Change backend"

This reverts commit 7a085711e3653d3bf642aa84f73650e22595e383.

* Update backend
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Revert e2e
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Ignore TSL relevant files
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Polish codes
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>

* Update certificates expires year to 2120
Signed-off-by: NGao Hongtao <hanahmily@gmail.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

Update the latest release commit to the e2e for release verification.

* support http api.

* e2e
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@apache.org>

* fix e2e miss.

* profide multi time range to query profile analyze

* use method to avoid direct reference field

* add profiled segment query

* fix query field error

* provide influxdb as a new storage plugin
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

Motivation:

Review code styles with some bots automatically.

Modifications:

Set up ReviewDog in GitHub Action to review code style.

Add more check rules to checkstyle plugin.

Result:

Obvious code styles can be reviewed and commented automatically.

* provide profile analyze query
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

* provide profiled segment query
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: NAirTrioa <44222967+AirTrioa@users.noreply.github.com>

* Add lombok dependency into root pom.xml

* Update all depending lombok pom.xml(s)

* Change lombok version into latest(1.18.10)

* Revert lombok dependency config of test projects

* Remove redundant lombok dependency of submodule

* Move Lombok into dependencies node for applying globally

### Motivation:

Make E2E tests more stable.

### Modifications:

Verify the traces loosely as other tests, and remove the `verify` method which verifies traces strictly, as we have preparation that also generates traces.

### Result:

E2E tests are more stable.

* merge e2e profile test project, Use `env` to distinguish different storage types

* remove maven child module

* change `env` -> `storage`
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* sniffer processing profile task and report status and snapshot

* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`

* resolve names

* change profile to single one thread run.

* 1. change to the ArrayList, because known the max size
2. rename issue resolved

* add profiling status enum

* change sniffer use full name issue

* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task

* remove unnecessary getter

* add test assert error message

* adding `AgentServiceRule`

* revert original assert

* remove unnecessary getter

* resolve issues

* reduce findService invoke

* resolve style error

* recheck profiling when change first span operatin name

* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting

* remove article link

* add `ProfileTask#maxSamplingCount` check

* resolve conflict (Downstream -> Commands)

* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump

* remove unused return val

* remove some `@param` and `@return`

* add profile task check result data bean

* change profiler slot to `AtomicReferenceArray`

* resolved java doc error

* fix doc error, remove meaningless descriptions

* resolve missed profile receiver on oap starter

* resolve method invoke error
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md
Co-Authored-By: Nkezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.
Co-authored-by: NJared Tan <jian.tan@daocloud.io>
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning

* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId
Co-authored-by: Nkezhenxu94 <kezhenxu94@163.com>

* Fix mave-os plugin issues for Eclipse m2e

* Add os-msven-plugin.version property

* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* test e2e

* add official_analysis.oal & update getServiceInstanceTopology

* test instance topo for single service

* fix gql

* fix verifyServiceInstances

* verifyServiceInstanceTopo

* fix verifyServiceInstanceTopo

* fix ServiceInstanceInventory#name

* fix ServiceInstanceInventory#name

* e2e test

* fix action

* fix provider port

* fix Instance Node Type

* re run

* fix expected-data

* add e2e mysql

* set @test timeout

* fix ci

* increase timeout

* re check

* test ttl es7

* test ttl

* fix action

* merge group

* test ttl es7

* replace official_analysis.oal

* replace official_analysis.oal

* comments

* add TODO

* add TODO

* fix filed name

* Enhancement TopoMatcher

* optimize parse

* Update test/e2e/e2e-base/src/main/java/org/apache/skywalking/e2e/assertor/VariableExpressParser.java
Co-Authored-By: Nkezhenxu94 <kezhenxu94@163.com>