@@ -83,10 +83,10 @@ To make the orchestration process easier, we're using a [docker-compose](https:/
Follow these steps:
1. Decide what (and how many) containers will be needed. For example, for cluster testing, you'll need > 2 OAP nodes, coordinators (e.g. zookeeper), storage (e.g. ElasticSearch), and instrumented services;
1. Define the containers in `docker-compose.yml`, and carefully specify the dependencies, starting orders, and most importantly, link them together, e.g. set the correct OAP address on the agent end, and set the correct coordinator address in OAP, etc.
1. Define the e2e case [config](https://skywalking.apache.org/docs/skywalking-infra-e2e/latest/en/setup/configuration-file/) in `e2e.yaml`.
1. Define the e2e case [config](https://skywalking.apache.org/docs/skywalking-infra-e2e/next/en/setup/configuration-file/) in `e2e.yaml`.
@@ -27,9 +27,8 @@ The value of `x-le` should be in JSON format. There are two options:
{
"logic-span":true
}
```
```
### References
1.[Java plugin API](https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/java-plugin-development-guide/#extension-logic-endpoint-tag-key-x-le) guides users to write plugins with logic endpoint.
2. Java agent's plugins include native included logic endpoints, also it provides ways to set the tag of logic span. The document could be found [here](https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/logic-endpoint/).
1.[Java plugin API](https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/java-plugin-development-guide/#extension-logic-endpoint-tag-key-x-le) guides users to write plugins with logic endpoint.
2. Java agent's plugins include native included logic endpoints, also it provides ways to set the tag of logic span. The document could be found [here](https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/logic-endpoint/).
@@ -24,8 +24,8 @@ Please read SkyWalking language agents documentation to see whether it is suppor
SkyWalking has a native metrics format, and supports widely used metric formats, such as Prometheus, OpenCensus, OpenTelemetry, and Zabbix.
The native metrics format definition could be found [here](https://github.com/apache/skywalking-data-collect-protocol/blob/master/language-agent/Meter.proto).
Typically, the agent meter plugin (e.g. [Java Meter Plugin](https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/java-plugin-development-guide/#meter-plugin)) and
Typically, the agent meter plugin (e.g. [Java Meter Plugin](https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/java-plugin-development-guide/#meter-plugin)) and
would convert metrics into native format and forward them to SkyWalking OAP server.
To learn more about receiving 3rd party formats metrics, see [Meter receiver](../setup/backend/backend-meter.md) and [OpenTelemetry receiver](../setup/backend/opentelemetry-receiver.md).
We could set up environment variables to configure this image. They are defined in [backend-setup](https://skywalking.apache.org/docs/main/latest/en/setup/backend/backend-setup/).
We could set up environment variables to configure this image. They are defined in [backend-setup](https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-setup/).
SkyWalking leverages [SkyWalking Rover](https://github.com/apache/skywalking-rover)[network profiling feature](https://skywalking.apache.org/docs/skywalking-rover/latest/en/setup/configuration/profiling/#network) for collecting metrics data from the network. SkyWalking Rover converts data from socket data to metrics using eBPF technology.
SkyWalking leverages [SkyWalking Rover](https://github.com/apache/skywalking-rover)[network profiling feature](https://skywalking.apache.org/docs/skywalking-rover/next/en/setup/configuration/profiling/#network) for collecting metrics data from the network. SkyWalking Rover converts data from socket data to metrics using eBPF technology.
## Data flow
1. SkyWalking OAP server observes which specific k8s pod needs to monitor the network.
3. The SkyWalking OAP Server accesses K8s's `API Server` to fetch meta info and parses the expression with [MAL](../../concepts-and-designs/mal.md) to aggregate.
@@ -47,7 +47,7 @@ License (SSPL), which is incompatible with Apache License 2.0. This license chan
version 7.11. So please choose the suitable ElasticSearch version according to your usage.
If you have concerns about SSPL, choose the versions before 7.11 or switch to OpenSearch.
Since 9.2.0, SkyWalking provides no-sharding/one-index mode to merge all metrics/meter and records(without super datasets)
Since 9.2.0, SkyWalking provides no-sharding/one-index mode to merge all metrics/meter and records(without super datasets)
indices into one physical index template `metrics-all` and `records-all` on the default setting.
In the current one index mode, users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out.
After merge all indices, the following indices are available:
...
...
@@ -61,7 +61,7 @@ After merge all indices, the following indices are available:
* sw_records-all-`${day-format}`
___
Provide system environment variable(`SW_STORAGE_ES_LOGIC_SHARDING`). Set it to `true` could shard metrics indices into multi-physical indices
Provide system environment variable(`SW_STORAGE_ES_LOGIC_SHARDING`). Set it to `true` could shard metrics indices into multi-physical indices
as same as the versions(one index template per metric/meter aggregation function) before 9.2.0.
___
...
...
@@ -108,7 +108,7 @@ storage:
### ElasticSearch With Https SSL Encrypting communications.
Example:
Example:
```yaml
storage:
...
...
@@ -133,17 +133,17 @@ In most cases, users don't need to change the value manually, as SkyWalking is d
But in some cases, users may want to set a long TTL value, such as more than 60 days. However, their ElasticSearch cluster may not be powerful enough due to low traffic in the production environment.
This value could be increased to 5 (or more) if users could ensure a single index could support the metrics and traces for these days (5 in this case).
For example, if dayStep == 11,
For example, if dayStep == 11,
1. Data in [2000-01-01, 2000-01-11] will be merged into the index-20000101.
1. Data in [2000-01-12, 2000-01-22] will be merged into the index-20000112.
`storage/elasticsearch/superDatasetDayStep` overrides the `storage/elasticsearch/dayStep` if the value is positive.
This would affect the record-related entities, such as trace segments. In some cases, the size of metrics is much smaller than the record (trace). This would improve the shards balance in the ElasticSearch cluster.
NOTE: TTL deletion would be affected by these steps. You should set an extra dayStep in your TTL. For example, if you want to have TTL == 30 days and dayStep == 10, you are recommended to set TTL = 40.
### Secrets Management File Of ElasticSearch Authentication
The value of `secretsManagementFile` should point to the secrets management file absolute path.
The value of `secretsManagementFile` should point to the secrets management file absolute path.
The file includes the username, password, and JKS password of the ElasticSearch server in the properties format.
```properties
user=xxx
...
...
@@ -151,8 +151,8 @@ password=yyy
trustStorePass=zzz
```
The major difference between using `user, password, trustStorePass` configs in the `application.yaml` file is that the **Secrets Management File** is being watched by the OAP server.
Once it is changed manually or through a 3rd party tool, such as [Vault](https://github.com/hashicorp/vault),
The major difference between using `user, password, trustStorePass` configs in the `application.yaml` file is that the **Secrets Management File** is being watched by the OAP server.
Once it is changed manually or through a 3rd party tool, such as [Vault](https://github.com/hashicorp/vault),
the storage provider will use the new username, password, and JKS password to establish the connection and close the old one. If the information exists in the file,
the `user/password` will be overridden.
...
...
@@ -186,9 +186,9 @@ We strongly recommend that you read more about these configurations from Elastic
When a namespace is set, all index names in ElasticSearch will use it as the prefix.
## MySQL
Activate MySQL as storage, and set storage provider to **mysql**.
Activate MySQL as storage, and set storage provider to **mysql**.
**NOTE:** MySQL driver is NOT allowed in Apache official distribution and source codes.
**NOTE:** MySQL driver is NOT allowed in Apache official distribution and source codes.
Please download the MySQL driver on your own. Copy the connection driver jar to `oap-libs`.
All connection-related settings, including URL link, username, and password, are found in `application.yml`.
All connection-related settings, including URL link, username, and password, are found in `application.yml`.
Only part of the settings is listed here. See the [HikariCP](https://github.com/brettwooldridge/HikariCP) connection pool document for full settings.
To understand the function of the parameter `rewriteBatchedStatements=true` in MySQL, see the [MySQL official document](https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-connp-props-performance-extensions.html#cj-conn-prop_rewriteBatchedStatements).
## TiDB
Tested TiDB Server 4.0.8 version, and MySQL Client driver 8.0.13 version is currently available.
Activate TiDB as storage, and set storage provider to **tidb**.
Activate TiDB as storage, and set storage provider to **tidb**.
All connection-related settings, including URL link, username, and password are found in `application.yml`.
All connection-related settings, including URL link, username, and password are found in `application.yml`.
For details on settings, refer to the configuration of *MySQL* above.
To understand the function of the parameter `rewriteBatchedStatements=true` in TiDB, see the document of [TiDB best practices](https://docs.pingcap.com/tidb/stable/java-app-best-practices#use-batch-api).
## PostgreSQL
PostgreSQL JDBC driver uses version 42.3.2. It supports PostgreSQL 8.2 or newer.
Activate PostgreSQL as storage, and set storage provider to **postgresql**.
Activate PostgreSQL as storage, and set storage provider to **postgresql**.
All connection-related settings, including URL link, username, and password, are found in `application.yml`.
All connection-related settings, including URL link, username, and password, are found in `application.yml`.
Only part of the settings is listed here. Please follow [HikariCP](https://github.com/brettwooldridge/HikariCP) connection pool document for full settings.
## BanyanDB
...
...
@@ -281,9 +281,9 @@ storage:
profileTaskQueryMaxSize:${SW_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE:200}# the max number of fetch task in a request
```
For more details, please refer to the documents of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/latest/readme/)
For more details, please refer to the documents of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/)
and [BanyanDB Java Client](https://github.com/apache/skywalking-banyandb-java-client) subprojects.
## More storage extension solutions
Follow the [Storage extension development guide](../../guides/storage-extention.md)
Follow the [Storage extension development guide](../../guides/storage-extention.md)
in the [Project Extensions document](../../guides/README.md#project-extensions).
@@ -6,13 +6,13 @@ format and maximize the analysis capabilities of the SkyWalking OAP server.
## Installing language agents in services
-[Java agent](https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/readme/). Learn how to install the Java agent in your service without affecting your code.
-[Java agent](https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/readme/). Learn how to install the Java agent in your service without affecting your code.
-[LUA agent](https://github.com/apache/skywalking-nginx-lua). Learn how to install the Lua agent in Nginx + LUA module or OpenResty.
-[Kong agent](https://github.com/apache/skywalking-kong). Learn how to install the Lua agent in Kong.
-[Python Agent](https://skywalking.apache.org/docs/skywalking-python/latest/en/setup/cli/). Learn how to install the Python Agent in a Python service without affecting your code.
-[Python Agent](https://skywalking.apache.org/docs/skywalking-python/next/en/setup/cli/). Learn how to install the Python Agent in a Python service without affecting your code.
-[Node.js agent](https://github.com/apache/skywalking-nodejs). Learn how to install the NodeJS Agent in a NodeJS service.
"content":"Satellite: an open-source agent designed for the cloud-native infrastructures, which provides a low-cost, high-efficient, and more secure way to collect telemetry data. It is the recommended load balancer for telemetry collecting.",