提交 f80b0312 编写于 作者: wu-sheng's avatar wu-sheng

Add many new chapters of v6 document. Make the document more clear.

上级 1b0d80c5
# Observability Analysis Platform Cluster Management
OAP(Observability Analysis Platform) backend is a distributed system, services need to find each
other. i.e. a web service needs to find query service, level 1 aggregate service need to find
level 2 aggregate service. Cluster management is just a client-side implementation. It must work
together with a distributed coordination service, (i.e. Zookeeper, Consul, Kubernetes.) unless
running in standalone mode.
## Architecture overview
<img src="https://skywalkingtest.github.io/page-resources/6.0.0-alpha/cluster_management.png"/>
### Cluster Management Plugins
By default, OAP backend provides two implementations for cluster management, which are standalone
and zookeeper. When the scale of services, which are under monitoring, is small, you can choose
the standalone mode. Otherwise, you must choose the cluster mode by zookeeper plugin, or you can
implement a new service discovery plugin for your own scene.
### Cluster Management Interface
There are two interfaces defined beforehand in the OAP server core, which are Module register and
module query, all the cluster management plugins must implement those two interfaces.
* Module Register: When any modules need to provide services for others, the module must do register
through this interface.
* Module Query: When any module needs to find other services, it can use this interface to retrieve
the service instance list in the certain order.
### Process Flow Between Client and Cluster Management
The client has two ways to connect the backend, one, use the direct link by a set of backend instance
endpoint list, or you can use naming service, which considers your given list is just the seed nodes
of the whole cluster. The following graph is showing you how naming service works. You need to check
the probe documents to know which way(s) is(are) supported.
```
Client lib Collector1 Collector2 Collector3
(Set collector.servers=Collector2) (Collector 1,2,3 constitute the cluster)
|
+-----------> naming service ---------------------------->|
|
|<------- receive gRPC IP:Port(s) of Collector 1,2,3---<--|
|
|Select a random gRPC service
|For example collector 3
|
|------------------------->Uplink gRPC service----------------------------------->|
```
\ No newline at end of file
......@@ -158,7 +158,7 @@ SpanLayer is the catalog of span. Here are 5 values:
1. MQ
Component IDs are defined and reserved by SkyWalking project.
For component name/ID extension, please follow [component library setting document](../../backend/Component-library-settings.md).
For component name/ID extension, please follow [component library setting document](../setup/backend/Component-library-settings.md).
## Develop a plugin
### Abstract
......
......@@ -11,6 +11,7 @@ breaking any Apache rule.
SkyWalking project supports many ways to extends existing features. If you are interesting in these ways,
read the following guides.
- [Java agent plugin development guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md).
- [Java agent plugin development guide](Java-Plugin-Development-Guide.md).
This guide helps you to develop SkyWalking agent plugin to support more frameworks. Both open source plugin
and private plugin developer should read this.
\ No newline at end of file
and private plugin developer should read this.
- [Storage extension development guide](storage-extention.md)
# Extend storage
SkyWalking has already provided H2 and ElasticSearch as storage solutions. In this document, you could
learn how to implement a new storage easily.
\ No newline at end of file
......@@ -18,7 +18,7 @@ v2.0
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/TraceSegmentService.proto)
- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
- Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md)
- Span data please refs to [Plugin Development Guide](../guides/Java-Plugin-Development-Guide.md)
- Id and name both exist, please use id if possible.
- operationNameId/operationName
- networkAddress/networkAddressId
......
# Alarm
\ No newline at end of file
# Cluster Management
In many product environments, backend need to support high throughputs and provide HA to keep robustness,
so you should need cluster management always in product env.
Backend provides several ways to do cluster management. Choose the one you need/want.
- [Zookeeper coordinator](#zookeeper-coordinator). Use Zookeeper to let backend detects and communicates
......
# Official OAL scripts
\ No newline at end of file
# Choose receiver
Receiver is a concept in SkyWalking backend. All modules, which are responsible for receiving telemetry
or tracing data from other being monitored system, are all being called **Receiver**. Although today, most of
receivers are using gRPC or HTTPRestful to provide service, actually, whether listening mode or pull mode
could be receiver. Such as a receiver could base on pull data from remote, like Kakfa MQ.
We have following receivers
1.
\ No newline at end of file
......@@ -27,17 +27,38 @@ core:
At the same time, modules includes required and optional, the required modules provide the skeleton of backend,
even modulization supported pluggable, remove those modules are meanless. We highly recommend you don't try to
switch those modules, unless you are well known SkyWalking project and its codes.
change APIs of those modules, unless you understand SkyWalking project and its codes very well.
List the required modules here
1. **core**
1. **cluster**
1. **storage**
1. **query**
1. **Core**. Do basic and major skeleton of all data analysis and stream dispatch.
1. **Cluster**. Manage multiple backend instances in a cluster, which could provide high throughputs process
capabilities.
1. **Storage**. Make the analysis result persistence.
1. **Query**. Provide query interfaces to UI.
For **Cluster** and **Storage** have provided multiple implementors(providers), see **Cluster management**
and **Choose storage** documents in the [link list](#advanced-feature-document-link-list).
Also, several **receiver** modules are provided.
Receiver is the module in charge of accepting incoming data requests to backend. Most(all) provide
service by some network(RPC) protocol, such as gRPC, HTTPRestful.
The receivers have many different module names, you could
read **Set receivers** document in the [link list](#advanced-feature-document-link-list).
## Advanced feature document link list
After understand the setting file structure, you could choose your interesting feature document.
We recommend you to read the feature documents in our following list.
We recommend you to read the feature documents in our following order.
1. [IP and port setting](backend-ip-port.md). Introduce how IP and port set and be used.
1. [Cluster management](backend-cluster.md). Guide you to set backend server in cluster mode.
1. [Deploy in kubernetes](backend-k8s.md). Guide you to build and use SkyWalking image, and deploy in k8s.
1. [Choose storage](backend-storage.md). As we know, in default quick start, backend is running with H2
DB. But clearly, it doesn't fit the product env. In here, you could find what other choices do you have.
Choose the one you like, we are also welcome anyone to contribute new storage implementor,
1. [Set receivers](backend-receivers.md). You could choose receivers by your requirements, most receivers
are harmless, at least our default receivers are. You would set and active all receivers provided.
1. Official [OAL scripts](backend-oal-scripts.md). As you known from our [OAL introduction](../../concepts-and-designs/oal.md),
most of backend analysis capabilities based on the scripts. Here is the description of official scripts,
which helps you to understand which metric data are in process, also could be used in alarm.
1. [Alarm](backend-alarm.md). Alarm provides a time-series based check mechanism. You could set alarm
rules targeting the analysis oal metric objects.
# Backend storage
SkyWalking storage is pluggable, we have provided the following storage solutions, you could easily
use is by changing the `application.yml`
- [**H2**](#h2)
- [**ElasticSearch 6**](#elasticsearch-6)
## H2
Active ElasticSearch 6 as storage, set storage provider to **H2**.
Setting fragment example
```yaml
```
## ElasticSearch 6
Active ElasticSearch 6 as storage, set storage provider to **elasticsearch**.
Setting fragment example
```yaml
storage:
elasticsearch:
clusterNodes: localhost:9200
indexShardsNumber: 2
indexReplicasNumber: 0
# Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html
bulkActions: 2000 # Execute the bulk every 2000 requests
bulkSize: 20 # flush the bulk every 20mb
flushInterval: 10 # flush the bulk every 10 seconds whatever the number of requests
concurrentRequests: 2 # the number of concurrent requests
# Set a timeout on metric data. After the timeout has expired, the metric data will automatically be deleted.
traceDataTTL: 90 # Unit is minute
minuteMetricDataTTL: 90 # Unit is minute
hourMetricDataTTL: 36 # Unit is hour
dayMetricDataTTL: 45 # Unit is day
monthMetricDataTTL: 18 # Unit is month
```
## More storage solution extension
Follow [Storage extension development guide](../../guides/storage-extention.md)
in [Project Extensions document](../../guides/README.md#project-extensions) in development guide.
\ No newline at end of file
......@@ -35,7 +35,7 @@ data to backend. Also, jetty service supported in JSON format.
- UI uses GraphQL(HTTP) query to access backend also in Jetty service(`core/default/rest*` in `application.yml`).
Now, let's go the backend and UI setting documents.
- [Backend setting document](backend-setup.md)
- [UI setting document](ui-setup.md)
- [Backend setup document](backend-setup.md)
- [UI setup document](ui-setup.md)
\ No newline at end of file
......@@ -70,7 +70,7 @@ Now, we have the following known plugins.
## Plugin Development Guide
SkyWalking java agent supports plugin to extend [the supported list](Supported-list.md). Please follow
our [Plugin Development Guide](Plugin-Development-Guide.md).
our [Plugin Development Guide](../../../guides/Java-Plugin-Development-Guide.md).
# Test
If you are interested in plugin compatible tests or agent performance, see the following reports.
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册