diff --git a/docs/README.md b/docs/README.md index 01f1347b7ac9c47568595ff641e21b23cb864876..4c0a176af9786c76d05bc16ebe482dc5ec9066ca 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,7 @@ # Welcome **SkyWalking 6 documents and codes are still on building, Status: WIP. Of course, we welcome everyone to join us.** -From here you can learn all about **SkyWalking**’s architecture, how to deploy and use SkyWalking and -resources related to the project. +From here you can learn all about **SkyWalking**’s architecture, how to deploy and use SkyWalking, even develop something based on SkyWalking. - [Concepts and Designs](en/concepts-and-designs/README.md). Concepts and designs explain the most important core ideas about SkyWalking. You can learn from here if you want to understand what is going on under our cool features and visualization. @@ -18,7 +17,7 @@ you are facing a problem, check here first. In addition, you might find these links interesting: -- The latest and old releases are all available in [Apache SkyWalking release page](http://skywalking.apache.org/downloads/) and [CHANGELOG](../CHANGES.md). +- The latest and old releases are all available in [Apache SkyWalking release page](http://skywalking.apache.org/downloads/). The change logs are [here](../CHANGES.md). - You can find the speaking schedules at Conf, online videos and articles about SkyWalking in [Community resource catalog](https://github.com/OpenSkywalking/Community). Also, if you have some related to us, welcome to submit a pull request to add. - We’re always looking for help improving our documentation and codes, so please don’t hesitate to [file an issue](https://github.com/apache/incubator-skywalking/issues/new) diff --git a/docs/en/concepts-and-designs/README.md b/docs/en/concepts-and-designs/README.md index 05d88b849b53e8b58f7fb3f5a3d6b0897a4fb58c..15e9cacc2ee9957916529d75e902f27e873050af 100644 --- a/docs/en/concepts-and-designs/README.md +++ b/docs/en/concepts-and-designs/README.md @@ -2,15 +2,23 @@ Concepts and Designs help you to learn and understand the SkyWalking and the landscape. - What is SkyWalking? - - [Overview](overview.md). Provides a high-level description and introduction, including the problems it solves. + - [Overview and Core concepts](overview.md). Provides a high-level description and introduction, including the problems it solves. - [Project Goals](project-goals.md). Provides the goals, which SkyWalking is trying to focus and provide features about them. + +After you read the above documents, you should understand the SkyWalking basic goals. Now, you can choose which following parts +you are interested, then dive in. + - Probe - [Introduction](probe-introduction.md). Lead readers to understand what the probe is, how many different probes existed and why need them. - [Service auto instrument agent](service-agent.md). Introduce what the auto instrument agents do and which languages does SkyWalking already support. - [Manual instrument SDK](manual-sdk.md). Introduce the role of the manual instrument SDKs in SkyWalking ecosystem. -- Observability Analysis Platform Backend - - [Overview](oap-overview.md). Provides a high level introduction about the OAP backend. + - [Service Mesh probe](service-mesh-probe.md). Introduce why and how SkyWalking receive telemetry data from Service mesh and proxy probe. + - [Zipkin receiver](zipkin.md). Helps you to know the purposes of receiving other libs data. Also how to process this kind of data format. +- Backend + - [Overview](backend-overview.md). Provides a high level introduction about the OAP backend. - [Observability Analysis Language](oal.md). Introduces the core languages, which is designed for aggregation behaviour definition. - [Query in OAP](query-oap.md). Introduces how to query data from backend, whether a simple metric data or topology map, even trace or log. +- UI + - [Overview](ui-overview.md). diff --git a/docs/en/concepts-and-designs/oap-overview.md b/docs/en/concepts-and-designs/backend-overview.md similarity index 89% rename from docs/en/concepts-and-designs/oap-overview.md rename to docs/en/concepts-and-designs/backend-overview.md index e7507b5f6691df33e0f7249f0d43c758b00b4d33..3dfb40d2a7629adccee67813e42560f60d546798 100644 --- a/docs/en/concepts-and-designs/oap-overview.md +++ b/docs/en/concepts-and-designs/backend-overview.md @@ -3,9 +3,7 @@ OAP(Observability Analysis Platform) is a new concept, which starts in SkyWalkin old SkyWalking whole backend. The capabilities of the platform are following. ## OAP capabilities - - -In SkyWalking 6 series, OAP accepts data from more sources, which belongs two groups: **Tracing** and **Metric**. +OAP accepts data from more sources, which belongs two groups: **Tracing** and **Metric**. - **Tracing**. Including, SkyWalking native data formats. Zipkin v1,v2 data formats and Jaeger data formats. - **Metric**. SkyWalking integrates with Service Mesh platforms, such as Istio, Envoy, Linkerd, to provide observability from data panel diff --git a/docs/en/concepts-and-designs/overview.md b/docs/en/concepts-and-designs/overview.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b442b9e4616459b33bff6061eb34ea575c90410f 100644 --- a/docs/en/concepts-and-designs/overview.md +++ b/docs/en/concepts-and-designs/overview.md @@ -0,0 +1,45 @@ +# Overview +SkyWalking: an open source observability platform to collect, analysis, aggregate and visualize data from services and cloud native +infrastructures. SkyWalking provide an easy way to keep you have clear view about your distributed system, even across Cloud. +It is more like an modern APM, especially designed for cloud native, container based and distributed system. + +## Why use SkyWalking? +SkyWalking provides solutions for observing and monitoring distributed system, in many different scenarios. First of all, +like traditional ways, SkyWalking provides auto instrument agents for service, such as Java, C# +and NodeJS. At same time, provide manual instrument SDKs for Go(Not yet), C++(Not yet). +Also with more languages required, risks in manipulating codes at runtime, cloud native infrastructures grow +more powerful, SkyWalking could use Service Mesher infra probes to collect data for understanding the whole distributed system. +In generally, it provides observability capabilities for **service**(s), **service instance**(s), **endpoint**(s). + +Service, instance and endpoint concepts are using everywhere in today, so let's be clear what we mean these in SkyWalking first. + +- **Service**. Represent a set/group of workloads to provide same behaviours for incoming requests. You can define the service +name when you are using instrument agents or SDKs. Or SkyWalking use the name you defined in platform, such as from Istio. +- **Service Instance**. Each one workload in the Service group, named as instance. Like `pods` in Kubernetes, this does't need +to be a single process in OS. Also if you are using instrument agents, instance is actually a real process in OS. +- **Endpoint**. It is a path in the certain service for incoming requests, such as HTTP URI path or gRPC service class + method +signature. + +By using SkyWalking, you can have the topology relationship between Services and Endpoints, metrics of every Service, Service +Instance and Endpoint and set alarm rules. + +In addition to all there, you can have distributed tracing from SkyWalking native agents and SDKs, also by using Zipkin, Jaeger +and OpenCensus. + +## Architecture +The SkyWalking is logically split into four parts: Probes, Platform backend, Storage, UI. + + + +- The **Probe**s are different based on different sources. They collects data and reformat them in SkyWalking requirements. +- THe **Platform backend** is a backend, supported cluster for sure. It is for aggregation, analysis and driving process flow +from probe to UI. Also it provide the pluggable capabilities for incoming formats(like Zipkin's), storage implementors and cluster +management. You even can customize aggregation and analysis by using [Observability Analysis Language](oal.md). +- The **Storage** is open. You can, either choose an existed implementor, such as ElasticSearch, H2 or MySQL cluster managed by +Sharding-Sphere, or implement your own. Of source, we are very welcome to have your contributions for new storage implementors. +- THe **UI** is cool and very powerful for end user of SkyWalking platform. UI is customizable too, because +you need match the backend if you do customize too. + + +## What is next? +- Learn SkyWalking's [Project Goals](project-goals.md) \ No newline at end of file diff --git a/docs/en/concepts-and-designs/probe-introduction.md b/docs/en/concepts-and-designs/probe-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..e439f4c08b9e75080043914b8d9ed1f34559ad55 --- /dev/null +++ b/docs/en/concepts-and-designs/probe-introduction.md @@ -0,0 +1,36 @@ +# Probe Introduction +In SkyWalking, probe means an agent or SDK library integrated into target system, which take charge of +collecting telemetry data including tracing and metric. Based on the target system tech stack, probe could use very different +ways to do so. But ultimately they are same, just collect and reformat data, then send to backend. + +In high level, there are three typical groups in all SkyWalking probes. +- **Language based native agent**. This kind of agents runs in target service user space, like a part of user codes. Such as +SkyWalking Java agent, use `-javaagent` commend line argument to manipulate codes in runtime, `manipulate` means change and inject +user's codes. Another kind of agents is using some hook or intercept mechanism provided by target libraries. So you can see, these kinds +of agents based on languages and libraries. + +- **Service Mesh probe**. Service Mesh probe collects data from sidecar, control panel in service mesh or proxy. In old days, proxy +is only used as ingress of the whole cluster, but with the Service Mesh and sidecar, now we can do observe based on that. + +- **3rd-party instrument library**. SkyWalking accepts other popular used instrument libraries data format. It analysis the +data, transfer it to SkyWalking formats of trace, metric or both. This feature starts with accepting Zipkin span data. See +[Zipkin receiver](zipkin.md) to know more. + +You don't need to use **Language based native agent** and **Service Mesh probe** at the same time, because they both collect +metric data. As a result of that, your system suffers twice payloads, and the analytic numbers are doubled. + +There are several recommend ways in using these probes: +1. Use **Language based native agent** only. +1. Use **3rd-party instrument library** only, like Zipkin instrument ecosystem. +1. Use **Service Mesh probe** only. +1. Use **Service Mesh probe** with **Language based native agent** or **3rd-party instrument library** in tracing status. (Advanced usage) + +In addition, let's example what is the meaning of **in tracing status**? + +In default, **Language based native agent** and **3rd-party instrument library** both send distributed traces to backend, +which do analysis and aggregate on those traces. **In tracing status** means, backend considers these traces as something +like logs, just save them, and build the links between traces and metrics, like `which endpoint and service does the trace belong?`. + +## What is next? +After understand the probe, read [backend overview](backend-overview.md) for understanding analysis and persistence. + diff --git a/docs/en/concepts-and-designs/project-goals.md b/docs/en/concepts-and-designs/project-goals.md new file mode 100644 index 0000000000000000000000000000000000000000..1af038cf8e4dc0efe042cf41df05328783cfdd27 --- /dev/null +++ b/docs/en/concepts-and-designs/project-goals.md @@ -0,0 +1,41 @@ +# Design Goals +The document outlines the core design goals for SkyWalking project. + +- **Keep Observability**. No matter how does the target system deploy, SkyWalking could provide a solution or +integration way to keep observability for it. Based on this, SkyWalking provides several runtime forms and probes. + +- **Topology, Metric and Trace Together**. The first step to see and understand a distributed system should be +from topology map. It visualizes the whole complex system as an easy map. Under that topology, OSS people requires +more about metric for service, instance, endpoint and calls. Trace exists as detail logs for making sense of those metric. +Such as when endpoint latency becomes long, you want to see the slowest the trace to find out why. So you can see, +they are from big picture to details, they are all needed. SkyWalking integrates and provides a lot of features to +make this possible and easy understand. + +- **Light Weight**. There two parts of light weight are needed. (1) In probe, we just depend on network +communication framework, prefer gRPC. By that, the probe should be as small as possible, to avoid the library +conflicts and the payload of VM, such as permsize requirement in JVM. +(2) As an observability platform, it is secondary and third level system in your project environment. +So we are using our own light weight framework to build the backend core. Then you don't need to +deploy big data tech platform and maintain them. SkyWalking should be simple in tech stack. + +- **Pluggable**. SkyWalking core team provides many default implementations, but definitely it is not enough, +and also don't fit every scenario. So, we provide a lot of features for being pluggable. + +- **Portability**. SkyWalking can run in multiple environments, including: +(1) Use traditional register center like eureka. +(2) Use RPC framework including service discovery, like Spring Cloud, Apache Dubbo. +(3) Use Service Mesh in modern infrastructure. +(4) Use cloud services. +(5) Across cloud deployment. +SkyWalking should run well in all these cases. + +- **Interop**. Observability is a big landscape, SkyWalking is impossible to support all, even by its community. +As that, it supports to interop with other OSS system, mostly probes, such as Zipkin, Jaeger, OpenTracing, OpenCensus. +To accept and understand their data formats makes sure SkyWalking more useful for end users. And don't require +the users to switch their libraries. + + +## What is next? +- See [probe Introduction](probe-introduction.md) to know SkyWalking's probe groups. +- From [backend overview](backend-overview.md), you can understand what backend does after it received probe data. +- If you want to customize UI, start with [UI overview](ui-overview.md) document. \ No newline at end of file diff --git a/docs/en/concepts-and-designs/service-agent.md b/docs/en/concepts-and-designs/service-agent.md new file mode 100644 index 0000000000000000000000000000000000000000..56213847c594a03b3898c260077d543221b784b9 --- /dev/null +++ b/docs/en/concepts-and-designs/service-agent.md @@ -0,0 +1,2 @@ +# Service Auto Instrument Agent +Service auto instrument agent is a subset of Language based native agents. \ No newline at end of file diff --git a/docs/en/concepts-and-designs/zipkin.md b/docs/en/concepts-and-designs/zipkin.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391