English | [简体中文](README-CN.md) | We are hiring, check [here](https://tdengine.com/careers)
English | [简体中文](README-CN.md) | We are hiring, check [here](https://tdengine.com/careers)
# What is TDengine？
# What is TDengine？
TDengine is an open source, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT. It enables efficient, real-time data ingestion, processing, and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine differentiates itself from other TSDBs with the following advantages.:
TDengine is an open source, high performance , cloud native time-series database (Time-Series Database, TSDB).
TDengine can be optimized for Internet of Things (IoT), Connected Cars, and Industrial IoT, IT operation and maintenance, finance and other fields. In addition to the core time series database functions, TDengine also provides functions such as caching, data subscription, and streaming computing. It is a minimalist time series data processing platform that minimizes the complexity of system design and reduces R&D and operating costs. Compared with other time series databases, the main advantages of TDengine are as follows:
- High-Performance: TDengine is the only time-series database to solve the high cardinality issue to support billions of data collection points while out performing other time-series databases for data ingestion, querying and data compression.
- High-Performance: TDengine is the only time-series database to solve the high cardinality issue to support billions of data collection points while out performing other time-series databases for data ingestion, querying and data compression.
- Simplified Solution: Through built-in caching, stream processing and data subscription features, TDengine provides a simplified solution for time-series data processing. It reduces system design complexity and operation costs significantly.
- Simplified Solution: Through built-in caching, stream processing and data subscription features, TDengine provides a simplified solution for time-series data processing. It reduces system design complexity and operation costs significantly.
- Cloud Native: Through native distributed design, sharding and partitioning, separation of compute and storage, RAFT, support for kubernetes deployment and full observability, TDengine can be deployed on public, private or hybrid clouds.
- Cloud Native: Through native distributed design, sharding and partitioning, separation of compute and storage, RAFT, support for kubernetes deployment and full observability, TDengine is a cloud native Time-Series Database and can be deployed on public, private or hybrid clouds.
- Open Source: TDengine’s core modules, including cluster feature, are all available under open source licenses. It has gathered 18.8k stars on GitHub, an active developer community, and over 137k running instances worldwide.
- Ease of Use: For administrators, TDengine significantly reduces the effort to deploy and maintain. For developers, it provides a simple interface, simplified solution and seamless integrations for third party tools. For data users, it gives easy data access.
- Ease of Use: For administrators, TDengine significantly reduces the effort to deploy and maintain. For developers, it provides a simple interface, simplified solution and seamless integrations for third party tools. For data users, it gives easy data access.
- Easy Data Analytics: Through super tables, storage and compute separation, data partitioning by time interval, pre-computation and other means, TDengine makes it easy to explore, format, and get access to data in a highly efficient way.
- Easy Data Analytics: Through super tables, storage and compute separation, data partitioning by time interval, pre-computation and other means, TDengine makes it easy to explore, format, and get access to data in a highly efficient way.
- Open Source: TDengine’s core modules, including cluster feature, are all available under open source licenses. It has gathered 18.8k stars on GitHub, an active developer community, and over 137k running instances worldwide.
# Documentation
# Documentation
For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.tdengine.com)(中文版请点击[这里](https://docs.taosdata.com))
For user manual, system design and architecture, please refer to [TDengine Documentation](https://docs.taosdata.com)([TDengine 文档](https://docs.taosdata.com))
# Building
# Building
At the moment, TDengine server only supports running on Linux systems. You can choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) or build it from the source code. This quick guide is for installation from the source only.
At the moment, TDengine server supports running on Linux, Windows systems.Any OS application can also choose the RESTful interface of taosAdapter to connect the taosd service . TDengine supports X64/ARM64 CPU , and it will support MIPS64, Alpha64, ARM32, RISC-V and other CPU architectures in the future.
You can choose to install through source code according to your needs, [container](https://docs.taosdata.com/get-started/docker/), [installation package](https://docs.taosdata.com/get-started/package/) or [Kubenetes](https://docs.taosdata.com/deployment/k8s/) to install. This quick guide only applies to installing from source.
TDengine provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. By default, TDengine compiling does not include taosTools. You can use `cmake .. -DBUILD_TOOLS=true` to make them be compiled with TDengine.
To build TDengine, use [CMake](https://cmake.org/) 3.0.2 or higher versions in the project directory.
To build TDengine, use [CMake](https://cmake.org/) 3.0.2 or higher versions in the project directory.
We provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. From TDengine 2.4.0.0, taosBenchmark and taosdump were not released together with TDengine.
By default, TDengine compiling does not include taosTools. You can use 'cmake .. -DBUILD_TOOLS=true' to make them be compiled with TDengine.
To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed.
To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed.
Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it leads a cmake prompt libsnappy not found. But snappy will works well.
Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it leads a cmake prompt libsnappy not found. But snappy still works well.
If the powertools installation fails, you can try to use:
```
sudo yum config-manager --set-enabled Powertools
```
### Setup golang environment
### Setup golang environment
TDengine includes a few components developed by Go language. Please refer to golang.org official documentation for golang environment setup.
TDengine includes a few components like taosAdapter developed by Go language. Please refer to golang.org official documentation for golang environment setup.
Please use version 1.14+. For the user in China, we recommend using a proxy to accelerate package downloading.
Please use version 1.14+. For the user in China, we recommend using a proxy to accelerate package downloading.
...
@@ -99,9 +126,15 @@ go env -w GO111MODULE=on
...
@@ -99,9 +126,15 @@ go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GOPROXY=https://goproxy.cn,direct
```
```
The default will not build taosAdapter, but you can use the following command to build taosAdapter as the service for RESTful interface.
```
cmake .. -DBUILD_HTTP=false
```
### Setup rust environment
### Setup rust environment
TDengine includees a few compoments developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup.
TDengine includes a few compoments developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup.
The connectors for go & Grafana and some tools have been moved to separated repositories.
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You will need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
```
```
[url "git@github.com:"]
[url "git@github.com:"]
insteadOf = https://github.com/
insteadOf = https://github.com/
```
```
## Special Note
[JDBC Connector](https://github.com/taosdata/taos-connector-jdbc)， [Go Connector](https://github.com/taosdata/driver-go)，[Python Connector](https://github.com/taosdata/taos-connector-python)，[Node.js Connector](https://github.com/taosdata/taos-connector-node)，[C# Connector](https://github.com/taosdata/taos-connector-dotnet) ，[Rust Connector](https://github.com/taosdata/taos-connector-rust) and [Grafana plugin](https://github.com/taosdata/grafanaplugin) has been moved to standalone repository.
## Build TDengine
## Build TDengine
### On Linux platform
### On Linux platform
You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
```bash
```bash
...
@@ -140,18 +178,6 @@ cmake .. -DBUILD_TOOLS=true
...
@@ -140,18 +178,6 @@ cmake .. -DBUILD_TOOLS=true
make
make
```
```
Note TDengine 2.3.x.0 and later use a component named 'taosAdapter' to play http daemon role by default instead of the http daemon embedded in the early version of TDengine. The taosAdapter is programmed by go language. If you pull TDengine source code to the latest from an existing codebase, please execute 'git submodule update --init --recursive' to pull taosAdapter source code. Please install go language version 1.14 or above for compiling taosAdapter. If you meet difficulties regarding 'go mod', especially you are from China, you can use a proxy to solve the problem.
```
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
```
The embedded http daemon still be built from TDengine source code by default. Or you can use the following command to choose to build taosAdapter.
```
cmake .. -DBUILD_HTTP=false
```
You can use Jemalloc as memory allocator instead of glibc:
You can use Jemalloc as memory allocator instead of glibc:
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
Please specify "amd64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
```cmd
mkdir debug && cd debug
mkdir debug && cd debug
...
@@ -184,7 +210,7 @@ nmake
...
@@ -184,7 +210,7 @@ nmake
If you use the Visual Studio 2019 or 2017:
If you use the Visual Studio 2019 or 2017:
please open a command window by executing "cmd.exe".
please open a command window by executing "cmd.exe".
Please specify "x64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
Please specify "x64" for 64 bits Windows or specify "x86" for 32 bits Windows when you execute vcvarsall.bat.
```cmd
```cmd
mkdir debug && cd debug
mkdir debug && cd debug
...
@@ -220,8 +246,9 @@ After building successfully, TDengine can be installed by
...
@@ -220,8 +246,9 @@ After building successfully, TDengine can be installed by
sudo make install
sudo make install
```
```
Users can find more information about directories installed on the system in the [directory and files](https://www.taosdata.com/en/documentation/administrator/#Directory-and-Files) section. Since version 2.0, installing from source code will also configure service management for TDengine.
Users can find more information about directories installed on the system in the [directory and files](https://docs.taosdata.com/reference/directory/) section.
Users can also choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) for it.
Installing from source code will also configure service management for TDengine.Users can also choose to [install from packages](https://docs.taosdata.com/get-started/package/) for it.
To start the service after installation, in a terminal, use:
To start the service after installation, in a terminal, use:
...
@@ -229,26 +256,13 @@ To start the service after installation, in a terminal, use:
...
@@ -229,26 +256,13 @@ To start the service after installation, in a terminal, use:
sudo systemctl start taosd
sudo systemctl start taosd
```
```
Then users can use the [TDengine shell](https://www.taosdata.com/en/getting-started/#TDengine-Shell) to connect the TDengine server. In a terminal, use:
Then users can use the TDengine CLI to connect the TDengine server. In a terminal, use:
```bash
```bash
taos
taos
```
```
If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
If TDengine CLI connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
### Install TDengine by apt-get
If you use Debian or Ubuntu system, you can use 'apt-get' command to install TDengine from official repository. Please use following commands to setup:
@@ -274,7 +288,7 @@ If you don't want to run TDengine as a service, you can run it in current shell.
...
@@ -274,7 +288,7 @@ If you don't want to run TDengine as a service, you can run it in current shell.
./build/bin/taosd -ctest/cfg
./build/bin/taosd -ctest/cfg
```
```
In another terminal, use the TDengine shell to connect the server:
In another terminal, use the TDengine CLI to connect the server:
```bash
```bash
./build/bin/taos -ctest/cfg
./build/bin/taos -ctest/cfg
...
@@ -284,7 +298,7 @@ option "-c test/cfg" specifies the system configuration file directory.
...
@@ -284,7 +298,7 @@ option "-c test/cfg" specifies the system configuration file directory.
# Try TDengine
# Try TDengine
It is easy to run SQL commands from TDengine shell which is the same as other SQL databases.
It is easy to run SQL commands from TDengine CLI which is the same as other SQL databases.
```sql
```sql
CREATEDATABASEdemo;
CREATEDATABASEdemo;
...
@@ -304,7 +318,7 @@ Query OK, 2 row(s) in set (0.001700s)
...
@@ -304,7 +318,7 @@ Query OK, 2 row(s) in set (0.001700s)
## Official Connectors
## Official Connectors
TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation.
TDengine provides abundant developing tools for users to develop on TDengine. include C/C++、Java、Python、Go、Node.js、C# 、RESTful ,Follow the links below to find your desired connectors and relevant documentation.
# How to run the test cases and how to add a new test case
TDengine's test framework and all test cases are fully open source.
Please refer to [this document](https://github.com/taosdata/TDengine/blob/develop/tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case.
# Contribute to TDengine
# Contribute to TDengine
Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project.
Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project.
...
@@ -327,7 +336,3 @@ Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to th
...
@@ -327,7 +336,3 @@ Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to th
# Join TDengine WeChat Group
# Join TDengine WeChat Group
Add WeChat “tdengine” to join the group，you can communicate with other users.
Add WeChat “tdengine” to join the group，you can communicate with other users.
If you are using TDengine and feel it helps or you'd like to do some contributions, please add your company to [user list](https://github.com/taosdata/TDengine/issues/2432) and let us know your needs.
It's very easy to install TDengine and would take you only a few minutes from downloading to finishing installation.
TDengine 的安装非常简单，从下载到安装成功仅仅只要几秒钟。
For the convenience of users, from version 2.4.0.10, the standard server side installation package includes `taos`, `taosd`, `taosAdapter`, `taosBenchmark` and sample code. If only the `taosd` server and C/C++ connector are required, you can also choose to download the lite package.
为方便使用，从 2.4.0.10 开始，标准的服务端安装包包含了 taos、taosd、taosAdapter、taosdump、taosBenchmark、TDinsight 安装脚本和示例代码；如果您只需要用到服务端程序和客户端连接的 C/C++ 语言支持，也可以仅下载 lite 版本的安装包。
Three kinds of packages are provided, tar.gz, rpm and deb. Especially the tar.gz package is provided for the convenience of enterprise customers on different kinds of operating systems, it includes `taosdump` and TDinsight installation script which are normally only provided in taos-tools rpm and deb packages.
在安装包格式上，我们提供 tar.gz, rpm 和 deb 格式，为企业客户提供 tar.gz 格式安装包，以方便在特定操作系统上使用。需要注意的是，rpm 和 deb 包不含 taosdump、taosBenchmark 和 TDinsight 安装脚本，这些工具需要通过安装 taosTool 包获得。
Between two major release versions, some beta versions may be delivered for users to try some new features.
For the details please refer to [Install and Uninstall](/operation/pkg-install)。
具体的安装方法，请参见[安装包的安装和卸载](/operation/pkg-install)。
To see the details of versions, please refer to [Download List](https://tdengine.com/all-downloads) and [Release Notes](https://github.com/taosdata/TDengine/releases).
The full package of TDengine includes the server(taosd), taosAdapter for connecting with third-party systems and providing a RESTful interface, client driver(taosc), command-line program(CLI, taos) and some tools. For the current version, the server taosd and taosAdapter can only be installed and run on Linux systems. In the future taosd and taosAdapter will also be supported on Windows, macOS and other systems. The client driver taosc and TDengine CLI can be installed and run on Windows or Linux. In addition to connectors for multiple languages, TDengine also provides a [RESTful interface](/reference/rest-api) through [taosAdapter](/reference/taosadapter). Prior to version 2.4.0.0, taosAdapter did not exist and the RESTful interface was provided by the built-in HTTP service of taosd.
```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useCurrentSidebarCategory} from '@docusaurus/theme-common';
TDengine supports X64/ARM64/MIPS64/Alpha64 hardware platforms, and will support ARM32, RISC-V and other CPU architectures in the future.
If docker is already installed on your computer, execute the following command:
```shell
docker run -d-p 6030-6049:6030-6049 -p 6030-6049:6030-6049/udp tdengine/tdengine
```
Make sure the container is running
```shell
docker ps
```
Enter into container and execute bash
```shell
docker exec-it <container name> bash
```
Then you can execute the Linux commands and access TDengine.
For detailed steps, please visit [Experience TDengine via Docker](/train-faq/docker)。
:::info
Starting from 2.4.0.10，besides taosd，TDengine docker image includes: taos，taosAdapter，taosdump，taosBenchmark，TDinsight, scripts and sample code. Once the TDengine container is started，it will start both taosAdapter and taosd automatically to support RESTful interface.
:::
</TabItem>
<TabItemvalue="apt-get"label="apt-get">
<AptGetInstall/>
</TabItem>
<TabItemvalue="pkg"label="Package">
<PkgInstall/>
</TabItem>
<TabItemvalue="src"label="Source Code">
If you like to check the source code, build the package by yourself or contribute to the project, please check [TDengine GitHub Repository](https://github.com/taosdata/TDengine)
</TabItem>
</Tabs>
## Quick Launch
After installation, you can launch the TDengine service by the 'systemctl' command to start 'taosd'.
```bash
systemctl start taosd
```
Check if taosd is running：
```bash
systemctl status taosd
```
If everything is fine, you can run TDengine command-line interface `taos` to access TDengine and test it out yourself.
:::info
- systemctl requires _root_ privileges，if you are not _root_ ，please add sudo before the command.
- To get feedback and keep improving the product, TDengine is collecting some basic usage information, but you can turn it off by setting telemetryReporting to 0 in configuration file taos.cfg.
- TDengine uses FQDN (usually hostname）as the ID for a node. To make the system work, you need to configure the FQDN for the server running taosd, and configure the DNS service or hosts file on the the machine where the application or TDengine CLI runs to ensure that the FQDN can be resolved.
-`systemctl stop taosd` won't stop the server right away, it will wait until all the data in memory are flushed to disk. It may takes time depending on the cache size.
TDengine supports the installation on system which runs [`systemd`](https://en.wikipedia.org/wiki/Systemd) for process management，use `which systemctl` to check if the system has `systemd` installed:
```bash
which systemctl
```
If the system does not have `systemd`，you can start TDengine manually by executing `/usr/local/taos/bin/taosd`
:::note
## Command Line Interface
To manage the TDengine running instance，or execute ad-hoc queries, TDengine provides a Command Line Interface (hereinafter referred to as TDengine CLI) taos. To enter into the interactive CLI，execute `taos` on a Linux terminal where TDengine is installed.
```bash
taos
```
If it connects to the TDengine server successfully, it will print out the version and welcome message. If it fails, it will print out the error message, please check [FAQ](/train-faq/faq) for trouble shooting connection issue. TDengine CLI's prompt is：
```cmd
taos>
```
Inside TDengine CLI，you can execute SQL commands to create/drop database/table, and run queries. The SQL command must be ended with a semicolon. For example:
```sql
createdatabasedemo;
usedemo;
createtablet(tstimestamp,speedint);
insertintotvalues('2019-07-15 00:00:00',10);
insertintotvalues('2019-07-15 01:00:00',20);
select*fromt;
ts|speed|
========================================
2019-07-1500:00:00.000|10|
2019-07-1501:00:00.000|20|
QueryOK,2row(s)inset(0.003128s)
```
Besides executing SQL commands, system administrators can check running status, add/drop user accounts and manage the running instances. TDengine CLI with client driver can be installed and run on either Linux or Windows machines. For more details on CLI, please [check here](../reference/taos-shell/).
## Experience the blazing fast speed
After TDengine server is running，execute `taosBenchmark` (previously named taosdemo) from a Linux terminal：
```bash
taosBenchmark
```
This command will create a super table "meters" under database "test". Under "meters", 10000 tables are created with names from "d0" to "d9999". Each table has 10000 rows and each row has four columns (ts, current, voltage, phase). Time stamp is starting from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999". Each table has tags "location" and "groupId". groupId is set 1 to 10 randomly, and location is set to "California.SanFrancisco" or "California.SanDiego".
This command will insert 100 million rows into the database quickly. Time to insert depends on the hardware configuration, it only takes a dozen seconds for a regular PC server.
taosBenchmark provides command-line options and a configuration file to customize the scenarios, like number of tables, number of rows per table, number of columns and more. Please execute `taosBenchmark --help` to list them. For details on running taosBenchmark, please check [reference for taosBenchmark](/reference/taosbenchmark)
## Experience query speed
After using taosBenchmark to insert a number of rows data, you can execute queries from TDengine CLI to experience the lightning fast query speed.
query the total number of rows under super table "meters"：
```sql
taos>selectcount(*)fromtest.meters;
```
query the average, maximum, minimum of 100 million rows:
@@ -8,16 +8,13 @@ TDengine provides a rich set of APIs (application development interface). To fac
...
@@ -8,16 +8,13 @@ TDengine provides a rich set of APIs (application development interface). To fac
## Supported platforms
## Supported platforms
Currently, TDengine's native interface connectors can support platforms such as X64/X86/ARM64/ARM32/MIPS/Alpha hardware platforms and Linux/Win64/Win32 development environments. The comparison matrix is as follows.
Currently, TDengine's native interface connectors can support platforms such as X64/ARM64 hardware platforms and Linux/Win64 development environments. The comparison matrix is as follows.
根据元数据信息中的 End Point 信息，将查询请求序列化后发送到该表所在的数据节点（dnode）。dnode 接收到查询请求后，识别出该查询请求指向的虚拟节点（vnode），将消息转发到 vnode 的查询执行队列。vnode 的查询执行线程建立基础的查询执行环境，并立即返回该查询请求，同时开始执行该查询。
