README.md 14.3 KB
Newer Older
L
liuyq-617 已提交
1
[![Build Status](https://cloud.drone.io/api/badges/taosdata/TDengine/status.svg?ref=refs/heads/master)](https://cloud.drone.io/taosdata/TDengine)
2
[![Build status](https://ci.appveyor.com/api/projects/status/kf3pwh2or5afsgl9/branch/master?svg=true)](https://ci.appveyor.com/project/sangshuduo/tdengine-2n8ge/branch/master)
3
[![Coverage Status](https://coveralls.io/repos/github/taosdata/TDengine/badge.svg?branch=develop)](https://coveralls.io/github/taosdata/TDengine?branch=develop)
S
Shuduo Sang 已提交
4
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4201/badge)](https://bestpractices.coreinfrastructure.org/projects/4201)
sangshuduo's avatar
sangshuduo 已提交
5
[![tdengine](https://snapcraft.io//tdengine/badge.svg)](https://snapcraft.io/tdengine)
6

J
Jeff Tao 已提交
7
[![TDengine](TDenginelogo.png)](https://www.taosdata.com)
H
hzcheng 已提交
8

9
English | [简体中文](./README-CN.md)
陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
10
We are hiring, check [here](https://www.taosdata.com/en/careers/)
涛思数据(TDengine)'s avatar
涛思数据(TDengine) 已提交
11

H
hzcheng 已提交
12 13
# What is TDengine?

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
14
TDengine is a high-performance, scalable time-series database with SQL support. Its code including cluster feature is open souece under [GNU AGPL v3.0](http://www.gnu.org/licenses/agpl-3.0.html). Besides the database, it provides caching, stream processing, data data subscription and other functionalities to reduce the complexity and cost of development and operation. It can be widely applied to Internet of Things (IoT), Connected Vehicles, Industrial IoT, DevOps, energy, finance and many other scenarios. It differentiates itself from other TSDBs with the following advanatages.
H
hzcheng 已提交
15

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
16
- **High Peroformance**: TDengine outperforms other time series databases in data ingestion and querying while significantly reducing storage cost and compute costs, with an innovatively designed and purpose-built storage engine. 
H
hzcheng 已提交
17

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
18
- **Scalable**: TDengine provides out-of-box scalability and high-availability through its native distributed design. Nodes can be added through simple configuration to achieve greater data processing power. In addition, this feature is open source.
H
hzcheng 已提交
19

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
20
- **SQL Support**: TDengine uses SQL as the query language, thereby reducing learning and migration costs, while adding SQL extensions to handle time-series data better, and supporting convenient and flexible schemaless data ingestion.
H
hzcheng 已提交
21

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
22
- **All in One**: TDengine has built-in caching, stream processing and data subscription functions, it is no longer necessary to integrate Kafka/Redis/HBase/Spark or other software in some scenarios. It makes the system architecture much simpler and easy to maintain.
H
hzcheng 已提交
23

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
24 25 26
- **Seamless Integration**: Without a single line of code, TDengine provide seamless integration with third-party tools such as Telegraf, Grafana, EMQ X, Prometheus, StatsD, collectd, etc. More will be integrated.
 
- **Zero Management**: Installation and cluster setup can be done in seconds. Data partitioning and sharding are done automatically. TDengine’s running status can be monitored via Grafana or other DevOps tools. 
H
hzcheng 已提交
27

陶建辉(Jeff)'s avatar
陶建辉(Jeff) 已提交
28 29 30
- **Zero Learning Cost**: With SQL as the query language, support for ubiquitous tools like Python, Java, C/C++, Go, Rust, Node.js connectors, there is zero learning cost.

- **Interactive Console**: TDengine provides convenient console access to the database to run ad hoc queries, maintain the database, or manage the cluster without any programming.
H
hzcheng 已提交
31 32

# Documentation
33

X
Xiaxin Li 已提交
34
For user manual, system design and architecture, engineering blogs, refer to [TDengine Documentation](https://www.taosdata.com/en/documentation/)(中文版请点击[这里](https://www.taosdata.com/cn/documentation20/))
S
StoneT2000 已提交
35
 for details. The documentation from our website can also be downloaded locally from *documentation/tdenginedocs-en* or *documentation/tdenginedocs-cn*.
H
hzcheng 已提交
36 37

# Building
38

H
hzcheng 已提交
39
At the moment, TDengine only supports building and running on Linux systems. You can choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) or from the source code. This quick guide is for installation from the source only.
H
hzcheng 已提交
40

41
To build TDengine, use [CMake](https://cmake.org/) 3.0.2 or higher versions in the project directory.
42

43
## Install build dependencies
44

45 46
### Ubuntu 16.04 and above or Debian

47 48
```bash
sudo apt-get install -y gcc cmake build-essential git
49 50
```

51 52
### Ubuntu 14.04

53 54 55 56 57
```bash
sudo apt-get install -y gcc cmake3 build-essential git binutils-2.26
export PATH=/usr/lib/binutils-2.26/bin:$PATH
```

58
To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed.
59

60
To install openjdk-8:
61

62 63 64 65 66
```bash
sudo apt-get install -y openjdk-8-jdk
```

To install Apache Maven:
67

68
```bash
69
sudo apt-get install -y maven
70 71
```

72 73
#### Install build dependencies for taosTools

74
We provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. From TDengine 2.4.0.0, taosBenchmark and taosdump were not released together with TDengine.
75 76 77
By default, TDengine compiling does not include taosTools. You can use 'cmake .. -DBUILD_TOOLS=true' to make them be compiled with TDengine.

To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed.
78

79
```bash
80
sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev pkg-config
81 82
```

83 84
### CentOS 7

85
```bash
86 87 88 89
sudo yum install epel-release
sudo yum update
sudo yum install -y gcc gcc-c++ make cmake3 git
sudo ln -sf /usr/bin/cmake3 /usr/bin/cmake
90 91 92
```

To install openjdk-8:
93

94 95 96 97 98
```bash
sudo yum install -y java-1.8.0-openjdk
```

To install Apache Maven:
99

100 101
```bash
sudo yum install -y maven
J
johnnyhou327 已提交
102
```
103

104 105
### CentOS 8 & Fedora

106 107
```bash
sudo dnf install -y gcc gcc-c++ make cmake epel-release git
J
johnnyhou327 已提交
108
```
109 110

To install openjdk-8:
111

112 113
```bash
sudo dnf install -y java-1.8.0-openjdk
J
johnnyhou327 已提交
114
```
115 116

To install Apache Maven:
117

118 119
```bash
sudo dnf install -y maven
J
johnnyhou327 已提交
120 121
```

122 123 124 125
#### Install build dependencies for taosTools on CentOS

To build the [taosTools](https://github.com/taosdata/taos-tools) on CentOS, the following packages need to be installed.

126
```bash
127
sudo yum install zlib-devel xz-devel snappy-devel jansson-devel pkgconfig libatomic libstdc++-static
128
```
129

130
Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it lead a cmake prompt libsnappy not found. But snappy will works well.
131

132
### Setup golang environment
133

134 135 136
TDengine includes few components developed by Go language. Please refer to golang.org official documentation for golang environment setup.

Please use version 1.14+. For the user in China, we recommend using a proxy to accelerate package downloading.
137

138 139
```
go env -w GO111MODULE=on
140
go env -w GOPROXY=https://goproxy.cn,direct
141 142
```

143 144
## Get the source codes

145
First of all, you may clone the source codes from github:
146

147 148
```bash
git clone https://github.com/taosdata/TDengine.git
B
Bison "goldeagle" Fan 已提交
149
cd TDengine
L
lihui 已提交
150
```
151

152
The connectors for go & grafana and some tools have been moved to separated repositories,
153
so you should run this command in the TDengine directory to install them:
154

155 156 157 158
```bash
git submodule update --init --recursive
```

sangshuduo's avatar
sangshuduo 已提交
159
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
160

161 162 163 164 165
```
[url "git@github.com:"]
    insteadOf = https://github.com/
```

166 167
## Build TDengine

168 169
### On Linux platform

170
```bash
S
Shuduo Sang 已提交
171
mkdir debug && cd debug
H
hzcheng 已提交
172 173
cmake .. && cmake --build .
```
J
John Zhuang 已提交
174

175
Note TDengine 2.3.x.0 and later use a component named 'taosAdapter' to play http daemon role by default instead of the http daemon embedded in the early version of TDengine. The taosAdapter is programmed by go language. If you pull TDengine source code to the latest from an existing codebase, please execute 'git submodule update --init --recursive' to pull taosAdapter source code. Please install go language version 1.14 or above for compiling taosAdapter. If you meet difficulties regarding 'go mod', especially you are from China, you can use a proxy to solve the problem.
176

177 178 179 180 181
```
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
```

sangshuduo's avatar
sangshuduo 已提交
182
The embedded http daemon still be built from TDengine source code by default. Or you can use the following command to choose to build taosAdapter.
183

184
```
sangshuduo's avatar
sangshuduo 已提交
185
cmake .. -DBUILD_HTTP=false
186 187
```

sangshuduo's avatar
sangshuduo 已提交
188
You can use Jemalloc as memory allocator instead of glibc:
189

sangshuduo's avatar
sangshuduo 已提交
190 191 192 193 194
```
apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true
```

195
TDengine build script can detect the host machine's architecture on X86-64, X86, arm64, arm32 and mips64 platform.
196
You can also specify CPUTYPE option like aarch64 or aarch32 too if the detection result is not correct:
L
lihui 已提交
197

H
huili 已提交
198
aarch64:
199

200
```bash
H
huili 已提交
201
cmake .. -DCPUTYPE=aarch64 && cmake --build .
H
huili 已提交
202
```
H
huili 已提交
203 204

aarch32:
205

B
Bison "goldeagle" Fan 已提交
206
```bash
H
huili 已提交
207
cmake .. -DCPUTYPE=aarch32 && cmake --build .
L
lihui 已提交
208 209
```

210
mips64:
211

212 213 214 215
```bash
cmake .. -DCPUTYPE=mips64 && cmake --build .
```

216 217 218
### On Windows platform

If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
219
Please specify "amd64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
220

221
```cmd
222
mkdir debug && cd debug
223
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
224 225 226 227
cmake .. -G "NMake Makefiles"
nmake
```

228 229 230
If you use the Visual Studio 2019 or 2017:

please open a command window by executing "cmd.exe".
231
Please specify "x64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
232 233

```cmd
234 235 236 237 238 239
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```

240
Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:
241

242
```cmd
243 244 245 246 247
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```

248 249 250 251 252 253 254 255 256
### On Mac OS X platform

Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.

```shell
mkdir debug && cd debug
cmake .. && cmake --build .
```

H
Hongze Cheng 已提交
257
# Installing
258

259
After building successfully, TDengine can be installed by: (On Windows platform, the following command should be `nmake install`)
260

261
```bash
262
sudo make install
H
Hongze Cheng 已提交
263
```
264

265
Users can find more information about directories installed on the system in the [directory and files](https://www.taosdata.com/en/documentation/administrator/#Directory-and-Files) section. Since version 2.0, installing from source code will also configure service management for TDengine.
H
Hongze Cheng 已提交
266
Users can also choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) for it.
H
hzcheng 已提交
267

H
Hongze Cheng 已提交
268
To start the service after installation, in a terminal, use:
269

270 271
```bash
sudo systemctl start taosd
H
hzcheng 已提交
272 273
```

J
Jeff Tao 已提交
274
Then users can use the [TDengine shell](https://www.taosdata.com/en/getting-started/#TDengine-Shell) to connect the TDengine server. In a terminal, use:
275

276
```bash
H
hzcheng 已提交
277 278 279
taos
```

J
Jeff Tao 已提交
280
If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
H
hzcheng 已提交
281

282 283
## Install TDengine by apt-get

sangshuduo's avatar
sangshuduo 已提交
284
If you use Debian or Ubuntu system, you can use 'apt-get' command to install TDengine from official repository. Please use following commands to setup:
285 286 287 288 289 290

```
wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add -
echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | sudo tee /etc/apt/sources.list.d/tdengine-stable.list
[Optional] echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list
sudo apt-get update
O
occoder 已提交
291
apt-cache policy tdengine
292 293 294
sudo apt-get install tdengine
```

295 296
## Quick Run

297
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
298

299 300 301 302 303
```bash
./build/bin/taosd -c test/cfg
```

In another terminal, use the TDengine shell to connect the server:
304

305 306 307 308
```bash
./build/bin/taos -c test/cfg
```

309
option "-c test/cfg" specifies the system configuration file directory.
310

H
hzcheng 已提交
311
# Try TDengine
312

J
Jeff Tao 已提交
313
It is easy to run SQL commands from TDengine shell which is the same as other SQL databases.
314

H
hzcheng 已提交
315 316 317 318 319 320 321 322 323 324 325
```sql
create database db;
use db;
create table t (ts timestamp, a int);
insert into t values ('2019-07-15 00:00:00', 1);
insert into t values ('2019-07-15 01:00:00', 2);
select * from t;
drop database db;
```

# Developing with TDengine
326 327

## Official Connectors
328 329

TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation.
H
hzcheng 已提交
330

331 332 333 334 335 336 337
- [Java](https://www.taosdata.com/en/documentation/connector/java)
- [C/C++](https://www.taosdata.com/en/documentation/connector#c-cpp)
- [Python](https://www.taosdata.com/en/documentation/connector#python)
- [Go](https://www.taosdata.com/en/documentation/connector#go)
- [RESTful API](https://www.taosdata.com/en/documentation/connector#restful)
- [Node.js](https://www.taosdata.com/en/documentation/connector#nodejs)
- [Rust](https://www.taosdata.com/en/documentation/connector/rust)
H
hzcheng 已提交
338

339
## Third Party Connectors
340 341 342

The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.

343
- [Rust Bindings](https://github.com/songtianyi/tdengine-rust-bindings/tree/master/examples)
344
- [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos)
345
- [Lua Connector](https://github.com/taosdata/TDengine/tree/develop/tests/examples/lua)
346

347 348
# How to run the test cases and how to add a new test case

sangshuduo's avatar
sangshuduo 已提交
349 350 351
  TDengine's test framework and all test cases are fully open source.
  Please refer to [this document](tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case.

J
Jeff Tao 已提交
352
# TDengine Roadmap
353

S
StoneT2000 已提交
354
- Support event-driven stream computing
J
Jeff Tao 已提交
355 356 357
- Support user defined functions
- Support MQTT connection
- Support OPC connection
S
StoneT2000 已提交
358
- Support Hadoop, Spark connections
J
Jeff Tao 已提交
359 360
- Support Tableau and other BI tools

W
winshining 已提交
361
# Contribute to TDengine
H
hzcheng 已提交
362 363

Please follow the [contribution guidelines](CONTRIBUTING.md) to contribute to the project.
L
lihui 已提交
364 365 366 367 368

# Join TDengine WeChat Group

Add WeChat “tdengine” to join the group,you can communicate with other users.

涛思数据(TDengine)'s avatar
涛思数据(TDengine) 已提交
369 370
# [User List](https://github.com/taosdata/TDengine/issues/2432)

E
Elias Soong 已提交
371
If you are using TDengine and feel it helps or you'd like to do some contributions, please add your company to [user list](https://github.com/taosdata/TDengine/issues/2432) and let us know your needs.