+
++
+
+
+
+
+ [](https://travis-ci.org/taosdata/TDengine) [](https://ci.appveyor.com/project/sangshuduo/tdengine-2n8ge/branch/master) [](https://coveralls.io/github/taosdata/TDengine?branch=develop) [](https://bestpractices.coreinfrastructure.org/projects/4201) [](https://snapcraft.io/tdengine) -[](https://www.taosdata.com) - -简体中文 | [English](./README.md) +简体中文 | [English](README.md) | 很多职位正在热招中，请看[这里](https://www.taosdata.com/cn/careers/) # TDengine 简介 -TDengine是涛思数据专为物联网、车联网、工业互联网、IT运维等设计和优化的大数据平台。除核心的快10倍以上的时序数据库功能外，还提供缓存、数据订阅、流式计算等功能，最大程度减少研发和运维的复杂度，且核心代码，包括集群功能全部开源（开源协议，AGPL v3.0）。 +TDengine 是一款高性能、分布式、支持 SQL 的时序数据库（Time-Series Database）。而且除时序数据库功能外，它还提供缓存、数据订阅、流式计算等功能，最大程度减少研发和运维的复杂度，且核心代码，包括集群功能全部开源（开源协议，AGPL v3.0）。与其他时序数据数据库相比，TDengine 有以下特点： + +- **高性能**：通过创新的存储引擎设计，无论是数据写入还是查询，TDengine 的性能比通用数据库快 10 倍以上，也远超其他时序数据库，而且存储空间也大为节省。 + +- **分布式**：通过原生分布式的设计，TDengine 提供了水平扩展的能力，只需要增加节点就能获得更强的数据处理能力，同时通过多副本机制保证了系统的高可用。 + +- **支持 SQL**：TDengine 采用 SQL 作为数据查询语言，减少学习和迁移成本，同时提供 SQL 扩展来处理时序数据特有的分析，而且支持方便灵活的 schemaless 数据写入。 + +- **All in One**：将数据库、消息队列、缓存、流式计算等功能融合一起，应用无需再集成 Kafka/Redis/HBase/Spark 等软件，大幅降低应用开发和维护成本。 + +- **零管理**：安装、集群几秒搞定，无任何依赖，不用分库分表，系统运行状态监测能与 Grafana 或其他运维工具无缝集成。 + +- **零学习成本**：采用 SQL 查询语言，支持 Python、Java、C/C++、Go、Rust、Node.js 等多种编程语言，与 MySQL 相似，零学习成本。 + +- **无缝集成**：不用一行代码，即可与 Telegraf、Grafana、EMQX、Prometheus、StatsD、collectd、Matlab、R 等第三方工具无缝集成。 -- 10 倍以上性能提升。定义了创新的数据存储结构，单核每秒就能处理至少2万次请求，插入数百万个数据点，读出一千万以上数据点，比现有通用数据库快了十倍以上。 -- 硬件或云服务成本降至1/5。由于超强性能，计算资源不到通用大数据方案的1/5；通过列式存储和先进的压缩算法，存储空间不到通用数据库的1/10。 -- 全栈时序数据处理引擎。将数据库、消息队列、缓存、流式计算等功能融合一起，应用无需再集成Kafka/Redis/HBase/Spark等软件，大幅降低应用开发和维护成本。 -- 强大的分析功能。无论是十年前还是一秒钟前的数据，指定时间范围即可查询。数据可在时间轴上或多个设备上进行聚合。即席查询可通过Shell/Python/R/Matlab随时进行。 -- 与第三方工具无缝连接。不用一行代码，即可与Telegraf, Grafana, EMQ X, Prometheus, Matlab, R集成。后续还将支持MQTT, OPC, Hadoop，Spark等, BI工具也将无缝连接。 -- 零运维成本、零学习成本。安装、集群一秒搞定，无需分库分表，实时备份。标准SQL，支持JDBC,RESTful，支持Python/Java/C/C++/Go/Node.JS, 与MySQL相似，零学习成本。 +- **互动 Console**: 通过命令行 console，不用编程，执行 SQL 语句就能做即席查询、各种数据库的操作、管理以及集群的维护. + +TDengine 可以广泛应用于物联网、工业互联网、车联网、IT 运维、能源、金融等领域，让大量设备、数据采集器每天产生的高达 TB 甚至 PB 级的数据能得到高效实时的处理，对业务的运行状态进行实时的监测、预警，从大数据中挖掘出商业价值。 # 文档 -TDengine是一个高效的存储、查询、分析时序大数据的平台，专为物联网、车联网、工业互联网、运维监测等优化而设计。您可以像使用关系型数据库MySQL一样来使用它，但建议您在使用前仔细阅读一遍下面的文档，特别是 [数据模型](https://www.taosdata.com/cn/documentation/architecture) 与 [数据建模](https://www.taosdata.com/cn/documentation/model)。除本文档之外，欢迎 [下载产品白皮书](https://www.taosdata.com/downloads/TDengine%20White%20Paper.pdf)。 +TDengine 采用传统的关系数据库模型，您可以像使用关系型数据库 MySQL 一样来使用它。但由于引入了超级表，一个采集点一张表的概念，建议您在使用前仔细阅读一遍下面的文档，特别是 [数据模型](https://www.taosdata.com/cn/documentation/architecture) 与 [数据建模](https://www.taosdata.com/cn/documentation/model)。除本文档之外，欢迎 [下载产品白皮书](https://www.taosdata.com/downloads/TDengine%20White%20Paper.pdf)。 # 构建 -TDengine目前2.0版服务器仅能在Linux系统上安装和运行，后续会支持Windows、macOS等系统。客户端可以在Windows或Linux上安装和运行。任何OS的应用也可以选择RESTful接口连接服务器taosd。CPU支持X64/ARM64/MIPS64/Alpha64，后续会支持ARM32、RISC-V等CPU架构。用户可根据需求选择通过[源码](https://www.taosdata.com/cn/getting-started/#通过源码安装)或者[安装包](https://www.taosdata.com/cn/getting-started/#通过安装包安装)来安装。本快速指南仅适用于通过源码安装。 +TDengine 目前 2.0 版服务器仅能在 Linux 系统上安装和运行，后续会支持 Windows、macOS 等系统。客户端可以在 Windows 或 Linux 上安装和运行。任何 OS 的应用也可以选择 RESTful 接口连接服务器 taosd。CPU 支持 X64/ARM64/MIPS64/Alpha64，后续会支持 ARM32、RISC-V 等 CPU 架构。用户可根据需求选择通过[源码](https://www.taosdata.com/cn/getting-started/#通过源码安装)或者[安装包](https://www.taosdata.com/cn/getting-started/#通过安装包安装)来安装。本快速指南仅适用于通过源码安装。 ## 安装工具 ### Ubuntu 16.04 及以上版本 & Debian： ```bash -sudo apt-get install -y gcc cmake build-essential git +sudo apt-get install -y gcc cmake build-essential git libssl-dev ``` ### Ubuntu 14.04： @@ -56,10 +77,22 @@ sudo apt-get install -y openjdk-8-jdk sudo apt-get install -y maven ``` +#### 为 taos-tools 安装编译需要的软件 + +taosTools 是用于 TDengine 的辅助工具软件集合。目前它包含 taosBenchmark（曾命名为 taosdemo）和 taosdump 两个软件。 + +默认 TDengine 编译不包含 taosTools。您可以在编译 TDengine 时使用`cmake .. -DBUILD_TOOLS=true` 来同时编译 taosTools。 + +为了在 Ubuntu/Debian 系统上编译 [taos-tools](https://github.com/taosdata/taos-tools) 需要安装如下软件： + +```bash +sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev pkg-config +``` + ### CentOS 7： ```bash -sudo yum install -y gcc gcc-c++ make cmake git +sudo yum install -y gcc gcc-c++ make cmake git openssl-devel ``` 安装 OpenJDK 8： @@ -74,10 +107,10 @@ sudo yum install -y java-1.8.0-openjdk sudo yum install -y maven ``` -### CentOS 8 & Fedora: +### CentOS 8 & Fedora ```bash -sudo dnf install -y gcc gcc-c++ make cmake epel-release git +sudo dnf install -y gcc gcc-c++ make cmake epel-release git openssl-devel ``` 安装 OpenJDK 8： @@ -92,6 +125,33 @@ sudo dnf install -y java-1.8.0-openjdk sudo dnf install -y maven ``` +#### 在 CentOS 上构建 taosTools 安装依赖软件 + +为了在 CentOS 上构建 [taosTools](https://github.com/taosdata/taos-tools) 需要安装如下依赖软件 + +```bash +sudo yum install zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel +``` + +注意：由于 snappy 缺乏 pkg-config 支持 +（参考 [链接](https://github.com/google/snappy/pull/86)），会导致 +cmake 提示无法发现 libsnappy，实际上工作正常。 + +### 设置 golang 开发环境 + +TDengine 包含数个使用 Go 语言开发的组件，请参考 golang.org 官方文档设置 go 开发环境。 + +请使用 1.14 及以上版本。对于中国用户，我们建议使用代理来加速软件包下载。 + +``` +go env -w GO111MODULE=on +go env -w GOPROXY=https://goproxy.cn,direct +``` + +### 设置 rust 开发环境 + +TDengine 包含数个使用 Rust 语言开发的组件. 请参考 rust-lang.org 官方文档设置 rust 开发环境。 + ## 获取源码 首先，你需要从 GitHub 克隆源码： @@ -107,22 +167,41 @@ Go 连接器和 Grafana 插件在其他独立仓库，如果安装它们的话 git submodule update --init --recursive ``` +如果使用 https 协议下载比较慢，可以通过修改 ~/.gitconfig 文件添加以下两行设置使用 ssh 协议下载。需要首先上传 ssh 密钥到 GitHub，详细方法请参考 GitHub 官方文档。 + +``` +[url "git@github.com:"] + insteadOf = https://github.com/ +``` + ## 构建 TDengine ### Linux 系统 +可以运行代码仓库中的 `build.sh` 脚本编译出 TDengine 和 taosTools（包含 taosBenchmark 和 taosdump）。 + ```bash -mkdir debug && cd debug -cmake .. && cmake --build . +./build.sh ``` -您可以选择使用 Jemalloc 作为内存分配器，替代默认的 glibc： +这个脚本等价于执行如下命令： + +```bash +git submodule update --init --recursive +mkdir debug +cd debug +cmake .. -DBUILD_TOOLS=true +make +``` + +您也可以选择使用 jemalloc 作为内存分配器，替代默认的 glibc： + ```bash apt install autoconf cmake .. -DJEMALLOC_ENABLED=true ``` -在X86-64、X86、arm64、arm32 和 mips64 平台上，TDengine 生成脚本可以自动检测机器架构。也可以手动配置 CPUTYPE 参数来指定 CPU 类型，如 aarch64 或 aarch32 等。 +在 X86-64、X86、arm64、arm32 和 mips64 平台上，TDengine 生成脚本可以自动检测机器架构。也可以手动配置 CPUTYPE 参数来指定 CPU 类型，如 aarch64 或 aarch32 等。 aarch64： @@ -157,7 +236,7 @@ nmake 如果你使用的是 Visual Studio 2019 或 2017 版本： -打开cmd.exe，执行 vcvarsall.bat 时，为 64 位操作系统指定“x64”，为 32 位操作系统指定“x86”。 +打开 cmd.exe，执行 vcvarsall.bat 时，为 64 位操作系统指定“x64”，为 32 位操作系统指定“x86”。 ```bash mkdir debug && cd debug @@ -174,9 +253,7 @@ cmake .. -G "NMake Makefiles" nmake ``` -如果你使用的是 Visual Studio 2022 版本， 脚本 `vcvarsall.bat` 的默认安装路径是 `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat`。 - -### Mac OS X 系统 +### macOS 系统 安装 Xcode 命令行工具和 cmake. 在 Catalina 和 Big Sur 操作系统上，需要安装 XCode 11.4+ 版本。 @@ -187,13 +264,17 @@ cmake .. && cmake --build . # 安装 -生成完成后，安装 TDengine（下文给出的指令以 Linux 为例，如果是在 Windows 下，那么对应的指令会是 `nmake install`）： +## Linux 系统 + +生成完成后，安装 TDengine： ```bash sudo make install ``` 用户可以在[文件目录结构](https://www.taosdata.com/cn/documentation/administrator#directories)中了解更多在操作系统中生成的目录或文件。 +从 2.0 版本开始, 从源代码安装也会为 TDengine 配置服务管理。 +用户也可以选择[从安装包中安装](https://www.taosdata.com/en/getting-started/#Install-from-Package)。 安装成功后，在终端中启动 TDengine 服务： @@ -209,6 +290,40 @@ taos 如果 TDengine Shell 连接服务成功，将会打印出欢迎消息和版本信息。如果失败，则会打印出错误消息。 +## Windows 系统 + +生成完成后，安装 TDengine： + +```cmd +nmake install +``` + +## macOS 系统 + +生成完成后，安装 TDengine： + +```bash +sudo make install +``` + +安装成功后，如果想以服务形式启动，先配置 `.plist` 文件，在终端中执行： + +```bash +sudo cp ../packaging/macOS/com.taosdata.tdengine.plist /Library/LaunchDaemons +``` + +在终端中启动 TDengine 服务： + +```bash +sudo launchctl load /Library/LaunchDaemons/com.taosdata.tdengine.plist +``` + +在终端中停止 TDengine 服务： + +```bash +sudo launchctl unload /Library/LaunchDaemons/com.taosdata.tdengine.plist +``` + ## 快速运行 如果不希望以服务方式运行 TDengine，也可以在终端中直接运行它。也即在生成完成后，执行以下命令（在 Windows 下，生成的可执行文件会带有 .exe 后缀，例如会名为 taosd.exe ）： @@ -227,15 +342,15 @@ taos # 体验 TDengine -在TDengine终端中，用户可以通过SQL命令来创建/删除数据库、表等，并进行插入查询操作。 +在 TDengine 终端中，用户可以通过 SQL 命令来创建/删除数据库、表等，并进行插入查询操作。 -```bash -create database demo; -use demo; -create table t (ts timestamp, speed int); -insert into t values ('2019-07-15 00:00:00', 10); -insert into t values ('2019-07-15 01:00:00', 20); -select * from t; +```sql +CREATE DATABASE demo; +USE demo; +CREATE TABLE t (ts TIMESTAMP, speed INT); +INSERT INTO t VALUES('2019-07-15 00:00:00', 10); +INSERT INTO t VALUES('2019-07-15 01:00:00', 20); +SELECT * FROM t; ts | speed | =================================== 19-07-15 00:00:00.000| 10| @@ -247,33 +362,35 @@ Query OK, 2 row(s) in set (0.001700s) ## 官方连接器 -TDengine 提供了丰富的应用程序开发接口，其中包括C/C++、Java、Python、Go、Node.js、C# 、RESTful 等，便于用户快速开发应用： +TDengine 提供了丰富的应用程序开发接口，其中包括 C/C++、Java、Python、Go、Node.js、C# 、RESTful 等，便于用户快速开发应用： + +- [Java](https://www.taosdata.com/cn/documentation/connector/java) -- Java +- [C/C++](https://www.taosdata.com/cn/documentation/connector#c-cpp) -- C/C++ +- [Python](https://www.taosdata.com/cn/documentation/connector#python) -- Python +- [Go](https://www.taosdata.com/cn/documentation/connector#go) -- Go +- [RESTful API](https://www.taosdata.com/cn/documentation/connector#restful) -- RESTful API +- [Node.js](https://www.taosdata.com/cn/documentation/connector#nodejs) -- Node.js +- [Rust](https://www.taosdata.com/cn/documentation/connector/rust) ## 第三方连接器 TDengine 社区生态中也有一些非常友好的第三方连接器，可以通过以下链接访问它们的源码。 -- [Rust Connector](https://github.com/taosdata/TDengine/tree/master/tests/examples/rust) +- [Rust Bindings](https://github.com/songtianyi/tdengine-rust-bindings/tree/master/examples) - [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos) -- [Lua Connector](https://github.com/taosdata/TDengine/tree/develop/tests/examples/lua) +- [Lua Connector](https://github.com/taosdata/TDengine/tree/develop/examples/lua) # 运行和添加测试例 TDengine 的测试框架和所有测试例全部开源。 -点击 [这里](tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md)，了解如何运行测试例和添加新的测试例。 +点击 [这里](https://github.com/taosdata/TDengine/blob/develop/tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md)，了解如何运行测试例和添加新的测试例。 # 成为社区贡献者 @@ -281,8 +398,8 @@ TDengine 的测试框架和所有测试例全部开源。 # 加入技术交流群 -TDengine 官方社群「物联网大数据群」对外开放，欢迎您加入讨论。搜索微信号 "tdengine"，加小T为好友，即可入群。 +TDengine 官方社群「物联网大数据群」对外开放，欢迎您加入讨论。搜索微信号 "tdengine"，加小 T 为好友，即可入群。 -# [谁在使用TDengine](https://github.com/taosdata/TDengine/issues/2432) +# [谁在使用 TDengine](https://github.com/taosdata/TDengine/issues/2432) -欢迎所有 TDengine 用户及贡献者在 [这里](https://github.com/taosdata/TDengine/issues/2432) 分享您在当前工作中开发/使用 TDengine 的故事。 +欢迎所有 TDengine 用户及贡献者在 [这里](https://github.com/taosdata/TDengine/issues/2432) 分享您在当前工作中开发/使用 TDengine 的故事。 diff --git a/README.md b/README.md index 48349e891eb4e0d32890fe3cd8fc89d688de48fd..d4b8321813235f46842e7307a238623fc952c46a 100644 --- a/README.md +++ b/README.md @@ -1,116 +1,218 @@ +
+
+
+
+
+
+
[](https://cloud.drone.io/taosdata/TDengine)
[](https://ci.appveyor.com/project/sangshuduo/tdengine-2n8ge/branch/master)
[](https://coveralls.io/github/taosdata/TDengine?branch=develop)
[](https://bestpractices.coreinfrastructure.org/projects/4201)
[](https://snapcraft.io/tdengine)
-[](https://www.taosdata.com)
-
-English | [简体中文](./README-CN.md)
+English | [简体中文](README-CN.md) | We are hiring, check [here](https://tdengine.com/careers)
# What is TDengine？
-TDengine is an open-sourced big data platform under [GNU AGPL v3.0](http://www.gnu.org/licenses/agpl-3.0.html), designed and optimized for the Internet of Things (IoT), Connected Cars, Industrial IoT, and IT Infrastructure and Application Monitoring. Besides the 10x faster time-series database, it provides caching, stream computing, message queuing and other functionalities to reduce the complexity and cost of development and operation.
+TDengine is a high-performance, scalable time-series database with SQL support. Its code including cluster feature is open source under [GNU AGPL v3.0](http://www.gnu.org/licenses/agpl-3.0.html). Besides the database, it provides caching, stream processing, data subscription and other functionalities to reduce the complexity and cost of development and operation. TDengine differentiates itself from other TSDBs with the following advantages.
+
+- **High Performance**: TDengine outperforms other time series databases in data ingestion and querying while significantly reducing storage cost and compute costs, with an innovatively designed and purpose-built storage engine.
+
+- **Scalable**: TDengine provides out-of-box scalability and high-availability through its native distributed design. Nodes can be added through simple configuration to achieve greater data processing power. In addition, this feature is open source.
-- **10x Faster on Insert/Query Speeds**: Through the innovative design on storage, on a single-core machine, over 20K requests can be processed, millions of data points can be ingested, and over 10 million data points can be retrieved in a second. It is 10 times faster than other databases.
+- **SQL Support**: TDengine uses SQL as the query language, thereby reducing learning and migration costs, while adding SQL extensions to handle time-series data better, and supporting convenient and flexible schemaless data ingestion.
-- **1/5 Hardware/Cloud Service Costs**: Compared with typical big data solutions, less than 1/5 of computing resources are required. Via column-based storage and tuned compression algorithms for different data types, less than 1/10 of storage space is needed.
+- **All in One**: TDengine has built-in caching, stream processing and data subscription functions, it is no longer necessary to integrate Kafka/Redis/HBase/Spark or other software in some scenarios. It makes the system architecture much simpler and easy to maintain.
-- **Full Stack for Time-Series Data**: By integrating a database with message queuing, caching, and stream computing features together, it is no longer necessary to integrate Kafka/Redis/HBase/Spark or other software. It makes the system architecture much simpler and more robust.
+- **Seamless Integration**: Without a single line of code, TDengine provide seamless integration with third-party tools such as Telegraf, Grafana, EMQX, Prometheus, StatsD, collectd, etc. More will be integrated.
-- **Powerful Data Analysis**: Whether it is 10 years or one minute ago, data can be queried just by specifying the time range. Data can be aggregated over time, multiple time streams or both. Ad Hoc queries or analyses can be executed via TDengine shell, Python, R or Matlab.
+- **Zero Management**: Installation and cluster setup can be done in seconds. Data partitioning and sharding are executed automatically. TDengine’s running status can be monitored via Grafana or other DevOps tools.
-- **Seamless Integration with Other Tools**: Telegraf, Grafana, Matlab, R, and other tools can be integrated with TDengine without a line of code. MQTT, OPC, Hadoop, Spark, and many others will be integrated soon.
+- **Zero Learning Cost**: With SQL as the query language, support for ubiquitous tools like Python, Java, C/C++, Go, Rust, Node.js connectors, there is zero learning cost.
-- **Zero Management, No Learning Curve**: It takes only seconds to download, install, and run it successfully; there are no other dependencies. Automatic partitioning on tables or DBs. Standard SQL is used, with C/C++, Python, JDBC, Go and RESTful connectors.
+- **Interactive Console**: TDengine provides convenient console access to the database to run ad hoc queries, maintain the database, or manage the cluster without any programming.
+
+TDengine can be widely applied to Internet of Things (IoT), Connected Vehicles, Industrial IoT, DevOps, energy, finance and many other scenarios.
# Documentation
+
For user manual, system design and architecture, engineering blogs, refer to [TDengine Documentation](https://www.taosdata.com/en/documentation/)(中文版请点击[这里](https://www.taosdata.com/cn/documentation20/))
- for details. The documentation from our website can also be downloaded locally from *documentation/tdenginedocs-en* or *documentation/tdenginedocs-cn*.
+for details. The documentation from our website can also be downloaded locally from _documentation/tdenginedocs-en_ or _documentation/tdenginedocs-cn_.
# Building
-At the moment, TDengine only supports building and running on Linux systems. You can choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) or from the source code. This quick guide is for installation from the source only.
-To build TDengine, use [CMake](https://cmake.org/) 2.8.12.x or higher versions in the project directory.
+At the moment, TDengine server only supports running on Linux systems. You can choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) or build it from the source code. This quick guide is for installation from the source only.
+
+To build TDengine, use [CMake](https://cmake.org/) 3.0.2 or higher versions in the project directory.
-## Install tools
+## Install build dependencies
+
+### Ubuntu 16.04 and above or Debian
-### Ubuntu 16.04 and above & Debian:
```bash
-sudo apt-get install -y gcc cmake build-essential git
+sudo apt-get install -y gcc cmake build-essential git libssl-dev
```
-### Ubuntu 14.04:
+### Ubuntu 14.04
+
```bash
sudo apt-get install -y gcc cmake3 build-essential git binutils-2.26
export PATH=/usr/lib/binutils-2.26/bin:$PATH
```
-To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed.
+To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed.
+
To install openjdk-8:
+
```bash
sudo apt-get install -y openjdk-8-jdk
```
To install Apache Maven:
+
```bash
-sudo apt-get install -y maven
+sudo apt-get install -y maven
```
-### Centos 7:
+#### Install build dependencies for taosTools
+
+We provide a few useful tools such as taosBenchmark (was named taosdemo) and taosdump. They were part of TDengine. From TDengine 2.4.0.0, taosBenchmark and taosdump were not released together with TDengine.
+By default, TDengine compiling does not include taosTools. You can use 'cmake .. -DBUILD_TOOLS=true' to make them be compiled with TDengine.
+
+To build the [taosTools](https://github.com/taosdata/taos-tools) on Ubuntu/Debian, the following packages need to be installed.
+
```bash
-sudo yum install -y gcc gcc-c++ make cmake git
+sudo apt install build-essential libjansson-dev libsnappy-dev liblzma-dev libz-dev pkg-config
+```
+
+### CentOS 7
+
+```bash
+sudo yum install epel-release
+sudo yum update
+sudo yum install -y gcc gcc-c++ make cmake3 git openssl-devel
+sudo ln -sf /usr/bin/cmake3 /usr/bin/cmake
```
To install openjdk-8:
+
```bash
sudo yum install -y java-1.8.0-openjdk
```
To install Apache Maven:
+
```bash
sudo yum install -y maven
```
-### Centos 8 & Fedora:
+### CentOS 8 & Fedora
+
```bash
-sudo dnf install -y gcc gcc-c++ make cmake epel-release git
+sudo dnf install -y gcc gcc-c++ make cmake epel-release git openssl-devel
```
To install openjdk-8:
+
```bash
sudo dnf install -y java-1.8.0-openjdk
```
To install Apache Maven:
+
```bash
sudo dnf install -y maven
```
+#### Install build dependencies for taosTools on CentOS
+
+To build the [taosTools](https://github.com/taosdata/taos-tools) on CentOS, the following packages need to be installed.
+
+```bash
+sudo yum install zlib-devel xz-devel snappy-devel jansson jansson-devel pkgconfig libatomic libstdc++-static openssl-devel
+```
+
+Note: Since snappy lacks pkg-config support (refer to [link](https://github.com/google/snappy/pull/86)), it lead a cmake prompt libsnappy not found. But snappy will works well.
+
+### Setup golang environment
+
+TDengine includes few components developed by Go language. Please refer to golang.org official documentation for golang environment setup.
+
+Please use version 1.14+. For the user in China, we recommend using a proxy to accelerate package downloading.
+
+```
+go env -w GO111MODULE=on
+go env -w GOPROXY=https://goproxy.cn,direct
+```
+
+### Setup rust environment
+
+TDengine includees few compoments developed by Rust language. Please refer to rust-lang.org official documentation for rust environment setup.
+
## Get the source codes
First of all, you may clone the source codes from github:
+
```bash
git clone https://github.com/taosdata/TDengine.git
cd TDengine
```
-The connectors for go & grafana have been moved to separated repositories,
+The connectors for go & Grafana and some tools have been moved to separated repositories,
so you should run this command in the TDengine directory to install them:
+
```bash
git submodule update --init --recursive
```
+You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
+
+```
+[url "git@github.com:"]
+ insteadOf = https://github.com/
+```
+
## Build TDengine
### On Linux platform
+You can run the bash script `build.sh` to build both TDengine and taosTools including taosBenchmark and taosdump as below:
+
```bash
-mkdir debug && cd debug
-cmake .. && cmake --build .
+./build.sh
+```
+
+It equals to execute following commands:
+
+```bash
+git submodule update --init --recursive
+mkdir debug
+cd debug
+cmake .. -DBUILD_TOOLS=true
+make
+```
+
+Note TDengine 2.3.x.0 and later use a component named 'taosAdapter' to play http daemon role by default instead of the http daemon embedded in the early version of TDengine. The taosAdapter is programmed by go language. If you pull TDengine source code to the latest from an existing codebase, please execute 'git submodule update --init --recursive' to pull taosAdapter source code. Please install go language version 1.14 or above for compiling taosAdapter. If you meet difficulties regarding 'go mod', especially you are from China, you can use a proxy to solve the problem.
+
+```
+go env -w GO111MODULE=on
+go env -w GOPROXY=https://goproxy.cn,direct
+```
+
+The embedded http daemon still be built from TDengine source code by default. Or you can use the following command to choose to build taosAdapter.
+
+```
+cmake .. -DBUILD_HTTP=false
```
You can use Jemalloc as memory allocator instead of glibc:
+
```
apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true
@@ -120,24 +222,28 @@ TDengine build script can detect the host machine's architecture on X86-64, X86,
You can also specify CPUTYPE option like aarch64 or aarch32 too if the detection result is not correct:
aarch64:
+
```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build .
```
aarch32:
+
```bash
cmake .. -DCPUTYPE=aarch32 && cmake --build .
```
mips64:
+
```bash
cmake .. -DCPUTYPE=mips64 && cmake --build .
```
### On Windows platform
-If you use Visual Studio 2013, please open a command window by executing "cmd.exe".
+If you use the Visual Studio 2013, please open a command window by executing "cmd.exe".
Please specify "amd64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
+
```cmd
mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
@@ -145,7 +251,7 @@ cmake .. -G "NMake Makefiles"
nmake
```
-If you use Visual Studio 2019 or 2017:
+If you use the Visual Studio 2019 or 2017:
please open a command window by executing "cmd.exe".
Please specify "x64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.
@@ -158,15 +264,14 @@ nmake
```
Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:
+
```cmd
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```
-If you use Visual Studio 2022, the only change is the default path of `vcvarsall.bat`, which is `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat`.
-
-### On Mac OS X platform
+### On macOS platform
Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.
@@ -177,7 +282,10 @@ cmake .. && cmake --build .
# Installing
-After building successfully, TDengine can be installed by: (On Windows platform, the following command should be `nmake install`)
+## On Linux platform
+
+After building successfully, TDengine can be installed by
+
```bash
sudo make install
```
@@ -186,68 +294,129 @@ Users can find more information about directories installed on the system in the
Users can also choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) for it.
To start the service after installation, in a terminal, use:
+
```bash
sudo systemctl start taosd
```
Then users can use the [TDengine shell](https://www.taosdata.com/en/getting-started/#TDengine-Shell) to connect the TDengine server. In a terminal, use:
+
```bash
taos
```
If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
+### Install TDengine by apt-get
+
+If you use Debian or Ubuntu system, you can use 'apt-get' command to install TDengine from official repository. Please use following commands to setup:
+
+```
+wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add -
+echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | sudo tee /etc/apt/sources.list.d/tdengine-stable.list
+[Optional] echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list
+sudo apt-get update
+apt-cache policy tdengine
+sudo apt-get install tdengine
+```
+
+## On Windows platform
+
+After building successfully, TDengine can be installed by:
+
+```cmd
+nmake install
+```
+
+## On macOS platform
+
+After building successfully, TDengine can be installed by:
+
+```bash
+sudo make install
+```
+
+To start the service after installation, config `.plist` file first, in a terminal, use:
+
+```bash
+sudo cp ../packaging/macOS/com.taosdata.tdengine.plist /Library/LaunchDaemons
+```
+
+To start the service, in a terminal, use:
+
+```bash
+sudo launchctl load /Library/LaunchDaemons/com.taosdata.tdengine.plist
+```
+
+To stop the service, in a terminal, use:
+
+```bash
+sudo launchctl unload /Library/LaunchDaemons/com.taosdata.tdengine.plist
+```
+
## Quick Run
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
+
```bash
./build/bin/taosd -c test/cfg
```
In another terminal, use the TDengine shell to connect the server:
+
```bash
./build/bin/taos -c test/cfg
```
-option "-c test/cfg" specifies the system configuration file directory.
+option "-c test/cfg" specifies the system configuration file directory.
# Try TDengine
+
It is easy to run SQL commands from TDengine shell which is the same as other SQL databases.
+
```sql
-create database db;
-use db;
-create table t (ts timestamp, a int);
-insert into t values ('2019-07-15 00:00:00', 1);
-insert into t values ('2019-07-15 01:00:00', 2);
-select * from t;
-drop database db;
+CREATE DATABASE demo;
+USE demo;
+CREATE TABLE t (ts TIMESTAMP, speed INT);
+INSERT INTO t VALUES('2019-07-15 00:00:00', 10);
+INSERT INTO t VALUES('2019-07-15 01:00:00', 20);
+SELECT * FROM t;
+ ts | speed |
+===================================
+ 19-07-15 00:00:00.000| 10|
+ 19-07-15 01:00:00.000| 20|
+Query OK, 2 row(s) in set (0.001700s)
```
# Developing with TDengine
-### Official Connectors
+
+## Official Connectors
TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation.
-- [Java](https://www.taosdata.com/en/documentation/connector/#Java-Connector)
-- [C/C++](https://www.taosdata.com/en/documentation/connector/#C/C++-Connector)
-- [Python](https://www.taosdata.com/en/documentation/connector/#Python-Connector)
-- [Go](https://www.taosdata.com/en/documentation/connector/#Go-Connector)
-- [RESTful API](https://www.taosdata.com/en/documentation/connector/#RESTful-Connector)
-- [Node.js](https://www.taosdata.com/en/documentation/connector/#Node.js-Connector)
+- [Java](https://www.taosdata.com/en/documentation/connector/java)
+- [C/C++](https://www.taosdata.com/en/documentation/connector#c-cpp)
+- [Python](https://www.taosdata.com/en/documentation/connector#python)
+- [Go](https://www.taosdata.com/en/documentation/connector#go)
+- [RESTful API](https://www.taosdata.com/en/documentation/connector#restful)
+- [Node.js](https://www.taosdata.com/en/documentation/connector#nodejs)
+- [Rust](https://www.taosdata.com/en/documentation/connector/rust)
-### Third Party Connectors
+## Third Party Connectors
The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.
-- [Rust Connector](https://github.com/taosdata/TDengine/tree/master/tests/examples/rust)
+- [Rust Bindings](https://github.com/songtianyi/tdengine-rust-bindings/tree/master/examples)
- [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos)
- [Lua Connector](https://github.com/taosdata/TDengine/tree/develop/tests/examples/lua)
-# How to run the test cases and how to add a new test case?
- TDengine's test framework and all test cases are fully open source.
- Please refer to [this document](tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case.
+# How to run the test cases and how to add a new test case
+
+TDengine's test framework and all test cases are fully open source.
+Please refer to [this document](https://github.com/taosdata/TDengine/blob/develop/tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md) for how to run test and develop new test case.
# TDengine Roadmap
+
- Support event-driven stream computing
- Support user defined functions
- Support MQTT connection
diff --git a/cmake/cmake.define b/cmake/cmake.define
index 1cc28ec6fdd0f522534493788f25a45a1945a031..a33db902cacb02076f3d5e9caca980163a46d152 100644
--- a/cmake/cmake.define
+++ b/cmake/cmake.define
@@ -18,6 +18,14 @@ if (NOT DEFINED TD_GRANT)
SET(TD_GRANT FALSE)
endif()
+IF ("${WEBSOCKET}" MATCHES "true")
+ SET(TD_WEBSOCKET TRUE)
+ MESSAGE("Enable websocket")
+ ADD_DEFINITIONS(-DWEBSOCKET)
+ELSE ()
+ SET(TD_WEBSOCKET FALSE)
+ENDIF ()
+
IF ("${BUILD_HTTP}" STREQUAL "")
IF (TD_LINUX)
IF (TD_ARM_32)
diff --git a/include/util/tencode.h b/include/util/tencode.h
index a13afd44480eef8397befb42c2fe2a12c322b01e..e318d4f2400065c3a8926c06f36c7d077fb5b45a 100644
--- a/include/util/tencode.h
+++ b/include/util/tencode.h
@@ -477,22 +477,6 @@ static FORCE_INLINE void* tDecoderMalloc(SDecoder* pCoder, int32_t size) {
return n; \
} while (0)
-#define tGetV(p, v) \
- do { \
- int32_t n = 0; \
- if (v) *v = 0; \
- for (;;) { \
- if (p[n] <= 0x7f) { \
- if (v) (*v) |= (p[n] << (7 * n)); \
- n++; \
- break; \
- } \
- if (v) (*v) |= ((p[n] & 0x7f) << (7 * n)); \
- n++; \
- } \
- return n; \
- } while (0)
-
// PUT
static FORCE_INLINE int32_t tPutU8(uint8_t* p, uint8_t v) {
if (p) ((uint8_t*)p)[0] = v;
@@ -607,7 +591,22 @@ static FORCE_INLINE int32_t tGetI64(uint8_t* p, int64_t* v) {
return sizeof(int64_t);
}
-static FORCE_INLINE int32_t tGetU16v(uint8_t* p, uint16_t* v) { tGetV(p, v); }
+static FORCE_INLINE int32_t tGetU16v(uint8_t* p, uint16_t* v) {
+ int32_t n = 0;
+
+ if (v) *v = 0;
+ for (;;) {
+ if (p[n] <= 0x7f) {
+ if (v) (*v) |= (((uint16_t)p[n]) << (7 * n));
+ n++;
+ break;
+ }
+ if (v) (*v) |= (((uint16_t)(p[n] & 0x7f)) << (7 * n));
+ n++;
+ }
+
+ return n;
+}
static FORCE_INLINE int32_t tGetI16v(uint8_t* p, int16_t* v) {
int32_t n;
@@ -619,7 +618,22 @@ static FORCE_INLINE int32_t tGetI16v(uint8_t* p, int16_t* v) {
return n;
}
-static FORCE_INLINE int32_t tGetU32v(uint8_t* p, uint32_t* v) { tGetV(p, v); }
+static FORCE_INLINE int32_t tGetU32v(uint8_t* p, uint32_t* v) {
+ int32_t n = 0;
+
+ if (v) *v = 0;
+ for (;;) {
+ if (p[n] <= 0x7f) {
+ if (v) (*v) |= (((uint32_t)p[n]) << (7 * n));
+ n++;
+ break;
+ }
+ if (v) (*v) |= (((uint32_t)(p[n] & 0x7f)) << (7 * n));
+ n++;
+ }
+
+ return n;
+}
static FORCE_INLINE int32_t tGetI32v(uint8_t* p, int32_t* v) {
int32_t n;
@@ -631,7 +645,22 @@ static FORCE_INLINE int32_t tGetI32v(uint8_t* p, int32_t* v) {
return n;
}
-static FORCE_INLINE int32_t tGetU64v(uint8_t* p, uint64_t* v) { tGetV(p, v); }
+static FORCE_INLINE int32_t tGetU64v(uint8_t* p, uint64_t* v) {
+ int32_t n = 0;
+
+ if (v) *v = 0;
+ for (;;) {
+ if (p[n] <= 0x7f) {
+ if (v) (*v) |= (((uint64_t)p[n]) << (7 * n));
+ n++;
+ break;
+ }
+ if (v) (*v) |= (((uint64_t)(p[n] & 0x7f)) << (7 * n));
+ n++;
+ }
+
+ return n;
+}
static FORCE_INLINE int32_t tGetI64v(uint8_t* p, int64_t* v) {
int32_t n;
diff --git a/packaging/deb/DEBIAN/preinst b/packaging/deb/DEBIAN/preinst
index 5217a8229571bf993c6c3df8f82beb1ed67c3e96..c9957fc89c9570f9195e7ab0859db4583cbce0cf 100644
--- a/packaging/deb/DEBIAN/preinst
+++ b/packaging/deb/DEBIAN/preinst
@@ -37,4 +37,5 @@ if [ -f "${install_main_dir}/taosadapter.service" ]; then
fi
# there can not libtaos.so*, otherwise ln -s error
-${csudo}rm -f ${install_main_dir}/driver/libtaos* || :
+${csudo}rm -f ${install_main_dir}/driver/libtaos.* || :
+${csudo}rm -f ${install_main_dir}/driver/libtaosws.* || :
diff --git a/packaging/deb/DEBIAN/prerm b/packaging/deb/DEBIAN/prerm
index 501398f350de7013840056c3ee2a8252f961e496..5ff970bb700f32ff2848183feef2395f55910ae7 100644
--- a/packaging/deb/DEBIAN/prerm
+++ b/packaging/deb/DEBIAN/prerm
@@ -29,8 +29,12 @@ else
${csudo}rm -f ${bin_link_dir}/taosdemo || :
${csudo}rm -f ${cfg_link_dir}/* || :
${csudo}rm -f ${inc_link_dir}/taos.h || :
+ ${csudo}rm -f ${inc_link_dir}/taosdef.h || :
+ ${csudo}rm -f ${inc_link_dir}/taoserror.h || :
${csudo}rm -f ${inc_link_dir}/taosudf.h || :
+ ${csudo}rm -f ${inc_link_dir}/taosws.h || :
${csudo}rm -f ${lib_link_dir}/libtaos.* || :
+ ${csudo}rm -f ${lib_link_dir}/libtaosws.* || :
${csudo}rm -f ${log_link_dir} || :
${csudo}rm -f ${data_link_dir} || :
diff --git a/packaging/deb/makedeb.sh b/packaging/deb/makedeb.sh
index 043c1456b8fc22e78e707a4f8bc0c0e5a86bae55..1a4131ec6f059d25be409976246f767b6c3ec840 100755
--- a/packaging/deb/makedeb.sh
+++ b/packaging/deb/makedeb.sh
@@ -30,6 +30,7 @@ mkdir -p ${pkg_dir}
cd ${pkg_dir}
libfile="libtaos.so.${tdengine_ver}"
+wslibfile="libtaosws.so"
# create install dir
install_home_path="/usr/local/taos"
@@ -67,10 +68,12 @@ fi
cp ${compile_dir}/build/bin/taos ${pkg_dir}${install_home_path}/bin
cp ${compile_dir}/build/lib/${libfile} ${pkg_dir}${install_home_path}/driver
+cp ${compile_dir}/build/lib/${wslibfile} ${pkg_dir}${install_home_path}/driver ||:
cp ${compile_dir}/../include/client/taos.h ${pkg_dir}${install_home_path}/include
cp ${compile_dir}/../include/common/taosdef.h ${pkg_dir}${install_home_path}/include
cp ${compile_dir}/../include/util/taoserror.h ${pkg_dir}${install_home_path}/include
cp ${compile_dir}/../include/libs/function/taosudf.h ${pkg_dir}${install_home_path}/include
+cp ${compile_dir}/../src/inc/taosws.h ${pkg_dir}${install_home_path}/include ||:
cp -r ${top_dir}/examples/* ${pkg_dir}${install_home_path}/examples
#cp -r ${top_dir}/src/connector/python ${pkg_dir}${install_home_path}/connector
#cp -r ${top_dir}/src/connector/go ${pkg_dir}${install_home_path}/connector
diff --git a/packaging/rpm/tdengine.spec b/packaging/rpm/tdengine.spec
index f440f72aa298daccfa2f60a6bacf0035d5d44f6a..02e7e9b5e5ba5587f7cc102a42aa6991c8ea2c8a 100644
--- a/packaging/rpm/tdengine.spec
+++ b/packaging/rpm/tdengine.spec
@@ -42,6 +42,7 @@ echo version: %{_version}
echo buildroot: %{buildroot}
libfile="libtaos.so.%{_version}"
+wslibfile="libtaosws.so"
# create install path, and cp file
mkdir -p %{buildroot}%{homepath}/bin
@@ -74,10 +75,12 @@ if [ -f %{_compiledir}/build/bin/taosadapter ]; then
cp %{_compiledir}/build/bin/taosadapter %{buildroot}%{homepath}/bin ||:
fi
cp %{_compiledir}/build/lib/${libfile} %{buildroot}%{homepath}/driver
+cp %{_compiledir}/build/lib/${wslibfile} %{buildroot}%{homepath}/driver ||:
cp %{_compiledir}/../include/client/taos.h %{buildroot}%{homepath}/include
cp %{_compiledir}/../include/common/taosdef.h %{buildroot}%{homepath}/include
cp %{_compiledir}/../include/util/taoserror.h %{buildroot}%{homepath}/include
cp %{_compiledir}/../include/libs/function/taosudf.h %{buildroot}%{homepath}/include
+cp %{_compiledir}/../src/inc/taosws.h %{buildroot}%{homepath}/include ||:
#cp -r %{_compiledir}/../src/connector/python %{buildroot}%{homepath}/connector
#cp -r %{_compiledir}/../src/connector/go %{buildroot}%{homepath}/connector
#cp -r %{_compiledir}/../src/connector/nodejs %{buildroot}%{homepath}/connector
diff --git a/packaging/tools/install.sh b/packaging/tools/install.sh
index 2d17ef481219f5ee747f7596d4d3d2a656e90fb9..bcc2df4b91b0bdaba6fb66b5e9eab89bf7f96d2f 100755
--- a/packaging/tools/install.sh
+++ b/packaging/tools/install.sh
@@ -227,9 +227,13 @@ function install_lib() {
${csudo}ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo}ln -s ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
+ ${csudo}ln -s ${lib_link_dir}/libtaosws.so ${lib_link_dir}/libtaosws.so || :
+
if [[ -d ${lib64_link_dir} && ! -e ${lib64_link_dir}/libtaos.so ]]; then
${csudo}ln -s ${install_main_dir}/driver/libtaos.* ${lib64_link_dir}/libtaos.so.1 || :
${csudo}ln -s ${lib64_link_dir}/libtaos.so.1 ${lib64_link_dir}/libtaos.so || :
+
+ ${csudo}ln -s ${lib64_link_dir}/libtaosws.so ${lib64_link_dir}/libtaosws.so || :
fi
${csudo}ldconfig
@@ -313,11 +317,16 @@ function install_jemalloc() {
function install_header() {
${csudo}rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taosdef.h ${inc_link_dir}/taoserror.h ${inc_link_dir}/taosudf.h || :
+
+ ${csudo}rm -f ${inc_link_dir}/taosws.h || :
+
${csudo}cp -f ${script_dir}/inc/* ${install_main_dir}/include && ${csudo}chmod 644 ${install_main_dir}/include/*
${csudo}ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h
${csudo}ln -s ${install_main_dir}/include/taosdef.h ${inc_link_dir}/taosdef.h
${csudo}ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h
${csudo}ln -s ${install_main_dir}/include/taosudf.h ${inc_link_dir}/taosudf.h
+
+ ${csudo}ln -s ${install_main_dir}/include/taosws.h ${inc_link_dir}/taosws.h || :
}
function add_newHostname_to_hosts() {
diff --git a/packaging/tools/make_install.sh b/packaging/tools/make_install.sh
index 4e4ebb72c01474832a8319b20fea96813a6a761a..8ad42811d4befc0bdb4f16bfa78c9137bb3099e9 100755
--- a/packaging/tools/make_install.sh
+++ b/packaging/tools/make_install.sh
@@ -294,21 +294,29 @@ function install_avro() {
function install_lib() {
# Remove links
${csudo}rm -f ${lib_link_dir}/libtaos.* || :
+ ${csudo}rm -f ${lib_link_dir}/libtaosws.* || :
if [ "$osType" != "Darwin" ]; then
${csudo}rm -f ${lib64_link_dir}/libtaos.* || :
+ ${csudo}rm -f ${lib64_link_dir}/libtaosws.* || :
fi
if [ "$osType" != "Darwin" ]; then
${csudo}cp ${binary_dir}/build/lib/libtaos.so.${verNumber} \
${install_main_dir}/driver &&
- ${csudo}chmod 777 ${install_main_dir}/driver/*
+ ${csudo}chmod 777 ${install_main_dir}/driver/libtaos.so.${verNumber}
+
+ ${csudo}cp ${binary_dir}/build/lib/libtaosws.so \
+ ${install_main_dir}/driver &&
+ ${csudo}chmod 777 ${install_main_dir}/driver/libtaosws.so
${csudo}ln -sf ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo}ln -sf ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
+ ${csudo}ln -sf ${install_main_dir}/driver/libtaosws.so ${lib_link_dir}/libtaosws.so || :
if [ -d "${lib64_link_dir}" ]; then
${csudo}ln -sf ${install_main_dir}/driver/libtaos.* ${lib64_link_dir}/libtaos.so.1
${csudo}ln -sf ${lib64_link_dir}/libtaos.so.1 ${lib64_link_dir}/libtaos.so
+ ${csudo}ln -sf ${lib64_link_dir}/libtaosws.so ${lib64_link_dir}/libtaosws.so || :
fi
else
${csudo}cp -Rf ${binary_dir}/build/lib/libtaos.${verNumber}.dylib \
@@ -337,8 +345,8 @@ function install_lib() {
fi
install_jemalloc
- install_avro lib
- install_avro lib64
+ #install_avro lib
+ #install_avro lib64
if [ "$osType" != "Darwin" ]; then
${csudo}ldconfig
@@ -350,11 +358,19 @@ function install_header() {
if [ "$osType" != "Darwin" ]; then
${csudo}rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taosdef.h ${inc_link_dir}/taoserror.h ${inc_link_dir}/taosudf.h || :
${csudo}cp -f ${source_dir}/include/client/taos.h ${source_dir}/include/common/taosdef.h ${source_dir}/include/util/taoserror.h ${source_dir}/include/libs/function/taosudf.h \
+ ${csudo}rm -f ${inc_link_dir}/taosws.h || :
+
+ ${csudo}cp -f ${source_dir}/src/inc/taos.h ${source_dir}/src/inc/taosdef.h ${source_dir}/src/inc/taoserror.h \
${install_main_dir}/include && ${csudo}chmod 644 ${install_main_dir}/include/*
+
+ ${csudo}cp -f ${binary_dir}/build/include/taosws.h ${install_main_dir}/include && ${csudo}chmod 644 ${install_main_dir}/include/taosws.h
+
${csudo}ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h
${csudo}ln -s ${install_main_dir}/include/taosdef.h ${inc_link_dir}/taosdef.h
${csudo}ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h
${csudo}ln -s ${install_main_dir}/include/taosudf.h ${inc_link_dir}/taosudf.h
+
+ ${csudo}ln -s ${install_main_dir}/include/taosws.h ${inc_link_dir}/taosws.h || :
else
${csudo}cp -f ${source_dir}/include/client/taos.h ${source_dir}/include/common/taosdef.h ${source_dir}/include/util/taoserror.h ${source_dir}/include/libs/function/taosudf.h \
${install_main_dir}/include ||
diff --git a/packaging/tools/makepkg.sh b/packaging/tools/makepkg.sh
index cd656fe6f21a960f6682d87a25084d8e125a1dcf..2965a02b49ba68706ef53bc0f4741f5b1e571228 100755
--- a/packaging/tools/makepkg.sh
+++ b/packaging/tools/makepkg.sh
@@ -92,8 +92,11 @@ else
fi
lib_files="${build_dir}/lib/libtaos.so.${version}"
+wslib_files="${build_dir}/lib/libtaosws.so."
header_files="${code_dir}/include/client/taos.h ${code_dir}/include/common/taosdef.h ${code_dir}/include/util/taoserror.h ${code_dir}/include/libs/function/taosudf.h"
+wsheader_files="${code_dir}/inc/taosws.h"
+
if [ "$dbName" != "taos" ]; then
cfg_dir="${top_dir}/../enterprise/packaging/cfg"
else
@@ -109,6 +112,9 @@ init_file_rpm=${script_dir}/../rpm/taosd
# make directories.
mkdir -p ${install_dir}
mkdir -p ${install_dir}/inc && cp ${header_files} ${install_dir}/inc
+
+${wsheader_files} ${install_dir}/inc || :
+
mkdir -p ${install_dir}/cfg && cp ${cfg_dir}/${configFile} ${install_dir}/cfg/${configFile}
if [ -f "${compile_dir}/test/cfg/taosadapter.toml" ]; then
@@ -283,6 +289,7 @@ fi
# Copy driver
mkdir -p ${install_dir}/driver && cp ${lib_files} ${install_dir}/driver && echo "${versionComp}" >${install_dir}/driver/vercomp.txt
+cp ${wslib_files} ${install_dir}/driver || :
# Copy connector
if [ "$verMode" == "cluster" ]; then
diff --git a/packaging/tools/remove.sh b/packaging/tools/remove.sh
index d34c81a4f59186ead2dd4ba58c4c14ebba52276d..ec836f2eac297751f969470e89fbffac7cdf3d47 100755
--- a/packaging/tools/remove.sh
+++ b/packaging/tools/remove.sh
@@ -102,7 +102,10 @@ function clean_local_bin() {
function clean_lib() {
# Remove link
${csudo}rm -f ${lib_link_dir}/libtaos.* || :
+ ${csudo}rm -f ${lib_link_dir}/libtaosws.* || :
+
${csudo}rm -f ${lib64_link_dir}/libtaos.* || :
+ ${csudo}rm -f ${lib64_link_dir}/libtaosws.* || :
#${csudo}rm -rf ${v15_java_app_dir} || :
}
@@ -111,6 +114,8 @@ function clean_header() {
${csudo}rm -f ${inc_link_dir}/taos.h || :
${csudo}rm -f ${inc_link_dir}/taosdef.h || :
${csudo}rm -f ${inc_link_dir}/taoserror.h || :
+
+ ${csudo}rm -f ${inc_link_dir}/taosws.h || :
}
function clean_config() {
diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
index 993a604b510de8911e1a56046fca116194490c47..5ffddd0127c26174e5747c5db7936695764e985b 100644
--- a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
+++ b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
@@ -384,6 +384,8 @@ SArray *vmGetMsgHandles() {
if (dmSetMgmtHandle(pArray, TDMT_SYNC_APPEND_ENTRIES, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_SYNC_APPEND_ENTRIES_BATCH, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_SYNC_APPEND_ENTRIES_REPLY, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
+ if (dmSetMgmtHandle(pArray, TDMT_SYNC_SNAPSHOT_SEND, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
+ if (dmSetMgmtHandle(pArray, TDMT_SYNC_SNAPSHOT_RSP, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_SYNC_SET_VNODE_STANDBY, vmPutMsgToSyncQueue, 0) == NULL) goto _OVER;
code = 0;
diff --git a/source/dnode/vnode/src/tsdb/tsdbCache.c b/source/dnode/vnode/src/tsdb/tsdbCache.c
index ea1dfc057f28ea81402f0bd25aac52edc83480fe..946984d9c2d9eb7d6df088d000fda4a4017b0897 100644
--- a/source/dnode/vnode/src/tsdb/tsdbCache.c
+++ b/source/dnode/vnode/src/tsdb/tsdbCache.c
@@ -155,7 +155,7 @@ int32_t tsdbCacheInsertLastrow(SLRUCache *pCache, STsdb *pTsdb, tb_uid_t uid, ST
/* tsdbCacheInsertLastrow(pCache, uid, row, dup); */
}
}
- } else {
+ } /*else {
if (dup) {
cacheRow = tdRowDup(row);
} else {
@@ -168,7 +168,7 @@ int32_t tsdbCacheInsertLastrow(SLRUCache *pCache, STsdb *pTsdb, tb_uid_t uid, ST
if (status != TAOS_LRU_STATUS_OK) {
code = -1;
}
- }
+ }*/
return code;
}
@@ -992,7 +992,7 @@ static int32_t mergeLast(tb_uid_t uid, STsdb *pTsdb, SArray **ppLastArray) {
*pColVal = COL_VAL_VALUE(pTColumn->colId, pTColumn->type, (SValue){.ts = maxKey});
// if (taosArrayPush(pColArray, pColVal) == NULL) {
- if (taosArrayPush(pColArray, &(SLastCol){.ts = TSKEY_MAX, .colVal = *pColVal}) == NULL) {
+ if (taosArrayPush(pColArray, &(SLastCol){.ts = maxKey, .colVal = *pColVal}) == NULL) {
code = TSDB_CODE_OUT_OF_MEMORY;
goto _err;
}
@@ -1127,7 +1127,7 @@ int32_t tsdbCacheGetLastrowH(SLRUCache *pCache, tb_uid_t uid, STsdb *pTsdb, LRUH
//*ppRow = (STSRow *)taosLRUCacheValue(pCache, h);
} else {
STSRow *pRow = NULL;
- bool dup = false;
+ bool dup = false; // which is always false for now
code = mergeLastRow(uid, pTsdb, &dup, &pRow);
// if table's empty or error, return code of -1
if (code < 0 || pRow == NULL) {
@@ -1139,7 +1139,14 @@ int32_t tsdbCacheGetLastrowH(SLRUCache *pCache, tb_uid_t uid, STsdb *pTsdb, LRUH
return 0;
}
- tsdbCacheInsertLastrow(pCache, pTsdb, uid, pRow, dup);
+ _taos_lru_deleter_t deleter = deleteTableCacheLastrow;
+ LRUStatus status =
+ taosLRUCacheInsert(pCache, key, keyLen, pRow, TD_ROW_LEN(pRow), deleter, NULL, TAOS_LRU_PRIORITY_LOW);
+ if (status != TAOS_LRU_STATUS_OK) {
+ code = -1;
+ }
+
+ // tsdbCacheInsertLastrow(pCache, pTsdb, uid, pRow, dup);
h = taosLRUCacheLookup(pCache, key, keyLen);
//*ppRow = (STSRow *)taosLRUCacheValue(pCache, h);
}
@@ -1202,7 +1209,7 @@ int32_t tsdbCacheGetLastH(SLRUCache *pCache, tb_uid_t uid, STsdb *pTsdb, LRUHand
if (status != TAOS_LRU_STATUS_OK) {
code = -1;
}
- /* tsdbCacheInsertLast(pCache, uid, pRow); */
+
h = taosLRUCacheLookup(pCache, key, keyLen);
//*ppRow = (STSRow *)taosLRUCacheValue(pCache, h);
}
@@ -1235,9 +1242,21 @@ int32_t tsdbCacheDelete(SLRUCache *pCache, tb_uid_t uid, TSKEY eKey) {
getTableCacheKey(uid, 1, key, &keyLen);
h = taosLRUCacheLookup(pCache, key, keyLen);
if (h) {
- // clear last cache anyway, no matter where eKey ends.
- taosLRUCacheRelease(pCache, h, true);
+ SArray *pLast = (SArray *)taosLRUCacheValue(pCache, h);
+ bool invalidate = false;
+ int16_t nCol = taosArrayGetSize(pLast);
+
+ for (int16_t iCol = 0; iCol < nCol; ++iCol) {
+ SLastCol *tTsVal = (SLastCol *)taosArrayGet(pLast, iCol);
+ if (eKey >= tTsVal->ts) {
+ invalidate = true;
+ break;
+ }
+ }
+ if (invalidate) {
+ taosLRUCacheRelease(pCache, h, true);
+ }
// void taosLRUCacheErase(SLRUCache * cache, const void *key, size_t keyLen);
}
diff --git a/source/dnode/vnode/src/tsdb/tsdbReaderWriter.c b/source/dnode/vnode/src/tsdb/tsdbReaderWriter.c
index a331f88e61523003187850c47c593b28f94b817a..e96ee03b03f36c0e4d45c8ca8310441ff6d5a85b 100644
--- a/source/dnode/vnode/src/tsdb/tsdbReaderWriter.c
+++ b/source/dnode/vnode/src/tsdb/tsdbReaderWriter.c
@@ -424,10 +424,12 @@ int32_t tsdbReadDelIdx(SDelFReader *pReader, SArray *aDelIdx, uint8_t **ppBuf) {
ASSERT(n == size - sizeof(TSCKSUM));
+ tFree(pBuf);
return code;
_err:
tsdbError("vgId:%d read del idx failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+ tFree(pBuf);
return code;
}
diff --git a/source/dnode/vnode/src/tsdb/tsdbUtil.c b/source/dnode/vnode/src/tsdb/tsdbUtil.c
index ed8bdc17a50d9f132a0b9651cc4143369e632b57..2e628edb7a2946521ed9f4aa5e4a4cdd173ee32a 100644
--- a/source/dnode/vnode/src/tsdb/tsdbUtil.c
+++ b/source/dnode/vnode/src/tsdb/tsdbUtil.c
@@ -1016,7 +1016,7 @@ int32_t tBlockDataAppendRow(SBlockData *pBlockData, TSDBROW *pRow, STSchema *pTS
SColData *pColData;
SColVal *pColVal;
- ASSERT(nColData > 0);
+ if (nColData == 0) goto _exit;
tRowIterInit(pIter, pRow, pTSchema);
pColData = tBlockDataGetColDataByIdx(pBlockData, iColData);
@@ -1046,6 +1046,7 @@ int32_t tBlockDataAppendRow(SBlockData *pBlockData, TSDBROW *pRow, STSchema *pTS
}
}
+_exit:
pBlockData->nRow++;
return code;
diff --git a/source/dnode/vnode/src/vnd/vnodeSync.c b/source/dnode/vnode/src/vnd/vnodeSync.c
index f75ccba4bba7faf54d2aa0abdacdd0b8cca09a24..0ca083d551db1e64d600e80b57e3954feb69be82 100644
--- a/source/dnode/vnode/src/vnd/vnodeSync.c
+++ b/source/dnode/vnode/src/vnd/vnodeSync.c
@@ -338,12 +338,12 @@ int32_t vnodeProcessSyncMsg(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) {
} else if (pMsg->msgType == TDMT_SYNC_REQUEST_VOTE) {
SyncRequestVote *pSyncMsg = syncRequestVoteFromRpcMsg2(pMsg);
ASSERT(pSyncMsg != NULL);
- code = syncNodeOnRequestVoteCb(pSyncNode, pSyncMsg);
+ code = syncNodeOnRequestVoteSnapshotCb(pSyncNode, pSyncMsg);
syncRequestVoteDestroy(pSyncMsg);
} else if (pMsg->msgType == TDMT_SYNC_REQUEST_VOTE_REPLY) {
SyncRequestVoteReply *pSyncMsg = syncRequestVoteReplyFromRpcMsg2(pMsg);
ASSERT(pSyncMsg != NULL);
- code = syncNodeOnRequestVoteReplyCb(pSyncNode, pSyncMsg);
+ code = syncNodeOnRequestVoteReplySnapshotCb(pSyncNode, pSyncMsg);
syncRequestVoteReplyDestroy(pSyncMsg);
} else if (pMsg->msgType == TDMT_SYNC_APPEND_ENTRIES_BATCH) {
SyncAppendEntriesBatch *pSyncMsg = syncAppendEntriesBatchFromRpcMsg2(pMsg);
@@ -355,6 +355,14 @@ int32_t vnodeProcessSyncMsg(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) {
ASSERT(pSyncMsg != NULL);
code = syncNodeOnAppendEntriesReplySnapshot2Cb(pSyncNode, pSyncMsg);
syncAppendEntriesReplyDestroy(pSyncMsg);
+ } else if (pMsg->msgType == TDMT_SYNC_SNAPSHOT_SEND) {
+ SyncSnapshotSend *pSyncMsg = syncSnapshotSendFromRpcMsg2(pMsg);
+ code = syncNodeOnSnapshotSendCb(pSyncNode, pSyncMsg);
+ syncSnapshotSendDestroy(pSyncMsg);
+ } else if (pMsg->msgType == TDMT_SYNC_SNAPSHOT_RSP) {
+ SyncSnapshotRsp *pSyncMsg = syncSnapshotRspFromRpcMsg2(pMsg);
+ code = syncNodeOnSnapshotRspCb(pSyncNode, pSyncMsg);
+ syncSnapshotRspDestroy(pSyncMsg);
} else if (pMsg->msgType == TDMT_SYNC_SET_VNODE_STANDBY) {
code = vnodeSetStandBy(pVnode);
if (code != 0 && terrno != 0) code = terrno;
diff --git a/source/libs/sync/src/syncAppendEntriesReply.c b/source/libs/sync/src/syncAppendEntriesReply.c
index 3f19b6759e9dcc4924681e5978065cbe602e019f..afea32d6729358e204f3e23ea580bf88c508425e 100644
--- a/source/libs/sync/src/syncAppendEntriesReply.c
+++ b/source/libs/sync/src/syncAppendEntriesReply.c
@@ -193,9 +193,10 @@ int32_t syncNodeOnAppendEntriesReplySnapshot2Cb(SSyncNode* ths, SyncAppendEntrie
// start snapshot