Skip to content

T
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1192
Star 22018
Fork 4786
T
TDengine
提交 e15abb2e 编写于 作者: S shenglian zhou
浏览文件
操作

Merge branch 'develop' into feature/szhou/regex-nmatch

上级 d889a8c0 10db0c2b
展开全部 隐藏空白更改
内联 并排
Showing with 5585 addition and 381 deletion
Jenkinsfile
浏览文件 @ e15abb2e
...@@ -235,11 +235,18 @@ pipeline { ...@@ -235,11 +235,18 @@ pipeline {
npm install td2.0-connector > /dev/null 2>&1 npm install td2.0-connector > /dev/null 2>&1
node nodejsChecker.js host=localhost node nodejsChecker.js host=localhost
node test1970.js node test1970.js
cd ${WKC}/tests/connectorTest/nodejsTest/nanosupport
npm install td2.0-connector > /dev/null 2>&1
node nanosecondTest.js
''' '''
sh ''' sh '''
cd ${WKC}/tests/examples/C#/taosdemo cd ${WKC}/tests/examples/C#/taosdemo
mcs -out:taosdemo *.cs > /dev/null 2>&1 mcs -out:taosdemo *.cs > /dev/null 2>&1
echo '' |./taosdemo -c /etc/taos echo '' |./taosdemo -c /etc/taos
cd ${WKC}/tests/connectorTest/C#Test/nanosupport
mcs -out:nano *.cs > /dev/null 2>&1
echo '' |./nano
''' '''
sh ''' sh '''
cd ${WKC}/tests/gotest cd ${WKC}/tests/gotest
......
documentation20/cn/05.insert/docs.md
浏览文件 @ e15abb2e
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
TDengine支持多种接口写入数据，包括SQL, Prometheus, Telegraf, EMQ MQTT Broker, HiveMQ Broker, CSV文件等，后续还将提供Kafka, OPC等接口。数据可以单条插入，也可以批量插入，可以插入一个数据采集点的数据，也可以同时插入多个数据采集点的数据。支持多线程插入，支持时间乱序数据插入，也支持历史数据插入。 TDengine支持多种接口写入数据，包括SQL, Prometheus, Telegraf, EMQ MQTT Broker, HiveMQ Broker, CSV文件等，后续还将提供Kafka, OPC等接口。数据可以单条插入，也可以批量插入，可以插入一个数据采集点的数据，也可以同时插入多个数据采集点的数据。支持多线程插入，支持时间乱序数据插入，也支持历史数据插入。
## <a class="anchor" id="sql"></a>SQL写入 ## <a class="anchor" id="sql"></a>SQL 写入
应用通过C/C++、JDBC、GO、C#或Python Connector 执行SQL insert语句来插入数据，用户还可以通过TAOS Shell，手动输入SQL insert语句插入数据。比如下面这条insert 就将一条记录写入到表d1001中： 应用通过C/C++、JDBC、GO、C#或Python Connector 执行SQL insert语句来插入数据，用户还可以通过TAOS Shell，手动输入SQL insert语句插入数据。比如下面这条insert 就将一条记录写入到表d1001中：
```mysql ```mysql
...@@ -27,11 +27,73 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6, ...@@ -27,11 +27,73 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6,
- 对同一张表，如果新插入记录的时间戳已经存在，默认情形下（UPDATE=0）新记录将被直接抛弃，也就是说，在一张表里，时间戳必须是唯一的。如果应用自动生成记录，很有可能生成的时间戳是一样的，这样，成功插入的记录条数会小于应用插入的记录条数。如果在创建数据库时使用了 UPDATE 1 选项，插入相同时间戳的新记录将覆盖原有记录。 - 对同一张表，如果新插入记录的时间戳已经存在，默认情形下（UPDATE=0）新记录将被直接抛弃，也就是说，在一张表里，时间戳必须是唯一的。如果应用自动生成记录，很有可能生成的时间戳是一样的，这样，成功插入的记录条数会小于应用插入的记录条数。如果在创建数据库时使用了 UPDATE 1 选项，插入相同时间戳的新记录将覆盖原有记录。
- 写入的数据的时间戳必须大于当前时间减去配置参数keep的时间。如果keep配置为3650天，那么无法写入比3650天还早的数据。写入数据的时间戳也不能大于当前时间加配置参数days。如果days为2，那么无法写入比当前时间还晚2天的数据。 - 写入的数据的时间戳必须大于当前时间减去配置参数keep的时间。如果keep配置为3650天，那么无法写入比3650天还早的数据。写入数据的时间戳也不能大于当前时间加配置参数days。如果days为2，那么无法写入比当前时间还晚2天的数据。
## <a class="anchor" id="prometheus"></a>Prometheus直接写入 ## <a class="anchor" id="schemaless"></a>Schemaless 写入
在物联网应用中，常会采集比较多的数据项，用于实现智能控制、业务分析、设备监控等。由于应用逻辑的版本升级，或者设备自身的硬件调整等原因，数据采集项就有可能比较频繁地出现变动。为了在这种情况下方便地完成数据记录工作，TDengine 从 2.2.0.0 版本开始，提供 Schemaless 写入方式，可以免于预先创建超级表/数据子表，而是随着数据写入，自动创建与数据对应的存储结构。并且在必要时，Schemaless 将自动增加必要的数据列，保证用户写入的数据可以被正确存储。目前，TDengine 的 C/C++ Connector 提供支持 Schemaless 的操作接口，详情请参见 [Schemaless 方式写入接口](https://www.taosdata.com/cn/documentation/connector#schemaless) 章节。这里对 Schemaless 的数据表达格式进行描述。
### Schemaless 数据行协议
Schemaless 采用一个字符串来表达最终存储的一个数据行（可以向 Schemaless 写入 API 中一次传入多个字符串来实现多个数据行的批量写入），其格式约定如下：
```json
measurement,tag_set field_set timestamp
```
其中，
* measurement 将作为数据表名。它与 tag_set 之间使用一个英文逗号来分隔。
* tag_set 将作为标签数据，其格式形如 `<tag_key>=<tag_value>,<tag_key>=<tag_value>`，也即可以使用英文逗号来分隔多个标签数据。它与 field_set 之间使用一个半角空格来分隔。
* field_set 将作为普通列数据，其格式形如 `<field_key>=<field_value>,<field_key>=<field_value>`，同样是使用英文逗号来分隔多个普通列的数据。它与 timestamp 之间使用一个半角空格来分隔。
* timestamp 即本行数据对应的主键时间戳。
在 Schemaless 的数据行协议中，tag_set、field_set 中的每个数据项都需要对自身的数据类型进行描述。具体来说：
* 如果两边有英文双引号，表示 BIANRY(32) 类型。例如 `"abc"`。
* 如果两边有英文双引号而且带有 L 前缀，表示 NCHAR(32) 类型。例如 `L"报错信息"`。
* 对空格、等号（=）、逗号（,）、双引号（"），前面需要使用反斜杠（\）进行转义。（都指的是英文半角符号）
* 数值类型将通过后缀来区分数据类型：
- 没有后缀，为 FLOAT 类型；
- 后缀为 f32，为 FLOAT 类型；
- 后缀为 f64，为 DOUBLE 类型；
- 后缀为 i8，表示为 TINYINT (INT8) 类型；
- 后缀为 i16，表示为 SMALLINT (INT16) 类型；
- 后缀为 i32，表示为 INT (INT32) 类型；
- 后缀为 i64，表示为 BIGINT (INT64) 类型；
- 后缀为 b，表示为 BOOL 类型。
* t, T, true, True, TRUE, f, F, false, False 将直接作为 BOOL 型来处理。
timestamp 位置的时间戳通过后缀来声明时间精度，具体如下：
* 不带任何后缀的长整数会被当作微秒来处理；
* 当后缀为 s 时，表示秒时间戳；
* 当后缀为 ms 时，表示毫秒时间戳；
* 当后缀为 us 时，表示微秒时间戳；
* 当后缀为 ns 时，表示纳秒时间戳；
* 当时间戳为 0 时，表示采用客户端的当前时间（因此，同一批提交的数据中，时间戳 0 会被解释为同一个时间点，于是就有可能导致时间戳重复）。
例如，如下 Schemaless 数据行表示：向名为 st 的超级表下的 t1 标签为 3（BIGINT 类型）、t2 标签为 4（DOUBLE 类型）、t3 标签为 "t3"（BINARY 类型）的数据子表，写入 c1 列为 3（BIGINT 类型）、c2 列为 false（BOOL 类型）、c3 列为 "passit"（NCHAR 类型）、c4 列为 4（DOUBLE 类型）、主键时间戳为 1626006833639000000（纳秒精度）的一行数据。
```json
st,t1=3i64,t2=4f64,t3="t3" c1=3i64,c3=L"passit",c2=false,c4=4f64 1626006833639000000ns
```
### Schemaless 的处理逻辑
Schemaless 按照如下原则来处理行数据：
1. 当 tag_set 中有 ID 字段时，该字段的值将作为数据子表的表名。
2. 没有 ID 字段时，将使用 `measurement + tag_value1 + tag_value2 + ...` 的 md5 值来作为子表名。
3. 如果指定的超级表名不存在，则 Schemaless 会创建这个超级表。
4. 如果指定的数据子表不存在，则 Schemaless 会使用 tag values 创建这个数据子表。
5. 如果数据行中指定的标签列或普通列不存在，则 Schemaless 会在超级表中增加对应的标签列或普通列（只增不减）。
6. 如果超级表中存在一些标签列或普通列未在一个数据行中被指定取值，那么这些列的值在这一行中会被置为 NULL。
7. 对 BINARY 或 NCHAR 列，如果数据行中所提供值的长度超出了列类型的限制，那么 Schemaless 会增加该列允许存储的字符长度上限（只增不减），以保证数据的完整保存。
8. 如果指定的数据子表已经存在，而且本次指定的标签列取值跟已保存的值不一样，那么最新的数据行中的值会覆盖旧的标签列取值。
9. 整个处理过程中遇到的错误会中断写入过程，并返回错误代码。
**注意：**Schemaless 所有的处理逻辑，仍会遵循 TDengine 对数据结构的底层限制，例如每行数据的总长度不能超过 16k 字节。这方面的具体限制约束请参见 [TAOS SQL 边界限制](https://www.taosdata.com/cn/documentation/taos-sql#limitation) 章节。
关于 Schemaless 的字符串编码处理、时区设置等，均会沿用 TAOSC 客户端的设置。
## <a class="anchor" id="prometheus"></a>Prometheus 直接写入
[Prometheus](https://www.prometheus.io/)作为Cloud Native Computing Fundation毕业的项目，在性能监控以及K8S性能监控领域有着非常广泛的应用。TDengine提供一个小工具[Bailongma](https://github.com/taosdata/Bailongma)，只需对Prometheus做简单配置，无需任何代码，就可将Prometheus采集的数据直接写入TDengine，并按规则在TDengine自动创建库和相关表项。博文[用Docker容器快速搭建一个Devops监控Demo](https://www.taosdata.com/blog/2020/02/03/1189.html)即是采用Bailongma将Prometheus和Telegraf的数据写入TDengine中的示例，可以参考。 [Prometheus](https://www.prometheus.io/)作为Cloud Native Computing Fundation毕业的项目，在性能监控以及K8S性能监控领域有着非常广泛的应用。TDengine提供一个小工具[Bailongma](https://github.com/taosdata/Bailongma)，只需对Prometheus做简单配置，无需任何代码，就可将Prometheus采集的数据直接写入TDengine，并按规则在TDengine自动创建库和相关表项。博文[用Docker容器快速搭建一个Devops监控Demo](https://www.taosdata.com/blog/2020/02/03/1189.html)即是采用Bailongma将Prometheus和Telegraf的数据写入TDengine中的示例，可以参考。
### 从源代码编译blm_prometheus ### 从源代码编译 blm_prometheus
用户需要从github下载[Bailongma](https://github.com/taosdata/Bailongma)的源码，使用Golang语言编译器编译生成可执行文件。在开始编译前，需要准备好以下条件： 用户需要从github下载[Bailongma](https://github.com/taosdata/Bailongma)的源码，使用Golang语言编译器编译生成可执行文件。在开始编译前，需要准备好以下条件：
- Linux操作系统的服务器 - Linux操作系统的服务器
...@@ -46,11 +108,11 @@ go build ...@@ -46,11 +108,11 @@ go build
一切正常的情况下，就会在对应的目录下生成一个blm_prometheus的可执行程序。 一切正常的情况下，就会在对应的目录下生成一个blm_prometheus的可执行程序。
### 安装Prometheus ### 安装 Prometheus
通过Prometheus的官网下载安装。具体请见：[下载地址](https://prometheus.io/download/)。 通过Prometheus的官网下载安装。具体请见：[下载地址](https://prometheus.io/download/)。
### 配置Prometheus ### 配置 Prometheus
参考Prometheus的[配置文档](https://prometheus.io/docs/prometheus/latest/configuration/configuration/)，在Prometheus的配置文件中的<remote_write>部分，增加以下配置： 参考Prometheus的[配置文档](https://prometheus.io/docs/prometheus/latest/configuration/configuration/)，在Prometheus的配置文件中的<remote_write>部分，增加以下配置：
...@@ -60,7 +122,8 @@ go build ...@@ -60,7 +122,8 @@ go build
启动Prometheus后，可以通过taos客户端查询确认数据是否成功写入。 启动Prometheus后，可以通过taos客户端查询确认数据是否成功写入。
### 启动blm_prometheus程序 ### 启动 blm_prometheus 程序
blm_prometheus程序有以下选项，在启动blm_prometheus程序时可以通过设定这些选项来设定blm_prometheus的配置。 blm_prometheus程序有以下选项，在启动blm_prometheus程序时可以通过设定这些选项来设定blm_prometheus的配置。
```bash ```bash
--tdengine-name --tdengine-name
...@@ -94,7 +157,8 @@ remote_write: ...@@ -94,7 +157,8 @@ remote_write:
- url: "http://10.1.2.3:8088/receive" - url: "http://10.1.2.3:8088/receive"
``` ```
### 查询prometheus写入数据 ### 查询 prometheus 写入数据
prometheus产生的数据格式如下： prometheus产生的数据格式如下：
```json ```json
{ {
...@@ -105,10 +169,10 @@ prometheus产生的数据格式如下： ...@@ -105,10 +169,10 @@ prometheus产生的数据格式如下：
instance="192.168.99.116:8443", instance="192.168.99.116:8443",
job="kubernetes-apiservers", job="kubernetes-apiservers",
le="125000", le="125000",
resource="persistentvolumes", s resource="persistentvolumes",
cope="cluster", scope="cluster",
verb="LIST", verb="LIST",
version=“v1" version="v1"
} }
} }
``` ```
...@@ -118,11 +182,11 @@ use prometheus; ...@@ -118,11 +182,11 @@ use prometheus;
select * from apiserver_request_latencies_bucket; select * from apiserver_request_latencies_bucket;
``` ```
## <a class="anchor" id="telegraf"></a>Telegraf直接写入 ## <a class="anchor" id="telegraf"></a>Telegraf 直接写入
[Telegraf](https://www.influxdata.com/time-series-platform/telegraf/)是一流行的IT运维数据采集开源工具，TDengine提供一个小工具[Bailongma](https://github.com/taosdata/Bailongma)，只需在Telegraf做简单配置，无需任何代码，就可将Telegraf采集的数据直接写入TDengine，并按规则在TDengine自动创建库和相关表项。博文[用Docker容器快速搭建一个Devops监控Demo](https://www.taosdata.com/blog/2020/02/03/1189.html)即是采用bailongma将Prometheus和Telegraf的数据写入TDengine中的示例，可以参考。 [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/)是一流行的IT运维数据采集开源工具，TDengine提供一个小工具[Bailongma](https://github.com/taosdata/Bailongma)，只需在Telegraf做简单配置，无需任何代码，就可将Telegraf采集的数据直接写入TDengine，并按规则在TDengine自动创建库和相关表项。博文[用Docker容器快速搭建一个Devops监控Demo](https://www.taosdata.com/blog/2020/02/03/1189.html)即是采用bailongma将Prometheus和Telegraf的数据写入TDengine中的示例，可以参考。
### 从源代码编译blm_telegraf ### 从源代码编译 blm_telegraf
用户需要从github下载[Bailongma](https://github.com/taosdata/Bailongma)的源码，使用Golang语言编译器编译生成可执行文件。在开始编译前，需要准备好以下条件： 用户需要从github下载[Bailongma](https://github.com/taosdata/Bailongma)的源码，使用Golang语言编译器编译生成可执行文件。在开始编译前，需要准备好以下条件：
...@@ -139,11 +203,11 @@ go build ...@@ -139,11 +203,11 @@ go build
一切正常的情况下，就会在对应的目录下生成一个blm_telegraf的可执行程序。 一切正常的情况下，就会在对应的目录下生成一个blm_telegraf的可执行程序。
### 安装Telegraf ### 安装 Telegraf
目前TDengine支持Telegraf 1.7.4以上的版本。用户可以根据当前的操作系统，到Telegraf官网下载安装包，并执行安装。下载地址如下：https://portal.influxdata.com/downloads 。 目前TDengine支持Telegraf 1.7.4以上的版本。用户可以根据当前的操作系统，到Telegraf官网下载安装包，并执行安装。下载地址如下：https://portal.influxdata.com/downloads 。
### 配置Telegraf ### 配置 Telegraf
修改Telegraf配置文件/etc/telegraf/telegraf.conf中与TDengine有关的配置项。 修改Telegraf配置文件/etc/telegraf/telegraf.conf中与TDengine有关的配置项。
...@@ -160,7 +224,8 @@ go build ...@@ -160,7 +224,8 @@ go build
关于如何使用Telegraf采集数据以及更多有关使用Telegraf的信息，请参考Telegraf官方的[文档](https://docs.influxdata.com/telegraf/v1.11/)。 关于如何使用Telegraf采集数据以及更多有关使用Telegraf的信息，请参考Telegraf官方的[文档](https://docs.influxdata.com/telegraf/v1.11/)。
### 启动blm_telegraf程序 ### 启动 blm_telegraf 程序
blm_telegraf程序有以下选项，在启动blm_telegraf程序时可以通过设定这些选项来设定blm_telegraf的配置。 blm_telegraf程序有以下选项，在启动blm_telegraf程序时可以通过设定这些选项来设定blm_telegraf的配置。
```bash ```bash
...@@ -196,7 +261,7 @@ blm_telegraf对telegraf提供服务的端口号。 ...@@ -196,7 +261,7 @@ blm_telegraf对telegraf提供服务的端口号。
url = "http://10.1.2.3:8089/telegraf" url = "http://10.1.2.3:8089/telegraf"
``` ```
### 查询telegraf写入数据 ### 查询 telegraf 写入数据
telegraf产生的数据格式如下： telegraf产生的数据格式如下：
```json ```json
......
documentation20/cn/08.connector/docs.md
浏览文件 @ e15abb2e
...@@ -403,6 +403,25 @@ typedef struct TAOS_MULTI_BIND { ...@@ -403,6 +403,25 @@ typedef struct TAOS_MULTI_BIND {
（2.1.3.0 版本新增） （2.1.3.0 版本新增）
用于在其他 stmt API 返回错误（返回错误码或空指针）时获取错误信息。 用于在其他 stmt API 返回错误（返回错误码或空指针）时获取错误信息。
<a class="anchor" id="schemaless"></a>
### Schemaless 方式写入接口
除了使用 SQL 方式或者使用参数绑定 API 写入数据外，还可以使用 Schemaless 的方式完成写入。Schemaless 可以免于预先创建超级表/数据子表的数据结构，而是可以直接写入数据，TDengine 系统会根据写入的数据内容自动创建和维护所需要的表结构。Schemaless 的使用方式详见 [Schemaless 写入](https://www.taosdata.com/cn/documentation/insert#schemaless) 章节，这里介绍与之配套使用的 C/C++ API。
- `int taos_insert_lines(TAOS* taos, char* lines[], int numLines)`
（2.2.0.0 版本新增）
以 Schemaless 格式写入多行数据。其中：
* taos：调用 taos_connect 返回的数据库连接。
* lines：由 char 字符串指针组成的数组，指向本次想要写入数据库的多行数据。
* numLines：lines 数据的总行数。
返回值为 0 表示写入成功，非零值表示出错。具体错误代码请参见 [taoserror.h](https://github.com/taosdata/TDengine/blob/develop/src/inc/taoserror.h) 文件。
说明：
1. 此接口是一个同步阻塞式接口，使用时机与 `taos_query()` 一致。
2. 在调用此接口之前，必须先调用 `taos_select_db()` 来确定目前是在向哪个 DB 来写入。
### 连续查询接口 ### 连续查询接口
TDengine提供时间驱动的实时流式计算API。可以每隔一指定的时间段，对一张或多张数据库的表(数据流)进行各种实时聚合计算操作。操作简单，仅有打开、关闭流的API。具体如下： TDengine提供时间驱动的实时流式计算API。可以每隔一指定的时间段，对一张或多张数据库的表(数据流)进行各种实时聚合计算操作。操作简单，仅有打开、关闭流的API。具体如下：
......
documentation20/cn/11.administrator/docs.md
浏览文件 @ e15abb2e
...@@ -568,6 +568,35 @@ COMPACT 命令对指定的一个或多个 VGroup 启动碎片重整，系统会 ...@@ -568,6 +568,35 @@ COMPACT 命令对指定的一个或多个 VGroup 启动碎片重整，系统会
需要注意的是，碎片重整操作会大幅消耗磁盘 I/O。因此在重整进行期间，有可能会影响节点的写入和查询性能，甚至在极端情况下导致短时间的阻写。 需要注意的是，碎片重整操作会大幅消耗磁盘 I/O。因此在重整进行期间，有可能会影响节点的写入和查询性能，甚至在极端情况下导致短时间的阻写。
<a class="anchor" id="tsz_compress"></a>
## 浮点数有损压缩
在车联网等物联网智能应用场景中，经常会采集和存储海量的浮点数类型数据，如果能更高效地对此类数据进行压缩，那么不但能够节省数据存储的硬件资源，也能够因降低磁盘 I/O 数据量而提升系统性能表现。
从 2.1.6.0 版本开始，TDengine 提供一种名为 TSZ 的新型数据压缩算法，无论设置为有损压缩还是无损压缩，都能够显著提升浮点数类型数据的压缩率表现。目前该功能以可选模块的方式进行发布，可以通过添加特定的编译参数来启用该功能（也即常规安装包中暂未包含该功能）。
**需要注意的是，该功能一旦启用，效果是全局的，也即会对系统中所有的 FLOAT、DOUBLE 类型的数据生效。同时，在启用了浮点数有损压缩功能后写入的数据，也无法被未启用该功能的版本载入，并有可能因此而导致数据库服务报错退出。**
### 创建支持 TSZ 压缩算法的 TDengine 版本
TSZ 模块保存在单独的代码仓库 https://github.com/taosdata/TSZ 中。可以通过以下步骤创建包含此模块的 TDengine 版本：
1. TDengine 中的插件目前只支持通过 SSH 的方式拉取和编译，所以需要自己先配置好通过 SSH 拉取 GitHub 代码的环境。
2. `git clone git@github.com:taosdata/TDengine -b your_branchname --recurse-submodules` 通过 `--recurse-submodules` 使依赖模块的源代码可以被一并下载。
3. `mkdir debug && cd debug` 进入单独的编译目录。
4. `cmake .. -DTSZ_ENABLED=true` 其中参数 `-DTSZ_ENABLED=true` 表示在编译过程中加入对 TSZ 插件功能的支持。如果成功激活对 TSZ 模块的编译，那么 CMAKE 过程中也会显示 `build with TSZ enabled` 字样。
5. 编译成功后，包含 TSZ 浮点压缩功能的插件便已经编译进了 TDengine 中了，可以通过调整 taos.cfg 中的配置参数来使用此功能了。
### 通过配置文件来启用 TSZ 压缩算法
如果要启用 TSZ 压缩算法，除了在 TDengine 的编译过程需要声明启用 TSZ 模块之外，还需要在 taos.cfg 配置文件中对以下参数进行设置：
* lossyColumns：配置要进行有损压缩的浮点数数据类型。参数值类型为字符串，含义为：空 - 关闭有损压缩；float - 只对 FLOAT 类型进行有损压缩；double - 只对 DOUBLE 类型进行有损压缩；float|double：对 FLOAT 和 DOUBLE 类型都进行有损压缩。默认值是“空”，也即关闭有损压缩。
* fPrecision：设置 float 类型浮点数压缩精度，小于此值的浮点数尾数部分将被截断。参数值类型为 FLOAT，最小值为 0.0，最大值为 100,000.0。缺省值为 0.00000001（1E-8）。
* dPrecision：设置 double 类型浮点数压缩精度，小于此值的浮点数尾数部分将被截断。参数值类型为 DOUBLE，最小值为 0.0，最大值为 100,000.0。缺省值为 0.0000000000000001（1E-16）。
* maxRange：表示数据的最大浮动范围。一般无需调整，在数据具有特定特征时可以配合 range 参数来实现极高的数据压缩率。默认值为 500。
* range：表示数据大体浮动范围。一般无需调整，在数据具有特定特征时可以配合 maxRange 参数来实现极高的数据压缩率。默认值为 100。
**注意：**对 cfg 配置文件中参数值的任何调整，都需要重新启动 taosd 才能生效。并且以上选项为全局配置选项，配置后对所有数据库中所有表的 FLOAT 及 DOUBLE 类型的字段生效。
## <a class="anchor" id="directories"></a>文件目录结构 ## <a class="anchor" id="directories"></a>文件目录结构
安装TDengine后，默认会在操作系统中生成下列目录或文件： 安装TDengine后，默认会在操作系统中生成下列目录或文件：
......
documentation20/cn/12.taos-sql/docs.md
浏览文件 @ e15abb2e
...@@ -730,6 +730,34 @@ Query OK, 1 row(s) in set (0.001091s) ...@@ -730,6 +730,34 @@ Query OK, 1 row(s) in set (0.001091s)
5. 从 2.0.17.0 版本开始，条件过滤开始支持 BETWEEN AND 语法，例如 `WHERE col2 BETWEEN 1.5 AND 3.25` 表示查询条件为“1.5 ≤ col2 ≤ 3.25”。 5. 从 2.0.17.0 版本开始，条件过滤开始支持 BETWEEN AND 语法，例如 `WHERE col2 BETWEEN 1.5 AND 3.25` 表示查询条件为“1.5 ≤ col2 ≤ 3.25”。
6. 从 2.1.4.0 版本开始，条件过滤开始支持 IN 算子，例如 `WHERE city IN ('Beijing', 'Shanghai')`。说明：BOOL 类型写作 `{true, false}` 或 `{0, 1}` 均可，但不能写作 0、1 之外的整数；FLOAT 和 DOUBLE 类型会受到浮点数精度影响，集合内的值在精度范围内认为和数据行的值完全相等才能匹配成功；TIMESTAMP 类型支持非主键的列。<!-- REPLACE_OPEN_TO_ENTERPRISE__IN_OPERATOR_AND_UNSIGNED_INTEGER --> 6. 从 2.1.4.0 版本开始，条件过滤开始支持 IN 算子，例如 `WHERE city IN ('Beijing', 'Shanghai')`。说明：BOOL 类型写作 `{true, false}` 或 `{0, 1}` 均可，但不能写作 0、1 之外的整数；FLOAT 和 DOUBLE 类型会受到浮点数精度影响，集合内的值在精度范围内认为和数据行的值完全相等才能匹配成功；TIMESTAMP 类型支持非主键的列。<!-- REPLACE_OPEN_TO_ENTERPRISE__IN_OPERATOR_AND_UNSIGNED_INTEGER -->
<a class="anchor" id="join"></a>
### JOIN 子句
从 2.2.0.0 版本开始，TDengine 对内连接（INNER JOIN）中的自然连接（Natural join）操作实现了完整的支持。也即支持“普通表与普通表之间”、“超级表与超级表之间”、“子查询与子查询之间”进行自然连接。自然连接与内连接的主要区别是，自然连接要求参与连接的字段在不同的表/超级表中必须是同名字段。也即，TDengine 在连接关系的表达中，要求必须使用同名数据列/标签列的相等关系。
在普通表与普通表之间的 JOIN 操作中，只能使用主键时间戳之间的相等关系。例如：
```sql
SELECT *
FROM temp_tb_1 t1, pressure_tb_1 t2
WHERE t1.ts = t2.ts
```
在超级表与超级表之间的 JOIN 操作中，除了主键时间戳一致的条件外，还要求引入能实现一一对应的标签列的相等关系。例如：
```sql
SELECT *
FROM temp_stable t1, temp_stable t2
WHERE t1.ts = t2.ts AND t1.deviceid = t2.deviceid AND t1.status=0;
```
类似地，也可以对多个子查询的查询结果进行 JOIN 操作。
注意，JOIN 操作存在如下限制要求：
1. 参与一条语句中 JOIN 操作的表/超级表最多可以有 10 个。
2. 在包含 JOIN 操作的查询语句中不支持 FILL。
3. 暂不支持参与 JOIN 操作的表之间聚合后的四则运算。
4. 不支持只对其中一部分表做 GROUP BY。
5. JOIN 查询的不同表的过滤条件之间不能为 OR。
<a class="anchor" id="nested"></a> <a class="anchor" id="nested"></a>
### 嵌套查询 ### 嵌套查询
...@@ -757,7 +785,7 @@ SELECT ... FROM (SELECT ... FROM ...) ...; ...@@ -757,7 +785,7 @@ SELECT ... FROM (SELECT ... FROM ...) ...;
* 外层查询不支持 GROUP BY。 * 外层查询不支持 GROUP BY。
<a class="anchor" id="union"></a> <a class="anchor" id="union"></a>
### UNION ALL 操作符 ### UNION ALL 子句
```mysql ```mysql
SELECT ... SELECT ...
...@@ -1486,12 +1514,6 @@ SELECT AVG(current), MAX(current), LEASTSQUARES(current, start_val, step_val), P ...@@ -1486,12 +1514,6 @@ SELECT AVG(current), MAX(current), LEASTSQUARES(current, start_val, step_val), P
TAOS SQL 支持对标签、TBNAME 进行 GROUP BY 操作，也支持普通列进行 GROUP BY，前提是：仅限一列且该列的唯一值小于 10 万个。 TAOS SQL 支持对标签、TBNAME 进行 GROUP BY 操作，也支持普通列进行 GROUP BY，前提是：仅限一列且该列的唯一值小于 10 万个。
**JOIN 操作的限制**
TAOS SQL 支持表之间按主键时间戳来 join 两张表的列，暂不支持两个表之间聚合后的四则运算。
JOIN 查询的不同表的过滤条件之间不能为 OR。
**IS NOT NULL 与不为空的表达式适用范围** **IS NOT NULL 与不为空的表达式适用范围**
IS NOT NULL 支持所有类型的列。不为空的表达式为 <>""，仅对非数值类型的列适用。 IS NOT NULL 支持所有类型的列。不为空的表达式为 <>""，仅对非数值类型的列适用。
......
documentation20/en/12.taos-sql/docs.md
浏览文件 @ e15abb2e
...@@ -37,7 +37,7 @@ With TDengine, the most important thing is timestamp. When creating and insertin ...@@ -37,7 +37,7 @@ With TDengine, the most important thing is timestamp. When creating and insertin
- Epch Time: a timestamp value can also be a long integer representing milliseconds since 1970-01-01 08:00:00.000. - Epch Time: a timestamp value can also be a long integer representing milliseconds since 1970-01-01 08:00:00.000.
- Arithmetic operations can be applied to timestamp. For example: now-2h represents a timestamp which is 2 hours ago from the current server time. Units include u( microsecond), a (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks). In `select * from t1 where ts > now-2w and ts <= now-1w`, which queries data of the whole week before two weeks. To specify the interval of down sampling, you can also use n(calendar month) and y(calendar year) as time units. - Arithmetic operations can be applied to timestamp. For example: now-2h represents a timestamp which is 2 hours ago from the current server time. Units include u( microsecond), a (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks). In `select * from t1 where ts > now-2w and ts <= now-1w`, which queries data of the whole week before two weeks. To specify the interval of down sampling, you can also use n(calendar month) and y(calendar year) as time units.
Default time precision of TDengine is millisecond, you can change it to microseocnd by setting parameter enableMicrosecond. TDengine's timestamp is set to millisecond accuracy by default. Microsecond/nanosecond accuracy can be set using CREATE DATABASE with PRECISION parameter. (Nanosecond resolution is supported from version 2.1.5.0 onwards.)
In TDengine, the following 10 data types can be used in data model of an ordinary table. In TDengine, the following 10 data types can be used in data model of an ordinary table.
......
src/client/inc/tscUtil.h
浏览文件 @ e15abb2e
...@@ -92,7 +92,7 @@ typedef struct SMergeTsCtx { ...@@ -92,7 +92,7 @@ typedef struct SMergeTsCtx {
}SMergeTsCtx; }SMergeTsCtx;
typedef struct SVgroupTableInfo { typedef struct SVgroupTableInfo {
SVgroupInfo vgInfo; SVgroupMsg vgInfo;
SArray *itemList; // SArray<STableIdInfo> SArray *itemList; // SArray<STableIdInfo>
} SVgroupTableInfo; } SVgroupTableInfo;
...@@ -174,7 +174,9 @@ void tscClearInterpInfo(SQueryInfo* pQueryInfo); ...@@ -174,7 +174,9 @@ void tscClearInterpInfo(SQueryInfo* pQueryInfo);
bool tscIsInsertData(char* sqlstr); bool tscIsInsertData(char* sqlstr);
int tscAllocPayload(SSqlCmd* pCmd, int size); // the memory is not reset in case of fast allocate payload function
int32_t tscAllocPayloadFast(SSqlCmd *pCmd, size_t size);
int32_t tscAllocPayload(SSqlCmd* pCmd, int size);
TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes); TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes);
...@@ -288,7 +290,11 @@ void doExecuteQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo); ...@@ -288,7 +290,11 @@ void doExecuteQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo);
SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *pInfo); SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *pInfo);
void* tscVgroupInfoClear(SVgroupsInfo *pInfo); void* tscVgroupInfoClear(SVgroupsInfo *pInfo);
#if 0
void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src); void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src);
#endif
/** /**
* The create object function must be successful expect for the out of memory issue. * The create object function must be successful expect for the out of memory issue.
* *
......
src/client/inc/tsclient.h
浏览文件 @ e15abb2e
...@@ -234,7 +234,6 @@ typedef struct STableDataBlocks { ...@@ -234,7 +234,6 @@ typedef struct STableDataBlocks {
typedef struct { typedef struct {
STableMeta *pTableMeta; STableMeta *pTableMeta;
SArray *vgroupIdList; SArray *vgroupIdList;
// SVgroupsInfo *pVgroupsInfo;
} STableMetaVgroupInfo; } STableMetaVgroupInfo;
typedef struct SInsertStatementParam { typedef struct SInsertStatementParam {
...@@ -286,20 +285,14 @@ typedef struct { ...@@ -286,20 +285,14 @@ typedef struct {
int32_t resColumnId; int32_t resColumnId;
} SSqlCmd; } SSqlCmd;
typedef struct SResRec {
int numOfRows;
int numOfTotal;
} SResRec;
typedef struct { typedef struct {
int32_t numOfRows; // num of results in current retrieval int32_t numOfRows; // num of results in current retrieval
int64_t numOfRowsGroup; // num of results of current group
int64_t numOfTotal; // num of total results int64_t numOfTotal; // num of total results
int64_t numOfClauseTotal; // num of total result in current subclause int64_t numOfClauseTotal; // num of total result in current subclause
char * pRsp; char * pRsp;
int32_t rspType; int32_t rspType;
int32_t rspLen; int32_t rspLen;
uint64_t qId; uint64_t qId; // query id of SQInfo
int64_t useconds; int64_t useconds;
int64_t offset; // offset value from vnode during projection query of stable int64_t offset; // offset value from vnode during projection query of stable
int32_t row; int32_t row;
...@@ -307,8 +300,6 @@ typedef struct { ...@@ -307,8 +300,6 @@ typedef struct {
int16_t precision; int16_t precision;
bool completed; bool completed;
int32_t code; int32_t code;
int32_t numOfGroups;
SResRec * pGroupRec;
char * data; char * data;
TAOS_ROW tsrow; TAOS_ROW tsrow;
TAOS_ROW urow; TAOS_ROW urow;
...@@ -316,8 +307,7 @@ typedef struct { ...@@ -316,8 +307,7 @@ typedef struct {
char ** buffer; // Buffer used to put multibytes encoded using unicode (wchar_t) char ** buffer; // Buffer used to put multibytes encoded using unicode (wchar_t)
SColumnIndex* pColumnIndex; SColumnIndex* pColumnIndex;
TAOS_FIELD* final; TAOS_FIELD* final;
SArithmeticSupport *pArithSup; // support the arithmetic expression calculation on agg functions
struct SGlobalMerger *pMerger; struct SGlobalMerger *pMerger;
} SSqlRes; } SSqlRes;
...@@ -377,7 +367,6 @@ typedef struct SSqlObj { ...@@ -377,7 +367,6 @@ typedef struct SSqlObj {
tsem_t rspSem; tsem_t rspSem;
SSqlCmd cmd; SSqlCmd cmd;
SSqlRes res; SSqlRes res;
bool isBind;
SSubqueryState subState; SSubqueryState subState;
struct SSqlObj **pSubs; struct SSqlObj **pSubs;
......
src/client/src/tscAsync.c
浏览文件 @ e15abb2e
...@@ -60,17 +60,25 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para ...@@ -60,17 +60,25 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para
tscDebugL("0x%"PRIx64" SQL: %s", pSql->self, pSql->sqlstr); tscDebugL("0x%"PRIx64" SQL: %s", pSql->self, pSql->sqlstr);
pCmd->resColumnId = TSDB_RES_COL_ID; pCmd->resColumnId = TSDB_RES_COL_ID;
taosAcquireRef(tscObjRef, pSql->self);
int32_t code = tsParseSql(pSql, true); int32_t code = tsParseSql(pSql, true);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) return;
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return;
}
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
pSql->res.code = code; pSql->res.code = code;
tscAsyncResultOnError(pSql); tscAsyncResultOnError(pSql);
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} }
SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd);
executeQuery(pSql, pQueryInfo); executeQuery(pSql, pQueryInfo);
taosReleaseRef(tscObjRef, pSql->self);
} }
// TODO return the correct error code to client in tscQueueAsyncError // TODO return the correct error code to client in tscQueueAsyncError
......
src/client/src/tscPrepare.c
浏览文件 @ e15abb2e
...@@ -1491,7 +1491,6 @@ TAOS_STMT* taos_stmt_init(TAOS* taos) { ...@@ -1491,7 +1491,6 @@ TAOS_STMT* taos_stmt_init(TAOS* taos) {
pSql->signature = pSql; pSql->signature = pSql;
pSql->pTscObj = pObj; pSql->pTscObj = pObj;
pSql->maxRetry = TSDB_MAX_REPLICA; pSql->maxRetry = TSDB_MAX_REPLICA;
pSql->isBind = true;
pStmt->pSql = pSql; pStmt->pSql = pSql;
pStmt->last = STMT_INIT; pStmt->last = STMT_INIT;
......
src/client/src/tscSQLParser.c
浏览文件 @ e15abb2e
...@@ -8697,7 +8697,7 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod ...@@ -8697,7 +8697,7 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
if (p->vgroupIdList != NULL) { if (p->vgroupIdList != NULL) {
size_t s = taosArrayGetSize(p->vgroupIdList); size_t s = taosArrayGetSize(p->vgroupIdList);
size_t vgroupsz = sizeof(SVgroupInfo) * s + sizeof(SVgroupsInfo); size_t vgroupsz = sizeof(SVgroupMsg) * s + sizeof(SVgroupsInfo);
pTableMetaInfo->vgroupList = calloc(1, vgroupsz); pTableMetaInfo->vgroupList = calloc(1, vgroupsz);
if (pTableMetaInfo->vgroupList == NULL) { if (pTableMetaInfo->vgroupList == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY; return TSDB_CODE_TSC_OUT_OF_MEMORY;
...@@ -8712,14 +8712,11 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod ...@@ -8712,14 +8712,11 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
taosHashGetClone(tscVgroupMap, id, sizeof(*id), NULL, &existVgroupInfo); taosHashGetClone(tscVgroupMap, id, sizeof(*id), NULL, &existVgroupInfo);
assert(existVgroupInfo.inUse >= 0); assert(existVgroupInfo.inUse >= 0);
SVgroupInfo *pVgroup = &pTableMetaInfo->vgroupList->vgroups[j]; SVgroupMsg *pVgroup = &pTableMetaInfo->vgroupList->vgroups[j];
pVgroup->numOfEps = existVgroupInfo.numOfEps; pVgroup->numOfEps = existVgroupInfo.numOfEps;
pVgroup->vgId = existVgroupInfo.vgId; pVgroup->vgId = existVgroupInfo.vgId;
for (int32_t k = 0; k < existVgroupInfo.numOfEps; ++k) { memcpy(&pVgroup->epAddr, &existVgroupInfo.ep, sizeof(pVgroup->epAddr));
pVgroup->epAddr[k].port = existVgroupInfo.ep[k].port;
pVgroup->epAddr[k].fqdn = strndup(existVgroupInfo.ep[k].fqdn, TSDB_FQDN_LEN);
}
} }
} }
} }
......
src/client/src/tscServer.c
浏览文件 @ e15abb2e
...@@ -73,7 +73,7 @@ static int32_t removeDupVgid(int32_t *src, int32_t sz) { ...@@ -73,7 +73,7 @@ static int32_t removeDupVgid(int32_t *src, int32_t sz) {
return ret; return ret;
} }
static void tscSetDnodeEpSet(SRpcEpSet* pEpSet, SVgroupInfo* pVgroupInfo) { static void tscSetDnodeEpSet(SRpcEpSet* pEpSet, SVgroupMsg* pVgroupInfo) {
assert(pEpSet != NULL && pVgroupInfo != NULL && pVgroupInfo->numOfEps > 0); assert(pEpSet != NULL && pVgroupInfo != NULL && pVgroupInfo->numOfEps > 0);
// Issue the query to one of the vnode among a vgroup randomly. // Issue the query to one of the vnode among a vgroup randomly.
...@@ -93,6 +93,7 @@ static void tscSetDnodeEpSet(SRpcEpSet* pEpSet, SVgroupInfo* pVgroupInfo) { ...@@ -93,6 +93,7 @@ static void tscSetDnodeEpSet(SRpcEpSet* pEpSet, SVgroupInfo* pVgroupInfo) {
existed = true; existed = true;
} }
} }
assert(existed); assert(existed);
} }
...@@ -723,7 +724,7 @@ static char *doSerializeTableInfo(SQueryTableMsg *pQueryMsg, SSqlObj *pSql, STab ...@@ -723,7 +724,7 @@ static char *doSerializeTableInfo(SQueryTableMsg *pQueryMsg, SSqlObj *pSql, STab
int32_t index = pTableMetaInfo->vgroupIndex; int32_t index = pTableMetaInfo->vgroupIndex;
assert(index >= 0); assert(index >= 0);
SVgroupInfo* pVgroupInfo = NULL; SVgroupMsg* pVgroupInfo = NULL;
if (pTableMetaInfo->vgroupList && pTableMetaInfo->vgroupList->numOfVgroups > 0) { if (pTableMetaInfo->vgroupList && pTableMetaInfo->vgroupList->numOfVgroups > 0) {
assert(index < pTableMetaInfo->vgroupList->numOfVgroups); assert(index < pTableMetaInfo->vgroupList->numOfVgroups);
pVgroupInfo = &pTableMetaInfo->vgroupList->vgroups[index]; pVgroupInfo = &pTableMetaInfo->vgroupList->vgroups[index];
...@@ -861,8 +862,8 @@ static int32_t serializeSqlExpr(SSqlExpr* pExpr, STableMetaInfo* pTableMetaInfo, ...@@ -861,8 +862,8 @@ static int32_t serializeSqlExpr(SSqlExpr* pExpr, STableMetaInfo* pTableMetaInfo,
(*pMsg) += sizeof(SSqlExpr); (*pMsg) += sizeof(SSqlExpr);
for (int32_t j = 0; j < pExpr->numOfParams; ++j) { // todo add log for (int32_t j = 0; j < pExpr->numOfParams; ++j) { // todo add log
pSqlExpr->param[j].nType = htons((uint16_t)pExpr->param[j].nType); pSqlExpr->param[j].nType = htonl(pExpr->param[j].nType);
pSqlExpr->param[j].nLen = htons(pExpr->param[j].nLen); pSqlExpr->param[j].nLen = htonl(pExpr->param[j].nLen);
if (pExpr->param[j].nType == TSDB_DATA_TYPE_BINARY) { if (pExpr->param[j].nType == TSDB_DATA_TYPE_BINARY) {
memcpy((*pMsg), pExpr->param[j].pz, pExpr->param[j].nLen); memcpy((*pMsg), pExpr->param[j].pz, pExpr->param[j].nLen);
...@@ -880,17 +881,22 @@ static int32_t serializeSqlExpr(SSqlExpr* pExpr, STableMetaInfo* pTableMetaInfo, ...@@ -880,17 +881,22 @@ static int32_t serializeSqlExpr(SSqlExpr* pExpr, STableMetaInfo* pTableMetaInfo,
int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
SQueryInfo *pQueryInfo = NULL;
STableMeta *pTableMeta = NULL;
STableMetaInfo *pTableMetaInfo = NULL;
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
int32_t size = tscEstimateQueryMsgSize(pSql); int32_t size = tscEstimateQueryMsgSize(pSql);
assert(size > 0);
if (TSDB_CODE_SUCCESS != tscAllocPayload(pCmd, size)) { if (TSDB_CODE_SUCCESS != tscAllocPayloadFast(pCmd, size)) {
tscError("%p failed to malloc for query msg", pSql); tscError("%p failed to malloc for query msg", pSql);
return TSDB_CODE_TSC_INVALID_OPERATION; // todo add test for this return TSDB_CODE_TSC_INVALID_OPERATION; // todo add test for this
} }
SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); pQueryInfo = tscGetQueryInfo(pCmd);
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableMeta * pTableMeta = pTableMetaInfo->pTableMeta; pTableMeta = pTableMetaInfo->pTableMeta;
SQueryAttr query = {{0}}; SQueryAttr query = {{0}};
tscCreateQueryFromQueryInfo(pQueryInfo, &query, pSql); tscCreateQueryFromQueryInfo(pQueryInfo, &query, pSql);
...@@ -941,14 +947,13 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -941,14 +947,13 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->pointInterpQuery = query.pointInterpQuery; pQueryMsg->pointInterpQuery = query.pointInterpQuery;
pQueryMsg->needReverseScan = query.needReverseScan; pQueryMsg->needReverseScan = query.needReverseScan;
pQueryMsg->stateWindow = query.stateWindow; pQueryMsg->stateWindow = query.stateWindow;
pQueryMsg->numOfTags = htonl(numOfTags); pQueryMsg->numOfTags = htonl(numOfTags);
pQueryMsg->sqlstrLen = htonl(sqlLen); pQueryMsg->sqlstrLen = htonl(sqlLen);
pQueryMsg->sw.gap = htobe64(query.sw.gap); pQueryMsg->sw.gap = htobe64(query.sw.gap);
pQueryMsg->sw.primaryColId = htonl(PRIMARYKEY_TIMESTAMP_COL_INDEX); pQueryMsg->sw.primaryColId = htonl(PRIMARYKEY_TIMESTAMP_COL_INDEX);
pQueryMsg->secondStageOutput = htonl(query.numOfExpr2); pQueryMsg->secondStageOutput = htonl(query.numOfExpr2);
pQueryMsg->numOfOutput = htons((int16_t)query.numOfOutput); // this is the stage one output column number pQueryMsg->numOfOutput = htons((int16_t)query.numOfOutput); // this is the stage one output column number
pQueryMsg->numOfGroupCols = htons(pQueryInfo->groupbyExpr.numOfGroupCols); pQueryMsg->numOfGroupCols = htons(pQueryInfo->groupbyExpr.numOfGroupCols);
pQueryMsg->tagNameRelType = htons(pQueryInfo->tagCond.relType); pQueryMsg->tagNameRelType = htons(pQueryInfo->tagCond.relType);
...@@ -968,7 +973,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -968,7 +973,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->tableCols[i].type = htons(pCol->type); pQueryMsg->tableCols[i].type = htons(pCol->type);
//pQueryMsg->tableCols[i].flist.numOfFilters = htons(pCol->flist.numOfFilters); //pQueryMsg->tableCols[i].flist.numOfFilters = htons(pCol->flist.numOfFilters);
pQueryMsg->tableCols[i].flist.numOfFilters = 0; pQueryMsg->tableCols[i].flist.numOfFilters = 0;
pQueryMsg->tableCols[i].flist.filterInfo = 0;
// append the filter information after the basic column information // append the filter information after the basic column information
//serializeColFilterInfo(pCol->flist.filterInfo, pCol->flist.numOfFilters, &pMsg); //serializeColFilterInfo(pCol->flist.filterInfo, pCol->flist.numOfFilters, &pMsg);
} }
...@@ -981,6 +986,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -981,6 +986,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += pCond->len; pMsg += pCond->len;
} }
} else {
pQueryMsg->colCondLen = 0;
} }
for (int32_t i = 0; i < query.numOfOutput; ++i) { for (int32_t i = 0; i < query.numOfOutput; ++i) {
...@@ -1060,6 +1067,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -1060,6 +1067,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += pCond->len; pMsg += pCond->len;
} }
} else {
pQueryMsg->tagCondLen = 0;
} }
if (pQueryInfo->bufLen > 0) { if (pQueryInfo->bufLen > 0) {
...@@ -1089,6 +1098,9 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -1089,6 +1098,9 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->tsBuf.tsOrder = htonl(pQueryInfo->tsBuf->tsOrder); pQueryMsg->tsBuf.tsOrder = htonl(pQueryInfo->tsBuf->tsOrder);
pQueryMsg->tsBuf.tsLen = htonl(pQueryMsg->tsBuf.tsLen); pQueryMsg->tsBuf.tsLen = htonl(pQueryMsg->tsBuf.tsLen);
pQueryMsg->tsBuf.tsNumOfBlocks = htonl(pQueryMsg->tsBuf.tsNumOfBlocks); pQueryMsg->tsBuf.tsNumOfBlocks = htonl(pQueryMsg->tsBuf.tsNumOfBlocks);
} else {
pQueryMsg->tsBuf.tsLen = 0;
pQueryMsg->tsBuf.tsNumOfBlocks = 0;
} }
int32_t numOfOperator = (int32_t) taosArrayGetSize(queryOperator); int32_t numOfOperator = (int32_t) taosArrayGetSize(queryOperator);
...@@ -1126,6 +1138,9 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -1126,6 +1138,9 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += pUdfInfo->contLen; pMsg += pUdfInfo->contLen;
} }
} else {
pQueryMsg->udfContentOffset = 0;
pQueryMsg->udfContentLen = 0;
} }
memcpy(pMsg, pSql->sqlstr, sqlLen); memcpy(pMsg, pSql->sqlstr, sqlLen);
...@@ -2146,7 +2161,7 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t ...@@ -2146,7 +2161,7 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t
*size = (int32_t)(sizeof(SVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsMsg)); *size = (int32_t)(sizeof(SVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsMsg));
size_t vgroupsz = sizeof(SVgroupInfo) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsInfo); size_t vgroupsz = sizeof(SVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsInfo);
SVgroupsInfo *pVgroupInfo = calloc(1, vgroupsz); SVgroupsInfo *pVgroupInfo = calloc(1, vgroupsz);
assert(pVgroupInfo != NULL); assert(pVgroupInfo != NULL);
...@@ -2156,7 +2171,7 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t ...@@ -2156,7 +2171,7 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t
} else { } else {
for (int32_t j = 0; j < pVgroupInfo->numOfVgroups; ++j) { for (int32_t j = 0; j < pVgroupInfo->numOfVgroups; ++j) {
// just init, no need to lock // just init, no need to lock
SVgroupInfo *pVgroup = &pVgroupInfo->vgroups[j]; SVgroupMsg *pVgroup = &pVgroupInfo->vgroups[j];
SVgroupMsg *vmsg = &pVgroupMsg->vgroups[j]; SVgroupMsg *vmsg = &pVgroupMsg->vgroups[j];
vmsg->vgId = htonl(vmsg->vgId); vmsg->vgId = htonl(vmsg->vgId);
...@@ -2168,7 +2183,8 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t ...@@ -2168,7 +2183,8 @@ static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t
pVgroup->vgId = vmsg->vgId; pVgroup->vgId = vmsg->vgId;
for (int32_t k = 0; k < vmsg->numOfEps; ++k) { for (int32_t k = 0; k < vmsg->numOfEps; ++k) {
pVgroup->epAddr[k].port = vmsg->epAddr[k].port; pVgroup->epAddr[k].port = vmsg->epAddr[k].port;
pVgroup->epAddr[k].fqdn = strndup(vmsg->epAddr[k].fqdn, TSDB_FQDN_LEN); tstrncpy(pVgroup->epAddr[k].fqdn, vmsg->epAddr[k].fqdn, TSDB_FQDN_LEN);
// pVgroup->epAddr[k].fqdn = strndup(vmsg->epAddr[k].fqdn, TSDB_FQDN_LEN);
} }
doUpdateVgroupInfo(pVgroup->vgId, vmsg); doUpdateVgroupInfo(pVgroup->vgId, vmsg);
...@@ -2618,7 +2634,11 @@ int tscProcessAlterTableMsgRsp(SSqlObj *pSql) { ...@@ -2618,7 +2634,11 @@ int tscProcessAlterTableMsgRsp(SSqlObj *pSql) {
tfree(pTableMetaInfo->pTableMeta); tfree(pTableMetaInfo->pTableMeta);
if (isSuperTable) { // if it is a super table, iterate the hashTable and remove all the childTableMeta if (isSuperTable) { // if it is a super table, iterate the hashTable and remove all the childTableMeta
taosHashClear(tscTableMetaMap); if (pSql->res.pRsp == NULL) {
tscDebug("0x%"PRIx64" unexpected resp from mnode, super table: %s failed to update super table meta ", pSql->self, name);
return 0;
}
return tscProcessTableMetaRsp(pSql);
} }
return 0; return 0;
......
src/client/src/tscSubquery.c
浏览文件 @ e15abb2e
...@@ -623,13 +623,12 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { ...@@ -623,13 +623,12 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid); int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid);
// set the tag column id for executor to extract correct tag value // set the tag column id for executor to extract correct tag value
#ifndef _TD_NINGSI_60 tVariant* pVariant = &pExpr->base.param[0];
pExpr->base.param[0] = (tVariant) {.i64 = colId, .nType = TSDB_DATA_TYPE_BIGINT, .nLen = sizeof(int64_t)};
#else pVariant->i64 = colId;
pExpr->base.param[0].i64 = colId; pVariant->nType = TSDB_DATA_TYPE_BIGINT;
pExpr->base.param[0].nType = TSDB_DATA_TYPE_BIGINT; pVariant->nLen = sizeof(int64_t);
pExpr->base.param[0].nLen = sizeof(int64_t);
#endif
pExpr->base.numOfParams = 1; pExpr->base.numOfParams = 1;
} }
...@@ -748,10 +747,11 @@ void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArr ...@@ -748,10 +747,11 @@ void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArr
SVgroupTableInfo info = {{0}}; SVgroupTableInfo info = {{0}};
for (int32_t m = 0; m < pvg->numOfVgroups; ++m) { for (int32_t m = 0; m < pvg->numOfVgroups; ++m) {
if (tt->vgId == pvg->vgroups[m].vgId) { if (tt->vgId == pvg->vgroups[m].vgId) {
tscSVgroupInfoCopy(&info.vgInfo, &pvg->vgroups[m]); memcpy(&info.vgInfo, &pvg->vgroups[m], sizeof(info.vgInfo));
break; break;
} }
} }
assert(info.vgInfo.numOfEps != 0); assert(info.vgInfo.numOfEps != 0);
vgTables = taosArrayInit(4, sizeof(STableIdInfo)); vgTables = taosArrayInit(4, sizeof(STableIdInfo));
...@@ -2463,7 +2463,7 @@ static void doConcurrentlySendSubQueries(SSqlObj* pSql) { ...@@ -2463,7 +2463,7 @@ static void doConcurrentlySendSubQueries(SSqlObj* pSql) {
SSubqueryState *pState = &pSql->subState; SSubqueryState *pState = &pSql->subState;
// concurrently sent the query requests. // concurrently sent the query requests.
const int32_t MAX_REQUEST_PER_TASK = 8; const int32_t MAX_REQUEST_PER_TASK = 4;
int32_t numOfTasks = (pState->numOfSub + MAX_REQUEST_PER_TASK - 1)/MAX_REQUEST_PER_TASK; int32_t numOfTasks = (pState->numOfSub + MAX_REQUEST_PER_TASK - 1)/MAX_REQUEST_PER_TASK;
assert(numOfTasks >= 1); assert(numOfTasks >= 1);
...@@ -2550,13 +2550,14 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { ...@@ -2550,13 +2550,14 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) {
trs->pExtMemBuffer = pMemoryBuf; trs->pExtMemBuffer = pMemoryBuf;
trs->pOrderDescriptor = pDesc; trs->pOrderDescriptor = pDesc;
trs->localBuffer = (tFilePage *)calloc(1, nBufferSize + sizeof(tFilePage)); trs->localBuffer = (tFilePage *)malloc(nBufferSize + sizeof(tFilePage));
if (trs->localBuffer == NULL) { if (trs->localBuffer == NULL) {
tscError("0x%"PRIx64" failed to malloc buffer for local buffer, orderOfSub:%d, reason:%s", pSql->self, i, strerror(errno)); tscError("0x%"PRIx64" failed to malloc buffer for local buffer, orderOfSub:%d, reason:%s", pSql->self, i, strerror(errno));
tfree(trs); tfree(trs);
break; break;
} }
trs->localBuffer->num = 0;
trs->subqueryIndex = i; trs->subqueryIndex = i;
trs->pParentSql = pSql; trs->pParentSql = pSql;
...@@ -2651,7 +2652,7 @@ static int32_t tscReissueSubquery(SRetrieveSupport *oriTrs, SSqlObj *pSql, int32 ...@@ -2651,7 +2652,7 @@ static int32_t tscReissueSubquery(SRetrieveSupport *oriTrs, SSqlObj *pSql, int32
int32_t subqueryIndex = trsupport->subqueryIndex; int32_t subqueryIndex = trsupport->subqueryIndex;
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0);
SVgroupInfo* pVgroup = &pTableMetaInfo->vgroupList->vgroups[0]; SVgroupMsg* pVgroup = &pTableMetaInfo->vgroupList->vgroups[0];
tExtMemBufferClear(trsupport->pExtMemBuffer[subqueryIndex]); tExtMemBufferClear(trsupport->pExtMemBuffer[subqueryIndex]);
...@@ -2879,7 +2880,6 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p ...@@ -2879,7 +2880,6 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p
pParentSql->res.precision = pSql->res.precision; pParentSql->res.precision = pSql->res.precision;
pParentSql->res.numOfRows = 0; pParentSql->res.numOfRows = 0;
pParentSql->res.row = 0; pParentSql->res.row = 0;
pParentSql->res.numOfGroups = 0;
tscFreeRetrieveSup(pSql); tscFreeRetrieveSup(pSql);
...@@ -2930,7 +2930,7 @@ static void tscRetrieveFromDnodeCallBack(void *param, TAOS_RES *tres, int numOfR ...@@ -2930,7 +2930,7 @@ static void tscRetrieveFromDnodeCallBack(void *param, TAOS_RES *tres, int numOfR
SSubqueryState* pState = &pParentSql->subState; SSubqueryState* pState = &pParentSql->subState;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0);
SVgroupInfo *pVgroup = &pTableMetaInfo->vgroupList->vgroups[0]; SVgroupMsg *pVgroup = &pTableMetaInfo->vgroupList->vgroups[0];
if (pParentSql->res.code != TSDB_CODE_SUCCESS) { if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
trsupport->numOfRetry = MAX_NUM_OF_SUBQUERY_RETRY; trsupport->numOfRetry = MAX_NUM_OF_SUBQUERY_RETRY;
...@@ -3058,7 +3058,7 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) { ...@@ -3058,7 +3058,7 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) {
assert(pQueryInfo->numOfTables == 1); assert(pQueryInfo->numOfTables == 1);
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0);
SVgroupInfo* pVgroup = &pTableMetaInfo->vgroupList->vgroups[trsupport->subqueryIndex]; SVgroupMsg* pVgroup = &pTableMetaInfo->vgroupList->vgroups[trsupport->subqueryIndex];
// stable query killed or other subquery failed, all query stopped // stable query killed or other subquery failed, all query stopped
if (pParentSql->res.code != TSDB_CODE_SUCCESS) { if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
...@@ -3404,7 +3404,6 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) { ...@@ -3404,7 +3404,6 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) {
return; return;
} }
// tscRestoreFuncForSTableQuery(pQueryInfo);
int32_t rowSize = tscGetResRowLength(pQueryInfo->exprList); int32_t rowSize = tscGetResRowLength(pQueryInfo->exprList);
assert(numOfRes * rowSize > 0); assert(numOfRes * rowSize > 0);
......
src/client/src/tscUtil.c
浏览文件 @ e15abb2e
...@@ -1347,14 +1347,7 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) { ...@@ -1347,14 +1347,7 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) {
tfree(pRes->buffer); tfree(pRes->buffer);
tfree(pRes->urow); tfree(pRes->urow);
tfree(pRes->pGroupRec);
tfree(pRes->pColumnIndex); tfree(pRes->pColumnIndex);
if (pRes->pArithSup != NULL) {
tfree(pRes->pArithSup->data);
tfree(pRes->pArithSup);
}
tfree(pRes->final); tfree(pRes->final);
pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free
...@@ -2087,32 +2080,35 @@ bool tscIsInsertData(char* sqlstr) { ...@@ -2087,32 +2080,35 @@ bool tscIsInsertData(char* sqlstr) {
} while (1); } while (1);
} }
int tscAllocPayload(SSqlCmd* pCmd, int size) { int32_t tscAllocPayloadFast(SSqlCmd *pCmd, size_t size) {
if (pCmd->payload == NULL) { if (pCmd->payload == NULL) {
assert(pCmd->allocSize == 0); assert(pCmd->allocSize == 0);
pCmd->payload = (char*)calloc(1, size); pCmd->payload = malloc(size);
if (pCmd->payload == NULL) { pCmd->allocSize = (uint32_t) size;
} else if (pCmd->allocSize < size) {
char* tmp = realloc(pCmd->payload, size);
if (tmp == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY; return TSDB_CODE_TSC_OUT_OF_MEMORY;
} }
pCmd->allocSize = size; pCmd->payload = tmp;
} else { pCmd->allocSize = (uint32_t) size;
if (pCmd->allocSize < (uint32_t)size) { }
char* b = realloc(pCmd->payload, size);
if (b == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
pCmd->payload = b; assert(pCmd->allocSize >= size);
pCmd->allocSize = size; return TSDB_CODE_SUCCESS;
} }
int32_t tscAllocPayload(SSqlCmd* pCmd, int size) {
assert(size > 0);
int32_t code = tscAllocPayloadFast(pCmd, (size_t) size);
if (code == TSDB_CODE_SUCCESS) {
memset(pCmd->payload, 0, pCmd->allocSize); memset(pCmd->payload, 0, pCmd->allocSize);
} }
assert(pCmd->allocSize >= (uint32_t)size && size > 0); return code;
return TSDB_CODE_SUCCESS;
} }
TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes) { TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes) {
...@@ -3369,11 +3365,11 @@ void tscFreeVgroupTableInfo(SArray* pVgroupTables) { ...@@ -3369,11 +3365,11 @@ void tscFreeVgroupTableInfo(SArray* pVgroupTables) {
size_t num = taosArrayGetSize(pVgroupTables); size_t num = taosArrayGetSize(pVgroupTables);
for (size_t i = 0; i < num; i++) { for (size_t i = 0; i < num; i++) {
SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTables, i); SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTables, i);
#if 0
for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) { for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
tfree(pInfo->vgInfo.epAddr[j].fqdn); tfree(pInfo->vgInfo.epAddr[j].fqdn);
} }
#endif
taosArrayDestroy(pInfo->itemList); taosArrayDestroy(pInfo->itemList);
} }
...@@ -3387,9 +3383,9 @@ void tscRemoveVgroupTableGroup(SArray* pVgroupTable, int32_t index) { ...@@ -3387,9 +3383,9 @@ void tscRemoveVgroupTableGroup(SArray* pVgroupTable, int32_t index) {
assert(size > index); assert(size > index);
SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTable, index); SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTable, index);
for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) { // for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
tfree(pInfo->vgInfo.epAddr[j].fqdn); // tfree(pInfo->vgInfo.epAddr[j].fqdn);
} // }
taosArrayDestroy(pInfo->itemList); taosArrayDestroy(pInfo->itemList);
taosArrayRemove(pVgroupTable, index); taosArrayRemove(pVgroupTable, index);
...@@ -3399,9 +3395,12 @@ void tscVgroupTableCopy(SVgroupTableInfo* info, SVgroupTableInfo* pInfo) { ...@@ -3399,9 +3395,12 @@ void tscVgroupTableCopy(SVgroupTableInfo* info, SVgroupTableInfo* pInfo) {
memset(info, 0, sizeof(SVgroupTableInfo)); memset(info, 0, sizeof(SVgroupTableInfo));
info->vgInfo = pInfo->vgInfo; info->vgInfo = pInfo->vgInfo;
#if 0
for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) { for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
info->vgInfo.epAddr[j].fqdn = strdup(pInfo->vgInfo.epAddr[j].fqdn); info->vgInfo.epAddr[j].fqdn = strdup(pInfo->vgInfo.epAddr[j].fqdn);
} }
#endif
if (pInfo->itemList) { if (pInfo->itemList) {
info->itemList = taosArrayDup(pInfo->itemList); info->itemList = taosArrayDup(pInfo->itemList);
...@@ -3464,13 +3463,9 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM ...@@ -3464,13 +3463,9 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM
} }
pTableMetaInfo->pTableMeta = pTableMeta; pTableMetaInfo->pTableMeta = pTableMeta;
if (pTableMetaInfo->pTableMeta == NULL) { pTableMetaInfo->tableMetaSize = (pTableMetaInfo->pTableMeta == NULL)? 0:tscGetTableMetaSize(pTableMeta);
pTableMetaInfo->tableMetaSize = 0;
} else {
pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pTableMeta);
}
pTableMetaInfo->tableMetaCapacity = (size_t)(pTableMetaInfo->tableMetaSize); pTableMetaInfo->tableMetaCapacity = (size_t)(pTableMetaInfo->tableMetaSize);
if (vgroupList != NULL) { if (vgroupList != NULL) {
pTableMetaInfo->vgroupList = tscVgroupInfoClone(vgroupList); pTableMetaInfo->vgroupList = tscVgroupInfoClone(vgroupList);
...@@ -3718,8 +3713,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3718,8 +3713,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto _error; goto _error;
} }
pNewQueryInfo->numOfFillVal = pQueryInfo->fieldsInfo.numOfOutput;
pNewQueryInfo->numOfFillVal = pQueryInfo->fieldsInfo.numOfOutput;
memcpy(pNewQueryInfo->fillVal, pQueryInfo->fillVal, pQueryInfo->fieldsInfo.numOfOutput * sizeof(int64_t)); memcpy(pNewQueryInfo->fillVal, pQueryInfo->fillVal, pQueryInfo->fieldsInfo.numOfOutput * sizeof(int64_t));
} }
...@@ -3760,7 +3755,6 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3760,7 +3755,6 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pTableMeta, pTableMetaInfo->vgroupList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pTableMeta, pTableMetaInfo->vgroupList,
pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables); pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables);
} else { // transfer the ownership of pTableMeta to the newly create sql object. } else { // transfer the ownership of pTableMeta to the newly create sql object.
STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, 0); STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, 0);
if (pPrevInfo->pTableMeta && pPrevInfo->pTableMeta->tableType < 0) { if (pPrevInfo->pTableMeta && pPrevInfo->pTableMeta->tableType < 0) {
...@@ -3770,8 +3764,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3770,8 +3764,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
STableMeta* pPrevTableMeta = tscTableMetaDup(pPrevInfo->pTableMeta); STableMeta* pPrevTableMeta = tscTableMetaDup(pPrevInfo->pTableMeta);
SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList; SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList;
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pPrevTableMeta, pVgroupsInfo, pTableMetaInfo->tagColList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pPrevTableMeta, pVgroupsInfo,
pTableMetaInfo->pVgroupTables); pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables);
} }
// this case cannot be happened // this case cannot be happened
...@@ -4415,8 +4409,8 @@ SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *vgroupList) { ...@@ -4415,8 +4409,8 @@ SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *vgroupList) {
return NULL; return NULL;
} }
size_t size = sizeof(SVgroupsInfo) + sizeof(SVgroupInfo) * vgroupList->numOfVgroups; size_t size = sizeof(SVgroupsInfo) + sizeof(SVgroupMsg) * vgroupList->numOfVgroups;
SVgroupsInfo* pNew = calloc(1, size); SVgroupsInfo* pNew = malloc(size);
if (pNew == NULL) { if (pNew == NULL) {
return NULL; return NULL;
} }
...@@ -4424,15 +4418,15 @@ SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *vgroupList) { ...@@ -4424,15 +4418,15 @@ SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *vgroupList) {
pNew->numOfVgroups = vgroupList->numOfVgroups; pNew->numOfVgroups = vgroupList->numOfVgroups;
for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) { for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) {
SVgroupInfo* pNewVInfo = &pNew->vgroups[i]; SVgroupMsg* pNewVInfo = &pNew->vgroups[i];
SVgroupInfo* pvInfo = &vgroupList->vgroups[i]; SVgroupMsg* pvInfo = &vgroupList->vgroups[i];
pNewVInfo->vgId = pvInfo->vgId; pNewVInfo->vgId = pvInfo->vgId;
pNewVInfo->numOfEps = pvInfo->numOfEps; pNewVInfo->numOfEps = pvInfo->numOfEps;
for(int32_t j = 0; j < pvInfo->numOfEps; ++j) { for(int32_t j = 0; j < pvInfo->numOfEps; ++j) {
pNewVInfo->epAddr[j].fqdn = strdup(pvInfo->epAddr[j].fqdn);
pNewVInfo->epAddr[j].port = pvInfo->epAddr[j].port; pNewVInfo->epAddr[j].port = pvInfo->epAddr[j].port;
tstrncpy(pNewVInfo->epAddr[j].fqdn, pvInfo->epAddr[j].fqdn, TSDB_FQDN_LEN);
} }
} }
...@@ -4444,8 +4438,9 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) { ...@@ -4444,8 +4438,9 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) {
return NULL; return NULL;
} }
#if 0
for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) { for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) {
SVgroupInfo* pVgroupInfo = &vgroupList->vgroups[i]; SVgroupMsg* pVgroupInfo = &vgroupList->vgroups[i];
for(int32_t j = 0; j < pVgroupInfo->numOfEps; ++j) { for(int32_t j = 0; j < pVgroupInfo->numOfEps; ++j) {
tfree(pVgroupInfo->epAddr[j].fqdn); tfree(pVgroupInfo->epAddr[j].fqdn);
...@@ -4456,10 +4451,11 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) { ...@@ -4456,10 +4451,11 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) {
} }
} }
#endif
tfree(vgroupList); tfree(vgroupList);
return NULL; return NULL;
} }
# if 0
void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src) { void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src) {
dst->vgId = src->vgId; dst->vgId = src->vgId;
dst->numOfEps = src->numOfEps; dst->numOfEps = src->numOfEps;
...@@ -4472,6 +4468,8 @@ void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src) { ...@@ -4472,6 +4468,8 @@ void tscSVgroupInfoCopy(SVgroupInfo* dst, const SVgroupInfo* src) {
} }
} }
#endif
char* serializeTagData(STagData* pTagData, char* pMsg) { char* serializeTagData(STagData* pTagData, char* pMsg) {
int32_t n = (int32_t) strlen(pTagData->name); int32_t n = (int32_t) strlen(pTagData->name);
*(int32_t*) pMsg = htonl(n); *(int32_t*) pMsg = htonl(n);
...@@ -4612,11 +4610,12 @@ STableMeta* tscTableMetaDup(STableMeta* pTableMeta) { ...@@ -4612,11 +4610,12 @@ STableMeta* tscTableMetaDup(STableMeta* pTableMeta) {
SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo) { SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo) {
assert(pVgroupsInfo != NULL); assert(pVgroupsInfo != NULL);
size_t size = sizeof(SVgroupInfo) * pVgroupsInfo->numOfVgroups + sizeof(SVgroupsInfo); size_t size = sizeof(SVgroupMsg) * pVgroupsInfo->numOfVgroups + sizeof(SVgroupsInfo);
SVgroupsInfo* pInfo = calloc(1, size); SVgroupsInfo* pInfo = calloc(1, size);
pInfo->numOfVgroups = pVgroupsInfo->numOfVgroups; pInfo->numOfVgroups = pVgroupsInfo->numOfVgroups;
for (int32_t m = 0; m < pVgroupsInfo->numOfVgroups; ++m) { for (int32_t m = 0; m < pVgroupsInfo->numOfVgroups; ++m) {
tscSVgroupInfoCopy(&pInfo->vgroups[m], &pVgroupsInfo->vgroups[m]); memcpy(&pInfo->vgroups[m], &pVgroupsInfo->vgroups[m], sizeof(SVgroupMsg));
// tscSVgroupInfoCopy(&pInfo->vgroups[m], &pVgroupsInfo->vgroups[m]);
} }
return pInfo; return pInfo;
} }
......
src/common/inc/tglobal.h
浏览文件 @ e15abb2e
...@@ -224,6 +224,8 @@ extern uint32_t maxRange; ...@@ -224,6 +224,8 @@ extern uint32_t maxRange;
extern uint32_t curRange; extern uint32_t curRange;
extern char Compressor[]; extern char Compressor[];
#endif #endif
// long query
extern int8_t tsDeadLockKillQuery;
typedef struct { typedef struct {
char dir[TSDB_FILENAME_LEN]; char dir[TSDB_FILENAME_LEN];
......
src/common/src/tglobal.c
浏览文件 @ e15abb2e
...@@ -279,6 +279,9 @@ uint32_t curRange = 100; // range ...@@ -279,6 +279,9 @@ uint32_t curRange = 100; // range
char Compressor[32] = "ZSTD_COMPRESSOR"; // ZSTD_COMPRESSOR or GZIP_COMPRESSOR char Compressor[32] = "ZSTD_COMPRESSOR"; // ZSTD_COMPRESSOR or GZIP_COMPRESSOR
#endif #endif
// long query death-lock
int8_t tsDeadLockKillQuery = 0;
int32_t (*monStartSystemFp)() = NULL; int32_t (*monStartSystemFp)() = NULL;
void (*monStopSystemFp)() = NULL; void (*monStopSystemFp)() = NULL;
void (*monExecuteSQLFp)(char *sql) = NULL; void (*monExecuteSQLFp)(char *sql) = NULL;
...@@ -1623,7 +1626,17 @@ static void doInitGlobalConfig(void) { ...@@ -1623,7 +1626,17 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
assert(tsGlobalConfigNum <= TSDB_CFG_MAX_NUM); // enable kill long query
cfg.option = "deadLockKillQuery";
cfg.ptr = &tsDeadLockKillQuery;
cfg.valType = TAOS_CFG_VTYPE_INT8;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW;
cfg.minValue = 0;
cfg.maxValue = 1;
cfg.ptrLength = 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
#ifdef TD_TSZ #ifdef TD_TSZ
// lossy compress // lossy compress
cfg.option = "lossyColumns"; cfg.option = "lossyColumns";
...@@ -1677,6 +1690,9 @@ static void doInitGlobalConfig(void) { ...@@ -1677,6 +1690,9 @@ static void doInitGlobalConfig(void) {
cfg.ptrLength = 0; cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
assert(tsGlobalConfigNum == TSDB_CFG_MAX_NUM);
#else
assert(tsGlobalConfigNum == TSDB_CFG_MAX_NUM - 5);
#endif #endif
} }
......
src/inc/query.h
浏览文件 @ e15abb2e
...@@ -76,6 +76,11 @@ void* qGetResultRetrieveMsg(qinfo_t qinfo); ...@@ -76,6 +76,11 @@ void* qGetResultRetrieveMsg(qinfo_t qinfo);
*/ */
int32_t qKillQuery(qinfo_t qinfo); int32_t qKillQuery(qinfo_t qinfo);
//kill by qid
int32_t qKillQueryByQId(void* pMgmt, int64_t qId, int32_t waitMs, int32_t waitCount);
bool qSolveCommitNoBlock(void* pRepo, void* pMgmt);
int32_t qQueryCompleted(qinfo_t qinfo); int32_t qQueryCompleted(qinfo_t qinfo);
/** /**
......
src/inc/taoserror.h
浏览文件 @ e15abb2e
...@@ -35,6 +35,7 @@ int32_t* taosGetErrno(); ...@@ -35,6 +35,7 @@ int32_t* taosGetErrno();
#define terrno (*taosGetErrno()) #define terrno (*taosGetErrno())
#define TSDB_CODE_SUCCESS 0 #define TSDB_CODE_SUCCESS 0
#define TSDB_CODE_FAILED -1 // unknown or needn't tell detail error
// rpc // rpc
#define TSDB_CODE_RPC_ACTION_IN_PROGRESS TAOS_DEF_ERROR_CODE(0, 0x0001) //"Action in progress") #define TSDB_CODE_RPC_ACTION_IN_PROGRESS TAOS_DEF_ERROR_CODE(0, 0x0001) //"Action in progress")
......
src/inc/taosmsg.h
浏览文件 @ e15abb2e
...@@ -766,27 +766,16 @@ typedef struct SSTableVgroupMsg { ...@@ -766,27 +766,16 @@ typedef struct SSTableVgroupMsg {
int32_t numOfTables; int32_t numOfTables;
} SSTableVgroupMsg, SSTableVgroupRspMsg; } SSTableVgroupMsg, SSTableVgroupRspMsg;
typedef struct {
int32_t vgId;
int8_t numOfEps;
SEpAddr1 epAddr[TSDB_MAX_REPLICA];
} SVgroupInfo;
typedef struct { typedef struct {
int32_t vgId; int32_t vgId;
int8_t numOfEps; int8_t numOfEps;
SEpAddrMsg epAddr[TSDB_MAX_REPLICA]; SEpAddrMsg epAddr[TSDB_MAX_REPLICA];
} SVgroupMsg; } SVgroupMsg;
typedef struct {
int32_t numOfVgroups;
SVgroupInfo vgroups[];
} SVgroupsInfo;
typedef struct { typedef struct {
int32_t numOfVgroups; int32_t numOfVgroups;
SVgroupMsg vgroups[]; SVgroupMsg vgroups[];
} SVgroupsMsg; } SVgroupsMsg, SVgroupsInfo;
typedef struct STableMetaMsg { typedef struct STableMetaMsg {
int32_t contLen; int32_t contLen;
......
src/inc/tsdb.h
浏览文件 @ e15abb2e
...@@ -39,6 +39,7 @@ extern "C" { ...@@ -39,6 +39,7 @@ extern "C" {
#define TSDB_STATUS_COMMIT_START 1 #define TSDB_STATUS_COMMIT_START 1
#define TSDB_STATUS_COMMIT_OVER 2 #define TSDB_STATUS_COMMIT_OVER 2
#define TSDB_STATUS_COMMIT_NOBLOCK 3 //commit no block, need to be solved
// TSDB STATE DEFINITION // TSDB STATE DEFINITION
#define TSDB_STATE_OK 0x0 #define TSDB_STATE_OK 0x0
...@@ -413,6 +414,11 @@ int tsdbSyncRecv(void *pRepo, SOCKET socketFd); ...@@ -413,6 +414,11 @@ int tsdbSyncRecv(void *pRepo, SOCKET socketFd);
// For TSDB Compact // For TSDB Compact
int tsdbCompact(STsdbRepo *pRepo); int tsdbCompact(STsdbRepo *pRepo);
// For TSDB Health Monitor
// no problem return true
bool tsdbNoProblem(STsdbRepo* pRepo);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
......
src/kit/taosdemo/taosdemo.c
浏览文件 @ e15abb2e
此差异已折叠。
src/kit/taosdump/taosdump.c
浏览文件 @ e15abb2e
...@@ -1209,14 +1209,14 @@ _dump_db_point: ...@@ -1209,14 +1209,14 @@ _dump_db_point:
fprintf(fp, "USE %s;\n\n", g_dbInfos[0]->name); fprintf(fp, "USE %s;\n\n", g_dbInfos[0]->name);
int32_t totalNumOfThread = 1; // 0: all normal talbe into .tables.tmp.0 int32_t totalNumOfThread = 1; // 0: all normal table into .tables.tmp.0
int normalTblFd = -1; int normalTblFd = -1;
int32_t retCode; int32_t retCode;
int superTblCnt = 0 ; int superTblCnt = 0 ;
for (int i = 1; g_args.arg_list[i]; i++) { for (int i = 1; g_args.arg_list[i]; i++) {
if (taosGetTableRecordInfo(g_args.arg_list[i], if (taosGetTableRecordInfo(g_args.arg_list[i],
&tableRecordInfo, taos) < 0) { &tableRecordInfo, taos) < 0) {
errorPrint("input the invalide table %s\n", errorPrint("input the invalid table %s\n",
g_args.arg_list[i]); g_args.arg_list[i]);
continue; continue;
} }
...@@ -1341,11 +1341,10 @@ static int taosGetTableDes( ...@@ -1341,11 +1341,10 @@ static int taosGetTableDes(
return count; return count;
} }
// if chidl-table have tag, using select tagName from table to get tagValue // if child-table have tag, using select tagName from table to get tagValue
for (int i = 0 ; i < count; i++) { for (int i = 0 ; i < count; i++) {
if (strcmp(stableDes->cols[i].note, "TAG") != 0) continue; if (strcmp(stableDes->cols[i].note, "TAG") != 0) continue;
sprintf(sqlstr, "select %s from %s.%s", sprintf(sqlstr, "select %s from %s.%s",
stableDes->cols[i].field, dbName, table); stableDes->cols[i].field, dbName, table);
...@@ -2443,7 +2442,7 @@ static int taosGetFilesNum(const char *directoryName, ...@@ -2443,7 +2442,7 @@ static int taosGetFilesNum(const char *directoryName,
} }
if (fileNum <= 0) { if (fileNum <= 0) {
errorPrint("directory:%s is empry\n", directoryName); errorPrint("directory:%s is empty\n", directoryName);
exit(-1); exit(-1);
} }
...@@ -2620,9 +2619,9 @@ static int taosDumpInOneFile(TAOS* taos, FILE* fp, char* fcharset, ...@@ -2620,9 +2619,9 @@ static int taosDumpInOneFile(TAOS* taos, FILE* fp, char* fcharset,
memcpy(cmd + cmd_len, line, read_len); memcpy(cmd + cmd_len, line, read_len);
cmd[read_len + cmd_len]= '\0'; cmd[read_len + cmd_len]= '\0';
if (queryDbImpl(taos, cmd)) { if (queryDbImpl(taos, cmd)) {
errorPrint("%s() LN%d, error sql: linenu:%d, file:%s\n", errorPrint("%s() LN%d, error sql: lineno:%d, file:%s\n",
__func__, __LINE__, lineNo, fileName); __func__, __LINE__, lineNo, fileName);
fprintf(g_fpOfResult, "error sql: linenu:%d, file:%s\n", lineNo, fileName); fprintf(g_fpOfResult, "error sql: lineno:%d, file:%s\n", lineNo, fileName);
} }
memset(cmd, 0, TSDB_MAX_ALLOWED_SQL_LEN); memset(cmd, 0, TSDB_MAX_ALLOWED_SQL_LEN);
......
src/mnode/src/mnodeTable.c
浏览文件 @ e15abb2e
...@@ -1231,7 +1231,9 @@ static int32_t mnodeAddSuperTableTagCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1231,7 +1231,9 @@ static int32_t mnodeAddSuperTableTagCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, add tag result:%s, numOfTags:%d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, add tag result:%s, numOfTags:%d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code), pStable->numOfTags); tstrerror(code), pStable->numOfTags);
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
...@@ -1287,6 +1289,9 @@ static int32_t mnodeDropSuperTableTagCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1287,6 +1289,9 @@ static int32_t mnodeDropSuperTableTagCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, drop tag result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, drop tag result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code)); tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
...@@ -1321,6 +1326,9 @@ static int32_t mnodeModifySuperTableTagNameCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1321,6 +1326,9 @@ static int32_t mnodeModifySuperTableTagNameCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, modify tag result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, modify tag result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code)); tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
...@@ -1376,6 +1384,9 @@ static int32_t mnodeAddSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1376,6 +1384,9 @@ static int32_t mnodeAddSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, add column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, add column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code)); tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
...@@ -1444,6 +1455,9 @@ static int32_t mnodeDropSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1444,6 +1455,9 @@ static int32_t mnodeDropSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, delete column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, delete column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code)); tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
...@@ -1489,6 +1503,9 @@ static int32_t mnodeChangeSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1489,6 +1503,9 @@ static int32_t mnodeChangeSuperTableColumnCb(SMnodeMsg *pMsg, int32_t code) {
SSTableObj *pStable = (SSTableObj *)pMsg->pTable; SSTableObj *pStable = (SSTableObj *)pMsg->pTable;
mLInfo("msg:%p, app:%p stable %s, change column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mLInfo("msg:%p, app:%p stable %s, change column result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
tstrerror(code)); tstrerror(code));
if (code == TSDB_CODE_SUCCESS) {
code = mnodeGetSuperTableMeta(pMsg);
}
return code; return code;
} }
......
src/plugins/http/inc/httpUtil.h
浏览文件 @ e15abb2e
...@@ -17,6 +17,7 @@ ...@@ -17,6 +17,7 @@
#define TDENGINE_HTTP_UTIL_H #define TDENGINE_HTTP_UTIL_H
bool httpCheckUsedbSql(char *sql); bool httpCheckUsedbSql(char *sql);
bool httpCheckAlterSql(char *sql);
void httpTimeToString(int32_t t, char *buf, int32_t buflen); void httpTimeToString(int32_t t, char *buf, int32_t buflen);
bool httpUrlMatch(HttpContext *pContext, int32_t pos, char *cmp); bool httpUrlMatch(HttpContext *pContext, int32_t pos, char *cmp);
......
src/plugins/http/src/httpHandle.c
浏览文件 @ e15abb2e
...@@ -35,6 +35,7 @@ bool httpProcessData(HttpContext* pContext) { ...@@ -35,6 +35,7 @@ bool httpProcessData(HttpContext* pContext) {
if (!httpAlterContextState(pContext, HTTP_CONTEXT_STATE_READY, HTTP_CONTEXT_STATE_HANDLING)) { if (!httpAlterContextState(pContext, HTTP_CONTEXT_STATE_READY, HTTP_CONTEXT_STATE_HANDLING)) {
httpTrace("context:%p, fd:%d, state:%s not in ready state, stop process request", pContext, pContext->fd, httpTrace("context:%p, fd:%d, state:%s not in ready state, stop process request", pContext, pContext->fd,
httpContextStateStr(pContext->state)); httpContextStateStr(pContext->state));
pContext->error = true;
httpCloseContextByApp(pContext); httpCloseContextByApp(pContext);
return false; return false;
} }
......
src/plugins/http/src/httpParser.c
浏览文件 @ e15abb2e
...@@ -1157,10 +1157,6 @@ static int32_t httpParseChar(HttpParser *parser, const char c, int32_t *again) { ...@@ -1157,10 +1157,6 @@ static int32_t httpParseChar(HttpParser *parser, const char c, int32_t *again) {
httpOnError(parser, HTTP_CODE_INTERNAL_SERVER_ERROR, TSDB_CODE_HTTP_PARSE_ERROR_STATE); httpOnError(parser, HTTP_CODE_INTERNAL_SERVER_ERROR, TSDB_CODE_HTTP_PARSE_ERROR_STATE);
} }
if (ok != 0) {
pContext->error = true;
}
return ok; return ok;
} }
......
src/plugins/http/src/httpResp.c
浏览文件 @ e15abb2e
...@@ -147,6 +147,8 @@ void httpSendErrorResp(HttpContext *pContext, int32_t errNo) { ...@@ -147,6 +147,8 @@ void httpSendErrorResp(HttpContext *pContext, int32_t errNo) {
httpCode = pContext->parser->httpCode; httpCode = pContext->parser->httpCode;
} }
pContext->error = true;
char *httpCodeStr = httpGetStatusDesc(httpCode); char *httpCodeStr = httpGetStatusDesc(httpCode);
httpSendErrorRespImp(pContext, httpCode, httpCodeStr, errNo & 0XFFFF, tstrerror(errNo)); httpSendErrorRespImp(pContext, httpCode, httpCodeStr, errNo & 0XFFFF, tstrerror(errNo));
} }
......
src/plugins/http/src/httpRestJson.c
浏览文件 @ e15abb2e
...@@ -16,6 +16,7 @@ ...@@ -16,6 +16,7 @@
#define _DEFAULT_SOURCE #define _DEFAULT_SOURCE
#include "os.h" #include "os.h"
#include "tglobal.h" #include "tglobal.h"
#include "tsclient.h"
#include "httpLog.h" #include "httpLog.h"
#include "httpJson.h" #include "httpJson.h"
#include "httpRestHandle.h" #include "httpRestHandle.h"
...@@ -62,13 +63,21 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result) ...@@ -62,13 +63,21 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result)
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonToken(jsonBuf, JsonArrStt); httpJsonToken(jsonBuf, JsonArrStt);
SSqlObj *pObj = (SSqlObj *) result;
bool isAlterSql = (pObj->sqlstr == NULL) ? false : httpCheckAlterSql(pObj->sqlstr);
if (num_fields == 0) { if (num_fields == 0) {
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, REST_JSON_AFFECT_ROWS, REST_JSON_AFFECT_ROWS_LEN); httpJsonString(jsonBuf, REST_JSON_AFFECT_ROWS, REST_JSON_AFFECT_ROWS_LEN);
} else { } else {
for (int32_t i = 0; i < num_fields; ++i) { if (isAlterSql == true) {
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, fields[i].name, (int32_t)strlen(fields[i].name)); httpJsonString(jsonBuf, REST_JSON_AFFECT_ROWS, REST_JSON_AFFECT_ROWS_LEN);
} else {
for (int32_t i = 0; i < num_fields; ++i) {
httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, fields[i].name, (int32_t)strlen(fields[i].name));
}
} }
} }
...@@ -99,8 +108,14 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result) ...@@ -99,8 +108,14 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result)
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonToken(jsonBuf, JsonArrStt); httpJsonToken(jsonBuf, JsonArrStt);
httpJsonItemToken(jsonBuf); if (isAlterSql == true) {
httpJsonString(jsonBuf, fields[i].name, (int32_t)strlen(fields[i].name)); httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, REST_JSON_AFFECT_ROWS, REST_JSON_AFFECT_ROWS_LEN);
} else {
httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, fields[i].name, (int32_t)strlen(fields[i].name));
}
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonInt(jsonBuf, fields[i].type); httpJsonInt(jsonBuf, fields[i].type);
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
......
src/plugins/http/src/httpServer.c
浏览文件 @ e15abb2e
...@@ -191,8 +191,6 @@ static void httpProcessHttpData(void *param) { ...@@ -191,8 +191,6 @@ static void httpProcessHttpData(void *param) {
if (httpReadData(pContext)) { if (httpReadData(pContext)) {
(*(pThread->processData))(pContext); (*(pThread->processData))(pContext);
atomic_fetch_add_32(&pServer->requestNum, 1); atomic_fetch_add_32(&pServer->requestNum, 1);
} else {
httpReleaseContext(pContext/*, false*/);
} }
} }
} }
...@@ -402,13 +400,17 @@ static bool httpReadData(HttpContext *pContext) { ...@@ -402,13 +400,17 @@ static bool httpReadData(HttpContext *pContext) {
} else if (nread < 0) { } else if (nread < 0) {
if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK) { if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK) {
httpDebug("context:%p, fd:%d, read from socket error:%d, wait another event", pContext, pContext->fd, errno); httpDebug("context:%p, fd:%d, read from socket error:%d, wait another event", pContext, pContext->fd, errno);
return false; // later again continue; // later again
} else { } else {
httpError("context:%p, fd:%d, read from socket error:%d, close connect", pContext, pContext->fd, errno); httpError("context:%p, fd:%d, read from socket error:%d, close connect", pContext, pContext->fd, errno);
taosCloseSocket(pContext->fd);
httpReleaseContext(pContext/*, false */);
return false; return false;
} }
} else { } else {
httpError("context:%p, fd:%d, nread:%d, wait another event", pContext, pContext->fd, nread); httpError("context:%p, fd:%d, nread:%d, wait another event", pContext, pContext->fd, nread);
taosCloseSocket(pContext->fd);
httpReleaseContext(pContext/*, false */);
return false; return false;
} }
} }
......
src/plugins/http/src/httpSql.c
浏览文件 @ e15abb2e
...@@ -405,7 +405,6 @@ void httpProcessRequestCb(void *param, TAOS_RES *result, int32_t code) { ...@@ -405,7 +405,6 @@ void httpProcessRequestCb(void *param, TAOS_RES *result, int32_t code) {
if (pContext->session == NULL) { if (pContext->session == NULL) {
httpSendErrorResp(pContext, TSDB_CODE_HTTP_SESSION_FULL); httpSendErrorResp(pContext, TSDB_CODE_HTTP_SESSION_FULL);
httpCloseContextByApp(pContext);
} else { } else {
httpExecCmd(pContext); httpExecCmd(pContext);
} }
......
src/plugins/http/src/httpUtil.c
浏览文件 @ e15abb2e
...@@ -21,6 +21,7 @@ ...@@ -21,6 +21,7 @@
#include "httpResp.h" #include "httpResp.h"
#include "httpSql.h" #include "httpSql.h"
#include "httpUtil.h" #include "httpUtil.h"
#include "ttoken.h"
bool httpCheckUsedbSql(char *sql) { bool httpCheckUsedbSql(char *sql) {
if (strstr(sql, "use ") != NULL) { if (strstr(sql, "use ") != NULL) {
...@@ -29,6 +30,17 @@ bool httpCheckUsedbSql(char *sql) { ...@@ -29,6 +30,17 @@ bool httpCheckUsedbSql(char *sql) {
return false; return false;
} }
bool httpCheckAlterSql(char *sql) {
int32_t index = 0;
do {
SStrToken t0 = tStrGetToken(sql, &index, false);
if (t0.type != TK_LP) {
return t0.type == TK_ALTER;
}
} while (1);
}
void httpTimeToString(int32_t t, char *buf, int32_t buflen) { void httpTimeToString(int32_t t, char *buf, int32_t buflen) {
memset(buf, 0, (size_t)buflen); memset(buf, 0, (size_t)buflen);
char ts[32] = {0}; char ts[32] = {0};
......
src/query/inc/qExecutor.h
浏览文件 @ e15abb2e
...@@ -86,11 +86,18 @@ typedef struct SResultRow { ...@@ -86,11 +86,18 @@ typedef struct SResultRow {
char *key; // start key of current result row char *key; // start key of current result row
} SResultRow; } SResultRow;
typedef struct SResultRowCell {
uint64_t groupId;
SResultRow *pRow;
} SResultRowCell;
typedef struct SGroupResInfo { typedef struct SGroupResInfo {
int32_t totalGroup; int32_t totalGroup;
int32_t currentGroup; int32_t currentGroup;
int32_t index; int32_t index;
SArray* pRows; // SArray<SResultRow*> SArray* pRows; // SArray<SResultRow*>
bool ordered;
int32_t position;
} SGroupResInfo; } SGroupResInfo;
/** /**
...@@ -284,8 +291,9 @@ typedef struct SQueryRuntimeEnv { ...@@ -284,8 +291,9 @@ typedef struct SQueryRuntimeEnv {
SDiskbasedResultBuf* pResultBuf; // query result buffer based on blocked-wised disk file SDiskbasedResultBuf* pResultBuf; // query result buffer based on blocked-wised disk file
SHashObj* pResultRowHashTable; // quick locate the window object for each result SHashObj* pResultRowHashTable; // quick locate the window object for each result
SHashObj* pResultRowListSet; // used to check if current ResultRowInfo has ResultRow object or not SHashObj* pResultRowListSet; // used to check if current ResultRowInfo has ResultRow object or not
SArray* pResultRowArrayList; // The array list that contains the Result rows
char* keyBuf; // window key buffer char* keyBuf; // window key buffer
SResultRowPool* pool; // window result object pool SResultRowPool* pool; // The window result objects pool, all the resultRow Objects are allocated and managed by this object.
char** prevRow; char** prevRow;
SArray* prevResult; // intermediate result, SArray<SInterResult> SArray* prevResult; // intermediate result, SArray<SInterResult>
......
src/query/src/qExecutor.c
浏览文件 @ e15abb2e
...@@ -544,6 +544,8 @@ static SResultRow* doSetResultOutBufByKey(SQueryRuntimeEnv* pRuntimeEnv, SResult ...@@ -544,6 +544,8 @@ static SResultRow* doSetResultOutBufByKey(SQueryRuntimeEnv* pRuntimeEnv, SResult
// add a new result set for a new group // add a new result set for a new group
taosHashPut(pRuntimeEnv->pResultRowHashTable, pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes), &pResult, POINTER_BYTES); taosHashPut(pRuntimeEnv->pResultRowHashTable, pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes), &pResult, POINTER_BYTES);
SResultRowCell cell = {.groupId = tableGroupId, .pRow = pResult};
taosArrayPush(pRuntimeEnv->pResultRowArrayList, &cell);
} else { } else {
pResult = *p1; pResult = *p1;
} }
...@@ -2107,9 +2109,10 @@ static int32_t setupQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv, int32_t numOf ...@@ -2107,9 +2109,10 @@ static int32_t setupQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv, int32_t numOf
pRuntimeEnv->pQueryAttr = pQueryAttr; pRuntimeEnv->pQueryAttr = pQueryAttr;
pRuntimeEnv->pResultRowHashTable = taosHashInit(numOfTables, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK); pRuntimeEnv->pResultRowHashTable = taosHashInit(numOfTables, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK);
pRuntimeEnv->pResultRowListSet = taosHashInit(numOfTables, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK); pRuntimeEnv->pResultRowListSet = taosHashInit(numOfTables * 10, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK);
pRuntimeEnv->keyBuf = malloc(pQueryAttr->maxTableColumnWidth + sizeof(int64_t) + POINTER_BYTES); pRuntimeEnv->keyBuf = malloc(pQueryAttr->maxTableColumnWidth + sizeof(int64_t) + POINTER_BYTES);
pRuntimeEnv->pool = initResultRowPool(getResultRowSize(pRuntimeEnv)); pRuntimeEnv->pool = initResultRowPool(getResultRowSize(pRuntimeEnv));
pRuntimeEnv->pResultRowArrayList = taosArrayInit(numOfTables, sizeof(SResultRowCell));
pRuntimeEnv->prevRow = malloc(POINTER_BYTES * pQueryAttr->numOfCols + pQueryAttr->srcRowSize); pRuntimeEnv->prevRow = malloc(POINTER_BYTES * pQueryAttr->numOfCols + pQueryAttr->srcRowSize);
pRuntimeEnv->tagVal = malloc(pQueryAttr->tagLen); pRuntimeEnv->tagVal = malloc(pQueryAttr->tagLen);
...@@ -2384,6 +2387,7 @@ static void teardownQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv) { ...@@ -2384,6 +2387,7 @@ static void teardownQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv) {
pRuntimeEnv->pool = destroyResultRowPool(pRuntimeEnv->pool); pRuntimeEnv->pool = destroyResultRowPool(pRuntimeEnv->pool);
taosArrayDestroyEx(pRuntimeEnv->prevResult, freeInterResult); taosArrayDestroyEx(pRuntimeEnv->prevResult, freeInterResult);
taosArrayDestroy(pRuntimeEnv->pResultRowArrayList);
pRuntimeEnv->prevResult = NULL; pRuntimeEnv->prevResult = NULL;
} }
...@@ -2398,11 +2402,11 @@ bool isQueryKilled(SQInfo *pQInfo) { ...@@ -2398,11 +2402,11 @@ bool isQueryKilled(SQInfo *pQInfo) {
// query has been executed more than tsShellActivityTimer, and the retrieve has not arrived // query has been executed more than tsShellActivityTimer, and the retrieve has not arrived
// abort current query execution. // abort current query execution.
if (pQInfo->owner != 0 && ((taosGetTimestampSec() - pQInfo->startExecTs) > getMaximumIdleDurationSec()) && if (pQInfo->owner != 0 && ((taosGetTimestampSec() - pQInfo->startExecTs/1000) > getMaximumIdleDurationSec()) &&
(!needBuildResAfterQueryComplete(pQInfo))) { (!needBuildResAfterQueryComplete(pQInfo))) {
assert(pQInfo->startExecTs != 0); assert(pQInfo->startExecTs != 0);
qDebug("QInfo:%" PRIu64 " retrieve not arrive beyond %d sec, abort current query execution, start:%" PRId64 qDebug("QInfo:%" PRIu64 " retrieve not arrive beyond %d ms, abort current query execution, start:%" PRId64
", current:%d", pQInfo->qId, 1, pQInfo->startExecTs, taosGetTimestampSec()); ", current:%d", pQInfo->qId, 1, pQInfo->startExecTs, taosGetTimestampSec());
return true; return true;
} }
...@@ -4808,7 +4812,6 @@ int32_t doInitQInfo(SQInfo* pQInfo, STSBuf* pTsBuf, void* tsdb, void* sourceOptr ...@@ -4808,7 +4812,6 @@ int32_t doInitQInfo(SQInfo* pQInfo, STSBuf* pTsBuf, void* tsdb, void* sourceOptr
SQueryAttr *pQueryAttr = pQInfo->runtimeEnv.pQueryAttr; SQueryAttr *pQueryAttr = pQInfo->runtimeEnv.pQueryAttr;
pQueryAttr->tsdb = tsdb; pQueryAttr->tsdb = tsdb;
if (tsdb != NULL) { if (tsdb != NULL) {
int32_t code = setupQueryHandle(tsdb, pRuntimeEnv, pQInfo->qId, pQueryAttr->stableQuery); int32_t code = setupQueryHandle(tsdb, pRuntimeEnv, pQInfo->qId, pQueryAttr->stableQuery);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
...@@ -6379,6 +6382,7 @@ static SSDataBlock* hashGroupbyAggregate(void* param, bool* newgroup) { ...@@ -6379,6 +6382,7 @@ static SSDataBlock* hashGroupbyAggregate(void* param, bool* newgroup) {
if (!pRuntimeEnv->pQueryAttr->stableQuery) { if (!pRuntimeEnv->pQueryAttr->stableQuery) {
sortGroupResByOrderList(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pInfo->binfo.pRes); sortGroupResByOrderList(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pInfo->binfo.pRes);
} }
toSSDataBlock(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pInfo->binfo.pRes); toSSDataBlock(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pInfo->binfo.pRes);
if (pInfo->binfo.pRes->info.rows == 0 || !hasRemainDataInCurrentGroup(&pRuntimeEnv->groupResInfo)) { if (pInfo->binfo.pRes->info.rows == 0 || !hasRemainDataInCurrentGroup(&pRuntimeEnv->groupResInfo)) {
...@@ -7600,8 +7604,8 @@ int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param) { ...@@ -7600,8 +7604,8 @@ int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param) {
pMsg += sizeof(SSqlExpr); pMsg += sizeof(SSqlExpr);
for (int32_t j = 0; j < pExprMsg->numOfParams; ++j) { for (int32_t j = 0; j < pExprMsg->numOfParams; ++j) {
pExprMsg->param[j].nType = htons(pExprMsg->param[j].nType); pExprMsg->param[j].nType = htonl(pExprMsg->param[j].nType);
pExprMsg->param[j].nLen = htons(pExprMsg->param[j].nLen); pExprMsg->param[j].nLen = htonl(pExprMsg->param[j].nLen);
if (pExprMsg->param[j].nType == TSDB_DATA_TYPE_BINARY) { if (pExprMsg->param[j].nType == TSDB_DATA_TYPE_BINARY) {
pExprMsg->param[j].pz = pMsg; pExprMsg->param[j].pz = pMsg;
...@@ -7648,8 +7652,8 @@ int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param) { ...@@ -7648,8 +7652,8 @@ int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param) {
pMsg += sizeof(SSqlExpr); pMsg += sizeof(SSqlExpr);
for (int32_t j = 0; j < pExprMsg->numOfParams; ++j) { for (int32_t j = 0; j < pExprMsg->numOfParams; ++j) {
pExprMsg->param[j].nType = htons(pExprMsg->param[j].nType); pExprMsg->param[j].nType = htonl(pExprMsg->param[j].nType);
pExprMsg->param[j].nLen = htons(pExprMsg->param[j].nLen); pExprMsg->param[j].nLen = htonl(pExprMsg->param[j].nLen);
if (pExprMsg->param[j].nType == TSDB_DATA_TYPE_BINARY) { if (pExprMsg->param[j].nType == TSDB_DATA_TYPE_BINARY) {
pExprMsg->param[j].pz = pMsg; pExprMsg->param[j].pz = pMsg;
...@@ -8409,6 +8413,7 @@ SQInfo* createQInfoImpl(SQueryTableMsg* pQueryMsg, SGroupbyExpr* pGroupbyExpr, S ...@@ -8409,6 +8413,7 @@ SQInfo* createQInfoImpl(SQueryTableMsg* pQueryMsg, SGroupbyExpr* pGroupbyExpr, S
} }
pQInfo->qId = qId; pQInfo->qId = qId;
pQInfo->startExecTs = 0;
pQInfo->runtimeEnv.pUdfInfo = pUdfInfo; pQInfo->runtimeEnv.pUdfInfo = pUdfInfo;
...@@ -8647,7 +8652,6 @@ int32_t initQInfo(STsBufInfo* pTsBufInfo, void* tsdb, void* sourceOptr, SQInfo* ...@@ -8647,7 +8652,6 @@ int32_t initQInfo(STsBufInfo* pTsBufInfo, void* tsdb, void* sourceOptr, SQInfo*
SArray* prevResult = NULL; SArray* prevResult = NULL;
if (prevResultLen > 0) { if (prevResultLen > 0) {
prevResult = interResFromBinary(param->prevResult, prevResultLen); prevResult = interResFromBinary(param->prevResult, prevResultLen);
pRuntimeEnv->prevResult = prevResult; pRuntimeEnv->prevResult = prevResult;
} }
......
src/query/src/qUtil.c
浏览文件 @ e15abb2e
...@@ -436,13 +436,13 @@ static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void * ...@@ -436,13 +436,13 @@ static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void *
} }
STableQueryInfo** pList = supporter->pTableQueryInfo; STableQueryInfo** pList = supporter->pTableQueryInfo;
SResultRow* pWindowRes1 = pList[left]->resInfo.pResult[leftPos];
SResultRowInfo *pWindowResInfo1 = &(pList[left]->resInfo); // SResultRow * pWindowRes1 = getResultRow(&(pList[left]->resInfo), leftPos);
SResultRow * pWindowRes1 = getResultRow(pWindowResInfo1, leftPos);
TSKEY leftTimestamp = pWindowRes1->win.skey; TSKEY leftTimestamp = pWindowRes1->win.skey;
SResultRowInfo *pWindowResInfo2 = &(pList[right]->resInfo); // SResultRowInfo *pWindowResInfo2 = &(pList[right]->resInfo);
SResultRow * pWindowRes2 = getResultRow(pWindowResInfo2, rightPos); // SResultRow * pWindowRes2 = getResultRow(pWindowResInfo2, rightPos);
SResultRow* pWindowRes2 = pList[right]->resInfo.pResult[rightPos];
TSKEY rightTimestamp = pWindowRes2->win.skey; TSKEY rightTimestamp = pWindowRes2->win.skey;
if (leftTimestamp == rightTimestamp) { if (leftTimestamp == rightTimestamp) {
...@@ -456,7 +456,77 @@ static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void * ...@@ -456,7 +456,77 @@ static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void *
} }
} }
static int32_t mergeIntoGroupResultImpl(SQueryRuntimeEnv *pRuntimeEnv, SGroupResInfo* pGroupResInfo, SArray *pTableList, int32_t tsAscOrder(const void* p1, const void* p2) {
SResultRowCell* pc1 = (SResultRowCell*) p1;
SResultRowCell* pc2 = (SResultRowCell*) p2;
if (pc1->groupId == pc2->groupId) {
if (pc1->pRow->win.skey == pc2->pRow->win.skey) {
return 0;
} else {
return (pc1->pRow->win.skey < pc2->pRow->win.skey)? -1:1;
}
} else {
return (pc1->groupId < pc2->groupId)? -1:1;
}
}
int32_t tsDescOrder(const void* p1, const void* p2) {
SResultRowCell* pc1 = (SResultRowCell*) p1;
SResultRowCell* pc2 = (SResultRowCell*) p2;
if (pc1->groupId == pc2->groupId) {
if (pc1->pRow->win.skey == pc2->pRow->win.skey) {
return 0;
} else {
return (pc1->pRow->win.skey < pc2->pRow->win.skey)? 1:-1;
}
} else {
return (pc1->groupId < pc2->groupId)? -1:1;
}
}
void orderTheResultRows(SQueryRuntimeEnv* pRuntimeEnv) {
__compar_fn_t fn = NULL;
if (pRuntimeEnv->pQueryAttr->order.order == TSDB_ORDER_ASC) {
fn = tsAscOrder;
} else {
fn = tsDescOrder;
}
taosArraySort(pRuntimeEnv->pResultRowArrayList, fn);
}
static int32_t mergeIntoGroupResultImplRv(SQueryRuntimeEnv *pRuntimeEnv, SGroupResInfo* pGroupResInfo, uint64_t groupId, int32_t* rowCellInfoOffset) {
if (!pGroupResInfo->ordered) {
orderTheResultRows(pRuntimeEnv);
pGroupResInfo->ordered = true;
}
if (pGroupResInfo->pRows == NULL) {
pGroupResInfo->pRows = taosArrayInit(100, POINTER_BYTES);
}
size_t len = taosArrayGetSize(pRuntimeEnv->pResultRowArrayList);
for(; pGroupResInfo->position < len; ++pGroupResInfo->position) {
SResultRowCell* pResultRowCell = taosArrayGet(pRuntimeEnv->pResultRowArrayList, pGroupResInfo->position);
if (pResultRowCell->groupId != groupId) {
break;
}
int64_t num = getNumOfResultWindowRes(pRuntimeEnv, pResultRowCell->pRow, rowCellInfoOffset);
if (num <= 0) {
continue;
}
taosArrayPush(pGroupResInfo->pRows, &pResultRowCell->pRow);
pResultRowCell->pRow->numOfRows = (uint32_t) num;
}
return TSDB_CODE_SUCCESS;
}
static UNUSED_FUNC int32_t mergeIntoGroupResultImpl(SQueryRuntimeEnv *pRuntimeEnv, SGroupResInfo* pGroupResInfo, SArray *pTableList,
int32_t* rowCellInfoOffset) { int32_t* rowCellInfoOffset) {
bool ascQuery = QUERY_IS_ASC_QUERY(pRuntimeEnv->pQueryAttr); bool ascQuery = QUERY_IS_ASC_QUERY(pRuntimeEnv->pQueryAttr);
...@@ -562,12 +632,7 @@ int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQueryRuntimeEnv* pRu ...@@ -562,12 +632,7 @@ int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQueryRuntimeEnv* pRu
int64_t st = taosGetTimestampUs(); int64_t st = taosGetTimestampUs();
while (pGroupResInfo->currentGroup < pGroupResInfo->totalGroup) { while (pGroupResInfo->currentGroup < pGroupResInfo->totalGroup) {
SArray *group = GET_TABLEGROUP(pRuntimeEnv, pGroupResInfo->currentGroup); mergeIntoGroupResultImplRv(pRuntimeEnv, pGroupResInfo, pGroupResInfo->currentGroup, offset);
int32_t ret = mergeIntoGroupResultImpl(pRuntimeEnv, pGroupResInfo, group, offset);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
// this group generates at least one result, return results // this group generates at least one result, return results
if (taosArrayGetSize(pGroupResInfo->pRows) > 0) { if (taosArrayGetSize(pGroupResInfo->pRows) > 0) {
...@@ -583,7 +648,6 @@ int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQueryRuntimeEnv* pRu ...@@ -583,7 +648,6 @@ int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQueryRuntimeEnv* pRu
qDebug("QInfo:%"PRIu64" merge res data into group, index:%d, total group:%d, elapsed time:%" PRId64 "us", GET_QID(pRuntimeEnv), qDebug("QInfo:%"PRIu64" merge res data into group, index:%d, total group:%d, elapsed time:%" PRId64 "us", GET_QID(pRuntimeEnv),
pGroupResInfo->currentGroup, pGroupResInfo->totalGroup, elapsedTime); pGroupResInfo->currentGroup, pGroupResInfo->totalGroup, elapsedTime);
// pQInfo->summary.firstStageMergeTime += elapsedTime;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
......
src/query/src/queryMain.c
浏览文件 @ e15abb2e
...@@ -35,7 +35,7 @@ typedef struct SQueryMgmt { ...@@ -35,7 +35,7 @@ typedef struct SQueryMgmt {
bool closed; bool closed;
} SQueryMgmt; } SQueryMgmt;
static void queryMgmtKillQueryFn(void* handle) { static void queryMgmtKillQueryFn(void* handle, void* param1) {
void** fp = (void**)handle; void** fp = (void**)handle;
qKillQuery(*fp); qKillQuery(*fp);
} }
...@@ -215,6 +215,51 @@ int32_t qCreateQueryInfo(void* tsdb, int32_t vgId, SQueryTableMsg* pQueryMsg, qi ...@@ -215,6 +215,51 @@ int32_t qCreateQueryInfo(void* tsdb, int32_t vgId, SQueryTableMsg* pQueryMsg, qi
return code; return code;
} }
#ifdef TEST_IMPL
// wait moment
int waitMoment(SQInfo* pQInfo){
if(pQInfo->sql) {
int ms = 0;
char* pcnt = strstr(pQInfo->sql, " count(*)");
if(pcnt) return 0;
char* pos = strstr(pQInfo->sql, " t_");
if(pos){
pos += 3;
ms = atoi(pos);
while(*pos >= '0' && *pos <= '9'){
pos ++;
}
char unit_char = *pos;
if(unit_char == 'h'){
ms *= 3600*1000;
} else if(unit_char == 'm'){
ms *= 60*1000;
} else if(unit_char == 's'){
ms *= 1000;
}
}
if(ms == 0) return 0;
printf("test wait sleep %dms. sql=%s ...\n", ms, pQInfo->sql);
if(ms < 1000) {
taosMsleep(ms);
} else {
int used_ms = 0;
while(used_ms < ms) {
taosMsleep(1000);
used_ms += 1000;
if(isQueryKilled(pQInfo)){
printf("test check query is canceled, sleep break.%s\n", pQInfo->sql);
break;
}
}
}
}
return 1;
}
#endif
bool qTableQuery(qinfo_t qinfo, uint64_t *qId) { bool qTableQuery(qinfo_t qinfo, uint64_t *qId) {
SQInfo *pQInfo = (SQInfo *)qinfo; SQInfo *pQInfo = (SQInfo *)qinfo;
assert(pQInfo && pQInfo->signature == pQInfo); assert(pQInfo && pQInfo->signature == pQInfo);
...@@ -228,7 +273,8 @@ bool qTableQuery(qinfo_t qinfo, uint64_t *qId) { ...@@ -228,7 +273,8 @@ bool qTableQuery(qinfo_t qinfo, uint64_t *qId) {
} }
*qId = pQInfo->qId; *qId = pQInfo->qId;
pQInfo->startExecTs = taosGetTimestampSec(); if(pQInfo->startExecTs == 0)
pQInfo->startExecTs = taosGetTimestampMs();
if (isQueryKilled(pQInfo)) { if (isQueryKilled(pQInfo)) {
qDebug("QInfo:0x%"PRIx64" it is already killed, abort", pQInfo->qId); qDebug("QInfo:0x%"PRIx64" it is already killed, abort", pQInfo->qId);
...@@ -259,7 +305,9 @@ bool qTableQuery(qinfo_t qinfo, uint64_t *qId) { ...@@ -259,7 +305,9 @@ bool qTableQuery(qinfo_t qinfo, uint64_t *qId) {
int64_t st = taosGetTimestampUs(); int64_t st = taosGetTimestampUs();
pRuntimeEnv->outputBuf = pRuntimeEnv->proot->exec(pRuntimeEnv->proot, &newgroup); pRuntimeEnv->outputBuf = pRuntimeEnv->proot->exec(pRuntimeEnv->proot, &newgroup);
pQInfo->summary.elapsedTime += (taosGetTimestampUs() - st); pQInfo->summary.elapsedTime += (taosGetTimestampUs() - st);
#ifdef TEST_IMPL
waitMoment(pQInfo);
#endif
publishOperatorProfEvent(pRuntimeEnv->proot, QUERY_PROF_AFTER_OPERATOR_EXEC); publishOperatorProfEvent(pRuntimeEnv->proot, QUERY_PROF_AFTER_OPERATOR_EXEC);
pRuntimeEnv->resultInfo.total += GET_NUM_OF_RESULTS(pRuntimeEnv); pRuntimeEnv->resultInfo.total += GET_NUM_OF_RESULTS(pRuntimeEnv);
...@@ -479,7 +527,7 @@ void qQueryMgmtNotifyClosed(void* pQMgmt) { ...@@ -479,7 +527,7 @@ void qQueryMgmtNotifyClosed(void* pQMgmt) {
pQueryMgmt->closed = true; pQueryMgmt->closed = true;
pthread_mutex_unlock(&pQueryMgmt->lock); pthread_mutex_unlock(&pQueryMgmt->lock);
taosCacheRefresh(pQueryMgmt->qinfoPool, queryMgmtKillQueryFn); taosCacheRefresh(pQueryMgmt->qinfoPool, queryMgmtKillQueryFn, NULL);
} }
void qQueryMgmtReOpen(void *pQMgmt) { void qQueryMgmtReOpen(void *pQMgmt) {
...@@ -574,3 +622,148 @@ void** qReleaseQInfo(void* pMgmt, void* pQInfo, bool freeHandle) { ...@@ -574,3 +622,148 @@ void** qReleaseQInfo(void* pMgmt, void* pQInfo, bool freeHandle) {
taosCacheRelease(pQueryMgmt->qinfoPool, pQInfo, freeHandle); taosCacheRelease(pQueryMgmt->qinfoPool, pQInfo, freeHandle);
return 0; return 0;
} }
//kill by qid
int32_t qKillQueryByQId(void* pMgmt, int64_t qId, int32_t waitMs, int32_t waitCount) {
int32_t error = TSDB_CODE_SUCCESS;
void** handle = qAcquireQInfo(pMgmt, qId);
if(handle == NULL) return terrno;
SQInfo* pQInfo = (SQInfo*)(*handle);
if (pQInfo == NULL || !isValidQInfo(pQInfo)) {
return TSDB_CODE_QRY_INVALID_QHANDLE;
}
qWarn("QId:0x%"PRIx64" be killed(no memory commit).", pQInfo->qId);
setQueryKilled(pQInfo);
// wait query stop
int32_t loop = 0;
while (pQInfo->owner != 0) {
taosMsleep(waitMs);
if(loop++ > waitCount){
error = TSDB_CODE_FAILED;
break;
}
}
qReleaseQInfo(pMgmt, (void **)&handle, true);
return error;
}
// local struct
typedef struct {
int64_t qId;
int64_t startExecTs;
} SLongQuery;
// callbark for sort compare
static int compareLongQuery(const void* p1, const void* p2) {
// sort desc
SLongQuery* plq1 = *(SLongQuery**)p1;
SLongQuery* plq2 = *(SLongQuery**)p2;
if(plq1->startExecTs == plq2->startExecTs) {
return 0;
} else if(plq1->startExecTs > plq2->startExecTs) {
return 1;
} else {
return -1;
}
}
// callback for taosCacheRefresh
static void cbFoundItem(void* handle, void* param1) {
SQInfo * qInfo = *(SQInfo**) handle;
if(qInfo == NULL) return ;
SArray* qids = (SArray*) param1;
if(qids == NULL) return ;
bool usedMem = true;
bool usedIMem = true;
SMemTable* mem = qInfo->query.memRef.snapshot.omem;
SMemTable* imem = qInfo->query.memRef.snapshot.imem;
if(mem == NULL || T_REF_VAL_GET(mem) == 0)
usedMem = false;
if(imem == NULL || T_REF_VAL_GET(mem) == 0)
usedIMem = false ;
if(!usedMem && !usedIMem)
return ;
// push to qids
SLongQuery* plq = (SLongQuery*)malloc(sizeof(SLongQuery));
plq->qId = qInfo->qId;
plq->startExecTs = qInfo->startExecTs;
taosArrayPush(qids, &plq);
}
// longquery
void* qObtainLongQuery(void* param){
SQueryMgmt* qMgmt = (SQueryMgmt*)param;
if(qMgmt == NULL || qMgmt->qinfoPool == NULL)
return NULL;
SArray* qids = taosArrayInit(4, sizeof(int64_t*));
if(qids == NULL) return NULL;
// Get each item
taosCacheRefresh(qMgmt->qinfoPool, cbFoundItem, qids);
size_t cnt = taosArrayGetSize(qids);
if(cnt == 0) {
taosArrayDestroy(qids);
return NULL;
}
if(cnt > 1)
taosArraySort(qids, compareLongQuery);
return qids;
}
//solve tsdb no block to commit
bool qFixedNoBlock(void* pRepo, void* pMgmt, int32_t longQueryMs) {
SQueryMgmt *pQueryMgmt = pMgmt;
bool fixed = false;
// qid top list
SArray *qids = (SArray*)qObtainLongQuery(pQueryMgmt);
if(qids == NULL) return false;
// kill Query
int64_t now = taosGetTimestampMs();
size_t cnt = taosArrayGetSize(qids);
size_t i;
SLongQuery* plq;
for(i=0; i < cnt; i++) {
plq = (SLongQuery* )taosArrayGetP(qids, i);
if(plq->startExecTs > now) continue;
if(now - plq->startExecTs >= longQueryMs) {
qKillQueryByQId(pMgmt, plq->qId, 500, 10); // wait 50*100 ms
if(tsdbNoProblem(pRepo)) {
fixed = true;
qWarn("QId:0x%"PRIx64" fixed problem after kill this query.", plq->qId);
break;
}
}
}
// free qids
for(i=0; i < cnt; i++) {
free(taosArrayGetP(qids, i));
}
taosArrayDestroy(qids);
return fixed;
}
//solve tsdb no block to commit
bool qSolveCommitNoBlock(void* pRepo, void* pMgmt) {
qWarn("pRepo=%p start solve problem.", pRepo);
if(qFixedNoBlock(pRepo, pMgmt, 10*60*1000)) {
return true;
}
if(qFixedNoBlock(pRepo, pMgmt, 2*60*1000)){
return true;
}
if(qFixedNoBlock(pRepo, pMgmt, 30*1000)){
return true;
}
qWarn("pRepo=%p solve problem failed.", pRepo);
return false;
}
src/rpc/src/rpcMain.c
浏览文件 @ e15abb2e
...@@ -407,7 +407,7 @@ void rpcSendRequest(void *shandle, const SRpcEpSet *pEpSet, SRpcMsg *pMsg, int64 ...@@ -407,7 +407,7 @@ void rpcSendRequest(void *shandle, const SRpcEpSet *pEpSet, SRpcMsg *pMsg, int64
if (type == TSDB_MSG_TYPE_QUERY || type == TSDB_MSG_TYPE_CM_RETRIEVE if (type == TSDB_MSG_TYPE_QUERY || type == TSDB_MSG_TYPE_CM_RETRIEVE
|| type == TSDB_MSG_TYPE_FETCH || type == TSDB_MSG_TYPE_CM_STABLE_VGROUP || type == TSDB_MSG_TYPE_FETCH || type == TSDB_MSG_TYPE_CM_STABLE_VGROUP
|| type == TSDB_MSG_TYPE_CM_TABLES_META || type == TSDB_MSG_TYPE_CM_TABLE_META || type == TSDB_MSG_TYPE_CM_TABLES_META || type == TSDB_MSG_TYPE_CM_TABLE_META
|| type == TSDB_MSG_TYPE_CM_SHOW || type == TSDB_MSG_TYPE_DM_STATUS) || type == TSDB_MSG_TYPE_CM_SHOW || type == TSDB_MSG_TYPE_DM_STATUS || type == TSDB_MSG_TYPE_CM_ALTER_TABLE)
pContext->connType = RPC_CONN_TCPC; pContext->connType = RPC_CONN_TCPC;
pContext->rid = taosAddRef(tsRpcRefId, pContext); pContext->rid = taosAddRef(tsRpcRefId, pContext);
......
src/tsdb/inc/tsdbBuffer.h
浏览文件 @ e15abb2e
...@@ -29,6 +29,7 @@ typedef struct { ...@@ -29,6 +29,7 @@ typedef struct {
int tBufBlocks; int tBufBlocks;
int nBufBlocks; int nBufBlocks;
int nRecycleBlocks; int nRecycleBlocks;
int nElasticBlocks;
int64_t index; int64_t index;
SList* bufBlockList; SList* bufBlockList;
} STsdbBufPool; } STsdbBufPool;
...@@ -41,6 +42,10 @@ int tsdbOpenBufPool(STsdbRepo* pRepo); ...@@ -41,6 +42,10 @@ int tsdbOpenBufPool(STsdbRepo* pRepo);
void tsdbCloseBufPool(STsdbRepo* pRepo); void tsdbCloseBufPool(STsdbRepo* pRepo);
SListNode* tsdbAllocBufBlockFromPool(STsdbRepo* pRepo); SListNode* tsdbAllocBufBlockFromPool(STsdbRepo* pRepo);
int tsdbExpandPool(STsdbRepo* pRepo, int32_t oldTotalBlocks); int tsdbExpandPool(STsdbRepo* pRepo, int32_t oldTotalBlocks);
void tsdbRecycleBufferBlock(STsdbBufPool* pPool, SListNode *pNode); void tsdbRecycleBufferBlock(STsdbBufPool* pPool, SListNode *pNode, bool bELastic);
// health cite
STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize);
void tsdbFreeBufBlock(STsdbBufBlock *pBufBlock);
#endif /* _TD_TSDB_BUFFER_H_ */ #endif /* _TD_TSDB_BUFFER_H_ */
src/tsdb/inc/tsdbHealth.h 0 → 100644
浏览文件 @ e15abb2e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef _TD_TSDB_HEALTH_H_
#define _TD_TSDB_HEALTH_H_
bool tsdbUrgeQueryFree(STsdbRepo* pRepo);
int32_t tsdbInsertNewBlock(STsdbRepo* pRepo);
bool tsdbIdleMemEnough();
bool tsdbAllowNewBlock(STsdbRepo* pRepo);
#endif /* _TD_TSDB_BUFFER_H_ */
src/tsdb/inc/tsdbMeta.h
浏览文件 @ e15abb2e
...@@ -100,7 +100,7 @@ static FORCE_INLINE int tsdbCompareSchemaVersion(const void *key1, const void *k ...@@ -100,7 +100,7 @@ static FORCE_INLINE int tsdbCompareSchemaVersion(const void *key1, const void *k
} }
static FORCE_INLINE STSchema* tsdbGetTableSchemaImpl(STable* pTable, bool lock, bool copy, int16_t _version) { static FORCE_INLINE STSchema* tsdbGetTableSchemaImpl(STable* pTable, bool lock, bool copy, int16_t _version) {
STable* pDTable = (TABLE_TYPE(pTable) == TSDB_CHILD_TABLE) ? pTable->pSuper : pTable; STable* pDTable = (pTable->pSuper != NULL) ? pTable->pSuper : pTable; // for performance purpose
STSchema* pSchema = NULL; STSchema* pSchema = NULL;
STSchema* pTSchema = NULL; STSchema* pTSchema = NULL;
......
src/tsdb/inc/tsdbint.h
浏览文件 @ e15abb2e
...@@ -97,6 +97,7 @@ struct STsdbRepo { ...@@ -97,6 +97,7 @@ struct STsdbRepo {
SMergeBuf mergeBuf; //used when update=2 SMergeBuf mergeBuf; //used when update=2
int8_t compactState; // compact state: inCompact/noCompact/waitingCompact? int8_t compactState; // compact state: inCompact/noCompact/waitingCompact?
pthread_t* pthread;
}; };
#define REPO_ID(r) (r)->config.tsdbId #define REPO_ID(r) (r)->config.tsdbId
......
src/tsdb/src/tsdbBuffer.c
浏览文件 @ e15abb2e
...@@ -14,12 +14,10 @@ ...@@ -14,12 +14,10 @@
*/ */
#include "tsdbint.h" #include "tsdbint.h"
#include "tsdbHealth.h"
#define POOL_IS_EMPTY(b) (listNEles((b)->bufBlockList) == 0) #define POOL_IS_EMPTY(b) (listNEles((b)->bufBlockList) == 0)
static STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize);
static void tsdbFreeBufBlock(STsdbBufBlock *pBufBlock);
// ---------------- INTERNAL FUNCTIONS ---------------- // ---------------- INTERNAL FUNCTIONS ----------------
STsdbBufPool *tsdbNewBufPool() { STsdbBufPool *tsdbNewBufPool() {
STsdbBufPool *pBufPool = (STsdbBufPool *)calloc(1, sizeof(*pBufPool)); STsdbBufPool *pBufPool = (STsdbBufPool *)calloc(1, sizeof(*pBufPool));
...@@ -65,10 +63,10 @@ int tsdbOpenBufPool(STsdbRepo *pRepo) { ...@@ -65,10 +63,10 @@ int tsdbOpenBufPool(STsdbRepo *pRepo) {
STsdbBufPool *pPool = pRepo->pPool; STsdbBufPool *pPool = pRepo->pPool;
ASSERT(pPool != NULL); ASSERT(pPool != NULL);
pPool->bufBlockSize = pCfg->cacheBlockSize * 1024 * 1024; // MB pPool->bufBlockSize = pCfg->cacheBlockSize * 1024 * 1024; // MB
pPool->tBufBlocks = pCfg->totalBlocks; pPool->tBufBlocks = pCfg->totalBlocks;
pPool->nBufBlocks = 0; pPool->nBufBlocks = 0;
pPool->nElasticBlocks = 0;
pPool->index = 0; pPool->index = 0;
pPool->nRecycleBlocks = 0; pPool->nRecycleBlocks = 0;
...@@ -120,6 +118,18 @@ SListNode *tsdbAllocBufBlockFromPool(STsdbRepo *pRepo) { ...@@ -120,6 +118,18 @@ SListNode *tsdbAllocBufBlockFromPool(STsdbRepo *pRepo) {
STsdbBufPool *pBufPool = pRepo->pPool; STsdbBufPool *pBufPool = pRepo->pPool;
while (POOL_IS_EMPTY(pBufPool)) { while (POOL_IS_EMPTY(pBufPool)) {
if(tsDeadLockKillQuery) {
// supply new Block
if(tsdbInsertNewBlock(pRepo) > 0) {
tsdbWarn("vgId:%d add new elastic block . elasticBlocks=%d cur free Blocks=%d", REPO_ID(pRepo), pBufPool->nElasticBlocks, pBufPool->bufBlockList->numOfEles);
break;
} else {
// no newBlock, kill query free
if(!tsdbUrgeQueryFree(pRepo))
tsdbWarn("vgId:%d Urge query free thread start failed.", REPO_ID(pRepo));
}
}
pRepo->repoLocked = false; pRepo->repoLocked = false;
pthread_cond_wait(&(pBufPool->poolNotEmpty), &(pRepo->mutex)); pthread_cond_wait(&(pBufPool->poolNotEmpty), &(pRepo->mutex));
pRepo->repoLocked = true; pRepo->repoLocked = true;
...@@ -139,11 +149,11 @@ SListNode *tsdbAllocBufBlockFromPool(STsdbRepo *pRepo) { ...@@ -139,11 +149,11 @@ SListNode *tsdbAllocBufBlockFromPool(STsdbRepo *pRepo) {
} }
// ---------------- LOCAL FUNCTIONS ---------------- // ---------------- LOCAL FUNCTIONS ----------------
static STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize) { STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize) {
STsdbBufBlock *pBufBlock = (STsdbBufBlock *)malloc(sizeof(*pBufBlock) + bufBlockSize); STsdbBufBlock *pBufBlock = (STsdbBufBlock *)malloc(sizeof(*pBufBlock) + bufBlockSize);
if (pBufBlock == NULL) { if (pBufBlock == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
goto _err; return NULL;
} }
pBufBlock->blockId = 0; pBufBlock->blockId = 0;
...@@ -151,13 +161,9 @@ static STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize) { ...@@ -151,13 +161,9 @@ static STsdbBufBlock *tsdbNewBufBlock(int bufBlockSize) {
pBufBlock->remain = bufBlockSize; pBufBlock->remain = bufBlockSize;
return pBufBlock; return pBufBlock;
_err:
tsdbFreeBufBlock(pBufBlock);
return NULL;
} }
static void tsdbFreeBufBlock(STsdbBufBlock *pBufBlock) { tfree(pBufBlock); } void tsdbFreeBufBlock(STsdbBufBlock *pBufBlock) { tfree(pBufBlock); }
int tsdbExpandPool(STsdbRepo* pRepo, int32_t oldTotalBlocks) { int tsdbExpandPool(STsdbRepo* pRepo, int32_t oldTotalBlocks) {
if (oldTotalBlocks == pRepo->config.totalBlocks) { if (oldTotalBlocks == pRepo->config.totalBlocks) {
...@@ -193,10 +199,16 @@ err: ...@@ -193,10 +199,16 @@ err:
return err; return err;
} }
void tsdbRecycleBufferBlock(STsdbBufPool* pPool, SListNode *pNode) { void tsdbRecycleBufferBlock(STsdbBufPool* pPool, SListNode *pNode, bool bELastic) {
STsdbBufBlock *pBufBlock = NULL; STsdbBufBlock *pBufBlock = NULL;
tdListNodeGetData(pPool->bufBlockList, pNode, (void *)(&pBufBlock)); tdListNodeGetData(pPool->bufBlockList, pNode, (void *)(&pBufBlock));
tsdbFreeBufBlock(pBufBlock); tsdbFreeBufBlock(pBufBlock);
free(pNode); free(pNode);
pPool->nBufBlocks--; if(bELastic)
} {
pPool->nElasticBlocks--;
tsdbWarn("pPool=%p elastic block reduce one . nElasticBlocks=%d cur free Blocks=%d", pPool, pPool->nElasticBlocks, pPool->bufBlockList->numOfEles);
}
else
pPool->nBufBlocks--;
}
\ No newline at end of file
src/tsdb/src/tsdbHealth.c 0 → 100644
浏览文件 @ e15abb2e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "os.h"
#include "taosmsg.h"
#include "tarray.h"
#include "query.h"
#include "tglobal.h"
#include "tlist.h"
#include "tsdbint.h"
#include "tsdbBuffer.h"
#include "tsdbLog.h"
#include "tsdbHealth.h"
#include "ttimer.h"
#include "tthread.h"
// return malloc new block count
int32_t tsdbInsertNewBlock(STsdbRepo * pRepo) {
STsdbBufPool *pPool = pRepo->pPool;
int32_t cnt = 0;
if(tsdbAllowNewBlock(pRepo)) {
STsdbBufBlock *pBufBlock = tsdbNewBufBlock(pPool->bufBlockSize);
if (pBufBlock) {
if (tdListAppend(pPool->bufBlockList, (void *)(&pBufBlock)) < 0) {
// append error
tsdbFreeBufBlock(pBufBlock);
} else {
pPool->nElasticBlocks ++;
cnt ++ ;
}
}
}
return cnt;
}
// switch anther thread to run
void* cbKillQueryFree(void* param) {
STsdbRepo* pRepo = (STsdbRepo*)param;
// vnode
if(pRepo->appH.notifyStatus) {
pRepo->appH.notifyStatus(pRepo->appH.appH, TSDB_STATUS_COMMIT_NOBLOCK, TSDB_CODE_SUCCESS);
}
// free
if(pRepo->pthread){
void* p = pRepo->pthread;
pRepo->pthread = NULL;
free(p);
}
return NULL;
}
// return true do free , false do nothing
bool tsdbUrgeQueryFree(STsdbRepo * pRepo) {
// check previous running
if(pRepo->pthread && taosThreadRunning(pRepo->pthread)) {
tsdbWarn("vgId:%d pre urge thread is runing. nBlocks=%d nElasticBlocks=%d", REPO_ID(pRepo), pRepo->pPool->nBufBlocks, pRepo->pPool->nElasticBlocks);
return false;
}
// create new
pRepo->pthread = taosCreateThread(cbKillQueryFree, pRepo);
if(pRepo->pthread == NULL) {
tsdbError("vgId:%d create urge thread error.", REPO_ID(pRepo));
return false;
}
return true;
}
bool tsdbAllowNewBlock(STsdbRepo* pRepo) {
int32_t nMaxElastic = pRepo->config.totalBlocks/3;
STsdbBufPool* pPool = pRepo->pPool;
if(pPool->nElasticBlocks >= nMaxElastic) {
tsdbWarn("vgId:%d tsdbAllowNewBlock return fasle. nElasticBlock(%d) >= MaxElasticBlocks(%d)", REPO_ID(pRepo), pPool->nElasticBlocks, nMaxElastic);
return false;
}
return true;
}
bool tsdbNoProblem(STsdbRepo* pRepo) {
if(listNEles(pRepo->pPool->bufBlockList) == 0)
return false;
return true;
}
\ No newline at end of file
src/tsdb/src/tsdbMain.c
浏览文件 @ e15abb2e
...@@ -16,6 +16,8 @@ ...@@ -16,6 +16,8 @@
// no test file errors here // no test file errors here
#include "taosdef.h" #include "taosdef.h"
#include "tsdbint.h" #include "tsdbint.h"
#include "ttimer.h"
#include "tthread.h"
#define IS_VALID_PRECISION(precision) \ #define IS_VALID_PRECISION(precision) \
(((precision) >= TSDB_TIME_PRECISION_MILLI) && ((precision) <= TSDB_TIME_PRECISION_NANO)) (((precision) >= TSDB_TIME_PRECISION_MILLI) && ((precision) <= TSDB_TIME_PRECISION_NANO))
...@@ -126,6 +128,10 @@ int tsdbCloseRepo(STsdbRepo *repo, int toCommit) { ...@@ -126,6 +128,10 @@ int tsdbCloseRepo(STsdbRepo *repo, int toCommit) {
terrno = TSDB_CODE_SUCCESS; terrno = TSDB_CODE_SUCCESS;
tsdbStopStream(pRepo); tsdbStopStream(pRepo);
if(pRepo->pthread){
taosDestoryThread(pRepo->pthread);
pRepo->pthread = NULL;
}
if (toCommit) { if (toCommit) {
tsdbSyncCommit(repo); tsdbSyncCommit(repo);
...@@ -547,6 +553,7 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH) { ...@@ -547,6 +553,7 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH) {
pRepo->appH = *pAppH; pRepo->appH = *pAppH;
} }
pRepo->repoLocked = false; pRepo->repoLocked = false;
pRepo->pthread = NULL;
int code = pthread_mutex_init(&(pRepo->mutex), NULL); int code = pthread_mutex_init(&(pRepo->mutex), NULL);
if (code != 0) { if (code != 0) {
......
src/tsdb/src/tsdbMemTable.c
浏览文件 @ e15abb2e
...@@ -99,17 +99,22 @@ int tsdbUnRefMemTable(STsdbRepo *pRepo, SMemTable *pMemTable) { ...@@ -99,17 +99,22 @@ int tsdbUnRefMemTable(STsdbRepo *pRepo, SMemTable *pMemTable) {
STsdbBufPool *pBufPool = pRepo->pPool; STsdbBufPool *pBufPool = pRepo->pPool;
SListNode *pNode = NULL; SListNode *pNode = NULL;
bool recycleBlocks = pBufPool->nRecycleBlocks > 0; bool addNew = false;
if (tsdbLockRepo(pRepo) < 0) return -1; if (tsdbLockRepo(pRepo) < 0) return -1;
while ((pNode = tdListPopHead(pMemTable->bufBlockList)) != NULL) { while ((pNode = tdListPopHead(pMemTable->bufBlockList)) != NULL) {
if (pBufPool->nRecycleBlocks > 0) { if (pBufPool->nRecycleBlocks > 0) {
tsdbRecycleBufferBlock(pBufPool, pNode); tsdbRecycleBufferBlock(pBufPool, pNode, false);
pBufPool->nRecycleBlocks -= 1; pBufPool->nRecycleBlocks -= 1;
} else { } else {
tdListAppendNode(pBufPool->bufBlockList, pNode); if(pBufPool->nElasticBlocks > 0 && listNEles(pBufPool->bufBlockList) > 2) {
tsdbRecycleBufferBlock(pBufPool, pNode, true);
} else {
tdListAppendNode(pBufPool->bufBlockList, pNode);
addNew = true;
}
} }
} }
if (!recycleBlocks) { if (addNew) {
int code = pthread_cond_signal(&pBufPool->poolNotEmpty); int code = pthread_cond_signal(&pBufPool->poolNotEmpty);
if (code != 0) { if (code != 0) {
if (tsdbUnlockRepo(pRepo) < 0) return -1; if (tsdbUnlockRepo(pRepo) < 0) return -1;
......
src/tsdb/src/tsdbRead.c
浏览文件 @ e15abb2e
...@@ -288,8 +288,6 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa ...@@ -288,8 +288,6 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa
STableKeyInfo* pKeyInfo = (STableKeyInfo*) taosArrayGet(group, j); STableKeyInfo* pKeyInfo = (STableKeyInfo*) taosArrayGet(group, j);
STableCheckInfo info = { .lastKey = pKeyInfo->lastKey, .pTableObj = pKeyInfo->pTable }; STableCheckInfo info = { .lastKey = pKeyInfo->lastKey, .pTableObj = pKeyInfo->pTable };
info.tableId = ((STable*)(pKeyInfo->pTable))->tableId;
assert(info.pTableObj != NULL && (info.pTableObj->type == TSDB_NORMAL_TABLE || assert(info.pTableObj != NULL && (info.pTableObj->type == TSDB_NORMAL_TABLE ||
info.pTableObj->type == TSDB_CHILD_TABLE || info.pTableObj->type == TSDB_STREAM_TABLE)); info.pTableObj->type == TSDB_CHILD_TABLE || info.pTableObj->type == TSDB_STREAM_TABLE));
...@@ -2218,7 +2216,7 @@ static int32_t createDataBlocksInfo(STsdbQueryHandle* pQueryHandle, int32_t numO ...@@ -2218,7 +2216,7 @@ static int32_t createDataBlocksInfo(STsdbQueryHandle* pQueryHandle, int32_t numO
SBlock* pBlock = pTableCheck->pCompInfo->blocks; SBlock* pBlock = pTableCheck->pCompInfo->blocks;
sup.numOfBlocksPerTable[numOfQualTables] = pTableCheck->numOfBlocks; sup.numOfBlocksPerTable[numOfQualTables] = pTableCheck->numOfBlocks;
char* buf = calloc(1, sizeof(STableBlockInfo) * pTableCheck->numOfBlocks); char* buf = malloc(sizeof(STableBlockInfo) * pTableCheck->numOfBlocks);
if (buf == NULL) { if (buf == NULL) {
cleanBlockOrderSupporter(&sup, numOfQualTables); cleanBlockOrderSupporter(&sup, numOfQualTables);
return TSDB_CODE_TDB_OUT_OF_MEMORY; return TSDB_CODE_TDB_OUT_OF_MEMORY;
...@@ -3618,8 +3616,6 @@ SArray* createTableGroup(SArray* pTableList, STSchema* pTagSchema, SColIndex* pC ...@@ -3618,8 +3616,6 @@ SArray* createTableGroup(SArray* pTableList, STSchema* pTagSchema, SColIndex* pC
for(int32_t i = 0; i < size; ++i) { for(int32_t i = 0; i < size; ++i) {
STableKeyInfo *pKeyInfo = taosArrayGet(pTableList, i); STableKeyInfo *pKeyInfo = taosArrayGet(pTableList, i);
assert(((STable*)pKeyInfo->pTable)->type == TSDB_CHILD_TABLE);
tsdbRefTable(pKeyInfo->pTable); tsdbRefTable(pKeyInfo->pTable);
STableKeyInfo info = {.pTable = pKeyInfo->pTable, .lastKey = skey}; STableKeyInfo info = {.pTable = pKeyInfo->pTable, .lastKey = skey};
......
src/util/inc/tcache.h
浏览文件 @ e15abb2e
...@@ -33,6 +33,7 @@ extern "C" { ...@@ -33,6 +33,7 @@ extern "C" {
#endif #endif
typedef void (*__cache_free_fn_t)(void*); typedef void (*__cache_free_fn_t)(void*);
typedef void (*__cache_trav_fn_t)(void*, void*);
typedef struct SCacheStatis { typedef struct SCacheStatis {
int64_t missCount; int64_t missCount;
...@@ -176,7 +177,7 @@ void taosCacheCleanup(SCacheObj *pCacheObj); ...@@ -176,7 +177,7 @@ void taosCacheCleanup(SCacheObj *pCacheObj);
* @param fp * @param fp
* @return * @return
*/ */
void taosCacheRefresh(SCacheObj *pCacheObj, __cache_free_fn_t fp); void taosCacheRefresh(SCacheObj *pCacheObj, __cache_trav_fn_t fp, void* param1);
/** /**
* stop background refresh worker thread * stop background refresh worker thread
......
src/util/inc/tconfig.h
浏览文件 @ e15abb2e
...@@ -20,7 +20,7 @@ ...@@ -20,7 +20,7 @@
extern "C" { extern "C" {
#endif #endif
#define TSDB_CFG_MAX_NUM 117 // 110 + 6 with lossy option #define TSDB_CFG_MAX_NUM 123
#define TSDB_CFG_PRINT_LEN 23 #define TSDB_CFG_PRINT_LEN 23
#define TSDB_CFG_OPTION_LEN 24 #define TSDB_CFG_OPTION_LEN 24
#define TSDB_CFG_VALUE_LEN 41 #define TSDB_CFG_VALUE_LEN 41
......
src/util/inc/tlosertree.h
浏览文件 @ e15abb2e
...@@ -26,7 +26,7 @@ typedef int (*__merge_compare_fn_t)(const void *, const void *, void *param); ...@@ -26,7 +26,7 @@ typedef int (*__merge_compare_fn_t)(const void *, const void *, void *param);
typedef struct SLoserTreeNode { typedef struct SLoserTreeNode {
int32_t index; int32_t index;
void * pData; void *pData;
} SLoserTreeNode; } SLoserTreeNode;
typedef struct SLoserTreeInfo { typedef struct SLoserTreeInfo {
...@@ -34,8 +34,7 @@ typedef struct SLoserTreeInfo { ...@@ -34,8 +34,7 @@ typedef struct SLoserTreeInfo {
int32_t totalEntries; int32_t totalEntries;
__merge_compare_fn_t comparFn; __merge_compare_fn_t comparFn;
void * param; void * param;
SLoserTreeNode *pNode;
SLoserTreeNode *pNode;
} SLoserTreeInfo; } SLoserTreeInfo;
uint32_t tLoserTreeCreate(SLoserTreeInfo **pTree, int32_t numOfEntries, void *param, __merge_compare_fn_t compareFn); uint32_t tLoserTreeCreate(SLoserTreeInfo **pTree, int32_t numOfEntries, void *param, __merge_compare_fn_t compareFn);
......
src/util/inc/tthread.h 0 → 100644
浏览文件 @ e15abb2e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_TTHREAD_H
#define TDENGINE_TTHREAD_H
#ifdef __cplusplus
extern "C" {
#endif
#include "os.h"
#include "taosdef.h"
// create new thread
pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param);
// destory thread
bool taosDestoryThread(pthread_t* pthread);
// thread running return true
bool taosThreadRunning(pthread_t* pthread);
#ifdef __cplusplus
}
#endif
#endif // TDENGINE_TTHREAD_H
src/util/src/hash.c
浏览文件 @ e15abb2e
...@@ -741,17 +741,19 @@ void taosHashTableResize(SHashObj *pHashObj) { ...@@ -741,17 +741,19 @@ void taosHashTableResize(SHashObj *pHashObj) {
} }
SHashNode *doCreateHashNode(const void *key, size_t keyLen, const void *pData, size_t dsize, uint32_t hashVal) { SHashNode *doCreateHashNode(const void *key, size_t keyLen, const void *pData, size_t dsize, uint32_t hashVal) {
SHashNode *pNewNode = calloc(1, sizeof(SHashNode) + keyLen + dsize); SHashNode *pNewNode = malloc(sizeof(SHashNode) + keyLen + dsize);
if (pNewNode == NULL) { if (pNewNode == NULL) {
uError("failed to allocate memory, reason:%s", strerror(errno)); uError("failed to allocate memory, reason:%s", strerror(errno));
return NULL; return NULL;
} }
pNewNode->keyLen = (uint32_t)keyLen; pNewNode->keyLen = (uint32_t)keyLen;
pNewNode->hashVal = hashVal; pNewNode->hashVal = hashVal;
pNewNode->dataLen = (uint32_t) dsize; pNewNode->dataLen = (uint32_t) dsize;
pNewNode->count = 1; pNewNode->count = 1;
pNewNode->removed = 0;
pNewNode->next = NULL;
memcpy(GET_HASH_NODE_DATA(pNewNode), pData, dsize); memcpy(GET_HASH_NODE_DATA(pNewNode), pData, dsize);
memcpy(GET_HASH_NODE_KEY(pNewNode), key, keyLen); memcpy(GET_HASH_NODE_KEY(pNewNode), key, keyLen);
......
src/util/src/tarray.c
浏览文件 @ e15abb2e
...@@ -24,11 +24,12 @@ void* taosArrayInit(size_t size, size_t elemSize) { ...@@ -24,11 +24,12 @@ void* taosArrayInit(size_t size, size_t elemSize) {
size = TARRAY_MIN_SIZE; size = TARRAY_MIN_SIZE;
} }
SArray* pArray = calloc(1, sizeof(SArray)); SArray* pArray = malloc(sizeof(SArray));
if (pArray == NULL) { if (pArray == NULL) {
return NULL; return NULL;
} }
pArray->size = 0;
pArray->pData = calloc(size, elemSize); pArray->pData = calloc(size, elemSize);
if (pArray->pData == NULL) { if (pArray->pData == NULL) {
free(pArray); free(pArray);
......
src/util/src/tcache.c
浏览文件 @ e15abb2e
...@@ -505,7 +505,8 @@ void taosCacheRelease(SCacheObj *pCacheObj, void **data, bool _remove) { ...@@ -505,7 +505,8 @@ void taosCacheRelease(SCacheObj *pCacheObj, void **data, bool _remove) {
typedef struct SHashTravSupp { typedef struct SHashTravSupp {
SCacheObj* pCacheObj; SCacheObj* pCacheObj;
int64_t time; int64_t time;
__cache_free_fn_t fp; __cache_trav_fn_t fp;
void* param1;
} SHashTravSupp; } SHashTravSupp;
static bool travHashTableEmptyFn(void* param, void* data) { static bool travHashTableEmptyFn(void* param, void* data) {
...@@ -667,17 +668,17 @@ bool travHashTableFn(void* param, void* data) { ...@@ -667,17 +668,17 @@ bool travHashTableFn(void* param, void* data) {
} }
if (ps->fp) { if (ps->fp) {
(ps->fp)(pNode->data); (ps->fp)(pNode->data, ps->param1);
} }
// do not remove element in hash table // do not remove element in hash table
return true; return true;
} }
static void doCacheRefresh(SCacheObj* pCacheObj, int64_t time, __cache_free_fn_t fp) { static void doCacheRefresh(SCacheObj* pCacheObj, int64_t time, __cache_trav_fn_t fp, void* param1) {
assert(pCacheObj != NULL); assert(pCacheObj != NULL);
SHashTravSupp sup = {.pCacheObj = pCacheObj, .fp = fp, .time = time}; SHashTravSupp sup = {.pCacheObj = pCacheObj, .fp = fp, .time = time, .param1 = param1};
taosHashCondTraverse(pCacheObj->pHashTable, travHashTableFn, &sup); taosHashCondTraverse(pCacheObj->pHashTable, travHashTableFn, &sup);
} }
...@@ -748,7 +749,7 @@ void* taosCacheTimedRefresh(void *handle) { ...@@ -748,7 +749,7 @@ void* taosCacheTimedRefresh(void *handle) {
// refresh data in hash table // refresh data in hash table
if (elemInHash > 0) { if (elemInHash > 0) {
int64_t now = taosGetTimestampMs(); int64_t now = taosGetTimestampMs();
doCacheRefresh(pCacheObj, now, NULL); doCacheRefresh(pCacheObj, now, NULL, NULL);
} }
taosTrashcanEmpty(pCacheObj, false); taosTrashcanEmpty(pCacheObj, false);
...@@ -766,13 +767,13 @@ void* taosCacheTimedRefresh(void *handle) { ...@@ -766,13 +767,13 @@ void* taosCacheTimedRefresh(void *handle) {
return NULL; return NULL;
} }
void taosCacheRefresh(SCacheObj *pCacheObj, __cache_free_fn_t fp) { void taosCacheRefresh(SCacheObj *pCacheObj, __cache_trav_fn_t fp, void* param1) {
if (pCacheObj == NULL) { if (pCacheObj == NULL) {
return; return;
} }
int64_t now = taosGetTimestampMs(); int64_t now = taosGetTimestampMs();
doCacheRefresh(pCacheObj, now, fp); doCacheRefresh(pCacheObj, now, fp, param1);
} }
void taosStopCacheRefreshWorker(void) { void taosStopCacheRefreshWorker(void) {
......
src/util/src/tlosertree.c
浏览文件 @ e15abb2e
...@@ -90,12 +90,13 @@ void tLoserTreeAdjust(SLoserTreeInfo* pTree, int32_t idx) { ...@@ -90,12 +90,13 @@ void tLoserTreeAdjust(SLoserTreeInfo* pTree, int32_t idx) {
SLoserTreeNode kLeaf = pTree->pNode[idx]; SLoserTreeNode kLeaf = pTree->pNode[idx];
while (parentId > 0) { while (parentId > 0) {
if (pTree->pNode[parentId].index == -1) { SLoserTreeNode* pCur = &pTree->pNode[parentId];
if (pCur->index == -1) {
pTree->pNode[parentId] = kLeaf; pTree->pNode[parentId] = kLeaf;
return; return;
} }
int32_t ret = pTree->comparFn(&pTree->pNode[parentId], &kLeaf, pTree->param); int32_t ret = pTree->comparFn(pCur, &kLeaf, pTree->param);
if (ret < 0) { if (ret < 0) {
SLoserTreeNode t = pTree->pNode[parentId]; SLoserTreeNode t = pTree->pNode[parentId];
pTree->pNode[parentId] = kLeaf; pTree->pNode[parentId] = kLeaf;
......
src/util/src/tthread.c 0 → 100644
浏览文件 @ e15abb2e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "os.h"
#include "tthread.h"
#include "tglobal.h"
#include "taosdef.h"
#include "tutil.h"
#include "tulog.h"
#include "taoserror.h"
// create new thread
pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param) {
pthread_t* pthread = (pthread_t*)malloc(sizeof(pthread_t));
pthread_attr_t thattr;
pthread_attr_init(&thattr);
pthread_attr_setdetachstate(&thattr, PTHREAD_CREATE_JOINABLE);
int32_t ret = pthread_create(pthread, &thattr, __start_routine, param);
pthread_attr_destroy(&thattr);
if (ret != 0) {
free(pthread);
return NULL;
}
return pthread;
}
// destory thread
bool taosDestoryThread(pthread_t* pthread) {
if(pthread == NULL) return false;
if(taosThreadRunning(pthread)) {
pthread_cancel(*pthread);
pthread_join(*pthread, NULL);
}
free(pthread);
return true;
}
// thread running return true
bool taosThreadRunning(pthread_t* pthread) {
if(pthread == NULL) return false;
int ret = pthread_kill(*pthread, 0);
if(ret == ESRCH)
return false;
if(ret == EINVAL)
return false;
// alive
return true;
}
src/vnode/src/vnodeMain.c
浏览文件 @ e15abb2e
...@@ -560,5 +560,10 @@ static int32_t vnodeProcessTsdbStatus(void *arg, int32_t status, int32_t eno) { ...@@ -560,5 +560,10 @@ static int32_t vnodeProcessTsdbStatus(void *arg, int32_t status, int32_t eno) {
return vnodeSaveVersion(pVnode); return vnodeSaveVersion(pVnode);
} }
// timer thread callback
if(status == TSDB_STATUS_COMMIT_NOBLOCK) {
qSolveCommitNoBlock(pVnode->tsdb, pVnode->qMgmt);
}
return 0; return 0;
} }
tests/connectorTest/C#Test/nanosupport/TDengineDriver.cs 0 → 100644
浏览文件 @ e15abb2e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
namespace TDengineDriver
{
enum TDengineDataType
{
TSDB_DATA_TYPE_NULL = 0, // 1 bytes
TSDB_DATA_TYPE_BOOL = 1, // 1 bytes
TSDB_DATA_TYPE_TINYINT = 2, // 1 bytes
TSDB_DATA_TYPE_SMALLINT = 3, // 2 bytes
TSDB_DATA_TYPE_INT = 4, // 4 bytes
TSDB_DATA_TYPE_BIGINT = 5, // 8 bytes
TSDB_DATA_TYPE_FLOAT = 6, // 4 bytes
TSDB_DATA_TYPE_DOUBLE = 7, // 8 bytes
TSDB_DATA_TYPE_BINARY = 8, // string
TSDB_DATA_TYPE_TIMESTAMP = 9,// 8 bytes
TSDB_DATA_TYPE_NCHAR = 10, // unicode string
TSDB_DATA_TYPE_UTINYINT = 11,// 1 byte
TSDB_DATA_TYPE_USMALLINT= 12,// 2 bytes
TSDB_DATA_TYPE_UINT = 13, // 4 bytes
TSDB_DATA_TYPE_UBIGINT= 14 // 8 bytes
}
enum TDengineInitOption
{
TSDB_OPTION_LOCALE = 0,
TSDB_OPTION_CHARSET = 1,
TSDB_OPTION_TIMEZONE = 2,
TDDB_OPTION_CONFIGDIR = 3,
TDDB_OPTION_SHELL_ACTIVITY_TIMER = 4
}
class TDengineMeta
{
public string name;
public short size;
public byte type;
public string TypeName()
{
switch ((TDengineDataType)type)
{
case TDengineDataType.TSDB_DATA_TYPE_BOOL:
return "BOOL";
case TDengineDataType.TSDB_DATA_TYPE_TINYINT:
return "TINYINT";
case TDengineDataType.TSDB_DATA_TYPE_SMALLINT:
return "SMALLINT";
case TDengineDataType.TSDB_DATA_TYPE_INT:
return "INT";
case TDengineDataType.TSDB_DATA_TYPE_BIGINT:
return "BIGINT";
case TDengineDataType.TSDB_DATA_TYPE_UTINYINT:
return "TINYINT UNSIGNED";
case TDengineDataType.TSDB_DATA_TYPE_USMALLINT:
return "SMALLINT UNSIGNED";
case TDengineDataType.TSDB_DATA_TYPE_UINT:
return "INT UNSIGNED";
case TDengineDataType.TSDB_DATA_TYPE_UBIGINT:
return "BIGINT UNSIGNED";
case TDengineDataType.TSDB_DATA_TYPE_FLOAT:
return "FLOAT";
case TDengineDataType.TSDB_DATA_TYPE_DOUBLE:
return "DOUBLE";
case TDengineDataType.TSDB_DATA_TYPE_BINARY:
return "STRING";
case TDengineDataType.TSDB_DATA_TYPE_TIMESTAMP:
return "TIMESTAMP";
case TDengineDataType.TSDB_DATA_TYPE_NCHAR:
return "NCHAR";
default:
return "undefine";
}
}
}
class TDengine
{
public const int TSDB_CODE_SUCCESS = 0;
[DllImport("taos", EntryPoint = "taos_init", CallingConvention = CallingConvention.Cdecl)]
static extern public void Init();
[DllImport("taos", EntryPoint = "taos_cleanup", CallingConvention = CallingConvention.Cdecl)]
static extern public void Cleanup();
[DllImport("taos", EntryPoint = "taos_options", CallingConvention = CallingConvention.Cdecl)]
static extern public void Options(int option, string value);
[DllImport("taos", EntryPoint = "taos_connect", CallingConvention = CallingConvention.Cdecl)]
static extern public IntPtr Connect(string ip, string user, string password, string db, short port);
[DllImport("taos", EntryPoint = "taos_errstr", CallingConvention = CallingConvention.Cdecl)]
static extern private IntPtr taos_errstr(IntPtr res);
static public string Error(IntPtr res)
{
IntPtr errPtr = taos_errstr(res);
return Marshal.PtrToStringAnsi(errPtr);
}
[DllImport("taos", EntryPoint = "taos_errno", CallingConvention = CallingConvention.Cdecl)]
static extern public int ErrorNo(IntPtr res);
[DllImport("taos", EntryPoint = "taos_query", CallingConvention = CallingConvention.Cdecl)]
static extern public IntPtr Query(IntPtr conn, string sqlstr);
[DllImport("taos", EntryPoint = "taos_affected_rows", CallingConvention = CallingConvention.Cdecl)]
static extern public int AffectRows(IntPtr res);
[DllImport("taos", EntryPoint = "taos_field_count", CallingConvention = CallingConvention.Cdecl)]
static extern public int FieldCount(IntPtr res);
[DllImport("taos", EntryPoint = "taos_fetch_fields", CallingConvention = CallingConvention.Cdecl)]
static extern private IntPtr taos_fetch_fields(IntPtr res);
static public List<TDengineMeta> FetchFields(IntPtr res)
{
const int fieldSize = 68;
List<TDengineMeta> metas = new List<TDengineMeta>();
if (res == IntPtr.Zero)
{
return metas;
}
int fieldCount = FieldCount(res);
IntPtr fieldsPtr = taos_fetch_fields(res);
for (int i = 0; i < fieldCount; ++i)
{
int offset = i * fieldSize;
TDengineMeta meta = new TDengineMeta();
meta.name = Marshal.PtrToStringAnsi(fieldsPtr + offset);
meta.type = Marshal.ReadByte(fieldsPtr + offset + 65);
meta.size = Marshal.ReadInt16(fieldsPtr + offset + 66);
metas.Add(meta);
}
return metas;
}
[DllImport("taos", EntryPoint = "taos_fetch_row", CallingConvention = CallingConvention.Cdecl)]
static extern public IntPtr FetchRows(IntPtr res);
[DllImport("taos", EntryPoint = "taos_free_result", CallingConvention = CallingConvention.Cdecl)]
static extern public IntPtr FreeResult(IntPtr res);
[DllImport("taos", EntryPoint = "taos_close", CallingConvention = CallingConvention.Cdecl)]
static extern public int Close(IntPtr taos);
//get precisionin parameter restultset
[DllImport("taos", EntryPoint = "taos_result_precision", CallingConvention = CallingConvention.Cdecl)]
static extern public int ResultPrecision(IntPtr taos);
}
}
tests/connectorTest/C#Test/nanosupport/nanotest.cs 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nanosupport/nanosecondTest.js 0 → 100644
浏览文件 @ e15abb2e
const taos = require('td2.0-connector');
var conn = taos.connect({host:"localhost", user:"root", password:"taosdata", config:"/etc/taos",port:6030})
var c1 = conn.cursor();
function checkData(sql,row,col,data){
console.log(sql)
c1.execute(sql)
var d = c1.fetchall();
let checkdata = d[row][col];
if (checkdata == data) {
console.log('check pass')
}
else{
console.log('check failed')
console.log('checked is :',checkdata)
console.log("expected is :",data)
}
}
// nano basic case
c1.execute('reset query cache')
c1.execute('drop database if exists db')
c1.execute('create database db precision "ns";')
c1.execute('use db');
c1.execute('create table tb (ts timestamp, speed int)')
c1.execute('insert into tb values(\'2021-06-10 00:00:00.100000001\', 1);')
c1.execute('insert into tb values(1623254400150000000, 2);')
c1.execute('import into tb values(1623254400300000000, 3);')
c1.execute('import into tb values(1623254400299999999, 4);')
c1.execute('insert into tb values(1623254400300000001, 5);')
c1.execute('insert into tb values(1623254400999999999, 7);')
c1.execute('insert into tb values(1623254400123456789, 8);')
sql = 'select * from tb;'
console.log('*******************************************')
console.log('this is area about checkdata result')
//check data about insert data
checkData(sql,0,0,'2021-06-10 00:00:00.100000001')
checkData(sql,1,0,'2021-06-10 00:00:00.123456789')
checkData(sql,2,0,'2021-06-10 00:00:00.150000000')
checkData(sql,3,0,'2021-06-10 00:00:00.299999999')
checkData(sql,4,0,'2021-06-10 00:00:00.300000000')
checkData(sql,5,0,'2021-06-10 00:00:00.300000001')
checkData(sql,6,0,'2021-06-10 00:00:00.999999999')
checkData(sql,0,1,1)
checkData(sql,1,1,8)
checkData(sql,2,1,2)
checkData(sql,5,1,5)
// us basic case
c1.execute('reset query cache')
c1.execute('drop database if exists usdb')
c1.execute('create database usdb precision "us";')
c1.execute('use usdb');
c1.execute('create table tb (ts timestamp, speed int)')
c1.execute('insert into tb values(\'2021-06-10 00:00:00.100001\', 1);')
c1.execute('insert into tb values(1623254400150000, 2);')
c1.execute('import into tb values(1623254400300000, 3);')
c1.execute('import into tb values(1623254400299999, 4);')
c1.execute('insert into tb values(1623254400300001, 5);')
c1.execute('insert into tb values(1623254400999999, 7);')
c1.execute('insert into tb values(1623254400123789, 8);')
sql = 'select * from tb;'
console.log('*******************************************')
//check data about insert data
checkData(sql,0,0,'2021-06-10 00:00:00.100001')
checkData(sql,1,0,'2021-06-10 00:00:00.123789')
checkData(sql,2,0,'2021-06-10 00:00:00.150000')
checkData(sql,3,0,'2021-06-10 00:00:00.299999')
checkData(sql,4,0,'2021-06-10 00:00:00.300000')
checkData(sql,5,0,'2021-06-10 00:00:00.300001')
checkData(sql,6,0,'2021-06-10 00:00:00.999999')
checkData(sql,0,1,1)
checkData(sql,1,1,8)
checkData(sql,2,1,2)
checkData(sql,5,1,5)
console.log('*******************************************')
// ms basic case
c1.execute('reset query cache')
c1.execute('drop database if exists msdb')
c1.execute('create database msdb precision "ms";')
c1.execute('use msdb');
c1.execute('create table tb (ts timestamp, speed int)')
c1.execute('insert into tb values(\'2021-06-10 00:00:00.101\', 1);')
c1.execute('insert into tb values(1623254400150, 2);')
c1.execute('import into tb values(1623254400300, 3);')
c1.execute('import into tb values(1623254400299, 4);')
c1.execute('insert into tb values(1623254400301, 5);')
c1.execute('insert into tb values(1623254400789, 7);')
c1.execute('insert into tb values(1623254400999, 8);')
sql = 'select * from tb;'
console.log('*******************************************')
console.log('this is area about checkdata result')
//check data about insert data
checkData(sql,0,0,'2021-06-10 00:00:00.101')
checkData(sql,1,0,'2021-06-10 00:00:00.150')
checkData(sql,2,0,'2021-06-10 00:00:00.299')
checkData(sql,3,0,'2021-06-10 00:00:00.300')
checkData(sql,4,0,'2021-06-10 00:00:00.301')
checkData(sql,5,0,'2021-06-10 00:00:00.789')
checkData(sql,6,0,'2021-06-10 00:00:00.999')
checkData(sql,0,1,1)
checkData(sql,1,1,2)
checkData(sql,2,1,4)
checkData(sql,5,1,7)
console.log('*******************************************')
// offfical query result to show
// console.log('this is area about fetch all data')
// var query = c1.query(sql)
// var promise = query.execute();
// promise.then(function(result) {
// result.pretty();
// });
console.log('*******************************************')
c1.execute('use db')
sql2 = 'select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400100000002;'
checkData(sql2,0,0,1)
sql3 = 'select count(*) from tb where ts > \'2021-06-10 0:00:00.100000001\' and ts < \'2021-06-10 0:00:00.160000000\';'
checkData(sql3,0,0,2)
sql4 = 'select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400150000000;'
checkData(sql4,0,0,2)
sql5 = 'select count(*) from tb where ts > \'2021-06-10 0:00:00.100000000\' and ts < \'2021-06-10 0:00:00.150000000\';'
checkData(sql5,0,0,2)
sql6 = 'select count(*) from tb where ts > 1623254400400000000;'
checkData(sql6,0,0,1)
sql7 = 'select count(*) from tb where ts < \'2021-06-10 00:00:00.400000000\';'
checkData(sql7,0,0,6)
sql8 = 'select count(*) from tb where ts > now + 400000000b;'
c1.execute(sql8)
sql9 = 'select count(*) from tb where ts >= \'2021-06-10 0:00:00.100000001\';'
checkData(sql9,0,0,7)
sql10 = 'select count(*) from tb where ts <= 1623254400300000000;'
checkData(sql10,0,0,5)
sql11 = 'select count(*) from tb where ts = \'2021-06-10 0:00:00.000000000\';'
c1.execute(sql11)
sql12 = 'select count(*) from tb where ts = 1623254400150000000;'
checkData(sql12,0,0,1)
sql13 = 'select count(*) from tb where ts = \'2021-06-10 0:00:00.100000001\';'
checkData(sql13,0,0,1)
sql14 = 'select count(*) from tb where ts between 1623254400000000000 and 1623254400400000000;'
checkData(sql14,0,0,6)
sql15 = 'select count(*) from tb where ts between \'2021-06-10 0:00:00.299999999\' and \'2021-06-10 0:00:00.300000001\';'
checkData(sql15,0,0,3)
sql16 = 'select avg(speed) from tb interval(5000000000b);'
checkData(sql16,0,0,'2021-06-10 00:00:00.000000000')
sql17 = 'select avg(speed) from tb interval(100000000b)'
checkData(sql17,0,1,3.6666666666666665)
checkData(sql17,1,1,4.000000000)
checkData(sql17,2,0,'2021-06-10 00:00:00.300000000')
checkData(sql17,3,0,'2021-06-10 00:00:00.900000000')
console.log("print break ")
// sql18 = 'select avg(speed) from tb interval(999b)'
// c1.execute(sql18)
console.log("print break2 ")
sql19 = 'select avg(speed) from tb interval(1u);'
checkData(sql19,2,1,2.000000000)
checkData(sql19,3,0,'2021-06-10 00:00:00.299999000')
sql20 = 'select avg(speed) from tb interval(100000000b) sliding (100000000b);'
checkData(sql20,2,1,4.000000000)
checkData(sql20,3,0,'2021-06-10 00:00:00.900000000')
sql21 = 'select last(*) from tb;'
checkData(sql21,0,0,'2021-06-10 00:00:00.999999999')
sql22 = 'select first(*) from tb;'
checkData(sql22,0,0,'2021-06-10 00:00:00.100000001')
// timezone support
console.log('testing nanosecond support in other timestamps')
c1.execute('create table tb2 (ts timestamp, speed int, ts2 timestamp);')
c1.execute('insert into tb2 values(\'2021-06-10 0:00:00.100000001\', 1, \'2021-06-11 0:00:00.100000001\');')
c1.execute('insert into tb2 values(1623254400150000000, 2, 1623340800150000000);')
c1.execute('import into tb2 values(1623254400300000000, 3, 1623340800300000000);')
c1.execute('import into tb2 values(1623254400299999999, 4, 1623340800299999999);')
c1.execute('insert into tb2 values(1623254400300000001, 5, 1623340800300000001);')
c1.execute('insert into tb2 values(1623254400999999999, 7, 1623513600999999999);')
sql23 = 'select * from tb2;'
checkData(sql23,0,0,'2021-06-10 00:00:00.100000001')
checkData(sql23,1,0,'2021-06-10 00:00:00.150000000')
checkData(sql23,2,1,4)
checkData(sql23,3,1,3)
checkData(sql23,4,2,'2021-06-11 00:00:00.300000001')
checkData(sql23,5,2,'2021-06-13 00:00:00.999999999')
sql24 = 'select count(*) from tb2 where ts2 >= \'2021-06-11 0:00:00.100000001\';'
checkData(sql24,0,0,6)
sql25 = 'select count(*) from tb2 where ts2 <= 1623340800400000000;'
checkData(sql25,0,0,5)
sql26 = 'select count(*) from tb2 where ts2 = \'2021-06-11 0:00:00.300000001\';'
checkData(sql26,0,0,1)
sql27 = 'select count(*) from tb2 where ts2 = 1623340800300000001;'
checkData(sql27,0,0,1)
sql28 = 'select count(*) from tb2 where ts2 between 1623340800000000000 and 1623340800450000000;'
checkData(sql28,0,0,5)
sql29 = 'select count(*) from tb2 where ts2 between \'2021-06-11 0:00:00.299999999\' and \'2021-06-11 0:00:00.300000001\';'
checkData(sql29,0,0,3)
sql30 = 'select count(*) from tb2 where ts2 <> 1623513600999999999;'
checkData(sql30,0,0,5)
sql31 = 'select count(*) from tb2 where ts2 <> \'2021-06-11 0:00:00.100000001\';'
checkData(sql31,0,0,5)
sql32 = 'select count(*) from tb2 where ts2 != 1623513600999999999;'
checkData(sql32,0,0,5)
sql33 = 'select count(*) from tb2 where ts2 != \'2021-06-11 0:00:00.100000001\';'
checkData(sql33,0,0,5)
c1.execute('insert into tb2 values(now + 500000000b, 6, now +2d);')
sql34 = 'select count(*) from tb2;'
checkData(sql34,0,0,7)
// check timezone support
c1.execute('use db;')
c1.execute('create stable st (ts timestamp ,speed float ) tags(time timestamp ,id int);')
c1.execute('insert into stb1 using st tags("2021-06-10 0:00:00.123456789" , 1 ) values("2021-06-10T0:00:00.123456789+07:00" , 1.0);' )
sql35 = 'select first(*) from stb1;'
checkData(sql35,0,0,'2021-06-10 01:00:00.123456789')
c1.execute('use usdb;')
c1.execute('create stable st (ts timestamp ,speed float ) tags(time timestamp ,id int);')
c1.execute('insert into stb1 using st tags("2021-06-10 0:00:00.123456" , 1 ) values("2021-06-10T0:00:00.123456+07:00" , 1.0);' )
sql36 = 'select first(*) from stb1;'
checkData(sql36,0,0,'2021-06-10 01:00:00.123456')
c1.execute('use msdb;')
c1.execute('create stable st (ts timestamp ,speed float ) tags(time timestamp ,id int);')
c1.execute('insert into stb1 using st tags("2021-06-10 0:00:00.123456" , 1 ) values("2021-06-10T0:00:00.123456+07:00" , 1.0);' )
sql36 = 'select first(*) from stb1;'
checkData(sql36,0,0,'2021-06-10 01:00:00.123')
tests/connectorTest/nodejsTest/nodetaos/cinterface.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/connection.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/constants.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/cursor.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/error.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/globalfunc.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/taosobjects.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/taosquery.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/nodetaos/taosresult.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/readme.md 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/tdengine.js 0 → 100644
浏览文件 @ e15abb2e
var TDengineConnection = require('./nodetaos/connection.js')
module.exports.connect = function (connection={}) {
return new TDengineConnection(connection);
}
tests/connectorTest/nodejsTest/test/performance.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/test/test.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/test/testMicroseconds.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/test/testNanoseconds.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/nodejsTest/test/testSubscribe.js 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/odbcTest/nanosupport/odbc.go 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/connectorTest/odbcTest/nanosupport/odbc.py 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/batchtest.bat
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/batchtest.sh
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/case001/case001.go
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/case001/case001.sh
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/case002/case002.bat
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/case002/case002.go
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/case002/case002.sh
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/nanosupport/connector/executor.go 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/nanosupport/nanoCase.bat 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/nanosupport/nanoCase.sh 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/gotest/nanosupport/nanosupport.go 0 → 100644
浏览文件 @ e15abb2e
此差异已折叠。
tests/pytest/tools/taosdemoTest.py
浏览文件 @ e15abb2e
此差异已折叠。
tests/pytest/tools/taosdumpTest2.py
浏览文件 @ e15abb2e
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或