提交 e5a2a446 编写于 作者: L liuyq-617

Merge branch 'develop' into test/testcase

...@@ -27,7 +27,6 @@ tests/hdfs/ ...@@ -27,7 +27,6 @@ tests/hdfs/
nmake/ nmake/
sln/ sln/
hdfs/ hdfs/
c/
taoshebei/ taoshebei/
taosdalipu/ taosdalipu/
Target/ Target/
......
...@@ -227,6 +227,8 @@ pipeline { ...@@ -227,6 +227,8 @@ pipeline {
./test-all.sh p4 ./test-all.sh p4
cd ${WKC}/tests cd ${WKC}/tests
./test-all.sh full jdbc ./test-all.sh full jdbc
cd ${WKC}/tests
./test-all.sh full unit
date''' date'''
} }
} }
......
...@@ -32,7 +32,7 @@ ELSEIF (TD_WINDOWS) ...@@ -32,7 +32,7 @@ ELSEIF (TD_WINDOWS)
#INSTALL(TARGETS taos RUNTIME DESTINATION driver) #INSTALL(TARGETS taos RUNTIME DESTINATION driver)
#INSTALL(TARGETS shell RUNTIME DESTINATION .) #INSTALL(TARGETS shell RUNTIME DESTINATION .)
IF (TD_MVN_INSTALLED) IF (TD_MVN_INSTALLED)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.20-dist.jar DESTINATION connector/jdbc) INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.21-dist.jar DESTINATION connector/jdbc)
ENDIF () ENDIF ()
ELSEIF (TD_DARWIN) ELSEIF (TD_DARWIN)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh") SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
......
...@@ -31,6 +31,20 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专 ...@@ -31,6 +31,20 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专
* [创建超级表](/model#create-stable):为同一类型的数据采集点创建一个超级表 * [创建超级表](/model#create-stable):为同一类型的数据采集点创建一个超级表
* [创建表](/model#create-table):使用超级表做模板,为每一个具体的数据采集点单独建表 * [创建表](/model#create-table):使用超级表做模板,为每一个具体的数据采集点单独建表
## [TAOS SQL](/taos-sql)
* [支持的数据类型](/taos-sql#data-type):支持时间戳、整型、浮点型、布尔型、字符型等多种数据类型
* [数据库管理](/taos-sql#management):添加、删除、查看数据库
* [表管理](/taos-sql#table):添加、删除、查看、修改表
* [超级表管理](/taos-sql#super-table):添加、删除、查看、修改超级表
* [标签管理](/taos-sql#tags):增加、删除、修改标签
* [数据写入](/taos-sql#insert):支持单表单条、多条、多表多条写入,支持历史数据写入
* [数据查询](/taos-sql#select):支持时间段、值过滤、排序、查询结果手动分页等
* [SQL函数](/taos-sql#functions):支持各种聚合函数、选择函数、计算函数,如avg, min, diff等
* [时间维度聚合](/taos-sql#aggregation):将表中数据按照时间段进行切割后聚合,降维处理
* [边界限制](/taos-sql#limitation):库、表、SQL等边界限制条件
* [错误码](/taos-sql/error-code):TDengine 2.0 错误码以及对应的十进制码
## [高效写入数据](/insert) ## [高效写入数据](/insert)
* [SQL写入](/insert#sql):使用SQL insert命令向一张或多张表写入单条或多条记录 * [SQL写入](/insert#sql):使用SQL insert命令向一张或多张表写入单条或多条记录
...@@ -94,20 +108,6 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专 ...@@ -94,20 +108,6 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专
* [文件目录结构](/administrator#directories):TDengine数据文件、配置文件等所在目录 * [文件目录结构](/administrator#directories):TDengine数据文件、配置文件等所在目录
* [参数限制与保留关键字](/administrator#keywords):TDengine的参数限制与保留关键字列表 * [参数限制与保留关键字](/administrator#keywords):TDengine的参数限制与保留关键字列表
## [TAOS SQL](/taos-sql)
* [支持的数据类型](/taos-sql#data-type):支持时间戳、整型、浮点型、布尔型、字符型等多种数据类型
* [数据库管理](/taos-sql#management):添加、删除、查看数据库
* [表管理](/taos-sql#table):添加、删除、查看、修改表
* [超级表管理](/taos-sql#super-table):添加、删除、查看、修改超级表
* [标签管理](/taos-sql#tags):增加、删除、修改标签
* [数据写入](/taos-sql#insert):支持单表单条、多条、多表多条写入,支持历史数据写入
* [数据查询](/taos-sql#select):支持时间段、值过滤、排序、查询结果手动分页等
* [SQL函数](/taos-sql#functions):支持各种聚合函数、选择函数、计算函数,如avg, min, diff等
* [时间维度聚合](/taos-sql#aggregation):将表中数据按照时间段进行切割后聚合,降维处理
* [边界限制](/taos-sql#limitation):库、表、SQL等边界限制条件
* [错误码](/taos-sql/error-code):TDengine 2.0 错误码以及对应的十进制码
## TDengine的技术设计 ## TDengine的技术设计
* [系统模块](/architecture/taosd):taosd的功能和模块划分 * [系统模块](/architecture/taosd):taosd的功能和模块划分
...@@ -120,6 +120,7 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专 ...@@ -120,6 +120,7 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专
* [TDengine性能对比测试工具](https://www.taosdata.com/blog/2020/01/18/1166.html) * [TDengine性能对比测试工具](https://www.taosdata.com/blog/2020/01/18/1166.html)
* [IDEA数据库管理工具可视化使用TDengine](https://www.taosdata.com/blog/2020/08/27/1767.html) * [IDEA数据库管理工具可视化使用TDengine](https://www.taosdata.com/blog/2020/08/27/1767.html)
* [基于eletron开发的跨平台TDengine图形化管理工具](https://github.com/skye0207/TDengineGUI) * [基于eletron开发的跨平台TDengine图形化管理工具](https://github.com/skye0207/TDengineGUI)
* [DataX,支持TDengine的离线数据采集/同步工具](https://github.com/alibaba/DataX)
## TDengine与其他数据库的对比测试 ## TDengine与其他数据库的对比测试
......
...@@ -178,11 +178,11 @@ TDengine 分布式架构的逻辑结构图如下: ...@@ -178,11 +178,11 @@ TDengine 分布式架构的逻辑结构图如下:
**FQDN配置**:一个数据节点有一个或多个FQDN,可以在系统配置文件taos.cfg通过参数“fqdn"进行指定,如果没有指定,系统将自动获取计算机的hostname作为其FQDN。如果节点没有配置FQDN,可以直接将该节点的配置参数fqdn设置为它的IP地址。但不建议使用IP,因为IP地址可变,一旦变化,将让集群无法正常工作。一个数据节点的EP(End Point)由FQDN + Port组成。采用FQDN,需要保证DNS服务正常工作,或者在节点以及应用所在的节点配置好hosts文件。 **FQDN配置**:一个数据节点有一个或多个FQDN,可以在系统配置文件taos.cfg通过参数“fqdn"进行指定,如果没有指定,系统将自动获取计算机的hostname作为其FQDN。如果节点没有配置FQDN,可以直接将该节点的配置参数fqdn设置为它的IP地址。但不建议使用IP,因为IP地址可变,一旦变化,将让集群无法正常工作。一个数据节点的EP(End Point)由FQDN + Port组成。采用FQDN,需要保证DNS服务正常工作,或者在节点以及应用所在的节点配置好hosts文件。
**端口配置:**一个数据节点对外的端口由TDengine的系统配置参数serverPort决定,对集群内部通讯的端口是serverPort+5。集群内数据节点之间的数据复制操作还占有一个TCP端口,是serverPort+10. 为支持多线程高效的处理UDP数据,每个对内和对外的UDP连接,都需要占用5个连续的端口。因此一个数据节点总的端口范围为serverPort到serverPort + 10,总共11个TCP/UDP端口。使用时,需要确保防火墙将这些端口打开。每个数据节点可以配置不同的serverPort。 **端口配置:**一个数据节点对外的端口由TDengine的系统配置参数serverPort决定,对集群内部通讯的端口是serverPort+5。集群内数据节点之间的数据复制操作还占有一个TCP端口,是serverPort+10. 为支持多线程高效的处理UDP数据,每个对内和对外的UDP连接,都需要占用5个连续的端口。因此一个数据节点总的端口范围为serverPort到serverPort + 10,总共11个TCP/UDP端口。(另外还可能有 RESTful、Arbitrator 所使用的端口,那样的话就一共是 13 个。)使用时,需要确保防火墙将这些端口打开,以备使用。每个数据节点可以配置不同的serverPort。
**集群对外连接:** TDengine集群可以容纳单个、多个甚至几千个数据节点。应用只需要向集群中任何一个数据节点发起连接即可,连接需要提供的网络参数是一数据节点的End Point(FQDN加配置的端口号)。通过命令行CLI启动应用taos时,可以通过选项-h来指定数据节点的FQDN, -P来指定其配置的端口号,如果端口不配置,将采用TDengine的系统配置参数serverPort。 **集群对外连接:** TDengine集群可以容纳单个、多个甚至几千个数据节点。应用只需要向集群中任何一个数据节点发起连接即可,连接需要提供的网络参数是一数据节点的End Point(FQDN加配置的端口号)。通过命令行CLI启动应用taos时,可以通过选项-h来指定数据节点的FQDN, -P来指定其配置的端口号,如果端口不配置,将采用TDengine的系统配置参数serverPort。
**集群内部通讯**: 各个数据节点之间通过TCP/UDP进行连接。一个数据节点启动时,将获取mnode所在的dnode的EP信息,然后与系统中的mnode建立起连接,交换信息。获取mnode的EP信息有三步,1:检查mnodeEpList文件是否存在,如果不存在或不能正常打开获得mnode EP信息,进入第二步;2:检查系统配置文件taos.cfg, 获取节点配置参数firstEp, secondEp,(这两个参数指定的节点可以是不带mnode的普通节点,这样的话,节点被连接时会尝试重定向到mnode节点)如果不存在或者taos.cfg里没有这两个配置参数,或无效,进入第三步;3:将自己的EP设为mnode EP, 并独立运行起来。获取mnode EP列表后,数据节点发起连接,如果连接成功,则成功加入进工作的集群,如果不成功,则尝试mnode EP列表中的下一个。如果都尝试了,但连接都仍然失败,则休眠几秒后,再进行尝试。 **集群内部通讯**: 各个数据节点之间通过TCP/UDP进行连接。一个数据节点启动时,将获取mnode所在的dnode的EP信息,然后与系统中的mnode建立起连接,交换信息。获取mnode的EP信息有三步,1:检查mnodeEpSet文件是否存在,如果不存在或不能正常打开获得mnode EP信息,进入第二步;2:检查系统配置文件taos.cfg, 获取节点配置参数firstEp, secondEp,(这两个参数指定的节点可以是不带mnode的普通节点,这样的话,节点被连接时会尝试重定向到mnode节点)如果不存在或者taos.cfg里没有这两个配置参数,或无效,进入第三步;3:将自己的EP设为mnode EP, 并独立运行起来。获取mnode EP列表后,数据节点发起连接,如果连接成功,则成功加入进工作的集群,如果不成功,则尝试mnode EP列表中的下一个。如果都尝试了,但连接都仍然失败,则休眠几秒后,再进行尝试。
**MNODE的选择:** TDengine逻辑上有管理节点,但没有单独的执行代码,服务器侧只有一套执行代码taosd。那么哪个数据节点会是管理节点呢?这是系统自动决定的,无需任何人工干预。原则如下:一个数据节点启动时,会检查自己的End Point, 并与获取的mnode EP List进行比对,如果在其中,该数据节点认为自己应该启动mnode模块,成为mnode。如果自己的EP不在mnode EP List里,则不启动mnode模块。在系统的运行过程中,由于负载均衡、宕机等原因,mnode有可能迁移至新的dnode,但一切都是透明的,无需人工干预,配置参数的修改,是mnode自己根据资源做出的决定。 **MNODE的选择:** TDengine逻辑上有管理节点,但没有单独的执行代码,服务器侧只有一套执行代码taosd。那么哪个数据节点会是管理节点呢?这是系统自动决定的,无需任何人工干预。原则如下:一个数据节点启动时,会检查自己的End Point, 并与获取的mnode EP List进行比对,如果在其中,该数据节点认为自己应该启动mnode模块,成为mnode。如果自己的EP不在mnode EP List里,则不启动mnode模块。在系统的运行过程中,由于负载均衡、宕机等原因,mnode有可能迁移至新的dnode,但一切都是透明的,无需人工干预,配置参数的修改,是mnode自己根据资源做出的决定。
......
...@@ -209,7 +209,7 @@ C/C++的API类似于MySQL的C API。应用程序使用时,需要包含TDengine ...@@ -209,7 +209,7 @@ C/C++的API类似于MySQL的C API。应用程序使用时,需要包含TDengine
- `TAOS_RES* taos_query(TAOS *taos, const char *sql)` - `TAOS_RES* taos_query(TAOS *taos, const char *sql)`
该API用来执行SQL语句,可以是DQL、DML或DDL语句。 其中的`taos`参数是通过`taos_connect`获得的指针。返回值 NULL 表示失败 该API用来执行SQL语句,可以是DQL、DML或DDL语句。 其中的`taos`参数是通过`taos_connect`获得的指针。不能通过返回值是否是 NULL 来判断执行结果是否失败,而是需要用`taos_errno`函数解析结果集中的错误代码来进行判断
- `int taos_result_precision(TAOS_RES *res)` - `int taos_result_precision(TAOS_RES *res)`
...@@ -591,7 +591,8 @@ curl -u username:password -d '<SQL>' <ip>:<PORT>/rest/sql ...@@ -591,7 +591,8 @@ curl -u username:password -d '<SQL>' <ip>:<PORT>/rest/sql
```json ```json
{ {
"status": "succ", "status": "succ",
"head": ["Time Stamp","current", ], "head": ["ts","current", ],
"column_meta": [["ts",9,8],["current",6,4], ],
"data": [ "data": [
["2018-10-03 14:38:05.000", 10.3, ], ["2018-10-03 14:38:05.000", 10.3, ],
["2018-10-03 14:38:15.000", 12.6, ] ["2018-10-03 14:38:15.000", 12.6, ]
...@@ -602,10 +603,23 @@ curl -u username:password -d '<SQL>' <ip>:<PORT>/rest/sql ...@@ -602,10 +603,23 @@ curl -u username:password -d '<SQL>' <ip>:<PORT>/rest/sql
说明: 说明:
- status: 告知操作结果是成功还是失败 - status: 告知操作结果是成功还是失败。
- head: 表的定义,如果不返回结果集,仅有一列“affected_rows” - head: 表的定义,如果不返回结果集,则仅有一列“affected_rows”。(从 2.0.17 版本开始,建议不要依赖 head 返回值来判断数据列类型,而推荐使用 column_meta。在未来版本中,有可能会从返回值中去掉 head 这一项。)
- data: 具体返回的数据,一排一排的呈现,如果不返回结果集,仅[[affected_rows]] - column_meta: 从 2.0.17 版本开始,返回值中增加这一项来说明 data 里每一列的数据类型。具体每个列会用三个值来说明,分别为:列名、列类型、类型长度。例如`["current",6,4]`表示列名为“current”;列类型为 6,也即 float 类型;类型长度为 4,也即对应 4 个字节表示的 float。如果列类型为 binary 或 nchar,则类型长度表示该列最多可以保存的内容长度,而不是本次返回值中的具体数据长度。当列类型是 nchar 的时候,其类型长度表示可以保存的 unicode 字符数量,而不是 bytes。
- rows: 表明总共多少行数据 - data: 具体返回的数据,一行一行的呈现,如果不返回结果集,那么就仅有[[affected_rows]]。data 中每一行的数据列顺序,与 column_meta 中描述数据列的顺序完全一致。
- rows: 表明总共多少行数据。
column_meta 中的列类型说明:
* 1:BOOL
* 2:TINYINT
* 3:SMALLINT
* 4:INT
* 5:BIGINT
* 6:FLOAT
* 7:DOUBLE
* 8:BINARY
* 9:TIMESTAMP
* 10:NCHAR
### 自定义授权码 ### 自定义授权码
...@@ -651,7 +665,8 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001 ...@@ -651,7 +665,8 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001
```json ```json
{ {
"status": "succ", "status": "succ",
"head": ["Time Stamp","current","voltage","phase"], "head": ["ts","current","voltage","phase"],
"column_meta": [["ts",9,8],["current",6,4],["voltage",4,4],["phase",6,4]],
"data": [ "data": [
["2018-10-03 14:38:05.000",10.3,219,0.31], ["2018-10-03 14:38:05.000",10.3,219,0.31],
["2018-10-03 14:38:15.000",12.6,218,0.33] ["2018-10-03 14:38:15.000",12.6,218,0.33]
...@@ -671,8 +686,9 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'create database demo' 19 ...@@ -671,8 +686,9 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'create database demo' 19
{ {
"status": "succ", "status": "succ",
"head": ["affected_rows"], "head": ["affected_rows"],
"column_meta": [["affected_rows",4,4]],
"data": [[1]], "data": [[1]],
"rows": 1, "rows": 1
} }
``` ```
...@@ -691,7 +707,8 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001 ...@@ -691,7 +707,8 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001
```json ```json
{ {
"status": "succ", "status": "succ",
"head": ["column1","column2","column3"], "head": ["ts","current","voltage","phase"],
"column_meta": [["ts",9,8],["current",6,4],["voltage",4,4],["phase",6,4]],
"data": [ "data": [
[1538548685000,10.3,219,0.31], [1538548685000,10.3,219,0.31],
[1538548695000,12.6,218,0.33] [1538548695000,12.6,218,0.33]
...@@ -712,7 +729,8 @@ HTTP请求URL采用`sqlutc`时,返回结果集的时间戳将采用UTC时间 ...@@ -712,7 +729,8 @@ HTTP请求URL采用`sqlutc`时,返回结果集的时间戳将采用UTC时间
```json ```json
{ {
"status": "succ", "status": "succ",
"head": ["column1","column2","column3"], "head": ["ts","current","voltage","phase"],
"column_meta": [["ts",9,8],["current",6,4],["voltage",4,4],["phase",6,4]],
"data": [ "data": [
["2018-10-03T14:38:05.000+0800",10.3,219,0.31], ["2018-10-03T14:38:05.000+0800",10.3,219,0.31],
["2018-10-03T14:38:15.000+0800",12.6,218,0.33] ["2018-10-03T14:38:15.000+0800",12.6,218,0.33]
......
...@@ -155,11 +155,3 @@ TDengine客户端暂不支持如下函数: ...@@ -155,11 +155,3 @@ TDengine客户端暂不支持如下函数:
- dbExistsTable(conn, "test"):是否存在表test - dbExistsTable(conn, "test"):是否存在表test
- dbListTables(conn):显示连接中的所有表 - dbListTables(conn):显示连接中的所有表
## <a class="anchor" id="datax"></a>DataX
[DataX](https://github.com/alibaba/DataX) 是阿里巴巴集团开源的一款通用离线数据采集/同步工具,能够简单、高效地接入 TDengine 进行数据写入和读取。
* 数据读取集成的方法请参见 [TSDBReader 插件文档](https://github.com/alibaba/DataX/blob/master/tsdbreader/doc/tsdbreader.md)
* 数据写入集成的方法请参见 [TSDBWriter 插件文档](https://github.com/alibaba/DataX/blob/master/tsdbwriter/doc/tsdbhttpwriter.md)
...@@ -13,7 +13,7 @@ TDengine的集群管理极其简单,除添加和删除节点需要人工干预 ...@@ -13,7 +13,7 @@ TDengine的集群管理极其简单,除添加和删除节点需要人工干预
**第零步**:规划集群所有物理节点的FQDN,将规划好的FQDN分别添加到每个物理节点的/etc/hostname;修改每个物理节点的/etc/hosts,将所有集群物理节点的IP与FQDN的对应添加好。【如部署了DNS,请联系网络管理员在DNS上做好相关配置】 **第零步**:规划集群所有物理节点的FQDN,将规划好的FQDN分别添加到每个物理节点的/etc/hostname;修改每个物理节点的/etc/hosts,将所有集群物理节点的IP与FQDN的对应添加好。【如部署了DNS,请联系网络管理员在DNS上做好相关配置】
**第一步**:如果搭建集群的物理节点中,存有之前的测试数据、装过1.X的版本,或者装过其他版本的TDengine,请先将其删除,并清空所有数据,具体步骤请参考博客[《TDengine多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html ) **第一步**:如果搭建集群的物理节点中,存有之前的测试数据、装过1.X的版本,或者装过其他版本的TDengine,请先将其删除,并清空所有数据,具体步骤请参考博客[《TDengine多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html )
**注意1:**因为FQDN的信息会写进文件,如果之前没有配置或者更改FQDN,且启动了TDengine。请一定在确保数据无用或者备份的前提下,清理一下之前的数据(rm -rf /var/lib/taos/); **注意1:**因为FQDN的信息会写进文件,如果之前没有配置或者更改FQDN,且启动了TDengine。请一定在确保数据无用或者备份的前提下,清理一下之前的数据(`rm -rf /var/lib/taos/*`);
**注意2:**客户端也需要配置,确保它可以正确解析每个节点的FQDN配置,不管是通过DNS服务,还是 Host 文件。 **注意2:**客户端也需要配置,确保它可以正确解析每个节点的FQDN配置,不管是通过DNS服务,还是 Host 文件。
**第二步**:建议关闭所有物理节点的防火墙,至少保证端口:6030 - 6042的TCP和UDP端口都是开放的。**强烈建议**先关闭防火墙,集群搭建完毕之后,再来配置端口; **第二步**:建议关闭所有物理节点的防火墙,至少保证端口:6030 - 6042的TCP和UDP端口都是开放的。**强烈建议**先关闭防火墙,集群搭建完毕之后,再来配置端口;
......
...@@ -627,19 +627,21 @@ Query OK, 1 row(s) in set (0.001091s) ...@@ -627,19 +627,21 @@ Query OK, 1 row(s) in set (0.001091s)
### 支持的条件过滤操作 ### 支持的条件过滤操作
| Operation | Note | Applicable Data Types | | Operation | Note | Applicable Data Types |
| --------- | ----------------------------- | ------------------------------------- | | ----------- | ----------------------------- | ------------------------------------- |
| > | larger than | **`timestamp`** and all numeric types | | > | larger than | **`timestamp`** and all numeric types |
| < | smaller than | **`timestamp`** and all numeric types | | < | smaller than | **`timestamp`** and all numeric types |
| >= | larger than or equal to | **`timestamp`** and all numeric types | | >= | larger than or equal to | **`timestamp`** and all numeric types |
| <= | smaller than or equal to | **`timestamp`** and all numeric types | | <= | smaller than or equal to | **`timestamp`** and all numeric types |
| = | equal to | all types | | = | equal to | all types |
| <> | not equal to | all types | | <> | not equal to | all types |
| % | match with any char sequences | **`binary`** **`nchar`** | | between and | within a certain range | **`timestamp`** and all numeric types |
| _ | match with a single char | **`binary`** **`nchar`** | | % | match with any char sequences | **`binary`** **`nchar`** |
| _ | match with a single char | **`binary`** **`nchar`** |
1. 同时进行多个字段的范围过滤,需要使用关键词 AND 来连接不同的查询条件,暂不支持 OR 连接的不同列之间的查询过滤条件。 1. 同时进行多个字段的范围过滤,需要使用关键词 AND 来连接不同的查询条件,暂不支持 OR 连接的不同列之间的查询过滤条件。
2. 针对单一字段的过滤,如果是时间过滤条件,则一条语句中只支持设定一个;但针对其他的(普通)列或标签列,则可以使用``` OR``` 关键字进行组合条件的查询过滤。例如:((value > 20 and value < 30) OR (value < 12)) 。 2. 针对单一字段的过滤,如果是时间过滤条件,则一条语句中只支持设定一个;但针对其他的(普通)列或标签列,则可以使用 `OR` 关键字进行组合条件的查询过滤。例如:((value > 20 AND value < 30) OR (value < 12)) 。
3. 从 2.0.17 版本开始,条件过滤开始支持 BETWEEN AND 语法,例如 `WHERE col2 BETWEEN 1.5 AND 3.25` 表示查询条件为“1.5 ≤ col2 ≤ 3.25”。
### SQL 示例 ### SQL 示例
......
...@@ -13,9 +13,8 @@ WORKDIR /root/${dirName}/ ...@@ -13,9 +13,8 @@ WORKDIR /root/${dirName}/
RUN /bin/bash install.sh -e no RUN /bin/bash install.sh -e no
ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib" ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib"
ENV LANG=en_US.UTF-8 ENV LANG=C.UTF-8
ENV LANGUAGE=en_US:en ENV LC_ALL=C.UTF-8
ENV LC_ALL=en_US.UTF-8
EXPOSE 6030 6031 6032 6033 6034 6035 6036 6037 6038 6039 6040 6041 6042 EXPOSE 6030 6031 6032 6033 6034 6035 6036 6037 6038 6039 6040 6041 6042
CMD ["taosd"] CMD ["taosd"]
VOLUME [ "/var/lib/taos", "/var/log/taos","/etc/taos/" ] VOLUME [ "/var/lib/taos", "/var/log/taos","/etc/taos/" ]
...@@ -270,7 +270,7 @@ void tscPrintSelectClause(SSqlObj* pSql, int32_t subClauseIndex); ...@@ -270,7 +270,7 @@ void tscPrintSelectClause(SSqlObj* pSql, int32_t subClauseIndex);
bool hasMoreVnodesToTry(SSqlObj *pSql); bool hasMoreVnodesToTry(SSqlObj *pSql);
bool hasMoreClauseToTry(SSqlObj* pSql); bool hasMoreClauseToTry(SSqlObj* pSql);
void tscFreeQueryInfo(SSqlCmd* pCmd); void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeMeta);
void tscTryQueryNextVnode(SSqlObj *pSql, __async_cb_func_t fp); void tscTryQueryNextVnode(SSqlObj *pSql, __async_cb_func_t fp);
void tscAsyncQuerySingleRowForNextVnode(void *param, TAOS_RES *tres, int numOfRows); void tscAsyncQuerySingleRowForNextVnode(void *param, TAOS_RES *tres, int numOfRows);
......
...@@ -442,6 +442,8 @@ void tscCloseTscObj(void *pObj); ...@@ -442,6 +442,8 @@ void tscCloseTscObj(void *pObj);
TAOS *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int), TAOS *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int),
void *param, TAOS **taos); void *param, TAOS **taos);
TAOS_RES* taos_query_h(TAOS* taos, const char *sqlstr, int64_t* res); TAOS_RES* taos_query_h(TAOS* taos, const char *sqlstr, int64_t* res);
TAOS_RES * taos_query_ra(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *param);
void waitForQueryRsp(void *param, TAOS_RES *tres, int code); void waitForQueryRsp(void *param, TAOS_RES *tres, int code);
void doAsyncQuery(STscObj *pObj, SSqlObj *pSql, __async_cb_func_t fp, void *param, const char *sqlstr, size_t sqlLen); void doAsyncQuery(STscObj *pObj, SSqlObj *pSql, __async_cb_func_t fp, void *param, const char *sqlstr, size_t sqlLen);
......
...@@ -74,12 +74,16 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para ...@@ -74,12 +74,16 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para
// TODO return the correct error code to client in tscQueueAsyncError // TODO return the correct error code to client in tscQueueAsyncError
void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *param) { void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *param) {
taos_query_ra(taos, sqlstr, fp, param);
}
TAOS_RES * taos_query_ra(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *param) {
STscObj *pObj = (STscObj *)taos; STscObj *pObj = (STscObj *)taos;
if (pObj == NULL || pObj->signature != pObj) { if (pObj == NULL || pObj->signature != pObj) {
tscError("bug!!! pObj:%p", pObj); tscError("bug!!! pObj:%p", pObj);
terrno = TSDB_CODE_TSC_DISCONNECTED; terrno = TSDB_CODE_TSC_DISCONNECTED;
tscQueueAsyncError(fp, param, TSDB_CODE_TSC_DISCONNECTED); tscQueueAsyncError(fp, param, TSDB_CODE_TSC_DISCONNECTED);
return; return NULL;
} }
int32_t sqlLen = (int32_t)strlen(sqlstr); int32_t sqlLen = (int32_t)strlen(sqlstr);
...@@ -87,7 +91,7 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa ...@@ -87,7 +91,7 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa
tscError("sql string exceeds max length:%d", tsMaxSQLStringLen); tscError("sql string exceeds max length:%d", tsMaxSQLStringLen);
terrno = TSDB_CODE_TSC_EXCEED_SQL_LIMIT; terrno = TSDB_CODE_TSC_EXCEED_SQL_LIMIT;
tscQueueAsyncError(fp, param, terrno); tscQueueAsyncError(fp, param, terrno);
return; return NULL;
} }
nPrintTsc("%s", sqlstr); nPrintTsc("%s", sqlstr);
...@@ -96,12 +100,15 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa ...@@ -96,12 +100,15 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa
if (pSql == NULL) { if (pSql == NULL) {
tscError("failed to malloc sqlObj"); tscError("failed to malloc sqlObj");
tscQueueAsyncError(fp, param, TSDB_CODE_TSC_OUT_OF_MEMORY); tscQueueAsyncError(fp, param, TSDB_CODE_TSC_OUT_OF_MEMORY);
return; return NULL;
} }
doAsyncQuery(pObj, pSql, fp, param, sqlstr, sqlLen); doAsyncQuery(pObj, pSql, fp, param, sqlstr, sqlLen);
return pSql;
} }
static void tscAsyncFetchRowsProxy(void *param, TAOS_RES *tres, int numOfRows) { static void tscAsyncFetchRowsProxy(void *param, TAOS_RES *tres, int numOfRows) {
if (tres == NULL) { if (tres == NULL) {
return; return;
......
...@@ -2802,7 +2802,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) ...@@ -2802,7 +2802,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows)
numOfFailed += 1; numOfFailed += 1;
// clean up tableMeta in cache // clean up tableMeta in cache
tscFreeQueryInfo(&pSql->cmd); tscFreeQueryInfo(&pSql->cmd, false);
SQueryInfo* pQueryInfo = tscGetQueryInfoDetailSafely(&pSql->cmd, 0); SQueryInfo* pQueryInfo = tscGetQueryInfoDetailSafely(&pSql->cmd, 0);
STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, pSql->cmd.clauseIndex, 0); STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, pSql->cmd.clauseIndex, 0);
tscAddTableMetaInfo(pQueryInfo, &pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL); tscAddTableMetaInfo(pQueryInfo, &pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL);
......
...@@ -30,7 +30,7 @@ ...@@ -30,7 +30,7 @@
#include "ttokendef.h" #include "ttokendef.h"
static void freeQueryInfoImpl(SQueryInfo* pQueryInfo); static void freeQueryInfoImpl(SQueryInfo* pQueryInfo);
static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo); static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta);
static void tscStrToLower(char *str, int32_t n) { static void tscStrToLower(char *str, int32_t n) {
if (str == NULL || n <= 0) { return;} if (str == NULL || n <= 0) { return;}
...@@ -367,7 +367,7 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) { ...@@ -367,7 +367,7 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) {
pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free
} }
void tscFreeQueryInfo(SSqlCmd* pCmd) { void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeMeta) {
if (pCmd == NULL || pCmd->numOfClause == 0) { if (pCmd == NULL || pCmd->numOfClause == 0) {
return; return;
} }
...@@ -376,7 +376,7 @@ void tscFreeQueryInfo(SSqlCmd* pCmd) { ...@@ -376,7 +376,7 @@ void tscFreeQueryInfo(SSqlCmd* pCmd) {
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, i); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, i);
freeQueryInfoImpl(pQueryInfo); freeQueryInfoImpl(pQueryInfo);
clearAllTableMetaInfo(pQueryInfo); clearAllTableMetaInfo(pQueryInfo, removeMeta);
tfree(pQueryInfo); tfree(pQueryInfo);
} }
...@@ -404,7 +404,7 @@ void tscResetSqlCmd(SSqlCmd* pCmd, bool removeMeta) { ...@@ -404,7 +404,7 @@ void tscResetSqlCmd(SSqlCmd* pCmd, bool removeMeta) {
pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList, removeMeta); pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList, removeMeta);
pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks);
tscFreeQueryInfo(pCmd); tscFreeQueryInfo(pCmd, removeMeta);
} }
void tscFreeSqlResult(SSqlObj* pSql) { void tscFreeSqlResult(SSqlObj* pSql) {
...@@ -1847,10 +1847,17 @@ SArray* tscVgroupTableInfoDup(SArray* pVgroupTables) { ...@@ -1847,10 +1847,17 @@ SArray* tscVgroupTableInfoDup(SArray* pVgroupTables) {
return pa; return pa;
} }
void clearAllTableMetaInfo(SQueryInfo* pQueryInfo) { void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta) {
for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i);
if (removeMeta) {
char name[TSDB_TABLE_FNAME_LEN] = {0};
tNameExtractFullName(&pTableMetaInfo->name, name);
taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
}
tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables); tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables);
tscClearTableMetaInfo(pTableMetaInfo); tscClearTableMetaInfo(pTableMetaInfo);
free(pTableMetaInfo); free(pTableMetaInfo);
...@@ -2714,7 +2721,11 @@ STableMeta* createSuperTableMeta(STableMetaMsg* pChild) { ...@@ -2714,7 +2721,11 @@ STableMeta* createSuperTableMeta(STableMetaMsg* pChild) {
uint32_t tscGetTableMetaSize(STableMeta* pTableMeta) { uint32_t tscGetTableMetaSize(STableMeta* pTableMeta) {
assert(pTableMeta != NULL); assert(pTableMeta != NULL);
int32_t totalCols = pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; int32_t totalCols = 0;
if (pTableMeta->tableInfo.numOfColumns >= 0 && pTableMeta->tableInfo.numOfTags >= 0) {
totalCols = pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags;
}
return sizeof(STableMeta) + totalCols * sizeof(SSchema); return sizeof(STableMeta) + totalCols * sizeof(SSchema);
} }
......
...@@ -430,10 +430,10 @@ static void doInitGlobalConfig(void) { ...@@ -430,10 +430,10 @@ static void doInitGlobalConfig(void) {
// port // port
cfg.option = "serverPort"; cfg.option = "serverPort";
cfg.ptr = &tsServerPort; cfg.ptr = &tsServerPort;
cfg.valType = TAOS_CFG_VTYPE_INT16; cfg.valType = TAOS_CFG_VTYPE_UINT16;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW | TSDB_CFG_CTYPE_B_CLIENT; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 1; cfg.minValue = 1;
cfg.maxValue = 65535; cfg.maxValue = 65056;
cfg.ptrLength = 0; cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
......
...@@ -8,7 +8,7 @@ IF (TD_MVN_INSTALLED) ...@@ -8,7 +8,7 @@ IF (TD_MVN_INSTALLED)
ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME} ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME}
POST_BUILD POST_BUILD
COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.20-dist.jar ${LIBRARY_OUTPUT_PATH} COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.21-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMENT "build jdbc driver") COMMENT "build jdbc driver")
ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME}) ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME})
......
...@@ -5,7 +5,7 @@ ...@@ -5,7 +5,7 @@
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.20</version> <version>2.0.21</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>JDBCDriver</name> <name>JDBCDriver</name>
......
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.20</version> <version>2.0.21</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>JDBCDriver</name> <name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url> <url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
......
...@@ -308,7 +308,7 @@ public class DatabaseMetaDataResultSet implements ResultSet { ...@@ -308,7 +308,7 @@ public class DatabaseMetaDataResultSet implements ResultSet {
return colMetaData.getColIndex() + 1; return colMetaData.getColIndex() + 1;
} }
} }
throw new SQLException(TSDBConstants.INVALID_VARIABLES); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_VARIABLE);
} }
@Override @Override
......
...@@ -14,16 +14,13 @@ ...@@ -14,16 +14,13 @@
*****************************************************************************/ *****************************************************************************/
package com.taosdata.jdbc; package com.taosdata.jdbc;
import java.sql.SQLException;
import java.sql.Types;
import java.util.HashMap; import java.util.HashMap;
import java.util.Map; import java.util.Map;
public abstract class TSDBConstants { public abstract class TSDBConstants {
public static final String STATEMENT_CLOSED = "statement is closed";
public static final String UNSUPPORTED_METHOD_EXCEPTION_MSG = "this operation is NOT supported currently!";
public static final String INVALID_VARIABLES = "invalid variables";
public static final String RESULT_SET_IS_CLOSED = "resultSet is closed";
public static final String DEFAULT_PORT = "6200"; public static final String DEFAULT_PORT = "6200";
public static Map<Integer, String> DATATYPE_MAP = null; public static Map<Integer, String> DATATYPE_MAP = null;
...@@ -77,8 +74,65 @@ public abstract class TSDBConstants { ...@@ -77,8 +74,65 @@ public abstract class TSDBConstants {
return WrapErrMsg("unkown error!"); return WrapErrMsg("unkown error!");
} }
public static int taosType2JdbcType(int taosType) throws SQLException {
switch (taosType) {
case TSDBConstants.TSDB_DATA_TYPE_NULL:
return Types.NULL;
case TSDBConstants.TSDB_DATA_TYPE_BOOL:
return Types.BOOLEAN;
case TSDBConstants.TSDB_DATA_TYPE_TINYINT:
return Types.TINYINT;
case TSDBConstants.TSDB_DATA_TYPE_SMALLINT:
return Types.SMALLINT;
case TSDBConstants.TSDB_DATA_TYPE_INT:
return Types.INTEGER;
case TSDBConstants.TSDB_DATA_TYPE_BIGINT:
return Types.BIGINT;
case TSDBConstants.TSDB_DATA_TYPE_FLOAT:
return Types.FLOAT;
case TSDBConstants.TSDB_DATA_TYPE_DOUBLE:
return Types.DOUBLE;
case TSDBConstants.TSDB_DATA_TYPE_BINARY:
return Types.BINARY;
case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP:
return Types.TIMESTAMP;
case TSDBConstants.TSDB_DATA_TYPE_NCHAR:
return Types.NCHAR;
}
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN_SQL_TYPE_IN_TDENGINE);
}
public static int jdbcType2TaosType(int jdbcType) throws SQLException {
switch (jdbcType){
case Types.NULL:
return TSDBConstants.TSDB_DATA_TYPE_NULL;
case Types.BOOLEAN:
return TSDBConstants.TSDB_DATA_TYPE_BOOL;
case Types.TINYINT:
return TSDBConstants.TSDB_DATA_TYPE_TINYINT;
case Types.SMALLINT:
return TSDBConstants.TSDB_DATA_TYPE_SMALLINT;
case Types.INTEGER:
return TSDBConstants.TSDB_DATA_TYPE_INT;
case Types.BIGINT:
return TSDBConstants.TSDB_DATA_TYPE_BIGINT;
case Types.FLOAT:
return TSDBConstants.TSDB_DATA_TYPE_FLOAT;
case Types.DOUBLE:
return TSDBConstants.TSDB_DATA_TYPE_DOUBLE;
case Types.BINARY:
return TSDBConstants.TSDB_DATA_TYPE_BINARY;
case Types.TIMESTAMP:
return TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP;
case Types.NCHAR:
return TSDBConstants.TSDB_DATA_TYPE_NCHAR;
}
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN_SQL_TYPE_IN_TDENGINE);
}
static { static {
DATATYPE_MAP = new HashMap<>(); DATATYPE_MAP = new HashMap<>();
DATATYPE_MAP.put(0, "NULL");
DATATYPE_MAP.put(1, "BOOL"); DATATYPE_MAP.put(1, "BOOL");
DATATYPE_MAP.put(2, "TINYINT"); DATATYPE_MAP.put(2, "TINYINT");
DATATYPE_MAP.put(3, "SMALLINT"); DATATYPE_MAP.put(3, "SMALLINT");
...@@ -90,4 +144,8 @@ public abstract class TSDBConstants { ...@@ -90,4 +144,8 @@ public abstract class TSDBConstants {
DATATYPE_MAP.put(9, "TIMESTAMP"); DATATYPE_MAP.put(9, "TIMESTAMP");
DATATYPE_MAP.put(10, "NCHAR"); DATATYPE_MAP.put(10, "NCHAR");
} }
public static String jdbcType2TaosTypeName(int type) throws SQLException {
return DATATYPE_MAP.get(jdbcType2TaosType(type));
}
} }
...@@ -18,6 +18,7 @@ public class TSDBErrorNumbers { ...@@ -18,6 +18,7 @@ public class TSDBErrorNumbers {
public static final int ERROR_INVALID_FOR_EXECUTE = 0x230c; //not a valid sql for execute: (SQL) public static final int ERROR_INVALID_FOR_EXECUTE = 0x230c; //not a valid sql for execute: (SQL)
public static final int ERROR_PARAMETER_INDEX_OUT_RANGE = 0x230d; // parameter index out of range public static final int ERROR_PARAMETER_INDEX_OUT_RANGE = 0x230d; // parameter index out of range
public static final int ERROR_SQLCLIENT_EXCEPTION_ON_CONNECTION_CLOSED = 0x230e; // connection already closed public static final int ERROR_SQLCLIENT_EXCEPTION_ON_CONNECTION_CLOSED = 0x230e; // connection already closed
public static final int ERROR_UNKNOWN_SQL_TYPE_IN_TDENGINE = 0x230f; //unknown sql type in tdengine
public static final int ERROR_UNKNOWN = 0x2350; //unknown error public static final int ERROR_UNKNOWN = 0x2350; //unknown error
...@@ -49,6 +50,7 @@ public class TSDBErrorNumbers { ...@@ -49,6 +50,7 @@ public class TSDBErrorNumbers {
errorNumbers.add(ERROR_INVALID_FOR_EXECUTE); errorNumbers.add(ERROR_INVALID_FOR_EXECUTE);
errorNumbers.add(ERROR_PARAMETER_INDEX_OUT_RANGE); errorNumbers.add(ERROR_PARAMETER_INDEX_OUT_RANGE);
errorNumbers.add(ERROR_SQLCLIENT_EXCEPTION_ON_CONNECTION_CLOSED); errorNumbers.add(ERROR_SQLCLIENT_EXCEPTION_ON_CONNECTION_CLOSED);
errorNumbers.add(ERROR_UNKNOWN_SQL_TYPE_IN_TDENGINE);
/*****************************************************/ /*****************************************************/
errorNumbers.add(ERROR_SUBSCRIBE_FAILED); errorNumbers.add(ERROR_SUBSCRIBE_FAILED);
......
...@@ -20,7 +20,7 @@ import java.sql.Timestamp; ...@@ -20,7 +20,7 @@ import java.sql.Timestamp;
import java.sql.Types; import java.sql.Types;
import java.util.List; import java.util.List;
public class TSDBResultSetMetaData implements ResultSetMetaData { public class TSDBResultSetMetaData extends WrapperImpl implements ResultSetMetaData {
List<ColumnMetaData> colMetaDataList = null; List<ColumnMetaData> colMetaDataList = null;
...@@ -28,14 +28,6 @@ public class TSDBResultSetMetaData implements ResultSetMetaData { ...@@ -28,14 +28,6 @@ public class TSDBResultSetMetaData implements ResultSetMetaData {
this.colMetaDataList = metaDataList; this.colMetaDataList = metaDataList;
} }
public <T> T unwrap(Class<T> iface) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG);
}
public boolean isWrapperFor(Class<?> iface) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG);
}
public int getColumnCount() throws SQLException { public int getColumnCount() throws SQLException {
return colMetaDataList.size(); return colMetaDataList.size();
} }
...@@ -94,7 +86,7 @@ public class TSDBResultSetMetaData implements ResultSetMetaData { ...@@ -94,7 +86,7 @@ public class TSDBResultSetMetaData implements ResultSetMetaData {
} }
public String getSchemaName(int column) throws SQLException { public String getSchemaName(int column) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
public int getPrecision(int column) throws SQLException { public int getPrecision(int column) throws SQLException {
...@@ -125,18 +117,18 @@ public class TSDBResultSetMetaData implements ResultSetMetaData { ...@@ -125,18 +117,18 @@ public class TSDBResultSetMetaData implements ResultSetMetaData {
} }
public String getTableName(int column) throws SQLException { public String getTableName(int column) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
public String getCatalogName(int column) throws SQLException { public String getCatalogName(int column) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
public int getColumnType(int column) throws SQLException { public int getColumnType(int column) throws SQLException {
ColumnMetaData meta = this.colMetaDataList.get(column - 1); ColumnMetaData meta = this.colMetaDataList.get(column - 1);
switch (meta.getColType()) { switch (meta.getColType()) {
case TSDBConstants.TSDB_DATA_TYPE_BOOL: case TSDBConstants.TSDB_DATA_TYPE_BOOL:
return java.sql.Types.BIT; return Types.BOOLEAN;
case TSDBConstants.TSDB_DATA_TYPE_TINYINT: case TSDBConstants.TSDB_DATA_TYPE_TINYINT:
return java.sql.Types.TINYINT; return java.sql.Types.TINYINT;
case TSDBConstants.TSDB_DATA_TYPE_SMALLINT: case TSDBConstants.TSDB_DATA_TYPE_SMALLINT:
...@@ -150,13 +142,13 @@ public class TSDBResultSetMetaData implements ResultSetMetaData { ...@@ -150,13 +142,13 @@ public class TSDBResultSetMetaData implements ResultSetMetaData {
case TSDBConstants.TSDB_DATA_TYPE_DOUBLE: case TSDBConstants.TSDB_DATA_TYPE_DOUBLE:
return java.sql.Types.DOUBLE; return java.sql.Types.DOUBLE;
case TSDBConstants.TSDB_DATA_TYPE_BINARY: case TSDBConstants.TSDB_DATA_TYPE_BINARY:
return java.sql.Types.CHAR; return Types.BINARY;
case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP: case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP:
return java.sql.Types.BIGINT; return java.sql.Types.TIMESTAMP;
case TSDBConstants.TSDB_DATA_TYPE_NCHAR: case TSDBConstants.TSDB_DATA_TYPE_NCHAR:
return java.sql.Types.CHAR; return Types.NCHAR;
} }
throw new SQLException(TSDBConstants.INVALID_VARIABLES); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_VARIABLE);
} }
public String getColumnTypeName(int column) throws SQLException { public String getColumnTypeName(int column) throws SQLException {
...@@ -173,7 +165,7 @@ public class TSDBResultSetMetaData implements ResultSetMetaData { ...@@ -173,7 +165,7 @@ public class TSDBResultSetMetaData implements ResultSetMetaData {
} }
public boolean isDefinitelyWritable(int column) throws SQLException { public boolean isDefinitelyWritable(int column) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
public String getColumnClassName(int column) throws SQLException { public String getColumnClassName(int column) throws SQLException {
......
...@@ -1153,11 +1153,11 @@ public class TSDBResultSetWrapper implements ResultSet { ...@@ -1153,11 +1153,11 @@ public class TSDBResultSetWrapper implements ResultSet {
} }
public <T> T getObject(int columnIndex, Class<T> type) throws SQLException { public <T> T getObject(int columnIndex, Class<T> type) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
public <T> T getObject(String columnLabel, Class<T> type) throws SQLException { public <T> T getObject(String columnLabel, Class<T> type) throws SQLException {
throw new SQLException(TSDBConstants.UNSUPPORTED_METHOD_EXCEPTION_MSG); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_METHOD);
} }
@Override @Override
......
...@@ -14,12 +14,11 @@ ...@@ -14,12 +14,11 @@
*****************************************************************************/ *****************************************************************************/
package com.taosdata.jdbc; package com.taosdata.jdbc;
import javax.management.OperationsException;
import java.sql.SQLException; import java.sql.SQLException;
public class TSDBSubscribe { public class TSDBSubscribe {
private TSDBJNIConnector connecter = null; private final TSDBJNIConnector connecter;
private long id = 0; private final long id;
TSDBSubscribe(TSDBJNIConnector connecter, long id) throws SQLException { TSDBSubscribe(TSDBJNIConnector connecter, long id) throws SQLException {
if (null != connecter) { if (null != connecter) {
......
...@@ -18,10 +18,10 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -18,10 +18,10 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
private final String database; private final String database;
private final Statement statement; private final Statement statement;
// data // data
private ArrayList<ArrayList<Object>> resultSet = new ArrayList<>(); private ArrayList<ArrayList<Object>> resultSet;
// meta // meta
private ArrayList<String> columnNames = new ArrayList<>(); private ArrayList<String> columnNames;
private ArrayList<Field> columns = new ArrayList<>(); private ArrayList<Field> columns;
private RestfulResultSetMetaData metaData; private RestfulResultSetMetaData metaData;
/** /**
...@@ -29,11 +29,36 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -29,11 +29,36 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
* *
* @param resultJson: 包含data信息的结果集,有sql返回的结果集 * @param resultJson: 包含data信息的结果集,有sql返回的结果集
***/ ***/
public RestfulResultSet(String database, Statement statement, JSONObject resultJson) { public RestfulResultSet(String database, Statement statement, JSONObject resultJson) throws SQLException {
this.database = database; this.database = database;
this.statement = statement; this.statement = statement;
// column metadata
JSONArray columnMeta = resultJson.getJSONArray("column_meta");
columnNames = new ArrayList<>();
columns = new ArrayList<>();
for (int colIndex = 0; colIndex < columnMeta.size(); colIndex++) {
JSONArray col = columnMeta.getJSONArray(colIndex);
String col_name = col.getString(0);
int col_type = TSDBConstants.taosType2JdbcType(col.getInteger(1));
int col_length = col.getInteger(2);
columnNames.add(col_name);
columns.add(new Field(col_name, col_type, col_length, ""));
}
this.metaData = new RestfulResultSetMetaData(this.database, columns, this);
// row data // row data
JSONArray data = resultJson.getJSONArray("data"); JSONArray data = resultJson.getJSONArray("data");
resultSet = new ArrayList<>();
for (int rowIndex = 0; rowIndex < data.size(); rowIndex++) {
ArrayList row = new ArrayList();
JSONArray jsonRow = data.getJSONArray(rowIndex);
for (int colIndex = 0; colIndex < jsonRow.size(); colIndex++) {
row.add(parseColumnData(jsonRow, colIndex, columns.get(colIndex).type));
}
resultSet.add(row);
}
/*
int columnIndex = 0; int columnIndex = 0;
for (; columnIndex < data.size(); columnIndex++) { for (; columnIndex < data.size(); columnIndex++) {
ArrayList oneRow = new ArrayList<>(); ArrayList oneRow = new ArrayList<>();
...@@ -52,50 +77,77 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -52,50 +77,77 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
columns.add(new Field(name, "", 0, "")); columns.add(new Field(name, "", 0, ""));
} }
this.metaData = new RestfulResultSetMetaData(this.database, columns, this); this.metaData = new RestfulResultSetMetaData(this.database, columns, this);
} */
}
/**
* 由多个resultSet的JSON构造结果集 private Object parseColumnData(JSONArray row, int colIndex, int sqlType) {
* switch (sqlType) {
* @param resultJson: 包含data信息的结果集,有sql返回的结果集 case Types.NULL:
* @param fieldJson: 包含多个(最多2个)meta信息的结果集,有describe xxx return null;
**/ case Types.BOOLEAN:
public RestfulResultSet(String database, Statement statement, JSONObject resultJson, List<JSONObject> fieldJson) { return row.getBoolean(colIndex);
this(database, statement, resultJson); case Types.TINYINT:
ArrayList<Field> newColumns = new ArrayList<>(); case Types.SMALLINT:
return row.getShort(colIndex);
for (Field column : columns) { case Types.INTEGER:
Field field = findField(column.name, fieldJson); return row.getInteger(colIndex);
if (field != null) { case Types.BIGINT:
newColumns.add(field); return row.getBigInteger(colIndex);
} else { case Types.FLOAT:
newColumns.add(column); return row.getFloat(colIndex);
} case Types.DOUBLE:
} return row.getDouble(colIndex);
this.columns = newColumns; case Types.TIMESTAMP:
this.metaData = new RestfulResultSetMetaData(this.database, this.columns, this); return new Timestamp(row.getDate(colIndex).getTime());
} case Types.BINARY:
case Types.NCHAR:
public Field findField(String columnName, List<JSONObject> fieldJsonList) { default:
for (JSONObject fieldJSON : fieldJsonList) { return row.getString(colIndex);
JSONArray fieldDataJson = fieldJSON.getJSONArray("data");
for (int i = 0; i < fieldDataJson.size(); i++) {
JSONArray field = fieldDataJson.getJSONArray(i);
if (columnName.equalsIgnoreCase(field.getString(0))) {
return new Field(field.getString(0), field.getString(1), field.getInteger(2), field.getString(3));
}
}
} }
return null;
} }
// /**
// * 由多个resultSet的JSON构造结果集
// *
// * @param resultJson: 包含data信息的结果集,有sql返回的结果集
// * @param fieldJson: 包含多个(最多2个)meta信息的结果集,有describe xxx
// **/
// public RestfulResultSet(String database, Statement statement, JSONObject resultJson, List<JSONObject> fieldJson) throws SQLException {
// this(database, statement, resultJson);
// ArrayList<Field> newColumns = new ArrayList<>();
//
// for (Field column : columns) {
// Field field = findField(column.name, fieldJson);
// if (field != null) {
// newColumns.add(field);
// } else {
// newColumns.add(column);
// }
// }
// this.columns = newColumns;
// this.metaData = new RestfulResultSetMetaData(this.database, this.columns, this);
// }
// public Field findField(String columnName, List<JSONObject> fieldJsonList) {
// for (JSONObject fieldJSON : fieldJsonList) {
// JSONArray fieldDataJson = fieldJSON.getJSONArray("data");
// for (int i = 0; i < fieldDataJson.size(); i++) {
// JSONArray field = fieldDataJson.getJSONArray(i);
// if (columnName.equalsIgnoreCase(field.getString(0))) {
// return new Field(field.getString(0), field.getString(1), field.getInteger(2), field.getString(3));
// }
// }
// }
// return null;
// }
public class Field { public class Field {
String name; String name;
String type; int type;
int length; int length;
String note; String note;
public Field(String name, String type, int length, String note) { public Field(String name, int type, int length, String note) {
this.name = name; this.name = name;
this.type = type; this.type = type;
this.length = length; this.length = length;
......
...@@ -5,6 +5,7 @@ import com.taosdata.jdbc.TSDBConstants; ...@@ -5,6 +5,7 @@ import com.taosdata.jdbc.TSDBConstants;
import java.sql.ResultSetMetaData; import java.sql.ResultSetMetaData;
import java.sql.SQLException; import java.sql.SQLException;
import java.sql.Timestamp; import java.sql.Timestamp;
import java.sql.Types;
import java.util.ArrayList; import java.util.ArrayList;
public class RestfulResultSetMetaData implements ResultSetMetaData { public class RestfulResultSetMetaData implements ResultSetMetaData {
...@@ -53,14 +54,14 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -53,14 +54,14 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public boolean isSigned(int column) throws SQLException { public boolean isSigned(int column) throws SQLException {
String type = this.fields.get(column - 1).type.toUpperCase(); int type = this.fields.get(column - 1).type;
switch (type) { switch (type) {
case "TINYINT": case Types.TINYINT:
case "SMALLINT": case Types.SMALLINT:
case "INT": case Types.INTEGER:
case "BIGINT": case Types.BIGINT:
case "FLOAT": case Types.FLOAT:
case "DOUBLE": case Types.DOUBLE:
return true; return true;
default: default:
return false; return false;
...@@ -89,14 +90,14 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -89,14 +90,14 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public int getPrecision(int column) throws SQLException { public int getPrecision(int column) throws SQLException {
String type = this.fields.get(column - 1).type.toUpperCase(); int type = this.fields.get(column - 1).type;
switch (type) { switch (type) {
case "FLOAT": case Types.FLOAT:
return 5; return 5;
case "DOUBLE": case Types.DOUBLE:
return 9; return 9;
case "BINARY": case Types.BINARY:
case "NCHAR": case Types.NCHAR:
return this.fields.get(column - 1).length; return this.fields.get(column - 1).length;
default: default:
return 0; return 0;
...@@ -105,11 +106,11 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -105,11 +106,11 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public int getScale(int column) throws SQLException { public int getScale(int column) throws SQLException {
String type = this.fields.get(column - 1).type.toUpperCase(); int type = this.fields.get(column - 1).type;
switch (type) { switch (type) {
case "FLOAT": case Types.FLOAT:
return 5; return 5;
case "DOUBLE": case Types.DOUBLE:
return 9; return 9;
default: default:
return 0; return 0;
...@@ -128,36 +129,13 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -128,36 +129,13 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public int getColumnType(int column) throws SQLException { public int getColumnType(int column) throws SQLException {
String type = this.fields.get(column - 1).type.toUpperCase(); return this.fields.get(column - 1).type;
switch (type) {
case "BOOL":
return java.sql.Types.BOOLEAN;
case "TINYINT":
return java.sql.Types.TINYINT;
case "SMALLINT":
return java.sql.Types.SMALLINT;
case "INT":
return java.sql.Types.INTEGER;
case "BIGINT":
return java.sql.Types.BIGINT;
case "FLOAT":
return java.sql.Types.FLOAT;
case "DOUBLE":
return java.sql.Types.DOUBLE;
case "BINARY":
return java.sql.Types.BINARY;
case "TIMESTAMP":
return java.sql.Types.TIMESTAMP;
case "NCHAR":
return java.sql.Types.NCHAR;
}
throw new SQLException(TSDBConstants.INVALID_VARIABLES);
} }
@Override @Override
public String getColumnTypeName(int column) throws SQLException { public String getColumnTypeName(int column) throws SQLException {
String type = fields.get(column - 1).type; int type = fields.get(column - 1).type;
return type.toUpperCase(); return TSDBConstants.jdbcType2TaosTypeName(type);
} }
@Override @Override
...@@ -177,26 +155,26 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -177,26 +155,26 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public String getColumnClassName(int column) throws SQLException { public String getColumnClassName(int column) throws SQLException {
String type = this.fields.get(column - 1).type; int type = this.fields.get(column - 1).type;
String columnClassName = ""; String columnClassName = "";
switch (type) { switch (type) {
case "BOOL": case Types.BOOLEAN:
return Boolean.class.getName(); return Boolean.class.getName();
case "TINYINT": case Types.TINYINT:
case "SMALLINT": case Types.SMALLINT:
return Short.class.getName(); return Short.class.getName();
case "INT": case Types.INTEGER:
return Integer.class.getName(); return Integer.class.getName();
case "BIGINT": case Types.BIGINT:
return Long.class.getName(); return Long.class.getName();
case "FLOAT": case Types.FLOAT:
return Float.class.getName(); return Float.class.getName();
case "DOUBLE": case Types.DOUBLE:
return Double.class.getName(); return Double.class.getName();
case "TIMESTAMP": case Types.TIMESTAMP:
return Timestamp.class.getName(); return Timestamp.class.getName();
case "BINARY": case Types.BINARY:
case "NCHAR": case Types.NCHAR:
return String.class.getName(); return String.class.getName();
} }
return columnClassName; return columnClassName;
......
...@@ -151,22 +151,21 @@ public class RestfulStatement extends AbstractStatement { ...@@ -151,22 +151,21 @@ public class RestfulStatement extends AbstractStatement {
throw new SQLException(TSDBConstants.WrapErrMsg("SQL execution error: " + resultJson.getString("desc") + "\n" + "error code: " + resultJson.getString("code"))); throw new SQLException(TSDBConstants.WrapErrMsg("SQL execution error: " + resultJson.getString("desc") + "\n" + "error code: " + resultJson.getString("code")));
} }
// parse table name from sql // parse table name from sql
String[] tableIdentifiers = parseTableIdentifier(sql); // String[] tableIdentifiers = parseTableIdentifier(sql);
if (tableIdentifiers != null) { // if (tableIdentifiers != null) {
List<JSONObject> fieldJsonList = new ArrayList<>(); // List<JSONObject> fieldJsonList = new ArrayList<>();
for (String tableIdentifier : tableIdentifiers) { // for (String tableIdentifier : tableIdentifiers) {
// field meta // String fields = HttpClientPoolUtil.execute(url, "DESCRIBE " + tableIdentifier);
String fields = HttpClientPoolUtil.execute(url, "DESCRIBE " + tableIdentifier); // JSONObject fieldJson = JSON.parseObject(fields);
JSONObject fieldJson = JSON.parseObject(fields); // if (fieldJson.getString("status").equals("error")) {
if (fieldJson.getString("status").equals("error")) { // throw new SQLException(TSDBConstants.WrapErrMsg("SQL execution error: " + fieldJson.getString("desc") + "\n" + "error code: " + fieldJson.getString("code")));
throw new SQLException(TSDBConstants.WrapErrMsg("SQL execution error: " + fieldJson.getString("desc") + "\n" + "error code: " + fieldJson.getString("code"))); // }
} // fieldJsonList.add(fieldJson);
fieldJsonList.add(fieldJson); // }
} // this.resultSet = new RestfulResultSet(database, this, resultJson, fieldJsonList);
this.resultSet = new RestfulResultSet(database, this, resultJson, fieldJsonList); // } else {
} else { this.resultSet = new RestfulResultSet(database, this, resultJson);
this.resultSet = new RestfulResultSet(database, this, resultJson); // }
}
this.affectedRows = 0; this.affectedRows = 0;
return resultSet; return resultSet;
} }
...@@ -201,7 +200,7 @@ public class RestfulStatement extends AbstractStatement { ...@@ -201,7 +200,7 @@ public class RestfulStatement extends AbstractStatement {
@Override @Override
public ResultSet getResultSet() throws SQLException { public ResultSet getResultSet() throws SQLException {
if (isClosed()) if (isClosed())
throw new SQLException(TSDBConstants.STATEMENT_CLOSED); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_STATEMENT_CLOSED);
return resultSet; return resultSet;
} }
......
...@@ -13,7 +13,6 @@ import java.util.HashMap; ...@@ -13,7 +13,6 @@ import java.util.HashMap;
import java.util.Properties; import java.util.Properties;
import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public class ResultSetTest { public class ResultSetTest {
static Connection connection; static Connection connection;
......
...@@ -48,29 +48,28 @@ public class SubscribeTest { ...@@ -48,29 +48,28 @@ public class SubscribeTest {
@Test @Test
public void subscribe() { public void subscribe() {
try { try {
String rawSql = "select * from " + dbName + "." + tName + ";"; String rawSql = "select * from " + dbName + "." + tName + ";";
System.out.println(rawSql); System.out.println(rawSql);
TSDBSubscribe subscribe = ((TSDBConnection) connection).subscribe(topic, rawSql, false); // TSDBSubscribe subscribe = ((TSDBConnection) connection).subscribe(topic, rawSql, false);
int a = 0; // int a = 0;
while (true) { // while (true) {
TimeUnit.MILLISECONDS.sleep(1000); // TimeUnit.MILLISECONDS.sleep(1000);
TSDBResultSet resSet = subscribe.consume(); // TSDBResultSet resSet = subscribe.consume();
while (resSet.next()) { // while (resSet.next()) {
for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) { // for (int i = 1; i <= resSet.getMetaData().getColumnCount(); i++) {
System.out.printf(i + ": " + resSet.getString(i) + "\t"); // System.out.printf(i + ": " + resSet.getString(i) + "\t");
} // }
System.out.println("\n======" + a + "=========="); // System.out.println("\n======" + a + "==========");
} // }
a++; // a++;
if (a >= 2) { // if (a >= 2) {
break; // break;
} // }
// resSet.close(); // resSet.close();
} // }
//
subscribe.close(true); // subscribe.close(true);
} catch (Exception e) { } catch (Exception e) {
e.printStackTrace(); e.printStackTrace();
} }
......
...@@ -10,7 +10,7 @@ import java.util.Random; ...@@ -10,7 +10,7 @@ import java.util.Random;
public class RestfulJDBCTest { public class RestfulJDBCTest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
// private static final String host = "master"; // private static final String host = "master";
private static Connection connection; private static Connection connection;
private Random random = new Random(System.currentTimeMillis()); private Random random = new Random(System.currentTimeMillis());
......
...@@ -12,7 +12,7 @@ import java.sql.*; ...@@ -12,7 +12,7 @@ import java.sql.*;
@FixMethodOrder(MethodSorters.NAME_ASCENDING) @FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SQLTest { public class SQLTest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
// private static final String host = "master"; // private static final String host = "master";
private static Connection connection; private static Connection connection;
@Test @Test
...@@ -323,6 +323,18 @@ public class SQLTest { ...@@ -323,6 +323,18 @@ public class SQLTest {
SQLExecutor.executeQuery(connection, sql); SQLExecutor.executeQuery(connection, sql);
} }
@Test
public void testCase052() {
String sql = "select server_status()";
SQLExecutor.executeQuery(connection, sql);
}
@Test
public void testCase053() {
String sql = "select avg(cpu_taosd), avg(cpu_system), max(cpu_cores), avg(mem_taosd), avg(mem_system), max(mem_total), avg(disk_used), max(disk_total), avg(band_speed), avg(io_read), avg(io_write), sum(req_http), sum(req_select), sum(req_insert) from log.dn1 where ts> now - 60m and ts<= now interval(1m) fill(value, 0)";
SQLExecutor.executeQuery(connection, sql);
}
@BeforeClass @BeforeClass
public static void before() throws ClassNotFoundException, SQLException { public static void before() throws ClassNotFoundException, SQLException {
Class.forName("com.taosdata.jdbc.rs.RestfulDriver"); Class.forName("com.taosdata.jdbc.rs.RestfulDriver");
......
...@@ -29,7 +29,7 @@ typedef struct { ...@@ -29,7 +29,7 @@ typedef struct {
static SCheckItem tsCheckItem[TSDB_CHECK_ITEM_MAX] = {{0}}; static SCheckItem tsCheckItem[TSDB_CHECK_ITEM_MAX] = {{0}};
int64_t tsMinFreeMemSizeForStart = 0; int64_t tsMinFreeMemSizeForStart = 0;
static int32_t bindTcpPort(int16_t port) { static int32_t bindTcpPort(uint16_t port) {
SOCKET serverSocket; SOCKET serverSocket;
struct sockaddr_in server_addr; struct sockaddr_in server_addr;
...@@ -85,9 +85,9 @@ static int32_t bindUdpPort(int16_t port) { ...@@ -85,9 +85,9 @@ static int32_t bindUdpPort(int16_t port) {
static int32_t dnodeCheckNetwork() { static int32_t dnodeCheckNetwork() {
int32_t ret; int32_t ret;
int16_t startPort = tsServerPort; uint16_t startPort = tsServerPort;
for (int16_t port = startPort; port < startPort + 12; port++) { for (uint16_t port = startPort; port < startPort + 12; port++) {
ret = bindTcpPort(port); ret = bindTcpPort(port);
if (0 != ret) { if (0 != ret) {
dError("failed to tcp bind port %d, quit", port); dError("failed to tcp bind port %d, quit", port);
......
...@@ -286,7 +286,7 @@ do { \ ...@@ -286,7 +286,7 @@ do { \
#define TSDB_MAX_COMP_LEVEL 2 #define TSDB_MAX_COMP_LEVEL 2
#define TSDB_DEFAULT_COMP_LEVEL 2 #define TSDB_DEFAULT_COMP_LEVEL 2
#define TSDB_MIN_WAL_LEVEL 1 #define TSDB_MIN_WAL_LEVEL 0
#define TSDB_MAX_WAL_LEVEL 2 #define TSDB_MAX_WAL_LEVEL 2
#define TSDB_DEFAULT_WAL_LEVEL 1 #define TSDB_DEFAULT_WAL_LEVEL 1
......
...@@ -8,10 +8,12 @@ ...@@ -8,10 +8,12 @@
"thread_count": 4, "thread_count": 4,
"thread_count_create_tbl": 4, "thread_count_create_tbl": 4,
"result_file": "./insert_res.txt", "result_file": "./insert_res.txt",
"confirm_parameter_prompt": "no", "confirm_parameter_prompt": "no",
"insert_interval": 0,
"num_of_records_per_req": 100,
"databases": [{ "databases": [{
"dbinfo": { "dbinfo": {
"name": "dbx", "name": "db",
"drop": "yes", "drop": "yes",
"replica": 1, "replica": 1,
"days": 10, "days": 10,
...@@ -30,14 +32,13 @@ ...@@ -30,14 +32,13 @@
}, },
"super_tables": [{ "super_tables": [{
"name": "stb", "name": "stb",
"child_table_exists":"no", "child_table_exists":"no",
"childtable_count": 100, "childtable_count": 100,
"childtable_prefix": "stb_", "childtable_prefix": "stb_",
"auto_create_table": "no", "auto_create_table": "no",
"data_source": "rand", "data_source": "rand",
"insert_mode": "taosc", "insert_mode": "taosc",
"insert_rate": 0, "insert_rows": 100000,
"insert_rows": 1000,
"multi_thread_write_one_tbl": "no", "multi_thread_write_one_tbl": "no",
"number_of_tbl_in_one_sql": 0, "number_of_tbl_in_one_sql": 0,
"rows_per_tbl": 100, "rows_per_tbl": 100,
......
...@@ -61,56 +61,6 @@ ...@@ -61,56 +61,6 @@
#define REQ_EXTRA_BUF_LEN 1024 #define REQ_EXTRA_BUF_LEN 1024
#define RESP_BUF_LEN 4096 #define RESP_BUF_LEN 4096
#ifdef WINDOWS
#include <windows.h>
// Some old MinGW/CYGWIN distributions don't define this:
#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif
static HANDLE g_stdoutHandle;
static DWORD g_consoleMode;
void setupForAnsiEscape(void) {
DWORD mode = 0;
g_stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE);
if(g_stdoutHandle == INVALID_HANDLE_VALUE) {
exit(GetLastError());
}
if(!GetConsoleMode(g_stdoutHandle, &mode)) {
exit(GetLastError());
}
g_consoleMode = mode;
// Enable ANSI escape codes
mode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
if(!SetConsoleMode(g_stdoutHandle, mode)) {
exit(GetLastError());
}
}
void resetAfterAnsiEscape(void) {
// Reset colors
printf("\x1b[0m");
// Reset console mode
if(!SetConsoleMode(g_stdoutHandle, g_consoleMode)) {
exit(GetLastError());
}
}
#else
void setupForAnsiEscape(void) {}
void resetAfterAnsiEscape(void) {
// Reset colors
printf("\x1b[0m");
}
#endif
extern char configDir[]; extern char configDir[];
#define INSERT_JSON_NAME "insert.json" #define INSERT_JSON_NAME "insert.json"
...@@ -130,7 +80,6 @@ extern char configDir[]; ...@@ -130,7 +80,6 @@ extern char configDir[];
#define OPT_ABORT 1 /* –abort */ #define OPT_ABORT 1 /* –abort */
#define STRING_LEN 60000 #define STRING_LEN 60000
#define MAX_PREPARED_RAND 1000000 #define MAX_PREPARED_RAND 1000000
//#define MAX_SQL_SIZE 65536
#define MAX_FILE_NAME_LEN 256 #define MAX_FILE_NAME_LEN 256
#define MAX_SAMPLES_ONCE_FROM_FILE 10000 #define MAX_SAMPLES_ONCE_FROM_FILE 10000
...@@ -163,7 +112,7 @@ enum MODE { ...@@ -163,7 +112,7 @@ enum MODE {
ASYNC, ASYNC,
MODE_BUT MODE_BUT
}; };
enum QUERY_TYPE { enum QUERY_TYPE {
NO_INSERT_TYPE, NO_INSERT_TYPE,
INSERT_TYPE, INSERT_TYPE,
...@@ -222,6 +171,7 @@ typedef struct { ...@@ -222,6 +171,7 @@ typedef struct {
/* Used by main to communicate with parse_opt. */ /* Used by main to communicate with parse_opt. */
typedef struct SArguments_S { typedef struct SArguments_S {
char * metaFile; char * metaFile;
int test_mode;
char * host; char * host;
uint16_t port; uint16_t port;
char * user; char * user;
...@@ -233,12 +183,14 @@ typedef struct SArguments_S { ...@@ -233,12 +183,14 @@ typedef struct SArguments_S {
bool use_metric; bool use_metric;
bool insert_only; bool insert_only;
bool answer_yes; bool answer_yes;
bool debug_print;
char * output_file; char * output_file;
int mode; int mode;
char * datatype[MAX_NUM_DATATYPE + 1]; char * datatype[MAX_NUM_DATATYPE + 1];
int len_of_binary; int len_of_binary;
int num_of_CPR; int num_of_CPR;
int num_of_threads; int num_of_threads;
int insert_interval;
int num_of_RPR; int num_of_RPR;
int num_of_tables; int num_of_tables;
int num_of_DPT; int num_of_DPT;
...@@ -266,7 +218,6 @@ typedef struct SSuperTable_S { ...@@ -266,7 +218,6 @@ typedef struct SSuperTable_S {
char childTblPrefix[MAX_TB_NAME_SIZE]; char childTblPrefix[MAX_TB_NAME_SIZE];
char dataSource[MAX_TB_NAME_SIZE+1]; // rand_gen or sample char dataSource[MAX_TB_NAME_SIZE+1]; // rand_gen or sample
char insertMode[MAX_TB_NAME_SIZE]; // taosc, restful char insertMode[MAX_TB_NAME_SIZE]; // taosc, restful
int insertRate; // 0: unlimit > 0 rows/s
int multiThreadWriteOneTbl; // 0: no, 1: yes int multiThreadWriteOneTbl; // 0: no, 1: yes
int numberOfTblInOneSql; // 0/1: one table, > 1: number of tbl int numberOfTblInOneSql; // 0/1: one table, > 1: number of tbl
...@@ -288,7 +239,7 @@ typedef struct SSuperTable_S { ...@@ -288,7 +239,7 @@ typedef struct SSuperTable_S {
StrColumn tags[MAX_TAG_COUNT]; StrColumn tags[MAX_TAG_COUNT];
char* childTblName; char* childTblName;
char* colsOfCreatChildTable; char* colsOfCreateChildTable;
int lenOfOneRow; int lenOfOneRow;
int lenOfTagOfOneRow; int lenOfTagOfOneRow;
...@@ -431,7 +382,7 @@ typedef struct SThreadInfo_S { ...@@ -431,7 +382,7 @@ typedef struct SThreadInfo_S {
int start_table_id; int start_table_id;
int end_table_id; int end_table_id;
int data_of_rate; int data_of_rate;
int64_t start_time; uint64_t start_time;
char* cols; char* cols;
bool use_metric; bool use_metric;
SSuperTable* superTblInfo; SSuperTable* superTblInfo;
...@@ -439,10 +390,9 @@ typedef struct SThreadInfo_S { ...@@ -439,10 +390,9 @@ typedef struct SThreadInfo_S {
// for async insert // for async insert
tsem_t lock_sem; tsem_t lock_sem;
int64_t counter; int64_t counter;
int64_t st; uint64_t st;
int64_t et; uint64_t et;
int64_t lastTs; int64_t lastTs;
int nrecords_per_request;
// statistics // statistics
int64_t totalRowsInserted; int64_t totalRowsInserted;
...@@ -457,224 +407,56 @@ typedef struct SThreadInfo_S { ...@@ -457,224 +407,56 @@ typedef struct SThreadInfo_S {
} threadInfo; } threadInfo;
#ifdef WINDOWS
#include <windows.h>
// Some old MinGW/CYGWIN distributions don't define this:
#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif
#ifdef LINUX static HANDLE g_stdoutHandle;
/* The options we understand. */ static DWORD g_consoleMode;
static struct argp_option options[] = {
{0, 'f', "meta file", 0, "The meta data to the execution procedure, if use -f, all others options invalid. Default is NULL.", 0},
#ifdef _TD_POWER_
{0, 'c', "config_directory", 0, "Configuration directory. Default is '/etc/power/'.", 1},
{0, 'P', "password", 0, "The password to use when connecting to the server. Default is 'powerdb'.", 2},
#else
{0, 'c', "config_directory", 0, "Configuration directory. Default is '/etc/taos/'.", 1},
{0, 'P', "password", 0, "The password to use when connecting to the server. Default is 'taosdata'.", 2},
#endif
{0, 'h', "host", 0, "The host to connect to TDengine. Default is localhost.", 2},
{0, 'p', "port", 0, "The TCP/IP port number to use for the connection. Default is 0.", 2},
{0, 'u', "user", 0, "The TDengine user name to use when connecting to the server. Default is 'root'.", 2},
{0, 'd', "database", 0, "Destination database. Default is 'test'.", 3},
{0, 'a', "replica", 0, "Set the replica parameters of the database, Default 1, min: 1, max: 3.", 4},
{0, 'm', "table_prefix", 0, "Table prefix name. Default is 't'.", 4},
{0, 's', "sql file", 0, "The select sql file.", 6},
{0, 'M', 0, 0, "Use metric flag.", 4},
{0, 'o', "outputfile", 0, "Direct output to the named file. Default is './output.txt'.", 6},
{0, 'q', "query_mode", 0, "Query mode--0: SYNC, 1: ASYNC. Default is SYNC.", 4},
{0, 'b', "type_of_cols", 0, "The data_type of columns, default: TINYINT,SMALLINT,INT,BIGINT,FLOAT,DOUBLE,BINARY,NCHAR,BOOL,TIMESTAMP.", 4},
{0, 'w', "length_of_chartype", 0, "The length of data_type 'BINARY' or 'NCHAR'. Default is 16", 4},
{0, 'l', "num_of_cols_per_record", 0, "The number of columns per record. Default is 10.", 4},
{0, 'T', "num_of_threads", 0, "The number of threads. Default is 10.", 4},
// {0, 'r', "num_of_records_per_req", 0, "The number of records per request. Default is 100.", 4},
{0, 't', "num_of_tables", 0, "The number of tables. Default is 10000.", 4},
{0, 'n', "num_of_records_per_table", 0, "The number of records per table. Default is 10000.", 4},
{0, 'x', 0, 0, "Not insert only flag.", 4},
{0, 'y', 0, 0, "Default input yes for prompt.", 4},
{0, 'O', "disorderRatio", 0, "Insert mode--0: In order, > 0: disorder ratio. Default is in order.", 4},
{0, 'R', "disorderRang", 0, "Out of order data's range, ms, default is 1000.", 4},
//{0, 'D', "delete database", 0, "if elete database if exists. 0: no, 1: yes, default is 1", 5},
{0}};
/* Parse a single option. */
static error_t parse_opt(int key, char *arg, struct argp_state *state) {
// Get the input argument from argp_parse, which we know is a pointer to our arguments structure.
SArguments *arguments = state->input;
wordexp_t full_path;
char **sptr;
switch (key) {
case 'f':
arguments->metaFile = arg;
break;
case 'h':
arguments->host = arg;
break;
case 'p':
arguments->port = atoi(arg);
break;
case 'u':
arguments->user = arg;
break;
case 'P':
arguments->password = arg;
break;
case 'o':
arguments->output_file = arg;
break;
case 's':
arguments->sqlFile = arg;
break;
case 'q':
arguments->mode = atoi(arg);
break;
case 'T':
arguments->num_of_threads = atoi(arg);
break;
//case 'r':
// arguments->num_of_RPR = atoi(arg);
// break;
case 't':
arguments->num_of_tables = atoi(arg);
break;
case 'n':
arguments->num_of_DPT = atoi(arg);
break;
case 'd':
arguments->database = arg;
break;
case 'l':
arguments->num_of_CPR = atoi(arg);
break;
case 'b':
sptr = arguments->datatype;
if (strstr(arg, ",") == NULL) {
if (strcasecmp(arg, "INT") != 0 && strcasecmp(arg, "FLOAT") != 0 &&
strcasecmp(arg, "TINYINT") != 0 && strcasecmp(arg, "BOOL") != 0 &&
strcasecmp(arg, "SMALLINT") != 0 && strcasecmp(arg, "TIMESTAMP") != 0 &&
strcasecmp(arg, "BIGINT") != 0 && strcasecmp(arg, "DOUBLE") != 0 &&
strcasecmp(arg, "BINARY") != 0 && strcasecmp(arg, "NCHAR") != 0) {
argp_error(state, "Invalid data_type!");
}
sptr[0] = arg;
} else {
int index = 0;
char *dupstr = strdup(arg);
char *running = dupstr;
char *token = strsep(&running, ",");
while (token != NULL) {
if (strcasecmp(token, "INT") != 0 && strcasecmp(token, "FLOAT") != 0 &&
strcasecmp(token, "TINYINT") != 0 && strcasecmp(token, "BOOL") != 0 &&
strcasecmp(token, "SMALLINT") != 0 && strcasecmp(token, "TIMESTAMP") != 0 &&
strcasecmp(token, "BIGINT") != 0 && strcasecmp(token, "DOUBLE") != 0 &&
strcasecmp(token, "BINARY") != 0 && strcasecmp(token, "NCHAR") != 0) {
argp_error(state, "Invalid data_type!");
}
sptr[index++] = token;
token = strsep(&running, ",");
if (index >= MAX_NUM_DATATYPE) break;
}
}
break;
case 'w':
arguments->len_of_binary = atoi(arg);
break;
case 'm':
arguments->tb_prefix = arg;
break;
case 'M':
arguments->use_metric = true;
break;
case 'x':
arguments->insert_only = true;
break;
case 'y': void setupForAnsiEscape(void) {
arguments->answer_yes = true; DWORD mode = 0;
break; g_stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE);
case 'c':
if (wordexp(arg, &full_path, 0) != 0) {
fprintf(stderr, "Invalid path %s\n", arg);
return -1;
}
taos_options(TSDB_OPTION_CONFIGDIR, full_path.we_wordv[0]);
wordfree(&full_path);
break;
case 'O':
arguments->disorderRatio = atoi(arg);
if (arguments->disorderRatio < 0 || arguments->disorderRatio > 100)
{
argp_error(state, "Invalid disorder ratio, should 1 ~ 100!");
}
break;
case 'R':
arguments->disorderRange = atoi(arg);
break;
case 'a':
arguments->replica = atoi(arg);
if (arguments->replica > 3 || arguments->replica < 1)
{
arguments->replica = 1;
}
break;
//case 'D':
// arguments->method_of_delete = atoi(arg);
// break;
case OPT_ABORT:
arguments->abort = 1;
break;
case ARGP_KEY_ARG:
/*arguments->arg_list = &state->argv[state->next-1];
state->next = state->argc;*/
argp_usage(state);
break;
default: if(g_stdoutHandle == INVALID_HANDLE_VALUE) {
return ARGP_ERR_UNKNOWN; exit(GetLastError());
} }
return 0;
if(!GetConsoleMode(g_stdoutHandle, &mode)) {
exit(GetLastError());
}
g_consoleMode = mode;
// Enable ANSI escape codes
mode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
if(!SetConsoleMode(g_stdoutHandle, mode)) {
exit(GetLastError());
}
} }
static struct argp argp = {options, parse_opt, 0, 0}; void resetAfterAnsiEscape(void) {
// Reset colors
printf("\x1b[0m");
void parse_args(int argc, char *argv[], SArguments *arguments) { // Reset console mode
argp_parse(&argp, argc, argv, 0, 0, arguments); if(!SetConsoleMode(g_stdoutHandle, g_consoleMode)) {
if (arguments->abort) { exit(GetLastError());
#ifndef _ALPINE
error(10, 0, "ABORTED");
#else
abort();
#endif
} }
} }
#else #else
void printHelp() { void setupForAnsiEscape(void) {}
char indent[10] = " ";
printf("%s%s\n", indent, "-f"); void resetAfterAnsiEscape(void) {
printf("%s%s%s\n", indent, indent, "The meta file to the execution procedure. Default is './meta.json'."); // Reset colors
printf("%s%s\n", indent, "-c"); printf("\x1b[0m");
printf("%s%s%s\n", indent, indent, "config_directory, Configuration directory. Default is '/etc/taos/'."); }
}
void parse_args(int argc, char *argv[], SArguments *arguments) {
for (int i = 1; i < argc; i++) {
if (strcmp(argv[i], "-f") == 0) {
arguments->metaFile = argv[++i];
} else if (strcmp(argv[i], "-c") == 0) {
strcpy(configDir, argv[++i]);
} else if (strcmp(argv[i], "--help") == 0) {
printHelp();
exit(EXIT_FAILURE);
} else {
fprintf(stderr, "wrong options\n");
printHelp();
exit(EXIT_FAILURE);
}
}
}
#endif #endif
static bool getInfoFromJsonFile(char* file);
//static int generateOneRowDataForStb(SSuperTable* stbInfo);
//static int getDataIntoMemForStb(SSuperTable* stbInfo);
static void init_rand_data();
static int createDatabases(); static int createDatabases();
static void createChildTables(); static void createChildTables();
static int queryDbExec(TAOS *taos, char *command, int type); static int queryDbExec(TAOS *taos, char *command, int type);
...@@ -685,9 +467,12 @@ int32_t randint[MAX_PREPARED_RAND]; ...@@ -685,9 +467,12 @@ int32_t randint[MAX_PREPARED_RAND];
int64_t randbigint[MAX_PREPARED_RAND]; int64_t randbigint[MAX_PREPARED_RAND];
float randfloat[MAX_PREPARED_RAND]; float randfloat[MAX_PREPARED_RAND];
double randdouble[MAX_PREPARED_RAND]; double randdouble[MAX_PREPARED_RAND];
char *aggreFunc[] = {"*", "count(*)", "avg(col0)", "sum(col0)", "max(col0)", "min(col0)", "first(col0)", "last(col0)"}; char *aggreFunc[] = {"*", "count(*)", "avg(col0)", "sum(col0)",
"max(col0)", "min(col0)", "first(col0)", "last(col0)"};
SArguments g_args = {NULL, SArguments g_args = {
NULL, // metaFile
0, // test_mode
"127.0.0.1", // host "127.0.0.1", // host
6030, // port 6030, // port
"root", // user "root", // user
...@@ -701,7 +486,8 @@ SArguments g_args = {NULL, ...@@ -701,7 +486,8 @@ SArguments g_args = {NULL,
"t", // tb_prefix "t", // tb_prefix
NULL, // sqlFile NULL, // sqlFile
false, // use_metric false, // use_metric
false, // insert_only false, // insert_only
false, // debug_print
false, // answer_yes; false, // answer_yes;
"./output.txt", // output_file "./output.txt", // output_file
0, // mode : sync or async 0, // mode : sync or async
...@@ -720,6 +506,7 @@ SArguments g_args = {NULL, ...@@ -720,6 +506,7 @@ SArguments g_args = {NULL,
16, // len_of_binary 16, // len_of_binary
10, // num_of_CPR 10, // num_of_CPR
10, // num_of_connections/thread 10, // num_of_connections/thread
0, // insert_interval
100, // num_of_RPR 100, // num_of_RPR
10000, // num_of_tables 10000, // num_of_tables
10000, // num_of_DPT 10000, // num_of_DPT
...@@ -731,13 +518,261 @@ SArguments g_args = {NULL, ...@@ -731,13 +518,261 @@ SArguments g_args = {NULL,
}; };
static int g_jsonType = 0;
static SDbs g_Dbs; static SDbs g_Dbs;
static int g_totalChildTables = 0; static int g_totalChildTables = 0;
static SQueryMetaInfo g_queryInfo; static SQueryMetaInfo g_queryInfo;
static FILE * g_fpOfInsertResult = NULL; static FILE * g_fpOfInsertResult = NULL;
#define debugPrint(fmt, ...) \
do { if (g_args.debug_print) fprintf(stderr, fmt, __VA_ARGS__); } while(0)
///////////////////////////////////////////////////
void printHelp() {
char indent[10] = " ";
printf("%s%s%s%s\n", indent, "-f", indent,
"The meta file to the execution procedure. Default is './meta.json'.");
printf("%s%s%s%s\n", indent, "-u", indent,
"The TDengine user name to use when connecting to the server. Default is 'root'.");
#ifdef _TD_POWER_
printf("%s%s%s%s\n", indent, "-P", indent,
"The password to use when connecting to the server. Default is 'powerdb'.");
printf("%s%s%s%s\n", indent, "-c", indent,
"Configuration directory. Default is '/etc/power/'.");
#else
printf("%s%s%s%s\n", indent, "-P", indent,
"The password to use when connecting to the server. Default is 'taosdata'.");
printf("%s%s%s%s\n", indent, "-c", indent,
"Configuration directory. Default is '/etc/taos/'.");
#endif
printf("%s%s%s%s\n", indent, "-h", indent,
"The host to connect to TDengine. Default is localhost.");
printf("%s%s%s%s\n", indent, "-p", indent,
"The TCP/IP port number to use for the connection. Default is 0.");
printf("%s%s%s%s\n", indent, "-d", indent,
"Destination database. Default is 'test'.");
printf("%s%s%s%s\n", indent, "-a", indent,
"Set the replica parameters of the database, Default 1, min: 1, max: 3.");
printf("%s%s%s%s\n", indent, "-m", indent,
"Table prefix name. Default is 't'.");
printf("%s%s%s%s\n", indent, "-s", indent, "The select sql file.");
printf("%s%s%s%s\n", indent, "-M", indent, "Use metric flag.");
printf("%s%s%s%s\n", indent, "-o", indent,
"Direct output to the named file. Default is './output.txt'.");
printf("%s%s%s%s\n", indent, "-q", indent,
"Query mode--0: SYNC, 1: ASYNC. Default is SYNC.");
printf("%s%s%s%s\n", indent, "-b", indent,
"The data_type of columns, default: TINYINT,SMALLINT,INT,BIGINT,FLOAT,DOUBLE,BINARY,NCHAR,BOOL,TIMESTAMP.");
printf("%s%s%s%s\n", indent, "-w", indent,
"The length of data_type 'BINARY' or 'NCHAR'. Default is 16");
printf("%s%s%s%s\n", indent, "-l", indent,
"The number of columns per record. Default is 10.");
printf("%s%s%s%s\n", indent, "-T", indent,
"The number of threads. Default is 10.");
printf("%s%s%s%s\n", indent, "-i", indent,
"The sleep time (ms) between insertion. Default is 0.");
printf("%s%s%s%s\n", indent, "-r", indent,
"The number of records per request. Default is 100.");
printf("%s%s%s%s\n", indent, "-t", indent,
"The number of tables. Default is 10000.");
printf("%s%s%s%s\n", indent, "-n", indent,
"The number of records per table. Default is 10000.");
printf("%s%s%s%s\n", indent, "-x", indent, "Not insert only flag.");
printf("%s%s%s%s\n", indent, "-y", indent, "Default input yes for prompt.");
printf("%s%s%s%s\n", indent, "-O", indent,
"Insert mode--0: In order, > 0: disorder ratio. Default is in order.");
printf("%s%s%s%s\n", indent, "-R", indent,
"Out of order data's range, ms, default is 1000.");
printf("%s%s%s%s\n", indent, "-g", indent,
"Print debug info.");
/* printf("%s%s%s%s\n", indent, "-D", indent,
"if elete database if exists. 0: no, 1: yes, default is 1");
*/
}
void parse_args(int argc, char *argv[], SArguments *arguments) {
char **sptr;
wordexp_t full_path;
for (int i = 1; i < argc; i++) {
if (strcmp(argv[i], "-f") == 0) {
arguments->metaFile = argv[++i];
} else if (strcmp(argv[i], "-c") == 0) {
char *configPath = argv[++i];
if (wordexp(configPath, &full_path, 0) != 0) {
fprintf(stderr, "Invalid path %s\n", configPath);
return;
}
taos_options(TSDB_OPTION_CONFIGDIR, full_path.we_wordv[0]);
wordfree(&full_path);
} else if (strcmp(argv[i], "-h") == 0) {
arguments->host = argv[++i];
} else if (strcmp(argv[i], "-p") == 0) {
arguments->port = atoi(argv[++i]);
} else if (strcmp(argv[i], "-u") == 0) {
arguments->user = argv[++i];
} else if (strcmp(argv[i], "-P") == 0) {
arguments->password = argv[++i];
} else if (strcmp(argv[i], "-o") == 0) {
arguments->output_file = argv[++i];
} else if (strcmp(argv[i], "-s") == 0) {
arguments->sqlFile = argv[++i];
} else if (strcmp(argv[i], "-q") == 0) {
arguments->mode = atoi(argv[++i]);
} else if (strcmp(argv[i], "-T") == 0) {
arguments->num_of_threads = atoi(argv[++i]);
} else if (strcmp(argv[i], "-i") == 0) {
arguments->insert_interval = atoi(argv[++i]);
} else if (strcmp(argv[i], "-r") == 0) {
arguments->num_of_RPR = atoi(argv[++i]);
} else if (strcmp(argv[i], "-t") == 0) {
arguments->num_of_tables = atoi(argv[++i]);
} else if (strcmp(argv[i], "-n") == 0) {
arguments->num_of_DPT = atoi(argv[++i]);
} else if (strcmp(argv[i], "-d") == 0) {
arguments->database = argv[++i];
} else if (strcmp(argv[i], "-l") == 0) {
arguments->num_of_CPR = atoi(argv[++i]);
} else if (strcmp(argv[i], "-b") == 0) {
sptr = arguments->datatype;
++i;
if (strstr(argv[i], ",") == NULL) {
// only one col
if (strcasecmp(argv[i], "INT")
&& strcasecmp(argv[i], "FLOAT")
&& strcasecmp(argv[i], "TINYINT")
&& strcasecmp(argv[i], "BOOL")
&& strcasecmp(argv[i], "SMALLINT")
&& strcasecmp(argv[i], "BIGINT")
&& strcasecmp(argv[i], "DOUBLE")
&& strcasecmp(argv[i], "BINARY")
&& strcasecmp(argv[i], "NCHAR")) {
fprintf(stderr, "Invalid data_type!\n");
printHelp();
exit(EXIT_FAILURE);
}
sptr[0] = argv[i];
} else {
// more than one col
int index = 0;
char *dupstr = strdup(argv[i]);
char *running = dupstr;
char *token = strsep(&running, ",");
while (token != NULL) {
if (strcasecmp(token, "INT")
&& strcasecmp(token, "FLOAT")
&& strcasecmp(token, "TINYINT")
&& strcasecmp(token, "BOOL")
&& strcasecmp(token, "SMALLINT")
&& strcasecmp(token, "BIGINT")
&& strcasecmp(token, "DOUBLE")
&& strcasecmp(token, "BINARY")
&& strcasecmp(token, "NCHAR")) {
fprintf(stderr, "Invalid data_type!\n");
printHelp();
exit(EXIT_FAILURE);
}
sptr[index++] = token;
token = strsep(&running, ",");
if (index >= MAX_NUM_DATATYPE) break;
}
sptr[index] = NULL;
}
} else if (strcmp(argv[i], "-w") == 0) {
arguments->len_of_binary = atoi(argv[++i]);
} else if (strcmp(argv[i], "-m") == 0) {
arguments->tb_prefix = argv[++i];
} else if (strcmp(argv[i], "-M") == 0) {
arguments->use_metric = true;
} else if (strcmp(argv[i], "-x") == 0) {
arguments->insert_only = true;
} else if (strcmp(argv[i], "-y") == 0) {
arguments->answer_yes = true;
} else if (strcmp(argv[i], "-g") == 0) {
arguments->debug_print = true;
} else if (strcmp(argv[i], "-c") == 0) {
strcpy(configDir, argv[++i]);
} else if (strcmp(argv[i], "-O") == 0) {
arguments->disorderRatio = atoi(argv[++i]);
if (arguments->disorderRatio > 1
|| arguments->disorderRatio < 0) {
arguments->disorderRatio = 0;
} else if (arguments->disorderRatio == 1) {
arguments->disorderRange = 10;
}
} else if (strcmp(argv[i], "-R") == 0) {
arguments->disorderRange = atoi(argv[++i]);
if (arguments->disorderRange == 1
&& (arguments->disorderRange > 50
|| arguments->disorderRange <= 0)) {
arguments->disorderRange = 10;
}
} else if (strcmp(argv[i], "-a") == 0) {
arguments->replica = atoi(argv[++i]);
if (arguments->replica > 3 || arguments->replica < 1) {
arguments->replica = 1;
}
} else if (strcmp(argv[i], "-D") == 0) {
arguments->method_of_delete = atoi(argv[++i]);
if (arguments->method_of_delete < 0
|| arguments->method_of_delete > 3) {
arguments->method_of_delete = 0;
}
} else if (strcmp(argv[i], "--help") == 0) {
printHelp();
exit(0);
} else {
fprintf(stderr, "wrong options\n");
printHelp();
exit(EXIT_FAILURE);
}
}
if (arguments->debug_print) {
printf("###################################################################\n");
printf("# meta file: %s\n", arguments->metaFile);
printf("# Server IP: %s:%hu\n",
arguments->host == NULL ? "localhost" : arguments->host,
arguments->port );
printf("# User: %s\n", arguments->user);
printf("# Password: %s\n", arguments->password);
printf("# Use metric: %s\n", arguments->use_metric ? "true" : "false");
if (*(arguments->datatype)) {
printf("# Specified data type: ");
for (int i = 0; i < MAX_NUM_DATATYPE; i++)
if (arguments->datatype[i])
printf("%s,", arguments->datatype[i]);
else
break;
printf("\n");
}
printf("# Insertion interval: %d\n", arguments->insert_interval);
printf("# Number of Columns per record: %d\n", arguments->num_of_RPR);
printf("# Number of Threads: %d\n", arguments->num_of_threads);
printf("# Number of Tables: %d\n", arguments->num_of_tables);
printf("# Number of Data per Table: %d\n", arguments->num_of_DPT);
printf("# Database name: %s\n", arguments->database);
printf("# Table prefix: %s\n", arguments->tb_prefix);
if (arguments->disorderRatio) {
printf("# Data order: %d\n", arguments->disorderRatio);
printf("# Data out of order rate: %d\n", arguments->disorderRange);
}
printf("# Delete method: %d\n", arguments->method_of_delete);
printf("# Answer yes when prompt: %d\n", arguments->answer_yes);
printf("# Print debug info: %d\n", arguments->debug_print);
printf("###################################################################\n");
if (!arguments->answer_yes) {
printf("Press enter key to continue\n\n");
(void) getchar();
}
}
}
static bool getInfoFromJsonFile(char* file);
//static int generateOneRowDataForStb(SSuperTable* stbInfo);
//static int getDataIntoMemForStb(SSuperTable* stbInfo);
static void init_rand_data();
void tmfclose(FILE *fp) { void tmfclose(FILE *fp) {
if (NULL != fp) { if (NULL != fp) {
fclose(fp); fclose(fp);
...@@ -760,7 +795,7 @@ static int queryDbExec(TAOS *taos, char *command, int type) { ...@@ -760,7 +795,7 @@ static int queryDbExec(TAOS *taos, char *command, int type) {
taos_free_result(res); taos_free_result(res);
res = NULL; res = NULL;
} }
res = taos_query(taos, command); res = taos_query(taos, command);
code = taos_errno(res); code = taos_errno(res);
if (0 == code) { if (0 == code) {
...@@ -769,6 +804,7 @@ static int queryDbExec(TAOS *taos, char *command, int type) { ...@@ -769,6 +804,7 @@ static int queryDbExec(TAOS *taos, char *command, int type) {
} }
if (code != 0) { if (code != 0) {
debugPrint("DEBUG %s() LN%d - command: %s\n", __func__, __LINE__, command);
fprintf(stderr, "Failed to run %s, reason: %s\n", command, taos_errstr(res)); fprintf(stderr, "Failed to run %s, reason: %s\n", command, taos_errstr(res));
taos_free_result(res); taos_free_result(res);
//taos_close(taos); //taos_close(taos);
...@@ -925,14 +961,37 @@ static void init_rand_data() { ...@@ -925,14 +961,37 @@ static void init_rand_data() {
} }
} }
#define SHOW_PARSE_RESULT_START() \
do { if (g_args.metaFile) \
printf("\033[1m\033[40;32m================ %s parse result START ================\033[0m\n", \
g_args.metaFile); } while(0)
#define SHOW_PARSE_RESULT_END() \
do { if (g_args.metaFile) \
printf("\033[1m\033[40;32m================ %s parse result END================\033[0m\n", \
g_args.metaFile); } while(0)
#define SHOW_PARSE_RESULT_START_TO_FILE(fp) \
do { if (g_args.metaFile) \
fprintf(fp, "\033[1m\033[40;32m================ %s parse result START ================\033[0m\n", \
g_args.metaFile); } while(0)
#define SHOW_PARSE_RESULT_END_TO_FILE(fp) \
do { if (g_args.metaFile) \
fprintf(fp, "\033[1m\033[40;32m================ %s parse result END================\033[0m\n", \
g_args.metaFile); } while(0)
static int printfInsertMeta() { static int printfInsertMeta() {
printf("\033[1m\033[40;32m================ insert.json parse result START ================\033[0m\n"); SHOW_PARSE_RESULT_START();
printf("host: \033[33m%s:%u\033[0m\n", g_Dbs.host, g_Dbs.port); printf("host: \033[33m%s:%u\033[0m\n", g_Dbs.host, g_Dbs.port);
printf("user: \033[33m%s\033[0m\n", g_Dbs.user); printf("user: \033[33m%s\033[0m\n", g_Dbs.user);
printf("password: \033[33m%s\033[0m\n", g_Dbs.password); printf("password: \033[33m%s\033[0m\n", g_Dbs.password);
printf("resultFile: \033[33m%s\033[0m\n", g_Dbs.resultFile); printf("resultFile: \033[33m%s\033[0m\n", g_Dbs.resultFile);
printf("thread num of insert data: \033[33m%d\033[0m\n", g_Dbs.threadCount); printf("thread num of insert data: \033[33m%d\033[0m\n", g_Dbs.threadCount);
printf("thread num of create table: \033[33m%d\033[0m\n", g_Dbs.threadCountByCreateTbl); printf("thread num of create table: \033[33m%d\033[0m\n", g_Dbs.threadCountByCreateTbl);
printf("insert interval: \033[33m%d\033[0m\n", g_args.insert_interval);
printf("number of records per req: \033[33m%d\033[0m\n", g_args.num_of_RPR);
printf("database count: \033[33m%d\033[0m\n", g_Dbs.dbCount); printf("database count: \033[33m%d\033[0m\n", g_Dbs.dbCount);
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
...@@ -981,11 +1040,13 @@ static int printfInsertMeta() { ...@@ -981,11 +1040,13 @@ static int printfInsertMeta() {
printf(" quorum: \033[33m%d\033[0m\n", g_Dbs.db[i].dbCfg.quorum); printf(" quorum: \033[33m%d\033[0m\n", g_Dbs.db[i].dbCfg.quorum);
} }
if (g_Dbs.db[i].dbCfg.precision[0] != 0) { if (g_Dbs.db[i].dbCfg.precision[0] != 0) {
if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2)) || (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) { if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2))
|| (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) {
printf(" precision: \033[33m%s\033[0m\n", g_Dbs.db[i].dbCfg.precision); printf(" precision: \033[33m%s\033[0m\n", g_Dbs.db[i].dbCfg.precision);
} else { } else {
printf(" precision error: \033[33m%s\033[0m\n", g_Dbs.db[i].dbCfg.precision); printf("\033[1m\033[40;31m precision error: %s\033[0m\n",
return -1; g_Dbs.db[i].dbCfg.precision);
return -1;
} }
} }
...@@ -1015,7 +1076,6 @@ static int printfInsertMeta() { ...@@ -1015,7 +1076,6 @@ static int printfInsertMeta() {
printf(" childTblPrefix: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].childTblPrefix); printf(" childTblPrefix: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].childTblPrefix);
printf(" dataSource: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].dataSource); printf(" dataSource: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].dataSource);
printf(" insertMode: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].insertMode); printf(" insertMode: \033[33m%s\033[0m\n", g_Dbs.db[i].superTbls[j].insertMode);
printf(" insertRate: \033[33m%d\033[0m\n", g_Dbs.db[i].superTbls[j].insertRate);
printf(" insertRows: \033[33m%"PRId64"\033[0m\n", g_Dbs.db[i].superTbls[j].insertRows); printf(" insertRows: \033[33m%"PRId64"\033[0m\n", g_Dbs.db[i].superTbls[j].insertRows);
if (0 == g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl) { if (0 == g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl) {
...@@ -1038,34 +1098,43 @@ static int printfInsertMeta() { ...@@ -1038,34 +1098,43 @@ static int printfInsertMeta() {
printf(" columnCount: \033[33m%d\033[0m\n ", g_Dbs.db[i].superTbls[j].columnCount); printf(" columnCount: \033[33m%d\033[0m\n ", g_Dbs.db[i].superTbls[j].columnCount);
for (int k = 0; k < g_Dbs.db[i].superTbls[j].columnCount; k++) { for (int k = 0; k < g_Dbs.db[i].superTbls[j].columnCount; k++) {
//printf("dataType:%s, dataLen:%d\t", g_Dbs.db[i].superTbls[j].columns[k].dataType, g_Dbs.db[i].superTbls[j].columns[k].dataLen); //printf("dataType:%s, dataLen:%d\t", g_Dbs.db[i].superTbls[j].columns[k].dataType, g_Dbs.db[i].superTbls[j].columns[k].dataLen);
if ((0 == strncasecmp(g_Dbs.db[i].superTbls[j].columns[k].dataType, "binary", 6)) || (0 == strncasecmp(g_Dbs.db[i].superTbls[j].columns[k].dataType, "nchar", 5))) { if ((0 == strncasecmp(g_Dbs.db[i].superTbls[j].columns[k].dataType, "binary", 6))
printf("column[\033[33m%d\033[0m]:\033[33m%s(%d)\033[0m ", k, g_Dbs.db[i].superTbls[j].columns[k].dataType, g_Dbs.db[i].superTbls[j].columns[k].dataLen); || (0 == strncasecmp(g_Dbs.db[i].superTbls[j].columns[k].dataType, "nchar", 5))) {
printf("column[\033[33m%d\033[0m]:\033[33m%s(%d)\033[0m ", k,
g_Dbs.db[i].superTbls[j].columns[k].dataType, g_Dbs.db[i].superTbls[j].columns[k].dataLen);
} else { } else {
printf("column[%d]:\033[33m%s\033[0m ", k, g_Dbs.db[i].superTbls[j].columns[k].dataType); printf("column[%d]:\033[33m%s\033[0m ", k,
g_Dbs.db[i].superTbls[j].columns[k].dataType);
} }
} }
printf("\n"); printf("\n");
printf(" tagCount: \033[33m%d\033[0m\n ", g_Dbs.db[i].superTbls[j].tagCount); printf(" tagCount: \033[33m%d\033[0m\n ",
g_Dbs.db[i].superTbls[j].tagCount);
for (int k = 0; k < g_Dbs.db[i].superTbls[j].tagCount; k++) { for (int k = 0; k < g_Dbs.db[i].superTbls[j].tagCount; k++) {
//printf("dataType:%s, dataLen:%d\t", g_Dbs.db[i].superTbls[j].tags[k].dataType, g_Dbs.db[i].superTbls[j].tags[k].dataLen); //printf("dataType:%s, dataLen:%d\t", g_Dbs.db[i].superTbls[j].tags[k].dataType, g_Dbs.db[i].superTbls[j].tags[k].dataLen);
if ((0 == strncasecmp(g_Dbs.db[i].superTbls[j].tags[k].dataType, "binary", 6)) || (0 == strncasecmp(g_Dbs.db[i].superTbls[j].tags[k].dataType, "nchar", 5))) { if ((0 == strncasecmp(g_Dbs.db[i].superTbls[j].tags[k].dataType, "binary", 6))
printf("tag[%d]:\033[33m%s(%d)\033[0m ", k, g_Dbs.db[i].superTbls[j].tags[k].dataType, g_Dbs.db[i].superTbls[j].tags[k].dataLen); || (0 == strncasecmp(g_Dbs.db[i].superTbls[j].tags[k].dataType, "nchar", 5))) {
printf("tag[%d]:\033[33m%s(%d)\033[0m ", k,
g_Dbs.db[i].superTbls[j].tags[k].dataType, g_Dbs.db[i].superTbls[j].tags[k].dataLen);
} else { } else {
printf("tag[%d]:\033[33m%s\033[0m ", k, g_Dbs.db[i].superTbls[j].tags[k].dataType); printf("tag[%d]:\033[33m%s\033[0m ", k,
g_Dbs.db[i].superTbls[j].tags[k].dataType);
} }
} }
printf("\n"); printf("\n");
} }
printf("\n"); printf("\n");
} }
printf("\033[1m\033[40;32m================ insert.json parse result END================\033[0m\n");
SHOW_PARSE_RESULT_END();
return 0; return 0;
} }
static void printfInsertMetaToFile(FILE* fp) { static void printfInsertMetaToFile(FILE* fp) {
fprintf(fp, "================ insert.json parse result START================\n"); SHOW_PARSE_RESULT_START_TO_FILE(fp);
fprintf(fp, "host: %s:%u\n", g_Dbs.host, g_Dbs.port); fprintf(fp, "host: %s:%u\n", g_Dbs.host, g_Dbs.port);
fprintf(fp, "user: %s\n", g_Dbs.user); fprintf(fp, "user: %s\n", g_Dbs.user);
fprintf(fp, "password: %s\n", g_Dbs.password); fprintf(fp, "password: %s\n", g_Dbs.password);
...@@ -1120,7 +1189,8 @@ static void printfInsertMetaToFile(FILE* fp) { ...@@ -1120,7 +1189,8 @@ static void printfInsertMetaToFile(FILE* fp) {
fprintf(fp, " quorum: %d\n", g_Dbs.db[i].dbCfg.quorum); fprintf(fp, " quorum: %d\n", g_Dbs.db[i].dbCfg.quorum);
} }
if (g_Dbs.db[i].dbCfg.precision[0] != 0) { if (g_Dbs.db[i].dbCfg.precision[0] != 0) {
if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2)) || (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) { if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2))
|| (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) {
fprintf(fp, " precision: %s\n", g_Dbs.db[i].dbCfg.precision); fprintf(fp, " precision: %s\n", g_Dbs.db[i].dbCfg.precision);
} else { } else {
fprintf(fp, " precision error: %s\n", g_Dbs.db[i].dbCfg.precision); fprintf(fp, " precision error: %s\n", g_Dbs.db[i].dbCfg.precision);
...@@ -1153,7 +1223,6 @@ static void printfInsertMetaToFile(FILE* fp) { ...@@ -1153,7 +1223,6 @@ static void printfInsertMetaToFile(FILE* fp) {
fprintf(fp, " childTblPrefix: %s\n", g_Dbs.db[i].superTbls[j].childTblPrefix); fprintf(fp, " childTblPrefix: %s\n", g_Dbs.db[i].superTbls[j].childTblPrefix);
fprintf(fp, " dataSource: %s\n", g_Dbs.db[i].superTbls[j].dataSource); fprintf(fp, " dataSource: %s\n", g_Dbs.db[i].superTbls[j].dataSource);
fprintf(fp, " insertMode: %s\n", g_Dbs.db[i].superTbls[j].insertMode); fprintf(fp, " insertMode: %s\n", g_Dbs.db[i].superTbls[j].insertMode);
fprintf(fp, " insertRate: %d\n", g_Dbs.db[i].superTbls[j].insertRate);
fprintf(fp, " insertRows: %"PRId64"\n", g_Dbs.db[i].superTbls[j].insertRows); fprintf(fp, " insertRows: %"PRId64"\n", g_Dbs.db[i].superTbls[j].insertRows);
if (0 == g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl) { if (0 == g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl) {
...@@ -1197,11 +1266,11 @@ static void printfInsertMetaToFile(FILE* fp) { ...@@ -1197,11 +1266,11 @@ static void printfInsertMetaToFile(FILE* fp) {
} }
fprintf(fp, "\n"); fprintf(fp, "\n");
} }
fprintf(fp, "================ insert.json parse result END ================\n\n"); SHOW_PARSE_RESULT_END_TO_FILE(fp);
} }
static void printfQueryMeta() { static void printfQueryMeta() {
printf("\033[1m\033[40;32m================ query.json parse result ================\033[0m\n"); SHOW_PARSE_RESULT_START();
printf("host: \033[33m%s:%u\033[0m\n", g_queryInfo.host, g_queryInfo.port); printf("host: \033[33m%s:%u\033[0m\n", g_queryInfo.host, g_queryInfo.port);
printf("user: \033[33m%s\033[0m\n", g_queryInfo.user); printf("user: \033[33m%s\033[0m\n", g_queryInfo.user);
printf("password: \033[33m%s\033[0m\n", g_queryInfo.password); printf("password: \033[33m%s\033[0m\n", g_queryInfo.password);
...@@ -1213,14 +1282,13 @@ static void printfQueryMeta() { ...@@ -1213,14 +1282,13 @@ static void printfQueryMeta() {
printf("concurrent: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.concurrent); printf("concurrent: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.concurrent);
printf("sqlCount: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.sqlCount); printf("sqlCount: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.sqlCount);
if (SUBSCRIBE_MODE == g_jsonType) { if (SUBSCRIBE_MODE == g_args.test_mode) {
printf("mod: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeMode); printf("mod: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeMode);
printf("interval: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeInterval); printf("interval: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeInterval);
printf("restart: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeRestart); printf("restart: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeRestart);
printf("keepProgress: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeKeepProgress); printf("keepProgress: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.subscribeKeepProgress);
} }
for (int i = 0; i < g_queryInfo.superQueryInfo.sqlCount; i++) { for (int i = 0; i < g_queryInfo.superQueryInfo.sqlCount; i++) {
printf(" sql[%d]: \033[33m%s\033[0m\n", i, g_queryInfo.superQueryInfo.sql[i]); printf(" sql[%d]: \033[33m%s\033[0m\n", i, g_queryInfo.superQueryInfo.sql[i]);
} }
...@@ -1231,7 +1299,7 @@ static void printfQueryMeta() { ...@@ -1231,7 +1299,7 @@ static void printfQueryMeta() {
printf("childTblCount: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.childTblCount); printf("childTblCount: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.childTblCount);
printf("stable name: \033[33m%s\033[0m\n", g_queryInfo.subQueryInfo.sTblName); printf("stable name: \033[33m%s\033[0m\n", g_queryInfo.subQueryInfo.sTblName);
if (SUBSCRIBE_MODE == g_jsonType) { if (SUBSCRIBE_MODE == g_args.test_mode) {
printf("mod: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeMode); printf("mod: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeMode);
printf("interval: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeInterval); printf("interval: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeInterval);
printf("restart: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeRestart); printf("restart: \033[33m%d\033[0m\n", g_queryInfo.subQueryInfo.subscribeRestart);
...@@ -1243,7 +1311,8 @@ static void printfQueryMeta() { ...@@ -1243,7 +1311,8 @@ static void printfQueryMeta() {
printf(" sql[%d]: \033[33m%s\033[0m\n", i, g_queryInfo.subQueryInfo.sql[i]); printf(" sql[%d]: \033[33m%s\033[0m\n", i, g_queryInfo.subQueryInfo.sql[i]);
} }
printf("\n"); printf("\n");
printf("\033[1m\033[40;32m================ query.json parse result ================\033[0m\n");
SHOW_PARSE_RESULT_END();
} }
...@@ -1410,7 +1479,9 @@ static int getDbFromServer(TAOS * taos, SDbInfo** dbInfos) { ...@@ -1410,7 +1479,9 @@ static int getDbFromServer(TAOS * taos, SDbInfo** dbInfos) {
dbInfos[count]->comp = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_COMP_INDEX])); dbInfos[count]->comp = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_COMP_INDEX]));
dbInfos[count]->cachelast = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_CACHELAST_INDEX])); dbInfos[count]->cachelast = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_CACHELAST_INDEX]));
tstrncpy(dbInfos[count]->precision, (char *)row[TSDB_SHOW_DB_PRECISION_INDEX], fields[TSDB_SHOW_DB_PRECISION_INDEX].bytes); tstrncpy(dbInfos[count]->precision,
(char *)row[TSDB_SHOW_DB_PRECISION_INDEX],
fields[TSDB_SHOW_DB_PRECISION_INDEX].bytes);
dbInfos[count]->update = *((int8_t *)row[TSDB_SHOW_DB_UPDATE_INDEX]); dbInfos[count]->update = *((int8_t *)row[TSDB_SHOW_DB_UPDATE_INDEX]);
tstrncpy(dbInfos[count]->status, (char *)row[TSDB_SHOW_DB_STATUS_INDEX], fields[TSDB_SHOW_DB_STATUS_INDEX].bytes); tstrncpy(dbInfos[count]->status, (char *)row[TSDB_SHOW_DB_STATUS_INDEX], fields[TSDB_SHOW_DB_STATUS_INDEX].bytes);
...@@ -1861,10 +1932,14 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls, ...@@ -1861,10 +1932,14 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls,
char* dataType = superTbls->columns[colIndex].dataType; char* dataType = superTbls->columns[colIndex].dataType;
if (strcasecmp(dataType, "BINARY") == 0) { if (strcasecmp(dataType, "BINARY") == 0) {
len += snprintf(cols + len, STRING_LEN - len, ", col%d %s(%d)", colIndex, "BINARY", superTbls->columns[colIndex].dataLen); len += snprintf(cols + len, STRING_LEN - len,
", col%d %s(%d)", colIndex, "BINARY",
superTbls->columns[colIndex].dataLen);
lenOfOneRow += superTbls->columns[colIndex].dataLen + 3; lenOfOneRow += superTbls->columns[colIndex].dataLen + 3;
} else if (strcasecmp(dataType, "NCHAR") == 0) { } else if (strcasecmp(dataType, "NCHAR") == 0) {
len += snprintf(cols + len, STRING_LEN - len, ", col%d %s(%d)", colIndex, "NCHAR", superTbls->columns[colIndex].dataLen); len += snprintf(cols + len, STRING_LEN - len,
", col%d %s(%d)", colIndex, "NCHAR",
superTbls->columns[colIndex].dataLen);
lenOfOneRow += superTbls->columns[colIndex].dataLen + 3; lenOfOneRow += superTbls->columns[colIndex].dataLen + 3;
} else if (strcasecmp(dataType, "INT") == 0) { } else if (strcasecmp(dataType, "INT") == 0) {
len += snprintf(cols + len, STRING_LEN - len, ", col%d %s", colIndex, "INT"); len += snprintf(cols + len, STRING_LEN - len, ", col%d %s", colIndex, "INT");
...@@ -1901,13 +1976,14 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls, ...@@ -1901,13 +1976,14 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls,
//printf("%s.%s column count:%d, column length:%d\n\n", g_Dbs.db[i].dbName, g_Dbs.db[i].superTbls[j].sTblName, g_Dbs.db[i].superTbls[j].columnCount, lenOfOneRow); //printf("%s.%s column count:%d, column length:%d\n\n", g_Dbs.db[i].dbName, g_Dbs.db[i].superTbls[j].sTblName, g_Dbs.db[i].superTbls[j].columnCount, lenOfOneRow);
// save for creating child table // save for creating child table
superTbls->colsOfCreatChildTable = (char*)calloc(len+20, 1); superTbls->colsOfCreateChildTable = (char*)calloc(len+20, 1);
if (NULL == superTbls->colsOfCreatChildTable) { if (NULL == superTbls->colsOfCreateChildTable) {
printf("Failed when calloc, size:%d", len+1); printf("Failed when calloc, size:%d", len+1);
taos_close(taos); taos_close(taos);
exit(-1); exit(-1);
} }
snprintf(superTbls->colsOfCreatChildTable, len+20, "(ts timestamp%s)", cols); snprintf(superTbls->colsOfCreateChildTable, len+20, "(ts timestamp%s)", cols);
debugPrint("DEBUG - %s() LN%d: %s\n", __func__, __LINE__, superTbls->colsOfCreateChildTable);
if (use_metric) { if (use_metric) {
char tags[STRING_LEN] = "\0"; char tags[STRING_LEN] = "\0";
...@@ -1956,12 +2032,17 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls, ...@@ -1956,12 +2032,17 @@ static int createSuperTable(TAOS * taos, char* dbName, SSuperTable* superTbls,
len += snprintf(tags + len, STRING_LEN - len, ")"); len += snprintf(tags + len, STRING_LEN - len, ")");
superTbls->lenOfTagOfOneRow = lenOfTagOfOneRow; superTbls->lenOfTagOfOneRow = lenOfTagOfOneRow;
snprintf(command, BUFFER_SIZE, "create table if not exists %s.%s (ts timestamp%s) tags %s", dbName, superTbls->sTblName, cols, tags); snprintf(command, BUFFER_SIZE,
"create table if not exists %s.%s (ts timestamp%s) tags %s",
dbName, superTbls->sTblName, cols, tags);
debugPrint("DEBUG - %s() LN%d: %s\n", __func__, __LINE__, command);
if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) { if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) {
return -1; fprintf(stderr, "create supertable %s failed!\n\n", superTbls->sTblName);
return -1;
} }
printf("\ncreate supertable %s success!\n\n", superTbls->sTblName); debugPrint("DEBUG - create supertable %s success!\n\n", superTbls->sTblName);
} }
return 0; return 0;
} }
...@@ -1973,75 +2054,94 @@ static int createDatabases() { ...@@ -1973,75 +2054,94 @@ static int createDatabases() {
taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, NULL, g_Dbs.port); taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, NULL, g_Dbs.port);
if (taos == NULL) { if (taos == NULL) {
fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL)); fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL));
exit(-1); return -1;
} }
char command[BUFFER_SIZE] = "\0"; char command[BUFFER_SIZE] = "\0";
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
if (g_Dbs.db[i].drop) { if (g_Dbs.db[i].drop) {
sprintf(command, "drop database if exists %s;", g_Dbs.db[i].dbName); sprintf(command, "drop database if exists %s;", g_Dbs.db[i].dbName);
debugPrint("DEBUG %s() %d command: %s\n", __func__, __LINE__, command);
if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) { if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) {
taos_close(taos); taos_close(taos);
return -1; return -1;
} }
} }
int dataLen = 0; int dataLen = 0;
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "create database if not exists %s ", g_Dbs.db[i].dbName); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "create database if not exists %s ", g_Dbs.db[i].dbName);
if (g_Dbs.db[i].dbCfg.blocks > 0) { if (g_Dbs.db[i].dbCfg.blocks > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "blocks %d ", g_Dbs.db[i].dbCfg.blocks); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "blocks %d ", g_Dbs.db[i].dbCfg.blocks);
} }
if (g_Dbs.db[i].dbCfg.cache > 0) { if (g_Dbs.db[i].dbCfg.cache > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "cache %d ", g_Dbs.db[i].dbCfg.cache); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "cache %d ", g_Dbs.db[i].dbCfg.cache);
} }
if (g_Dbs.db[i].dbCfg.days > 0) { if (g_Dbs.db[i].dbCfg.days > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "days %d ", g_Dbs.db[i].dbCfg.days); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "days %d ", g_Dbs.db[i].dbCfg.days);
} }
if (g_Dbs.db[i].dbCfg.keep > 0) { if (g_Dbs.db[i].dbCfg.keep > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "keep %d ", g_Dbs.db[i].dbCfg.keep); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "keep %d ", g_Dbs.db[i].dbCfg.keep);
} }
if (g_Dbs.db[i].dbCfg.replica > 0) { if (g_Dbs.db[i].dbCfg.replica > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "replica %d ", g_Dbs.db[i].dbCfg.replica); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "replica %d ", g_Dbs.db[i].dbCfg.replica);
} }
if (g_Dbs.db[i].dbCfg.update > 0) { if (g_Dbs.db[i].dbCfg.update > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "update %d ", g_Dbs.db[i].dbCfg.update); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "update %d ", g_Dbs.db[i].dbCfg.update);
} }
//if (g_Dbs.db[i].dbCfg.maxtablesPerVnode > 0) { //if (g_Dbs.db[i].dbCfg.maxtablesPerVnode > 0) {
// dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "tables %d ", g_Dbs.db[i].dbCfg.maxtablesPerVnode); // dataLen += snprintf(command + dataLen,
// BUFFER_SIZE - dataLen, "tables %d ", g_Dbs.db[i].dbCfg.maxtablesPerVnode);
//} //}
if (g_Dbs.db[i].dbCfg.minRows > 0) { if (g_Dbs.db[i].dbCfg.minRows > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "minrows %d ", g_Dbs.db[i].dbCfg.minRows); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "minrows %d ", g_Dbs.db[i].dbCfg.minRows);
} }
if (g_Dbs.db[i].dbCfg.maxRows > 0) { if (g_Dbs.db[i].dbCfg.maxRows > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "maxrows %d ", g_Dbs.db[i].dbCfg.maxRows); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "maxrows %d ", g_Dbs.db[i].dbCfg.maxRows);
} }
if (g_Dbs.db[i].dbCfg.comp > 0) { if (g_Dbs.db[i].dbCfg.comp > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "comp %d ", g_Dbs.db[i].dbCfg.comp); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "comp %d ", g_Dbs.db[i].dbCfg.comp);
} }
if (g_Dbs.db[i].dbCfg.walLevel > 0) { if (g_Dbs.db[i].dbCfg.walLevel > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "wal %d ", g_Dbs.db[i].dbCfg.walLevel); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "wal %d ", g_Dbs.db[i].dbCfg.walLevel);
} }
if (g_Dbs.db[i].dbCfg.cacheLast > 0) { if (g_Dbs.db[i].dbCfg.cacheLast > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "cachelast %d ", g_Dbs.db[i].dbCfg.cacheLast); dataLen += snprintf(command + dataLen,
BUFFER_SIZE - dataLen, "cachelast %d ", g_Dbs.db[i].dbCfg.cacheLast);
} }
if (g_Dbs.db[i].dbCfg.fsync > 0) { if (g_Dbs.db[i].dbCfg.fsync > 0) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "fsync %d ", g_Dbs.db[i].dbCfg.fsync); dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "fsync %d ", g_Dbs.db[i].dbCfg.fsync);
} }
if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2)) || (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) { if ((0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "ms", 2))
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen, "precision \'%s\';", g_Dbs.db[i].dbCfg.precision); || (0 == strncasecmp(g_Dbs.db[i].dbCfg.precision, "us", 2))) {
dataLen += snprintf(command + dataLen, BUFFER_SIZE - dataLen,
"precision \'%s\';", g_Dbs.db[i].dbCfg.precision);
} }
debugPrint("DEBUG %s() %d command: %s\n", __func__, __LINE__, command);
if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) { if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) {
taos_close(taos); taos_close(taos);
printf("\ncreate database %s failed!\n\n", g_Dbs.db[i].dbName);
return -1; return -1;
} }
printf("\ncreate database %s success!\n\n", g_Dbs.db[i].dbName); printf("\ncreate database %s success!\n\n", g_Dbs.db[i].dbName);
debugPrint("DEBUG %s() %d count:%d\n", __func__, __LINE__, g_Dbs.db[i].superTblCount);
for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) {
// describe super table, if exists // describe super table, if exists
sprintf(command, "describe %s.%s;", g_Dbs.db[i].dbName, g_Dbs.db[i].superTbls[j].sTblName); sprintf(command, "describe %s.%s;", g_Dbs.db[i].dbName, g_Dbs.db[i].superTbls[j].sTblName);
debugPrint("DEBUG %s() %d command: %s\n", __func__, __LINE__, command);
if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) { if (0 != queryDbExec(taos, command, NO_INSERT_TYPE)) {
g_Dbs.db[i].superTbls[j].superTblExists = TBL_NO_EXISTS; g_Dbs.db[i].superTbls[j].superTblExists = TBL_NO_EXISTS;
ret = createSuperTable(taos, g_Dbs.db[i].dbName, &g_Dbs.db[i].superTbls[j], g_Dbs.use_metric); ret = createSuperTable(taos, g_Dbs.db[i].dbName, &g_Dbs.db[i].superTbls[j], g_Dbs.use_metric);
...@@ -2051,6 +2151,7 @@ static int createDatabases() { ...@@ -2051,6 +2151,7 @@ static int createDatabases() {
} }
if (0 != ret) { if (0 != ret) {
printf("\ncreate super table %d failed!\n\n", j);
taos_close(taos); taos_close(taos);
return -1; return -1;
} }
...@@ -2061,50 +2162,74 @@ static int createDatabases() { ...@@ -2061,50 +2162,74 @@ static int createDatabases() {
return 0; return 0;
} }
static void* createTable(void *sarg)
void * createTable(void *sarg)
{ {
threadInfo *winfo = (threadInfo *)sarg; threadInfo *winfo = (threadInfo *)sarg;
SSuperTable* superTblInfo = winfo->superTblInfo; SSuperTable* superTblInfo = winfo->superTblInfo;
int64_t lastPrintTime = taosGetTimestampMs(); int64_t lastPrintTime = taosGetTimestampMs();
char* buffer = calloc(superTblInfo->maxSqlLen, 1); int buff_len;
if (superTblInfo)
buff_len = superTblInfo->maxSqlLen;
else
buff_len = BUFFER_SIZE;
char *buffer = calloc(buff_len, 1);
if (buffer == NULL) {
fprintf(stderr, "Memory allocated failed!");
exit(-1);
}
int len = 0; int len = 0;
int batchNum = 0; int batchNum = 0;
//printf("Creating table from %d to %d\n", winfo->start_table_id, winfo->end_table_id); //printf("Creating table from %d to %d\n", winfo->start_table_id, winfo->end_table_id);
for (int i = winfo->start_table_id; i <= winfo->end_table_id; i++) { for (int i = winfo->start_table_id; i <= winfo->end_table_id; i++) {
if (0 == g_Dbs.use_metric) { if (0 == g_Dbs.use_metric) {
snprintf(buffer, BUFFER_SIZE, "create table if not exists %s.%s%d %s;", winfo->db_name, superTblInfo->childTblPrefix, i, superTblInfo->colsOfCreatChildTable); snprintf(buffer, buff_len,
"create table if not exists %s.%s%d %s;",
winfo->db_name,
g_args.tb_prefix, i,
winfo->cols);
} else { } else {
if (0 == len) { if (0 == len) {
batchNum = 0; batchNum = 0;
memset(buffer, 0, superTblInfo->maxSqlLen); memset(buffer, 0, buff_len);
len += snprintf(buffer + len, superTblInfo->maxSqlLen - len, "create table "); len += snprintf(buffer + len,
buff_len - len, "create table ");
} }
char* tagsValBuf = NULL; char* tagsValBuf = NULL;
if (0 == superTblInfo->tagSource) { if (0 == superTblInfo->tagSource) {
tagsValBuf = generateTagVaulesForStb(superTblInfo); tagsValBuf = generateTagVaulesForStb(superTblInfo);
} else { } else {
tagsValBuf = getTagValueFromTagSample(superTblInfo, i % superTblInfo->tagSampleCount); tagsValBuf = getTagValueFromTagSample(
superTblInfo,
i % superTblInfo->tagSampleCount);
} }
if (NULL == tagsValBuf) { if (NULL == tagsValBuf) {
free(buffer); free(buffer);
return NULL; return NULL;
} }
len += snprintf(buffer + len, superTblInfo->maxSqlLen - len, "if not exists %s.%s%d using %s.%s tags %s ", winfo->db_name, superTblInfo->childTblPrefix, i, winfo->db_name, superTblInfo->sTblName, tagsValBuf); len += snprintf(buffer + len,
superTblInfo->maxSqlLen - len,
"if not exists %s.%s%d using %s.%s tags %s ",
winfo->db_name, superTblInfo->childTblPrefix,
i, winfo->db_name,
superTblInfo->sTblName, tagsValBuf);
free(tagsValBuf); free(tagsValBuf);
batchNum++; batchNum++;
if ((batchNum < superTblInfo->batchCreateTableNum) && ((superTblInfo->maxSqlLen - len) >= (superTblInfo->lenOfTagOfOneRow + 256))) { if ((batchNum < superTblInfo->batchCreateTableNum)
&& ((superTblInfo->maxSqlLen - len)
>= (superTblInfo->lenOfTagOfOneRow + 256))) {
continue; continue;
} }
} }
len = 0; len = 0;
debugPrint("DEBUG %s() LN%d %s\n", __func__, __LINE__, buffer);
if (0 != queryDbExec(winfo->taos, buffer, NO_INSERT_TYPE)){ if (0 != queryDbExec(winfo->taos, buffer, NO_INSERT_TYPE)){
free(buffer); free(buffer);
return NULL; return NULL;
...@@ -2112,20 +2237,24 @@ void * createTable(void *sarg) ...@@ -2112,20 +2237,24 @@ void * createTable(void *sarg)
int64_t currentPrintTime = taosGetTimestampMs(); int64_t currentPrintTime = taosGetTimestampMs();
if (currentPrintTime - lastPrintTime > 30*1000) { if (currentPrintTime - lastPrintTime > 30*1000) {
printf("thread[%d] already create %d - %d tables\n", winfo->threadID, winfo->start_table_id, i); printf("thread[%d] already create %d - %d tables\n",
winfo->threadID, winfo->start_table_id, i);
lastPrintTime = currentPrintTime; lastPrintTime = currentPrintTime;
} }
} }
if (0 != len) { if (0 != len) {
debugPrint("DEBUG %s() %d buffer: %s\n", __func__, __LINE__, buffer);
(void)queryDbExec(winfo->taos, buffer, NO_INSERT_TYPE); (void)queryDbExec(winfo->taos, buffer, NO_INSERT_TYPE);
} }
free(buffer); free(buffer);
return NULL; return NULL;
} }
void startMultiThreadCreateChildTable(char* cols, int threads, int ntables, char* db_name, SSuperTable* superTblInfo) { int startMultiThreadCreateChildTable(
char* cols, int threads, int ntables,
char* db_name, SSuperTable* superTblInfo) {
pthread_t *pids = malloc(threads * sizeof(pthread_t)); pthread_t *pids = malloc(threads * sizeof(pthread_t));
threadInfo *infos = malloc(threads * sizeof(threadInfo)); threadInfo *infos = malloc(threads * sizeof(threadInfo));
...@@ -2146,14 +2275,23 @@ void startMultiThreadCreateChildTable(char* cols, int threads, int ntables, char ...@@ -2146,14 +2275,23 @@ void startMultiThreadCreateChildTable(char* cols, int threads, int ntables, char
int b = 0; int b = 0;
b = ntables % threads; b = ntables % threads;
int last = 0; int last = 0;
for (int i = 0; i < threads; i++) { for (int i = 0; i < threads; i++) {
threadInfo *t_info = infos + i; threadInfo *t_info = infos + i;
t_info->threadID = i; t_info->threadID = i;
tstrncpy(t_info->db_name, db_name, MAX_DB_NAME_SIZE); tstrncpy(t_info->db_name, db_name, MAX_DB_NAME_SIZE);
t_info->superTblInfo = superTblInfo; t_info->superTblInfo = superTblInfo;
t_info->taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, db_name, g_Dbs.port); t_info->taos = taos_connect(
g_Dbs.host,
g_Dbs.user,
g_Dbs.password,
db_name,
g_Dbs.port);
if (t_info->taos == NULL) {
fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL));
return -1;
}
t_info->start_table_id = last; t_info->start_table_id = last;
t_info->end_table_id = i < b ? last + a : last + a - 1; t_info->end_table_id = i < b ? last + a : last + a - 1;
last = t_info->end_table_id + 1; last = t_info->end_table_id + 1;
...@@ -2174,18 +2312,60 @@ void startMultiThreadCreateChildTable(char* cols, int threads, int ntables, char ...@@ -2174,18 +2312,60 @@ void startMultiThreadCreateChildTable(char* cols, int threads, int ntables, char
free(pids); free(pids);
free(infos); free(infos);
return 0;
} }
static void createChildTables() { static void createChildTables() {
char tblColsBuf[MAX_SQL_SIZE];
int len;
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { if (g_Dbs.db[i].superTblCount > 0) {
if ((AUTO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable) || (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) { // with super table
continue; for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) {
if ((AUTO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable)
|| (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) {
continue;
}
debugPrint("DEBUG - %s() LN%d: %s\n", __func__, __LINE__,
g_Dbs.db[i].superTbls[j].colsOfCreateChildTable);
startMultiThreadCreateChildTable(
g_Dbs.db[i].superTbls[j].colsOfCreateChildTable,
g_Dbs.threadCountByCreateTbl,
g_Dbs.db[i].superTbls[j].childTblCount,
g_Dbs.db[i].dbName, &(g_Dbs.db[i].superTbls[j]));
g_totalChildTables += g_Dbs.db[i].superTbls[j].childTblCount;
} }
startMultiThreadCreateChildTable(g_Dbs.db[i].superTbls[j].colsOfCreatChildTable, g_Dbs.threadCountByCreateTbl, g_Dbs.db[i].superTbls[j].childTblCount, g_Dbs.db[i].dbName, &(g_Dbs.db[i].superTbls[j])); } else {
g_totalChildTables += g_Dbs.db[i].superTbls[j].childTblCount; // normal table
} len = snprintf(tblColsBuf, MAX_SQL_SIZE, "(TS TIMESTAMP");
for (int i = 0; i < MAX_COLUMN_COUNT; i++) {
if (g_args.datatype[i]) {
if ((strncasecmp(g_args.datatype[i], "BINARY", strlen("BINARY")) == 0)
|| (strncasecmp(g_args.datatype[i], "NCHAR", strlen("NCHAR")) == 0)) {
len = snprintf(tblColsBuf + len, MAX_SQL_SIZE, ", COL%d %s(60)", i, g_args.datatype[i]);
} else {
len = snprintf(tblColsBuf + len, MAX_SQL_SIZE, ", COL%d %s", i, g_args.datatype[i]);
}
len = strlen(tblColsBuf);
} else {
len = snprintf(tblColsBuf + len, MAX_SQL_SIZE, ")");
break;
}
}
debugPrint("DEBUG - %s() LN%d: %s\n", __func__, __LINE__,
tblColsBuf);
startMultiThreadCreateChildTable(
tblColsBuf,
g_Dbs.threadCountByCreateTbl,
g_args.num_of_DPT,
g_Dbs.db[i].dbName,
NULL);
}
} }
} }
...@@ -2220,10 +2400,11 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) { ...@@ -2220,10 +2400,11 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) {
size_t n = 0; size_t n = 0;
ssize_t readLen = 0; ssize_t readLen = 0;
char * line = NULL; char * line = NULL;
FILE *fp = fopen(superTblInfo->tagsFile, "r"); FILE *fp = fopen(superTblInfo->tagsFile, "r");
if (fp == NULL) { if (fp == NULL) {
printf("Failed to open tags file: %s, reason:%s\n", superTblInfo->tagsFile, strerror(errno)); printf("Failed to open tags file: %s, reason:%s\n",
superTblInfo->tagsFile, strerror(errno));
return -1; return -1;
} }
...@@ -2231,7 +2412,7 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) { ...@@ -2231,7 +2412,7 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) {
free(superTblInfo->tagDataBuf); free(superTblInfo->tagDataBuf);
superTblInfo->tagDataBuf = NULL; superTblInfo->tagDataBuf = NULL;
} }
int tagCount = 10000; int tagCount = 10000;
int count = 0; int count = 0;
char* tagDataBuf = calloc(1, superTblInfo->lenOfTagOfOneRow * tagCount); char* tagDataBuf = calloc(1, superTblInfo->lenOfTagOfOneRow * tagCount);
...@@ -2254,11 +2435,13 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) { ...@@ -2254,11 +2435,13 @@ int readTagFromCsvFileToMem(SSuperTable * superTblInfo) {
count++; count++;
if (count >= tagCount - 1) { if (count >= tagCount - 1) {
char *tmp = realloc(tagDataBuf, (size_t)tagCount*1.5*superTblInfo->lenOfTagOfOneRow); char *tmp = realloc(tagDataBuf,
(size_t)tagCount*1.5*superTblInfo->lenOfTagOfOneRow);
if (tmp != NULL) { if (tmp != NULL) {
tagDataBuf = tmp; tagDataBuf = tmp;
tagCount = (int)(tagCount*1.5); tagCount = (int)(tagCount*1.5);
memset(tagDataBuf + count*superTblInfo->lenOfTagOfOneRow, 0, (size_t)((tagCount-count)*superTblInfo->lenOfTagOfOneRow)); memset(tagDataBuf + count*superTblInfo->lenOfTagOfOneRow,
0, (size_t)((tagCount-count)*superTblInfo->lenOfTagOfOneRow));
} else { } else {
// exit, if allocate more memory failed // exit, if allocate more memory failed
printf("realloc fail for save tag val from %s\n", superTblInfo->tagsFile); printf("realloc fail for save tag val from %s\n", superTblInfo->tagsFile);
...@@ -2298,7 +2481,8 @@ int readSampleFromCsvFileToMem(FILE *fp, SSuperTable* superTblInfo, char* sample ...@@ -2298,7 +2481,8 @@ int readSampleFromCsvFileToMem(FILE *fp, SSuperTable* superTblInfo, char* sample
readLen = tgetline(&line, &n, fp); readLen = tgetline(&line, &n, fp);
if (-1 == readLen) { if (-1 == readLen) {
if(0 != fseek(fp, 0, SEEK_SET)) { if(0 != fseek(fp, 0, SEEK_SET)) {
printf("Failed to fseek file: %s, reason:%s\n", superTblInfo->sampleFile, strerror(errno)); printf("Failed to fseek file: %s, reason:%s\n",
superTblInfo->sampleFile, strerror(errno));
return -1; return -1;
} }
continue; continue;
...@@ -2313,7 +2497,8 @@ int readSampleFromCsvFileToMem(FILE *fp, SSuperTable* superTblInfo, char* sample ...@@ -2313,7 +2497,8 @@ int readSampleFromCsvFileToMem(FILE *fp, SSuperTable* superTblInfo, char* sample
} }
if (readLen > superTblInfo->lenOfOneRow) { if (readLen > superTblInfo->lenOfOneRow) {
printf("sample row len[%d] overflow define schema len[%d], so discard this row\n", (int32_t)readLen, superTblInfo->lenOfOneRow); printf("sample row len[%d] overflow define schema len[%d], so discard this row\n",
(int32_t)readLen, superTblInfo->lenOfOneRow);
continue; continue;
} }
...@@ -2359,14 +2544,15 @@ static bool getColumnAndTagTypeFromInsertJsonFile(cJSON* stbInfo, SSuperTable* s ...@@ -2359,14 +2544,15 @@ static bool getColumnAndTagTypeFromInsertJsonFile(cJSON* stbInfo, SSuperTable* s
int columnSize = cJSON_GetArraySize(columns); int columnSize = cJSON_GetArraySize(columns);
if (columnSize > MAX_COLUMN_COUNT) { if (columnSize > MAX_COLUMN_COUNT) {
printf("failed to read json, column size overflow, max column size is %d\n", MAX_COLUMN_COUNT); printf("failed to read json, column size overflow, max column size is %d\n",
MAX_COLUMN_COUNT);
goto PARSE_OVER; goto PARSE_OVER;
} }
int count = 1; int count = 1;
int index = 0; int index = 0;
StrColumn columnCase; StrColumn columnCase;
//superTbls->columnCount = columnSize; //superTbls->columnCount = columnSize;
for (int k = 0; k < columnSize; ++k) { for (int k = 0; k < columnSize; ++k) {
cJSON* column = cJSON_GetArrayItem(columns, k); cJSON* column = cJSON_GetArrayItem(columns, k);
...@@ -2544,8 +2730,30 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2544,8 +2730,30 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
goto PARSE_OVER; goto PARSE_OVER;
} }
cJSON* insertInterval = cJSON_GetObjectItem(root, "insert_interval");
if (insertInterval && insertInterval->type == cJSON_Number) {
g_args.insert_interval = insertInterval->valueint;
} else if (!insertInterval) {
g_args.insert_interval = 0;
} else {
printf("failed to read json, insert_interval not found");
goto PARSE_OVER;
}
cJSON* numRecPerReq = cJSON_GetObjectItem(root, "num_of_records_per_req");
if (numRecPerReq && numRecPerReq->type == cJSON_Number) {
g_args.num_of_RPR = numRecPerReq->valueint;
} else if (!numRecPerReq) {
g_args.num_of_RPR = 100;
} else {
printf("failed to read json, num_of_records_per_req not found");
goto PARSE_OVER;
}
cJSON *answerPrompt = cJSON_GetObjectItem(root, "confirm_parameter_prompt"); // yes, no, cJSON *answerPrompt = cJSON_GetObjectItem(root, "confirm_parameter_prompt"); // yes, no,
if (answerPrompt && answerPrompt->type == cJSON_String && answerPrompt->valuestring != NULL) { if (answerPrompt
&& answerPrompt->type == cJSON_String
&& answerPrompt->valuestring != NULL) {
if (0 == strncasecmp(answerPrompt->valuestring, "yes", 3)) { if (0 == strncasecmp(answerPrompt->valuestring, "yes", 3)) {
g_args.answer_yes = false; g_args.answer_yes = false;
} else if (0 == strncasecmp(answerPrompt->valuestring, "no", 2)) { } else if (0 == strncasecmp(answerPrompt->valuestring, "no", 2)) {
...@@ -2790,7 +2998,9 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2790,7 +2998,9 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
tstrncpy(g_Dbs.db[i].superTbls[j].childTblPrefix, prefix->valuestring, MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[i].superTbls[j].childTblPrefix, prefix->valuestring, MAX_DB_NAME_SIZE);
cJSON *autoCreateTbl = cJSON_GetObjectItem(stbInfo, "auto_create_table"); // yes, no, null cJSON *autoCreateTbl = cJSON_GetObjectItem(stbInfo, "auto_create_table"); // yes, no, null
if (autoCreateTbl && autoCreateTbl->type == cJSON_String && autoCreateTbl->valuestring != NULL) { if (autoCreateTbl
&& autoCreateTbl->type == cJSON_String
&& autoCreateTbl->valuestring != NULL) {
if (0 == strncasecmp(autoCreateTbl->valuestring, "yes", 3)) { if (0 == strncasecmp(autoCreateTbl->valuestring, "yes", 3)) {
g_Dbs.db[i].superTbls[j].autoCreateTable = AUTO_CREATE_SUBTBL; g_Dbs.db[i].superTbls[j].autoCreateTable = AUTO_CREATE_SUBTBL;
} else if (0 == strncasecmp(autoCreateTbl->valuestring, "no", 2)) { } else if (0 == strncasecmp(autoCreateTbl->valuestring, "no", 2)) {
...@@ -2816,7 +3026,9 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2816,7 +3026,9 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
} }
cJSON *childTblExists = cJSON_GetObjectItem(stbInfo, "child_table_exists"); // yes, no cJSON *childTblExists = cJSON_GetObjectItem(stbInfo, "child_table_exists"); // yes, no
if (childTblExists && childTblExists->type == cJSON_String && childTblExists->valuestring != NULL) { if (childTblExists
&& childTblExists->type == cJSON_String
&& childTblExists->valuestring != NULL) {
if (0 == strncasecmp(childTblExists->valuestring, "yes", 3)) { if (0 == strncasecmp(childTblExists->valuestring, "yes", 3)) {
g_Dbs.db[i].superTbls[j].childTblExists = TBL_ALREADY_EXISTS; g_Dbs.db[i].superTbls[j].childTblExists = TBL_ALREADY_EXISTS;
} else if (0 == strncasecmp(childTblExists->valuestring, "no", 2)) { } else if (0 == strncasecmp(childTblExists->valuestring, "no", 2)) {
...@@ -2839,8 +3051,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2839,8 +3051,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
g_Dbs.db[i].superTbls[j].childTblCount = count->valueint; g_Dbs.db[i].superTbls[j].childTblCount = count->valueint;
cJSON *dataSource = cJSON_GetObjectItem(stbInfo, "data_source"); cJSON *dataSource = cJSON_GetObjectItem(stbInfo, "data_source");
if (dataSource && dataSource->type == cJSON_String && dataSource->valuestring != NULL) { if (dataSource && dataSource->type == cJSON_String
tstrncpy(g_Dbs.db[i].superTbls[j].dataSource, dataSource->valuestring, MAX_DB_NAME_SIZE); && dataSource->valuestring != NULL) {
tstrncpy(g_Dbs.db[i].superTbls[j].dataSource,
dataSource->valuestring, MAX_DB_NAME_SIZE);
} else if (!dataSource) { } else if (!dataSource) {
tstrncpy(g_Dbs.db[i].superTbls[j].dataSource, "rand", MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[i].superTbls[j].dataSource, "rand", MAX_DB_NAME_SIZE);
} else { } else {
...@@ -2849,8 +3063,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2849,8 +3063,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
} }
cJSON *insertMode = cJSON_GetObjectItem(stbInfo, "insert_mode"); // taosc , restful cJSON *insertMode = cJSON_GetObjectItem(stbInfo, "insert_mode"); // taosc , restful
if (insertMode && insertMode->type == cJSON_String && insertMode->valuestring != NULL) { if (insertMode && insertMode->type == cJSON_String
tstrncpy(g_Dbs.db[i].superTbls[j].insertMode, insertMode->valuestring, MAX_DB_NAME_SIZE); && insertMode->valuestring != NULL) {
tstrncpy(g_Dbs.db[i].superTbls[j].insertMode,
insertMode->valuestring, MAX_DB_NAME_SIZE);
} else if (!insertMode) { } else if (!insertMode) {
tstrncpy(g_Dbs.db[i].superTbls[j].insertMode, "taosc", MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[i].superTbls[j].insertMode, "taosc", MAX_DB_NAME_SIZE);
} else { } else {
...@@ -2893,7 +3109,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2893,7 +3109,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
cJSON *sampleFormat = cJSON_GetObjectItem(stbInfo, "sample_format"); cJSON *sampleFormat = cJSON_GetObjectItem(stbInfo, "sample_format");
if (sampleFormat && sampleFormat->type == cJSON_String && sampleFormat->valuestring != NULL) { if (sampleFormat && sampleFormat->type == cJSON_String && sampleFormat->valuestring != NULL) {
tstrncpy(g_Dbs.db[i].superTbls[j].sampleFormat, sampleFormat->valuestring, MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[i].superTbls[j].sampleFormat,
sampleFormat->valuestring, MAX_DB_NAME_SIZE);
} else if (!sampleFormat) { } else if (!sampleFormat) {
tstrncpy(g_Dbs.db[i].superTbls[j].sampleFormat, "csv", MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[i].superTbls[j].sampleFormat, "csv", MAX_DB_NAME_SIZE);
} else { } else {
...@@ -2903,7 +3120,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2903,7 +3120,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
cJSON *sampleFile = cJSON_GetObjectItem(stbInfo, "sample_file"); cJSON *sampleFile = cJSON_GetObjectItem(stbInfo, "sample_file");
if (sampleFile && sampleFile->type == cJSON_String && sampleFile->valuestring != NULL) { if (sampleFile && sampleFile->type == cJSON_String && sampleFile->valuestring != NULL) {
tstrncpy(g_Dbs.db[i].superTbls[j].sampleFile, sampleFile->valuestring, MAX_FILE_NAME_LEN); tstrncpy(g_Dbs.db[i].superTbls[j].sampleFile,
sampleFile->valuestring, MAX_FILE_NAME_LEN);
} else if (!sampleFile) { } else if (!sampleFile) {
memset(g_Dbs.db[i].superTbls[j].sampleFile, 0, MAX_FILE_NAME_LEN); memset(g_Dbs.db[i].superTbls[j].sampleFile, 0, MAX_FILE_NAME_LEN);
} else { } else {
...@@ -2913,7 +3131,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2913,7 +3131,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
cJSON *tagsFile = cJSON_GetObjectItem(stbInfo, "tags_file"); cJSON *tagsFile = cJSON_GetObjectItem(stbInfo, "tags_file");
if (tagsFile && tagsFile->type == cJSON_String && tagsFile->valuestring != NULL) { if (tagsFile && tagsFile->type == cJSON_String && tagsFile->valuestring != NULL) {
tstrncpy(g_Dbs.db[i].superTbls[j].tagsFile, tagsFile->valuestring, MAX_FILE_NAME_LEN); tstrncpy(g_Dbs.db[i].superTbls[j].tagsFile,
tagsFile->valuestring, MAX_FILE_NAME_LEN);
if (0 == g_Dbs.db[i].superTbls[j].tagsFile[0]) { if (0 == g_Dbs.db[i].superTbls[j].tagsFile[0]) {
g_Dbs.db[i].superTbls[j].tagSource = 0; g_Dbs.db[i].superTbls[j].tagSource = 0;
} else { } else {
...@@ -2943,8 +3162,11 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2943,8 +3162,11 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
goto PARSE_OVER; goto PARSE_OVER;
} }
cJSON *multiThreadWriteOneTbl = cJSON_GetObjectItem(stbInfo, "multi_thread_write_one_tbl"); // no , yes cJSON *multiThreadWriteOneTbl =
if (multiThreadWriteOneTbl && multiThreadWriteOneTbl->type == cJSON_String && multiThreadWriteOneTbl->valuestring != NULL) { cJSON_GetObjectItem(stbInfo, "multi_thread_write_one_tbl"); // no , yes
if (multiThreadWriteOneTbl
&& multiThreadWriteOneTbl->type == cJSON_String
&& multiThreadWriteOneTbl->valuestring != NULL) {
if (0 == strncasecmp(multiThreadWriteOneTbl->valuestring, "yes", 3)) { if (0 == strncasecmp(multiThreadWriteOneTbl->valuestring, "yes", 3)) {
g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl = 1; g_Dbs.db[i].superTbls[j].multiThreadWriteOneTbl = 1;
} else { } else {
...@@ -2997,16 +3219,7 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -2997,16 +3219,7 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
goto PARSE_OVER; goto PARSE_OVER;
} }
cJSON* insertRate = cJSON_GetObjectItem(stbInfo, "insert_rate");
if (insertRate && insertRate->type == cJSON_Number) {
g_Dbs.db[i].superTbls[j].insertRate = insertRate->valueint;
} else if (!insertRate) {
g_Dbs.db[i].superTbls[j].insertRate = 0;
} else {
printf("failed to read json, insert_rate not found");
goto PARSE_OVER;
}
cJSON* insertRows = cJSON_GetObjectItem(stbInfo, "insert_rows"); cJSON* insertRows = cJSON_GetObjectItem(stbInfo, "insert_rows");
if (insertRows && insertRows->type == cJSON_Number) { if (insertRows && insertRows->type == cJSON_Number) {
g_Dbs.db[i].superTbls[j].insertRows = insertRows->valueint; g_Dbs.db[i].superTbls[j].insertRows = insertRows->valueint;
...@@ -3020,7 +3233,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { ...@@ -3020,7 +3233,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) {
goto PARSE_OVER; goto PARSE_OVER;
} }
if (NO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable || (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) { if (NO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable
|| (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) {
continue; continue;
} }
...@@ -3080,7 +3294,8 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { ...@@ -3080,7 +3294,8 @@ static bool getMetaFromQueryJsonFile(cJSON* root) {
} }
cJSON *answerPrompt = cJSON_GetObjectItem(root, "confirm_parameter_prompt"); // yes, no, cJSON *answerPrompt = cJSON_GetObjectItem(root, "confirm_parameter_prompt"); // yes, no,
if (answerPrompt && answerPrompt->type == cJSON_String && answerPrompt->valuestring != NULL) { if (answerPrompt && answerPrompt->type == cJSON_String
&& answerPrompt->valuestring != NULL) {
if (0 == strncasecmp(answerPrompt->valuestring, "yes", 3)) { if (0 == strncasecmp(answerPrompt->valuestring, "yes", 3)) {
g_args.answer_yes = false; g_args.answer_yes = false;
} else if (0 == strncasecmp(answerPrompt->valuestring, "no", 2)) { } else if (0 == strncasecmp(answerPrompt->valuestring, "no", 2)) {
...@@ -3174,7 +3389,9 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { ...@@ -3174,7 +3389,9 @@ static bool getMetaFromQueryJsonFile(cJSON* root) {
} }
cJSON* keepProgress = cJSON_GetObjectItem(superQuery, "keepProgress"); cJSON* keepProgress = cJSON_GetObjectItem(superQuery, "keepProgress");
if (keepProgress && keepProgress->type == cJSON_String && keepProgress->valuestring != NULL) { if (keepProgress
&& keepProgress->type == cJSON_String
&& keepProgress->valuestring != NULL) {
if (0 == strcmp("yes", keepProgress->valuestring)) { if (0 == strcmp("yes", keepProgress->valuestring)) {
g_queryInfo.superQueryInfo.subscribeKeepProgress = 1; g_queryInfo.superQueryInfo.subscribeKeepProgress = 1;
} else if (0 == strcmp("no", keepProgress->valuestring)) { } else if (0 == strcmp("no", keepProgress->valuestring)) {
...@@ -3365,6 +3582,8 @@ PARSE_OVER: ...@@ -3365,6 +3582,8 @@ PARSE_OVER:
} }
static bool getInfoFromJsonFile(char* file) { static bool getInfoFromJsonFile(char* file) {
debugPrint("DEBUG - %s %d %s\n", __func__, __LINE__, file);
FILE *fp = fopen(file, "r"); FILE *fp = fopen(file, "r");
if (!fp) { if (!fp) {
printf("failed to read %s, reason:%s\n", file, strerror(errno)); printf("failed to read %s, reason:%s\n", file, strerror(errno));
...@@ -3392,32 +3611,32 @@ static bool getInfoFromJsonFile(char* file) { ...@@ -3392,32 +3611,32 @@ static bool getInfoFromJsonFile(char* file) {
cJSON* filetype = cJSON_GetObjectItem(root, "filetype"); cJSON* filetype = cJSON_GetObjectItem(root, "filetype");
if (filetype && filetype->type == cJSON_String && filetype->valuestring != NULL) { if (filetype && filetype->type == cJSON_String && filetype->valuestring != NULL) {
if (0 == strcasecmp("insert", filetype->valuestring)) { if (0 == strcasecmp("insert", filetype->valuestring)) {
g_jsonType = INSERT_MODE; g_args.test_mode = INSERT_MODE;
} else if (0 == strcasecmp("query", filetype->valuestring)) { } else if (0 == strcasecmp("query", filetype->valuestring)) {
g_jsonType = QUERY_MODE; g_args.test_mode = QUERY_MODE;
} else if (0 == strcasecmp("subscribe", filetype->valuestring)) { } else if (0 == strcasecmp("subscribe", filetype->valuestring)) {
g_jsonType = SUBSCRIBE_MODE; g_args.test_mode = SUBSCRIBE_MODE;
} else { } else {
printf("failed to read json, filetype not support\n"); printf("failed to read json, filetype not support\n");
goto PARSE_OVER; goto PARSE_OVER;
} }
} else if (!filetype) { } else if (!filetype) {
g_jsonType = INSERT_MODE; g_args.test_mode = INSERT_MODE;
} else { } else {
printf("failed to read json, filetype not found\n"); printf("failed to read json, filetype not found\n");
goto PARSE_OVER; goto PARSE_OVER;
} }
if (INSERT_MODE == g_jsonType) { if (INSERT_MODE == g_args.test_mode) {
ret = getMetaFromInsertJsonFile(root); ret = getMetaFromInsertJsonFile(root);
} else if (QUERY_MODE == g_jsonType) { } else if (QUERY_MODE == g_args.test_mode) {
ret = getMetaFromQueryJsonFile(root); ret = getMetaFromQueryJsonFile(root);
} else if (SUBSCRIBE_MODE == g_jsonType) { } else if (SUBSCRIBE_MODE == g_args.test_mode) {
ret = getMetaFromQueryJsonFile(root); ret = getMetaFromQueryJsonFile(root);
} else { } else {
printf("input json file type error! please input correct file type: insert or query or subscribe\n"); printf("input json file type error! please input correct file type: insert or query or subscribe\n");
goto PARSE_OVER; goto PARSE_OVER;
} }
PARSE_OVER: PARSE_OVER:
free(content); free(content);
...@@ -3426,14 +3645,13 @@ PARSE_OVER: ...@@ -3426,14 +3645,13 @@ PARSE_OVER:
return ret; return ret;
} }
void prePareSampleData() { void prePareSampleData() {
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) {
//if (0 == strncasecmp(g_Dbs.db[i].superTbls[j].dataSource, "sample", 6)) { //if (0 == strncasecmp(g_Dbs.db[i].superTbls[j].dataSource, "sample", 6)) {
// readSampleFromFileToMem(&g_Dbs.db[i].superTbls[j]); // readSampleFromFileToMem(&g_Dbs.db[i].superTbls[j]);
//} //}
if (g_Dbs.db[i].superTbls[j].tagsFile[0] != 0) { if (g_Dbs.db[i].superTbls[j].tagsFile[0] != 0) {
(void)readTagFromCsvFileToMem(&g_Dbs.db[i].superTbls[j]); (void)readTagFromCsvFileToMem(&g_Dbs.db[i].superTbls[j]);
} }
...@@ -3445,9 +3663,9 @@ void postFreeResource() { ...@@ -3445,9 +3663,9 @@ void postFreeResource() {
tmfclose(g_fpOfInsertResult); tmfclose(g_fpOfInsertResult);
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) {
if (0 != g_Dbs.db[i].superTbls[j].colsOfCreatChildTable) { if (0 != g_Dbs.db[i].superTbls[j].colsOfCreateChildTable) {
free(g_Dbs.db[i].superTbls[j].colsOfCreatChildTable); free(g_Dbs.db[i].superTbls[j].colsOfCreateChildTable);
g_Dbs.db[i].superTbls[j].colsOfCreatChildTable = NULL; g_Dbs.db[i].superTbls[j].colsOfCreateChildTable = NULL;
} }
if (0 != g_Dbs.db[i].superTbls[j].sampleDataBuf) { if (0 != g_Dbs.db[i].superTbls[j].sampleDataBuf) {
free(g_Dbs.db[i].superTbls[j].sampleDataBuf); free(g_Dbs.db[i].superTbls[j].sampleDataBuf);
...@@ -3525,11 +3743,12 @@ int generateRowData(char* dataBuf, int maxLen, int64_t timestamp, SSuperTable* ...@@ -3525,11 +3743,12 @@ int generateRowData(char* dataBuf, int maxLen, int64_t timestamp, SSuperTable*
} }
dataLen -= 2; dataLen -= 2;
dataLen += snprintf(dataBuf + dataLen, maxLen - dataLen, ")"); dataLen += snprintf(dataBuf + dataLen, maxLen - dataLen, ")");
return dataLen; return dataLen;
} }
void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDataBuf) { static void syncWriteForNumberOfTblInOneSql(
threadInfo *winfo, FILE *fp, char* sampleDataBuf) {
SSuperTable* superTblInfo = winfo->superTblInfo; SSuperTable* superTblInfo = winfo->superTblInfo;
int samplePos = 0; int samplePos = 0;
...@@ -3551,30 +3770,18 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3551,30 +3770,18 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
numberOfTblInOneSql = tbls; numberOfTblInOneSql = tbls;
} }
int64_t time_counter = winfo->start_time; uint64_t time_counter = winfo->start_time;
int64_t tmp_time; int64_t tmp_time;
int sampleUsePos; int sampleUsePos;
int64_t st = 0; int64_t st = 0;
int64_t et = 0; int64_t et = 0;
for (int i = 0; i < superTblInfo->insertRows;) { for (int i = 0; i < superTblInfo->insertRows;) {
if (superTblInfo->insertRate && (et - st) < 1000) {
taosMsleep(1000 - (et - st)); // ms
//printf("========sleep duration:%"PRId64 "========inserted rows:%d, table range:%d - %d\n", (1000 - (et - st)), i, winfo->start_table_id, winfo->end_table_id);
}
if (superTblInfo->insertRate) {
st = taosGetTimestampMs();
}
int32_t tbl_id = 0; int32_t tbl_id = 0;
for (int tID = winfo->start_table_id; tID <= winfo->end_table_id; ) { for (int tID = winfo->start_table_id; tID <= winfo->end_table_id; ) {
int inserted = i; int inserted = i;
int k = 0; for (int k = 0; k < g_args.num_of_RPR;) {
int batchRowsSql = 0;
while (1)
{
int len = 0; int len = 0;
memset(buffer, 0, superTblInfo->maxSqlLen); memset(buffer, 0, superTblInfo->maxSqlLen);
char *pstr = buffer; char *pstr = buffer;
...@@ -3583,6 +3790,7 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3583,6 +3790,7 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
if (end_tbl_id > winfo->end_table_id) { if (end_tbl_id > winfo->end_table_id) {
end_tbl_id = winfo->end_table_id+1; end_tbl_id = winfo->end_table_id+1;
} }
for (tbl_id = tID; tbl_id < end_tbl_id; tbl_id++) { for (tbl_id = tID; tbl_id < end_tbl_id; tbl_id++) {
sampleUsePos = samplePos; sampleUsePos = samplePos;
if (AUTO_CREATE_SUBTBL == superTblInfo->autoCreateTable) { if (AUTO_CREATE_SUBTBL == superTblInfo->autoCreateTable) {
...@@ -3590,47 +3798,97 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3590,47 +3798,97 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
if (0 == superTblInfo->tagSource) { if (0 == superTblInfo->tagSource) {
tagsValBuf = generateTagVaulesForStb(superTblInfo); tagsValBuf = generateTagVaulesForStb(superTblInfo);
} else { } else {
tagsValBuf = getTagValueFromTagSample(superTblInfo, tbl_id % superTblInfo->tagSampleCount); tagsValBuf = getTagValueFromTagSample(
superTblInfo, tbl_id % superTblInfo->tagSampleCount);
} }
if (NULL == tagsValBuf) { if (NULL == tagsValBuf) {
goto free_and_statistics; goto free_and_statistics;
} }
if (0 == len) { if (0 == len) {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s%d using %s.%s tags %s values ", winfo->db_name, superTblInfo->childTblPrefix, tbl_id, winfo->db_name, superTblInfo->sTblName, tagsValBuf); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s%d using %s.%s tags %s values ",
winfo->db_name,
superTblInfo->childTblPrefix,
tbl_id,
winfo->db_name,
superTblInfo->sTblName,
tagsValBuf);
} else { } else {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, " %s.%s%d using %s.%s tags %s values ", winfo->db_name, superTblInfo->childTblPrefix, tbl_id, winfo->db_name, superTblInfo->sTblName, tagsValBuf); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
" %s.%s%d using %s.%s tags %s values ",
winfo->db_name,
superTblInfo->childTblPrefix,
tbl_id,
winfo->db_name,
superTblInfo->sTblName,
tagsValBuf);
} }
tmfree(tagsValBuf); tmfree(tagsValBuf);
} else if (TBL_ALREADY_EXISTS == superTblInfo->childTblExists) { } else if (TBL_ALREADY_EXISTS == superTblInfo->childTblExists) {
if (0 == len) { if (0 == len) {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s values ", winfo->db_name, superTblInfo->childTblName + tbl_id * TSDB_TABLE_NAME_LEN); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s values ",
winfo->db_name,
superTblInfo->childTblName + tbl_id * TSDB_TABLE_NAME_LEN);
} else { } else {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, " %s.%s values ", winfo->db_name, superTblInfo->childTblName + tbl_id * TSDB_TABLE_NAME_LEN); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
" %s.%s values ",
winfo->db_name,
superTblInfo->childTblName + tbl_id * TSDB_TABLE_NAME_LEN);
} }
} else { // pre-create child table } else { // pre-create child table
if (0 == len) { if (0 == len) {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s%d values ", winfo->db_name, superTblInfo->childTblPrefix, tbl_id); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s%d values ",
winfo->db_name,
superTblInfo->childTblPrefix,
tbl_id);
} else { } else {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, " %s.%s%d values ", winfo->db_name, superTblInfo->childTblPrefix, tbl_id); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
" %s.%s%d values ",
winfo->db_name,
superTblInfo->childTblPrefix,
tbl_id);
} }
} }
tmp_time = time_counter; tmp_time = time_counter;
for (k = 0; k < superTblInfo->rowsPerTbl;) { for (k = 0; k < superTblInfo->rowsPerTbl;) {
int retLen = 0; int retLen = 0;
if (0 == strncasecmp(superTblInfo->dataSource, "sample", 6)) { if (0 == strncasecmp(superTblInfo->dataSource,
retLen = getRowDataFromSample(pstr + len, superTblInfo->maxSqlLen - len, tmp_time += superTblInfo->timeStampStep, superTblInfo, &sampleUsePos, fp, sampleDataBuf); "sample", strlen("sample"))) {
retLen = getRowDataFromSample(pstr + len,
superTblInfo->maxSqlLen - len,
tmp_time += superTblInfo->timeStampStep,
superTblInfo,
&sampleUsePos,
fp,
sampleDataBuf);
if (retLen < 0) { if (retLen < 0) {
goto free_and_statistics; goto free_and_statistics;
} }
} else if (0 == strncasecmp(superTblInfo->dataSource, "rand", 8)) { } else if (0 == strncasecmp(
superTblInfo->dataSource, "rand", strlen("rand"))) {
int rand_num = rand_tinyint() % 100; int rand_num = rand_tinyint() % 100;
if (0 != superTblInfo->disorderRatio && rand_num < superTblInfo->disorderRatio) { if (0 != superTblInfo->disorderRatio
&& rand_num < superTblInfo->disorderRatio) {
int64_t d = tmp_time - rand() % superTblInfo->disorderRange; int64_t d = tmp_time - rand() % superTblInfo->disorderRange;
retLen = generateRowData(pstr + len, superTblInfo->maxSqlLen - len, d, superTblInfo); retLen = generateRowData(pstr + len,
superTblInfo->maxSqlLen - len,
d,
superTblInfo);
} else { } else {
retLen = generateRowData(pstr + len, superTblInfo->maxSqlLen - len, tmp_time += superTblInfo->timeStampStep, superTblInfo); retLen = generateRowData(pstr + len,
superTblInfo->maxSqlLen - len,
tmp_time += superTblInfo->timeStampStep,
superTblInfo);
} }
if (retLen < 0) { if (retLen < 0) {
goto free_and_statistics; goto free_and_statistics;
...@@ -3640,30 +3898,44 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3640,30 +3898,44 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
//inserted++; //inserted++;
k++; k++;
totalRowsInserted++; totalRowsInserted++;
batchRowsSql++;
if (inserted >= superTblInfo->insertRows ||
if (inserted >= superTblInfo->insertRows || (superTblInfo->maxSqlLen - len) < (superTblInfo->lenOfOneRow + 128) || batchRowsSql >= INT16_MAX - 1) { (superTblInfo->maxSqlLen - len) < (superTblInfo->lenOfOneRow + 128)) {
tID = tbl_id + 1; tID = tbl_id + 1;
printf("config rowsPerTbl and numberOfTblInOneSql not match with max_sql_lenth, please reconfig![lenOfOneRow:%d]\n", superTblInfo->lenOfOneRow); printf("config rowsPerTbl and numberOfTblInOneSql not match with max_sql_lenth, please reconfig![lenOfOneRow:%d]\n",
superTblInfo->lenOfOneRow);
goto send_to_server; goto send_to_server;
} }
} }
} }
tID = tbl_id; tID = tbl_id;
inserted += superTblInfo->rowsPerTbl; inserted += superTblInfo->rowsPerTbl;
send_to_server: send_to_server:
batchRowsSql = 0; if (g_args.insert_interval && (g_args.insert_interval > (et - st))) {
if (0 == strncasecmp(superTblInfo->insertMode, "taosc", 5)) { int sleep_time = g_args.insert_interval - (et -st);
debugPrint("DEBUG sleep: %d ms\n", sleep_time);
taosMsleep(sleep_time); // ms
}
if (g_args.insert_interval) {
st = taosGetTimestampMs();
}
if (0 == strncasecmp(superTblInfo->insertMode,
"taosc",
strlen("taosc"))) {
//printf("multi table===== sql: %s \n\n", buffer); //printf("multi table===== sql: %s \n\n", buffer);
//int64_t t1 = taosGetTimestampMs(); //int64_t t1 = taosGetTimestampMs();
int64_t startTs; int64_t startTs;
int64_t endTs; int64_t endTs;
startTs = taosGetTimestampUs(); startTs = taosGetTimestampUs();
int affectedRows = queryDbExec(winfo->taos, buffer, INSERT_TYPE); debugPrint("DEBUG %s() LN%d buff: %s\n", __func__, __LINE__, buffer);
int affectedRows = queryDbExec(
winfo->taos, buffer, INSERT_TYPE);
if (0 > affectedRows) { if (0 > affectedRows) {
goto free_and_statistics; goto free_and_statistics;
} else { } else {
...@@ -3679,29 +3951,34 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3679,29 +3951,34 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
int64_t currentPrintTime = taosGetTimestampMs(); int64_t currentPrintTime = taosGetTimestampMs();
if (currentPrintTime - lastPrintTime > 30*1000) { if (currentPrintTime - lastPrintTime > 30*1000) {
printf("thread[%d] has currently inserted rows: %"PRId64 ", affected rows: %"PRId64 "\n", winfo->threadID, totalRowsInserted, totalAffectedRows); printf("thread[%d] has currently inserted rows: %"PRId64 ", affected rows: %"PRId64 "\n",
winfo->threadID,
totalRowsInserted,
totalAffectedRows);
lastPrintTime = currentPrintTime; lastPrintTime = currentPrintTime;
} }
//int64_t t2 = taosGetTimestampMs(); //int64_t t2 = taosGetTimestampMs();
//printf("taosc insert sql return, Spent %.4f seconds \n", (double)(t2 - t1)/1000.0); //printf("taosc insert sql return, Spent %.4f seconds \n", (double)(t2 - t1)/1000.0);
} else { } else {
//int64_t t1 = taosGetTimestampMs(); //int64_t t1 = taosGetTimestampMs();
int retCode = postProceSql(g_Dbs.host, g_Dbs.port, buffer); int retCode = postProceSql(g_Dbs.host, g_Dbs.port, buffer);
//int64_t t2 = taosGetTimestampMs(); //int64_t t2 = taosGetTimestampMs();
//printf("http insert sql return, Spent %ld ms \n", t2 - t1); //printf("http insert sql return, Spent %ld ms \n", t2 - t1);
if (0 != retCode) { if (0 != retCode) {
printf("========restful return fail, threadID[%d]\n", winfo->threadID); printf("========restful return fail, threadID[%d]\n", winfo->threadID);
goto free_and_statistics; goto free_and_statistics;
} }
} }
if (g_args.insert_interval) {
//printf("========tID:%d, k:%d, loop_cnt:%d\n", tID, k, loop_cnt); et = taosGetTimestampMs();
}
break; break;
} }
if (tID > winfo->end_table_id) { if (tID > winfo->end_table_id) {
if (0 == strncasecmp(superTblInfo->dataSource, "sample", 6)) { if (0 == strncasecmp(superTblInfo->dataSource, "sample", strlen("sample"))) {
samplePos = sampleUsePos; samplePos = sampleUsePos;
} }
i = inserted; i = inserted;
...@@ -3709,13 +3986,10 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3709,13 +3986,10 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
} }
} }
if (superTblInfo->insertRate) {
et = taosGetTimestampMs();
}
//printf("========loop %d childTables duration:%"PRId64 "========inserted rows:%d\n", winfo->end_table_id - winfo->start_table_id, et - st, i); //printf("========loop %d childTables duration:%"PRId64 "========inserted rows:%d\n", winfo->end_table_id - winfo->start_table_id, et - st, i);
} }
free_and_statistics: free_and_statistics:
tmfree(buffer); tmfree(buffer);
winfo->totalRowsInserted = totalRowsInserted; winfo->totalRowsInserted = totalRowsInserted;
winfo->totalAffectedRows = totalAffectedRows; winfo->totalAffectedRows = totalAffectedRows;
...@@ -3723,6 +3997,64 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3723,6 +3997,64 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
return; return;
} }
int32_t generateData(char *res, char **data_type,
int num_of_cols, int64_t timestamp, int len_of_binary) {
memset(res, 0, MAX_DATA_SIZE);
char *pstr = res;
pstr += sprintf(pstr, "(%" PRId64, timestamp);
int c = 0;
for (; c < MAX_NUM_DATATYPE; c++) {
if (data_type[c] == NULL) {
break;
}
}
if (0 == c) {
perror("data type error!");
exit(-1);
}
for (int i = 0; i < num_of_cols; i++) {
if (strcasecmp(data_type[i % c], "tinyint") == 0) {
pstr += sprintf(pstr, ", %d", rand_tinyint() );
} else if (strcasecmp(data_type[i % c], "smallint") == 0) {
pstr += sprintf(pstr, ", %d", rand_smallint());
} else if (strcasecmp(data_type[i % c], "int") == 0) {
pstr += sprintf(pstr, ", %d", rand_int());
} else if (strcasecmp(data_type[i % c], "bigint") == 0) {
pstr += sprintf(pstr, ", %" PRId64, rand_bigint());
} else if (strcasecmp(data_type[i % c], "float") == 0) {
pstr += sprintf(pstr, ", %10.4f", rand_float());
} else if (strcasecmp(data_type[i % c], "double") == 0) {
double t = rand_double();
pstr += sprintf(pstr, ", %20.8f", t);
} else if (strcasecmp(data_type[i % c], "bool") == 0) {
bool b = rand() & 1;
pstr += sprintf(pstr, ", %s", b ? "true" : "false");
} else if (strcasecmp(data_type[i % c], "binary") == 0) {
char *s = malloc(len_of_binary);
rand_string(s, len_of_binary);
pstr += sprintf(pstr, ", \"%s\"", s);
free(s);
}else if (strcasecmp(data_type[i % c], "nchar") == 0) {
char *s = malloc(len_of_binary);
rand_string(s, len_of_binary);
pstr += sprintf(pstr, ", \"%s\"", s);
free(s);
}
if (pstr - res > MAX_DATA_SIZE) {
perror("column length too long, abort");
exit(-1);
}
}
pstr += sprintf(pstr, ")");
return (int32_t)(pstr - res);
}
// sync insertion // sync insertion
/* /*
1 thread: 100 tables * 2000 rows/s 1 thread: 100 tables * 2000 rows/s
...@@ -3731,11 +4063,118 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa ...@@ -3731,11 +4063,118 @@ void syncWriteForNumberOfTblInOneSql(threadInfo *winfo, FILE *fp, char* sampleDa
2 taosinsertdata , 1 thread: 10 tables * 20000 rows/s 2 taosinsertdata , 1 thread: 10 tables * 20000 rows/s
*/ */
void *syncWrite(void *sarg) { static void* syncWrite(void *sarg) {
int64_t totalRowsInserted = 0;
int64_t totalAffectedRows = 0; threadInfo *winfo = (threadInfo *)sarg;
int64_t lastPrintTime = taosGetTimestampMs();
char buffer[BUFFER_SIZE] = "\0";
char data[MAX_DATA_SIZE];
char **data_type = g_args.datatype;
int len_of_binary = g_args.len_of_binary;
int ncols_per_record = 1; // count first col ts
for (int i = 0; i < MAX_COLUMN_COUNT; i ++) {
if (NULL == g_args.datatype[i])
break;
else
ncols_per_record ++;
}
srand((uint32_t)time(NULL));
int64_t time_counter = winfo->start_time;
uint64_t st = 0;
uint64_t et = 0;
for (int i = 0; i < g_args.num_of_DPT;) {
for (int tID = winfo->start_table_id; tID <= winfo->end_table_id; tID++) {
int inserted = i;
int64_t tmp_time = time_counter;
char *pstr = buffer;
pstr += sprintf(pstr,
"insert into %s.%s%d values",
winfo->db_name, g_args.tb_prefix, tID);
int k;
for (k = 0; k < g_args.num_of_RPR;) {
int rand_num = rand() % 100;
int len = -1;
if ((g_args.disorderRatio != 0)
&& (rand_num < g_args.disorderRange)) {
int64_t d = tmp_time - rand() % 1000000 + rand_num;
len = generateData(data, data_type,
ncols_per_record, d, len_of_binary);
} else {
len = generateData(data, data_type,
ncols_per_record, tmp_time += 1000, len_of_binary);
}
//assert(len + pstr - buffer < BUFFER_SIZE);
if (len + pstr - buffer >= BUFFER_SIZE) { // too long
break;
}
pstr += sprintf(pstr, " %s", data);
inserted++;
k++;
if (inserted >= g_args.num_of_DPT)
break;
}
/* puts(buffer); */
int64_t startTs;
int64_t endTs;
startTs = taosGetTimestampUs();
//queryDB(winfo->taos, buffer);
if (i > 0 && g_args.insert_interval
&& (g_args.insert_interval > (et - st) )) {
int sleep_time = g_args.insert_interval - (et -st);
debugPrint("DEBUG sleep: %d ms\n", sleep_time);
taosMsleep(sleep_time); // ms
}
if (g_args.insert_interval) {
st = taosGetTimestampMs();
}
debugPrint("DEBUG - %s() LN%d %s\n", __func__, __LINE__, buffer);
int affectedRows = queryDbExec(winfo->taos, buffer, 1);
if (0 <= affectedRows){
endTs = taosGetTimestampUs();
int64_t delay = endTs - startTs;
if (delay > winfo->maxDelay)
winfo->maxDelay = delay;
if (delay < winfo->minDelay)
winfo->minDelay = delay;
winfo->cntDelay++;
winfo->totalDelay += delay;
//winfo->avgDelay = (double)winfo->totalDelay / winfo->cntDelay;
}
if (g_args.insert_interval) {
et = taosGetTimestampMs();
}
if (tID == winfo->end_table_id) {
i = inserted;
time_counter = tmp_time;
}
}
}
return NULL;
}
static void* syncWriteWithStb(void *sarg) {
uint64_t totalRowsInserted = 0;
uint64_t totalAffectedRows = 0;
uint64_t lastPrintTime = taosGetTimestampMs();
threadInfo *winfo = (threadInfo *)sarg; threadInfo *winfo = (threadInfo *)sarg;
SSuperTable* superTblInfo = winfo->superTblInfo; SSuperTable* superTblInfo = winfo->superTblInfo;
...@@ -3744,20 +4183,27 @@ void *syncWrite(void *sarg) { ...@@ -3744,20 +4183,27 @@ void *syncWrite(void *sarg) {
int samplePos = 0; int samplePos = 0;
// each thread read sample data from csv file // each thread read sample data from csv file
if (0 == strncasecmp(superTblInfo->dataSource, "sample", 6)) { if (0 == strncasecmp(superTblInfo->dataSource,
sampleDataBuf = calloc(superTblInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE, 1); "sample",
strlen("sample"))) {
sampleDataBuf = calloc(
superTblInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE, 1);
if (sampleDataBuf == NULL) { if (sampleDataBuf == NULL) {
printf("Failed to calloc %d Bytes, reason:%s\n", superTblInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE, strerror(errno)); printf("Failed to calloc %d Bytes, reason:%s\n",
superTblInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE,
strerror(errno));
return NULL; return NULL;
} }
fp = fopen(superTblInfo->sampleFile, "r"); fp = fopen(superTblInfo->sampleFile, "r");
if (fp == NULL) { if (fp == NULL) {
printf("Failed to open sample file: %s, reason:%s\n", superTblInfo->sampleFile, strerror(errno)); printf("Failed to open sample file: %s, reason:%s\n",
superTblInfo->sampleFile, strerror(errno));
tmfree(sampleDataBuf); tmfree(sampleDataBuf);
return NULL; return NULL;
} }
int ret = readSampleFromCsvFileToMem(fp, superTblInfo, sampleDataBuf); int ret = readSampleFromCsvFileToMem(fp,
superTblInfo, sampleDataBuf);
if (0 != ret) { if (0 != ret) {
tmfree(sampleDataBuf); tmfree(sampleDataBuf);
tmfclose(fp); tmfclose(fp);
...@@ -3772,68 +4218,32 @@ void *syncWrite(void *sarg) { ...@@ -3772,68 +4218,32 @@ void *syncWrite(void *sarg) {
return NULL; return NULL;
} }
//printf("========threadID[%d], table rang: %d - %d \n", winfo->threadID, winfo->start_table_id, winfo->end_table_id);
char* buffer = calloc(superTblInfo->maxSqlLen, 1); char* buffer = calloc(superTblInfo->maxSqlLen, 1);
if (NULL == buffer) {
int nrecords_per_request = 0; printf("Failed to calloc %d Bytes, reason:%s\n",
if (AUTO_CREATE_SUBTBL == superTblInfo->autoCreateTable) { superTblInfo->maxSqlLen,
nrecords_per_request = (superTblInfo->maxSqlLen - 1280 - superTblInfo->lenOfTagOfOneRow) / superTblInfo->lenOfOneRow; strerror(errno));
} else { tmfree(sampleDataBuf);
nrecords_per_request = (superTblInfo->maxSqlLen - 1280) / superTblInfo->lenOfOneRow; tmfclose(fp);
} return NULL;
int nrecords_no_last_req = nrecords_per_request;
int nrecords_last_req = 0;
int loop_cnt = 0;
if (0 != superTblInfo->insertRate) {
if (nrecords_no_last_req >= superTblInfo->insertRate) {
nrecords_no_last_req = superTblInfo->insertRate;
} else {
nrecords_last_req = superTblInfo->insertRate % nrecords_per_request;
loop_cnt = (superTblInfo->insertRate / nrecords_per_request) + (superTblInfo->insertRate % nrecords_per_request ? 1 : 0) ;
}
}
if (nrecords_no_last_req <= 0) {
nrecords_no_last_req = 1;
}
if (nrecords_no_last_req >= INT16_MAX) {
nrecords_no_last_req = INT16_MAX - 1;
}
if (nrecords_last_req >= INT16_MAX) {
nrecords_last_req = INT16_MAX - 1;
} }
int nrecords_cur_req = nrecords_no_last_req;
int loop_cnt_orig = loop_cnt;
//printf("========nrecords_per_request:%d, nrecords_no_last_req:%d, nrecords_last_req:%d, loop_cnt:%d\n", nrecords_per_request, nrecords_no_last_req, nrecords_last_req, loop_cnt);
int64_t time_counter = winfo->start_time; int64_t time_counter = winfo->start_time;
uint64_t st = 0;
uint64_t et = 0;
debugPrint("DEBUG - %s() LN%d insertRows=%"PRId64"\n", __func__, __LINE__, superTblInfo->insertRows);
int64_t st = 0;
int64_t et = 0;
for (int i = 0; i < superTblInfo->insertRows;) { for (int i = 0; i < superTblInfo->insertRows;) {
if (superTblInfo->insertRate && (et - st) < 1000) {
taosMsleep(1000 - (et - st)); // ms
//printf("========sleep duration:%"PRId64 "========inserted rows:%d, table range:%d - %d\n", (1000 - (et - st)), i, winfo->start_table_id, winfo->end_table_id);
}
if (superTblInfo->insertRate) { for (uint32_t tID = winfo->start_table_id; tID <= winfo->end_table_id; tID++) {
st = taosGetTimestampMs(); uint64_t inserted = i;
} uint64_t tmp_time = time_counter;
for (int tID = winfo->start_table_id; tID <= winfo->end_table_id; tID++) {
int inserted = i;
int64_t tmp_time = time_counter;
int sampleUsePos = samplePos; int sampleUsePos = samplePos;
int k = 0; int k = 0;
while (1) debugPrint("DEBUG - %s() LN%d num_of_RPR=%d\n", __func__, __LINE__, g_args.num_of_RPR);
{ for (k = 0; k < g_args.num_of_RPR;) {
int len = 0; int len = 0;
memset(buffer, 0, superTblInfo->maxSqlLen); memset(buffer, 0, superTblInfo->maxSqlLen);
char *pstr = buffer; char *pstr = buffer;
...@@ -3843,56 +4253,106 @@ void *syncWrite(void *sarg) { ...@@ -3843,56 +4253,106 @@ void *syncWrite(void *sarg) {
if (0 == superTblInfo->tagSource) { if (0 == superTblInfo->tagSource) {
tagsValBuf = generateTagVaulesForStb(superTblInfo); tagsValBuf = generateTagVaulesForStb(superTblInfo);
} else { } else {
tagsValBuf = getTagValueFromTagSample(superTblInfo, tID % superTblInfo->tagSampleCount); tagsValBuf = getTagValueFromTagSample(
superTblInfo,
tID % superTblInfo->tagSampleCount);
} }
if (NULL == tagsValBuf) { if (NULL == tagsValBuf) {
goto free_and_statistics_2; goto free_and_statistics_2;
} }
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s%d using %s.%s tags %s values", winfo->db_name, superTblInfo->childTblPrefix, tID, winfo->db_name, superTblInfo->sTblName, tagsValBuf); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s%d using %s.%s tags %s values",
winfo->db_name,
superTblInfo->childTblPrefix,
tID,
winfo->db_name,
superTblInfo->sTblName,
tagsValBuf);
tmfree(tagsValBuf); tmfree(tagsValBuf);
} else if (TBL_ALREADY_EXISTS == superTblInfo->childTblExists) { } else if (TBL_ALREADY_EXISTS == superTblInfo->childTblExists) {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s values", winfo->db_name, superTblInfo->childTblName + tID * TSDB_TABLE_NAME_LEN); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s values",
winfo->db_name,
superTblInfo->childTblName + tID * TSDB_TABLE_NAME_LEN);
} else { } else {
len += snprintf(pstr + len, superTblInfo->maxSqlLen - len, "insert into %s.%s%d values", winfo->db_name, superTblInfo->childTblPrefix, tID); len += snprintf(pstr + len,
superTblInfo->maxSqlLen - len,
"insert into %s.%s%d values",
winfo->db_name,
superTblInfo->childTblPrefix,
tID);
} }
for (k = 0; k < nrecords_cur_req;) { int retLen = 0;
int retLen = 0; if (0 == strncasecmp(superTblInfo->dataSource, "sample", strlen("sample"))) {
if (0 == strncasecmp(superTblInfo->dataSource, "sample", 6)) { retLen = getRowDataFromSample(
retLen = getRowDataFromSample(pstr + len, superTblInfo->maxSqlLen - len, tmp_time += superTblInfo->timeStampStep, superTblInfo, &sampleUsePos, fp, sampleDataBuf); pstr + len,
superTblInfo->maxSqlLen - len,
tmp_time += superTblInfo->timeStampStep,
superTblInfo,
&sampleUsePos,
fp,
sampleDataBuf);
if (retLen < 0) { if (retLen < 0) {
goto free_and_statistics_2; goto free_and_statistics_2;
} }
} else if (0 == strncasecmp(superTblInfo->dataSource, "rand", 8)) { } else if (0 == strncasecmp(superTblInfo->dataSource, "rand", strlen("rand"))) {
int rand_num = rand_tinyint() % 100; int rand_num = rand_tinyint() % 100;
if (0 != superTblInfo->disorderRatio && rand_num < superTblInfo->disorderRatio) { if (0 != superTblInfo->disorderRatio
&& rand_num < superTblInfo->disorderRatio) {
int64_t d = tmp_time - rand() % superTblInfo->disorderRange; int64_t d = tmp_time - rand() % superTblInfo->disorderRange;
retLen = generateRowData(pstr + len, superTblInfo->maxSqlLen - len, d, superTblInfo); retLen = generateRowData(
pstr + len,
superTblInfo->maxSqlLen - len, d,
superTblInfo);
//printf("disorder rows, rand_num:%d, last ts:%"PRId64" current ts:%"PRId64"\n", rand_num, tmp_time, d); //printf("disorder rows, rand_num:%d, last ts:%"PRId64" current ts:%"PRId64"\n", rand_num, tmp_time, d);
} else { } else {
retLen = generateRowData(pstr + len, superTblInfo->maxSqlLen - len, tmp_time += superTblInfo->timeStampStep, superTblInfo); retLen = generateRowData(
pstr + len,
superTblInfo->maxSqlLen - len,
tmp_time += superTblInfo->timeStampStep,
superTblInfo);
} }
if (retLen < 0) { if (retLen < 0) {
goto free_and_statistics_2; goto free_and_statistics_2;
} }
}
len += retLen;
inserted++;
k++;
totalRowsInserted++;
if (inserted >= superTblInfo->insertRows || (superTblInfo->maxSqlLen - len) < (superTblInfo->lenOfOneRow + 128)) break;
} }
/* len += retLen;
*/
inserted++;
k++;
totalRowsInserted++;
if (0 == strncasecmp(superTblInfo->insertMode, "taosc", 5)) { if (inserted > superTblInfo->insertRows)
break;
/* if (inserted >= superTblInfo->insertRows
|| (superTblInfo->maxSqlLen - len) < (superTblInfo->lenOfOneRow + 128))
break;
*/
if (i > 0 && g_args.insert_interval
&& (g_args.insert_interval > (et - st) )) {
int sleep_time = g_args.insert_interval - (et -st);
debugPrint("DEBUG sleep: %d ms\n", sleep_time);
taosMsleep(sleep_time); // ms
}
if (g_args.insert_interval) {
st = taosGetTimestampMs();
}
if (0 == strncasecmp(superTblInfo->insertMode, "taosc", strlen("taosc"))) {
//printf("===== sql: %s \n\n", buffer); //printf("===== sql: %s \n\n", buffer);
//int64_t t1 = taosGetTimestampMs(); //int64_t t1 = taosGetTimestampMs();
int64_t startTs; int64_t startTs;
int64_t endTs; int64_t endTs;
startTs = taosGetTimestampUs(); startTs = taosGetTimestampUs();
debugPrint("DEBUG %s() LN%d %s\n", __func__, __LINE__, buffer);
int affectedRows = queryDbExec(winfo->taos, buffer, INSERT_TYPE); int affectedRows = queryDbExec(winfo->taos, buffer, INSERT_TYPE);
if (0 > affectedRows){ if (0 > affectedRows){
goto free_and_statistics_2; goto free_and_statistics_2;
} else { } else {
...@@ -3908,10 +4368,13 @@ void *syncWrite(void *sarg) { ...@@ -3908,10 +4368,13 @@ void *syncWrite(void *sarg) {
int64_t currentPrintTime = taosGetTimestampMs(); int64_t currentPrintTime = taosGetTimestampMs();
if (currentPrintTime - lastPrintTime > 30*1000) { if (currentPrintTime - lastPrintTime > 30*1000) {
printf("thread[%d] has currently inserted rows: %"PRId64 ", affected rows: %"PRId64 "\n", winfo->threadID, totalRowsInserted, totalAffectedRows); printf("thread[%d] has currently inserted rows: %"PRId64 ", affected rows: %"PRId64 "\n",
winfo->threadID,
totalRowsInserted,
totalAffectedRows);
lastPrintTime = currentPrintTime; lastPrintTime = currentPrintTime;
} }
//int64_t t2 = taosGetTimestampMs(); //int64_t t2 = taosGetTimestampMs();
//printf("taosc insert sql return, Spent %.4f seconds \n", (double)(t2 - t1)/1000.0); //printf("taosc insert sql return, Spent %.4f seconds \n", (double)(t2 - t1)/1000.0);
} else { } else {
//int64_t t1 = taosGetTimestampMs(); //int64_t t1 = taosGetTimestampMs();
...@@ -3924,9 +4387,10 @@ void *syncWrite(void *sarg) { ...@@ -3924,9 +4387,10 @@ void *syncWrite(void *sarg) {
goto free_and_statistics_2; goto free_and_statistics_2;
} }
} }
if (g_args.insert_interval) {
//printf("========tID:%d, k:%d, loop_cnt:%d\n", tID, k, loop_cnt); et = taosGetTimestampMs();
}
/*
if (loop_cnt) { if (loop_cnt) {
loop_cnt--; loop_cnt--;
if ((1 == loop_cnt) && (0 != nrecords_last_req)) { if ((1 == loop_cnt) && (0 != nrecords_last_req)) {
...@@ -3939,24 +4403,24 @@ void *syncWrite(void *sarg) { ...@@ -3939,24 +4403,24 @@ void *syncWrite(void *sarg) {
} else { } else {
break; break;
} }
*/
} }
if (tID == winfo->end_table_id) { if (tID == winfo->end_table_id) {
if (0 == strncasecmp(superTblInfo->dataSource, "sample", 6)) { if (0 == strncasecmp(
superTblInfo->dataSource, "sample", strlen("sample"))) {
samplePos = sampleUsePos; samplePos = sampleUsePos;
} }
i = inserted; i = inserted;
time_counter = tmp_time; time_counter = tmp_time;
} }
}
if (superTblInfo->insertRate) {
et = taosGetTimestampMs();
} }
//printf("========loop %d childTables duration:%"PRId64 "========inserted rows:%d\n", winfo->end_table_id - winfo->start_table_id, et - st, i); //printf("========loop %d childTables duration:%"PRId64 "========inserted rows:%d\n", winfo->end_table_id - winfo->start_table_id, et - st, i);
} }
free_and_statistics_2: free_and_statistics_2:
tmfree(buffer); tmfree(buffer);
tmfree(sampleDataBuf); tmfree(sampleDataBuf);
tmfclose(fp); tmfclose(fp);
...@@ -3964,14 +4428,17 @@ void *syncWrite(void *sarg) { ...@@ -3964,14 +4428,17 @@ void *syncWrite(void *sarg) {
winfo->totalRowsInserted = totalRowsInserted; winfo->totalRowsInserted = totalRowsInserted;
winfo->totalAffectedRows = totalAffectedRows; winfo->totalAffectedRows = totalAffectedRows;
printf("====thread[%d] completed total inserted rows: %"PRId64 ", total affected rows: %"PRId64 "====\n", winfo->threadID, totalRowsInserted, totalAffectedRows); printf("====thread[%d] completed total inserted rows: %"PRId64 ", total affected rows: %"PRId64 "====\n",
winfo->threadID,
totalRowsInserted,
totalAffectedRows);
return NULL; return NULL;
} }
void callBack(void *param, TAOS_RES *res, int code) { void callBack(void *param, TAOS_RES *res, int code) {
threadInfo* winfo = (threadInfo*)param; threadInfo* winfo = (threadInfo*)param;
if (winfo->superTblInfo->insertRate) { if (g_args.insert_interval) {
winfo->et = taosGetTimestampMs(); winfo->et = taosGetTimestampMs();
if (winfo->et - winfo->st < 1000) { if (winfo->et - winfo->st < 1000) {
taosMsleep(1000 - (winfo->et - winfo->st)); // ms taosMsleep(1000 - (winfo->et - winfo->st)); // ms
...@@ -3994,7 +4461,7 @@ void callBack(void *param, TAOS_RES *res, int code) { ...@@ -3994,7 +4461,7 @@ void callBack(void *param, TAOS_RES *res, int code) {
return; return;
} }
for (int i = 0; i < winfo->nrecords_per_request; i++) { for (int i = 0; i < g_args.num_of_RPR; i++) {
int rand_num = rand() % 100; int rand_num = rand() % 100;
if (0 != winfo->superTblInfo->disorderRatio && rand_num < winfo->superTblInfo->disorderRatio) if (0 != winfo->superTblInfo->disorderRatio && rand_num < winfo->superTblInfo->disorderRatio)
{ {
...@@ -4013,7 +4480,7 @@ void callBack(void *param, TAOS_RES *res, int code) { ...@@ -4013,7 +4480,7 @@ void callBack(void *param, TAOS_RES *res, int code) {
} }
} }
if (winfo->superTblInfo->insertRate) { if (g_args.insert_interval) {
winfo->st = taosGetTimestampMs(); winfo->st = taosGetTimestampMs();
} }
taos_query_a(winfo->taos, buffer, callBack, winfo); taos_query_a(winfo->taos, buffer, callBack, winfo);
...@@ -4026,36 +4493,11 @@ void callBack(void *param, TAOS_RES *res, int code) { ...@@ -4026,36 +4493,11 @@ void callBack(void *param, TAOS_RES *res, int code) {
void *asyncWrite(void *sarg) { void *asyncWrite(void *sarg) {
threadInfo *winfo = (threadInfo *)sarg; threadInfo *winfo = (threadInfo *)sarg;
winfo->nrecords_per_request = 0;
//if (AUTO_CREATE_SUBTBL == winfo->superTblInfo->autoCreateTable) {
winfo->nrecords_per_request = (winfo->superTblInfo->maxSqlLen - 1280 - winfo->superTblInfo->lenOfTagOfOneRow) / winfo->superTblInfo->lenOfOneRow;
//} else {
// winfo->nrecords_per_request = (winfo->superTblInfo->maxSqlLen - 1280) / winfo->superTblInfo->lenOfOneRow;
//}
if (0 != winfo->superTblInfo->insertRate) {
if (winfo->nrecords_per_request >= winfo->superTblInfo->insertRate) {
winfo->nrecords_per_request = winfo->superTblInfo->insertRate;
}
}
if (winfo->nrecords_per_request <= 0) {
winfo->nrecords_per_request = 1;
}
if (winfo->nrecords_per_request >= INT16_MAX) {
winfo->nrecords_per_request = INT16_MAX - 1;
}
if (winfo->nrecords_per_request >= INT16_MAX) {
winfo->nrecords_per_request = INT16_MAX - 1;
}
winfo->st = 0; winfo->st = 0;
winfo->et = 0; winfo->et = 0;
winfo->lastTs = winfo->start_time; winfo->lastTs = winfo->start_time;
if (winfo->superTblInfo->insertRate) { if (g_args.insert_interval) {
winfo->st = taosGetTimestampMs(); winfo->st = taosGetTimestampMs();
} }
taos_query_a(winfo->taos, "show databases", callBack, winfo); taos_query_a(winfo->taos, "show databases", callBack, winfo);
...@@ -4065,23 +4507,30 @@ void *asyncWrite(void *sarg) { ...@@ -4065,23 +4507,30 @@ void *asyncWrite(void *sarg) {
return NULL; return NULL;
} }
void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSuperTable* superTblInfo) { void startMultiThreadInsertData(int threads, char* db_name, char* precision,
pthread_t *pids = malloc(threads * sizeof(pthread_t)); SSuperTable* superTblInfo) {
threadInfo *infos = malloc(threads * sizeof(threadInfo));
memset(pids, 0, threads * sizeof(pthread_t));
memset(infos, 0, threads * sizeof(threadInfo));
int ntables = superTblInfo->childTblCount;
int a = ntables / threads; pthread_t *pids = malloc(threads * sizeof(pthread_t));
if (a < 1) { threadInfo *infos = malloc(threads * sizeof(threadInfo));
threads = ntables; memset(pids, 0, threads * sizeof(pthread_t));
a = 1; memset(infos, 0, threads * sizeof(threadInfo));
}
int b = 0; int ntables = 0;
if (threads != 0) { if (superTblInfo)
b = ntables % threads; ntables = superTblInfo->childTblCount;
} else
ntables = g_args.num_of_tables;
int a = ntables / threads;
if (a < 1) {
threads = ntables;
a = 1;
}
int b = 0;
if (threads != 0) {
b = ntables % threads;
}
//TAOS* taos; //TAOS* taos;
//if (0 == strncasecmp(superTblInfo->insertMode, "taosc", 5)) { //if (0 == strncasecmp(superTblInfo->insertMode, "taosc", 5)) {
...@@ -4104,11 +4553,19 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu ...@@ -4104,11 +4553,19 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu
} }
} }
int64_t start_time; int64_t start_time;
if (0 == strncasecmp(superTblInfo->startTimestamp, "now", 3)) { if (superTblInfo) {
start_time = taosGetTimestamp(timePrec); if (0 == strncasecmp(superTblInfo->startTimestamp, "now", 3)) {
} else { start_time = taosGetTimestamp(timePrec);
(void)taosParseTime(superTblInfo->startTimestamp, &start_time, strlen(superTblInfo->startTimestamp), timePrec, 0); } else {
taosParseTime(
superTblInfo->startTimestamp,
&start_time,
strlen(superTblInfo->startTimestamp),
timePrec, 0);
}
} else {
start_time = 1500000000000;
} }
double start = getCurrentTime(); double start = getCurrentTime();
...@@ -4123,18 +4580,23 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu ...@@ -4123,18 +4580,23 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu
t_info->start_time = start_time; t_info->start_time = start_time;
t_info->minDelay = INT16_MAX; t_info->minDelay = INT16_MAX;
if (0 == strncasecmp(superTblInfo->insertMode, "taosc", 5)) { if ((NULL == superTblInfo) ||
(0 == strncasecmp(superTblInfo->insertMode, "taosc", 5))) {
//t_info->taos = taos; //t_info->taos = taos;
t_info->taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, db_name, g_Dbs.port); t_info->taos = taos_connect(
g_Dbs.host, g_Dbs.user,
g_Dbs.password, db_name, g_Dbs.port);
if (NULL == t_info->taos) { if (NULL == t_info->taos) {
printf("connect to server fail from insert sub thread, reason: %s\n", taos_errstr(NULL)); printf("connect to server fail from insert sub thread, reason: %s\n",
taos_errstr(NULL));
exit(-1); exit(-1);
} }
} else { } else {
t_info->taos = NULL; t_info->taos = NULL;
} }
if (0 == superTblInfo->multiThreadWriteOneTbl) { if ((NULL == superTblInfo)
|| (0 == superTblInfo->multiThreadWriteOneTbl)) {
t_info->start_table_id = last; t_info->start_table_id = last;
t_info->end_table_id = i < b ? last + a : last + a - 1; t_info->end_table_id = i < b ? last + a : last + a - 1;
last = t_info->end_table_id + 1; last = t_info->end_table_id + 1;
...@@ -4145,9 +4607,12 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu ...@@ -4145,9 +4607,12 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu
} }
tsem_init(&(t_info->lock_sem), 0, 0); tsem_init(&(t_info->lock_sem), 0, 0);
if (SYNC == g_Dbs.queryMode) { if (SYNC == g_Dbs.queryMode) {
pthread_create(pids + i, NULL, syncWrite, t_info); if (superTblInfo) {
pthread_create(pids + i, NULL, syncWriteWithStb, t_info);
} else {
pthread_create(pids + i, NULL, syncWrite, t_info);
}
} else { } else {
pthread_create(pids + i, NULL, asyncWrite, t_info); pthread_create(pids + i, NULL, asyncWrite, t_info);
} }
...@@ -4169,13 +4634,15 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu ...@@ -4169,13 +4634,15 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu
tsem_destroy(&(t_info->lock_sem)); tsem_destroy(&(t_info->lock_sem));
taos_close(t_info->taos); taos_close(t_info->taos);
superTblInfo->totalAffectedRows += t_info->totalAffectedRows; if (superTblInfo) {
superTblInfo->totalRowsInserted += t_info->totalRowsInserted; superTblInfo->totalAffectedRows += t_info->totalAffectedRows;
superTblInfo->totalRowsInserted += t_info->totalRowsInserted;
totalDelay += t_info->totalDelay; totalDelay += t_info->totalDelay;
cntDelay += t_info->cntDelay; cntDelay += t_info->cntDelay;
if (t_info->maxDelay > maxDelay) maxDelay = t_info->maxDelay; if (t_info->maxDelay > maxDelay) maxDelay = t_info->maxDelay;
if (t_info->minDelay < minDelay) minDelay = t_info->minDelay; if (t_info->minDelay < minDelay) minDelay = t_info->minDelay;
}
} }
cntDelay -= 1; cntDelay -= 1;
...@@ -4184,23 +4651,29 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu ...@@ -4184,23 +4651,29 @@ void startMultiThreadInsertData(int threads, char* db_name, char* precision, SSu
double end = getCurrentTime(); double end = getCurrentTime();
double t = end - start; double t = end - start;
printf("Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s) into %s.%s. %2.f records/second\n\n",
t, superTblInfo->totalRowsInserted, superTblInfo->totalAffectedRows, threads, db_name, superTblInfo->sTblName, superTblInfo->totalRowsInserted / t);
fprintf(g_fpOfInsertResult, "Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s) into %s.%s. %2.f records/second\n\n",
t, superTblInfo->totalRowsInserted, superTblInfo->totalAffectedRows, threads, db_name, superTblInfo->sTblName, superTblInfo->totalRowsInserted / t);
if (superTblInfo) {
printf("Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s) into %s.%s. %2.f records/second\n\n",
t, superTblInfo->totalRowsInserted,
superTblInfo->totalAffectedRows,
threads, db_name, superTblInfo->sTblName,
superTblInfo->totalRowsInserted / t);
fprintf(g_fpOfInsertResult, "Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s) into %s.%s. %2.f records/second\n\n",
t, superTblInfo->totalRowsInserted,
superTblInfo->totalAffectedRows,
threads, db_name, superTblInfo->sTblName,
superTblInfo->totalRowsInserted / t);
}
printf("insert delay, avg: %10.6fms, max: %10.6fms, min: %10.6fms\n\n", printf("insert delay, avg: %10.6fms, max: %10.6fms, min: %10.6fms\n\n",
avgDelay/1000.0, (double)maxDelay/1000.0, (double)minDelay/1000.0); avgDelay/1000.0, (double)maxDelay/1000.0, (double)minDelay/1000.0);
fprintf(g_fpOfInsertResult, "insert delay, avg:%10.6fms, max: %10.6fms, min: %10.6fms\n\n", fprintf(g_fpOfInsertResult, "insert delay, avg:%10.6fms, max: %10.6fms, min: %10.6fms\n\n",
avgDelay/1000.0, (double)maxDelay/1000.0, (double)minDelay/1000.0); avgDelay/1000.0, (double)maxDelay/1000.0, (double)minDelay/1000.0);
//taos_close(taos); //taos_close(taos);
free(pids); free(pids);
free(infos); free(infos);
} }
...@@ -4209,7 +4682,7 @@ void *readTable(void *sarg) { ...@@ -4209,7 +4682,7 @@ void *readTable(void *sarg) {
threadInfo *rinfo = (threadInfo *)sarg; threadInfo *rinfo = (threadInfo *)sarg;
TAOS *taos = rinfo->taos; TAOS *taos = rinfo->taos;
char command[BUFFER_SIZE] = "\0"; char command[BUFFER_SIZE] = "\0";
int64_t sTime = rinfo->start_time; uint64_t sTime = rinfo->start_time;
char *tb_prefix = rinfo->tb_prefix; char *tb_prefix = rinfo->tb_prefix;
FILE *fp = fopen(rinfo->fp, "a"); FILE *fp = fopen(rinfo->fp, "a");
if (NULL == fp) { if (NULL == fp) {
...@@ -4217,7 +4690,13 @@ void *readTable(void *sarg) { ...@@ -4217,7 +4690,13 @@ void *readTable(void *sarg) {
return NULL; return NULL;
} }
int num_of_DPT = rinfo->superTblInfo->insertRows; // nrecords_per_table; int num_of_DPT;
if (rinfo->superTblInfo) {
num_of_DPT = rinfo->superTblInfo->insertRows; // nrecords_per_table;
} else {
num_of_DPT = g_args.num_of_DPT;
}
int num_of_tables = rinfo->end_table_id - rinfo->start_table_id + 1; int num_of_tables = rinfo->end_table_id - rinfo->start_table_id + 1;
int totalData = num_of_DPT * num_of_tables; int totalData = num_of_DPT * num_of_tables;
bool do_aggreFunc = g_Dbs.do_aggreFunc; bool do_aggreFunc = g_Dbs.do_aggreFunc;
...@@ -4343,19 +4822,21 @@ void *readMetric(void *sarg) { ...@@ -4343,19 +4822,21 @@ void *readMetric(void *sarg) {
int insertTestProcess() { int insertTestProcess() {
g_fpOfInsertResult = fopen(g_Dbs.resultFile, "a");
if (NULL == g_fpOfInsertResult) {
fprintf(stderr, "Failed to open %s for save result\n", g_Dbs.resultFile);
return 1;
};
setupForAnsiEscape(); setupForAnsiEscape();
int ret = printfInsertMeta(); int ret = printfInsertMeta();
resetAfterAnsiEscape(); resetAfterAnsiEscape();
if (ret == -1) if (ret == -1)
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
printfInsertMetaToFile(g_fpOfInsertResult); debugPrint("DEBUG - %d result file: %s\n", __LINE__, g_Dbs.resultFile);
g_fpOfInsertResult = fopen(g_Dbs.resultFile, "a");
if (NULL == g_fpOfInsertResult) {
fprintf(stderr, "Failed to open %s for save result\n", g_Dbs.resultFile);
return -1;
} {
printfInsertMetaToFile(g_fpOfInsertResult);
}
if (!g_args.answer_yes) { if (!g_args.answer_yes) {
printf("Press enter key to continue\n\n"); printf("Press enter key to continue\n\n");
...@@ -4365,11 +4846,14 @@ int insertTestProcess() { ...@@ -4365,11 +4846,14 @@ int insertTestProcess() {
init_rand_data(); init_rand_data();
// create database and super tables // create database and super tables
(void)createDatabases(); if(createDatabases() != 0) {
fclose(g_fpOfInsertResult);
return -1;
}
// pretreatement // pretreatement
prePareSampleData(); prePareSampleData();
double start; double start;
double end; double end;
...@@ -4377,26 +4861,41 @@ int insertTestProcess() { ...@@ -4377,26 +4861,41 @@ int insertTestProcess() {
start = getCurrentTime(); start = getCurrentTime();
createChildTables(); createChildTables();
end = getCurrentTime(); end = getCurrentTime();
if (g_totalChildTables > 0) { if (g_totalChildTables > 0) {
printf("Spent %.4f seconds to create %d tables with %d thread(s)\n\n", end - start, g_totalChildTables, g_Dbs.threadCount); printf("Spent %.4f seconds to create %d tables with %d thread(s)\n\n",
fprintf(g_fpOfInsertResult, "Spent %.4f seconds to create %d tables with %d thread(s)\n\n", end - start, g_totalChildTables, g_Dbs.threadCount); end - start, g_totalChildTables, g_Dbs.threadCount);
fprintf(g_fpOfInsertResult,
"Spent %.4f seconds to create %d tables with %d thread(s)\n\n",
end - start, g_totalChildTables, g_Dbs.threadCount);
} }
taosMsleep(1000);
taosMsleep(1000);
// create sub threads for inserting data // create sub threads for inserting data
//start = getCurrentTime(); //start = getCurrentTime();
for (int i = 0; i < g_Dbs.dbCount; i++) { for (int i = 0; i < g_Dbs.dbCount; i++) {
for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { if (g_Dbs.db[i].superTblCount > 0) {
SSuperTable* superTblInfo = &g_Dbs.db[i].superTbls[j]; for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) {
if (0 == g_Dbs.db[i].superTbls[j].insertRows) { SSuperTable* superTblInfo = &g_Dbs.db[i].superTbls[j];
continue; if (0 == g_Dbs.db[i].superTbls[j].insertRows) {
} continue;
startMultiThreadInsertData(g_Dbs.threadCount, g_Dbs.db[i].dbName, g_Dbs.db[i].dbCfg.precision, superTblInfo); }
startMultiThreadInsertData(
g_Dbs.threadCount,
g_Dbs.db[i].dbName,
g_Dbs.db[i].dbCfg.precision,
superTblInfo);
}
} else {
startMultiThreadInsertData(
g_Dbs.threadCount,
g_Dbs.db[i].dbName,
g_Dbs.db[i].dbCfg.precision,
NULL);
}
} }
}
//end = getCurrentTime(); //end = getCurrentTime();
//int64_t totalRowsInserted = 0; //int64_t totalRowsInserted = 0;
//int64_t totalAffectedRows = 0; //int64_t totalAffectedRows = 0;
//for (int i = 0; i < g_Dbs.dbCount; i++) { //for (int i = 0; i < g_Dbs.dbCount; i++) {
...@@ -4405,31 +4904,8 @@ int insertTestProcess() { ...@@ -4405,31 +4904,8 @@ int insertTestProcess() {
// totalAffectedRows += g_Dbs.db[i].superTbls[j].totalAffectedRows; // totalAffectedRows += g_Dbs.db[i].superTbls[j].totalAffectedRows;
//} //}
//printf("Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s)\n\n", end - start, totalRowsInserted, totalAffectedRows, g_Dbs.threadCount); //printf("Spent %.4f seconds to insert rows: %"PRId64", affected rows: %"PRId64" with %d thread(s)\n\n", end - start, totalRowsInserted, totalAffectedRows, g_Dbs.threadCount);
if (NULL == g_args.metaFile && false == g_Dbs.insert_only) {
// query data
pthread_t read_id;
threadInfo *rInfo = malloc(sizeof(threadInfo));
rInfo->start_time = 1500000000000; // 2017-07-14 10:40:00.000
rInfo->start_table_id = 0;
rInfo->end_table_id = g_Dbs.db[0].superTbls[0].childTblCount - 1;
//rInfo->do_aggreFunc = g_Dbs.do_aggreFunc;
//rInfo->nrecords_per_table = g_Dbs.db[0].superTbls[0].insertRows;
rInfo->superTblInfo = &g_Dbs.db[0].superTbls[0];
rInfo->taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, g_Dbs.db[0].dbName, g_Dbs.port);
strcpy(rInfo->tb_prefix, g_Dbs.db[0].superTbls[0].childTblPrefix);
strcpy(rInfo->fp, g_Dbs.resultFile);
if (!g_Dbs.use_metric) {
pthread_create(&read_id, NULL, readTable, rInfo);
} else {
pthread_create(&read_id, NULL, readMetric, rInfo);
}
pthread_join(read_id, NULL);
taos_close(rInfo->taos);
}
postFreeResource(); postFreeResource();
return 0; return 0;
} }
...@@ -4458,12 +4934,15 @@ void *superQueryProcess(void *sarg) { ...@@ -4458,12 +4934,15 @@ void *superQueryProcess(void *sarg) {
} }
selectAndGetResult(winfo->taos, g_queryInfo.superQueryInfo.sql[i], tmpFile); selectAndGetResult(winfo->taos, g_queryInfo.superQueryInfo.sql[i], tmpFile);
int64_t t2 = taosGetTimestampUs(); int64_t t2 = taosGetTimestampUs();
printf("=[taosc] thread[%"PRId64"] complete one sql, Spent %f s\n", taosGetSelfPthreadId(), (t2 - t1)/1000000.0); printf("=[taosc] thread[%"PRId64"] complete one sql, Spent %f s\n",
taosGetSelfPthreadId(), (t2 - t1)/1000000.0);
} else { } else {
int64_t t1 = taosGetTimestampUs(); int64_t t1 = taosGetTimestampUs();
int retCode = postProceSql(g_queryInfo.host, g_queryInfo.port, g_queryInfo.superQueryInfo.sql[i]); int retCode = postProceSql(g_queryInfo.host,
g_queryInfo.port, g_queryInfo.superQueryInfo.sql[i]);
int64_t t2 = taosGetTimestampUs(); int64_t t2 = taosGetTimestampUs();
printf("=[restful] thread[%"PRId64"] complete one sql, Spent %f s\n", taosGetSelfPthreadId(), (t2 - t1)/1000000.0); printf("=[restful] thread[%"PRId64"] complete one sql, Spent %f s\n",
taosGetSelfPthreadId(), (t2 - t1)/1000000.0);
if (0 != retCode) { if (0 != retCode) {
printf("====restful return fail, threadID[%d]\n", winfo->threadID); printf("====restful return fail, threadID[%d]\n", winfo->threadID);
...@@ -4472,7 +4951,8 @@ void *superQueryProcess(void *sarg) { ...@@ -4472,7 +4951,8 @@ void *superQueryProcess(void *sarg) {
} }
} }
et = taosGetTimestampMs(); et = taosGetTimestampMs();
printf("==thread[%"PRId64"] complete all sqls to specify tables once queries duration:%.6fs\n\n", taosGetSelfPthreadId(), (double)(et - st)/1000.0); printf("==thread[%"PRId64"] complete all sqls to specify tables once queries duration:%.6fs\n\n",
taosGetSelfPthreadId(), (double)(et - st)/1000.0);
} }
return NULL; return NULL;
} }
...@@ -4480,7 +4960,9 @@ void *superQueryProcess(void *sarg) { ...@@ -4480,7 +4960,9 @@ void *superQueryProcess(void *sarg) {
void replaceSubTblName(char* inSql, char* outSql, int tblIndex) { void replaceSubTblName(char* inSql, char* outSql, int tblIndex) {
char sourceString[32] = "xxxx"; char sourceString[32] = "xxxx";
char subTblName[MAX_TB_NAME_SIZE*3]; char subTblName[MAX_TB_NAME_SIZE*3];
sprintf(subTblName, "%s.%s", g_queryInfo.dbName, g_queryInfo.subQueryInfo.childTblName + tblIndex*TSDB_TABLE_NAME_LEN); sprintf(subTblName, "%s.%s",
g_queryInfo.dbName,
g_queryInfo.subQueryInfo.childTblName + tblIndex*TSDB_TABLE_NAME_LEN);
//printf("inSql: %s\n", inSql); //printf("inSql: %s\n", inSql);
...@@ -4514,28 +4996,42 @@ void *subQueryProcess(void *sarg) { ...@@ -4514,28 +4996,42 @@ void *subQueryProcess(void *sarg) {
memset(sqlstr,0,sizeof(sqlstr)); memset(sqlstr,0,sizeof(sqlstr));
replaceSubTblName(g_queryInfo.subQueryInfo.sql[j], sqlstr, i); replaceSubTblName(g_queryInfo.subQueryInfo.sql[j], sqlstr, i);
char tmpFile[MAX_FILE_NAME_LEN*2] = {0}; char tmpFile[MAX_FILE_NAME_LEN*2] = {0};
if (g_queryInfo.subQueryInfo.result[j][0] != 0) { if (g_queryInfo.subQueryInfo.result[i][0] != 0) {
sprintf(tmpFile, "%s-%d", g_queryInfo.subQueryInfo.result[j], winfo->threadID); sprintf(tmpFile, "%s-%d",
g_queryInfo.subQueryInfo.result[i],
winfo->threadID);
} }
selectAndGetResult(winfo->taos, sqlstr, tmpFile); selectAndGetResult(winfo->taos, sqlstr, tmpFile);
} }
} }
et = taosGetTimestampMs(); et = taosGetTimestampMs();
printf("####thread[%"PRId64"] complete all sqls to allocate all sub-tables[%d - %d] once queries duration:%.4fs\n\n", taosGetSelfPthreadId(), winfo->start_table_id, winfo->end_table_id, (double)(et - st)/1000.0); printf("####thread[%"PRId64"] complete all sqls to allocate all sub-tables[%d - %d] once queries duration:%.4fs\n\n",
taosGetSelfPthreadId(),
winfo->start_table_id,
winfo->end_table_id,
(double)(et - st)/1000.0);
} }
return NULL; return NULL;
} }
int queryTestProcess() { static int queryTestProcess() {
TAOS * taos = NULL; TAOS * taos = NULL;
taos = taos_connect(g_queryInfo.host, g_queryInfo.user, g_queryInfo.password, NULL, g_queryInfo.port); taos = taos_connect(g_queryInfo.host,
g_queryInfo.user,
g_queryInfo.password,
NULL,
g_queryInfo.port);
if (taos == NULL) { if (taos == NULL) {
fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL)); fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL));
exit(-1); exit(-1);
} }
if (0 != g_queryInfo.subQueryInfo.sqlCount) { if (0 != g_queryInfo.subQueryInfo.sqlCount) {
(void)getAllChildNameOfSuperTable(taos, g_queryInfo.dbName, g_queryInfo.subQueryInfo.sTblName, &g_queryInfo.subQueryInfo.childTblName, &g_queryInfo.subQueryInfo.childTblCount); (void)getAllChildNameOfSuperTable(taos,
g_queryInfo.dbName,
g_queryInfo.subQueryInfo.sTblName,
&g_queryInfo.subQueryInfo.childTblName,
&g_queryInfo.subQueryInfo.childTblCount);
} }
printfQueryMeta(); printfQueryMeta();
...@@ -4569,6 +5065,7 @@ int queryTestProcess() { ...@@ -4569,6 +5065,7 @@ int queryTestProcess() {
char sqlStr[MAX_TB_NAME_SIZE*2]; char sqlStr[MAX_TB_NAME_SIZE*2];
sprintf(sqlStr, "use %s", g_queryInfo.dbName); sprintf(sqlStr, "use %s", g_queryInfo.dbName);
debugPrint("DEBUG %s() %d sqlStr: %s\n", __func__, __LINE__, sqlStr);
(void)queryDbExec(t_info->taos, sqlStr, NO_INSERT_TYPE); (void)queryDbExec(t_info->taos, sqlStr, NO_INSERT_TYPE);
} else { } else {
t_info->taos = NULL; t_info->taos = NULL;
...@@ -4655,9 +5152,14 @@ static TAOS_SUB* subscribeImpl(TAOS *taos, char *sql, char* topic, char* resultF ...@@ -4655,9 +5152,14 @@ static TAOS_SUB* subscribeImpl(TAOS *taos, char *sql, char* topic, char* resultF
TAOS_SUB* tsub = NULL; TAOS_SUB* tsub = NULL;
if (g_queryInfo.superQueryInfo.subscribeMode) { if (g_queryInfo.superQueryInfo.subscribeMode) {
tsub = taos_subscribe(taos, g_queryInfo.superQueryInfo.subscribeRestart, topic, sql, subscribe_callback, (void*)resultFileName, g_queryInfo.superQueryInfo.subscribeInterval); tsub = taos_subscribe(taos,
g_queryInfo.superQueryInfo.subscribeRestart,
topic, sql, subscribe_callback, (void*)resultFileName,
g_queryInfo.superQueryInfo.subscribeInterval);
} else { } else {
tsub = taos_subscribe(taos, g_queryInfo.superQueryInfo.subscribeRestart, topic, sql, NULL, NULL, 0); tsub = taos_subscribe(taos,
g_queryInfo.superQueryInfo.subscribeRestart,
topic, sql, NULL, NULL, 0);
} }
if (tsub == NULL) { if (tsub == NULL) {
...@@ -4674,6 +5176,7 @@ void *subSubscribeProcess(void *sarg) { ...@@ -4674,6 +5176,7 @@ void *subSubscribeProcess(void *sarg) {
char sqlStr[MAX_TB_NAME_SIZE*2]; char sqlStr[MAX_TB_NAME_SIZE*2];
sprintf(sqlStr, "use %s", g_queryInfo.dbName); sprintf(sqlStr, "use %s", g_queryInfo.dbName);
debugPrint("DEBUG %s() %d sqlStr: %s\n", __func__, __LINE__, sqlStr);
if (0 != queryDbExec(winfo->taos, sqlStr, NO_INSERT_TYPE)){ if (0 != queryDbExec(winfo->taos, sqlStr, NO_INSERT_TYPE)){
return NULL; return NULL;
} }
...@@ -4717,7 +5220,9 @@ void *subSubscribeProcess(void *sarg) { ...@@ -4717,7 +5220,9 @@ void *subSubscribeProcess(void *sarg) {
if (res) { if (res) {
char tmpFile[MAX_FILE_NAME_LEN*2] = {0}; char tmpFile[MAX_FILE_NAME_LEN*2] = {0};
if (g_queryInfo.subQueryInfo.result[i][0] != 0) { if (g_queryInfo.subQueryInfo.result[i][0] != 0) {
sprintf(tmpFile, "%s-%d", g_queryInfo.subQueryInfo.result[i], winfo->threadID); sprintf(tmpFile, "%s-%d",
g_queryInfo.subQueryInfo.result[i],
winfo->threadID);
} }
getResult(res, tmpFile); getResult(res, tmpFile);
} }
...@@ -4726,7 +5231,8 @@ void *subSubscribeProcess(void *sarg) { ...@@ -4726,7 +5231,8 @@ void *subSubscribeProcess(void *sarg) {
taos_free_result(res); taos_free_result(res);
for (int i = 0; i < g_queryInfo.subQueryInfo.sqlCount; i++) { for (int i = 0; i < g_queryInfo.subQueryInfo.sqlCount; i++) {
taos_unsubscribe(g_queryInfo.subQueryInfo.tsub[i], g_queryInfo.subQueryInfo.subscribeKeepProgress); taos_unsubscribe(g_queryInfo.subQueryInfo.tsub[i],
g_queryInfo.subQueryInfo.subscribeKeepProgress);
} }
return NULL; return NULL;
} }
...@@ -4736,6 +5242,7 @@ void *superSubscribeProcess(void *sarg) { ...@@ -4736,6 +5242,7 @@ void *superSubscribeProcess(void *sarg) {
char sqlStr[MAX_TB_NAME_SIZE*2]; char sqlStr[MAX_TB_NAME_SIZE*2];
sprintf(sqlStr, "use %s", g_queryInfo.dbName); sprintf(sqlStr, "use %s", g_queryInfo.dbName);
debugPrint("DEBUG %s() %d sqlStr: %s\n", __func__, __LINE__, sqlStr);
if (0 != queryDbExec(winfo->taos, sqlStr, NO_INSERT_TYPE)) { if (0 != queryDbExec(winfo->taos, sqlStr, NO_INSERT_TYPE)) {
return NULL; return NULL;
} }
...@@ -4754,9 +5261,13 @@ void *superSubscribeProcess(void *sarg) { ...@@ -4754,9 +5261,13 @@ void *superSubscribeProcess(void *sarg) {
sprintf(topic, "taosdemo-subscribe-%d", i); sprintf(topic, "taosdemo-subscribe-%d", i);
char tmpFile[MAX_FILE_NAME_LEN*2] = {0}; char tmpFile[MAX_FILE_NAME_LEN*2] = {0};
if (g_queryInfo.subQueryInfo.result[i][0] != 0) { if (g_queryInfo.subQueryInfo.result[i][0] != 0) {
sprintf(tmpFile, "%s-%d", g_queryInfo.superQueryInfo.result[i], winfo->threadID); sprintf(tmpFile, "%s-%d",
g_queryInfo.superQueryInfo.result[i], winfo->threadID);
} }
g_queryInfo.superQueryInfo.tsub[i] = subscribeImpl(winfo->taos, g_queryInfo.superQueryInfo.sql[i], topic, tmpFile); g_queryInfo.superQueryInfo.tsub[i] =
subscribeImpl(winfo->taos,
g_queryInfo.superQueryInfo.sql[i],
topic, tmpFile);
if (NULL == g_queryInfo.superQueryInfo.tsub[i]) { if (NULL == g_queryInfo.superQueryInfo.tsub[i]) {
return NULL; return NULL;
} }
...@@ -4777,7 +5288,8 @@ void *superSubscribeProcess(void *sarg) { ...@@ -4777,7 +5288,8 @@ void *superSubscribeProcess(void *sarg) {
if (res) { if (res) {
char tmpFile[MAX_FILE_NAME_LEN*2] = {0}; char tmpFile[MAX_FILE_NAME_LEN*2] = {0};
if (g_queryInfo.superQueryInfo.result[i][0] != 0) { if (g_queryInfo.superQueryInfo.result[i][0] != 0) {
sprintf(tmpFile, "%s-%d", g_queryInfo.superQueryInfo.result[i], winfo->threadID); sprintf(tmpFile, "%s-%d",
g_queryInfo.superQueryInfo.result[i], winfo->threadID);
} }
getResult(res, tmpFile); getResult(res, tmpFile);
} }
...@@ -4786,12 +5298,13 @@ void *superSubscribeProcess(void *sarg) { ...@@ -4786,12 +5298,13 @@ void *superSubscribeProcess(void *sarg) {
taos_free_result(res); taos_free_result(res);
for (int i = 0; i < g_queryInfo.superQueryInfo.sqlCount; i++) { for (int i = 0; i < g_queryInfo.superQueryInfo.sqlCount; i++) {
taos_unsubscribe(g_queryInfo.superQueryInfo.tsub[i], g_queryInfo.superQueryInfo.subscribeKeepProgress); taos_unsubscribe(g_queryInfo.superQueryInfo.tsub[i],
g_queryInfo.superQueryInfo.subscribeKeepProgress);
} }
return NULL; return NULL;
} }
int subscribeTestProcess() { static int subscribeTestProcess() {
printfQueryMeta(); printfQueryMeta();
if (!g_args.answer_yes) { if (!g_args.answer_yes) {
...@@ -4800,21 +5313,30 @@ int subscribeTestProcess() { ...@@ -4800,21 +5313,30 @@ int subscribeTestProcess() {
} }
TAOS * taos = NULL; TAOS * taos = NULL;
taos = taos_connect(g_queryInfo.host, g_queryInfo.user, g_queryInfo.password, g_queryInfo.dbName, g_queryInfo.port); taos = taos_connect(g_queryInfo.host,
g_queryInfo.user,
g_queryInfo.password,
g_queryInfo.dbName,
g_queryInfo.port);
if (taos == NULL) { if (taos == NULL) {
fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL)); fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL));
exit(-1); exit(-1);
} }
if (0 != g_queryInfo.subQueryInfo.sqlCount) { if (0 != g_queryInfo.subQueryInfo.sqlCount) {
(void)getAllChildNameOfSuperTable(taos, g_queryInfo.dbName, g_queryInfo.subQueryInfo.sTblName, &g_queryInfo.subQueryInfo.childTblName, &g_queryInfo.subQueryInfo.childTblCount); (void)getAllChildNameOfSuperTable(taos,
g_queryInfo.dbName,
g_queryInfo.subQueryInfo.sTblName,
&g_queryInfo.subQueryInfo.childTblName,
&g_queryInfo.subQueryInfo.childTblCount);
} }
pthread_t *pids = NULL; pthread_t *pids = NULL;
threadInfo *infos = NULL; threadInfo *infos = NULL;
//==== create sub threads for query from super table //==== create sub threads for query from super table
if (g_queryInfo.superQueryInfo.sqlCount > 0 && g_queryInfo.superQueryInfo.concurrent > 0) { if (g_queryInfo.superQueryInfo.sqlCount > 0
&& g_queryInfo.superQueryInfo.concurrent > 0) {
pids = malloc(g_queryInfo.superQueryInfo.concurrent * sizeof(pthread_t)); pids = malloc(g_queryInfo.superQueryInfo.concurrent * sizeof(pthread_t));
infos = malloc(g_queryInfo.superQueryInfo.concurrent * sizeof(threadInfo)); infos = malloc(g_queryInfo.superQueryInfo.concurrent * sizeof(threadInfo));
if ((NULL == pids) || (NULL == infos)) { if ((NULL == pids) || (NULL == infos)) {
...@@ -4834,9 +5356,12 @@ int subscribeTestProcess() { ...@@ -4834,9 +5356,12 @@ int subscribeTestProcess() {
//==== create sub threads for query from sub table //==== create sub threads for query from sub table
pthread_t *pidsOfSub = NULL; pthread_t *pidsOfSub = NULL;
threadInfo *infosOfSub = NULL; threadInfo *infosOfSub = NULL;
if ((g_queryInfo.subQueryInfo.sqlCount > 0) && (g_queryInfo.subQueryInfo.threadCnt > 0)) { if ((g_queryInfo.subQueryInfo.sqlCount > 0)
pidsOfSub = malloc(g_queryInfo.subQueryInfo.threadCnt * sizeof(pthread_t)); && (g_queryInfo.subQueryInfo.threadCnt > 0)) {
infosOfSub = malloc(g_queryInfo.subQueryInfo.threadCnt * sizeof(threadInfo)); pidsOfSub = malloc(g_queryInfo.subQueryInfo.threadCnt *
sizeof(pthread_t));
infosOfSub = malloc(g_queryInfo.subQueryInfo.threadCnt *
sizeof(threadInfo));
if ((NULL == pidsOfSub) || (NULL == infosOfSub)) { if ((NULL == pidsOfSub) || (NULL == infosOfSub)) {
printf("malloc failed for create threads\n"); printf("malloc failed for create threads\n");
taos_close(taos); taos_close(taos);
...@@ -4890,23 +5415,23 @@ int subscribeTestProcess() { ...@@ -4890,23 +5415,23 @@ int subscribeTestProcess() {
void initOfInsertMeta() { void initOfInsertMeta() {
memset(&g_Dbs, 0, sizeof(SDbs)); memset(&g_Dbs, 0, sizeof(SDbs));
// set default values // set default values
tstrncpy(g_Dbs.host, "127.0.0.1", MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.host, "127.0.0.1", MAX_DB_NAME_SIZE);
g_Dbs.port = 6030; g_Dbs.port = 6030;
tstrncpy(g_Dbs.user, TSDB_DEFAULT_USER, MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.user, TSDB_DEFAULT_USER, MAX_DB_NAME_SIZE);
tstrncpy(g_Dbs.password, TSDB_DEFAULT_PASS, MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.password, TSDB_DEFAULT_PASS, MAX_DB_NAME_SIZE);
g_Dbs.threadCount = 2; g_Dbs.threadCount = 2;
g_Dbs.use_metric = true; g_Dbs.use_metric = true;
} }
void initOfQueryMeta() { void initOfQueryMeta() {
memset(&g_queryInfo, 0, sizeof(SQueryMetaInfo)); memset(&g_queryInfo, 0, sizeof(SQueryMetaInfo));
// set default values // set default values
tstrncpy(g_queryInfo.host, "127.0.0.1", MAX_DB_NAME_SIZE); tstrncpy(g_queryInfo.host, "127.0.0.1", MAX_DB_NAME_SIZE);
g_queryInfo.port = 6030; g_queryInfo.port = 6030;
tstrncpy(g_queryInfo.user, TSDB_DEFAULT_USER, MAX_DB_NAME_SIZE); tstrncpy(g_queryInfo.user, TSDB_DEFAULT_USER, MAX_DB_NAME_SIZE);
tstrncpy(g_queryInfo.password, TSDB_DEFAULT_PASS, MAX_DB_NAME_SIZE); tstrncpy(g_queryInfo.password, TSDB_DEFAULT_PASS, MAX_DB_NAME_SIZE);
} }
void setParaFromArg(){ void setParaFromArg(){
...@@ -4928,42 +5453,21 @@ void setParaFromArg(){ ...@@ -4928,42 +5453,21 @@ void setParaFromArg(){
g_Dbs.port = g_args.port; g_Dbs.port = g_args.port;
} }
g_Dbs.threadCount = g_args.num_of_threads;
g_Dbs.threadCountByCreateTbl = g_args.num_of_threads;
g_Dbs.dbCount = 1; g_Dbs.dbCount = 1;
g_Dbs.db[0].drop = 1; g_Dbs.db[0].drop = 1;
tstrncpy(g_Dbs.db[0].dbName, g_args.database, MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[0].dbName, g_args.database, MAX_DB_NAME_SIZE);
g_Dbs.db[0].dbCfg.replica = g_args.replica; g_Dbs.db[0].dbCfg.replica = g_args.replica;
tstrncpy(g_Dbs.db[0].dbCfg.precision, "ms", MAX_DB_NAME_SIZE); tstrncpy(g_Dbs.db[0].dbCfg.precision, "ms", MAX_DB_NAME_SIZE);
tstrncpy(g_Dbs.resultFile, g_args.output_file, MAX_FILE_NAME_LEN); tstrncpy(g_Dbs.resultFile, g_args.output_file, MAX_FILE_NAME_LEN);
g_Dbs.use_metric = g_args.use_metric; g_Dbs.use_metric = g_args.use_metric;
g_Dbs.insert_only = g_args.insert_only; g_Dbs.insert_only = g_args.insert_only;
g_Dbs.db[0].superTblCount = 1;
tstrncpy(g_Dbs.db[0].superTbls[0].sTblName, "meters", MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].childTblCount = g_args.num_of_tables;
g_Dbs.threadCount = g_args.num_of_threads;
g_Dbs.threadCountByCreateTbl = 1;
g_Dbs.queryMode = g_args.mode;
g_Dbs.db[0].superTbls[0].autoCreateTable = PRE_CREATE_SUBTBL;
g_Dbs.db[0].superTbls[0].superTblExists = TBL_NO_EXISTS;
g_Dbs.db[0].superTbls[0].childTblExists = TBL_NO_EXISTS;
g_Dbs.db[0].superTbls[0].insertRate = 0;
g_Dbs.db[0].superTbls[0].disorderRange = g_args.disorderRange;
g_Dbs.db[0].superTbls[0].disorderRatio = g_args.disorderRatio;
tstrncpy(g_Dbs.db[0].superTbls[0].childTblPrefix, g_args.tb_prefix, MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].dataSource, "rand", MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].insertMode, "taosc", MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].startTimestamp, "2017-07-14 10:40:00.000", MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].timeStampStep = 10;
// g_args.num_of_RPR;
g_Dbs.db[0].superTbls[0].insertRows = g_args.num_of_DPT;
g_Dbs.db[0].superTbls[0].maxSqlLen = TSDB_PAYLOAD_SIZE;
g_Dbs.do_aggreFunc = true; g_Dbs.do_aggreFunc = true;
char dataString[STRING_LEN]; char dataString[STRING_LEN];
...@@ -4971,32 +5475,58 @@ void setParaFromArg(){ ...@@ -4971,32 +5475,58 @@ void setParaFromArg(){
memset(dataString, 0, STRING_LEN); memset(dataString, 0, STRING_LEN);
if (strcasecmp(data_type[0], "BINARY") == 0 || strcasecmp(data_type[0], "BOOL") == 0 || strcasecmp(data_type[0], "NCHAR") == 0 ) { if (strcasecmp(data_type[0], "BINARY") == 0
|| strcasecmp(data_type[0], "BOOL") == 0
|| strcasecmp(data_type[0], "NCHAR") == 0 ) {
g_Dbs.do_aggreFunc = false; g_Dbs.do_aggreFunc = false;
} }
g_Dbs.db[0].superTbls[0].columnCount = 0; if (g_args.use_metric) {
for (int i = 0; i < MAX_NUM_DATATYPE; i++) { g_Dbs.db[0].superTblCount = 1;
if (data_type[i] == NULL) { tstrncpy(g_Dbs.db[0].superTbls[0].sTblName, "meters", MAX_TB_NAME_SIZE);
break; g_Dbs.db[0].superTbls[0].childTblCount = g_args.num_of_tables;
} g_Dbs.threadCount = g_args.num_of_threads;
g_Dbs.threadCountByCreateTbl = 1;
tstrncpy(g_Dbs.db[0].superTbls[0].columns[i].dataType, data_type[i], MAX_TB_NAME_SIZE); g_Dbs.queryMode = g_args.mode;
g_Dbs.db[0].superTbls[0].columns[i].dataLen = g_args.len_of_binary;
g_Dbs.db[0].superTbls[0].columnCount++; g_Dbs.db[0].superTbls[0].autoCreateTable = PRE_CREATE_SUBTBL;
} g_Dbs.db[0].superTbls[0].superTblExists = TBL_NO_EXISTS;
g_Dbs.db[0].superTbls[0].childTblExists = TBL_NO_EXISTS;
g_Dbs.db[0].superTbls[0].disorderRange = g_args.disorderRange;
g_Dbs.db[0].superTbls[0].disorderRatio = g_args.disorderRatio;
tstrncpy(g_Dbs.db[0].superTbls[0].childTblPrefix,
g_args.tb_prefix, MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].dataSource, "rand", MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].insertMode, "taosc", MAX_TB_NAME_SIZE);
tstrncpy(g_Dbs.db[0].superTbls[0].startTimestamp,
"2017-07-14 10:40:00.000", MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].timeStampStep = 10;
g_Dbs.db[0].superTbls[0].insertRows = g_args.num_of_DPT;
g_Dbs.db[0].superTbls[0].maxSqlLen = TSDB_PAYLOAD_SIZE;
if (g_Dbs.db[0].superTbls[0].columnCount > g_args.num_of_CPR) { g_Dbs.db[0].superTbls[0].columnCount = 0;
g_Dbs.db[0].superTbls[0].columnCount = g_args.num_of_CPR; for (int i = 0; i < MAX_NUM_DATATYPE; i++) {
} else { if (data_type[i] == NULL) {
for (int i = g_Dbs.db[0].superTbls[0].columnCount; i < g_args.num_of_CPR; i++) { break;
tstrncpy(g_Dbs.db[0].superTbls[0].columns[i].dataType, "INT", MAX_TB_NAME_SIZE); }
g_Dbs.db[0].superTbls[0].columns[i].dataLen = 0;
tstrncpy(g_Dbs.db[0].superTbls[0].columns[i].dataType,
data_type[i], MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].columns[i].dataLen = g_args.len_of_binary;
g_Dbs.db[0].superTbls[0].columnCount++; g_Dbs.db[0].superTbls[0].columnCount++;
} }
}
if (g_Dbs.db[0].superTbls[0].columnCount > g_args.num_of_CPR) {
g_Dbs.db[0].superTbls[0].columnCount = g_args.num_of_CPR;
} else {
for (int i = g_Dbs.db[0].superTbls[0].columnCount; i < g_args.num_of_CPR; i++) {
tstrncpy(g_Dbs.db[0].superTbls[0].columns[i].dataType, "INT", MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].columns[i].dataLen = 0;
g_Dbs.db[0].superTbls[0].columnCount++;
}
}
if (g_Dbs.use_metric) {
tstrncpy(g_Dbs.db[0].superTbls[0].tags[0].dataType, "INT", MAX_TB_NAME_SIZE); tstrncpy(g_Dbs.db[0].superTbls[0].tags[0].dataType, "INT", MAX_TB_NAME_SIZE);
g_Dbs.db[0].superTbls[0].tags[0].dataLen = 0; g_Dbs.db[0].superTbls[0].tags[0].dataLen = 0;
...@@ -5004,8 +5534,10 @@ void setParaFromArg(){ ...@@ -5004,8 +5534,10 @@ void setParaFromArg(){
g_Dbs.db[0].superTbls[0].tags[1].dataLen = g_args.len_of_binary; g_Dbs.db[0].superTbls[0].tags[1].dataLen = g_args.len_of_binary;
g_Dbs.db[0].superTbls[0].tagCount = 2; g_Dbs.db[0].superTbls[0].tagCount = 2;
} else { } else {
g_Dbs.threadCountByCreateTbl = 1;
g_Dbs.db[0].superTbls[0].tagCount = 0; g_Dbs.db[0].superTbls[0].tagCount = 0;
} }
} }
/* Function to do regular expression check */ /* Function to do regular expression check */
...@@ -5050,7 +5582,7 @@ void querySqlFile(TAOS* taos, char* sqlFile) ...@@ -5050,7 +5582,7 @@ void querySqlFile(TAOS* taos, char* sqlFile)
printf("failed to open file %s, reason:%s\n", sqlFile, strerror(errno)); printf("failed to open file %s, reason:%s\n", sqlFile, strerror(errno));
return; return;
} }
int read_len = 0; int read_len = 0;
char * cmd = calloc(1, MAX_SQL_SIZE); char * cmd = calloc(1, MAX_SQL_SIZE);
size_t cmd_len = 0; size_t cmd_len = 0;
...@@ -5058,7 +5590,7 @@ void querySqlFile(TAOS* taos, char* sqlFile) ...@@ -5058,7 +5590,7 @@ void querySqlFile(TAOS* taos, char* sqlFile)
size_t line_len = 0; size_t line_len = 0;
double t = getCurrentTime(); double t = getCurrentTime();
while ((read_len = tgetline(&line, &line_len, fp)) != -1) { while ((read_len = tgetline(&line, &line_len, fp)) != -1) {
if (read_len >= MAX_SQL_SIZE) continue; if (read_len >= MAX_SQL_SIZE) continue;
line[--read_len] = '\0'; line[--read_len] = '\0';
...@@ -5075,6 +5607,7 @@ void querySqlFile(TAOS* taos, char* sqlFile) ...@@ -5075,6 +5607,7 @@ void querySqlFile(TAOS* taos, char* sqlFile)
} }
memcpy(cmd + cmd_len, line, read_len); memcpy(cmd + cmd_len, line, read_len);
debugPrint("DEBUG %s() LN%d cmd: %s\n", __func__, __LINE__, cmd);
queryDbExec(taos, cmd, NO_INSERT_TYPE); queryDbExec(taos, cmd, NO_INSERT_TYPE);
memset(cmd, 0, MAX_SQL_SIZE); memset(cmd, 0, MAX_SQL_SIZE);
cmd_len = 0; cmd_len = 0;
...@@ -5089,59 +5622,63 @@ void querySqlFile(TAOS* taos, char* sqlFile) ...@@ -5089,59 +5622,63 @@ void querySqlFile(TAOS* taos, char* sqlFile)
return; return;
} }
int main(int argc, char *argv[]) { static void testMetaFile() {
parse_args(argc, argv, &g_args); if (INSERT_MODE == g_args.test_mode) {
if (g_args.metaFile) {
initOfInsertMeta();
initOfQueryMeta();
if (false == getInfoFromJsonFile(g_args.metaFile)) {
printf("Failed to read %s\n", g_args.metaFile);
return 1;
}
if (INSERT_MODE == g_jsonType) {
if (g_Dbs.cfgDir[0]) taos_options(TSDB_OPTION_CONFIGDIR, g_Dbs.cfgDir); if (g_Dbs.cfgDir[0]) taos_options(TSDB_OPTION_CONFIGDIR, g_Dbs.cfgDir);
(void)insertTestProcess(); insertTestProcess();
} else if (QUERY_MODE == g_jsonType) { } else if (QUERY_MODE == g_args.test_mode) {
if (g_queryInfo.cfgDir[0]) taos_options(TSDB_OPTION_CONFIGDIR, g_queryInfo.cfgDir); if (g_queryInfo.cfgDir[0])
(void)queryTestProcess(); taos_options(TSDB_OPTION_CONFIGDIR, g_queryInfo.cfgDir);
} else if (SUBSCRIBE_MODE == g_jsonType) { queryTestProcess();
if (g_queryInfo.cfgDir[0]) taos_options(TSDB_OPTION_CONFIGDIR, g_queryInfo.cfgDir); } else if (SUBSCRIBE_MODE == g_args.test_mode) {
(void)subscribeTestProcess(); if (g_queryInfo.cfgDir[0])
taos_options(TSDB_OPTION_CONFIGDIR, g_queryInfo.cfgDir);
subscribeTestProcess();
} else { } else {
; ;
} }
} else { }
memset(&g_Dbs, 0, sizeof(SDbs));
g_jsonType = INSERT_MODE;
setParaFromArg();
if (NULL != g_args.sqlFile) { static void testCmdLine() {
TAOS* qtaos = taos_connect(
g_Dbs.host, g_Dbs.user, g_Dbs.password, g_Dbs.db[0].dbName, g_Dbs.port); g_args.test_mode = INSERT_MODE;
querySqlFile(qtaos, g_args.sqlFile); insertTestProcess();
taos_close(qtaos);
return 0; if (g_Dbs.insert_only)
} return;
(void)insertTestProcess();
if (g_Dbs.insert_only) return 0;
// select // select
if (false == g_Dbs.insert_only) { if (false == g_Dbs.insert_only) {
// query data // query data
pthread_t read_id; pthread_t read_id;
threadInfo *rInfo = malloc(sizeof(threadInfo)); threadInfo *rInfo = malloc(sizeof(threadInfo));
rInfo->start_time = 1500000000000; // 2017-07-14 10:40:00.000 rInfo->start_time = 1500000000000; // 2017-07-14 10:40:00.000
rInfo->start_table_id = 0; rInfo->start_table_id = 0;
rInfo->end_table_id = g_Dbs.db[0].superTbls[0].childTblCount - 1;
//rInfo->do_aggreFunc = g_Dbs.do_aggreFunc; //rInfo->do_aggreFunc = g_Dbs.do_aggreFunc;
//rInfo->nrecords_per_table = g_Dbs.db[0].superTbls[0].insertRows; if (g_args.use_metric) {
rInfo->superTblInfo = &g_Dbs.db[0].superTbls[0]; rInfo->end_table_id = g_Dbs.db[0].superTbls[0].childTblCount - 1;
rInfo->taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, g_Dbs.db[0].dbName, g_Dbs.port); rInfo->superTblInfo = &g_Dbs.db[0].superTbls[0];
strcpy(rInfo->tb_prefix, g_Dbs.db[0].superTbls[0].childTblPrefix); strcpy(rInfo->tb_prefix,
g_Dbs.db[0].superTbls[0].childTblPrefix);
} else {
rInfo->end_table_id = g_args.num_of_tables -1;
strcpy(rInfo->tb_prefix, g_args.tb_prefix);
}
rInfo->taos = taos_connect(
g_Dbs.host,
g_Dbs.user,
g_Dbs.password,
g_Dbs.db[0].dbName,
g_Dbs.port);
if (rInfo->taos == NULL) {
fprintf(stderr, "Failed to connect to TDengine, reason:%s\n", taos_errstr(NULL));
free(rInfo);
exit(-1);
}
strcpy(rInfo->fp, g_Dbs.resultFile); strcpy(rInfo->fp, g_Dbs.resultFile);
if (!g_Dbs.use_metric) { if (!g_Dbs.use_metric) {
...@@ -5153,9 +5690,42 @@ int main(int argc, char *argv[]) { ...@@ -5153,9 +5690,42 @@ int main(int argc, char *argv[]) {
taos_close(rInfo->taos); taos_close(rInfo->taos);
free(rInfo); free(rInfo);
} }
}
int main(int argc, char *argv[]) {
parse_args(argc, argv, &g_args);
debugPrint("DEBUG - meta file: %s\n", g_args.metaFile);
if (g_args.metaFile) {
initOfInsertMeta();
initOfQueryMeta();
if (false == getInfoFromJsonFile(g_args.metaFile)) {
printf("Failed to read %s\n", g_args.metaFile);
return 1;
}
testMetaFile();
} else {
memset(&g_Dbs, 0, sizeof(SDbs));
setParaFromArg();
if (NULL != g_args.sqlFile) {
TAOS* qtaos = taos_connect(
g_Dbs.host,
g_Dbs.user,
g_Dbs.password,
g_Dbs.db[0].dbName,
g_Dbs.port);
querySqlFile(qtaos, g_args.sqlFile);
taos_close(qtaos);
} else {
testCmdLine();
}
} }
taos_cleanup();
return 0; return 0;
} }
...@@ -832,12 +832,13 @@ static int32_t mnodeProcessBatchCreateTableMsg(SMnodeMsg *pMsg) { ...@@ -832,12 +832,13 @@ static int32_t mnodeProcessBatchCreateTableMsg(SMnodeMsg *pMsg) {
return code; return code;
} else if (code != TSDB_CODE_MND_ACTION_IN_PROGRESS) { } else if (code != TSDB_CODE_MND_ACTION_IN_PROGRESS) {
++pMsg->pBatchMasterMsg->received; ++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = code;
mnodeDestroySubMsg(pMsg); mnodeDestroySubMsg(pMsg);
} }
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) { >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, TSDB_CODE_SUCCESS); dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, pMsg->pBatchMasterMsg->code);
} }
return TSDB_CODE_MND_ACTION_IN_PROGRESS; return TSDB_CODE_MND_ACTION_IN_PROGRESS;
...@@ -916,11 +917,13 @@ static int32_t mnodeProcessDropTableMsg(SMnodeMsg *pMsg) { ...@@ -916,11 +917,13 @@ static int32_t mnodeProcessDropTableMsg(SMnodeMsg *pMsg) {
return TSDB_CODE_MND_DB_IN_DROPPING; return TSDB_CODE_MND_DB_IN_DROPPING;
} }
#if 0
if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) {
mError("msg:%p, app:%p table:%s, failed to drop table, in monitor database", pMsg, pMsg->rpcMsg.ahandle, mError("msg:%p, app:%p table:%s, failed to drop table, in monitor database", pMsg, pMsg->rpcMsg.ahandle,
pDrop->name); pDrop->name);
return TSDB_CODE_MND_MONITOR_DB_FORBIDDEN; return TSDB_CODE_MND_MONITOR_DB_FORBIDDEN;
} }
#endif
if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pDrop->name); if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pDrop->name);
if (pMsg->pTable == NULL) { if (pMsg->pTable == NULL) {
...@@ -1906,7 +1909,8 @@ static int32_t mnodeDoCreateChildTableCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1906,7 +1909,8 @@ static int32_t mnodeDoCreateChildTableCb(SMnodeMsg *pMsg, int32_t code) {
sdbDeleteRow(&desc); sdbDeleteRow(&desc);
if (pMsg->pBatchMasterMsg) { if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->successed; ++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = code;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) { >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code); dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
...@@ -2688,6 +2692,7 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) { ...@@ -2688,6 +2692,7 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) {
if (pMsg->pBatchMasterMsg) { if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->received; ++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = code;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) { >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code); dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
...@@ -2726,6 +2731,7 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) { ...@@ -2726,6 +2731,7 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) {
if (pMsg->pBatchMasterMsg) { if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->received; ++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = rpcMsg->code;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) { >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, rpcMsg->code); dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, rpcMsg->code);
...@@ -3020,10 +3026,12 @@ static int32_t mnodeProcessAlterTableMsg(SMnodeMsg *pMsg) { ...@@ -3020,10 +3026,12 @@ static int32_t mnodeProcessAlterTableMsg(SMnodeMsg *pMsg) {
return TSDB_CODE_MND_DB_IN_DROPPING; return TSDB_CODE_MND_DB_IN_DROPPING;
} }
#if 0
if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) {
mError("msg:%p, app:%p table:%s, failed to alter table, its log db", pMsg, pMsg->rpcMsg.ahandle, pAlter->tableFname); mError("msg:%p, app:%p table:%s, failed to alter table, its log db", pMsg, pMsg->rpcMsg.ahandle, pAlter->tableFname);
return TSDB_CODE_MND_MONITOR_DB_FORBIDDEN; return TSDB_CODE_MND_MONITOR_DB_FORBIDDEN;
} }
#endif
if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pAlter->tableFname); if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pAlter->tableFname);
if (pMsg->pTable == NULL) { if (pMsg->pTable == NULL) {
......
...@@ -537,6 +537,7 @@ static int32_t mnodeCreateVgroupCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -537,6 +537,7 @@ static int32_t mnodeCreateVgroupCb(SMnodeMsg *pMsg, int32_t code) {
if (pMsg->pBatchMasterMsg) { if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->received; ++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = pMsg->code;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) { >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, pMsg->code); dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, pMsg->code);
...@@ -1002,6 +1003,7 @@ static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { ...@@ -1002,6 +1003,7 @@ static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) {
if (mnodeMsg->pBatchMasterMsg) { if (mnodeMsg->pBatchMasterMsg) {
++mnodeMsg->pBatchMasterMsg->received; ++mnodeMsg->pBatchMasterMsg->received;
mnodeMsg->pBatchMasterMsg->code = code;
if (mnodeMsg->pBatchMasterMsg->successed + mnodeMsg->pBatchMasterMsg->received if (mnodeMsg->pBatchMasterMsg->successed + mnodeMsg->pBatchMasterMsg->received
>= mnodeMsg->pBatchMasterMsg->expected) { >= mnodeMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(mnodeMsg->pBatchMasterMsg, code); dnodeSendRpcMWriteRsp(mnodeMsg->pBatchMasterMsg, code);
...@@ -1024,6 +1026,7 @@ static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { ...@@ -1024,6 +1026,7 @@ static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) {
if (mnodeMsg->pBatchMasterMsg) { if (mnodeMsg->pBatchMasterMsg) {
++mnodeMsg->pBatchMasterMsg->received; ++mnodeMsg->pBatchMasterMsg->received;
mnodeMsg->pBatchMasterMsg->code = mnodeMsg->code;
if (mnodeMsg->pBatchMasterMsg->successed + mnodeMsg->pBatchMasterMsg->received if (mnodeMsg->pBatchMasterMsg->successed + mnodeMsg->pBatchMasterMsg->received
>= mnodeMsg->pBatchMasterMsg->expected) { >= mnodeMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(mnodeMsg->pBatchMasterMsg, mnodeMsg->code); dnodeSendRpcMWriteRsp(mnodeMsg->pBatchMasterMsg, mnodeMsg->code);
......
...@@ -83,6 +83,20 @@ extern "C" { ...@@ -83,6 +83,20 @@ extern "C" {
} \ } \
} while (0) } while (0)
#define DEFAULT_DOUBLE_COMP(x, y) \
do { \
if (isnan(x) && isnan(y)) { return 0; } \
if (isnan(x)) { return -1; } \
if (isnan(y)) { return 1; } \
if ((x) == (y)) { \
return 0; \
} else { \
return (x) < (y) ? -1 : 1; \
} \
} while (0)
#define DEFAULT_FLOAT_COMP(x, y) DEFAULT_DOUBLE_COMP(x, y)
#define ALIGN_NUM(n, align) (((n) + ((align)-1)) & (~((align)-1))) #define ALIGN_NUM(n, align) (((n) + ((align)-1)) & (~((align)-1)))
// align to 8bytes // align to 8bytes
......
...@@ -34,6 +34,8 @@ ...@@ -34,6 +34,8 @@
#define REST_JSON_DATA_LEN 4 #define REST_JSON_DATA_LEN 4
#define REST_JSON_HEAD "head" #define REST_JSON_HEAD "head"
#define REST_JSON_HEAD_LEN 4 #define REST_JSON_HEAD_LEN 4
#define REST_JSON_HEAD_INFO "column_meta"
#define REST_JSON_HEAD_INFO_LEN 11
#define REST_JSON_ROWS "rows" #define REST_JSON_ROWS "rows"
#define REST_JSON_ROWS_LEN 4 #define REST_JSON_ROWS_LEN 4
#define REST_JSON_AFFECT_ROWS "affected_rows" #define REST_JSON_AFFECT_ROWS "affected_rows"
...@@ -51,4 +53,4 @@ bool restBuildSqlLocalTimeStringJson(HttpContext *pContext, HttpSqlCmd *cmd, TAO ...@@ -51,4 +53,4 @@ bool restBuildSqlLocalTimeStringJson(HttpContext *pContext, HttpSqlCmd *cmd, TAO
bool restBuildSqlUtcTimeStringJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result, int32_t numOfRows); bool restBuildSqlUtcTimeStringJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result, int32_t numOfRows);
void restStopSqlJson(HttpContext *pContext, HttpSqlCmd *cmd); void restStopSqlJson(HttpContext *pContext, HttpSqlCmd *cmd);
#endif #endif
\ No newline at end of file
...@@ -59,7 +59,9 @@ void httpDispatchToResultQueue(void *param, TAOS_RES *result, int32_t code, int3 ...@@ -59,7 +59,9 @@ void httpDispatchToResultQueue(void *param, TAOS_RES *result, int32_t code, int3
pMsg->fp = fp; pMsg->fp = fp;
taosWriteQitem(tsHttpQueue, TAOS_QTYPE_RPC, pMsg); taosWriteQitem(tsHttpQueue, TAOS_QTYPE_RPC, pMsg);
} else { } else {
(*fp)(param, result, code, rows); taos_stop_query(result);
taos_free_result(result);
//(*fp)(param, result, code, rows);
} }
} }
......
...@@ -75,6 +75,44 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result) ...@@ -75,6 +75,44 @@ void restStartSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result)
// head array end // head array end
httpJsonToken(jsonBuf, JsonArrEnd); httpJsonToken(jsonBuf, JsonArrEnd);
// column_meta begin
httpJsonItemToken(jsonBuf);
httpJsonPairHead(jsonBuf, REST_JSON_HEAD_INFO, REST_JSON_HEAD_INFO_LEN);
// column_meta array begin
httpJsonItemToken(jsonBuf);
httpJsonToken(jsonBuf, JsonArrStt);
if (num_fields == 0) {
httpJsonItemToken(jsonBuf);
httpJsonToken(jsonBuf, JsonArrStt);
httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, REST_JSON_AFFECT_ROWS, REST_JSON_AFFECT_ROWS_LEN);
httpJsonItemToken(jsonBuf);
httpJsonInt(jsonBuf, TSDB_DATA_TYPE_INT);
httpJsonItemToken(jsonBuf);
httpJsonInt(jsonBuf, 4);
httpJsonToken(jsonBuf, JsonArrEnd);
} else {
for (int32_t i = 0; i < num_fields; ++i) {
httpJsonItemToken(jsonBuf);
httpJsonToken(jsonBuf, JsonArrStt);
httpJsonItemToken(jsonBuf);
httpJsonString(jsonBuf, fields[i].name, (int32_t)strlen(fields[i].name));
httpJsonItemToken(jsonBuf);
httpJsonInt(jsonBuf, fields[i].type);
httpJsonItemToken(jsonBuf);
httpJsonInt(jsonBuf, fields[i].bytes);
httpJsonToken(jsonBuf, JsonArrEnd);
}
}
// column_meta array end
httpJsonToken(jsonBuf, JsonArrEnd);
// data begin // data begin
httpJsonItemToken(jsonBuf); httpJsonItemToken(jsonBuf);
httpJsonPairHead(jsonBuf, REST_JSON_DATA, REST_JSON_DATA_LEN); httpJsonPairHead(jsonBuf, REST_JSON_DATA, REST_JSON_DATA_LEN);
......
...@@ -362,20 +362,10 @@ static FORCE_INLINE int32_t columnValueAscendingComparator(char *f1, char *f2, i ...@@ -362,20 +362,10 @@ static FORCE_INLINE int32_t columnValueAscendingComparator(char *f1, char *f2, i
return (first < second) ? -1 : 1; return (first < second) ? -1 : 1;
}; };
case TSDB_DATA_TYPE_DOUBLE: { case TSDB_DATA_TYPE_DOUBLE: {
double first = GET_DOUBLE_VAL(f1); DEFAULT_DOUBLE_COMP(GET_DOUBLE_VAL(f1), GET_DOUBLE_VAL(f2));
double second = GET_DOUBLE_VAL(f2);
if (first == second) {
return 0;
}
return (first < second) ? -1 : 1;
}; };
case TSDB_DATA_TYPE_FLOAT: { case TSDB_DATA_TYPE_FLOAT: {
float first = GET_FLOAT_VAL(f1); DEFAULT_FLOAT_COMP(GET_FLOAT_VAL(f1), GET_FLOAT_VAL(f2));
float second = GET_FLOAT_VAL(f2);
if (first == second) {
return 0;
}
return (first < second) ? -1 : 1;
}; };
case TSDB_DATA_TYPE_BIGINT: { case TSDB_DATA_TYPE_BIGINT: {
int64_t first = *(int64_t *)f1; int64_t first = *(int64_t *)f1;
......
...@@ -58,6 +58,15 @@ SSqlInfo qSQLParse(const char *pStr) { ...@@ -58,6 +58,15 @@ SSqlInfo qSQLParse(const char *pStr) {
sqlInfo.valid = false; sqlInfo.valid = false;
goto abort_parse; goto abort_parse;
} }
case TK_HEX:
case TK_OCT:
case TK_BIN:{
snprintf(sqlInfo.msg, tListLen(sqlInfo.msg), "unsupported token: \"%s\"", t0.z);
sqlInfo.valid = false;
goto abort_parse;
}
default: default:
Parse(pParser, t0.type, t0, &sqlInfo); Parse(pParser, t0.type, t0, &sqlInfo);
if (sqlInfo.valid == false) { if (sqlInfo.valid == false) {
......
...@@ -48,7 +48,7 @@ tMemBucket *createUnsignedDataBucket(int32_t start, int32_t end, int32_t type) { ...@@ -48,7 +48,7 @@ tMemBucket *createUnsignedDataBucket(int32_t start, int32_t end, int32_t type) {
uint64_t k = i; uint64_t k = i;
int32_t ret = tMemBucketPut(pBucket, &k, 1); int32_t ret = tMemBucketPut(pBucket, &k, 1);
if (ret != 0) { if (ret != 0) {
printf("value out of range:%f", k); printf("value out of range:%" PRId64, k);
} }
} }
...@@ -245,7 +245,7 @@ void unsignedDataTest() { ...@@ -245,7 +245,7 @@ void unsignedDataTest() {
} // namespace } // namespace
TEST(testCase, percentileTest) { TEST(testCase, percentileTest) {
// qsortTest(); // qsortTest();
intDataTest(); intDataTest();
bigintDataTest(); bigintDataTest();
doubleDataTest(); doubleDataTest();
......
...@@ -227,10 +227,10 @@ TEST(testCase, db_table_name) { ...@@ -227,10 +227,10 @@ TEST(testCase, db_table_name) {
EXPECT_EQ(testValidateName(t60_1), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t60_1), TSDB_CODE_TSC_INVALID_SQL);
char t61[] = "' ABC '"; char t61[] = "' ABC '";
EXPECT_EQ(testValidateName(t61), TSDB_CODE_SUCCESS); EXPECT_EQ(testValidateName(t61), TSDB_CODE_TSC_INVALID_SQL);
char t61_1[] = "' ABC '"; char t61_1[] = "' ABC '";
EXPECT_EQ(testValidateName(t61_1), TSDB_CODE_SUCCESS); EXPECT_EQ(testValidateName(t61_1), TSDB_CODE_TSC_INVALID_SQL);
char t62[] = " ABC . def "; char t62[] = " ABC . def ";
EXPECT_EQ(testValidateName(t62), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t62), TSDB_CODE_TSC_INVALID_SQL);
...@@ -249,13 +249,13 @@ TEST(testCase, db_table_name) { ...@@ -249,13 +249,13 @@ TEST(testCase, db_table_name) {
EXPECT_EQ(testValidateName(t65), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t65), TSDB_CODE_TSC_INVALID_SQL);
char t66[] = "' ABC '.' DEF '"; char t66[] = "' ABC '.' DEF '";
EXPECT_EQ(testValidateName(t66), TSDB_CODE_SUCCESS); EXPECT_EQ(testValidateName(t66), TSDB_CODE_TSC_INVALID_SQL);
char t67[] = "abc . ' DEF '"; char t67[] = "abc . ' DEF '";
EXPECT_EQ(testValidateName(t67), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t67), TSDB_CODE_TSC_INVALID_SQL);
char t68[] = "' abc '.' DEF '"; char t68[] = "' abc '.' DEF '";
EXPECT_EQ(testValidateName(t68), TSDB_CODE_SUCCESS); EXPECT_EQ(testValidateName(t68), TSDB_CODE_TSC_INVALID_SQL);
// do not use key words // do not use key words
char t69[] = "table.'DEF'"; char t69[] = "table.'DEF'";
...@@ -265,7 +265,7 @@ TEST(testCase, db_table_name) { ...@@ -265,7 +265,7 @@ TEST(testCase, db_table_name) {
EXPECT_EQ(testValidateName(t70), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t70), TSDB_CODE_TSC_INVALID_SQL);
char t71[] = "'_abXYZ1234 '.' deFF '"; char t71[] = "'_abXYZ1234 '.' deFF '";
EXPECT_EQ(testValidateName(t71), TSDB_CODE_SUCCESS); EXPECT_EQ(testValidateName(t71), TSDB_CODE_TSC_INVALID_SQL);
char t72[] = "'_abDEF&^%1234'.' DIef'"; char t72[] = "'_abDEF&^%1234'.' DIef'";
EXPECT_EQ(testValidateName(t72), TSDB_CODE_TSC_INVALID_SQL); EXPECT_EQ(testValidateName(t72), TSDB_CODE_TSC_INVALID_SQL);
......
...@@ -1281,7 +1281,7 @@ static void rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) { ...@@ -1281,7 +1281,7 @@ static void rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) {
SRpcConn *pConn = rpcSetupConnToServer(pContext); SRpcConn *pConn = rpcSetupConnToServer(pContext);
if (pConn == NULL) { if (pConn == NULL) {
pContext->code = terrno; pContext->code = terrno;
taosTmrStart(rpcProcessConnError, 0, pContext, pRpc->tmrCtrl); taosTmrStart(rpcProcessConnError, 1, pContext, pRpc->tmrCtrl);
return; return;
} }
......
...@@ -37,6 +37,7 @@ typedef struct { ...@@ -37,6 +37,7 @@ typedef struct {
TSKEY keyLast; TSKEY keyLast;
int64_t numOfRows; int64_t numOfRows;
SSkipList* pData; SSkipList* pData;
T_REF_DECLARE()
} STableData; } STableData;
typedef struct { typedef struct {
...@@ -76,7 +77,7 @@ typedef struct { ...@@ -76,7 +77,7 @@ typedef struct {
int tsdbRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable); int tsdbRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable);
int tsdbUnRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable); int tsdbUnRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable);
int tsdbTakeMemSnapshot(STsdbRepo* pRepo, SMemTable** pMem, SMemTable** pIMem); int tsdbTakeMemSnapshot(STsdbRepo* pRepo, SMemTable** pMem, SMemTable** pIMem, SArray* pATable);
void tsdbUnTakeMemSnapShot(STsdbRepo* pRepo, SMemTable* pMem, SMemTable* pIMem); void tsdbUnTakeMemSnapShot(STsdbRepo* pRepo, SMemTable* pMem, SMemTable* pIMem);
void* tsdbAllocBytes(STsdbRepo* pRepo, int bytes); void* tsdbAllocBytes(STsdbRepo* pRepo, int bytes);
int tsdbAsyncCommit(STsdbRepo* pRepo); int tsdbAsyncCommit(STsdbRepo* pRepo);
......
...@@ -597,7 +597,7 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) { ...@@ -597,7 +597,7 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) {
// Get the data in row // Get the data in row
ASSERT(pTable->lastRow == NULL); ASSERT(pTable->lastRow == NULL);
STSchema *pSchema = tsdbGetTableSchema(pTable); STSchema *pSchema = tsdbGetTableSchema(pTable);
pTable->lastRow = taosTMalloc(schemaTLen(pSchema)); pTable->lastRow = taosTMalloc(dataRowMaxBytesFromSchema(pSchema));
if (pTable->lastRow == NULL) { if (pTable->lastRow == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
tsdbDestroyReadH(&readh); tsdbDestroyReadH(&readh);
......
...@@ -124,17 +124,66 @@ int tsdbUnRefMemTable(STsdbRepo *pRepo, SMemTable *pMemTable) { ...@@ -124,17 +124,66 @@ int tsdbUnRefMemTable(STsdbRepo *pRepo, SMemTable *pMemTable) {
return 0; return 0;
} }
int tsdbTakeMemSnapshot(STsdbRepo *pRepo, SMemTable **pMem, SMemTable **pIMem) { int tsdbTakeMemSnapshot(STsdbRepo *pRepo, SMemTable **pMem, SMemTable **pIMem, SArray *pATable) {
SMemTable *tmem;
// Get snap object
if (tsdbLockRepo(pRepo) < 0) return -1; if (tsdbLockRepo(pRepo) < 0) return -1;
*pMem = pRepo->mem; tmem = pRepo->mem;
*pIMem = pRepo->imem; *pIMem = pRepo->imem;
tsdbRefMemTable(pRepo, *pMem); tsdbRefMemTable(pRepo, tmem);
tsdbRefMemTable(pRepo, *pIMem); tsdbRefMemTable(pRepo, *pIMem);
if (tsdbUnlockRepo(pRepo) < 0) return -1; if (tsdbUnlockRepo(pRepo) < 0) return -1;
if (*pMem != NULL) taosRLockLatch(&((*pMem)->latch)); // Copy mem objects and ref needed STableData
if (tmem) {
taosRLockLatch(&(tmem->latch));
*pMem = (SMemTable *)calloc(1, sizeof(**pMem));
if (*pMem == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
taosRUnLockLatch(&(tmem->latch));
tsdbUnRefMemTable(pRepo, tmem);
tsdbUnRefMemTable(pRepo, *pIMem);
*pMem = NULL;
*pIMem = NULL;
return -1;
}
(*pMem)->tData = (STableData **)calloc(tmem->maxTables, sizeof(STableData *));
if ((*pMem)->tData == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
taosRUnLockLatch(&(tmem->latch));
free(*pMem);
tsdbUnRefMemTable(pRepo, tmem);
tsdbUnRefMemTable(pRepo, *pIMem);
*pMem = NULL;
*pIMem = NULL;
return -1;
}
(*pMem)->keyFirst = tmem->keyFirst;
(*pMem)->keyLast = tmem->keyLast;
(*pMem)->numOfRows = tmem->numOfRows;
(*pMem)->maxTables = tmem->maxTables;
for (size_t i = 0; i < taosArrayGetSize(pATable); i++) {
STable * pTable = *(STable **)taosArrayGet(pATable, i);
int32_t tid = TABLE_TID(pTable);
STableData *pTableData = (tid < tmem->maxTables) ? tmem->tData[tid] : NULL;
if ((pTableData == NULL) || (TABLE_UID(pTable) != pTableData->uid)) continue;
(*pMem)->tData[tid] = tmem->tData[tid];
T_REF_INC(tmem->tData[tid]);
}
taosRUnLockLatch(&(tmem->latch));
}
tsdbUnRefMemTable(pRepo, tmem);
tsdbDebug("vgId:%d take memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), *pMem, *pIMem); tsdbDebug("vgId:%d take memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), *pMem, *pIMem);
return 0; return 0;
...@@ -144,8 +193,14 @@ void tsdbUnTakeMemSnapShot(STsdbRepo *pRepo, SMemTable *pMem, SMemTable *pIMem) ...@@ -144,8 +193,14 @@ void tsdbUnTakeMemSnapShot(STsdbRepo *pRepo, SMemTable *pMem, SMemTable *pIMem)
tsdbDebug("vgId:%d untake memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), pMem, pIMem); tsdbDebug("vgId:%d untake memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), pMem, pIMem);
if (pMem != NULL) { if (pMem != NULL) {
taosRUnLockLatch(&(pMem->latch)); for (size_t i = 0; i < pMem->maxTables; i++) {
tsdbUnRefMemTable(pRepo, pMem); STableData *pTableData = pMem->tData[i];
if (pTableData) {
tsdbFreeTableData(pTableData);
}
}
free(pMem->tData);
free(pMem);
} }
if (pIMem != NULL) { if (pIMem != NULL) {
...@@ -436,7 +491,7 @@ static STableData *tsdbNewTableData(STsdbCfg *pCfg, STable *pTable) { ...@@ -436,7 +491,7 @@ static STableData *tsdbNewTableData(STsdbCfg *pCfg, STable *pTable) {
STableData *pTableData = (STableData *)calloc(1, sizeof(*pTableData)); STableData *pTableData = (STableData *)calloc(1, sizeof(*pTableData));
if (pTableData == NULL) { if (pTableData == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
goto _err; return NULL;
} }
pTableData->uid = TABLE_UID(pTable); pTableData->uid = TABLE_UID(pTable);
...@@ -449,20 +504,22 @@ static STableData *tsdbNewTableData(STsdbCfg *pCfg, STable *pTable) { ...@@ -449,20 +504,22 @@ static STableData *tsdbNewTableData(STsdbCfg *pCfg, STable *pTable) {
tkeyComparFn, pCfg->update ? SL_UPDATE_DUP_KEY : SL_DISCARD_DUP_KEY, tsdbGetTsTupleKey); tkeyComparFn, pCfg->update ? SL_UPDATE_DUP_KEY : SL_DISCARD_DUP_KEY, tsdbGetTsTupleKey);
if (pTableData->pData == NULL) { if (pTableData->pData == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
goto _err; free(pTableData);
return NULL;
} }
return pTableData; T_REF_INC(pTableData);
_err: return pTableData;
tsdbFreeTableData(pTableData);
return NULL;
} }
static void tsdbFreeTableData(STableData *pTableData) { static void tsdbFreeTableData(STableData *pTableData) {
if (pTableData) { if (pTableData) {
tSkipListDestroy(pTableData->pData); int32_t ref = T_REF_DEC(pTableData);
free(pTableData); if (ref == 0) {
tSkipListDestroy(pTableData->pData);
free(pTableData);
}
} }
} }
......
...@@ -187,13 +187,15 @@ static SArray* getDefaultLoadColumns(STsdbQueryHandle* pQueryHandle, bool loadTS ...@@ -187,13 +187,15 @@ static SArray* getDefaultLoadColumns(STsdbQueryHandle* pQueryHandle, bool loadTS
return pLocalIdList; return pLocalIdList;
} }
static void tsdbMayTakeMemSnapshot(STsdbQueryHandle* pQueryHandle) { static void tsdbMayTakeMemSnapshot(STsdbQueryHandle* pQueryHandle, SArray* psTable) {
assert(pQueryHandle != NULL && pQueryHandle->pMemRef != NULL); assert(pQueryHandle != NULL && pQueryHandle->pMemRef != NULL);
SMemRef* pMemRef = pQueryHandle->pMemRef; SMemRef* pMemRef = pQueryHandle->pMemRef;
if (pQueryHandle->pMemRef->ref++ == 0) { if (pQueryHandle->pMemRef->ref++ == 0) {
tsdbTakeMemSnapshot(pQueryHandle->pTsdb, (SMemTable**)&(pMemRef->mem), (SMemTable**)&(pMemRef->imem)); tsdbTakeMemSnapshot(pQueryHandle->pTsdb, (SMemTable**)&(pMemRef->mem), (SMemTable**)&(pMemRef->imem), psTable);
} }
taosArrayDestroy(psTable);
} }
static void tsdbMayUnTakeMemSnapshot(STsdbQueryHandle* pQueryHandle) { static void tsdbMayUnTakeMemSnapshot(STsdbQueryHandle* pQueryHandle) {
...@@ -242,7 +244,7 @@ int64_t tsdbGetNumOfRowsInMemTable(TsdbQueryHandleT* pHandle) { ...@@ -242,7 +244,7 @@ int64_t tsdbGetNumOfRowsInMemTable(TsdbQueryHandleT* pHandle) {
return rows; return rows;
} }
static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STableGroupInfo* pGroupList, STsdbMeta* pMeta) { static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STableGroupInfo* pGroupList, STsdbMeta* pMeta, SArray** psTable) {
size_t sizeOfGroup = taosArrayGetSize(pGroupList->pGroupList); size_t sizeOfGroup = taosArrayGetSize(pGroupList->pGroupList);
assert(sizeOfGroup >= 1 && pMeta != NULL); assert(sizeOfGroup >= 1 && pMeta != NULL);
...@@ -252,6 +254,12 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa ...@@ -252,6 +254,12 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa
return NULL; return NULL;
} }
SArray* pTable = taosArrayInit(4, sizeof(STable*));
if (pTable == NULL) {
taosArrayDestroy(pTableCheckInfo);
return NULL;
}
// todo apply the lastkey of table check to avoid to load header file // todo apply the lastkey of table check to avoid to load header file
for (int32_t i = 0; i < sizeOfGroup; ++i) { for (int32_t i = 0; i < sizeOfGroup; ++i) {
SArray* group = *(SArray**) taosArrayGet(pGroupList->pGroupList, i); SArray* group = *(SArray**) taosArrayGet(pGroupList->pGroupList, i);
...@@ -284,24 +292,40 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa ...@@ -284,24 +292,40 @@ static SArray* createCheckInfoFromTableGroup(STsdbQueryHandle* pQueryHandle, STa
} }
taosArraySort(pTableCheckInfo, tsdbCheckInfoCompar); taosArraySort(pTableCheckInfo, tsdbCheckInfoCompar);
size_t gsize = taosArrayGetSize(pTableCheckInfo);
for (int32_t i = 0; i < gsize; ++i) {
STableCheckInfo* pInfo = (STableCheckInfo*) taosArrayGet(pTableCheckInfo, i);
taosArrayPush(pTable, &pInfo->pTableObj);
}
*psTable = pTable;
return pTableCheckInfo; return pTableCheckInfo;
} }
static SArray* createCheckInfoFromCheckInfo(SArray* pTableCheckInfo, TSKEY skey) { static SArray* createCheckInfoFromCheckInfo(SArray* pTableCheckInfo, TSKEY skey, SArray** psTable) {
size_t si = taosArrayGetSize(pTableCheckInfo); size_t si = taosArrayGetSize(pTableCheckInfo);
SArray* pNew = taosArrayInit(si, sizeof(STableCheckInfo)); SArray* pNew = taosArrayInit(si, sizeof(STableCheckInfo));
if (pNew == NULL) { if (pNew == NULL) {
return NULL; return NULL;
} }
SArray* pTable = taosArrayInit(si, sizeof(STable*));
for (int32_t j = 0; j < si; ++j) { for (int32_t j = 0; j < si; ++j) {
STableCheckInfo* pCheckInfo = (STableCheckInfo*) taosArrayGet(pTableCheckInfo, j); STableCheckInfo* pCheckInfo = (STableCheckInfo*) taosArrayGet(pTableCheckInfo, j);
STableCheckInfo info = { .lastKey = skey, .pTableObj = pCheckInfo->pTableObj}; STableCheckInfo info = { .lastKey = skey, .pTableObj = pCheckInfo->pTableObj};
info.tableId = pCheckInfo->tableId; info.tableId = pCheckInfo->tableId;
taosArrayPush(pNew, &info); taosArrayPush(pNew, &info);
taosArrayPush(pTable, &pCheckInfo->pTableObj);
} }
*psTable = pTable;
// it is ordered already, no need to sort again. // it is ordered already, no need to sort again.
taosArraySort(pNew, tsdbCheckInfoCompar); taosArraySort(pNew, tsdbCheckInfoCompar);
return pNew; return pNew;
...@@ -332,7 +356,7 @@ static STsdbQueryHandle* tsdbQueryTablesImpl(STsdbRepo* tsdb, STsdbQueryCond* pC ...@@ -332,7 +356,7 @@ static STsdbQueryHandle* tsdbQueryTablesImpl(STsdbRepo* tsdb, STsdbQueryCond* pC
goto out_of_memory; goto out_of_memory;
} }
tsdbMayTakeMemSnapshot(pQueryHandle); //tsdbMayTakeMemSnapshot(pQueryHandle);
assert(pCond != NULL && pCond->numOfCols > 0 && pMemRef != NULL); assert(pCond != NULL && pCond->numOfCols > 0 && pMemRef != NULL);
if (ASCENDING_TRAVERSE(pCond->order)) { if (ASCENDING_TRAVERSE(pCond->order)) {
...@@ -393,14 +417,18 @@ TsdbQueryHandleT* tsdbQueryTables(STsdbRepo* tsdb, STsdbQueryCond* pCond, STable ...@@ -393,14 +417,18 @@ TsdbQueryHandleT* tsdbQueryTables(STsdbRepo* tsdb, STsdbQueryCond* pCond, STable
STsdbMeta* pMeta = tsdbGetMeta(tsdb); STsdbMeta* pMeta = tsdbGetMeta(tsdb);
assert(pMeta != NULL); assert(pMeta != NULL);
SArray* psTable = NULL;
// todo apply the lastkey of table check to avoid to load header file // todo apply the lastkey of table check to avoid to load header file
pQueryHandle->pTableCheckInfo = createCheckInfoFromTableGroup(pQueryHandle, groupList, pMeta); pQueryHandle->pTableCheckInfo = createCheckInfoFromTableGroup(pQueryHandle, groupList, pMeta, &psTable);
if (pQueryHandle->pTableCheckInfo == NULL) { if (pQueryHandle->pTableCheckInfo == NULL) {
tsdbCleanupQueryHandle(pQueryHandle); tsdbCleanupQueryHandle(pQueryHandle);
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
return NULL; return NULL;
} }
tsdbMayTakeMemSnapshot(pQueryHandle, psTable);
tsdbDebug("%p total numOfTable:%" PRIzu " in query, %p", pQueryHandle, taosArrayGetSize(pQueryHandle->pTableCheckInfo), pQueryHandle->qinfo); tsdbDebug("%p total numOfTable:%" PRIzu " in query, %p", pQueryHandle, taosArrayGetSize(pQueryHandle->pTableCheckInfo), pQueryHandle->qinfo);
return (TsdbQueryHandleT) pQueryHandle; return (TsdbQueryHandleT) pQueryHandle;
} }
...@@ -2337,12 +2365,18 @@ static int32_t doGetExternalRow(STsdbQueryHandle* pQueryHandle, int16_t type, SM ...@@ -2337,12 +2365,18 @@ static int32_t doGetExternalRow(STsdbQueryHandle* pQueryHandle, int16_t type, SM
pSecQueryHandle = tsdbQueryTablesImpl(pQueryHandle->pTsdb, &cond, pQueryHandle->qinfo, pMemRef); pSecQueryHandle = tsdbQueryTablesImpl(pQueryHandle->pTsdb, &cond, pQueryHandle->qinfo, pMemRef);
tfree(cond.colList); tfree(cond.colList);
pSecQueryHandle->pTableCheckInfo = createCheckInfoFromCheckInfo(pQueryHandle->pTableCheckInfo, pSecQueryHandle->window.skey);
SArray* psTable = NULL;
pSecQueryHandle->pTableCheckInfo = createCheckInfoFromCheckInfo(pQueryHandle->pTableCheckInfo, pSecQueryHandle->window.skey, &psTable);
if (pSecQueryHandle->pTableCheckInfo == NULL) { if (pSecQueryHandle->pTableCheckInfo == NULL) {
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; terrno = TSDB_CODE_QRY_OUT_OF_MEMORY;
goto out_of_memory; goto out_of_memory;
} }
tsdbMayTakeMemSnapshot(pSecQueryHandle, psTable);
if (!tsdbNextDataBlock((void*)pSecQueryHandle)) { if (!tsdbNextDataBlock((void*)pSecQueryHandle)) {
// no result in current query, free the corresponding result rows structure // no result in current query, free the corresponding result rows structure
if (type == TSDB_PREV_ROW) { if (type == TSDB_PREV_ROW) {
......
...@@ -44,6 +44,7 @@ enum { ...@@ -44,6 +44,7 @@ enum {
TAOS_CFG_VTYPE_INT8, TAOS_CFG_VTYPE_INT8,
TAOS_CFG_VTYPE_INT16, TAOS_CFG_VTYPE_INT16,
TAOS_CFG_VTYPE_INT32, TAOS_CFG_VTYPE_INT32,
TAOS_CFG_VTYPE_UINT16,
TAOS_CFG_VTYPE_FLOAT, TAOS_CFG_VTYPE_FLOAT,
TAOS_CFG_VTYPE_STRING, TAOS_CFG_VTYPE_STRING,
TAOS_CFG_VTYPE_IPSTR, TAOS_CFG_VTYPE_IPSTR,
......
...@@ -392,8 +392,8 @@ __compar_fn_t getKeyComparFunc(int32_t keyType) { ...@@ -392,8 +392,8 @@ __compar_fn_t getKeyComparFunc(int32_t keyType) {
int32_t doCompare(const char* f1, const char* f2, int32_t type, size_t size) { int32_t doCompare(const char* f1, const char* f2, int32_t type, size_t size) {
switch (type) { switch (type) {
case TSDB_DATA_TYPE_INT: DEFAULT_COMP(GET_INT32_VAL(f1), GET_INT32_VAL(f2)); case TSDB_DATA_TYPE_INT: DEFAULT_COMP(GET_INT32_VAL(f1), GET_INT32_VAL(f2));
case TSDB_DATA_TYPE_DOUBLE: DEFAULT_COMP(GET_DOUBLE_VAL(f1), GET_DOUBLE_VAL(f2)); case TSDB_DATA_TYPE_DOUBLE: DEFAULT_DOUBLE_COMP(GET_DOUBLE_VAL(f1), GET_DOUBLE_VAL(f2));
case TSDB_DATA_TYPE_FLOAT: DEFAULT_COMP(GET_FLOAT_VAL(f1), GET_FLOAT_VAL(f2)); case TSDB_DATA_TYPE_FLOAT: DEFAULT_FLOAT_COMP(GET_FLOAT_VAL(f1), GET_FLOAT_VAL(f2));
case TSDB_DATA_TYPE_BIGINT: DEFAULT_COMP(GET_INT64_VAL(f1), GET_INT64_VAL(f2)); case TSDB_DATA_TYPE_BIGINT: DEFAULT_COMP(GET_INT64_VAL(f1), GET_INT64_VAL(f2));
case TSDB_DATA_TYPE_SMALLINT: DEFAULT_COMP(GET_INT16_VAL(f1), GET_INT16_VAL(f2)); case TSDB_DATA_TYPE_SMALLINT: DEFAULT_COMP(GET_INT16_VAL(f1), GET_INT16_VAL(f2));
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
......
...@@ -95,6 +95,23 @@ static void taosReadInt16Config(SGlobalCfg *cfg, char *input_value) { ...@@ -95,6 +95,23 @@ static void taosReadInt16Config(SGlobalCfg *cfg, char *input_value) {
} }
} }
static void taosReadUInt16Config(SGlobalCfg *cfg, char *input_value) {
int32_t value = atoi(input_value);
uint16_t *option = (uint16_t *)cfg->ptr;
if (value < cfg->minValue || value > cfg->maxValue) {
uError("config option:%s, input value:%s, out of range[%f, %f], use default value:%d",
cfg->option, input_value, cfg->minValue, cfg->maxValue, *option);
} else {
if (cfg->cfgStatus <= TAOS_CFG_CSTATUS_FILE) {
*option = (uint16_t)value;
cfg->cfgStatus = TAOS_CFG_CSTATUS_FILE;
} else {
uWarn("config option:%s, input value:%s, is configured by %s, use %d", cfg->option, input_value,
tsCfgStatusStr[cfg->cfgStatus], *option);
}
}
}
static void taosReadInt8Config(SGlobalCfg *cfg, char *input_value) { static void taosReadInt8Config(SGlobalCfg *cfg, char *input_value) {
int32_t value = atoi(input_value); int32_t value = atoi(input_value);
int8_t *option = (int8_t *)cfg->ptr; int8_t *option = (int8_t *)cfg->ptr;
...@@ -239,6 +256,9 @@ static void taosReadConfigOption(const char *option, char *value, char *value2, ...@@ -239,6 +256,9 @@ static void taosReadConfigOption(const char *option, char *value, char *value2,
case TAOS_CFG_VTYPE_INT32: case TAOS_CFG_VTYPE_INT32:
taosReadInt32Config(cfg, value); taosReadInt32Config(cfg, value);
break; break;
case TAOS_CFG_VTYPE_UINT16:
taosReadUInt16Config(cfg, value);
break;
case TAOS_CFG_VTYPE_FLOAT: case TAOS_CFG_VTYPE_FLOAT:
taosReadFloatConfig(cfg, value); taosReadFloatConfig(cfg, value);
break; break;
...@@ -422,6 +442,9 @@ void taosPrintGlobalCfg() { ...@@ -422,6 +442,9 @@ void taosPrintGlobalCfg() {
case TAOS_CFG_VTYPE_INT32: case TAOS_CFG_VTYPE_INT32:
uInfo(" %s:%s%d%s", cfg->option, blank, *((int32_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]); uInfo(" %s:%s%d%s", cfg->option, blank, *((int32_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break; break;
case TAOS_CFG_VTYPE_UINT16:
uInfo(" %s:%s%d%s", cfg->option, blank, *((uint16_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break;
case TAOS_CFG_VTYPE_FLOAT: case TAOS_CFG_VTYPE_FLOAT:
uInfo(" %s:%s%f%s", cfg->option, blank, *((float *)cfg->ptr), tsGlobalUnit[cfg->unitType]); uInfo(" %s:%s%f%s", cfg->option, blank, *((float *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break; break;
...@@ -459,6 +482,9 @@ static void taosDumpCfg(SGlobalCfg *cfg) { ...@@ -459,6 +482,9 @@ static void taosDumpCfg(SGlobalCfg *cfg) {
case TAOS_CFG_VTYPE_INT32: case TAOS_CFG_VTYPE_INT32:
printf(" %s:%s%d%s\n", cfg->option, blank, *((int32_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]); printf(" %s:%s%d%s\n", cfg->option, blank, *((int32_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break; break;
case TAOS_CFG_VTYPE_UINT16:
printf(" %s:%s%d%s\n", cfg->option, blank, *((uint16_t *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break;
case TAOS_CFG_VTYPE_FLOAT: case TAOS_CFG_VTYPE_FLOAT:
printf(" %s:%s%f%s\n", cfg->option, blank, *((float *)cfg->ptr), tsGlobalUnit[cfg->unitType]); printf(" %s:%s%f%s\n", cfg->option, blank, *((float *)cfg->ptr), tsGlobalUnit[cfg->unitType]);
break; break;
......
...@@ -55,9 +55,15 @@ pipeline { ...@@ -55,9 +55,15 @@ pipeline {
sh ''' sh '''
cd ${WKC}/tests cd ${WKC}/tests
./test-all.sh b1 ./test-all.sh b1
date'''
sh '''
cd ${WKC}/tests cd ${WKC}/tests
./test-all.sh full jdbc ./test-all.sh full jdbc
date''' date'''
sh '''
cd ${WKC}/tests
./test-all.sh full unit
date'''
} }
} }
......
...@@ -63,7 +63,9 @@ ...@@ -63,7 +63,9 @@
<dependency> <dependency>
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.18</version> <version>2.0.20</version>
<!-- <scope>system</scope>-->
<!-- <systemPath>${project.basedir}/src/main/resources/taos-jdbcdriver-2.0.20-dist.jar</systemPath>-->
</dependency> </dependency>
<dependency> <dependency>
......
...@@ -10,4 +10,4 @@ public class SpringbootdemoApplication { ...@@ -10,4 +10,4 @@ public class SpringbootdemoApplication {
public static void main(String[] args) { public static void main(String[] args) {
SpringApplication.run(SpringbootdemoApplication.class, args); SpringApplication.run(SpringbootdemoApplication.class, args);
} }
} }
\ No newline at end of file
...@@ -6,6 +6,7 @@ import org.springframework.beans.factory.annotation.Autowired; ...@@ -6,6 +6,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import java.util.List; import java.util.List;
import java.util.Map;
@RequestMapping("/weather") @RequestMapping("/weather")
@RestController @RestController
...@@ -20,7 +21,7 @@ public class WeatherController { ...@@ -20,7 +21,7 @@ public class WeatherController {
* @return * @return
*/ */
@GetMapping("/init") @GetMapping("/init")
public boolean init() { public int init() {
return weatherService.init(); return weatherService.init();
} }
...@@ -44,19 +45,23 @@ public class WeatherController { ...@@ -44,19 +45,23 @@ public class WeatherController {
* @return * @return
*/ */
@PostMapping("/{temperature}/{humidity}") @PostMapping("/{temperature}/{humidity}")
public int saveWeather(@PathVariable int temperature, @PathVariable float humidity) { public int saveWeather(@PathVariable float temperature, @PathVariable int humidity) {
return weatherService.save(temperature, humidity); return weatherService.save(temperature, humidity);
} }
/** @GetMapping("/count")
* upload multi weather info public int count() {
* return weatherService.count();
* @param weatherList }
* @return
*/ @GetMapping("/subTables")
@PostMapping("/batch") public List<String> getSubTables() {
public int batchSaveWeather(@RequestBody List<Weather> weatherList) { return weatherService.getSubTables();
return weatherService.save(weatherList); }
@GetMapping("/avg")
public List<Weather> avg() {
return weatherService.avg();
} }
} }
...@@ -4,16 +4,26 @@ import com.taosdata.example.springbootdemo.domain.Weather; ...@@ -4,16 +4,26 @@ import com.taosdata.example.springbootdemo.domain.Weather;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;
import java.util.List; import java.util.List;
import java.util.Map;
public interface WeatherMapper { public interface WeatherMapper {
void dropDB();
void createDB();
void createSuperTable();
void createTable(Weather weather);
List<Weather> select(@Param("limit") Long limit, @Param("offset") Long offset);
int insert(Weather weather); int insert(Weather weather);
int batchInsert(List<Weather> weatherList); int count();
List<Weather> select(@Param("limit") Long limit, @Param("offset")Long offset); List<String> getSubTables();
void createDB(); List<Weather> avg();
void createTable();
} }
...@@ -4,28 +4,29 @@ ...@@ -4,28 +4,29 @@
<mapper namespace="com.taosdata.example.springbootdemo.dao.WeatherMapper"> <mapper namespace="com.taosdata.example.springbootdemo.dao.WeatherMapper">
<resultMap id="BaseResultMap" type="com.taosdata.example.springbootdemo.domain.Weather"> <resultMap id="BaseResultMap" type="com.taosdata.example.springbootdemo.domain.Weather">
<id column="ts" jdbcType="TIMESTAMP" property="ts" /> <id column="ts" jdbcType="TIMESTAMP" property="ts"/>
<result column="temperature" jdbcType="INTEGER" property="temperature" /> <result column="temperature" jdbcType="FLOAT" property="temperature"/>
<result column="humidity" jdbcType="FLOAT" property="humidity" /> <result column="humidity" jdbcType="FLOAT" property="humidity"/>
</resultMap> </resultMap>
<update id="createDB" > <update id="dropDB">
create database if not exists test; drop database if exists test
</update>
<update id="createDB">
create database if not exists test
</update> </update>
<update id="createTable" > <update id="createSuperTable">
create table if not exists test.weather(ts timestamp, temperature int, humidity float); create table if not exists test.weather(ts timestamp, temperature float, humidity float) tags(location nchar(64), groupId int)
</update> </update>
<sql id="Base_Column_List"> <update id="createTable" parameterType="com.taosdata.example.springbootdemo.domain.Weather">
ts, temperature, humidity create table if not exists test.t#{groupId} using test.weather tags(#{location}, #{groupId})
</sql> </update>
<select id="select" resultMap="BaseResultMap"> <select id="select" resultMap="BaseResultMap">
select select * from test.weather order by ts desc
<include refid="Base_Column_List" />
from test.weather
order by ts desc
<if test="limit != null"> <if test="limit != null">
limit #{limit,jdbcType=BIGINT} limit #{limit,jdbcType=BIGINT}
</if> </if>
...@@ -34,16 +35,26 @@ ...@@ -34,16 +35,26 @@
</if> </if>
</select> </select>
<insert id="insert" parameterType="com.taosdata.example.springbootdemo.domain.Weather" > <insert id="insert" parameterType="com.taosdata.example.springbootdemo.domain.Weather">
insert into test.weather (ts, temperature, humidity) values (now, #{temperature,jdbcType=INTEGER}, #{humidity,jdbcType=FLOAT}) insert into test.t#{groupId} (ts, temperature, humidity) values (#{ts}, ${temperature}, ${humidity})
</insert> </insert>
<insert id="batchInsert" parameterType="java.util.List" > <select id="getSubTables" resultType="String">
insert into test.weather (ts, temperature, humidity) values select tbname from test.weather
<foreach separator=" " collection="list" item="weather" index="index" > </select>
(now + #{index}a, #{weather.temperature}, #{weather.humidity})
</foreach>
</insert>
<select id="count" resultType="int">
select count(*) from test.weather
</select>
<resultMap id="avgResultSet" type="com.taosdata.example.springbootdemo.domain.Weather">
<id column="ts" jdbcType="TIMESTAMP" property="ts" />
<result column="avg(temperature)" jdbcType="FLOAT" property="temperature" />
<result column="avg(humidity)" jdbcType="FLOAT" property="humidity" />
</resultMap>
<select id="avg" resultMap="avgResultSet">
select avg(temperature), avg(humidity)from test.weather interval(1m)
</select>
</mapper> </mapper>
\ No newline at end of file
...@@ -6,12 +6,21 @@ import java.sql.Timestamp; ...@@ -6,12 +6,21 @@ import java.sql.Timestamp;
public class Weather { public class Weather {
@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss.SSS",timezone = "GMT+8") @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss.SSS", timezone = "GMT+8")
private Timestamp ts; private Timestamp ts;
private float temperature;
private float humidity;
private String location;
private int groupId;
private int temperature; public Weather() {
}
private float humidity; public Weather(Timestamp ts, float temperature, float humidity) {
this.ts = ts;
this.temperature = temperature;
this.humidity = humidity;
}
public Timestamp getTs() { public Timestamp getTs() {
return ts; return ts;
...@@ -21,11 +30,11 @@ public class Weather { ...@@ -21,11 +30,11 @@ public class Weather {
this.ts = ts; this.ts = ts;
} }
public int getTemperature() { public float getTemperature() {
return temperature; return temperature;
} }
public void setTemperature(int temperature) { public void setTemperature(float temperature) {
this.temperature = temperature; this.temperature = temperature;
} }
...@@ -36,4 +45,20 @@ public class Weather { ...@@ -36,4 +45,20 @@ public class Weather {
public void setHumidity(float humidity) { public void setHumidity(float humidity) {
this.humidity = humidity; this.humidity = humidity;
} }
public String getLocation() {
return location;
}
public void setLocation(String location) {
this.location = location;
}
public int getGroupId() {
return groupId;
}
public void setGroupId(int groupId) {
this.groupId = groupId;
}
} }
...@@ -5,25 +5,41 @@ import com.taosdata.example.springbootdemo.domain.Weather; ...@@ -5,25 +5,41 @@ import com.taosdata.example.springbootdemo.domain.Weather;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import java.sql.Timestamp;
import java.util.List; import java.util.List;
import java.util.Map;
import java.util.Random;
@Service @Service
public class WeatherService { public class WeatherService {
@Autowired @Autowired
private WeatherMapper weatherMapper; private WeatherMapper weatherMapper;
private Random random = new Random(System.currentTimeMillis());
private String[] locations = {"北京", "上海", "广州", "深圳", "天津"};
public boolean init() { public int init() {
weatherMapper.dropDB();
weatherMapper.createDB(); weatherMapper.createDB();
weatherMapper.createTable(); weatherMapper.createSuperTable();
return true; long ts = System.currentTimeMillis();
long thirtySec = 1000 * 30;
int count = 0;
for (int i = 0; i < 20; i++) {
Weather weather = new Weather(new Timestamp(ts + (thirtySec * i)), 30 * random.nextFloat(), random.nextInt(100));
weather.setLocation(locations[random.nextInt(locations.length)]);
weather.setGroupId(i % locations.length);
weatherMapper.createTable(weather);
count += weatherMapper.insert(weather);
}
return count;
} }
public List<Weather> query(Long limit, Long offset) { public List<Weather> query(Long limit, Long offset) {
return weatherMapper.select(limit, offset); return weatherMapper.select(limit, offset);
} }
public int save(int temperature, float humidity) { public int save(float temperature, int humidity) {
Weather weather = new Weather(); Weather weather = new Weather();
weather.setTemperature(temperature); weather.setTemperature(temperature);
weather.setHumidity(humidity); weather.setHumidity(humidity);
...@@ -31,8 +47,15 @@ public class WeatherService { ...@@ -31,8 +47,15 @@ public class WeatherService {
return weatherMapper.insert(weather); return weatherMapper.insert(weather);
} }
public int save(List<Weather> weatherList) { public int count() {
return weatherMapper.batchInsert(weatherList); return weatherMapper.count();
} }
public List<String> getSubTables() {
return weatherMapper.getSubTables();
}
public List<Weather> avg() {
return weatherMapper.avg();
}
} }
# datasource config - JDBC-JNI # datasource config - JDBC-JNI
spring.datasource.driver-class-name=com.taosdata.jdbc.TSDBDriver #spring.datasource.driver-class-name=com.taosdata.jdbc.TSDBDriver
spring.datasource.url=jdbc:TAOS://127.0.0.1:6030/test?timezone=UTC-8&charset=UTF-8&locale=en_US.UTF-8 #spring.datasource.url=jdbc:TAOS://127.0.0.1:6030/test?timezone=UTC-8&charset=UTF-8&locale=en_US.UTF-8
spring.datasource.username=root #spring.datasource.username=root
spring.datasource.password=taosdata #spring.datasource.password=taosdata
# datasource config - JDBC-RESTful # datasource config - JDBC-RESTful
#spring.datasource.driver-class-name=com.taosdata.jdbc.rs.RestfulDriver spring.datasource.driver-class-name=com.taosdata.jdbc.rs.RestfulDriver
#spring.datasource.url=jdbc:TAOS-RS://master:6041/test?user=root&password=taosdata spring.datasource.url=jdbc:TAOS-RS://master:6041/test?timezone=UTC-8&charset=UTF-8&locale=en_US.UTF-8
spring.datasource.username=root
spring.datasource.password=taosdata
spring.datasource.druid.initial-size=5 spring.datasource.druid.initial-size=5
spring.datasource.druid.min-idle=5 spring.datasource.druid.min-idle=5
......
...@@ -4,7 +4,7 @@ import com.taosdata.taosdemo.components.DataSourceFactory; ...@@ -4,7 +4,7 @@ import com.taosdata.taosdemo.components.DataSourceFactory;
import com.taosdata.taosdemo.components.JdbcTaosdemoConfig; import com.taosdata.taosdemo.components.JdbcTaosdemoConfig;
import com.taosdata.taosdemo.domain.SuperTableMeta; import com.taosdata.taosdemo.domain.SuperTableMeta;
import com.taosdata.taosdemo.service.DatabaseService; import com.taosdata.taosdemo.service.DatabaseService;
import com.taosdata.taosdemo.service.QueryService; import com.taosdata.taosdemo.service.SqlExecuteTask;
import com.taosdata.taosdemo.service.SubTableService; import com.taosdata.taosdemo.service.SubTableService;
import com.taosdata.taosdemo.service.SuperTableService; import com.taosdata.taosdemo.service.SuperTableService;
import com.taosdata.taosdemo.service.data.SuperTableMetaGenerator; import com.taosdata.taosdemo.service.data.SuperTableMetaGenerator;
...@@ -32,6 +32,17 @@ public class TaosDemoApplication { ...@@ -32,6 +32,17 @@ public class TaosDemoApplication {
} }
// 初始化 // 初始化
final DataSource dataSource = DataSourceFactory.getInstance(config.host, config.port, config.user, config.password); final DataSource dataSource = DataSourceFactory.getInstance(config.host, config.port, config.user, config.password);
if (config.executeSql != null && !config.executeSql.isEmpty() && !config.executeSql.replaceAll("\\s", "").isEmpty()) {
Thread task = new Thread(new SqlExecuteTask(dataSource, config.executeSql));
task.start();
try {
task.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return;
}
final DatabaseService databaseService = new DatabaseService(dataSource); final DatabaseService databaseService = new DatabaseService(dataSource);
final SuperTableService superTableService = new SuperTableService(dataSource); final SuperTableService superTableService = new SuperTableService(dataSource);
final SubTableService subTableService = new SubTableService(dataSource); final SubTableService subTableService = new SubTableService(dataSource);
...@@ -96,7 +107,6 @@ public class TaosDemoApplication { ...@@ -96,7 +107,6 @@ public class TaosDemoApplication {
// 查询 // 查询
/**********************************************************************************/ /**********************************************************************************/
// 删除表 // 删除表
if (config.dropTable) { if (config.dropTable) {
......
...@@ -42,7 +42,7 @@ public final class JdbcTaosdemoConfig { ...@@ -42,7 +42,7 @@ public final class JdbcTaosdemoConfig {
public int rate = 10; public int rate = 10;
public long range = 1000l; public long range = 1000l;
// select task // select task
public String executeSql;
// drop task // drop task
public boolean dropTable = false; public boolean dropTable = false;
...@@ -89,7 +89,7 @@ public final class JdbcTaosdemoConfig { ...@@ -89,7 +89,7 @@ public final class JdbcTaosdemoConfig {
System.out.println("-rate The proportion of data out of order. effective only if order is 1. min 0, max 100, default is 10"); System.out.println("-rate The proportion of data out of order. effective only if order is 1. min 0, max 100, default is 10");
System.out.println("-range The range of data out of order. effective only if order is 1. default is 1000 ms"); System.out.println("-range The range of data out of order. effective only if order is 1. default is 1000 ms");
// query task // query task
// System.out.println("-sqlFile The select sql file"); System.out.println("-executeSql execute a specific sql.");
// drop task // drop task
System.out.println("-dropTable Drop data before quit. Default is false"); System.out.println("-dropTable Drop data before quit. Default is false");
System.out.println("--help Give this help list"); System.out.println("--help Give this help list");
...@@ -207,6 +207,9 @@ public final class JdbcTaosdemoConfig { ...@@ -207,6 +207,9 @@ public final class JdbcTaosdemoConfig {
range = Integer.parseInt(args[++i]); range = Integer.parseInt(args[++i]);
} }
// select task // select task
if ("-executeSql".equals(args[i]) && i < args.length - 1) {
executeSql = args[++i];
}
// drop task // drop task
if ("-dropTable".equals(args[i]) && i < args.length - 1) { if ("-dropTable".equals(args[i]) && i < args.length - 1) {
......
package com.taosdata.taosdemo.service;
import com.taosdata.taosdemo.utils.Printer;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class SqlExecuteTask implements Runnable {
private final DataSource dataSource;
private final String sql;
public SqlExecuteTask(DataSource dataSource, String sql) {
this.dataSource = dataSource;
this.sql = sql;
}
@Override
public void run() {
try (Connection conn = dataSource.getConnection(); Statement stmt = conn.createStatement()) {
long start = System.currentTimeMillis();
boolean execute = stmt.execute(sql);
long end = System.currentTimeMillis();
if (execute) {
ResultSet rs = stmt.getResultSet();
Printer.printResult(rs);
} else {
Printer.printSql(sql, true, (end - start));
}
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.taosdemo.utils;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
public class Printer {
public static void printResult(ResultSet resultSet) throws SQLException {
ResultSetMetaData metaData = resultSet.getMetaData();
while (resultSet.next()) {
for (int i = 1; i <= metaData.getColumnCount(); i++) {
String columnLabel = metaData.getColumnLabel(i);
String value = resultSet.getString(i);
System.out.printf("%s: %s\t", columnLabel, value);
}
System.out.println();
}
}
public static void printSql(String sql, boolean succeed, long cost) {
System.out.println("[ " + (succeed ? "OK" : "ERROR!") + " ] time cost: " + cost + " ms, execute statement ====> " + sql);
}
private Printer() {
}
}
...@@ -92,15 +92,14 @@ void Test(TAOS *taos, char *qstr, int index) { ...@@ -92,15 +92,14 @@ void Test(TAOS *taos, char *qstr, int index) {
// printf("insert row: %i, reason:%s\n", i, taos_errstr(taos)); // printf("insert row: %i, reason:%s\n", i, taos_errstr(taos));
// } // }
TAOS_RES *result1 = taos_query(taos, qstr); TAOS_RES *result1 = taos_query(taos, qstr);
if (result1) { if (result1 == NULL || taos_errno(result1) != 0) {
printf("insert row: %i\n", i); printf("failed to insert row, reason:%s\n", taos_errstr(result1));
} else {
printf("failed to insert row: %i, reason:%s\n", i, "null result"/*taos_errstr(result)*/);
taos_free_result(result1); taos_free_result(result1);
exit(1); exit(1);
} else {
printf("insert row: %i\n", i);
} }
taos_free_result(result1); taos_free_result(result1);
} }
printf("success to insert rows, total %d rows\n", i); printf("success to insert rows, total %d rows\n", i);
......
...@@ -11,15 +11,9 @@ ...@@ -11,15 +11,9 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
import os
import sys
sys.path.insert(0, os.getcwd())
from fabric import Connection from fabric import Connection
from util.sql import *
from util.log import *
import taos
import random import random
import threading import time
import logging import logging
class Node: class Node:
...@@ -76,6 +70,19 @@ class Node: ...@@ -76,6 +70,19 @@ class Node:
print("remove taosd error for node %d " % self.index) print("remove taosd error for node %d " % self.index)
logging.exception(e) logging.exception(e)
def forceStopOneTaosd(self):
try:
self.conn.run("kill -9 $(ps -ax|grep taosd|awk '{print $1}')")
except Exception as e:
print("kill taosd error on node%d " % self.index)
def startOneTaosd(self):
try:
self.conn.run("nohup taosd -c /etc/taos/ > /dev/null 2>&1 &")
except Exception as e:
print("start taosd error on node%d " % self.index)
logging.exception(e)
def installTaosd(self, packagePath): def installTaosd(self, packagePath):
self.conn.put(packagePath, self.homeDir) self.conn.put(packagePath, self.homeDir)
self.conn.cd(self.homeDir) self.conn.cd(self.homeDir)
...@@ -122,100 +129,51 @@ class Node: ...@@ -122,100 +129,51 @@ class Node:
class Nodes: class Nodes:
def __init__(self): def __init__(self):
self.node1 = Node(1, 'root', '52.151.60.239', 'node1', 'r', '/root/') self.tdnodes = []
self.node2 = Node(2, 'root', '52.183.32.246', 'node1', 'r', '/root/') self.tdnodes.append(Node(0, 'root', '52.143.103.7', 'node1', 'a', '/root/'))
self.node3 = Node(3, 'root', '51.143.46.79', 'node1', 'r', '/root/') self.tdnodes.append(Node(1, 'root', '52.250.48.222', 'node2', 'a', '/root/'))
self.node4 = Node(4, 'root', '52.183.2.76', 'node1', 'r', '/root/') self.tdnodes.append(Node(2, 'root', '51.141.167.23', 'node3', 'a', '/root/'))
self.node5 = Node(5, 'root', '13.66.225.87', 'node1', 'r', '/root/') self.tdnodes.append(Node(3, 'root', '52.247.207.173', 'node4', 'a', '/root/'))
self.tdnodes.append(Node(4, 'root', '51.141.166.100', 'node5', 'a', '/root/'))
def stopOneNode(self, index):
self.tdnodes[index].forceStopOneTaosd()
def startOneNode(self, index):
self.tdnodes[index].startOneTaosd()
def stopAllTaosd(self): def stopAllTaosd(self):
self.node1.stopTaosd() for i in range(len(self.tdnodes)):
self.node2.stopTaosd() self.tdnodes[i].stopTaosd()
self.node3.stopTaosd()
def startAllTaosd(self): def startAllTaosd(self):
self.node1.startTaosd() for i in range(len(self.tdnodes)):
self.node2.startTaosd() self.tdnodes[i].startTaosd()
self.node3.startTaosd()
def restartAllTaosd(self): def restartAllTaosd(self):
self.node1.restartTaosd() for i in range(len(self.tdnodes)):
self.node2.restartTaosd() self.tdnodes[i].restartTaosd()
self.node3.restartTaosd()
def addConfigs(self, configKey, configValue): def addConfigs(self, configKey, configValue):
self.node1.configTaosd(configKey, configValue) for i in range(len(self.tdnodes)):
self.node2.configTaosd(configKey, configValue) self.tdnodes[i].configTaosd(configKey, configValue)
self.node3.configTaosd(configKey, configValue)
def removeConfigs(self, configKey, configValue): def removeConfigs(self, configKey, configValue):
self.node1.removeTaosConfig(configKey, configValue) for i in range(len(self.tdnodes)):
self.node2.removeTaosConfig(configKey, configValue) self.tdnodes[i].removeTaosConfig(configKey, configValue)
self.node3.removeTaosConfig(configKey, configValue)
def removeAllDataFiles(self): def removeAllDataFiles(self):
self.node1.removeData() for i in range(len(self.tdnodes)):
self.node2.removeData() self.tdnodes[i].removeData()
self.node3.removeData()
# kill taosd randomly every 10 mins
class ClusterTest: nodes = Nodes()
def __init__(self, hostName): loop = 0
self.host = hostName while True:
self.user = "root" loop = loop + 1
self.password = "taosdata" index = random.randint(0, 4)
self.config = "/etc/taos" print("loop: %d, kill taosd on node%d" %(loop, index))
self.dbName = "mytest" nodes.stopOneNode(index)
self.stbName = "meters" time.sleep(60)
self.numberOfThreads = 20 nodes.startOneNode(index)
self.numberOfTables = 10000 time.sleep(600)
self.numberOfRecords = 1000 \ No newline at end of file
self.tbPrefix = "t"
self.ts = 1538548685000
self.repeat = 1
def connectDB(self):
self.conn = taos.connect(
host=self.host,
user=self.user,
password=self.password,
config=self.config)
def createSTable(self, replica):
cursor = self.conn.cursor()
tdLog.info("drop database if exists %s" % self.dbName)
cursor.execute("drop database if exists %s" % self.dbName)
tdLog.info("create database %s replica %d" % (self.dbName, replica))
cursor.execute("create database %s replica %d" % (self.dbName, replica))
tdLog.info("use %s" % self.dbName)
cursor.execute("use %s" % self.dbName)
tdLog.info("drop table if exists %s" % self.stbName)
cursor.execute("drop table if exists %s" % self.stbName)
tdLog.info("create table %s(ts timestamp, current float, voltage int, phase int) tags(id int)" % self.stbName)
cursor.execute("create table %s(ts timestamp, current float, voltage int, phase int) tags(id int)" % self.stbName)
cursor.close()
def insertData(self, threadID):
print("Thread %d: starting" % threadID)
cursor = self.conn.cursor()
tablesPerThread = int(self.numberOfTables / self.numberOfThreads)
baseTableID = tablesPerThread * threadID
for i in range (tablesPerThread):
cursor.execute("create table %s%d using %s tags(%d)" % (self.tbPrefix, baseTableID + i, self.stbName, baseTableID + i))
query = "insert into %s%d values" % (self.tbPrefix, baseTableID + i)
base = self.numberOfRecords * i
for j in range(self.numberOfRecords):
query += "(%d, %f, %d, %d)" % (self.ts + base + j, random.random(), random.randint(210, 230), random.randint(0, 10))
cursor.execute(query)
cursor.close()
print("Thread %d: finishing" % threadID)
def run(self):
threads = []
tdLog.info("Inserting data")
for i in range(self.numberOfThreads):
thread = threading.Thread(target=self.insertData, args=(i,))
threads.append(thread)
thread.start()
for i in range(self.numberOfThreads):
threads[i].join()
\ No newline at end of file
...@@ -350,18 +350,27 @@ class ConcurrentInquiry: ...@@ -350,18 +350,27 @@ class ConcurrentInquiry:
cl.execute("create database if not exists %s;" %self.dbname) cl.execute("create database if not exists %s;" %self.dbname)
cl.execute("use %s" % self.dbname) cl.execute("use %s" % self.dbname)
for k in range(stableNum): for k in range(stableNum):
sql="create table %s (ts timestamp, c1 int, c2 float, c3 bigint, c4 smallint, c5 tinyint, c6 double, c7 bool,c8 binary(20),c9 nchar(20)) \ sql="create table %s (ts timestamp, c1 int, c2 float, c3 bigint, c4 smallint, c5 tinyint, c6 double, c7 bool,c8 binary(20),c9 nchar(20),c11 int unsigned,c12 smallint unsigned,c13 tinyint unsigned,c14 bigint unsigned) \
tags(t1 int, t2 float, t3 bigint, t4 smallint, t5 tinyint, t6 double, t7 bool,t8 binary(20),t9 nchar(20))" % (self.stb_prefix+str(k)) tags(t1 int, t2 float, t3 bigint, t4 smallint, t5 tinyint, t6 double, t7 bool,t8 binary(20),t9 nchar(20), t11 int unsigned , t12 smallint unsigned , t13 tinyint unsigned , t14 bigint unsigned)" % (self.stb_prefix+str(k))
cl.execute(sql) cl.execute(sql)
for j in range(subtableNum): for j in range(subtableNum):
sql = "create table %s using %s tags(%d,%d,%d,%d,%d,%d,%d,'%s','%s')" % \ if j % 100 == 0:
(self.subtb_prefix+str(k)+'_'+str(j),self.stb_prefix+str(k),j,j/2.0,j%41,j%51,j%53,j*1.0,j%2,'taos'+str(j),'涛思'+str(j)) sql = "create table %s using %s tags(NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL)" % \
(self.subtb_prefix+str(k)+'_'+str(j),self.stb_prefix+str(k))
else:
sql = "create table %s using %s tags(%d,%d,%d,%d,%d,%d,%d,'%s','%s',%d,%d,%d,%d)" % \
(self.subtb_prefix+str(k)+'_'+str(j),self.stb_prefix+str(k),j,j/2.0,j%41,j%51,j%53,j*1.0,j%2,'taos'+str(j),'涛思'+str(j), j%43, j%23 , j%17 , j%3167)
print(sql) print(sql)
cl.execute(sql) cl.execute(sql)
for i in range(insertRows): for i in range(insertRows):
ret = cl.execute( if i % 100 == 0 :
"insert into %s values (%d , %d,%d,%d,%d,%d,%d,%d,'%s','%s')" % ret = cl.execute(
(self.subtb_prefix+str(k)+'_'+str(j),t0+i,i%100,i/2.0,i%41,i%51,i%53,i*1.0,i%2,'taos'+str(i),'涛思'+str(i))) "insert into %s values (%d , NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL)" %
(self.subtb_prefix+str(k)+'_'+str(j), t0+i))
else:
ret = cl.execute(
"insert into %s values (%d , %d,%d,%d,%d,%d,%d,%d,'%s','%s',%d,%d,%d,%d)" %
(self.subtb_prefix+str(k)+'_'+str(j), t0+i, i%100, i/2.0, i%41, i%51, i%53, i*1.0, i%2,'taos'+str(i),'涛思'+str(i), i%43, i%23 , i%17 , i%3167))
cl.close() cl.close()
conn.close() conn.close()
......
...@@ -52,8 +52,8 @@ class TDTestCase: ...@@ -52,8 +52,8 @@ class TDTestCase:
p.start() p.start()
p.join() p.join()
p.terminate() p.terminate()
tdSql.execute("insert into tb values(%d, 1, 2)" % (self.ts + 1)) tdSql.execute("insert into tb(ts, col1, col2) values(%d, 1, 2)" % (self.ts + 2))
print("==============step2") print("==============step2")
tdSql.query("select * from tb") tdSql.query("select * from tb")
......
...@@ -25,18 +25,23 @@ class TDTestCase: ...@@ -25,18 +25,23 @@ class TDTestCase:
self.tables = 10 self.tables = 10
self.rows = 20 self.rows = 20
self.columns = 50
self.perfix = 't' self.perfix = 't'
self.ts = 1601481600000 self.ts = 1601481600000
def insertData(self): def insertData(self):
print("==============step1") print("==============step1")
tdSql.execute("create table st (ts timestamp, c1 int) tags(t1 int)") sql = "create table st(ts timestamp, "
for i in range(self.columns - 1):
sql += "c%d int, " % (i + 1)
sql += "c50 int) tags(t1 int)"
tdSql.execute(sql)
for i in range(self.tables): for i in range(self.tables):
tdSql.execute("create table %s%d using st tags(%d)" % (self.perfix, i, i)) tdSql.execute("create table %s%d using st tags(%d)" % (self.perfix, i, i))
for j in range(self.rows): for j in range(self.rows):
tc = self.ts + j * 60000 tc = self.ts + j * 60000
tdSql.execute("insert into %s%d values(%d, %d)" %(self.perfix, i, tc, j)) tdSql.execute("insert into %s%d(ts, c1) values(%d, %d)" %(self.perfix, i, tc, j))
def executeQueries(self): def executeQueries(self):
print("==============step2") print("==============step2")
...@@ -66,29 +71,29 @@ class TDTestCase: ...@@ -66,29 +71,29 @@ class TDTestCase:
tdSql.checkData(0, 0, 19) tdSql.checkData(0, 0, 19)
tc = self.ts + 1 * 3600000 tc = self.ts + 1 * 3600000
tdSql.execute("insert into %s%d values(%d, %d)" %(self.perfix, 1, tc, 10)) tdSql.execute("insert into %s%d(ts, c1) values(%d, %d)" %(self.perfix, 1, tc, 10))
tc = self.ts + 3 * 3600000 tc = self.ts + 3 * 3600000
tdSql.execute("insert into %s%d values(%d, null)" %(self.perfix, 1, tc)) tdSql.execute("insert into %s%d(ts, c1) values(%d, null)" %(self.perfix, 1, tc))
tc = self.ts + 5 * 3600000 tc = self.ts + 5 * 3600000
tdSql.execute("insert into %s%d values(%d, %d)" %(self.perfix, 1, tc, -1)) tdSql.execute("insert into %s%d(ts, c1) values(%d, %d)" %(self.perfix, 1, tc, -1))
tc = self.ts + 7 * 3600000 tc = self.ts + 7 * 3600000
tdSql.execute("insert into %s%d values(%d, null)" %(self.perfix, 1, tc)) tdSql.execute("insert into %s%d(ts, c1) values(%d, null)" %(self.perfix, 1, tc))
def insertData2(self): def insertData2(self):
tc = self.ts + 1 * 3600000 tc = self.ts + 1 * 3600000
tdSql.execute("insert into %s%d values(%d, %d)" %(self.perfix, 1, tc, 10)) tdSql.execute("insert into %s%d(ts, c1) values(%d, %d)" %(self.perfix, 1, tc, 10))
tc = self.ts + 3 * 3600000 tc = self.ts + 3 * 3600000
tdSql.execute("insert into %s%d values(%d, null)" %(self.perfix, 1, tc)) tdSql.execute("insert into %s%d(ts, c1) values(%d, null)" %(self.perfix, 1, tc))
tc = self.ts + 5 * 3600000 tc = self.ts + 5 * 3600000
tdSql.execute("insert into %s%d values(%d, %d)" %(self.perfix, 1, tc, -1)) tdSql.execute("insert into %s%d(ts, c1) values(%d, %d)" %(self.perfix, 1, tc, -1))
tc = self.ts + 7 * 3600000 tc = self.ts + 7 * 3600000
tdSql.execute("insert into %s%d values(%d, null)" %(self.perfix, 1, tc)) tdSql.execute("insert into %s%d(ts, c1) values(%d, null)" %(self.perfix, 1, tc))
def executeQueries2(self): def executeQueries2(self):
# For stable # For stable
...@@ -164,6 +169,9 @@ class TDTestCase: ...@@ -164,6 +169,9 @@ class TDTestCase:
self.executeQueries() self.executeQueries()
self.insertData2() self.insertData2()
self.executeQueries2() self.executeQueries2()
tdDnodes.stop(1)
tdDnodes.start(1)
self.executeQueries2()
tdSql.execute("alter database test2 cachelast 0") tdSql.execute("alter database test2 cachelast 0")
self.executeQueries2() self.executeQueries2()
......
...@@ -24,7 +24,7 @@ class TDTestCase: ...@@ -24,7 +24,7 @@ class TDTestCase:
tdLog.debug("start to execute %s" % __file__) tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql) tdSql.init(conn.cursor(), logSql)
self.numberOfTables = 10000 self.numberOfTables = 1000
self.numberOfRecords = 100 self.numberOfRecords = 100
def getBuildPath(self): def getBuildPath(self):
......
...@@ -209,7 +209,7 @@ sql alter database db wal 1 ...@@ -209,7 +209,7 @@ sql alter database db wal 1
sql alter database db wal 2 sql alter database db wal 2
sql alter database db wal 1 sql alter database db wal 1
sql alter database db wal 2 sql alter database db wal 2
sql_error alter database db wal 0 sql alter database db wal 0
sql_error alter database db wal 3 sql_error alter database db wal 3
sql_error alter database db wal 4 sql_error alter database db wal 4
sql_error alter database db wal -1 sql_error alter database db wal -1
......
...@@ -39,14 +39,14 @@ print =============== step3 - query data ...@@ -39,14 +39,14 @@ print =============== step3 - query data
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from d1.table_rest' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from d1.table_rest' 127.0.0.1:7111/rest/sql
print curl 127.0.0.1:7111/rest/sql -----> $system_content print curl 127.0.0.1:7111/rest/sql -----> $system_content
if $system_content != @{"status":"succ","head":["ts","i"],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10]],"rows":10}@ then if $system_content != @{"status":"succ","head":["ts","i"],"column_meta":[["ts",9,8],["i",4,4]],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10]],"rows":10}@ then
return -1 return -1
endi endi
print =============== step4 - insert data print =============== step4 - insert data
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d1.table_rest values('2017-12-25 21:28:51.022', 11)" 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d1.table_rest values('2017-12-25 21:28:51.022', 11)" 127.0.0.1:7111/rest/sql
print curl 127.0.0.1:7111/rest/sql -----> $system_content print curl 127.0.0.1:7111/rest/sql -----> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[1]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[1]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -54,7 +54,7 @@ print =============== step5 - query data ...@@ -54,7 +54,7 @@ print =============== step5 - query data
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from d1.table_rest' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from d1.table_rest' 127.0.0.1:7111/rest/sql
print curl 127.0.0.1:7111/rest/sql -----> $system_content print curl 127.0.0.1:7111/rest/sql -----> $system_content
if $system_content != @{"status":"succ","head":["ts","i"],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10],["2017-12-25 21:28:51.022",11]],"rows":11}@ then if $system_content != @{"status":"succ","head":["ts","i"],"column_meta":[["ts",9,8],["i",4,4]],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10],["2017-12-25 21:28:51.022",11]],"rows":11}@ then
return -1 return -1
endi endi
...@@ -79,4 +79,4 @@ if $system_content != @{"status":"error","code":3,"desc":"Authentication failure ...@@ -79,4 +79,4 @@ if $system_content != @{"status":"error","code":3,"desc":"Authentication failure
return -1 return -1
endi endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s stop -x SIGINT
\ No newline at end of file
...@@ -88,13 +88,13 @@ print =============== step2 - no db ...@@ -88,13 +88,13 @@ print =============== step2 - no db
#11 #11
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql
print 11-> $system_content print 11-> $system_content
if $system_content != @{"status":"succ","head":["name","created_time","ntables","vgroups","replica","quorum","days","keep0,keep1,keep(D)","cache(MB)","blocks","minrows","maxrows","wallevel","fsync","comp","cachelast","precision","update","status"],"data":[],"rows":0}@ then if $system_content != @{"status":"succ","head":["name","created_time","ntables","vgroups","replica","quorum","days","keep0,keep1,keep(D)","cache(MB)","blocks","minrows","maxrows","wallevel","fsync","comp","cachelast","precision","update","status"],"column_meta":[["name",8,32],["created_time",9,8],["ntables",4,4],["vgroups",4,4],["replica",3,2],["quorum",3,2],["days",3,2],["keep0,keep1,keep(D)",8,24],["cache(MB)",4,4],["blocks",4,4],["minrows",4,4],["maxrows",4,4],["wallevel",2,1],["fsync",4,4],["comp",2,1],["cachelast",2,1],["precision",8,3],["update",2,1],["status",8,10]],"data":[],"rows":0}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database d1' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database d1' 127.0.0.1:7111/rest/sql
print 12-> $system_content print 12-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[0]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -160,26 +160,26 @@ endi ...@@ -160,26 +160,26 @@ endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' create table d1.t1 (ts timestamp, speed int)' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' create table d1.t1 (ts timestamp, speed int)' 127.0.0.1:7111/rest/sql
print 22-> $system_content print 22-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[0]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql
print 23-> $system_content print 23-> $system_content
if $system_content != @{"status":"succ","head":["ts","speed"],"data":[],"rows":0}@ then if $system_content != @{"status":"succ","head":["ts","speed"],"column_meta":[["ts",9,8],["speed",4,4]],"data":[],"rows":0}@ then
return -1 return -1
endi endi
#24 #24
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d1.t1 values('2017-12-25 21:28:41.022', 1)" 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d1.t1 values('2017-12-25 21:28:41.022', 1)" 127.0.0.1:7111/rest/sql
print 24-> $system_content print 24-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[1]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[1]],"rows":1}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql
print 25-> $system_content print 25-> $system_content
if $system_content != @{"status":"succ","head":["ts","speed"],"data":[["2017-12-25 21:28:41.022",1]],"rows":1}@ then if $system_content != @{"status":"succ","head":["ts","speed"],"column_meta":[["ts",9,8],["speed",4,4]],"data":[["2017-12-25 21:28:41.022",1]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -208,32 +208,32 @@ system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl ...@@ -208,32 +208,32 @@ system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl
#27 #27
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d1.t1 ' 127.0.0.1:7111/rest/sql
print 27-> $system_content print 27-> $system_content
if $system_content != @{"status":"succ","head":["ts","speed"],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10],["2017-12-25 21:28:51.022",11]],"rows":11}@ then if $system_content != @{"status":"succ","head":["ts","speed"],"column_meta":[["ts",9,8],["speed",4,4]],"data":[["2017-12-25 21:28:41.022",1],["2017-12-25 21:28:42.022",2],["2017-12-25 21:28:43.022",3],["2017-12-25 21:28:44.022",4],["2017-12-25 21:28:45.022",5],["2017-12-25 21:28:46.022",6],["2017-12-25 21:28:47.022",7],["2017-12-25 21:28:48.022",8],["2017-12-25 21:28:49.022",9],["2017-12-25 21:28:50.022",10],["2017-12-25 21:28:51.022",11]],"rows":11}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database d2' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database d2' 127.0.0.1:7111/rest/sql
print 28-> $system_content print 28-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[0]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' create table d2.t1 (ts timestamp, speed int)' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' create table d2.t1 (ts timestamp, speed int)' 127.0.0.1:7111/rest/sql
print 29-> $system_content print 29-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[0]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then
return -1 return -1
endi endi
#30 #30
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d2.t1 values('2017-12-25 21:28:41.022', 1)" 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d "insert into d2.t1 values('2017-12-25 21:28:41.022', 1)" 127.0.0.1:7111/rest/sql
print 30-> $system_content print 30-> $system_content
if $system_content != @{"status":"succ","head":["affected_rows"],"data":[[1]],"rows":1}@ then if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[1]],"rows":1}@ then
return -1 return -1
endi endi
system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d2.t1 ' 127.0.0.1:7111/rest/sql system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d ' select * from d2.t1 ' 127.0.0.1:7111/rest/sql
print 31-> $system_content print 31-> $system_content
if $system_content != @{"status":"succ","head":["ts","speed"],"data":[["2017-12-25 21:28:41.022",1]],"rows":1}@ then if $system_content != @{"status":"succ","head":["ts","speed"],"column_meta":[["ts",9,8],["speed",4,4]],"data":[["2017-12-25 21:28:41.022",1]],"rows":1}@ then
return -1 return -1
endi endi
......
...@@ -285,8 +285,8 @@ system_content curl -u root:taosdata -d 'select count(*) from db.win_cpu' 127.0 ...@@ -285,8 +285,8 @@ system_content curl -u root:taosdata -d 'select count(*) from db.win_cpu' 127.0
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["count(*)"],"data":[[3]],"rows":1}@ then if $system_content != @{"status":"succ","head":["count(*)"],"column_meta":[["count(*)",5,8]],"data":[[3]],"rows":1}@ then
return -1 return -1
endi endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s stop -x SIGINT
\ No newline at end of file
...@@ -169,8 +169,8 @@ sql_error create database $db cache 0 ...@@ -169,8 +169,8 @@ sql_error create database $db cache 0
sql_error create database $db ctime 29 sql_error create database $db ctime 29
sql_error create database $db ctime 40961 sql_error create database $db ctime 40961
# wal {1, 2} # wal {0, 2}
sql_error create database $db wal 0 #sql_error create database $db wal 0
sql_error create database $db wal -1 sql_error create database $db wal -1
sql_error create database $db wal 3 sql_error create database $db wal 3
......
...@@ -15,18 +15,18 @@ print ========== step2 ...@@ -15,18 +15,18 @@ print ========== step2
# return -1 # return -1
#step21: #step21:
sql drop table log.dn -x step22 sql drop table log.dn -x step22
return -1 # return -1
step22: step22:
sql drop user log -x step23 sql drop user log -x step23
return -1 # return -1
step23: step23:
print ========== step3 print ========== step3
sleep 2000 sleep 2000
sql select * from log.dn #sql select * from log.dn
if $rows == 0 then #if $rows == 0 then
return -1 # return -1
endi #endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s stop -x SIGINT
\ No newline at end of file
...@@ -169,7 +169,7 @@ endi ...@@ -169,7 +169,7 @@ endi
system_content curl -u root:taosdata -d 'select * from db.sys_cpu_d_bbb_lga_1_web01' 127.0.0.1:7111/rest/sql/ system_content curl -u root:taosdata -d 'select * from db.sys_cpu_d_bbb_lga_1_web01' 127.0.0.1:7111/rest/sql/
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["ts","value"],"data":[["2012-09-05 20:00:00.000",18.000000000]],"rows":1}@ then if $system_content != @{"status":"succ","head":["ts","value"],"column_meta":[["ts",9,8],["value",7,8]],"data":[["2012-09-05 20:00:00.000",18.000000000]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -186,7 +186,7 @@ system_content curl -u root:taosdata -d 'select * from db.sys_cpu_d_bbb_lga_1_w ...@@ -186,7 +186,7 @@ system_content curl -u root:taosdata -d 'select * from db.sys_cpu_d_bbb_lga_1_w
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["ts","value"],"data":[["2012-09-05 20:00:00.000",18.000000000],["2012-09-05 20:00:05.000",18.000000000]],"rows":2}@ then if $system_content != @{"status":"succ","head":["ts","value"],"column_meta":[["ts",9,8],["value",7,8]],"data":[["2012-09-05 20:00:00.000",18.000000000],["2012-09-05 20:00:05.000",18.000000000]],"rows":2}@ then
return -1 return -1
endi endi
...@@ -194,7 +194,7 @@ system_content curl -u root:taosdata -d 'select count(*) from db.sys_cpu_d_bbb' ...@@ -194,7 +194,7 @@ system_content curl -u root:taosdata -d 'select count(*) from db.sys_cpu_d_bbb'
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["count(*)"],"data":[[3]],"rows":1}@ then if $system_content != @{"status":"succ","head":["count(*)"],"column_meta":[["count(*)",5,8]],"data":[[3]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -211,7 +211,7 @@ system_content curl -u root:taosdata -d 'select * from db.sys_mem_d_bbb_lga_1_w ...@@ -211,7 +211,7 @@ system_content curl -u root:taosdata -d 'select * from db.sys_mem_d_bbb_lga_1_w
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["ts","value"],"data":[["2012-09-05 20:00:00.000",8.000000000],["2012-09-05 20:00:05.000",9.000000000]],"rows":2}@ then if $system_content != @{"status":"succ","head":["ts","value"],"column_meta":[["ts",9,8],["value",7,8]],"data":[["2012-09-05 20:00:00.000",8.000000000],["2012-09-05 20:00:05.000",9.000000000]],"rows":2}@ then
return -1 return -1
endi endi
...@@ -219,7 +219,7 @@ system_content curl -u root:taosdata -d 'select count(*) from db.sys_mem_d_bbb' ...@@ -219,7 +219,7 @@ system_content curl -u root:taosdata -d 'select count(*) from db.sys_mem_d_bbb'
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["count(*)"],"data":[[2]],"rows":1}@ then if $system_content != @{"status":"succ","head":["count(*)"],"column_meta":[["count(*)",5,8]],"data":[[2]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -233,7 +233,7 @@ system_content curl -u root:taosdata -d '[{"metric": "sys_cpu","timestamp": 134 ...@@ -233,7 +233,7 @@ system_content curl -u root:taosdata -d '[{"metric": "sys_cpu","timestamp": 134
system_content curl -u root:taosdata -d 'select count(*) from db.sys_cpu_d_bbb' 127.0.0.1:7111/rest/sql/ system_content curl -u root:taosdata -d 'select count(*) from db.sys_cpu_d_bbb' 127.0.0.1:7111/rest/sql/
print $system_content print $system_content
if $system_content != @{"status":"succ","head":["count(*)"],"data":[[7]],"rows":1}@ then if $system_content != @{"status":"succ","head":["count(*)"],"column_meta":[["count(*)",5,8]],"data":[[7]],"rows":1}@ then
return -1 return -1
endi endi
...@@ -244,4 +244,4 @@ system sh/exec.sh -n dnode4 -s stop -x SIGINT ...@@ -244,4 +244,4 @@ system sh/exec.sh -n dnode4 -s stop -x SIGINT
system sh/exec.sh -n dnode5 -s stop -x SIGINT system sh/exec.sh -n dnode5 -s stop -x SIGINT
system sh/exec.sh -n dnode6 -s stop -x SIGINT system sh/exec.sh -n dnode6 -s stop -x SIGINT
system sh/exec.sh -n dnode7 -s stop -x SIGINT system sh/exec.sh -n dnode7 -s stop -x SIGINT
system sh/exec.sh -n dnode8 -s stop -x SIGINT system sh/exec.sh -n dnode8 -s stop -x SIGINT
\ No newline at end of file
...@@ -160,9 +160,10 @@ function runPyCaseOneByOnefq { ...@@ -160,9 +160,10 @@ function runPyCaseOneByOnefq {
totalFailed=0 totalFailed=0
totalPyFailed=0 totalPyFailed=0
totalJDBCFailed=0 totalJDBCFailed=0
totalUnitFailed=0
corepath=`grep -oP '.*(?=core_)' /proc/sys/kernel/core_pattern||grep -oP '.*(?=core-)' /proc/sys/kernel/core_pattern` corepath=`grep -oP '.*(?=core_)' /proc/sys/kernel/core_pattern||grep -oP '.*(?=core-)' /proc/sys/kernel/core_pattern`
if [ "$2" != "jdbc" ] && [ "$2" != "python" ]; then if [ "$2" != "jdbc" ] && [ "$2" != "python" ] && [ "$2" != "unit" ]; then
echo "### run TSIM test case ###" echo "### run TSIM test case ###"
cd $tests_dir/script cd $tests_dir/script
...@@ -231,7 +232,7 @@ if [ "$2" != "jdbc" ] && [ "$2" != "python" ]; then ...@@ -231,7 +232,7 @@ if [ "$2" != "jdbc" ] && [ "$2" != "python" ]; then
fi fi
fi fi
if [ "$2" != "sim" ] && [ "$2" != "jdbc" ] ; then if [ "$2" != "sim" ] && [ "$2" != "jdbc" ] && [ "$2" != "unit" ]; then
echo "### run Python test case ###" echo "### run Python test case ###"
cd $tests_dir cd $tests_dir
...@@ -300,8 +301,8 @@ if [ "$2" != "sim" ] && [ "$2" != "jdbc" ] ; then ...@@ -300,8 +301,8 @@ if [ "$2" != "sim" ] && [ "$2" != "jdbc" ] ; then
fi fi
if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$1" == "full" ]; then if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$2" != "unit" ] && [ "$1" == "full" ]; then
echo "### run JDBC test case ###" echo "### run JDBC test cases ###"
cd $tests_dir cd $tests_dir
...@@ -318,7 +319,7 @@ if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$1" == "full" ]; then ...@@ -318,7 +319,7 @@ if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$1" == "full" ]; then
nohup build/bin/taosd -c /etc/taos/ > /dev/null 2>&1 & nohup build/bin/taosd -c /etc/taos/ > /dev/null 2>&1 &
sleep 30 sleep 30
cd $tests_dir/../src/connector/jdbc cd $tests_dir/../src/connector/jdbc
mvn test > jdbc-out.log 2>&1 mvn test > jdbc-out.log 2>&1
tail -n 20 jdbc-out.log tail -n 20 jdbc-out.log
...@@ -343,4 +344,40 @@ if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$1" == "full" ]; then ...@@ -343,4 +344,40 @@ if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$1" == "full" ]; then
dohavecore 1 dohavecore 1
fi fi
exit $(($totalFailed + $totalPyFailed + $totalJDBCFailed)) if [ "$2" != "sim" ] && [ "$2" != "python" ] && [ "$2" != "jdbc" ] && [ "$1" == "full" ]; then
echo "### run Unit tests ###"
stopTaosd
cd $tests_dir
if [[ "$tests_dir" == *"$IN_TDINTERNAL"* ]]; then
cd ../../
else
cd ../
fi
pwd
cd debug/build/bin
nohup ./taosd -c /etc/taos/ > /dev/null 2>&1 &
sleep 30
pwd
./queryTest > unittest-out.log 2>&1
tail -n 20 unittest-out.log
totalUnitTests=`grep "Running" unittest-out.log | awk '{print $3}'`
totalUnitSuccess=`grep 'PASSED' unittest-out.log | awk '{print $4}'`
totalUnitFailed=`expr $totalUnitTests - $totalUnitSuccess`
if [ "$totalUnitSuccess" -gt "0" ]; then
echo -e "\n${GREEN} ### Total $totalUnitSuccess Unit test succeed! ### ${NC}"
fi
if [ "$totalUnitFailed" -ne "0" ]; then
echo -e "\n${RED} ### Total $totalUnitFailed Unit test failed! ### ${NC}"
fi
dohavecore 1
fi
exit $(($totalFailed + $totalPyFailed + $totalJDBCFailed + $totalUnitFailed))
...@@ -146,7 +146,7 @@ char *simGetVariable(SScript *script, char *varName, int32_t varLen) { ...@@ -146,7 +146,7 @@ char *simGetVariable(SScript *script, char *varName, int32_t varLen) {
int32_t simExecuteExpression(SScript *script, char *exp) { int32_t simExecuteExpression(SScript *script, char *exp) {
char * op1, *op2, *var1, *var2, *var3, *rest; char * op1, *op2, *var1, *var2, *var3, *rest;
int32_t op1Len, op2Len, var1Len, var2Len, var3Len, val0, val1; int32_t op1Len, op2Len, var1Len, var2Len, var3Len, val0, val1;
char t0[512], t1[512], t2[512], t3[1024]; char t0[1024], t1[1024], t2[1024], t3[2048];
int32_t result; int32_t result;
rest = paGetToken(exp, &var1, &var1Len); rest = paGetToken(exp, &var1, &var1Len);
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册