You need to sign in or sign up before continuing.
未验证 提交 d3702a7d 编写于 作者: H Haojun Liao 提交者: GitHub

Merge branch 'develop' into origin/feature/TD-3086

...@@ -32,7 +32,7 @@ ELSEIF (TD_WINDOWS) ...@@ -32,7 +32,7 @@ ELSEIF (TD_WINDOWS)
#INSTALL(TARGETS taos RUNTIME DESTINATION driver) #INSTALL(TARGETS taos RUNTIME DESTINATION driver)
#INSTALL(TARGETS shell RUNTIME DESTINATION .) #INSTALL(TARGETS shell RUNTIME DESTINATION .)
IF (TD_MVN_INSTALLED) IF (TD_MVN_INSTALLED)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.30.jar DESTINATION connector/jdbc) INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.31.jar DESTINATION connector/jdbc)
ENDIF () ENDIF ()
ELSEIF (TD_DARWIN) ELSEIF (TD_DARWIN)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh") SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
......
...@@ -81,7 +81,7 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专 ...@@ -81,7 +81,7 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专
## [与其他工具的连接](/connections) ## [与其他工具的连接](/connections)
* [Grafana](/connections#grafana):获取并可视化保存在TDengine的数据 * [Grafana](/connections#grafana):获取并可视化保存在TDengine的数据
* [Matlab](/connections#matlab):通过配置Matlab的JDBC数据源访问保存在TDengine的数据 * [MATLAB](/connections#matlab):通过配置MATLAB的JDBC数据源访问保存在TDengine的数据
* [R](/connections#r):通过配置R的JDBC数据源访问保存在TDengine的数据 * [R](/connections#r):通过配置R的JDBC数据源访问保存在TDengine的数据
* [IDEA Database](https://www.taosdata.com/blog/2020/08/27/1767.html):通过IDEA 数据库管理工具可视化使用 TDengine * [IDEA Database](https://www.taosdata.com/blog/2020/08/27/1767.html):通过IDEA 数据库管理工具可视化使用 TDengine
......
...@@ -9,8 +9,8 @@ TDengine的模块之一是时序数据库。但除此之外,为减少研发的 ...@@ -9,8 +9,8 @@ TDengine的模块之一是时序数据库。但除此之外,为减少研发的
* __10倍以上的性能提升__:定义了创新的数据存储结构,单核每秒能处理至少2万次请求,插入数百万个数据点,读出一千万以上数据点,比现有通用数据库快十倍以上。 * __10倍以上的性能提升__:定义了创新的数据存储结构,单核每秒能处理至少2万次请求,插入数百万个数据点,读出一千万以上数据点,比现有通用数据库快十倍以上。
* __硬件或云服务成本降至1/5__:由于超强性能,计算资源不到通用大数据方案的1/5;通过列式存储和先进的压缩算法,存储空间不到通用数据库的1/10。 * __硬件或云服务成本降至1/5__:由于超强性能,计算资源不到通用大数据方案的1/5;通过列式存储和先进的压缩算法,存储空间不到通用数据库的1/10。
* __全栈时序数据处理引擎__:将数据库、消息队列、缓存、流式计算等功能融为一体,应用无需再集成Kafka/Redis/HBase/Spark/HDFS等软件,大幅降低应用开发和维护的复杂度成本。 * __全栈时序数据处理引擎__:将数据库、消息队列、缓存、流式计算等功能融为一体,应用无需再集成Kafka/Redis/HBase/Spark/HDFS等软件,大幅降低应用开发和维护的复杂度成本。
* __强大的分析功能__:无论是十年前还是一秒钟前的数据,指定时间范围即可查询。数据可在时间轴上或多个设备上进行聚合。即席查询可通过Shell, Python, R, Matlab随时进行。 * __强大的分析功能__:无论是十年前还是一秒钟前的数据,指定时间范围即可查询。数据可在时间轴上或多个设备上进行聚合。即席查询可通过Shell, Python, R, MATLAB随时进行。
* __与第三方工具无缝连接__:不用一行代码,即可与Telegraf, Grafana, EMQ, HiveMQ, Prometheus, Matlab, R等集成。后续将支持OPC, Hadoop, Spark等, BI工具也将无缝连接。 * __与第三方工具无缝连接__:不用一行代码,即可与Telegraf, Grafana, EMQ, HiveMQ, Prometheus, MATLAB, R等集成。后续将支持OPC, Hadoop, Spark等, BI工具也将无缝连接。
* __零运维成本、零学习成本__:安装集群简单快捷,无需分库分表,实时备份。类似标准SQL,支持RESTful, 支持Python/Java/C/C++/C#/Go/Node.js, 与MySQL相似,零学习成本。 * __零运维成本、零学习成本__:安装集群简单快捷,无需分库分表,实时备份。类似标准SQL,支持RESTful, 支持Python/Java/C/C++/C#/Go/Node.js, 与MySQL相似,零学习成本。
采用TDengine,可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。但需要指出的是,因充分利用了物联网时序数据的特点,它无法用来处理网络爬虫、微博、微信、电商、ERP、CRM等通用型数据。 采用TDengine,可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。但需要指出的是,因充分利用了物联网时序数据的特点,它无法用来处理网络爬虫、微博、微信、电商、ERP、CRM等通用型数据。
......
...@@ -56,7 +56,7 @@ TDengine提供了丰富的应用程序开发接口,其中包括C/C++、Java、 ...@@ -56,7 +56,7 @@ TDengine提供了丰富的应用程序开发接口,其中包括C/C++、Java、
*taos.tar.gz*:应用驱动安装包 *taos.tar.gz*:应用驱动安装包
*driver*:TDengine应用驱动driver *driver*:TDengine应用驱动driver
*connector*: 各种编程语言连接器(go/grafanaplugin/nodejs/python/JDBC) *connector*: 各种编程语言连接器(go/grafanaplugin/nodejs/python/JDBC)
*examples*: 各种编程语言的示例程序(c/C#/go/JDBC/matlab/python/R) *examples*: 各种编程语言的示例程序(c/C#/go/JDBC/MATLAB/python/R)
运行install_client.sh进行安装 运行install_client.sh进行安装
......
...@@ -75,17 +75,17 @@ sudo cp -rf /usr/local/taos/connector/grafanaplugin /var/lib/grafana/plugins/tde ...@@ -75,17 +75,17 @@ sudo cp -rf /usr/local/taos/connector/grafanaplugin /var/lib/grafana/plugins/tde
![img](page://images/connections/import_dashboard2.jpg) ![img](page://images/connections/import_dashboard2.jpg)
## <a class="anchor" id="matlab"></a>Matlab ## <a class="anchor" id="matlab"></a>MATLAB
MatLab可以通过安装包内提供的JDBC Driver直接连接到TDengine获取数据到本地工作空间。 MATLAB可以通过安装包内提供的JDBC Driver直接连接到TDengine获取数据到本地工作空间。
### MatLab的JDBC接口适配 ### MATLAB的JDBC接口适配
MatLab的适配有下面几个步骤,下面以Windows10上适配MatLab2017a为例: MATLAB的适配有下面几个步骤,下面以Windows10上适配MATLAB2017a为例:
- 将TDengine安装包内的驱动程序JDBCDriver-1.0.0-dist.jar拷贝到${matlab_root}\MATLAB\R2017a\java\jar\toolbox - 将TDengine安装包内的驱动程序JDBCDriver-1.0.0-dist.jar拷贝到${matlab_root}\MATLAB\R2017a\java\jar\toolbox
- 将TDengine安装包内的taos.lib文件拷贝至${matlab_ root _dir}\MATLAB\R2017a\lib\win64 - 将TDengine安装包内的taos.lib文件拷贝至${matlab_ root _dir}\MATLAB\R2017a\lib\win64
- 将新添加的驱动jar包加入MatLab的classpath。在${matlab_ root _dir}\MATLAB\R2017a\toolbox\local\classpath.txt文件中添加下面一行 - 将新添加的驱动jar包加入MATLAB的classpath。在${matlab_ root _dir}\MATLAB\R2017a\toolbox\local\classpath.txt文件中添加下面一行
``` ```
$matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar $matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar
...@@ -96,9 +96,9 @@ $matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar ...@@ -96,9 +96,9 @@ $matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar
C:\Windows\System32 C:\Windows\System32
``` ```
### 在MatLab中连接TDengine获取数据 ### 在MATLAB中连接TDengine获取数据
在成功进行了上述配置后,打开MatLab 在成功进行了上述配置后,打开MATLAB
- 创建一个连接: - 创建一个连接:
......
...@@ -116,20 +116,22 @@ taosd -C ...@@ -116,20 +116,22 @@ taosd -C
**注意:**对于端口,TDengine会使用从serverPort起13个连续的TCP和UDP端口号,请务必在防火墙打开。因此如果是缺省配置,需要打开从6030到6042共13个端口,而且必须TCP和UDP都打开。(详细的端口情况请参见 [TDengine 2.0 端口说明](https://www.taosdata.com/cn/documentation/faq#port) **注意:**对于端口,TDengine会使用从serverPort起13个连续的TCP和UDP端口号,请务必在防火墙打开。因此如果是缺省配置,需要打开从6030到6042共13个端口,而且必须TCP和UDP都打开。(详细的端口情况请参见 [TDengine 2.0 端口说明](https://www.taosdata.com/cn/documentation/faq#port)
不同应用场景的数据往往具有不同的数据特征,比如保留天数、副本数、采集频次、记录大小、采集点的数量、压缩等都可完全不同。为获得在存储上的最高效率,TDengine提供如下存储相关的系统配置参数: 不同应用场景的数据往往具有不同的数据特征,比如保留天数、副本数、采集频次、记录大小、采集点的数量、压缩等都可完全不同。为获得在存储上的最高效率,TDengine提供如下存储相关的系统配置参数(既可以作为 create database 指令的参数,也可以写在 taos.cfg 配置文件中用来设定创建新数据库时所采用的默认值)
- days:一个数据文件存储数据的时间跨度单位为天,默认值:10。 - days:一个数据文件存储数据的时间跨度单位为天,默认值:10。
- keep:数据库中数据保留的天数单位为天,默认值:3650。(可通过 alter database 修改) - keep:数据库中数据保留的天数单位为天,默认值:3650。(可通过 alter database 修改)
- minRows:文件块中记录的最小条数单位为条,默认值:100。 - minRows:文件块中记录的最小条数单位为条,默认值:100。
- maxRows:文件块中记录的最大条数单位为条,默认值:4096。 - maxRows:文件块中记录的最大条数单位为条,默认值:4096。
- comp:文件压缩标志位0:关闭;1:一阶段压缩;2:两阶段压缩。默认值:2。(可通过 alter database 修改) - comp:文件压缩标志位0:关闭;1:一阶段压缩;2:两阶段压缩。默认值:2。(可通过 alter database 修改)
- walLevel:WAL级别。1:写wal,但不执行fsync;2:写wal, 而且执行fsync。默认值:1。 - wal:WAL级别。1:写wal,但不执行fsync;2:写wal, 而且执行fsync。默认值:1。(在 taos.cfg 中参数名需要写作 walLevel)(可通过 alter database 修改)
- fsync:当wal设置为2时,执行fsync的周期。设置为0,表示每次写入,立即执行fsync。单位为毫秒,默认值:3000。 - fsync:当wal设置为2时,执行fsync的周期。设置为0,表示每次写入,立即执行fsync。单位为毫秒,默认值:3000。(可通过 alter database 修改)
- cache:内存块的大小单位为兆字节(MB),默认值:16。 - cache:内存块的大小单位为兆字节(MB),默认值:16。
- blocks:每个VNODE(TSDB)中有多少cache大小的内存块。因此一个VNODE的用的内存大小粗略为(cache * blocks)。单位为块,默认值:4。(可通过 alter database 修改) - blocks:每个VNODE(TSDB)中有多少cache大小的内存块。因此一个VNODE的用的内存大小粗略为(cache * blocks)。单位为块,默认值:4。(可通过 alter database 修改)
- replica:副本个数,取值范围:1-3。单位为个,默认值:1。(可通过 alter database 修改) - replica:副本个数。取值范围:1-3,单位为个,默认值:1。(可通过 alter database 修改)
- precision:时间戳精度标识,ms表示毫秒,us表示微秒。默认值:ms。 - quorum:多副本环境下指令执行的确认数要求。取值范围:1、2,单位为个,默认值:1。(可通过 alter database 修改)
- cacheLast:是否在内存中缓存子表的最近数据,0:关闭;1:缓存子表最近一行数据;2:缓存子表每一列的最近的非NULL值,3:同时打开缓存最近行和列功能,默认值:0。(可通过 alter database 修改)(从 2.0.11 版本开始支持此参数) - precision:时间戳精度标识。ms表示毫秒,us表示微秒,默认值:ms。(2.1.2.0 版本之前、2.0.20.7 版本之前在 taos.cfg 文件中不支持此参数。)
- cacheLast:是否在内存中缓存子表的最近数据。0:关闭;1:缓存子表最近一行数据;2:缓存子表每一列的最近的非NULL值;3:同时打开缓存最近行和列功能。默认值:0。(可通过 alter database 修改)(从 2.1.2.0 版本开始此参数支持 0~3 的取值范围,在此之前取值只能是 [0, 1];而 2.0.11.0 之前的版本在 SQL 指令中不支持此参数。)(2.1.2.0 版本之前、2.0.20.7 版本之前在 taos.cfg 文件中不支持此参数。)
- update:是否允许更新。0:不允许;1:允许。默认值:0。(可通过 alter database 修改)
对于一个应用场景,可能有多种数据特征的数据并存,最佳的设计是将具有相同数据特征的表放在一个库里,这样一个应用有多个库,而每个库可以配置不同的存储参数,从而保证系统有最优的性能。TDengine允许应用在创建库时指定上述存储参数,如果指定,该参数就将覆盖对应的系统配置参数。举例,有下述SQL: 对于一个应用场景,可能有多种数据特征的数据并存,最佳的设计是将具有相同数据特征的表放在一个库里,这样一个应用有多个库,而每个库可以配置不同的存储参数,从而保证系统有最优的性能。TDengine允许应用在创建库时指定上述存储参数,如果指定,该参数就将覆盖对应的系统配置参数。举例,有下述SQL:
...@@ -142,7 +144,6 @@ taosd -C ...@@ -142,7 +144,6 @@ taosd -C
TDengine集群中加入一个新的dnode时,涉及集群相关的一些参数必须与已有集群的配置相同,否则不能成功加入到集群中。会进行校验的参数如下: TDengine集群中加入一个新的dnode时,涉及集群相关的一些参数必须与已有集群的配置相同,否则不能成功加入到集群中。会进行校验的参数如下:
- numOfMnodes:系统中管理节点个数。默认值:3。 - numOfMnodes:系统中管理节点个数。默认值:3。
- balance:是否启动负载均衡。0:否,1:是。默认值:1。
- mnodeEqualVnodeNum: 一个mnode等同于vnode消耗的个数。默认值:4。 - mnodeEqualVnodeNum: 一个mnode等同于vnode消耗的个数。默认值:4。
- offlineThreshold: dnode离线阈值,超过该时间将导致该dnode从集群中删除。单位为秒,默认值:86400*10(即10天)。 - offlineThreshold: dnode离线阈值,超过该时间将导致该dnode从集群中删除。单位为秒,默认值:86400*10(即10天)。
- statusInterval: dnode向mnode报告状态时长。单位为秒,默认值:1。 - statusInterval: dnode向mnode报告状态时长。单位为秒,默认值:1。
...@@ -150,6 +151,10 @@ TDengine集群中加入一个新的dnode时,涉及集群相关的一些参数 ...@@ -150,6 +151,10 @@ TDengine集群中加入一个新的dnode时,涉及集群相关的一些参数
- maxVgroupsPerDb: 每个数据库中能够使用的最大vgroup个数。 - maxVgroupsPerDb: 每个数据库中能够使用的最大vgroup个数。
- arbitrator: 系统中裁决器的end point,缺省为空。 - arbitrator: 系统中裁决器的end point,缺省为空。
- timezone、locale、charset 的配置见客户端配置。(2.0.20.0 及以上的版本里,集群中加入新节点已不要求 locale 和 charset 参数取值一致) - timezone、locale、charset 的配置见客户端配置。(2.0.20.0 及以上的版本里,集群中加入新节点已不要求 locale 和 charset 参数取值一致)
- balance:是否启用负载均衡。0:否,1:是。默认值:1。
- flowctrl:是否启用非阻塞流控。0:否,1:是。默认值:1。
- slaveQuery:是否启用 slave vnode 参与查询。0:否,1:是。默认值:1。
- adjustMaster:是否启用 vnode master 负载均衡。0:否,1:是。默认值:1。
为方便调试,可通过SQL语句临时调整每个dnode的日志配置,系统重启后会失效: 为方便调试,可通过SQL语句临时调整每个dnode的日志配置,系统重启后会失效:
......
...@@ -126,9 +126,25 @@ TDengine 缺省的时间戳是毫秒精度,但通过在 CREATE DATABASE 时传 ...@@ -126,9 +126,25 @@ TDengine 缺省的时间戳是毫秒精度,但通过在 CREATE DATABASE 时传
```mysql ```mysql
ALTER DATABASE db_name CACHELAST 0; ALTER DATABASE db_name CACHELAST 0;
``` ```
CACHELAST 参数控制是否在内存中缓存数据子表的 last_row。缺省值为 0,取值范围 [0, 1]。其中 0 表示不启用、1 表示启用。(从 2.0.11.0 版本开始支持。从 2.1.1.0 版本开始,修改此参数后无需重启服务器即可生效。) CACHELAST 参数控制是否在内存中缓存子表的最近数据。缺省值为 0,取值范围 [0, 1, 2, 3]。其中 0 表示不缓存,1 表示缓存子表最近一行数据,2 表示缓存子表每一列的最近的非 NULL 值,3 表示同时打开缓存最近行和列功能。(从 2.0.11.0 版本开始支持参数值 [0, 1],从 2.1.2.0 版本开始支持参数值 [0, 1, 2, 3]。)
说明:缓存最近行,将显著改善 LAST_ROW 函数的性能表现;缓存每列的最近非 NULL 值,将显著改善无特殊影响(WHERE、ORDER BY、GROUP BY、INTERVAL)下的 LAST 函数的性能表现。
**Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。 ```mysql
ALTER DATABASE db_name WAL 1;
```
WAL 参数控制 WAL 日志的落盘方式。缺省值为 1,取值范围为 [1, 2]。1 表示写 WAL,但不执行 fsync;2 表示写 WAL,而且执行 fsync。
```mysql
ALTER DATABASE db_name FSYNC 3000;
```
FSYNC 参数控制执行 fsync 操作的周期。缺省值为 3000,单位是毫秒,取值范围为 [0, 180000]。如果设置为 0,表示每次写入,立即执行 fsync。该设置项主要用于调节 WAL 参数设为 2 时的系统行为。
```mysql
ALTER DATABASE db_name UPDATE 0;
```
UPDATE 参数控制是否允许更新数据。缺省值为 0,取值范围为 [0, 1]。0 表示会直接丢弃后写入的相同时间戳的数据;1 表示会使用后写入的数据覆盖已有的相同时间戳的数据。
**Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。另外,从 2.1.1.0 版本开始,修改这些参数后无需重启服务器即可生效。
- **显示系统所有数据库** - **显示系统所有数据库**
......
...@@ -82,7 +82,7 @@ TDengine is a highly efficient platform to store, query, and analyze time-series ...@@ -82,7 +82,7 @@ TDengine is a highly efficient platform to store, query, and analyze time-series
## [Connections with Other Tools](/connections) ## [Connections with Other Tools](/connections)
- [Grafana](/connections#grafana): query the data saved in TDengine and provide visualization - [Grafana](/connections#grafana): query the data saved in TDengine and provide visualization
- [Matlab](/connections#matlab): access data stored in TDengine server via JDBC configured within Matlab - [MATLAB](/connections#matlab): access data stored in TDengine server via JDBC configured within MATLAB
- [R](/connections#r): access data stored in TDengine server via JDBC configured within R - [R](/connections#r): access data stored in TDengine server via JDBC configured within R
- [IDEA Database](https://www.taosdata.com/blog/2020/08/27/1767.html): use TDengine visually through IDEA Database Management Tool - [IDEA Database](https://www.taosdata.com/blog/2020/08/27/1767.html): use TDengine visually through IDEA Database Management Tool
......
...@@ -9,8 +9,8 @@ One of the modules of TDengine is the time-series database. However, in addition ...@@ -9,8 +9,8 @@ One of the modules of TDengine is the time-series database. However, in addition
- **Performance improvement over 10 times**: An innovative data storage structure is defined, with each single core can process at least 20,000 requests per second, insert millions of data points, and read more than 10 million data points, which is more than 10 times faster than other existing general database. - **Performance improvement over 10 times**: An innovative data storage structure is defined, with each single core can process at least 20,000 requests per second, insert millions of data points, and read more than 10 million data points, which is more than 10 times faster than other existing general database.
- **Reduce the cost of hardware or cloud services to 1/5**: Due to its ultra-performance, TDengine’s computing resources consumption is less than 1/5 of other common Big Data solutions; through columnar storage and advanced compression algorithms, the storage consumption is less than 1/10 of other general databases. - **Reduce the cost of hardware or cloud services to 1/5**: Due to its ultra-performance, TDengine’s computing resources consumption is less than 1/5 of other common Big Data solutions; through columnar storage and advanced compression algorithms, the storage consumption is less than 1/10 of other general databases.
- **Full-stack time-series data processing engine**: Integrate database, message queue, cache, stream computing, and other functions, and the applications do not need to integrate with software such as Kafka/Redis/HBase/Spark/HDFS, thus greatly reducing the complexity cost of application development and maintenance. - **Full-stack time-series data processing engine**: Integrate database, message queue, cache, stream computing, and other functions, and the applications do not need to integrate with software such as Kafka/Redis/HBase/Spark/HDFS, thus greatly reducing the complexity cost of application development and maintenance.
- **Powerful analysis functions**: Data from ten years ago or one second ago, can all be queried based on a specified time range. Data can be aggregated on a timeline or multiple devices. Ad-hoc queries can be made at any time through Shell, Python, R, and Matlab. - **Powerful analysis functions**: Data from ten years ago or one second ago, can all be queried based on a specified time range. Data can be aggregated on a timeline or multiple devices. Ad-hoc queries can be made at any time through Shell, Python, R, and MATLAB.
- **Seamless connection with third-party tools**: Integration with Telegraf, Grafana, EMQ, HiveMQ, Prometheus, Matlab, R, etc. without even one single line of code. OPC, Hadoop, Spark, etc. will be supported in the future, and more BI tools will be seamlessly connected to. - **Seamless connection with third-party tools**: Integration with Telegraf, Grafana, EMQ, HiveMQ, Prometheus, MATLAB, R, etc. without even one single line of code. OPC, Hadoop, Spark, etc. will be supported in the future, and more BI tools will be seamlessly connected to.
- **Zero operation cost & zero learning cost**: Installing clusters is simple and quick, with real-time backup built-in, and no need to split libraries or tables. Similar to standard SQL, TDengine can support RESTful, Python/Java/C/C + +/C#/Go/Node.js, and similar to MySQL with zero learning cost. - **Zero operation cost & zero learning cost**: Installing clusters is simple and quick, with real-time backup built-in, and no need to split libraries or tables. Similar to standard SQL, TDengine can support RESTful, Python/Java/C/C + +/C#/Go/Node.js, and similar to MySQL with zero learning cost.
With TDengine, the total cost of ownership of typical IoT, Internet of Vehicles, and Industrial Internet Big Data platforms can be greatly reduced. However, it should be pointed out that due to making full use of the characteristics of IoT time-series data, TDengine cannot be used to process general data from web crawlers, microblogs, WeChat, e-commerce, ERP, CRM, and other sources. With TDengine, the total cost of ownership of typical IoT, Internet of Vehicles, and Industrial Internet Big Data platforms can be greatly reduced. However, it should be pointed out that due to making full use of the characteristics of IoT time-series data, TDengine cannot be used to process general data from web crawlers, microblogs, WeChat, e-commerce, ERP, CRM, and other sources.
......
...@@ -58,7 +58,7 @@ After extracting the package, you will see the following files (directories) in ...@@ -58,7 +58,7 @@ After extracting the package, you will see the following files (directories) in
*connector*: Connectors for various programming languages (go/grafanaplugin/nodejs/python/JDBC) *connector*: Connectors for various programming languages (go/grafanaplugin/nodejs/python/JDBC)
*Examples*: Sample programs for various programming languages (C/C #/go/JDBC/matlab/python/R) *Examples*: Sample programs for various programming languages (C/C #/go/JDBC/MATLAB/python/R)
Run install_client.sh to install. Run install_client.sh to install.
......
...@@ -74,17 +74,17 @@ You can see as follows after Dashboard imported. ...@@ -74,17 +74,17 @@ You can see as follows after Dashboard imported.
![img](page://images/connections/import_dashboard2.jpg) ![img](page://images/connections/import_dashboard2.jpg)
## <a class="anchor" id="matlab"></a> Matlab ## <a class="anchor" id="matlab"></a> MATLAB
MatLab can access data to the local workspace by connecting directly to the TDengine via the JDBC Driver provided in the installation package. MATLAB can access data to the local workspace by connecting directly to the TDengine via the JDBC Driver provided in the installation package.
### JDBC Interface Adaptation of MatLab ### JDBC Interface Adaptation of MATLAB
Several steps are required to adapt Matlab to TDengine. Taking adapting Matlab2017a on Windows10 as an example: Several steps are required to adapt MATLAB to TDengine. Taking adapting MATLAB2017a on Windows10 as an example:
- Copy the file JDBCDriver-1.0.0-dist.ja*r* in TDengine package to the directory ${matlab_root}\MATLAB\R2017a\java\jar\toolbox - Copy the file JDBCDriver-1.0.0-dist.ja*r* in TDengine package to the directory ${matlab_root}\MATLAB\R2017a\java\jar\toolbox
- Copy the file taos.lib in TDengine package to ${matlab root dir}\MATLAB\R2017a\lib\win64 - Copy the file taos.lib in TDengine package to ${matlab root dir}\MATLAB\R2017a\lib\win64
- Add the .jar package just copied to the Matlab classpath. Append the line below as the end of the file of ${matlab root dir}\MATLAB\R2017a\toolbox\local\classpath.txt - Add the .jar package just copied to the MATLAB classpath. Append the line below as the end of the file of ${matlab root dir}\MATLAB\R2017a\toolbox\local\classpath.txt
- ``` - ```
$matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar $matlabroot/java/jar/toolbox/JDBCDriver-1.0.0-dist.jar
``` ```
...@@ -94,9 +94,9 @@ Several steps are required to adapt Matlab to TDengine. Taking adapting Matlab20 ...@@ -94,9 +94,9 @@ Several steps are required to adapt Matlab to TDengine. Taking adapting Matlab20
C:\Windows\System32 C:\Windows\System32
``` ```
- ### Connect to TDengine in MatLab to get data - ### Connect to TDengine in MATLAB to get data
After the above configured successfully, open MatLab. After the above configured successfully, open MATLAB.
- Create a connection: - Create a connection:
......
#!/bin/bash
#
# This file is used to install database on linux systems. The operating system
# is required to use systemd to manage services at boot
set -e
#set -x
verMode=edge
pagMode=full
iplist=""
serverFqdn=""
# -----------------------Variables definition---------------------
script_dir="../release"
# Dynamic directory
data_dir="/var/lib/taos"
log_dir="/var/log/taos"
data_link_dir="/usr/local/taos/data"
log_link_dir="/usr/local/taos/log"
cfg_install_dir="/etc/taos"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
lib64_link_dir="/usr/lib64"
inc_link_dir="/usr/include"
#install main path
install_main_dir="/usr/local/taos"
# old bin dir
sbin_dir="/usr/local/taos/bin"
temp_version=""
fin_result=""
service_config_dir="/etc/systemd/system"
nginx_port=6060
nginx_dir="/usr/local/nginxd"
# Color setting
RED='\033[0;31m'
GREEN='\033[1;32m'
GREEN_DARK='\033[0;32m'
GREEN_UNDERLINE='\033[4;32m'
NC='\033[0m'
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
# ============================= get input parameters =================================================
# install.sh -v [server | client] -e [yes | no] -i [systemd | service | ...]
# set parameters by default value
interactiveFqdn=yes # [yes | no]
verType=server # [server | client]
initType=systemd # [systemd | service | ...]
while getopts "hv:d:" arg
do
case $arg in
d)
#echo "interactiveFqdn=$OPTARG"
script_dir=$( echo $OPTARG )
;;
h)
echo "Usage: `basename $0` -d scripy_path"
exit 0
;;
?) #unknow option
echo "unkonw argument"
exit 1
;;
esac
done
#echo "verType=${verType} interactiveFqdn=${interactiveFqdn}"
function kill_process() {
pid=$(ps -ef | grep "$1" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function check_file() {
#check file whether exists
if [ ! -e $1/$2 ];then
echo -e "$1/$2 \033[31mnot exists\033[0m!quit"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
}
function get_package_name() {
var=$1
if [[ $1 =~ 'aarch' ]];then
echo ${var::-21}
else
echo ${var::-17}
fi
}
function check_link() {
#check Link whether exists or broken
if [ -L $1 ] ; then
if [ ! -e $1 ] ; then
echo -e "$1 \033[31Broken link\033[0m"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
else
echo -e "$1 \033[31mnot exists\033[0m!quit"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
}
function check_main_path() {
#check install main dir and all sub dir
main_dir=("" "cfg" "bin" "connector" "driver" "examples" "include" "init.d")
for i in ${main_dir[@]};do
check_file ${install_main_dir} $i
done
if [ "$verMode" == "cluster" ]; then
nginx_main_dir=("admin" "conf" "html" "sbin" "logs")
for i in ${nginx_main_dir[@]};do
check_file ${nginx_dir} $i
done
fi
echo -e "Check main path:\033[32mOK\033[0m!"
}
function check_bin_path() {
# check install bin dir and all sub dir
bin_dir=("taos" "taosd" "taosdemo" "taosdump" "remove.sh" "tarbitrator" "set_core.sh")
for i in ${bin_dir[@]};do
check_file ${sbin_dir} $i
done
lbin_dir=("taos" "taosd" "taosdemo" "taosdump" "rmtaos" "tarbitrator" "set_core")
for i in ${lbin_dir[@]};do
check_link ${bin_link_dir}/$i
done
if [ "$verMode" == "cluster" ]; then
check_file ${nginx_dir}/sbin nginx
fi
echo -e "Check bin path:\033[32mOK\033[0m!"
}
function check_lib_path() {
# check all links
check_link ${lib_link_dir}/libtaos.so
check_link ${lib_link_dir}/libtaos.so.1
if [[ -d ${lib64_link_dir} ]]; then
check_link ${lib64_link_dir}/libtaos.so
check_link ${lib64_link_dir}/libtaos.so.1
fi
echo -e "Check lib path:\033[32mOK\033[0m!"
}
function check_header_path() {
# check all header
header_dir=("taos.h" "taoserror.h")
for i in ${header_dir[@]};do
check_link ${inc_link_dir}/$i
done
echo -e "Check bin path:\033[32mOK\033[0m!"
}
function check_config_dir() {
# check all config
check_file ${cfg_install_dir} taos.cfg
check_file ${install_main_dir}/cfg taos.cfg.org
echo -e "Check conf path:\033[32mOK\033[0m!"
}
function check_log_path() {
# check log path
check_file ${log_dir}
echo -e "Check log path:\033[32mOK\033[0m!"
}
function check_data_path() {
# check data path
check_file ${data_dir}
echo -e "Check data path:\033[32mOK\033[0m!"
}
function install_TDengine() {
cd ${script_dir}
tar zxf $1
temp_version=$(get_package_name $1)
cd $(get_package_name $1)
echo -e "\033[32muninstall TDengine && install TDengine...\033[0m"
rmtaos >/dev/null 2>&1 || echo 'taosd not installed' && echo -e '\n\n' |./install.sh >/dev/null 2>&1
echo -e "\033[32mTDengine has been installed!\033[0m"
echo -e "\033[32mTDengine is starting...\033[0m"
kill_process taos && systemctl start taosd && sleep 10
}
function test_TDengine() {
check_main_path
check_bin_path
check_lib_path
check_header_path
check_config_dir
check_log_path
check_data_path
result=`taos -s 'create database test ;create table test.tt(ts timestamp ,i int);insert into test.tt values(now,11);select * from test.tt' 2>&1 ||:`
if [[ $result =~ "Unable to establish" ]];then
echo -e "\033[31mTDengine connect failed\033[0m"
fin_result=$fin_result"\033[31m$temp_version\033[0m test failed!\n"
echo -e $fin_result
exit 8
fi
echo -e "Check TDengine connect:\033[32mOK\033[0m!"
fin_result=$fin_result"\033[32m$temp_version\033[0m test OK!\n"
}
# ## ==============================Main program starts from here============================
TD_package_name=`ls ${script_dir}/*server*gz |awk -F '/' '{print $NF}' `
temp=`pwd`
for i in $TD_package_name;do
if [[ $i =~ 'enterprise' ]];then
verMode="cluster"
else
verMode=""
fi
cd $temp
install_TDengine $i
test_TDengine
done
echo "============================================================"
echo -e $fin_result
\ No newline at end of file
...@@ -367,6 +367,7 @@ static bool bnMonitorBalance() { ...@@ -367,6 +367,7 @@ static bool bnMonitorBalance() {
for (int32_t dest = 0; dest < src; dest++) { for (int32_t dest = 0; dest < src; dest++) {
SDnodeObj *pDestDnode = tsBnDnodes.list[dest]; SDnodeObj *pDestDnode = tsBnDnodes.list[dest];
if (bnCheckDnodeInVgroup(pDestDnode, pVgroup)) continue; if (bnCheckDnodeInVgroup(pDestDnode, pVgroup)) continue;
if (taosGetTimestampMs() - pDestDnode->createdTime < 2000) continue;
float destScore = bnTryCalcDnodeScore(pDestDnode, 1); float destScore = bnTryCalcDnodeScore(pDestDnode, 1);
if (srcScore + 0.0001 < destScore) continue; if (srcScore + 0.0001 < destScore) continue;
......
...@@ -123,7 +123,8 @@ int32_t tscGetDataBlockFromList(SHashObj* pHashList, int64_t id, int32_t size, i ...@@ -123,7 +123,8 @@ int32_t tscGetDataBlockFromList(SHashObj* pHashList, int64_t id, int32_t size, i
*/ */
bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo); bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo);
bool tscIsTWAQuery(SQueryInfo* pQueryInfo); bool tscIsTWAQuery(SQueryInfo* pQueryInfo);
bool tscIsDiffQuery(SQueryInfo* pQueryInfo); bool tscIsIrateQuery(SQueryInfo* pQueryInfo);
bool tscIsSessionWindowQuery(SQueryInfo* pQueryInfo); bool tscIsSessionWindowQuery(SQueryInfo* pQueryInfo);
bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo); bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo);
bool tsIsArithmeticQueryOnAggResult(SQueryInfo* pQueryInfo); bool tsIsArithmeticQueryOnAggResult(SQueryInfo* pQueryInfo);
......
...@@ -51,10 +51,10 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getTsCharset ...@@ -51,10 +51,10 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getTsCharset
/* /*
* Class: com_taosdata_jdbc_TSDBJNIConnector * Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: getResultTimePrecision * Method: getResultTimePrecisionImp
* Signature: (J)J * Signature: (JJ)I
*/ */
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TDDBJNIConnector_getResultTimePrecision JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TDDBJNIConnector_getResultTimePrecisionImp
(JNIEnv *, jobject, jlong, jlong); (JNIEnv *, jobject, jlong, jlong);
/* /*
......
...@@ -113,7 +113,7 @@ static void jniGetGlobalMethod(JNIEnv *env) { ...@@ -113,7 +113,7 @@ static void jniGetGlobalMethod(JNIEnv *env) {
g_rowdataSetFloatFp = (*env)->GetMethodID(env, g_rowdataClass, "setFloat", "(IF)V"); g_rowdataSetFloatFp = (*env)->GetMethodID(env, g_rowdataClass, "setFloat", "(IF)V");
g_rowdataSetDoubleFp = (*env)->GetMethodID(env, g_rowdataClass, "setDouble", "(ID)V"); g_rowdataSetDoubleFp = (*env)->GetMethodID(env, g_rowdataClass, "setDouble", "(ID)V");
g_rowdataSetStringFp = (*env)->GetMethodID(env, g_rowdataClass, "setString", "(ILjava/lang/String;)V"); g_rowdataSetStringFp = (*env)->GetMethodID(env, g_rowdataClass, "setString", "(ILjava/lang/String;)V");
g_rowdataSetTimestampFp = (*env)->GetMethodID(env, g_rowdataClass, "setTimestamp", "(IJ)V"); g_rowdataSetTimestampFp = (*env)->GetMethodID(env, g_rowdataClass, "setTimestamp", "(IJI)V");
g_rowdataSetByteArrayFp = (*env)->GetMethodID(env, g_rowdataClass, "setByteArray", "(I[B)V"); g_rowdataSetByteArrayFp = (*env)->GetMethodID(env, g_rowdataClass, "setByteArray", "(I[B)V");
(*env)->DeleteLocalRef(env, rowdataClass); (*env)->DeleteLocalRef(env, rowdataClass);
...@@ -519,9 +519,11 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn ...@@ -519,9 +519,11 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn
jniFromNCharToByteArray(env, (char *)row[i], length[i])); jniFromNCharToByteArray(env, (char *)row[i], length[i]));
break; break;
} }
case TSDB_DATA_TYPE_TIMESTAMP: case TSDB_DATA_TYPE_TIMESTAMP: {
(*env)->CallVoidMethod(env, rowobj, g_rowdataSetTimestampFp, i, (jlong) * ((int64_t *)row[i])); int precision = taos_result_precision(result);
(*env)->CallVoidMethod(env, rowobj, g_rowdataSetTimestampFp, i, (jlong) * ((int64_t *)row[i]), precision);
break; break;
}
default: default:
break; break;
} }
...@@ -672,7 +674,15 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getTsCharset(J ...@@ -672,7 +674,15 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getTsCharset(J
return (*env)->NewStringUTF(env, (const char *)tsCharset); return (*env)->NewStringUTF(env, (const char *)tsCharset);
} }
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TDDBJNIConnector_getResultTimePrecision(JNIEnv *env, jobject jobj, jlong con, /**
* Get Result Time Precision
* @param env vm
* @param jobj the TSDBJNIConnector java object
* @param con the c connection pointer
* @param res the TAOS_RES object, i.e. the SSqlObject
* @return precision 0:ms 1:us 2:ns
*/
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultTimePrecisionImp(JNIEnv *env, jobject jobj, jlong con,
jlong res) { jlong res) {
TAOS *tscon = (TAOS *)con; TAOS *tscon = (TAOS *)con;
if (tscon == NULL) { if (tscon == NULL) {
......
...@@ -906,7 +906,7 @@ static int insertStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) { ...@@ -906,7 +906,7 @@ static int insertStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) {
int code = doBindParam(pBlock, data, param, &bind[param->idx], 1); int code = doBindParam(pBlock, data, param, &bind[param->idx], 1);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
tscDebug("0x%"PRIx64" bind column %d: type mismatch or invalid", pStmt->pSql->self, param->idx); tscDebug("0x%"PRIx64" bind column %d: type mismatch or invalid", pStmt->pSql->self, param->idx);
return invalidOperationMsg(tscGetErrorMsgPayload(&stmt->pSql->cmd), "bind column type mismatch or invalid"); return invalidOperationMsg(tscGetErrorMsgPayload(&stmt->pSql->cmd), "bind column type mismatch or invalid");
} }
} }
...@@ -1256,7 +1256,7 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { ...@@ -1256,7 +1256,7 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
if (sToken.n <= 0 || sToken.type != TK_USING) { if (sToken.n <= 0 || sToken.type != TK_USING) {
tscError("keywords USING is expected, sql:%s", pCmd->insertParam.sql); tscError("keywords USING is expected, sql:%s", pCmd->insertParam.sql);
return tscSQLSyntaxErrMsg(pCmd->payload, "keywords USING is expected", sToken.z ? sToken.z : pCmd->insertParam.sql); return tscSQLSyntaxErrMsg(pCmd->payload, "keywords USING is expected", sToken.z ? sToken.z : pCmd->insertParam.sql);
} }
...@@ -1403,7 +1403,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO ...@@ -1403,7 +1403,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO
} else { } else {
tfree(pSql->sqlstr); tfree(pSql->sqlstr);
} }
pSql->sqlstr = str; pSql->sqlstr = str;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
...@@ -1415,7 +1415,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO ...@@ -1415,7 +1415,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO
TAOS_STMT* taos_stmt_init(TAOS* taos) { TAOS_STMT* taos_stmt_init(TAOS* taos) {
STscObj* pObj = (STscObj*)taos; STscObj* pObj = (STscObj*)taos;
STscStmt* pStmt = NULL; STscStmt* pStmt = NULL;
if (pObj == NULL || pObj->signature != pObj) { if (pObj == NULL || pObj->signature != pObj) {
terrno = TSDB_CODE_TSC_DISCONNECTED; terrno = TSDB_CODE_TSC_DISCONNECTED;
tscError("connection disconnected"); tscError("connection disconnected");
...@@ -1469,7 +1469,7 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) { ...@@ -1469,7 +1469,7 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) {
tscError("sql is NULL"); tscError("sql is NULL");
STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "sql is NULL")); STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "sql is NULL"));
} }
if (pStmt->last != STMT_INIT) { if (pStmt->last != STMT_INIT) {
tscError("prepare status error, last:%d", pStmt->last); tscError("prepare status error, last:%d", pStmt->last);
STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "prepare status error")); STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "prepare status error"));
......
...@@ -148,16 +148,16 @@ int16_t getNewResColId(SSqlCmd* pCmd) { ...@@ -148,16 +148,16 @@ int16_t getNewResColId(SSqlCmd* pCmd) {
return pCmd->resColumnId--; return pCmd->resColumnId--;
} }
// serialize expr in exprlist to binary // serialize expr in exprlist to binary
// formate "type | size | value" // formate "type | size | value"
bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, uint8_t precision) { bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, uint8_t precision) {
bool ret = false; bool ret = false;
if (!pList || pList->size <= 0 || colType < 0) { if (!pList || pList->size <= 0 || colType < 0) {
return ret; return ret;
} }
tSqlExprItem* item = (tSqlExprItem *)taosArrayGet(pList, 0); tSqlExprItem* item = (tSqlExprItem *)taosArrayGet(pList, 0);
int32_t firstTokenType = item->pNode->token.type; int32_t firstTokenType = item->pNode->token.type;
int32_t type = firstTokenType; int32_t type = firstTokenType;
//nchar to binary and other xxint to bigint //nchar to binary and other xxint to bigint
...@@ -170,12 +170,13 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, ...@@ -170,12 +170,13 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType,
type = colType; type = colType;
SBufferWriter bw = tbufInitWriter( NULL, false); SBufferWriter bw = tbufInitWriter( NULL, false);
tbufEnsureCapacity(&bw, 512); tbufEnsureCapacity(&bw, 512);
int32_t size = (int32_t)(pList->size); int32_t size = (int32_t)(pList->size);
tbufWriteUint32(&bw, type); tbufWriteUint32(&bw, type);
tbufWriteInt32(&bw, size); tbufWriteInt32(&bw, size);
for (int32_t i = 0; i < size; i++) { for (int32_t i = 0; i < size; i++) {
tSqlExpr* pSub = ((tSqlExprItem*)(taosArrayGet(pList, i)))->pNode; tSqlExpr* pSub = ((tSqlExprItem*)(taosArrayGet(pList, i)))->pNode;
...@@ -203,11 +204,11 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, ...@@ -203,11 +204,11 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType,
} else if (type == TSDB_DATA_TYPE_BINARY){ } else if (type == TSDB_DATA_TYPE_BINARY){
tbufWriteBinary(&bw, var.pz, var.nLen); tbufWriteBinary(&bw, var.pz, var.nLen);
} else if (type == TSDB_DATA_TYPE_NCHAR) { } else if (type == TSDB_DATA_TYPE_NCHAR) {
char *buf = (char *)calloc(1, (var.nLen + 1)*TSDB_NCHAR_SIZE); char *buf = (char *)calloc(1, (var.nLen + 1)*TSDB_NCHAR_SIZE);
if (tVariantDump(&var, buf, type, false) != TSDB_CODE_SUCCESS) { if (tVariantDump(&var, buf, type, false) != TSDB_CODE_SUCCESS) {
free(buf); free(buf);
tVariantDestroy(&var); tVariantDestroy(&var);
break; break;
} }
tbufWriteBinary(&bw, buf, twcslen((wchar_t *)buf) * TSDB_NCHAR_SIZE); tbufWriteBinary(&bw, buf, twcslen((wchar_t *)buf) * TSDB_NCHAR_SIZE);
free(buf); free(buf);
...@@ -229,21 +230,21 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, ...@@ -229,21 +230,21 @@ bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType,
if (ret == true) { if (ret == true) {
if ((*dst = calloc(1, sizeof(tVariant))) != NULL) { if ((*dst = calloc(1, sizeof(tVariant))) != NULL) {
tVariantCreateFromBinary(*dst, tbufGetData(&bw, false), tbufTell(&bw), TSDB_DATA_TYPE_BINARY); tVariantCreateFromBinary(*dst, tbufGetData(&bw, false), tbufTell(&bw), TSDB_DATA_TYPE_BINARY);
} else { } else {
ret = false; ret = false;
} }
} }
tbufCloseWriter(&bw); tbufCloseWriter(&bw);
return ret; return ret;
} }
static int32_t validateParamOfRelationIn(tVariant *pVar, int32_t colType) { static int32_t validateParamOfRelationIn(tVariant *pVar, int32_t colType) {
if (pVar->nType != TSDB_DATA_TYPE_BINARY) { if (pVar->nType != TSDB_DATA_TYPE_BINARY) {
return -1; return -1;
} }
SBufferReader br = tbufInitReader(pVar->pz, pVar->nLen, false); SBufferReader br = tbufInitReader(pVar->pz, pVar->nLen, false);
return colType == TSDB_DATA_TYPE_NCHAR ? 0 : (tbufReadUint32(&br) == colType ? 0: -1); return colType == TSDB_DATA_TYPE_NCHAR ? 0 : (tbufReadUint32(&br) == colType ? 0: -1);
} }
static uint8_t convertOptr(SStrToken *pToken) { static uint8_t convertOptr(SStrToken *pToken) {
...@@ -447,18 +448,6 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -447,18 +448,6 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if(code != TSDB_CODE_SUCCESS) { if(code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
if (pInfo->pMiscInfo->tableType == TSDB_SUPER_TABLE) {
//// code = tscGetTableMeta(pSql, pTableMetaInfo);
//// if (code != TSDB_CODE_SUCCESS) {
//// return code;
//// }
//
// if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
// return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
// }
}
} else if (pInfo->type == TSDB_SQL_DROP_DNODE) { } else if (pInfo->type == TSDB_SQL_DROP_DNODE) {
if (pzName->type == TK_STRING) { if (pzName->type == TK_STRING) {
pzName->n = strdequote(pzName->z); pzName->n = strdequote(pzName->z);
...@@ -531,7 +520,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -531,7 +520,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
break; break;
} }
case TSDB_SQL_CREATE_DNODE: { case TSDB_SQL_CREATE_DNODE: {
const char* msg = "invalid host name (ip address)"; const char* msg = "invalid host name (ip address)";
...@@ -796,6 +785,11 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -796,6 +785,11 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
pCmd->active = pCmd->pQueryInfo; pCmd->active = pCmd->pQueryInfo;
pCmd->command = pCmd->pQueryInfo->command; pCmd->command = pCmd->pQueryInfo->command;
STableMetaInfo* pTableMetaInfo1 = tscGetMetaInfo(pCmd->active, 0);
if (pTableMetaInfo1->pTableMeta != NULL) {
pSql->res.precision = tscGetTableInfo(pTableMetaInfo1->pTableMeta).precision;
}
return TSDB_CODE_SUCCESS; // do not build query message here return TSDB_CODE_SUCCESS; // do not build query message here
} }
...@@ -827,12 +821,12 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -827,12 +821,12 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
} }
break; break;
} }
case TSDB_SQL_COMPACT_VNODE:{ case TSDB_SQL_COMPACT_VNODE:{
const char* msg = "invalid compact"; const char* msg = "invalid compact";
if (setCompactVnodeInfo(pSql, pInfo) != TSDB_CODE_SUCCESS) { if (setCompactVnodeInfo(pSql, pInfo) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
} }
break; break;
} }
default: default:
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression"); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression");
...@@ -980,20 +974,22 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS ...@@ -980,20 +974,22 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS
static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, bool isStable) { static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, bool isStable) {
const char* msg1 = "invalid column name"; const char* msg1 = "invalid column name";
const char* msg2 = "invalid column type";
const char* msg3 = "not support state_window with group by "; const char* msg3 = "not support state_window with group by ";
const char* msg4 = "function not support for super table query"; const char* msg4 = "function not support for super table query";
const char* msg5 = "not support state_window on tag column";
SStrToken *col = &(pSqlNode->windowstateVal.col) ; SStrToken *col = &(pSqlNode->windowstateVal.col) ;
if (col->z == NULL || col->n <= 0) { if (col->z == NULL || col->n <= 0) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
if (pQueryInfo->colList == NULL) { if (pQueryInfo->colList == NULL) {
pQueryInfo->colList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->colList = taosArrayInit(4, POINTER_BYTES);
} }
if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) { if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
pQueryInfo->groupbyExpr.numOfGroupCols = 1; pQueryInfo->groupbyExpr.numOfGroupCols = 1;
//TODO(dengyihao): check tag column //TODO(dengyihao): check tag column
...@@ -1004,23 +1000,27 @@ static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS ...@@ -1004,23 +1000,27 @@ static int32_t validateStateWindowNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
SColumnIndex index = COLUMN_INDEX_INITIALIZER; SColumnIndex index = COLUMN_INDEX_INITIALIZER;
if (getColumnIndexByName(pCmd, col, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(pCmd, col, pQueryInfo, &index) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
} }
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; STableMeta* pTableMeta = pTableMetaInfo->pTableMeta;
int32_t numOfCols = tscGetNumOfColumns(pTableMeta); int32_t numOfCols = tscGetNumOfColumns(pTableMeta);
if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX || index.columnIndex >= numOfCols) { if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} else if (index.columnIndex >= numOfCols) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
} }
SGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr; SGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr;
if (pGroupExpr->columnInfo == NULL) { if (pGroupExpr->columnInfo == NULL) {
pGroupExpr->columnInfo = taosArrayInit(4, sizeof(SColIndex)); pGroupExpr->columnInfo = taosArrayInit(4, sizeof(SColIndex));
} }
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, index.columnIndex); SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, index.columnIndex);
if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP || pSchema->type == TSDB_DATA_TYPE_FLOAT || pSchema->type == TSDB_DATA_TYPE_DOUBLE) { if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP || pSchema->type == TSDB_DATA_TYPE_FLOAT
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); || pSchema->type == TSDB_DATA_TYPE_DOUBLE || pSchema->type == TSDB_DATA_TYPE_NCHAR
|| pSchema->type == TSDB_DATA_TYPE_BINARY) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
} }
tscColumnListInsert(pQueryInfo->colList, index.columnIndex, pTableMeta->id.uid, pSchema); tscColumnListInsert(pQueryInfo->colList, index.columnIndex, pTableMeta->id.uid, pSchema);
...@@ -1069,7 +1069,7 @@ int32_t validateSessionNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode * pS ...@@ -1069,7 +1069,7 @@ int32_t validateSessionNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode * pS
} }
if (index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX) { if (index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
pQueryInfo->sessionWindow.primaryColId = PRIMARYKEY_TIMESTAMP_COL_INDEX; pQueryInfo->sessionWindow.primaryColId = PRIMARYKEY_TIMESTAMP_COL_INDEX;
...@@ -1819,7 +1819,6 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS ...@@ -1819,7 +1819,6 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS
assert(pSelNodeList != NULL && pCmd != NULL); assert(pSelNodeList != NULL && pCmd != NULL);
const char* msg1 = "too many items in selection clause"; const char* msg1 = "too many items in selection clause";
const char* msg2 = "functions or others can not be mixed up"; const char* msg2 = "functions or others can not be mixed up";
const char* msg3 = "not support query expression"; const char* msg3 = "not support query expression";
const char* msg4 = "only support distinct one tag"; const char* msg4 = "only support distinct one tag";
...@@ -2047,8 +2046,10 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t ...@@ -2047,8 +2046,10 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t
if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) {
SSchema colSchema = *tGetTbnameColumnSchema(); SSchema colSchema = *tGetTbnameColumnSchema();
getColumnName(pItem, colSchema.name, colSchema.name, sizeof(colSchema.name) - 1); char name[TSDB_COL_NAME_LEN] = {0};
getColumnName(pItem, name, colSchema.name, sizeof(colSchema.name) - 1);
tstrncpy(colSchema.name, name, TSDB_COL_NAME_LEN);
/*SExprInfo* pExpr = */tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, &colSchema, TSDB_COL_TAG, getNewResColId(pCmd)); /*SExprInfo* pExpr = */tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, &colSchema, TSDB_COL_TAG, getNewResColId(pCmd));
} else { } else {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
...@@ -2362,7 +2363,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col ...@@ -2362,7 +2363,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
tickPerSec /= 1000000; tickPerSec /= 1000000;
} else if (info.precision == TSDB_TIME_PRECISION_MICRO) { } else if (info.precision == TSDB_TIME_PRECISION_MICRO) {
tickPerSec /= 1000; tickPerSec /= 1000;
} }
if (tickPerSec <= 0 || tickPerSec < TSDB_TICK_PER_SECOND(info.precision)) { if (tickPerSec <= 0 || tickPerSec < TSDB_TICK_PER_SECOND(info.precision)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg10); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg10);
...@@ -2748,6 +2749,8 @@ void getColumnName(tSqlExprItem* pItem, char* resultFieldName, char* rawName, in ...@@ -2748,6 +2749,8 @@ void getColumnName(tSqlExprItem* pItem, char* resultFieldName, char* rawName, in
strncpy(rawName, pItem->pNode->token.z, len); strncpy(rawName, pItem->pNode->token.z, len);
if (pItem->aliasName != NULL) { if (pItem->aliasName != NULL) {
int32_t aliasNameLen = (int32_t) strlen(pItem->aliasName);
len = (aliasNameLen < nameLength)? aliasNameLen:nameLength;
strncpy(resultFieldName, pItem->aliasName, len); strncpy(resultFieldName, pItem->aliasName, len);
} else { } else {
strncpy(resultFieldName, rawName, len); strncpy(resultFieldName, rawName, len);
...@@ -2997,7 +3000,7 @@ int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killType) { ...@@ -2997,7 +3000,7 @@ int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killType) {
static int32_t setCompactVnodeInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { static int32_t setCompactVnodeInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SSqlCmd* pCmd = &pSql->cmd; SSqlCmd* pCmd = &pSql->cmd;
pCmd->command = pInfo->type; pCmd->command = pInfo->type;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
bool validateIpAddress(const char* ip, size_t size) { bool validateIpAddress(const char* ip, size_t size) {
...@@ -3085,8 +3088,8 @@ void tscRestoreFuncForSTableQuery(SQueryInfo* pQueryInfo) { ...@@ -3085,8 +3088,8 @@ void tscRestoreFuncForSTableQuery(SQueryInfo* pQueryInfo) {
} }
bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) {
const char* msg1 = "TWA/Diff not allowed to apply to super table directly"; const char* msg1 = "TWA/Diff/Derivative/Irate not allowed to apply to super table directly";
const char* msg2 = "TWA/Diff only support group by tbname for super table query"; const char* msg2 = "TWA/Diff/Derivative/Irate only support group by tbname for super table query";
const char* msg3 = "function not support for super table query"; const char* msg3 = "function not support for super table query";
// filter sql function not supported by metric query yet. // filter sql function not supported by metric query yet.
...@@ -3099,7 +3102,7 @@ bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) ...@@ -3099,7 +3102,7 @@ bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo)
} }
} }
if (tscIsTWAQuery(pQueryInfo) || tscIsDiffQuery(pQueryInfo)) { if (tscIsTWAQuery(pQueryInfo) || tscIsDiffDerivQuery(pQueryInfo) || tscIsIrateQuery(pQueryInfo)) {
if (pQueryInfo->groupbyExpr.numOfGroupCols == 0) { if (pQueryInfo->groupbyExpr.numOfGroupCols == 0) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
return true; return true;
...@@ -3117,7 +3120,7 @@ bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) ...@@ -3117,7 +3120,7 @@ bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo)
} }
} else if (tscIsSessionWindowQuery(pQueryInfo)) { } else if (tscIsSessionWindowQuery(pQueryInfo)) {
invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
return true; return true;
} }
return false; return false;
...@@ -3360,12 +3363,11 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, ...@@ -3360,12 +3363,11 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo,
// TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd // TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd
} else if (pExpr->tokenId == TK_IN) { } else if (pExpr->tokenId == TK_IN) {
tVariant *pVal; tVariant *pVal;
if (pRight->tokenId != TK_SET || !serializeExprListToVariant(pRight->pParam, &pVal, colType, timePrecision)) { if (pRight->tokenId != TK_SET || !serializeExprListToVariant(pRight->pParam, &pVal, colType, timePrecision)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
} }
if (validateParamOfRelationIn(pVal, colType) != TSDB_CODE_SUCCESS) { if (validateParamOfRelationIn(pVal, colType) != TSDB_CODE_SUCCESS) {
tVariantDestroy(pVal); tVariantDestroy(pVal);
free(pVal); free(pVal);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
} }
...@@ -3374,9 +3376,9 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, ...@@ -3374,9 +3376,9 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo,
pColumnFilter->filterstr = 1; pColumnFilter->filterstr = 1;
memcpy((char *)(pColumnFilter->pz), (char *)(pVal->pz), pVal->nLen); memcpy((char *)(pColumnFilter->pz), (char *)(pVal->pz), pVal->nLen);
tVariantDestroy(pVal); tVariantDestroy(pVal);
free(pVal); free(pVal);
} else if (colType == TSDB_DATA_TYPE_BINARY) { } else if (colType == TSDB_DATA_TYPE_BINARY) {
pColumnFilter->pz = (int64_t)calloc(1, bufLen * TSDB_NCHAR_SIZE); pColumnFilter->pz = (int64_t)calloc(1, bufLen * TSDB_NCHAR_SIZE);
pColumnFilter->len = pRight->value.nLen; pColumnFilter->len = pRight->value.nLen;
...@@ -4907,7 +4909,7 @@ int32_t getTimeRange(STimeWindow* win, tSqlExpr* pRight, int32_t optr, int16_t t ...@@ -4907,7 +4909,7 @@ int32_t getTimeRange(STimeWindow* win, tSqlExpr* pRight, int32_t optr, int16_t t
if (pRight->flags & (1 << EXPR_FLAG_NS_TIMESTAMP)) { if (pRight->flags & (1 << EXPR_FLAG_NS_TIMESTAMP)) {
pRight->value.i64 = convertTimePrecision(pRight->value.i64, TSDB_TIME_PRECISION_NANO, timePrecision); pRight->value.i64 = convertTimePrecision(pRight->value.i64, TSDB_TIME_PRECISION_NANO, timePrecision);
} }
tVariantDump(&pRight->value, (char*)&val, TSDB_DATA_TYPE_BIGINT, true); tVariantDump(&pRight->value, (char*)&val, TSDB_DATA_TYPE_BIGINT, true);
} }
...@@ -5565,7 +5567,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -5565,7 +5567,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (pItem->type != TSDB_DATA_TYPE_BINARY && pItem->type != TSDB_DATA_TYPE_NCHAR) { if (pItem->type != TSDB_DATA_TYPE_BINARY && pItem->type != TSDB_DATA_TYPE_NCHAR) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg21); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg21);
} }
SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER;
SStrToken name = {.type = TK_STRING, .z = pItem->name, .n = (uint32_t)strlen(pItem->name)}; SStrToken name = {.type = TK_STRING, .z = pItem->name, .n = (uint32_t)strlen(pItem->name)};
if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) {
...@@ -5586,11 +5588,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -5586,11 +5588,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
(pItem->type == TSDB_DATA_TYPE_NCHAR && (pItem->bytes <= 0 || pItem->bytes > TSDB_MAX_NCHAR_LEN))) { (pItem->type == TSDB_DATA_TYPE_NCHAR && (pItem->bytes <= 0 || pItem->bytes > TSDB_MAX_NCHAR_LEN))) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg24); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg24);
} }
if (pItem->bytes <= pColSchema->bytes) { if (pItem->bytes <= pColSchema->bytes) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg22); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg22);
} }
TAOS_FIELD f = tscCreateField(pColSchema->type, name.z, pItem->bytes); TAOS_FIELD f = tscCreateField(pColSchema->type, name.z, pItem->bytes);
tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
}else if (pAlterSQL->type == TSDB_ALTER_TABLE_MODIFY_TAG_COLUMN) { }else if (pAlterSQL->type == TSDB_ALTER_TABLE_MODIFY_TAG_COLUMN) {
...@@ -5602,7 +5604,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -5602,7 +5604,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (pItem->type != TSDB_DATA_TYPE_BINARY && pItem->type != TSDB_DATA_TYPE_NCHAR) { if (pItem->type != TSDB_DATA_TYPE_BINARY && pItem->type != TSDB_DATA_TYPE_NCHAR) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg21); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg21);
} }
SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER;
SStrToken name = {.type = TK_STRING, .z = pItem->name, .n = (uint32_t)strlen(pItem->name)}; SStrToken name = {.type = TK_STRING, .z = pItem->name, .n = (uint32_t)strlen(pItem->name)};
if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) {
...@@ -5627,11 +5629,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -5627,11 +5629,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
(pItem->type == TSDB_DATA_TYPE_NCHAR && (pItem->bytes <= 0 || pItem->bytes > TSDB_MAX_NCHAR_LEN))) { (pItem->type == TSDB_DATA_TYPE_NCHAR && (pItem->bytes <= 0 || pItem->bytes > TSDB_MAX_NCHAR_LEN))) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg24); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg24);
} }
if (pItem->bytes <= pColSchema->bytes) { if (pItem->bytes <= pColSchema->bytes) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg22); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg22);
} }
TAOS_FIELD f = tscCreateField(pColSchema->type, name.z, pItem->bytes); TAOS_FIELD f = tscCreateField(pColSchema->type, name.z, pItem->bytes);
tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
} }
...@@ -5997,7 +5999,7 @@ static int32_t setKeepOption(SSqlCmd* pCmd, SCreateDbMsg* pMsg, SCreateDbInfo* p ...@@ -5997,7 +5999,7 @@ static int32_t setKeepOption(SSqlCmd* pCmd, SCreateDbMsg* pMsg, SCreateDbInfo* p
tVariantListItem* p0 = taosArrayGet(pKeep, 0); tVariantListItem* p0 = taosArrayGet(pKeep, 0);
tVariantListItem* p1 = (s > 1) ? taosArrayGet(pKeep, 1) : p0; tVariantListItem* p1 = (s > 1) ? taosArrayGet(pKeep, 1) : p0;
tVariantListItem* p2 = (s > 2) ? taosArrayGet(pKeep, 2) : p1; tVariantListItem* p2 = (s > 2) ? taosArrayGet(pKeep, 2) : p1;
if ((int32_t)p0->pVar.i64 <= 0 || (int32_t)p1->pVar.i64 <= 0 || (int32_t)p2->pVar.i64 <= 0) { if ((int32_t)p0->pVar.i64 <= 0 || (int32_t)p1->pVar.i64 <= 0 || (int32_t)p2->pVar.i64 <= 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
} }
...@@ -7111,7 +7113,7 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) { ...@@ -7111,7 +7113,7 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) {
} else { } else {
if (pQueryInfo->interval.interval == 0) { if (pQueryInfo->interval.interval == 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
} }
} }
// set the created table[stream] name // set the created table[stream] name
...@@ -7784,6 +7786,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -7784,6 +7786,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
const char* msg4 = "interval query not supported, since the result of sub query not include valid timestamp column"; const char* msg4 = "interval query not supported, since the result of sub query not include valid timestamp column";
const char* msg5 = "only tag query not compatible with normal column filter"; const char* msg5 = "only tag query not compatible with normal column filter";
const char* msg6 = "not support stddev/percentile in outer query yet"; const char* msg6 = "not support stddev/percentile in outer query yet";
const char* msg7 = "drivative requires timestamp column exists in subquery";
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
...@@ -7833,11 +7836,27 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -7833,11 +7836,27 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
} }
} }
// todo derivative function requires ts column exists in subquery
STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta;
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, 0);
if (tscNumOfExprs(pQueryInfo) > 1) {
SExprInfo* pExpr = tscExprGet(pQueryInfo, 1);
if (pExpr->base.functionId == TSDB_FUNC_DERIVATIVE && pSchema->type != TSDB_DATA_TYPE_TIMESTAMP) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
}
}
// validate the query filter condition info // validate the query filter condition info
if (pSqlNode->pWhere != NULL) { if (pSqlNode->pWhere != NULL) {
if (validateWhereNode(pQueryInfo, &pSqlNode->pWhere, pSql) != TSDB_CODE_SUCCESS) { if (validateWhereNode(pQueryInfo, &pSqlNode->pWhere, pSql) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
if (pTableMeta->tableInfo.precision == TSDB_TIME_PRECISION_MILLI) {
pQueryInfo->window.skey = pQueryInfo->window.skey / 1000;
pQueryInfo->window.ekey = pQueryInfo->window.ekey / 1000;
}
} }
// validate the interval info // validate the interval info
...@@ -7858,7 +7877,6 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -7858,7 +7877,6 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
} }
// set order by info // set order by info
STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta;
if (validateOrderbyNode(pCmd, pQueryInfo, pSqlNode, tscGetTableSchema(pTableMeta)) != TSDB_CODE_SUCCESS) { if (validateOrderbyNode(pCmd, pQueryInfo, pSqlNode, tscGetTableSchema(pTableMeta)) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
...@@ -7911,7 +7929,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -7911,7 +7929,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
TSDB_CODE_SUCCESS) { TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
// parse the window_state // parse the window_state
if (validateStateWindowNode(pCmd, pQueryInfo, pSqlNode, isSTable) != TSDB_CODE_SUCCESS) { if (validateStateWindowNode(pCmd, pQueryInfo, pSqlNode, isSTable) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
...@@ -8009,7 +8027,6 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -8009,7 +8027,6 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
pQueryInfo->arithmeticOnAgg = tsIsArithmeticQueryOnAggResult(pQueryInfo); pQueryInfo->arithmeticOnAgg = tsIsArithmeticQueryOnAggResult(pQueryInfo);
pQueryInfo->orderProjectQuery = tscOrderedProjectionQueryOnSTable(pQueryInfo, 0); pQueryInfo->orderProjectQuery = tscOrderedProjectionQueryOnSTable(pQueryInfo, 0);
// pQueryInfo->diffQuery = tscIsDiffQuery(pQueryInfo);
SExprInfo** p = NULL; SExprInfo** p = NULL;
int32_t numOfExpr = 0; int32_t numOfExpr = 0;
...@@ -8137,7 +8154,7 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS ...@@ -8137,7 +8154,7 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS
SColIndex* idx = taosArrayGet(pCols, 0); SColIndex* idx = taosArrayGet(pCols, 0);
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex); SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex);
if (pSchema != NULL) { if (pSchema != NULL) {
colType = pSchema->type; colType = pSchema->type;
} }
} }
tVariant *pVal; tVariant *pVal;
......
...@@ -283,6 +283,7 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) { ...@@ -283,6 +283,7 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) {
SArray* tables = getTableList(pSql); SArray* tables = getTableList(pSql);
if (tables == NULL) { if (tables == NULL) {
pSub->lastSyncTime = 0; //force to get table list next time
return 0; return 0;
} }
size_t numOfTables = taosArrayGetSize(tables); size_t numOfTables = taosArrayGetSize(tables);
...@@ -489,7 +490,15 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) { ...@@ -489,7 +490,15 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) {
SSub *pSub = (SSub *)tsub; SSub *pSub = (SSub *)tsub;
if (pSub == NULL) return NULL; if (pSub == NULL) return NULL;
if (pSub->pSql->cmd.command == TSDB_SQL_RETRIEVE_EMPTY_RESULT) { if (pSub->pTimer == NULL) {
int64_t duration = taosGetTimestampMs() - pSub->lastConsumeTime;
if (duration < (int64_t)(pSub->interval)) {
tscDebug("subscription consume too frequently, blocking...");
taosMsleep(pSub->interval - (int32_t)duration);
}
}
if (pSub->pSql->cmd.command == TSDB_SQL_RETRIEVE_EMPTY_RESULT) { //may reach here when retireve stable vgroup failed
SSqlObj* pSql = recreateSqlObj(pSub); SSqlObj* pSql = recreateSqlObj(pSub);
if (pSql == NULL) { if (pSql == NULL) {
return NULL; return NULL;
...@@ -503,6 +512,11 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) { ...@@ -503,6 +512,11 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) {
pSub->pSql = pSql; pSub->pSql = pSql;
pSql->pSubscription = pSub; pSql->pSubscription = pSub;
// no table list now, force to update it
tscDebug("begin table synchronization");
if (!tscUpdateSubscription(pSub->taos, pSub)) return NULL;
tscDebug("table synchronization completed");
} }
tscSaveSubscriptionProgress(pSub); tscSaveSubscriptionProgress(pSub);
...@@ -527,14 +541,6 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) { ...@@ -527,14 +541,6 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) {
tscDebug("subscribe:%s set next round subscribe skey:%"PRId64, pSub->topic, pQueryInfo->window.skey); tscDebug("subscribe:%s set next round subscribe skey:%"PRId64, pSub->topic, pQueryInfo->window.skey);
} }
if (pSub->pTimer == NULL) {
int64_t duration = taosGetTimestampMs() - pSub->lastConsumeTime;
if (duration < (int64_t)(pSub->interval)) {
tscDebug("subscription consume too frequently, blocking...");
taosMsleep(pSub->interval - (int32_t)duration);
}
}
size_t size = taosArrayGetSize(pSub->progress) * sizeof(STableIdInfo); size_t size = taosArrayGetSize(pSub->progress) * sizeof(STableIdInfo);
size += sizeof(SQueryTableMsg) + 4096; size += sizeof(SQueryTableMsg) + 4096;
int code = tscAllocPayload(&pSql->cmd, (int)size); int code = tscAllocPayload(&pSql->cmd, (int)size);
......
...@@ -109,7 +109,7 @@ bool subAndCheckDone(SSqlObj *pSql, SSqlObj *pParentSql, int idx) { ...@@ -109,7 +109,7 @@ bool subAndCheckDone(SSqlObj *pSql, SSqlObj *pParentSql, int idx) {
// pthread_mutex_unlock(&subState->mutex); // pthread_mutex_unlock(&subState->mutex);
// return false; // return false;
// } // }
tscDebug("0x%"PRIx64" subquery:0x%"PRIx64", index:%d state set to 1", pParentSql->self, pSql->self, idx); tscDebug("0x%"PRIx64" subquery:0x%"PRIx64", index:%d state set to 1", pParentSql->self, pSql->self, idx);
subState->states[idx] = 1; subState->states[idx] = 1;
...@@ -622,7 +622,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { ...@@ -622,7 +622,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid); int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid);
// set the tag column id for executor to extract correct tag value // set the tag column id for executor to extract correct tag value
#ifndef _TD_NINGSI_60 #ifndef _TD_NINGSI_60
pExpr->base.param[0] = (tVariant) {.i64 = colId, .nType = TSDB_DATA_TYPE_BIGINT, .nLen = sizeof(int64_t)}; pExpr->base.param[0] = (tVariant) {.i64 = colId, .nType = TSDB_DATA_TYPE_BIGINT, .nLen = sizeof(int64_t)};
#else #else
pExpr->base.param[0].i64 = colId; pExpr->base.param[0].i64 = colId;
...@@ -1843,7 +1843,7 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter ...@@ -1843,7 +1843,7 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter
// refactor as one method // refactor as one method
SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd); SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd);
assert(pNewQueryInfo != NULL); assert(pNewQueryInfo != NULL);
pSupporter->colList = pNewQueryInfo->colList; pSupporter->colList = pNewQueryInfo->colList;
pNewQueryInfo->colList = NULL; pNewQueryInfo->colList = NULL;
...@@ -3158,7 +3158,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { ...@@ -3158,7 +3158,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) {
pRes->code = TSDB_CODE_TSC_APP_ERROR; pRes->code = TSDB_CODE_TSC_APP_ERROR;
return pRes->code; return pRes->code;
} }
for(int32_t i = 0; i < pSql->subState.numOfSub; ++i) { for(int32_t i = 0; i < pSql->subState.numOfSub; ++i) {
SSqlObj* pSub = pSql->pSubs[i]; SSqlObj* pSub = pSql->pSubs[i];
SInsertSupporter* pSup = calloc(1, sizeof(SInsertSupporter)); SInsertSupporter* pSup = calloc(1, sizeof(SInsertSupporter));
......
...@@ -288,16 +288,24 @@ static int taos_options_imp(TSDB_OPTION option, const char *pStr) { ...@@ -288,16 +288,24 @@ static int taos_options_imp(TSDB_OPTION option, const char *pStr) {
if (strlen(tsLocale) == 0) { // locale does not set yet if (strlen(tsLocale) == 0) { // locale does not set yet
char* defaultLocale = setlocale(LC_CTYPE, ""); char* defaultLocale = setlocale(LC_CTYPE, "");
// The locale of the current OS does not be set correctly, so the default locale cannot be acquired.
// The launch of current system will abort soon.
if (defaultLocale == NULL) {
tscError("failed to get default locale, please set the correct locale in current OS");
return -1;
}
tstrncpy(tsLocale, defaultLocale, TSDB_LOCALE_LEN); tstrncpy(tsLocale, defaultLocale, TSDB_LOCALE_LEN);
} }
// set the user specified locale // set the user specified locale
char *locale = setlocale(LC_CTYPE, pStr); char *locale = setlocale(LC_CTYPE, pStr);
if (locale != NULL) { if (locale != NULL) { // failed to set the user specified locale
tscInfo("locale set, prev locale:%s, new locale:%s", tsLocale, locale); tscInfo("locale set, prev locale:%s, new locale:%s", tsLocale, locale);
cfg->cfgStatus = TAOS_CFG_CSTATUS_OPTION; cfg->cfgStatus = TAOS_CFG_CSTATUS_OPTION;
} else { // set the user-specified localed failed, use default LC_CTYPE as current locale } else { // set the user specified locale failed, use default LC_CTYPE as current locale
locale = setlocale(LC_CTYPE, tsLocale); locale = setlocale(LC_CTYPE, tsLocale);
tscInfo("failed to set locale:%s, current locale:%s", pStr, tsLocale); tscInfo("failed to set locale:%s, current locale:%s", pStr, tsLocale);
} }
......
...@@ -460,17 +460,15 @@ bool tscIsTWAQuery(SQueryInfo* pQueryInfo) { ...@@ -460,17 +460,15 @@ bool tscIsTWAQuery(SQueryInfo* pQueryInfo) {
return false; return false;
} }
bool tscIsDiffQuery(SQueryInfo* pQueryInfo) { bool tscIsIrateQuery(SQueryInfo* pQueryInfo) {
size_t num = tscNumOfExprs(pQueryInfo); size_t numOfExprs = tscNumOfExprs(pQueryInfo);
for(int32_t i = 0; i < num; ++i) { for (int32_t i = 0; i < numOfExprs; ++i) {
SExprInfo* pExpr = tscExprGet(pQueryInfo, i); SExprInfo* pExpr = tscExprGet(pQueryInfo, i);
if (pExpr == NULL) {
int32_t f = pExpr->base.functionId;
if (pExpr == NULL || f == TSDB_FUNC_TS_DUMMY) {
continue; continue;
} }
if (f == TSDB_FUNC_DIFF || f == TSDB_FUNC_DERIVATIVE) { if (pExpr->base.functionId == TSDB_FUNC_IRATE) {
return true; return true;
} }
} }
...@@ -516,7 +514,7 @@ bool isSimpleAggregateRv(SQueryInfo* pQueryInfo) { ...@@ -516,7 +514,7 @@ bool isSimpleAggregateRv(SQueryInfo* pQueryInfo) {
return false; return false;
} }
if (tscIsDiffQuery(pQueryInfo)) { if (tscIsDiffDerivQuery(pQueryInfo)) {
return false; return false;
} }
...@@ -3495,6 +3493,7 @@ static void tscSubqueryRetrieveCallback(void* param, TAOS_RES* tres, int code) { ...@@ -3495,6 +3493,7 @@ static void tscSubqueryRetrieveCallback(void* param, TAOS_RES* tres, int code) {
if (pSql->res.code == TSDB_CODE_SUCCESS) { if (pSql->res.code == TSDB_CODE_SUCCESS) {
(*pSql->fp)(pParentSql->param, pParentSql, pParentSql->res.numOfRows); (*pSql->fp)(pParentSql->param, pParentSql, pParentSql->res.numOfRows);
} else { } else {
pParentSql->res.code = pSql->res.code;
tscAsyncResultOnError(pParentSql); tscAsyncResultOnError(pParentSql);
} }
} }
...@@ -4260,7 +4259,7 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt ...@@ -4260,7 +4259,7 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt
pQueryAttr->hasTagResults = hasTagValOutput(pQueryInfo); pQueryAttr->hasTagResults = hasTagValOutput(pQueryInfo);
pQueryAttr->stabledev = isStabledev(pQueryInfo); pQueryAttr->stabledev = isStabledev(pQueryInfo);
pQueryAttr->tsCompQuery = isTsCompQuery(pQueryInfo); pQueryAttr->tsCompQuery = isTsCompQuery(pQueryInfo);
pQueryAttr->diffQuery = tscIsDiffQuery(pQueryInfo); pQueryAttr->diffQuery = tscIsDiffDerivQuery(pQueryInfo);
pQueryAttr->simpleAgg = isSimpleAggregateRv(pQueryInfo); pQueryAttr->simpleAgg = isSimpleAggregateRv(pQueryInfo);
pQueryAttr->needReverseScan = tscNeedReverseScan(pQueryInfo); pQueryAttr->needReverseScan = tscNeedReverseScan(pQueryInfo);
pQueryAttr->stableQuery = QUERY_IS_STABLE_QUERY(pQueryInfo->type); pQueryAttr->stableQuery = QUERY_IS_STABLE_QUERY(pQueryInfo->type);
......
...@@ -7,7 +7,12 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib) ...@@ -7,7 +7,12 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib)
IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR) IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR)
MESSAGE(STATUS "gTest library found, build unit test") MESSAGE(STATUS "gTest library found, build unit test")
INCLUDE_DIRECTORIES(${HEADER_GTEST_INCLUDE_DIR}) # GoogleTest requires at least C++11
SET(CMAKE_CXX_STANDARD 11)
INCLUDE_DIRECTORIES(/usr/include /usr/local/include)
LINK_DIRECTORIES(/usr/lib /usr/local/lib)
AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST) AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST)
ADD_EXECUTABLE(cliTest ${SOURCE_LIST}) ADD_EXECUTABLE(cliTest ${SOURCE_LIST})
......
...@@ -8,9 +8,8 @@ IF (TD_MVN_INSTALLED) ...@@ -8,9 +8,8 @@ IF (TD_MVN_INSTALLED)
ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME} ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME}
POST_BUILD POST_BUILD
COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.30.jar ${LIBRARY_OUTPUT_PATH} COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.31.jar ${LIBRARY_OUTPUT_PATH}
COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMENT "build jdbc driver") COMMENT "build jdbc driver")
ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME}) ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME})
ENDIF () ENDIF ()
\ No newline at end of file
...@@ -5,7 +5,7 @@ ...@@ -5,7 +5,7 @@
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.30</version> <version>2.0.31</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>JDBCDriver</name> <name>JDBCDriver</name>
......
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId> <groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId> <artifactId>taos-jdbcdriver</artifactId>
<version>2.0.30</version> <version>2.0.31</version>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>JDBCDriver</name> <name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url> <url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
...@@ -122,7 +122,7 @@ ...@@ -122,7 +122,7 @@
<exclude>**/FailOverTest.java</exclude> <exclude>**/FailOverTest.java</exclude>
<exclude>**/InvalidResultSetPointerTest.java</exclude> <exclude>**/InvalidResultSetPointerTest.java</exclude>
<exclude>**/RestfulConnectionTest.java</exclude> <exclude>**/RestfulConnectionTest.java</exclude>
<exclude>**/TD4144Test.java</exclude> <exclude>**/ConnectMultiTaosdByRestfulWithDifferentTokenTest.java</exclude>
</excludes> </excludes>
<testFailureIgnore>true</testFailureIgnore> <testFailureIgnore>true</testFailureIgnore>
</configuration> </configuration>
......
package com.taosdata.jdbc; package com.taosdata.jdbc;
import com.taosdata.jdbc.rs.enums.TimestampFormat;
import java.sql.*; import java.sql.*;
import java.util.Enumeration; import java.util.Enumeration;
import java.util.Map; import java.util.Map;
...@@ -18,7 +20,7 @@ public abstract class AbstractConnection extends WrapperImpl implements Connecti ...@@ -18,7 +20,7 @@ public abstract class AbstractConnection extends WrapperImpl implements Connecti
for (String propName : propNames) { for (String propName : propNames) {
clientInfoProps.setProperty(propName, properties.getProperty(propName)); clientInfoProps.setProperty(propName, properties.getProperty(propName));
} }
String timestampFormat = properties.getProperty(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT, "STRING"); String timestampFormat = properties.getProperty(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT, String.valueOf(TimestampFormat.STRING));
clientInfoProps.setProperty(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT, timestampFormat); clientInfoProps.setProperty(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT, timestampFormat);
} }
...@@ -516,7 +518,6 @@ public abstract class AbstractConnection extends WrapperImpl implements Connecti ...@@ -516,7 +518,6 @@ public abstract class AbstractConnection extends WrapperImpl implements Connecti
public void abort(Executor executor) throws SQLException { public void abort(Executor executor) throws SQLException {
if (isClosed()) if (isClosed())
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_CONNECTION_CLOSED); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_CONNECTION_CLOSED);
// do nothing // do nothing
} }
......
...@@ -9,8 +9,6 @@ public abstract class AbstractStatement extends WrapperImpl implements Statement ...@@ -9,8 +9,6 @@ public abstract class AbstractStatement extends WrapperImpl implements Statement
protected List<String> batchedArgs; protected List<String> batchedArgs;
private int fetchSize; private int fetchSize;
@Override @Override
public abstract ResultSet executeQuery(String sql) throws SQLException; public abstract ResultSet executeQuery(String sql) throws SQLException;
......
...@@ -32,6 +32,7 @@ public class TSDBError { ...@@ -32,6 +32,7 @@ public class TSDBError {
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_INVALID_SQL, "invalid sql"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_INVALID_SQL, "invalid sql");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE, "numeric value out of range"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE, "numeric value out of range");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE, "unknown taos type in tdengine"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE, "unknown taos type in tdengine");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN_TIMESTAMP_PERCISION, "unknown timestamp precision");
/**************************************************/ /**************************************************/
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN, "unknown error"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN, "unknown error");
......
...@@ -25,6 +25,7 @@ public class TSDBErrorNumbers { ...@@ -25,6 +25,7 @@ public class TSDBErrorNumbers {
public static final int ERROR_INVALID_SQL = 0x2313; // invalid sql public static final int ERROR_INVALID_SQL = 0x2313; // invalid sql
public static final int ERROR_NUMERIC_VALUE_OUT_OF_RANGE = 0x2314; // numeric value out of range public static final int ERROR_NUMERIC_VALUE_OUT_OF_RANGE = 0x2314; // numeric value out of range
public static final int ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE = 0x2315; //unknown taos type in tdengine public static final int ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE = 0x2315; //unknown taos type in tdengine
public static final int ERROR_UNKNOWN_TIMESTAMP_PERCISION = 0x2316; // unknown timestamp precision
public static final int ERROR_UNKNOWN = 0x2350; //unknown error public static final int ERROR_UNKNOWN = 0x2350; //unknown error
...@@ -62,6 +63,7 @@ public class TSDBErrorNumbers { ...@@ -62,6 +63,7 @@ public class TSDBErrorNumbers {
errorNumbers.add(ERROR_INVALID_SQL); errorNumbers.add(ERROR_INVALID_SQL);
errorNumbers.add(ERROR_NUMERIC_VALUE_OUT_OF_RANGE); errorNumbers.add(ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
errorNumbers.add(ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE); errorNumbers.add(ERROR_UNKNOWN_TAOS_TYPE_IN_TDENGINE);
errorNumbers.add(ERROR_UNKNOWN_TIMESTAMP_PERCISION);
/*****************************************************/ /*****************************************************/
errorNumbers.add(ERROR_SUBSCRIBE_FAILED); errorNumbers.add(ERROR_SUBSCRIBE_FAILED);
......
...@@ -216,6 +216,16 @@ public class TSDBJNIConnector { ...@@ -216,6 +216,16 @@ public class TSDBJNIConnector {
private native int fetchBlockImp(long connection, long resultSet, TSDBResultSetBlockData blockData); private native int fetchBlockImp(long connection, long resultSet, TSDBResultSetBlockData blockData);
/**
* Get Result Time Precision.
* @return 0: ms, 1: us, 2: ns
*/
public int getResultTimePrecision(long sqlObj) {
return this.getResultTimePrecisionImp(this.taos, sqlObj);
}
private native int getResultTimePrecisionImp(long connection, long result);
/** /**
* Execute close operation from C to release connection pointer by JNI * Execute close operation from C to release connection pointer by JNI
* *
......
...@@ -38,7 +38,6 @@ import java.util.regex.Pattern; ...@@ -38,7 +38,6 @@ import java.util.regex.Pattern;
public class TSDBPreparedStatement extends TSDBStatement implements PreparedStatement { public class TSDBPreparedStatement extends TSDBStatement implements PreparedStatement {
private String rawSql; private String rawSql;
private Object[] parameters; private Object[] parameters;
private boolean isPrepared;
private ArrayList<ColumnInfo> colData; private ArrayList<ColumnInfo> colData;
private ArrayList<TableTagInfo> tableTags; private ArrayList<TableTagInfo> tableTags;
...@@ -47,8 +46,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat ...@@ -47,8 +46,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
private String tableName; private String tableName;
private long nativeStmtHandle = 0; private long nativeStmtHandle = 0;
private volatile TSDBParameterMetaData parameterMetaData;
TSDBPreparedStatement(TSDBConnection connection, String sql) { TSDBPreparedStatement(TSDBConnection connection, String sql) {
super(connection); super(connection);
init(sql); init(sql);
...@@ -61,7 +58,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat ...@@ -61,7 +58,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
} }
} }
parameters = new Object[parameterCnt]; parameters = new Object[parameterCnt];
this.isPrepared = true;
} }
if (parameterCnt > 1) { if (parameterCnt > 1) {
...@@ -76,11 +72,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat ...@@ -76,11 +72,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
preprocessSql(); preprocessSql();
} }
@Override
public int[] executeBatch() throws SQLException {
return super.executeBatch();
}
/* /*
* Some of the SQLs sent by other popular frameworks or tools like Spark, contains syntax that cannot be parsed by * Some of the SQLs sent by other popular frameworks or tools like Spark, contains syntax that cannot be parsed by
* the TDengine client. Thus, some simple parsers/filters are intentionally added in this JDBC implementation in * the TDengine client. Thus, some simple parsers/filters are intentionally added in this JDBC implementation in
...@@ -137,29 +128,15 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat ...@@ -137,29 +128,15 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
/***** for inner queries *****/ /***** for inner queries *****/
} }
/**
* Populate parameters into prepared sql statements
*
* @return a string of the native sql statement for TSDB
*/
private String getNativeSql(String rawSql) throws SQLException {
return Utils.getNativeSql(rawSql, this.parameters);
}
@Override @Override
public ResultSet executeQuery() throws SQLException { public ResultSet executeQuery() throws SQLException {
if (!isPrepared) final String sql = Utils.getNativeSql(this.rawSql, this.parameters);
return executeQuery(this.rawSql);
final String sql = getNativeSql(this.rawSql);
return executeQuery(sql); return executeQuery(sql);
} }
@Override @Override
public int executeUpdate() throws SQLException { public int executeUpdate() throws SQLException {
if (!isPrepared) String sql = Utils.getNativeSql(this.rawSql, this.parameters);
return executeUpdate(this.rawSql);
String sql = getNativeSql(this.rawSql);
return executeUpdate(sql); return executeUpdate(sql);
} }
...@@ -282,25 +259,14 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat ...@@ -282,25 +259,14 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
@Override @Override
public boolean execute() throws SQLException { public boolean execute() throws SQLException {
if (!isPrepared) final String sql = Utils.getNativeSql(this.rawSql, this.parameters);
return execute(this.rawSql);
final String sql = getNativeSql(this.rawSql);
return execute(sql); return execute(sql);
} }
@Override @Override
public void addBatch() throws SQLException { public void addBatch() throws SQLException {
if (this.batchedArgs == null) { String sql = Utils.getNativeSql(this.rawSql, this.parameters);
batchedArgs = new ArrayList<>(); addBatch(sql);
}
if (!isPrepared) {
addBatch(this.rawSql);
} else {
String sql = this.getConnection().nativeSQL(this.rawSql);
addBatch(sql);
}
} }
@Override @Override
......
...@@ -147,30 +147,14 @@ public class TSDBResultSetRowData { ...@@ -147,30 +147,14 @@ public class TSDBResultSetRowData {
case TSDBConstants.TSDB_DATA_TYPE_NCHAR: case TSDBConstants.TSDB_DATA_TYPE_NCHAR:
case TSDBConstants.TSDB_DATA_TYPE_BINARY: case TSDBConstants.TSDB_DATA_TYPE_BINARY:
return Integer.parseInt((String) obj); return Integer.parseInt((String) obj);
case TSDBConstants.TSDB_DATA_TYPE_UTINYINT: { case TSDBConstants.TSDB_DATA_TYPE_UTINYINT:
Byte value = (byte) obj; return parseUnsignedTinyIntToInt(obj);
if (value < 0) case TSDBConstants.TSDB_DATA_TYPE_USMALLINT:
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE); return parseUnsignedSmallIntToInt(obj);
return value; case TSDBConstants.TSDB_DATA_TYPE_UINT:
} return parseUnsignedIntegerToInt(obj);
case TSDBConstants.TSDB_DATA_TYPE_USMALLINT: { case TSDBConstants.TSDB_DATA_TYPE_UBIGINT:
short value = (short) obj; return parseUnsignedBigIntToInt(obj);
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value;
}
case TSDBConstants.TSDB_DATA_TYPE_UINT: {
int value = (int) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value;
}
case TSDBConstants.TSDB_DATA_TYPE_UBIGINT: {
long value = (long) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return Long.valueOf(value).intValue();
}
case TSDBConstants.TSDB_DATA_TYPE_FLOAT: case TSDBConstants.TSDB_DATA_TYPE_FLOAT:
return ((Float) obj).intValue(); return ((Float) obj).intValue();
case TSDBConstants.TSDB_DATA_TYPE_DOUBLE: case TSDBConstants.TSDB_DATA_TYPE_DOUBLE:
...@@ -180,6 +164,35 @@ public class TSDBResultSetRowData { ...@@ -180,6 +164,35 @@ public class TSDBResultSetRowData {
} }
} }
private byte parseUnsignedTinyIntToInt(Object obj) throws SQLException {
byte value = (byte) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value;
}
private short parseUnsignedSmallIntToInt(Object obj) throws SQLException {
short value = (short) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value;
}
private int parseUnsignedIntegerToInt(Object obj) throws SQLException {
int value = (int) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value;
}
private int parseUnsignedBigIntToInt(Object obj) throws SQLException {
long value = (long) obj;
if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return Long.valueOf(value).intValue();
}
/** /**
* $$$ this method is invoked by databaseMetaDataResultSet and so on which use a index start from 1 in JDBC api * $$$ this method is invoked by databaseMetaDataResultSet and so on which use a index start from 1 in JDBC api
*/ */
...@@ -216,7 +229,7 @@ public class TSDBResultSetRowData { ...@@ -216,7 +229,7 @@ public class TSDBResultSetRowData {
case TSDBConstants.TSDB_DATA_TYPE_BINARY: case TSDBConstants.TSDB_DATA_TYPE_BINARY:
return Long.parseLong((String) obj); return Long.parseLong((String) obj);
case TSDBConstants.TSDB_DATA_TYPE_UTINYINT: { case TSDBConstants.TSDB_DATA_TYPE_UTINYINT: {
Byte value = (byte) obj; byte value = (byte) obj;
if (value < 0) if (value < 0)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_NUMERIC_VALUE_OUT_OF_RANGE);
return value; return value;
...@@ -414,26 +427,40 @@ public class TSDBResultSetRowData { ...@@ -414,26 +427,40 @@ public class TSDBResultSetRowData {
* $$$ this method is invoked by databaseMetaDataResultSet and so on which use a index start from 1 in JDBC api * $$$ this method is invoked by databaseMetaDataResultSet and so on which use a index start from 1 in JDBC api
*/ */
public void setTimestampValue(int colIndex, long value) { public void setTimestampValue(int colIndex, long value) {
setTimestamp(colIndex - 1, value); setTimestamp(colIndex - 1, value, 0);
} }
/** /**
* !!! this method is invoked by JNI method and the index start from 0 in C implementations * !!! this method is invoked by JNI method and the index start from 0 in C implementations
* @param precision 0 : ms, 1 : us, 2 : ns
*/ */
public void setTimestamp(int col, long ts) { public void setTimestamp(int col, long ts, int precision) {
//TODO: this implementation contains logical error long milliseconds = 0;
// when precision is us the (long ts) is 16 digital number int fracNanoseconds = 0;
// when precision is ms, the (long ts) is 13 digital number switch (precision) {
// we need a JNI function like this: case 0: {
// public void setTimestamp(int col, long epochSecond, long nanoAdjustment) milliseconds = ts;
if (ts < 1_0000_0000_0000_0L) { fracNanoseconds = (int)(ts*1_000_000%1_000_000_000);
data.set(col, new Timestamp(ts)); break;
} else { }
long epochSec = ts / 1000_000l; case 1: {
long nanoAdjustment = ts % 1000_000l * 1000l; milliseconds = ts/1_000;
Timestamp timestamp = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment)); fracNanoseconds = (int)(ts*1_000%1_000_000_000);
data.set(col, timestamp); break;
}
case 2: {
milliseconds = ts/1_000_000;
fracNanoseconds = (int)(ts%1_000_000_000);
break;
}
default: {
throw new IllegalArgumentException("precision is not valid. precision: " + precision);
}
} }
Timestamp tsObj = new Timestamp(milliseconds);
tsObj.setNanos(fracNanoseconds);
data.set(col, tsObj);
} }
public Timestamp getTimestamp(int col, int nativeType) { public Timestamp getTimestamp(int col, int nativeType) {
......
package com.taosdata.jdbc.enums;
public enum TimestampPrecision {
MS,
US,
NS,
UNKNOWN
}
...@@ -17,16 +17,18 @@ public class RestfulConnection extends AbstractConnection { ...@@ -17,16 +17,18 @@ public class RestfulConnection extends AbstractConnection {
private final int port; private final int port;
private final String url; private final String url;
private volatile String database; private volatile String database;
private final String token;
/******************************************************/ /******************************************************/
private boolean isClosed; private boolean isClosed;
private final DatabaseMetaData metadata; private final DatabaseMetaData metadata;
public RestfulConnection(String host, String port, Properties props, String database, String url) { public RestfulConnection(String host, String port, Properties props, String database, String url, String token) {
super(props); super(props);
this.host = host; this.host = host;
this.port = Integer.parseInt(port); this.port = Integer.parseInt(port);
this.database = database; this.database = database;
this.url = url; this.url = url;
this.token = token;
this.metadata = new RestfulDatabaseMetaData(url, props.getProperty(TSDBDriver.PROPERTY_KEY_USER), this); this.metadata = new RestfulDatabaseMetaData(url, props.getProperty(TSDBDriver.PROPERTY_KEY_USER), this);
} }
...@@ -66,6 +68,7 @@ public class RestfulConnection extends AbstractConnection { ...@@ -66,6 +68,7 @@ public class RestfulConnection extends AbstractConnection {
return this.metadata; return this.metadata;
} }
// getters
public String getHost() { public String getHost() {
return host; return host;
} }
...@@ -81,4 +84,8 @@ public class RestfulConnection extends AbstractConnection { ...@@ -81,4 +84,8 @@ public class RestfulConnection extends AbstractConnection {
public String getUrl() { public String getUrl() {
return url; return url;
} }
public String getToken() {
return token;
}
} }
\ No newline at end of file
...@@ -38,15 +38,11 @@ public class RestfulDriver extends AbstractDriver { ...@@ -38,15 +38,11 @@ public class RestfulDriver extends AbstractDriver {
String port = props.getProperty(TSDBDriver.PROPERTY_KEY_PORT, "6041"); String port = props.getProperty(TSDBDriver.PROPERTY_KEY_PORT, "6041");
String database = props.containsKey(TSDBDriver.PROPERTY_KEY_DBNAME) ? props.getProperty(TSDBDriver.PROPERTY_KEY_DBNAME) : null; String database = props.containsKey(TSDBDriver.PROPERTY_KEY_DBNAME) ? props.getProperty(TSDBDriver.PROPERTY_KEY_DBNAME) : null;
String loginUrl = "http://" + props.getProperty(TSDBDriver.PROPERTY_KEY_HOST) + ":" String loginUrl = "http://" + host + ":" + port + "/rest/login/" + props.getProperty(TSDBDriver.PROPERTY_KEY_USER) + "/" + props.getProperty(TSDBDriver.PROPERTY_KEY_PASSWORD) + "";
+ props.getProperty(TSDBDriver.PROPERTY_KEY_PORT) + "/rest/login/"
+ props.getProperty(TSDBDriver.PROPERTY_KEY_USER) + "/"
+ props.getProperty(TSDBDriver.PROPERTY_KEY_PASSWORD) + "";
try { try {
String user = URLEncoder.encode(props.getProperty(TSDBDriver.PROPERTY_KEY_USER), "UTF-8"); String user = URLEncoder.encode(props.getProperty(TSDBDriver.PROPERTY_KEY_USER), "UTF-8");
String password = URLEncoder.encode(props.getProperty(TSDBDriver.PROPERTY_KEY_PASSWORD), "UTF-8"); String password = URLEncoder.encode(props.getProperty(TSDBDriver.PROPERTY_KEY_PASSWORD), "UTF-8");
loginUrl = "http://" + props.getProperty(TSDBDriver.PROPERTY_KEY_HOST) + ":" loginUrl = "http://" + props.getProperty(TSDBDriver.PROPERTY_KEY_HOST) + ":" + props.getProperty(TSDBDriver.PROPERTY_KEY_PORT) + "/rest/login/" + user + "/" + password + "";
+ props.getProperty(TSDBDriver.PROPERTY_KEY_PORT) + "/rest/login/" + user + "/" + password + "";
} catch (UnsupportedEncodingException e) { } catch (UnsupportedEncodingException e) {
e.printStackTrace(); e.printStackTrace();
} }
...@@ -55,12 +51,12 @@ public class RestfulDriver extends AbstractDriver { ...@@ -55,12 +51,12 @@ public class RestfulDriver extends AbstractDriver {
JSONObject jsonResult = JSON.parseObject(result); JSONObject jsonResult = JSON.parseObject(result);
String status = jsonResult.getString("status"); String status = jsonResult.getString("status");
String token = jsonResult.getString("desc"); String token = jsonResult.getString("desc");
HttpClientPoolUtil.token = token;
if (!status.equals("succ")) { if (!status.equals("succ")) {
throw new SQLException(jsonResult.getString("desc")); throw new SQLException(jsonResult.getString("desc"));
} }
RestfulConnection conn = new RestfulConnection(host, port, props, database, url); RestfulConnection conn = new RestfulConnection(host, port, props, database, url, token);
if (database != null && !database.trim().replaceAll("\\s", "").isEmpty()) { if (database != null && !database.trim().replaceAll("\\s", "").isEmpty()) {
Statement stmt = conn.createStatement(); Statement stmt = conn.createStatement();
stmt.execute("use " + database); stmt.execute("use " + database);
......
...@@ -44,7 +44,7 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar ...@@ -44,7 +44,7 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar
if (!isPrepared) if (!isPrepared)
return executeQuery(this.rawSql); return executeQuery(this.rawSql);
final String sql = getNativeSql(this.rawSql); final String sql = Utils.getNativeSql(this.rawSql, this.parameters);
return executeQuery(sql); return executeQuery(sql);
} }
...@@ -55,20 +55,10 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar ...@@ -55,20 +55,10 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar
if (!isPrepared) if (!isPrepared)
return executeUpdate(this.rawSql); return executeUpdate(this.rawSql);
final String sql = getNativeSql(this.rawSql); final String sql = Utils.getNativeSql(rawSql, this.parameters);
return executeUpdate(sql); return executeUpdate(sql);
} }
/****
* 将rawSql转换成一条可执行的sql语句,使用属性parameters中的变脸进行替换
* 对于insert into ?.? (?,?,?) using ?.? (?,?,?) tags(?, ?, ?) values(?, ?, ?)
* @param rawSql,可能是insert、select或其他,使用?做占位符
* @return
*/
private String getNativeSql(String rawSql) {
return Utils.getNativeSql(rawSql, this.parameters);
}
@Override @Override
public void setNull(int parameterIndex, int sqlType) throws SQLException { public void setNull(int parameterIndex, int sqlType) throws SQLException {
if (isClosed()) if (isClosed())
...@@ -224,16 +214,13 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar ...@@ -224,16 +214,13 @@ public class RestfulPreparedStatement extends RestfulStatement implements Prepar
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_STATEMENT_CLOSED); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_STATEMENT_CLOSED);
if (!isPrepared) if (!isPrepared)
return execute(this.rawSql); return execute(this.rawSql);
final String sql = getNativeSql(rawSql); final String sql = Utils.getNativeSql(rawSql, this.parameters);
return execute(sql); return execute(sql);
} }
@Override @Override
public void addBatch() throws SQLException { public void addBatch() throws SQLException {
if (isClosed()) final String sql = Utils.getNativeSql(rawSql, this.parameters);
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_STATEMENT_CLOSED);
final String sql = getNativeSql(this.rawSql);
addBatch(sql); addBatch(sql);
} }
......
...@@ -6,6 +6,8 @@ import com.google.common.primitives.Ints; ...@@ -6,6 +6,8 @@ import com.google.common.primitives.Ints;
import com.google.common.primitives.Longs; import com.google.common.primitives.Longs;
import com.google.common.primitives.Shorts; import com.google.common.primitives.Shorts;
import com.taosdata.jdbc.*; import com.taosdata.jdbc.*;
import com.taosdata.jdbc.enums.TimestampPrecision;
import com.taosdata.jdbc.rs.enums.TimestampFormat;
import com.taosdata.jdbc.utils.Utils; import com.taosdata.jdbc.utils.Utils;
import java.math.BigDecimal; import java.math.BigDecimal;
...@@ -46,6 +48,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -46,6 +48,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
columnNames.clear(); columnNames.clear();
columns.clear(); columns.clear();
this.resultSet.clear(); this.resultSet.clear();
this.metaData = new RestfulResultSetMetaData(this.database, null, this);
return; return;
} }
// get head // get head
...@@ -131,7 +134,6 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -131,7 +134,6 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
} }
} }
private Object parseColumnData(JSONArray row, int colIndex, int taosType) throws SQLException { private Object parseColumnData(JSONArray row, int colIndex, int taosType) throws SQLException {
switch (taosType) { switch (taosType) {
case TSDBConstants.TSDB_DATA_TYPE_NULL: case TSDBConstants.TSDB_DATA_TYPE_NULL:
...@@ -150,44 +152,8 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -150,44 +152,8 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
return row.getFloat(colIndex); return row.getFloat(colIndex);
case TSDBConstants.TSDB_DATA_TYPE_DOUBLE: case TSDBConstants.TSDB_DATA_TYPE_DOUBLE:
return row.getDouble(colIndex); return row.getDouble(colIndex);
case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP: { case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP:
if (row.get(colIndex) == null) return parseTimestampColumnData(row, colIndex);
return null;
String timestampFormat = this.statement.getConnection().getClientInfo(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT);
if ("TIMESTAMP".equalsIgnoreCase(timestampFormat)) {
Long value = row.getLong(colIndex);
//TODO: this implementation has bug if the timestamp bigger than 9999_9999_9999_9
if (value < 1_0000_0000_0000_0L)
return new Timestamp(value);
long epochSec = value / 1000_000l;
long nanoAdjustment = value % 1000_000l * 1000l;
return Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
}
if ("UTC".equalsIgnoreCase(timestampFormat)) {
String value = row.getString(colIndex);
long epochSec = Timestamp.valueOf(value.substring(0, 19).replace("T", " ")).getTime() / 1000;
int fractionalSec = Integer.parseInt(value.substring(20, value.length() - 5));
long nanoAdjustment = 0;
if (value.length() > 28) {
// ms timestamp: yyyy-MM-ddTHH:mm:ss.SSSSSS+0x00
nanoAdjustment = fractionalSec * 1000l;
} else {
// ms timestamp: yyyy-MM-ddTHH:mm:ss.SSS+0x00
nanoAdjustment = fractionalSec * 1000_000l;
}
ZoneOffset zoneOffset = ZoneOffset.of(value.substring(value.length() - 5));
Instant instant = Instant.ofEpochSecond(epochSec, nanoAdjustment).atOffset(zoneOffset).toInstant();
return Timestamp.from(instant);
}
String value = row.getString(colIndex);
if (value.length() <= 23) // ms timestamp: yyyy-MM-dd HH:mm:ss.SSS
return row.getTimestamp(colIndex);
// us timestamp: yyyy-MM-dd HH:mm:ss.SSSSSS
long epochSec = Timestamp.valueOf(value.substring(0, 19)).getTime() / 1000;
long nanoAdjustment = Integer.parseInt(value.substring(20)) * 1000l;
Timestamp timestamp = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
return timestamp;
}
case TSDBConstants.TSDB_DATA_TYPE_BINARY: case TSDBConstants.TSDB_DATA_TYPE_BINARY:
return row.getString(colIndex) == null ? null : row.getString(colIndex).getBytes(); return row.getString(colIndex) == null ? null : row.getString(colIndex).getBytes();
case TSDBConstants.TSDB_DATA_TYPE_NCHAR: case TSDBConstants.TSDB_DATA_TYPE_NCHAR:
...@@ -197,7 +163,66 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -197,7 +163,66 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
} }
} }
public class Field { private Timestamp parseTimestampColumnData(JSONArray row, int colIndex) throws SQLException {
if (row.get(colIndex) == null)
return null;
String tsFormatUpperCase = this.statement.getConnection().getClientInfo(TSDBDriver.PROPERTY_KEY_TIMESTAMP_FORMAT).toUpperCase();
TimestampFormat timestampFormat = TimestampFormat.valueOf(tsFormatUpperCase);
switch (timestampFormat) {
case TIMESTAMP: {
Long value = row.getLong(colIndex);
//TODO: this implementation has bug if the timestamp bigger than 9999_9999_9999_9
if (value < 1_0000_0000_0000_0L)
return new Timestamp(value);
long epochSec = value / 1000_000l;
long nanoAdjustment = value % 1000_000l * 1000l;
return Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
}
case UTC: {
String value = row.getString(colIndex);
long epochSec = Timestamp.valueOf(value.substring(0, 19).replace("T", " ")).getTime() / 1000;
int fractionalSec = Integer.parseInt(value.substring(20, value.length() - 5));
long nanoAdjustment;
if (value.length() > 31) {
// ns timestamp: yyyy-MM-ddTHH:mm:ss.SSSSSSSSS+0x00
nanoAdjustment = fractionalSec;
} else if (value.length() > 28) {
// ms timestamp: yyyy-MM-ddTHH:mm:ss.SSSSSS+0x00
nanoAdjustment = fractionalSec * 1000L;
} else {
// ms timestamp: yyyy-MM-ddTHH:mm:ss.SSS+0x00
nanoAdjustment = fractionalSec * 1000_000L;
}
ZoneOffset zoneOffset = ZoneOffset.of(value.substring(value.length() - 5));
Instant instant = Instant.ofEpochSecond(epochSec, nanoAdjustment).atOffset(zoneOffset).toInstant();
return Timestamp.from(instant);
}
case STRING:
default: {
String value = row.getString(colIndex);
TimestampPrecision precision = Utils.guessTimestampPrecision(value);
if (precision == TimestampPrecision.MS) {
// ms timestamp: yyyy-MM-dd HH:mm:ss.SSS
return row.getTimestamp(colIndex);
}
if (precision == TimestampPrecision.US) {
// us timestamp: yyyy-MM-dd HH:mm:ss.SSSSSS
long epochSec = Timestamp.valueOf(value.substring(0, 19)).getTime() / 1000;
long nanoAdjustment = Integer.parseInt(value.substring(20)) * 1000L;
return Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
}
if (precision == TimestampPrecision.NS) {
// ms timestamp: yyyy-MM-dd HH:mm:ss.SSSSSSSSS
long epochSec = Timestamp.valueOf(value.substring(0, 19)).getTime() / 1000;
long nanoAdjustment = Integer.parseInt(value.substring(20));
return Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
}
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN_TIMESTAMP_PERCISION);
}
}
}
public static class Field {
String name; String name;
int type; int type;
int length; int length;
...@@ -211,6 +236,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -211,6 +236,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
this.note = note; this.note = note;
this.taos_type = taos_type; this.taos_type = taos_type;
} }
} }
@Override @Override
...@@ -334,6 +360,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet { ...@@ -334,6 +360,7 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
wasNull = true; wasNull = true;
return 0; return 0;
} }
wasNull = false; wasNull = false;
if (value instanceof Timestamp) { if (value instanceof Timestamp) {
return ((Timestamp) value).getTime(); return ((Timestamp) value).getTime();
......
...@@ -8,20 +8,22 @@ import java.sql.SQLException; ...@@ -8,20 +8,22 @@ import java.sql.SQLException;
import java.sql.Timestamp; import java.sql.Timestamp;
import java.sql.Types; import java.sql.Types;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class RestfulResultSetMetaData extends WrapperImpl implements ResultSetMetaData { public class RestfulResultSetMetaData extends WrapperImpl implements ResultSetMetaData {
private final String database; private final String database;
private ArrayList<RestfulResultSet.Field> fields; private List<RestfulResultSet.Field> fields;
private final RestfulResultSet resultSet; private final RestfulResultSet resultSet;
public RestfulResultSetMetaData(String database, ArrayList<RestfulResultSet.Field> fields, RestfulResultSet resultSet) { public RestfulResultSetMetaData(String database, ArrayList<RestfulResultSet.Field> fields, RestfulResultSet resultSet) {
this.database = database; this.database = database;
this.fields = fields; this.fields = fields == null ? Collections.emptyList() : fields;
this.resultSet = resultSet; this.resultSet = resultSet;
} }
public ArrayList<RestfulResultSet.Field> getFields() { public List<RestfulResultSet.Field> getFields() {
return fields; return fields;
} }
......
...@@ -83,7 +83,7 @@ public class RestfulStatement extends AbstractStatement { ...@@ -83,7 +83,7 @@ public class RestfulStatement extends AbstractStatement {
} }
if (SqlSyntaxValidator.isUseSql(sql)) { if (SqlSyntaxValidator.isUseSql(sql)) {
HttpClientPoolUtil.execute(url, sql); HttpClientPoolUtil.execute(url, sql, this.conn.getToken());
this.database = sql.trim().replace("use", "").trim(); this.database = sql.trim().replace("use", "").trim();
this.conn.setCatalog(this.database); this.conn.setCatalog(this.database);
result = false; result = false;
...@@ -116,7 +116,7 @@ public class RestfulStatement extends AbstractStatement { ...@@ -116,7 +116,7 @@ public class RestfulStatement extends AbstractStatement {
if ("UTC".equalsIgnoreCase(timestampFormat)) if ("UTC".equalsIgnoreCase(timestampFormat))
url = "http://" + conn.getHost() + ":" + conn.getPort() + "/rest/sqlutc"; url = "http://" + conn.getHost() + ":" + conn.getPort() + "/rest/sqlutc";
String result = HttpClientPoolUtil.execute(url, sql); String result = HttpClientPoolUtil.execute(url, sql, this.conn.getToken());
JSONObject resultJson = JSON.parseObject(result); JSONObject resultJson = JSON.parseObject(result);
if (resultJson.getString("status").equals("error")) { if (resultJson.getString("status").equals("error")) {
throw TSDBError.createSQLException(resultJson.getInteger("code"), resultJson.getString("desc")); throw TSDBError.createSQLException(resultJson.getInteger("code"), resultJson.getString("desc"));
...@@ -130,7 +130,7 @@ public class RestfulStatement extends AbstractStatement { ...@@ -130,7 +130,7 @@ public class RestfulStatement extends AbstractStatement {
if (!SqlSyntaxValidator.isValidForExecuteUpdate(sql)) if (!SqlSyntaxValidator.isValidForExecuteUpdate(sql))
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_FOR_EXECUTE_UPDATE, "not a valid sql for executeUpdate: " + sql); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_FOR_EXECUTE_UPDATE, "not a valid sql for executeUpdate: " + sql);
String result = HttpClientPoolUtil.execute(url, sql); String result = HttpClientPoolUtil.execute(url, sql, this.conn.getToken());
JSONObject jsonObject = JSON.parseObject(result); JSONObject jsonObject = JSON.parseObject(result);
if (jsonObject.getString("status").equals("error")) { if (jsonObject.getString("status").equals("error")) {
throw TSDBError.createSQLException(jsonObject.getInteger("code"), jsonObject.getString("desc")); throw TSDBError.createSQLException(jsonObject.getInteger("code"), jsonObject.getString("desc"));
......
package com.taosdata.jdbc.rs.enums;
public enum TimestampFormat {
STRING,
TIMESTAMP,
UTC
}
...@@ -16,43 +16,32 @@ import org.apache.http.protocol.HTTP; ...@@ -16,43 +16,32 @@ import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpContext; import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils; import org.apache.http.util.EntityUtils;
import java.nio.charset.Charset; import java.nio.charset.StandardCharsets;
public class HttpClientPoolUtil { public class HttpClientPoolUtil {
public static PoolingHttpClientConnectionManager cm = null;
public static CloseableHttpClient httpClient = null;
public static String token = "cm9vdDp0YW9zZGF0YQ==";
/**
* 默认content 类型
*/
private static final String DEFAULT_CONTENT_TYPE = "application/json"; private static final String DEFAULT_CONTENT_TYPE = "application/json";
/**
* 默认请求超时时间30s
*/
private static final int DEFAULT_TIME_OUT = 15000; private static final int DEFAULT_TIME_OUT = 15000;
private static final int count = 32; private static final int DEFAULT_MAX_PER_ROUTE = 32;
private static final int totalCount = 1000; private static final int DEFAULT_MAX_TOTAL = 1000;
private static final int Http_Default_Keep_Time = 15000; private static final int DEFAULT_HTTP_KEEP_TIME = 15000;
private static PoolingHttpClientConnectionManager connectionManager;
private static CloseableHttpClient httpClient;
/**
* 初始化连接池
*/
private static synchronized void initPools() { private static synchronized void initPools() {
if (httpClient == null) { if (httpClient == null) {
cm = new PoolingHttpClientConnectionManager(); connectionManager = new PoolingHttpClientConnectionManager();
cm.setDefaultMaxPerRoute(count); connectionManager.setDefaultMaxPerRoute(DEFAULT_MAX_PER_ROUTE);
cm.setMaxTotal(totalCount); connectionManager.setMaxTotal(DEFAULT_MAX_TOTAL);
httpClient = HttpClients.custom().setKeepAliveStrategy(defaultStrategy).setConnectionManager(cm).build(); httpClient = HttpClients.custom().setKeepAliveStrategy(DEFAULT_KEEP_ALIVE_STRATEGY).setConnectionManager(connectionManager).build();
} }
} }
/** private static ConnectionKeepAliveStrategy DEFAULT_KEEP_ALIVE_STRATEGY = (response, context) -> {
* Http connection keepAlive 设置
*/
private static ConnectionKeepAliveStrategy defaultStrategy = (response, context) -> {
HeaderElementIterator it = new BasicHeaderElementIterator(response.headerIterator(HTTP.CONN_KEEP_ALIVE)); HeaderElementIterator it = new BasicHeaderElementIterator(response.headerIterator(HTTP.CONN_KEEP_ALIVE));
int keepTime = Http_Default_Keep_Time * 1000; int keepTime = DEFAULT_HTTP_KEEP_TIME * 1000;
while (it.hasNext()) { while (it.hasNext()) {
HeaderElement headerElement = it.nextElement(); HeaderElement headerElement = it.nextElement();
String param = headerElement.getName(); String param = headerElement.getName();
...@@ -76,7 +65,7 @@ public class HttpClientPoolUtil { ...@@ -76,7 +65,7 @@ public class HttpClientPoolUtil {
* @param data 请求数据 * @param data 请求数据
* @return responseBody * @return responseBody
*/ */
public static String execute(String uri, String data) { public static String execute(String uri, String data, String token) {
long startTime = System.currentTimeMillis(); long startTime = System.currentTimeMillis();
HttpEntity httpEntity = null; HttpEntity httpEntity = null;
HttpEntityEnclosingRequestBase method = null; HttpEntityEnclosingRequestBase method = null;
...@@ -90,7 +79,7 @@ public class HttpClientPoolUtil { ...@@ -90,7 +79,7 @@ public class HttpClientPoolUtil {
method.setHeader("Connection", "keep-alive"); method.setHeader("Connection", "keep-alive");
method.setHeader("Authorization", "Taosd " + token); method.setHeader("Authorization", "Taosd " + token);
method.setEntity(new StringEntity(data, Charset.forName("UTF-8"))); method.setEntity(new StringEntity(data, StandardCharsets.UTF_8));
HttpContext context = HttpClientContext.create(); HttpContext context = HttpClientContext.create();
CloseableHttpResponse httpResponse = httpClient.execute(method, context); CloseableHttpResponse httpResponse = httpClient.execute(method, context);
httpEntity = httpResponse.getEntity(); httpEntity = httpResponse.getEntity();
...@@ -175,28 +164,18 @@ public class HttpClientPoolUtil { ...@@ -175,28 +164,18 @@ public class HttpClientPoolUtil {
httpEntity = httpResponse.getEntity(); httpEntity = httpResponse.getEntity();
if (httpEntity != null) { if (httpEntity != null) {
responseBody = EntityUtils.toString(httpEntity, "UTF-8"); responseBody = EntityUtils.toString(httpEntity, "UTF-8");
// logger.info("请求URL: " + uri + "+ 返回状态码:" + httpResponse.getStatusLine().getStatusCode());
} }
} catch (Exception e) { } catch (Exception e) {
if (method != null) { if (method != null) {
method.abort(); method.abort();
} }
e.printStackTrace(); e.printStackTrace();
// logger.error("execute get request exception, url:" + uri + ", exception:" + e.toString() + ",cost time(ms):"
// + (System.currentTimeMillis() - startTime));
System.out.println("log:调用 HttpClientPoolUtil execute get request exception, url:" + uri + ", exception:" + e.toString() + ",cost time(ms):"
+ (System.currentTimeMillis() - startTime));
} finally { } finally {
if (httpEntity != null) { if (httpEntity != null) {
try { try {
EntityUtils.consumeQuietly(httpEntity); EntityUtils.consumeQuietly(httpEntity);
} catch (Exception e) { } catch (Exception e) {
// e.printStackTrace(); new Exception("close response exception, url:" + uri + ", exception:" + e.toString() + ",cost time(ms):" + (System.currentTimeMillis() - startTime)).printStackTrace();
// logger.error("close response exception, url:" + uri + ", exception:" + e.toString()
// + ",cost time(ms):" + (System.currentTimeMillis() - startTime));
new Exception("close response exception, url:" + uri + ", exception:" + e.toString()
+ ",cost time(ms):" + (System.currentTimeMillis() - startTime))
.printStackTrace();
} }
} }
} }
......
...@@ -3,14 +3,13 @@ package com.taosdata.jdbc.utils; ...@@ -3,14 +3,13 @@ package com.taosdata.jdbc.utils;
import com.google.common.collect.Range; import com.google.common.collect.Range;
import com.google.common.collect.RangeSet; import com.google.common.collect.RangeSet;
import com.google.common.collect.TreeRangeSet; import com.google.common.collect.TreeRangeSet;
import com.taosdata.jdbc.enums.TimestampPrecision;
import java.nio.charset.Charset; import java.nio.charset.Charset;
import java.sql.Date; import java.sql.Date;
import java.sql.Time; import java.sql.Time;
import java.sql.Timestamp; import java.sql.Timestamp;
import java.time.LocalDate;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeFormatterBuilder; import java.time.format.DateTimeFormatterBuilder;
import java.time.format.DateTimeParseException; import java.time.format.DateTimeParseException;
...@@ -25,39 +24,52 @@ public class Utils { ...@@ -25,39 +24,52 @@ public class Utils {
private static Pattern ptn = Pattern.compile(".*?'"); private static Pattern ptn = Pattern.compile(".*?'");
private static final DateTimeFormatter formatter = new DateTimeFormatterBuilder() private static final DateTimeFormatter milliSecFormatter = new DateTimeFormatterBuilder().appendPattern("yyyy-MM-dd HH:mm:ss.SSS").toFormatter();
.appendPattern("yyyy-MM-dd HH:mm:ss.SSS").toFormatter(); private static final DateTimeFormatter microSecFormatter = new DateTimeFormatterBuilder().appendPattern("yyyy-MM-dd HH:mm:ss.SSSSSS").toFormatter();
private static final DateTimeFormatter formatter2 = new DateTimeFormatterBuilder() private static final DateTimeFormatter nanoSecFormatter = new DateTimeFormatterBuilder().appendPattern("yyyy-MM-dd HH:mm:ss.SSSSSSSSS").toFormatter();
.appendPattern("yyyy-MM-dd HH:mm:ss.SSSSSS").toFormatter();
public static Time parseTime(String timestampStr) throws DateTimeParseException { public static Time parseTime(String timestampStr) throws DateTimeParseException {
LocalTime time; LocalDateTime dateTime = parseLocalDateTime(timestampStr);
try { return dateTime != null ? Time.valueOf(dateTime.toLocalTime()) : null;
time = LocalTime.parse(timestampStr, formatter);
} catch (DateTimeParseException e) {
time = LocalTime.parse(timestampStr, formatter2);
}
return Time.valueOf(time);
} }
public static Date parseDate(String timestampStr) throws DateTimeParseException { public static Date parseDate(String timestampStr) {
LocalDate date; LocalDateTime dateTime = parseLocalDateTime(timestampStr);
try { return dateTime != null ? Date.valueOf(String.valueOf(dateTime)) : null;
date = LocalDate.parse(timestampStr, formatter);
} catch (DateTimeParseException e) {
date = LocalDate.parse(timestampStr, formatter2);
}
return Date.valueOf(date);
} }
public static Timestamp parseTimestamp(String timeStampStr) { public static Timestamp parseTimestamp(String timeStampStr) {
LocalDateTime dateTime; LocalDateTime dateTime = parseLocalDateTime(timeStampStr);
return dateTime != null ? Timestamp.valueOf(dateTime) : null;
}
private static LocalDateTime parseLocalDateTime(String timeStampStr) {
try { try {
dateTime = LocalDateTime.parse(timeStampStr, formatter); return parseMilliSecTimestamp(timeStampStr);
} catch (DateTimeParseException e) { } catch (DateTimeParseException e) {
dateTime = LocalDateTime.parse(timeStampStr, formatter2); try {
return parseMicroSecTimestamp(timeStampStr);
} catch (DateTimeParseException ee) {
try {
return parseNanoSecTimestamp(timeStampStr);
} catch (DateTimeParseException eee) {
eee.printStackTrace();
}
}
} }
return Timestamp.valueOf(dateTime); return null;
}
private static LocalDateTime parseMilliSecTimestamp(String timeStampStr) throws DateTimeParseException {
return LocalDateTime.parse(timeStampStr, milliSecFormatter);
}
private static LocalDateTime parseMicroSecTimestamp(String timeStampStr) throws DateTimeParseException {
return LocalDateTime.parse(timeStampStr, microSecFormatter);
}
private static LocalDateTime parseNanoSecTimestamp(String timeStampStr) throws DateTimeParseException {
return LocalDateTime.parse(timeStampStr, nanoSecFormatter);
} }
public static String escapeSingleQuota(String origin) { public static String escapeSingleQuota(String origin) {
...@@ -93,6 +105,8 @@ public class Utils { ...@@ -93,6 +105,8 @@ public class Utils {
} }
public static String getNativeSql(String rawSql, Object[] parameters) { public static String getNativeSql(String rawSql, Object[] parameters) {
if (parameters == null || !rawSql.contains("?"))
return rawSql;
// toLowerCase // toLowerCase
String preparedSql = rawSql.trim().toLowerCase(); String preparedSql = rawSql.trim().toLowerCase();
String[] clause = new String[]{"values\\s*\\(.*?\\)", "tags\\s*\\(.*?\\)", "where\\s*.*"}; String[] clause = new String[]{"values\\s*\\(.*?\\)", "tags\\s*\\(.*?\\)", "where\\s*.*"};
...@@ -167,13 +181,47 @@ public class Utils { ...@@ -167,13 +181,47 @@ public class Utils {
}).collect(Collectors.joining()); }).collect(Collectors.joining());
} }
public static String formatTimestamp(Timestamp timestamp) { public static String formatTimestamp(Timestamp timestamp) {
int nanos = timestamp.getNanos(); int nanos = timestamp.getNanos();
if (nanos % 1000000l != 0) if (nanos % 1000000l != 0)
return timestamp.toLocalDateTime().format(formatter2); return timestamp.toLocalDateTime().format(microSecFormatter);
return timestamp.toLocalDateTime().format(formatter); return timestamp.toLocalDateTime().format(milliSecFormatter);
}
public static TimestampPrecision guessTimestampPrecision(String value) {
if (isMilliSecFormat(value))
return TimestampPrecision.MS;
if (isMicroSecFormat(value))
return TimestampPrecision.US;
if (isNanoSecFormat(value))
return TimestampPrecision.NS;
return TimestampPrecision.UNKNOWN;
}
private static boolean isMilliSecFormat(String timestampStr) {
try {
milliSecFormatter.parse(timestampStr);
} catch (DateTimeParseException e) {
return false;
}
return true;
} }
private static boolean isMicroSecFormat(String timestampStr) {
try {
microSecFormatter.parse(timestampStr);
} catch (DateTimeParseException e) {
return false;
}
return true;
}
private static boolean isNanoSecFormat(String timestampStr) {
try {
nanoSecFormatter.parse(timestampStr);
} catch (DateTimeParseException e) {
return false;
}
return true;
}
} }
package com.taosdata.jdbc; package com.taosdata.jdbc;
import org.junit.Test; import org.junit.Test;
import static org.junit.Assert.*;
import java.sql.SQLException; import java.sql.SQLException;
import java.sql.SQLWarning; import java.sql.SQLWarning;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;
import java.lang.management.ThreadMXBean;
public class TSDBJNIConnectorTest { public class TSDBJNIConnectorTest {
private static TSDBResultSetRowData rowData; private static TSDBResultSetRowData rowData;
...@@ -14,17 +19,68 @@ public class TSDBJNIConnectorTest { ...@@ -14,17 +19,68 @@ public class TSDBJNIConnectorTest {
@Test @Test
public void test() { public void test() {
try { try {
try {
//change sleepSeconds when debugging with attach to process to find PID
int sleepSeconds = -1;
if (sleepSeconds>0) {
RuntimeMXBean runtimeBean = ManagementFactory.getRuntimeMXBean();
String jvmName = runtimeBean.getName();
long pid = Long.valueOf(jvmName.split("@")[0]);
System.out.println("JVM PID = " + pid);
Thread.sleep(sleepSeconds*1000);
}
}
catch (Exception e) {
e.printStackTrace();
}
// init // init
TSDBJNIConnector.init("/etc/taos/taos.cfg", null, null, null); TSDBJNIConnector.init("/etc/taos", null, null, null);
// connect // connect
TSDBJNIConnector connector = new TSDBJNIConnector(); TSDBJNIConnector connector = new TSDBJNIConnector();
connector.connect("127.0.0.1", 6030, "unsign_jni", "root", "taosdata"); connector.connect("127.0.0.1", 6030, null, "root", "taosdata");
// setup
String setupSqlStrs[] = {"create database if not exists d precision \"us\"",
"create table if not exists d.t(ts timestamp, f int)",
"create database if not exists d2",
"create table if not exists d2.t2(ts timestamp, f int)",
"insert into d.t values(now+100s, 100)",
"insert into d2.t2 values(now+200s, 200)"
};
for (String setupSqlStr : setupSqlStrs) {
long setupSql = connector.executeQuery(setupSqlStr);
assertEquals(0, connector.getResultTimePrecision(setupSql));
if (connector.isUpdateQuery(setupSql)) {
connector.freeResultSet(setupSql);
}
}
{
long sqlObj1 = connector.executeQuery("select * from d2.t2");
assertEquals(0, connector.getResultTimePrecision(sqlObj1));
List<ColumnMetaData> columnMetaDataList = new ArrayList<>();
int code = connector.getSchemaMetaData(sqlObj1, columnMetaDataList);
rowData = new TSDBResultSetRowData(columnMetaDataList.size());
assertTrue(next(connector, sqlObj1));
assertEquals(0, connector.getResultTimePrecision(sqlObj1));
connector.freeResultSet(sqlObj1);
}
// executeQuery // executeQuery
long pSql = connector.executeQuery("select * from unsign_jni.us_table"); long pSql = connector.executeQuery("select * from d.t");
if (connector.isUpdateQuery(pSql)) { if (connector.isUpdateQuery(pSql)) {
connector.freeResultSet(pSql); connector.freeResultSet(pSql);
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_WITH_EXECUTEQUERY); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_WITH_EXECUTEQUERY);
} }
assertEquals(1, connector.getResultTimePrecision(pSql));
// get schema // get schema
List<ColumnMetaData> columnMetaDataList = new ArrayList<>(); List<ColumnMetaData> columnMetaDataList = new ArrayList<>();
int code = connector.getSchemaMetaData(pSql, columnMetaDataList); int code = connector.getSchemaMetaData(pSql, columnMetaDataList);
...@@ -37,6 +93,8 @@ public class TSDBJNIConnectorTest { ...@@ -37,6 +93,8 @@ public class TSDBJNIConnectorTest {
if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) { if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} }
assertEquals(1, connector.getResultTimePrecision(pSql));
int columnSize = columnMetaDataList.size(); int columnSize = columnMetaDataList.size();
// print metadata // print metadata
for (int i = 0; i < columnSize; i++) { for (int i = 0; i < columnSize; i++) {
...@@ -45,9 +103,8 @@ public class TSDBJNIConnectorTest { ...@@ -45,9 +103,8 @@ public class TSDBJNIConnectorTest {
rowData = new TSDBResultSetRowData(columnSize); rowData = new TSDBResultSetRowData(columnSize);
// iterate resultSet // iterate resultSet
for (int i = 0; next(connector, pSql); i++) { for (int i = 0; next(connector, pSql); i++) {
// System.out.println("col[" + i + "] size: " + rowData.getColSize()); assertEquals(1, connector.getResultTimePrecision(pSql));
// rowData.getData().stream().forEach(col -> System.out.print(col + "\t")); System.out.println();
// System.out.println();
} }
// close resultSet // close resultSet
code = connector.freeResultSet(pSql); code = connector.freeResultSet(pSql);
...@@ -86,4 +143,4 @@ public class TSDBJNIConnectorTest { ...@@ -86,4 +143,4 @@ public class TSDBJNIConnectorTest {
} }
} }
} }
\ No newline at end of file
...@@ -5,7 +5,6 @@ import org.junit.*; ...@@ -5,7 +5,6 @@ import org.junit.*;
import java.io.IOException; import java.io.IOException;
import java.math.BigDecimal; import java.math.BigDecimal;
import java.sql.*; import java.sql.*;
import java.time.LocalTime;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Random; import java.util.Random;
...@@ -15,6 +14,7 @@ public class TSDBPreparedStatementTest { ...@@ -15,6 +14,7 @@ public class TSDBPreparedStatementTest {
private static Connection conn; private static Connection conn;
private static final String sql_insert = "insert into t1 values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; private static final String sql_insert = "insert into t1 values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)";
private static final String sql_select = "select * from t1 where ts >= ? and ts < ? and f1 >= ?"; private static final String sql_select = "select * from t1 where ts >= ? and ts < ? and f1 >= ?";
private static final String dbname = "test_pstmt_jni";
private PreparedStatement pstmt_insert; private PreparedStatement pstmt_insert;
private PreparedStatement pstmt_select; private PreparedStatement pstmt_select;
...@@ -299,53 +299,53 @@ public class TSDBPreparedStatementTest { ...@@ -299,53 +299,53 @@ public class TSDBPreparedStatementTest {
} }
@Test @Test
public void executeTest() throws SQLException { public void executeTest() throws SQLException {
Statement stmt = conn.createStatement(); Statement stmt = conn.createStatement();
int numOfRows = 1000; int numOfRows = 1000;
for (int loop = 0; loop < 10; loop++){ for (int loop = 0; loop < 10; loop++) {
stmt.execute("drop table if exists weather_test"); stmt.execute("drop table if exists weather_test");
stmt.execute("create table weather_test(ts timestamp, f1 nchar(4), f2 float, f3 double, f4 timestamp, f5 int, f6 bool, f7 binary(10))"); stmt.execute("create table weather_test(ts timestamp, f1 nchar(4), f2 float, f3 double, f4 timestamp, f5 int, f6 bool, f7 binary(10))");
TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? values(?, ?, ?, ?, ?, ?, ?, ?)"); TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? values(?, ?, ?, ?, ?, ?, ?, ?)");
Random r = new Random(); Random r = new Random();
s.setTableName("weather_test"); s.setTableName("weather_test");
ArrayList<Long> ts = new ArrayList<Long>(); ArrayList<Long> ts = new ArrayList<Long>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
ts.add(System.currentTimeMillis() + i); ts.add(System.currentTimeMillis() + i);
} }
s.setTimestamp(0, ts); s.setTimestamp(0, ts);
int random = 10 + r.nextInt(5); int random = 10 + r.nextInt(5);
ArrayList<String> s2 = new ArrayList<String>(); ArrayList<String> s2 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s2.add(null); s2.add(null);
}else{ } else {
s2.add("分支" + i % 4); s2.add("分支" + i % 4);
} }
} }
s.setNString(1, s2, 4); s.setNString(1, s2, 4);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<Float> s3 = new ArrayList<Float>(); ArrayList<Float> s3 = new ArrayList<Float>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s3.add(null); s3.add(null);
}else{ } else {
s3.add(r.nextFloat()); s3.add(r.nextFloat());
} }
} }
s.setFloat(2, s3); s.setFloat(2, s3);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<Double> s4 = new ArrayList<Double>(); ArrayList<Double> s4 = new ArrayList<Double>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s4.add(null); s4.add(null);
}else{ } else {
s4.add(r.nextDouble()); s4.add(r.nextDouble());
} }
} }
...@@ -353,47 +353,47 @@ public class TSDBPreparedStatementTest { ...@@ -353,47 +353,47 @@ public class TSDBPreparedStatementTest {
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<Long> ts2 = new ArrayList<Long>(); ArrayList<Long> ts2 = new ArrayList<Long>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
ts2.add(null); ts2.add(null);
}else{ } else {
ts2.add(System.currentTimeMillis() + i); ts2.add(System.currentTimeMillis() + i);
} }
} }
s.setTimestamp(4, ts2); s.setTimestamp(4, ts2);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<Integer> vals = new ArrayList<>(); ArrayList<Integer> vals = new ArrayList<>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
vals.add(null); vals.add(null);
}else{ } else {
vals.add(r.nextInt()); vals.add(r.nextInt());
} }
} }
s.setInt(5, vals); s.setInt(5, vals);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<Boolean> sb = new ArrayList<>(); ArrayList<Boolean> sb = new ArrayList<>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
sb.add(null); sb.add(null);
}else{ } else {
sb.add(i % 2 == 0 ? true : false); sb.add(i % 2 == 0 ? true : false);
} }
} }
s.setBoolean(6, sb); s.setBoolean(6, sb);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<String> s5 = new ArrayList<String>(); ArrayList<String> s5 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s5.add(null); s5.add(null);
}else{ } else {
s5.add("test" + i % 10); s5.add("test" + i % 10);
} }
} }
s.setString(7, s5, 10); s.setString(7, s5, 10);
s.columnDataAddBatch(); s.columnDataAddBatch();
s.columnDataExecuteBatch(); s.columnDataExecuteBatch();
...@@ -403,54 +403,54 @@ public class TSDBPreparedStatementTest { ...@@ -403,54 +403,54 @@ public class TSDBPreparedStatementTest {
PreparedStatement statement = conn.prepareStatement(sql); PreparedStatement statement = conn.prepareStatement(sql);
ResultSet rs = statement.executeQuery(); ResultSet rs = statement.executeQuery();
int rows = 0; int rows = 0;
while(rs.next()) { while (rs.next()) {
rows++; rows++;
} }
Assert.assertEquals(numOfRows, rows); Assert.assertEquals(numOfRows, rows);
} }
} }
@Test @Test
public void bindDataSelectColumnTest() throws SQLException { public void bindDataSelectColumnTest() throws SQLException {
Statement stmt = conn.createStatement(); Statement stmt = conn.createStatement();
int numOfRows = 1000; int numOfRows = 1000;
for (int loop = 0; loop < 10; loop++){ for (int loop = 0; loop < 10; loop++) {
stmt.execute("drop table if exists weather_test"); stmt.execute("drop table if exists weather_test");
stmt.execute("create table weather_test(ts timestamp, f1 nchar(4), f2 float, f3 double, f4 timestamp, f5 int, f6 bool, f7 binary(10))"); stmt.execute("create table weather_test(ts timestamp, f1 nchar(4), f2 float, f3 double, f4 timestamp, f5 int, f6 bool, f7 binary(10))");
TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? (ts, f1, f7) values(?, ?, ?)"); TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? (ts, f1, f7) values(?, ?, ?)");
Random r = new Random(); Random r = new Random();
s.setTableName("weather_test"); s.setTableName("weather_test");
ArrayList<Long> ts = new ArrayList<Long>(); ArrayList<Long> ts = new ArrayList<Long>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
ts.add(System.currentTimeMillis() + i); ts.add(System.currentTimeMillis() + i);
} }
s.setTimestamp(0, ts); s.setTimestamp(0, ts);
int random = 10 + r.nextInt(5); int random = 10 + r.nextInt(5);
ArrayList<String> s2 = new ArrayList<String>(); ArrayList<String> s2 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s2.add(null); s2.add(null);
}else{ } else {
s2.add("分支" + i % 4); s2.add("分支" + i % 4);
} }
} }
s.setNString(1, s2, 4); s.setNString(1, s2, 4);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<String> s3 = new ArrayList<String>(); ArrayList<String> s3 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
if(i % random == 0) { if (i % random == 0) {
s3.add(null); s3.add(null);
}else{ } else {
s3.add("test" + i % 10); s3.add("test" + i % 10);
} }
} }
s.setString(2, s3, 10); s.setString(2, s3, 10);
s.columnDataAddBatch(); s.columnDataAddBatch();
s.columnDataExecuteBatch(); s.columnDataExecuteBatch();
...@@ -460,30 +460,30 @@ public class TSDBPreparedStatementTest { ...@@ -460,30 +460,30 @@ public class TSDBPreparedStatementTest {
PreparedStatement statement = conn.prepareStatement(sql); PreparedStatement statement = conn.prepareStatement(sql);
ResultSet rs = statement.executeQuery(); ResultSet rs = statement.executeQuery();
int rows = 0; int rows = 0;
while(rs.next()) { while (rs.next()) {
rows++; rows++;
} }
Assert.assertEquals(numOfRows, rows); Assert.assertEquals(numOfRows, rows);
} }
} }
@Test @Test
public void bindDataWithSingleTagTest() throws SQLException { public void bindDataWithSingleTagTest() throws SQLException {
Statement stmt = conn.createStatement(); Statement stmt = conn.createStatement();
String types[] = new String[] {"tinyint", "smallint", "int", "bigint", "bool", "float", "double", "binary(10)", "nchar(10)"}; String types[] = new String[]{"tinyint", "smallint", "int", "bigint", "bool", "float", "double", "binary(10)", "nchar(10)"};
for (String type : types) { for (String type : types) {
stmt.execute("drop table if exists weather_test"); stmt.execute("drop table if exists weather_test");
stmt.execute("create table weather_test(ts timestamp, f1 nchar(10), f2 binary(10)) tags (t " + type + ")"); stmt.execute("create table weather_test(ts timestamp, f1 nchar(10), f2 binary(10)) tags (t " + type + ")");
int numOfRows = 1; int numOfRows = 1;
TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? using weather_test tags(?) values(?, ?, ?)"); TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? using weather_test tags(?) values(?, ?, ?)");
Random r = new Random(); Random r = new Random();
s.setTableName("w1"); s.setTableName("w1");
switch(type) { switch (type) {
case "tinyint": case "tinyint":
case "smallint": case "smallint":
case "int": case "int":
...@@ -508,37 +508,37 @@ public class TSDBPreparedStatementTest { ...@@ -508,37 +508,37 @@ public class TSDBPreparedStatementTest {
default: default:
break; break;
} }
ArrayList<Long> ts = new ArrayList<Long>(); ArrayList<Long> ts = new ArrayList<Long>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
ts.add(System.currentTimeMillis() + i); ts.add(System.currentTimeMillis() + i);
} }
s.setTimestamp(0, ts); s.setTimestamp(0, ts);
int random = 10 + r.nextInt(5); int random = 10 + r.nextInt(5);
ArrayList<String> s2 = new ArrayList<String>(); ArrayList<String> s2 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
s2.add("分支" + i % 4); s2.add("分支" + i % 4);
} }
s.setNString(1, s2, 10); s.setNString(1, s2, 10);
random = 10 + r.nextInt(5); random = 10 + r.nextInt(5);
ArrayList<String> s3 = new ArrayList<String>(); ArrayList<String> s3 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
s3.add("test" + i % 4); s3.add("test" + i % 4);
} }
s.setString(2, s3, 10); s.setString(2, s3, 10);
s.columnDataAddBatch(); s.columnDataAddBatch();
s.columnDataExecuteBatch(); s.columnDataExecuteBatch();
s.columnDataCloseBatch(); s.columnDataCloseBatch();
String sql = "select * from weather_test"; String sql = "select * from weather_test";
PreparedStatement statement = conn.prepareStatement(sql); PreparedStatement statement = conn.prepareStatement(sql);
ResultSet rs = statement.executeQuery(); ResultSet rs = statement.executeQuery();
int rows = 0; int rows = 0;
while(rs.next()) { while (rs.next()) {
rows++; rows++;
} }
Assert.assertEquals(numOfRows, rows); Assert.assertEquals(numOfRows, rows);
...@@ -547,32 +547,32 @@ public class TSDBPreparedStatementTest { ...@@ -547,32 +547,32 @@ public class TSDBPreparedStatementTest {
@Test @Test
public void bindDataWithMultipleTagsTest() throws SQLException { public void bindDataWithMultipleTagsTest() throws SQLException {
Statement stmt = conn.createStatement(); Statement stmt = conn.createStatement();
stmt.execute("drop table if exists weather_test"); stmt.execute("drop table if exists weather_test");
stmt.execute("create table weather_test(ts timestamp, f1 nchar(10), f2 binary(10)) tags (t1 int, t2 binary(10))"); stmt.execute("create table weather_test(ts timestamp, f1 nchar(10), f2 binary(10)) tags (t1 int, t2 binary(10))");
int numOfRows = 1; int numOfRows = 1;
TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? using weather_test tags(?,?) (ts, f2) values(?, ?)"); TSDBPreparedStatement s = (TSDBPreparedStatement) conn.prepareStatement("insert into ? using weather_test tags(?,?) (ts, f2) values(?, ?)");
s.setTableName("w2"); s.setTableName("w2");
s.setTagInt(0, 1); s.setTagInt(0, 1);
s.setTagString(1, "test"); s.setTagString(1, "test");
ArrayList<Long> ts = new ArrayList<Long>(); ArrayList<Long> ts = new ArrayList<Long>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
ts.add(System.currentTimeMillis() + i); ts.add(System.currentTimeMillis() + i);
} }
s.setTimestamp(0, ts); s.setTimestamp(0, ts);
ArrayList<String> s2 = new ArrayList<String>(); ArrayList<String> s2 = new ArrayList<String>();
for(int i = 0; i < numOfRows; i++) { for (int i = 0; i < numOfRows; i++) {
s2.add("test" + i % 4); s2.add("test" + i % 4);
} }
s.setString(1, s2, 10); s.setString(1, s2, 10);
s.columnDataAddBatch(); s.columnDataAddBatch();
s.columnDataExecuteBatch(); s.columnDataExecuteBatch();
s.columnDataCloseBatch(); s.columnDataCloseBatch();
...@@ -581,21 +581,20 @@ public class TSDBPreparedStatementTest { ...@@ -581,21 +581,20 @@ public class TSDBPreparedStatementTest {
PreparedStatement statement = conn.prepareStatement(sql); PreparedStatement statement = conn.prepareStatement(sql);
ResultSet rs = statement.executeQuery(); ResultSet rs = statement.executeQuery();
int rows = 0; int rows = 0;
while(rs.next()) { while (rs.next()) {
rows++; rows++;
} }
Assert.assertEquals(numOfRows, rows); Assert.assertEquals(numOfRows, rows);
} }
@Test @Test(expected = SQLException.class)
public void createTwoSameDbTest() throws SQLException { public void createTwoSameDbTest() throws SQLException {
Statement stmt = conn.createStatement(); // when
Statement stmt = conn.createStatement();
stmt.execute("create database dbtest");
stmt.execute("create database dbtest"); stmt.execute("create database dbtest");
Assert.assertThrows(SQLException.class, () -> stmt.execute("create database dbtest"));
} }
@Test @Test
public void setBoolean() throws SQLException { public void setBoolean() throws SQLException {
// given // given
...@@ -1097,9 +1096,9 @@ public class TSDBPreparedStatementTest { ...@@ -1097,9 +1096,9 @@ public class TSDBPreparedStatementTest {
try { try {
conn = DriverManager.getConnection("jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata"); conn = DriverManager.getConnection("jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata");
try (Statement stmt = conn.createStatement()) { try (Statement stmt = conn.createStatement()) {
stmt.execute("drop database if exists test_pstmt_jni"); stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists test_pstmt_jni"); stmt.execute("create database if not exists " + dbname);
stmt.execute("use test_pstmt_jni"); stmt.execute("use " + dbname);
} }
} catch (SQLException e) { } catch (SQLException e) {
e.printStackTrace(); e.printStackTrace();
...@@ -1109,6 +1108,9 @@ public class TSDBPreparedStatementTest { ...@@ -1109,6 +1108,9 @@ public class TSDBPreparedStatementTest {
@AfterClass @AfterClass
public static void afterClass() { public static void afterClass() {
try { try {
Statement statement = conn.createStatement();
statement.execute("drop database if exists " + dbname);
statement.close();
if (conn != null) if (conn != null)
conn.close(); conn.close();
} catch (SQLException e) { } catch (SQLException e) {
......
...@@ -28,9 +28,7 @@ public class BatchInsertTest { ...@@ -28,9 +28,7 @@ public class BatchInsertTest {
@Before @Before
public void before() { public void before() {
try { try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties(); Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
...@@ -44,7 +42,7 @@ public class BatchInsertTest { ...@@ -44,7 +42,7 @@ public class BatchInsertTest {
String createTableSql = "create table " + stbName + "(ts timestamp, f1 int, f2 int, f3 int) tags(areaid int, loc binary(20))"; String createTableSql = "create table " + stbName + "(ts timestamp, f1 int, f2 int, f3 int) tags(areaid int, loc binary(20))";
statement.executeUpdate(createTableSql); statement.executeUpdate(createTableSql);
// create tables // create tables
for(int i = 0; i < numOfTables; i++) { for (int i = 0; i < numOfTables; i++) {
String loc = i % 2 == 0 ? "beijing" : "shanghai"; String loc = i % 2 == 0 ? "beijing" : "shanghai";
String createSubTalbesSql = "create table " + tablePrefix + i + " using " + stbName + " tags(" + i + ", '" + loc + "')"; String createSubTalbesSql = "create table " + tablePrefix + i + " using " + stbName + " tags(" + i + ", '" + loc + "')";
statement.executeUpdate(createSubTalbesSql); statement.executeUpdate(createSubTalbesSql);
......
package com.taosdata.jdbc.cases;
import com.taosdata.jdbc.TSDBDriver;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
import java.util.Properties;
public class ConnectMultiTaosdByRestfulWithDifferentTokenTest {
private static String host1 = "192.168.17.156";
private static String user1 = "root";
private static String password1 = "tqueue";
private Connection conn1;
private static String host2 = "192.168.17.82";
private static String user2 = "root";
private static String password2 = "taosdata";
private Connection conn2;
@Test
public void test() {
//when
executeSelectStatus(conn1);
executeSelectStatus(conn2);
executeSelectStatus(conn1);
}
private void executeSelectStatus(Connection connection) {
try (Statement stmt = connection.createStatement()) {
ResultSet rs = stmt.executeQuery("select server_status()");
ResultSetMetaData meta = rs.getMetaData();
while (rs.next()) {
for (int i = 1; i <= meta.getColumnCount(); i++) {
System.out.println(meta.getColumnLabel(i) + ": " + rs.getString(i));
}
}
} catch (SQLException e) {
e.printStackTrace();
}
}
@Before
public void before() {
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
String url1 = "jdbc:TAOS-RS://" + host1 + ":6041/?user=" + user1 + "&password=" + password1;
String url2 = "jdbc:TAOS-RS://" + host2 + ":6041/?user=" + user2 + "&password=" + password2;
try {
conn1 = DriverManager.getConnection(url1, properties);
conn2 = DriverManager.getConnection(url2, properties);
} catch (SQLException e) {
e.printStackTrace();
}
}
}
...@@ -7,56 +7,64 @@ import org.junit.*; ...@@ -7,56 +7,64 @@ import org.junit.*;
import java.sql.*; import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class TD4174Test { public class DoubleQuoteInSqlTest {
private Connection conn;
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
private static final String dbname = "td4174";
private Connection conn;
@Test @Test
public void test() { public void test() {
// given
long ts = System.currentTimeMillis(); long ts = System.currentTimeMillis();
JSONObject value = new JSONObject();
value.put("name", "John Smith");
value.put("age", 20);
// when
int ret = 0;
try (PreparedStatement pstmt = conn.prepareStatement("insert into weather values(" + ts + ", ?)")) { try (PreparedStatement pstmt = conn.prepareStatement("insert into weather values(" + ts + ", ?)")) {
JSONObject value = new JSONObject();
value.put("name", "John Smith");
value.put("age", 20);
Assert.assertEquals("{\"name\":\"John Smith\",\"age\":20}",value.toJSONString());
pstmt.setString(1, value.toJSONString()); pstmt.setString(1, value.toJSONString());
ret = pstmt.executeUpdate();
int ret = pstmt.executeUpdate();
Assert.assertEquals(1, ret);
} catch (SQLException e) { } catch (SQLException e) {
e.printStackTrace(); e.printStackTrace();
} }
}
public static void main(String[] args) { // then
JSONObject value = new JSONObject(); Assert.assertEquals("{\"name\":\"John Smith\",\"age\":20}", value.toJSONString());
value.put("name", "John Smith"); Assert.assertEquals(1, ret);
value.put("age", 20);
System.out.println(value.toJSONString());
} }
@Before @Before
public void before() throws SQLException { public void before() {
String url = "jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata"; String url = "jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata";
Properties properties = new Properties(); Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
conn = DriverManager.getConnection(url, properties); try {
try (Statement stmt = conn.createStatement()) { conn = DriverManager.getConnection(url, properties);
stmt.execute("drop database if exists td4174"); Statement stmt = conn.createStatement();
stmt.execute("create database if not exists td4174"); stmt.execute("drop database if exists " + dbname);
stmt.execute("use td4174"); stmt.execute("create database if not exists " + dbname);
stmt.execute("use " + dbname);
stmt.execute("create table weather(ts timestamp, text binary(64))"); stmt.execute("create table weather(ts timestamp, text binary(64))");
} catch (SQLException e) {
e.printStackTrace();
} }
} }
@After @After
public void after() throws SQLException { public void after() {
if (conn != null) try {
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.close();
conn.close(); conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
} }
} }
...@@ -10,7 +10,7 @@ import org.junit.Test; ...@@ -10,7 +10,7 @@ import org.junit.Test;
import java.sql.*; import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class TwoTypeTimestampPercisionInJniTest { public class MicroSecondPrecisionJNITest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
private static final String ms_timestamp_db = "ms_precision_test"; private static final String ms_timestamp_db = "ms_precision_test";
......
...@@ -10,10 +10,9 @@ import org.junit.Test; ...@@ -10,10 +10,9 @@ import org.junit.Test;
import java.sql.*; import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class TwoTypeTimestampPercisionInRestfulTest { public class MicroSecondPrecisionRestfulTest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
private static final String ms_timestamp_db = "ms_precision_test"; private static final String ms_timestamp_db = "ms_precision_test";
private static final String us_timestamp_db = "us_precision_test"; private static final String us_timestamp_db = "us_precision_test";
private static final long timestamp1 = System.currentTimeMillis(); private static final long timestamp1 = System.currentTimeMillis();
......
package com.taosdata.jdbc.cases;
import org.junit.Assert;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import java.sql.*;
import java.time.Instant;
import java.util.Random;
public class NanoSecondTimestampJNITest {
private static final String host = "127.0.0.1";
private static final String dbname = "nano_sec_test";
private static final Random random = new Random(System.currentTimeMillis());
private static Connection conn;
@Test
public void insertUsingLongValue() {
// given
long ms = System.currentTimeMillis();
long ns = ms * 1000_000 + random.nextInt(1000_000);
// when
int ret = 0;
try (Statement stmt = conn.createStatement()) {
ret = stmt.executeUpdate("insert into weather(ts, temperature, humidity) values(" + ns + ", 12.3, 4)");
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void insertUsingStringValue() {
// given
// when
int ret = 0;
try (Statement stmt = conn.createStatement()) {
ret = stmt.executeUpdate("insert into weather(ts, temperature, humidity) values('2021-01-01 12:00:00.123456789', 12.3, 4)");
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void insertUsingTimestampValue() {
// given
long epochSec = System.currentTimeMillis() / 1000;
long nanoAdjustment = random.nextInt(1000_000_000);
Timestamp ts = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
// when
int ret = 0;
String sql = "insert into weather(ts, temperature, humidity) values( ?, ?, ?)";
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
pstmt.setTimestamp(1, ts);
pstmt.setFloat(2, 12.34f);
pstmt.setInt(3, 55);
ret = pstmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void selectUsingLongValue() throws SQLException {
// given
long ms = System.currentTimeMillis();
long ns = ms * 1000_000L + random.nextInt(1000_000);
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
stmt.executeUpdate("insert into weather(ts, temperature, humidity) values(" + ns + ", 12.3, 4)");
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
long actual = rs.getLong(1);
Assert.assertEquals(ms, actual);
actual = rs.getLong("ts");
Assert.assertEquals(ms, actual);
}
@Test
public void selectUsingStringValue() throws SQLException {
// given
String timestampStr = "2021-01-01 12:00:00.123456789";
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
stmt.executeUpdate("insert into weather(ts, temperature, humidity) values('" + timestampStr + "', 12.3, 4)");
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
String actual = rs.getString(1);
Assert.assertEquals(timestampStr, actual);
actual = rs.getString("ts");
Assert.assertEquals(timestampStr, actual);
}
@Test
public void selectUsingTimestampValue() throws SQLException {
// given
long timeMillis = System.currentTimeMillis();
long epochSec = timeMillis / 1000;
long nanoAdjustment = (timeMillis % 1000) * 1000_000L + random.nextInt(1000_000);
Timestamp ts = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
// insert one row
String sql = "insert into weather(ts, temperature, humidity) values( ?, ?, ?)";
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
pstmt.setTimestamp(1, ts);
pstmt.setFloat(2, 12.34f);
pstmt.setInt(3, 55);
pstmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Timestamp actual = rs.getTimestamp(1);
Assert.assertEquals(ts, actual);
actual = rs.getTimestamp("ts");
Assert.assertEquals(ts, actual);
Assert.assertEquals(timeMillis, actual.getTime());
Assert.assertEquals(nanoAdjustment, actual.getNanos());
}
@Before
public void before() {
try (Statement stmt = conn.createStatement()) {
stmt.execute("drop table if exists weather");
stmt.execute("create table weather(ts timestamp, temperature float, humidity int)");
} catch (SQLException e) {
e.printStackTrace();
}
}
@BeforeClass
public static void beforeClass() {
final String url = "jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata";
try {
conn = DriverManager.getConnection(url);
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists " + dbname + " precision 'ns'");
stmt.execute("use " + dbname);
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.jdbc.cases;
import org.junit.Assert;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import java.sql.*;
import java.time.Instant;
import java.util.Random;
public class NanoSecondTimestampRestfulTest {
private static final String host = "127.0.0.1";
private static final String dbname = "nano_sec_test";
private static final Random random = new Random(System.currentTimeMillis());
private static Connection conn;
@Test
public void insertUsingLongValue() {
// given
long ms = System.currentTimeMillis();
long ns = ms * 1000_000 + random.nextInt(1000_000);
// when
int ret = 0;
try (Statement stmt = conn.createStatement()) {
ret = stmt.executeUpdate("insert into weather(ts, temperature, humidity) values(" + ns + ", 12.3, 4)");
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void insertUsingStringValue() {
// given
// when
int ret = 0;
try (Statement stmt = conn.createStatement()) {
ret = stmt.executeUpdate("insert into weather(ts, temperature, humidity) values('2021-01-01 12:00:00.123456789', 12.3, 4)");
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void insertUsingTimestampValue() {
// given
long epochSec = System.currentTimeMillis() / 1000;
long nanoAdjustment = random.nextInt(1000_000_000);
Timestamp ts = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
// when
int ret = 0;
String sql = "insert into weather(ts, temperature, humidity) values( ?, ?, ?)";
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
pstmt.setTimestamp(1, ts);
pstmt.setFloat(2, 12.34f);
pstmt.setInt(3, 55);
ret = pstmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(1, ret);
}
@Test
public void selectUsingLongValue() throws SQLException {
// given
long ms = System.currentTimeMillis();
long ns = ms * 1000_000L + random.nextInt(1000_000);
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
stmt.executeUpdate("insert into weather(ts, temperature, humidity) values(" + ns + ", 12.3, 4)");
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
long actual = rs.getLong(1);
Assert.assertEquals(ms, actual);
actual = rs.getLong("ts");
Assert.assertEquals(ms, actual);
}
@Test
public void selectUsingStringValue() throws SQLException {
// given
String timestampStr = "2021-01-01 12:00:00.123456789";
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
stmt.executeUpdate("insert into weather(ts, temperature, humidity) values('" + timestampStr + "', 12.3, 4)");
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
String actual = rs.getString(1);
Assert.assertEquals(timestampStr, actual);
actual = rs.getString("ts");
Assert.assertEquals(timestampStr, actual);
}
@Test
public void selectUsingTimestampValue() throws SQLException {
// given
long timeMillis = System.currentTimeMillis();
long epochSec = timeMillis / 1000;
long nanoAdjustment = (timeMillis % 1000) * 1000_000L + random.nextInt(1000_000);
Timestamp ts = Timestamp.from(Instant.ofEpochSecond(epochSec, nanoAdjustment));
// insert one row
String sql = "insert into weather(ts, temperature, humidity) values( ?, ?, ?)";
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
pstmt.setTimestamp(1, ts);
pstmt.setFloat(2, 12.34f);
pstmt.setInt(3, 55);
pstmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
// when
ResultSet rs = null;
try (Statement stmt = conn.createStatement()) {
rs = stmt.executeQuery("select * from weather");
rs.next();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Timestamp actual = rs.getTimestamp(1);
Assert.assertEquals(ts, actual);
actual = rs.getTimestamp("ts");
Assert.assertEquals(ts, actual);
Assert.assertEquals(timeMillis, actual.getTime());
Assert.assertEquals(nanoAdjustment, actual.getNanos());
}
@Before
public void before() {
try (Statement stmt = conn.createStatement()) {
stmt.execute("drop table if exists weather");
stmt.execute("create table weather(ts timestamp, temperature float, humidity int)");
} catch (SQLException e) {
e.printStackTrace();
}
}
@BeforeClass
public static void beforeClass() {
final String url = "jdbc:TAOS-RS://" + host + ":6041/?user=root&password=taosdata";
try {
conn = DriverManager.getConnection(url);
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists " + dbname + " precision 'ns'");
stmt.execute("use " + dbname);
} catch (SQLException e) {
e.printStackTrace();
}
}
}
...@@ -6,7 +6,7 @@ import org.junit.Test; ...@@ -6,7 +6,7 @@ import org.junit.Test;
import java.sql.*; import java.sql.*;
public class NullValueInResultSetForJdbcJniTest { public class NullValueInResultSetJNITest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
Connection conn; Connection conn;
......
...@@ -6,7 +6,7 @@ import org.junit.Test; ...@@ -6,7 +6,7 @@ import org.junit.Test;
import java.sql.*; import java.sql.*;
public class NullValueInResultSetForJdbcRestfulTest { public class NullValueInResultSetRestfulTest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
Connection conn; Connection conn;
......
...@@ -7,7 +7,7 @@ import org.junit.*; ...@@ -7,7 +7,7 @@ import org.junit.*;
import java.sql.*; import java.sql.*;
import java.util.Properties; import java.util.Properties;
public class TD3841Test { public class NullValueInResultSetTest {
private static final String host = "127.0.0.1"; private static final String host = "127.0.0.1";
private static Properties properties; private static Properties properties;
private static Connection conn_restful; private static Connection conn_restful;
......
package com.taosdata.jdbc.cases;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
import java.util.List;
import java.util.Random;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
public class PreparedStatementBatchInsertJNITest {
private static final String host = "127.0.0.1";
private static final String dbname = "td4668";
private final Random random = new Random(System.currentTimeMillis());
private Connection conn;
@Test
public void test() {
// given
long ts = System.currentTimeMillis();
List<Object[]> rows = IntStream.range(0, 10).mapToObj(i -> {
Object[] row = new Object[6];
final String groupId = String.format("%02d", random.nextInt(100));
// table name (d + groupId)组合
row[0] = "d" + groupId;
// tag
row[1] = groupId;
// ts
row[2] = ts + i;
// current 电流
row[3] = random.nextFloat();
// voltage 电压
row[4] = Math.random() > 0.5 ? 220 : 380;
// phase 相位
row[5] = random.nextInt(10);
return row;
}).collect(Collectors.toList());
final String sql = "INSERT INTO ? (TS,CURRENT,VOLTAGE,PHASE) USING METERS TAGS (?) VALUES (?,?,?,?)";
// when
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
for (Object[] row : rows) {
for (int i = 0; i < row.length; i++) {
pstmt.setObject(i + 1, row[i]);
}
pstmt.addBatch();
}
pstmt.executeBatch();
} catch (SQLException e) {
e.printStackTrace();
Assert.fail();
}
// then
try (Statement stmt = conn.createStatement()) {
ResultSet rs = stmt.executeQuery("select * from meters");
int count = 0;
while (rs.next()) {
count++;
}
Assert.assertEquals(10, count);
} catch (SQLException e) {
e.printStackTrace();
}
}
@Before
public void before() {
try {
conn = DriverManager.getConnection("jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata");
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists " + dbname);
stmt.execute("use " + dbname);
stmt.execute("create table meters(ts timestamp, current float, voltage int, phase int) tags(groupId int)");
} catch (SQLException e) {
e.printStackTrace();
}
}
@After
public void after() {
try {
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.close();
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.jdbc.cases;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
import java.util.List;
import java.util.Random;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
public class PreparedStatementBatchInsertRestfulTest {
private static final String host = "127.0.0.1";
private static final String dbname = "td4668";
private final Random random = new Random(System.currentTimeMillis());
private Connection conn;
@Test
public void test() {
// given
long ts = System.currentTimeMillis();
List<Object[]> rows = IntStream.range(0, 10).mapToObj(i -> {
Object[] row = new Object[6];
final String groupId = String.format("%02d", random.nextInt(100));
// table name (d + groupId)组合
row[0] = "d" + groupId;
// tag
row[1] = groupId;
// ts
row[2] = ts + i;
// current 电流
row[3] = random.nextFloat();
// voltage 电压
row[4] = Math.random() > 0.5 ? 220 : 380;
// phase 相位
row[5] = random.nextInt(10);
return row;
}).collect(Collectors.toList());
final String sql = "INSERT INTO ? (TS,CURRENT,VOLTAGE,PHASE) USING METERS TAGS (?) VALUES (?,?,?,?)";
// when
try (PreparedStatement pstmt = conn.prepareStatement(sql)) {
for (Object[] row : rows) {
for (int i = 0; i < row.length; i++) {
pstmt.setObject(i + 1, row[i]);
}
pstmt.addBatch();
}
pstmt.executeBatch();
} catch (SQLException e) {
e.printStackTrace();
Assert.fail();
}
// then
try (Statement stmt = conn.createStatement()) {
ResultSet rs = stmt.executeQuery("select * from meters");
int count = 0;
while (rs.next()) {
count++;
}
Assert.assertEquals(10, count);
} catch (SQLException e) {
e.printStackTrace();
}
}
@Before
public void before() {
try {
conn = DriverManager.getConnection("jdbc:TAOS-RS://" + host + ":6041/?user=root&password=taosdata");
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists " + dbname);
stmt.execute("use " + dbname);
stmt.execute("create table meters(ts timestamp, current float, voltage int, phase int) tags(groupId int)");
} catch (SQLException e) {
e.printStackTrace();
}
}
@After
public void after() {
try {
Statement stmt = conn.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.close();
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.jdbc.cases;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.sql.*;
public class ResultSetMetaShouldNotBeNullRestfulTest {
private static final String host = "127.0.0.1";
private static final String dbname = "td4745";
private Connection connection;
@Test
public void testExecuteQuery() {
// given
ResultSetMetaData metaData = null;
int columnCount = -1;
// when
try {
Statement statement = connection.createStatement();
metaData = statement.executeQuery("select * from weather").getMetaData();
columnCount = metaData.getColumnCount();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertNotNull(metaData);
Assert.assertEquals(0, columnCount);
}
@Test
public void testExecute() {
// given
ResultSetMetaData metaData = null;
int columnCount = -1;
boolean execute = false;
// when
try {
Statement statement = connection.createStatement();
execute = statement.execute("select * from weather");
metaData = statement.getResultSet().getMetaData();
columnCount = metaData.getColumnCount();
} catch (SQLException e) {
e.printStackTrace();
}
// then
Assert.assertEquals(true, execute);
Assert.assertNotNull(metaData);
Assert.assertEquals(0, columnCount);
}
@Before
public void before() {
try {
connection = DriverManager.getConnection("jdbc:TAOS-RS://" + host + ":6041/?user=root&password=taosdata");
Statement stmt = connection.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.execute("create database if not exists " + dbname);
stmt.execute("use " + dbname);
stmt.execute("create table weather (ts timestamp, temperature float)");
stmt.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
@After
public void after() {
try {
Statement stmt = connection.createStatement();
stmt.execute("drop database if exists " + dbname);
stmt.close();
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.jdbc.cases;
import com.taosdata.jdbc.TSDBConnection;
import com.taosdata.jdbc.TSDBDriver;
import com.taosdata.jdbc.TSDBResultSet;
import com.taosdata.jdbc.TSDBSubscribe;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.sql.DriverManager;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
public class TD4144Test {
private static TSDBConnection connection;
private static final String host = "127.0.0.1";
private static final String topic = "topic-meter-current-bg-10";
private static final String sql = "select * from meters where current > 10";
private static final String sql2 = "select * from meters where ts >= '2020-08-15 12:20:00.000'";
@Test
public void test() throws SQLException {
TSDBSubscribe subscribe = null;
TSDBResultSet res = null;
boolean hasNext = false;
try {
subscribe = connection.subscribe(topic, sql, false);
int count = 0;
while (true) {
// 等待1秒,避免频繁调用 consume,给服务端造成压力
TimeUnit.SECONDS.sleep(1);
if (res == null) {
// 消费数据
res = subscribe.consume();
hasNext = res.next();
}
if (res == null) {
continue;
}
ResultSetMetaData metaData = res.getMetaData();
int number = 0;
while (hasNext) {
int columnCount = metaData.getColumnCount();
for (int i = 1; i <= columnCount; i++) {
System.out.print(metaData.getColumnLabel(i) + ": " + res.getString(i) + "\t");
}
System.out.println();
count++;
number++;
hasNext = res.next();
if (!hasNext) {
res.close();
res = null;
System.out.println("rows: " + count);
}
if (hasNext == true && number >= 10) {
System.out.println("batch" + number);
break;
}
}
}
} catch (SQLException | InterruptedException throwables) {
throwables.printStackTrace();
} finally {
if (subscribe != null)
subscribe.close(true);
}
}
@BeforeClass
public static void beforeClass() throws SQLException {
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
String url = "jdbc:TAOS://" + host + ":6030/?user=root&password=taosdata";
connection = (DriverManager.getConnection(url, properties)).unwrap(TSDBConnection.class);
try (Statement stmt = connection.createStatement()) {
stmt.execute("drop database if exists power");
stmt.execute("create database if not exists power");
stmt.execute("use power");
stmt.execute("create table meters(ts timestamp, current float, voltage int, phase int) tags(location binary(64), groupId int)");
stmt.execute("create table d1001 using meters tags(\"Beijing.Chaoyang\", 2)");
stmt.execute("create table d1002 using meters tags(\"Beijing.Haidian\", 2)");
stmt.execute("insert into d1001 values(\"2020-08-15 12:00:00.000\", 12, 220, 1),(\"2020-08-15 12:10:00.000\", 12.3, 220, 2),(\"2020-08-15 12:20:00.000\", 12.2, 220, 1)");
stmt.execute("insert into d1002 values(\"2020-08-15 12:00:00.000\", 9.9, 220, 1),(\"2020-08-15 12:10:00.000\", 10.3, 220, 1),(\"2020-08-15 12:20:00.000\", 11.2, 220, 1)");
}
}
@AfterClass
public static void afterClass() throws SQLException {
if (connection != null)
connection.close();
}
}
...@@ -3,22 +3,98 @@ package com.taosdata.jdbc.utils; ...@@ -3,22 +3,98 @@ package com.taosdata.jdbc.utils;
import org.junit.Assert; import org.junit.Assert;
import org.junit.Test; import org.junit.Test;
import static org.junit.Assert.*; import java.util.stream.Stream;
public class UtilsTest { public class UtilsTest {
@Test @Test
public void escapeSingleQuota() { public void escapeSingleQuota() {
// given
String s = "'''''a\\'"; String s = "'''''a\\'";
// when
String news = Utils.escapeSingleQuota(s); String news = Utils.escapeSingleQuota(s);
// then
Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news); Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news);
// given
s = "\'''''a\\'"; s = "\'''''a\\'";
// when
news = Utils.escapeSingleQuota(s); news = Utils.escapeSingleQuota(s);
// then
Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news); Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news);
// given
s = "\'\'\'\''a\\'"; s = "\'\'\'\''a\\'";
// when
news = Utils.escapeSingleQuota(s); news = Utils.escapeSingleQuota(s);
// then
Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news); Assert.assertEquals("\\'\\'\\'\\'\\'a\\'", news);
} }
@Test
public void getNativeSqlReplaceQuestionMarks() {
// given
String nativeSql = "insert into ?.? (ts, temperature, humidity) using ?.? tags(?,?) values(now, ?, ?)";
Object[] parameters = Stream.of("test", "t1", "test", "weather", "beijing", 1, 12.2, 4).toArray();
// when
String actual = Utils.getNativeSql(nativeSql, parameters);
// then
String expected = "insert into test.t1 (ts, temperature, humidity) using test.weather tags('beijing',1) values(now, 12.2, 4)";
Assert.assertEquals(expected, actual);
}
@Test
public void getNativeSqlReplaceQuestionMarks2() {
// given
String nativeSql = "INSERT INTO ? (TS,CURRENT,VOLTAGE,PHASE) USING METERS TAGS (?) VALUES (?,?,?,?)";
Object[] parameters = Stream.of("d1", 1, 123, 3.14, 220, 4).toArray();
// when
String actual = Utils.getNativeSql(nativeSql, parameters);
// then
String expected = "INSERT INTO d1 (TS,CURRENT,VOLTAGE,PHASE) USING METERS TAGS (1) VALUES (123,3.14,220,4)";
Assert.assertEquals(expected, actual);
}
@Test
public void getNativeSqlReplaceNothing() {
// given
String nativeSql = "insert into test.t1 (ts, temperature, humidity) using test.weather tags('beijing',1) values(now, 12.2, 4)";
// when
String actual = Utils.getNativeSql(nativeSql, null);
// then
Assert.assertEquals(nativeSql, actual);
}
@Test
public void getNativeSqlReplaceNothing2() {
// given
String nativeSql = "insert into test.t1 (ts, temperature, humidity) using test.weather tags('beijing',1) values(now, 12.2, 4)";
Object[] parameters = Stream.of("test", "t1", "test", "weather", "beijing", 1, 12.2, 4).toArray();
// when
String actual = Utils.getNativeSql(nativeSql, parameters);
// then
Assert.assertEquals(nativeSql, actual);
}
@Test
public void getNativeSqlReplaceNothing3() {
// given
String nativeSql = "insert into ?.? (ts, temperature, humidity) using ?.? tags(?,?) values(now, ?, ?)";
// when
String actual = Utils.getNativeSql(nativeSql, null);
// then
Assert.assertEquals(nativeSql, actual);
}
} }
\ No newline at end of file
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.10", version="2.0.11",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
...@@ -14,12 +14,22 @@ def _convert_microsecond_to_datetime(micro): ...@@ -14,12 +14,22 @@ def _convert_microsecond_to_datetime(micro):
return datetime.datetime.fromtimestamp(micro / 1000000.0) return datetime.datetime.fromtimestamp(micro / 1000000.0)
def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): def _convert_nanosecond_to_datetime(nanosec):
return nanosec
def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C bool row to python row """Function to convert C bool row to python row
""" """
_timestamp_converter = _convert_millisecond_to_datetime _timestamp_converter = _convert_millisecond_to_datetime
if micro: if precision == FieldType.C_TIMESTAMP_MILLI:
_timestamp_converter = _convert_millisecond_to_datetime
elif precision == FieldType.C_TIMESTAMP_MICRO:
_timestamp_converter = _convert_microsecond_to_datetime _timestamp_converter = _convert_microsecond_to_datetime
elif precision == FieldType.C_TIMESTAMP_NANO:
_timestamp_converter = _convert_nanosecond_to_datetime
else:
raise DatabaseError("Unknown precision returned from database")
return [ return [
None if ele == FieldType.C_BIGINT_NULL else _timestamp_converter(ele) for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_NULL else _timestamp_converter(ele) for ele in ctypes.cast(
...@@ -28,7 +38,7 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -28,7 +38,7 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C bool row to python row """Function to convert C bool row to python row
""" """
return [ return [
...@@ -38,7 +48,7 @@ def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -38,7 +48,7 @@ def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_tinyint_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_tinyint_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C tinyint row to python row """Function to convert C tinyint row to python row
""" """
return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast(
...@@ -49,7 +59,7 @@ def _crow_tinyint_unsigned_to_python( ...@@ -49,7 +59,7 @@ def _crow_tinyint_unsigned_to_python(
data, data,
num_of_rows, num_of_rows,
nbytes=None, nbytes=None,
micro=False): precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C tinyint row to python row """Function to convert C tinyint row to python row
""" """
return [ return [
...@@ -59,7 +69,7 @@ def _crow_tinyint_unsigned_to_python( ...@@ -59,7 +69,7 @@ def _crow_tinyint_unsigned_to_python(
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_smallint_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_smallint_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C smallint row to python row """Function to convert C smallint row to python row
""" """
return [ return [
...@@ -70,7 +80,7 @@ def _crow_smallint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -70,7 +80,7 @@ def _crow_smallint_to_python(data, num_of_rows, nbytes=None, micro=False):
def _crow_smallint_unsigned_to_python( def _crow_smallint_unsigned_to_python(
data, num_of_rows, nbytes=None, micro=False): data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C smallint row to python row """Function to convert C smallint row to python row
""" """
return [ return [
...@@ -80,14 +90,14 @@ def _crow_smallint_unsigned_to_python( ...@@ -80,14 +90,14 @@ def _crow_smallint_unsigned_to_python(
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_int_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_int_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C int row to python row """Function to convert C int row to python row
""" """
return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]]
def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C int row to python row """Function to convert C int row to python row
""" """
return [ return [
...@@ -97,7 +107,7 @@ def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -97,7 +107,7 @@ def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False):
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bigint_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C bigint row to python row """Function to convert C bigint row to python row
""" """
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
...@@ -108,7 +118,7 @@ def _crow_bigint_unsigned_to_python( ...@@ -108,7 +118,7 @@ def _crow_bigint_unsigned_to_python(
data, data,
num_of_rows, num_of_rows,
nbytes=None, nbytes=None,
micro=False): precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C bigint row to python row """Function to convert C bigint row to python row
""" """
return [ return [
...@@ -118,21 +128,21 @@ def _crow_bigint_unsigned_to_python( ...@@ -118,21 +128,21 @@ def _crow_bigint_unsigned_to_python(
:abs(num_of_rows)]] :abs(num_of_rows)]]
def _crow_float_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_float_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C float row to python row """Function to convert C float row to python row
""" """
return [None if math.isnan(ele) else ele for ele in ctypes.cast( return [None if math.isnan(ele) else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]]
def _crow_double_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_double_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C double row to python row """Function to convert C double row to python row
""" """
return [None if math.isnan(ele) else ele for ele in ctypes.cast( return [None if math.isnan(ele) else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]]
def _crow_binary_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_binary_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C binary row to python row """Function to convert C binary row to python row
""" """
assert(nbytes is not None) assert(nbytes is not None)
...@@ -140,7 +150,7 @@ def _crow_binary_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -140,7 +150,7 @@ def _crow_binary_to_python(data, num_of_rows, nbytes=None, micro=False):
'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] 'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]]
def _crow_nchar_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_nchar_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C nchar row to python row """Function to convert C nchar row to python row
""" """
assert(nbytes is not None) assert(nbytes is not None)
...@@ -159,7 +169,7 @@ def _crow_nchar_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -159,7 +169,7 @@ def _crow_nchar_to_python(data, num_of_rows, nbytes=None, micro=False):
return res return res
def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, micro=False): def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C binary row to python row """Function to convert C binary row to python row
""" """
assert(nbytes is not None) assert(nbytes is not None)
...@@ -178,7 +188,7 @@ def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, micro=False): ...@@ -178,7 +188,7 @@ def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, micro=False):
return res return res
def _crow_nchar_to_python_block(data, num_of_rows, nbytes=None, micro=False): def _crow_nchar_to_python_block(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
"""Function to convert C nchar row to python row """Function to convert C nchar row to python row
""" """
assert(nbytes is not None) assert(nbytes is not None)
...@@ -448,8 +458,7 @@ class CTaosInterface(object): ...@@ -448,8 +458,7 @@ class CTaosInterface(object):
result, ctypes.byref(pblock)) result, ctypes.byref(pblock))
if num_of_rows == 0: if num_of_rows == 0:
return None, 0 return None, 0
isMicro = (CTaosInterface.libtaos.taos_result_precision( precision = CTaosInterface.libtaos.taos_result_precision(result)
result) == FieldType.C_TIMESTAMP_MICRO)
blocks = [None] * len(fields) blocks = [None] * len(fields)
fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result)
fieldLen = [ fieldLen = [
...@@ -462,7 +471,7 @@ class CTaosInterface(object): ...@@ -462,7 +471,7 @@ class CTaosInterface(object):
if fields[i]['type'] not in _CONVERT_FUNC_BLOCK: if fields[i]['type'] not in _CONVERT_FUNC_BLOCK:
raise DatabaseError("Invalid data type returned from database") raise DatabaseError("Invalid data type returned from database")
blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']]( blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']](
data, num_of_rows, fieldLen[i], isMicro) data, num_of_rows, fieldLen[i], precision)
return blocks, abs(num_of_rows) return blocks, abs(num_of_rows)
...@@ -472,8 +481,7 @@ class CTaosInterface(object): ...@@ -472,8 +481,7 @@ class CTaosInterface(object):
pblock = CTaosInterface.libtaos.taos_fetch_row(result) pblock = CTaosInterface.libtaos.taos_fetch_row(result)
if pblock: if pblock:
num_of_rows = 1 num_of_rows = 1
isMicro = (CTaosInterface.libtaos.taos_result_precision( precision = CTaosInterface.libtaos.taos_result_precision(result)
result) == FieldType.C_TIMESTAMP_MICRO)
blocks = [None] * len(fields) blocks = [None] * len(fields)
fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result)
fieldLen = [ fieldLen = [
...@@ -490,7 +498,7 @@ class CTaosInterface(object): ...@@ -490,7 +498,7 @@ class CTaosInterface(object):
blocks[i] = [None] blocks[i] = [None]
else: else:
blocks[i] = _CONVERT_FUNC[fields[i]['type']]( blocks[i] = _CONVERT_FUNC[fields[i]['type']](
data, num_of_rows, fieldLen[i], isMicro) data, num_of_rows, fieldLen[i], precision)
else: else:
return None, 0 return None, 0
return blocks, abs(num_of_rows) return blocks, abs(num_of_rows)
......
...@@ -40,3 +40,5 @@ class FieldType(object): ...@@ -40,3 +40,5 @@ class FieldType(object):
# Timestamp precision definition # Timestamp precision definition
C_TIMESTAMP_MILLI = 0 C_TIMESTAMP_MILLI = 0
C_TIMESTAMP_MICRO = 1 C_TIMESTAMP_MICRO = 1
C_TIMESTAMP_NANO = 2
C_TIMESTAMP_UNKNOWN = 3
...@@ -27,6 +27,7 @@ void dnodeUpdateCfg(SDnodeCfg *cfg); ...@@ -27,6 +27,7 @@ void dnodeUpdateCfg(SDnodeCfg *cfg);
int32_t dnodeGetDnodeId(); int32_t dnodeGetDnodeId();
void dnodeGetClusterId(char *clusterId); void dnodeGetClusterId(char *clusterId);
void dnodeGetCfg(int32_t *dnodeId, char *clusterId); void dnodeGetCfg(int32_t *dnodeId, char *clusterId);
void dnodeSetDropped();
#ifdef __cplusplus #ifdef __cplusplus
} }
......
...@@ -21,6 +21,7 @@ ...@@ -21,6 +21,7 @@
static SDnodeCfg tsCfg = {0}; static SDnodeCfg tsCfg = {0};
static pthread_mutex_t tsCfgMutex; static pthread_mutex_t tsCfgMutex;
static int32_t tsDnodeDropped;
static int32_t dnodeReadCfg(); static int32_t dnodeReadCfg();
static int32_t dnodeWriteCfg(); static int32_t dnodeWriteCfg();
...@@ -34,6 +35,10 @@ int32_t dnodeInitCfg() { ...@@ -34,6 +35,10 @@ int32_t dnodeInitCfg() {
if (ret == 0) { if (ret == 0) {
dInfo("dnode cfg is initialized"); dInfo("dnode cfg is initialized");
} }
if (tsDnodeDropped) {
dInfo("dnode is dropped, exiting");
return -1;
}
return ret; return ret;
} }
...@@ -44,6 +49,14 @@ void dnodeUpdateCfg(SDnodeCfg *cfg) { ...@@ -44,6 +49,14 @@ void dnodeUpdateCfg(SDnodeCfg *cfg) {
dnodeResetCfg(cfg); dnodeResetCfg(cfg);
} }
void dnodeSetDropped() {
pthread_mutex_lock(&tsCfgMutex);
tsDnodeDropped = 1;
dnodeWriteCfg();
pthread_mutex_unlock(&tsCfgMutex);
}
int32_t dnodeGetDnodeId() { int32_t dnodeGetDnodeId() {
int32_t dnodeId = 0; int32_t dnodeId = 0;
pthread_mutex_lock(&tsCfgMutex); pthread_mutex_lock(&tsCfgMutex);
...@@ -119,6 +132,14 @@ static int32_t dnodeReadCfg() { ...@@ -119,6 +132,14 @@ static int32_t dnodeReadCfg() {
} }
cfg.dnodeId = (int32_t)dnodeId->valueint; cfg.dnodeId = (int32_t)dnodeId->valueint;
cJSON *dnodeDropped = cJSON_GetObjectItem(root, "dnodeDropped");
if (!dnodeDropped || dnodeDropped->type != cJSON_Number) {
dError("failed to read %s, dnodeDropped not found", file);
//goto PARSE_CFG_OVER;
} else {
tsDnodeDropped = (int32_t)dnodeDropped->valueint;
}
cJSON *clusterId = cJSON_GetObjectItem(root, "clusterId"); cJSON *clusterId = cJSON_GetObjectItem(root, "clusterId");
if (!clusterId || clusterId->type != cJSON_String) { if (!clusterId || clusterId->type != cJSON_String) {
dError("failed to read %s, clusterId not found", file); dError("failed to read %s, clusterId not found", file);
...@@ -154,6 +175,7 @@ static int32_t dnodeWriteCfg() { ...@@ -154,6 +175,7 @@ static int32_t dnodeWriteCfg() {
len += snprintf(content + len, maxLen - len, "{\n"); len += snprintf(content + len, maxLen - len, "{\n");
len += snprintf(content + len, maxLen - len, " \"dnodeId\": %d,\n", tsCfg.dnodeId); len += snprintf(content + len, maxLen - len, " \"dnodeId\": %d,\n", tsCfg.dnodeId);
len += snprintf(content + len, maxLen - len, " \"dnodeDropped\": %d,\n", tsDnodeDropped);
len += snprintf(content + len, maxLen - len, " \"clusterId\": \"%s\"\n", tsCfg.clusterId); len += snprintf(content + len, maxLen - len, " \"clusterId\": \"%s\"\n", tsCfg.clusterId);
len += snprintf(content + len, maxLen - len, "}\n"); len += snprintf(content + len, maxLen - len, "}\n");
......
...@@ -202,6 +202,7 @@ static void dnodeProcessStatusRsp(SRpcMsg *pMsg) { ...@@ -202,6 +202,7 @@ static void dnodeProcessStatusRsp(SRpcMsg *pMsg) {
char clusterId[TSDB_CLUSTER_ID_LEN]; char clusterId[TSDB_CLUSTER_ID_LEN];
dnodeGetClusterId(clusterId); dnodeGetClusterId(clusterId);
if (clusterId[0] != '\0') { if (clusterId[0] != '\0') {
dnodeSetDropped();
dError("exit zombie dropped dnode"); dError("exit zombie dropped dnode");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
......
...@@ -1475,6 +1475,7 @@ static int32_t mnodeChangeSuperTableColumn(SMnodeMsg *pMsg) { ...@@ -1475,6 +1475,7 @@ static int32_t mnodeChangeSuperTableColumn(SMnodeMsg *pMsg) {
SSchema *schema = (SSchema *) (pStable->schema + col); SSchema *schema = (SSchema *) (pStable->schema + col);
ASSERT(schema->type == TSDB_DATA_TYPE_BINARY || schema->type == TSDB_DATA_TYPE_NCHAR); ASSERT(schema->type == TSDB_DATA_TYPE_BINARY || schema->type == TSDB_DATA_TYPE_NCHAR);
schema->bytes = pAlter->schema[0].bytes; schema->bytes = pAlter->schema[0].bytes;
pStable->sversion++;
mInfo("msg:%p, app:%p stable %s, start to modify column %s len to %d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mInfo("msg:%p, app:%p stable %s, start to modify column %s len to %d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
name, schema->bytes); name, schema->bytes);
...@@ -1504,6 +1505,7 @@ static int32_t mnodeChangeSuperTableTag(SMnodeMsg *pMsg) { ...@@ -1504,6 +1505,7 @@ static int32_t mnodeChangeSuperTableTag(SMnodeMsg *pMsg) {
SSchema *schema = (SSchema *) (pStable->schema + col + pStable->numOfColumns); SSchema *schema = (SSchema *) (pStable->schema + col + pStable->numOfColumns);
ASSERT(schema->type == TSDB_DATA_TYPE_BINARY || schema->type == TSDB_DATA_TYPE_NCHAR); ASSERT(schema->type == TSDB_DATA_TYPE_BINARY || schema->type == TSDB_DATA_TYPE_NCHAR);
schema->bytes = pAlter->schema[0].bytes; schema->bytes = pAlter->schema[0].bytes;
pStable->tversion++;
mInfo("msg:%p, app:%p stable %s, start to modify tag len %s to %d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, mInfo("msg:%p, app:%p stable %s, start to modify tag len %s to %d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId,
name, schema->bytes); name, schema->bytes);
......
...@@ -7,7 +7,12 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib) ...@@ -7,7 +7,12 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib)
IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR) IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR)
MESSAGE(STATUS "gTest library found, build unit test") MESSAGE(STATUS "gTest library found, build unit test")
INCLUDE_DIRECTORIES(${HEADER_GTEST_INCLUDE_DIR}) # GoogleTest requires at least C++11
SET(CMAKE_CXX_STANDARD 11)
INCLUDE_DIRECTORIES(/usr/include /usr/local/include)
LINK_DIRECTORIES(/usr/lib /usr/local/lib)
AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST) AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST)
ADD_EXECUTABLE(osTest ${SOURCE_LIST}) ADD_EXECUTABLE(osTest ${SOURCE_LIST})
......
...@@ -19,6 +19,70 @@ ...@@ -19,6 +19,70 @@
#define HTTP_MAX_URL 5 // http url stack size #define HTTP_MAX_URL 5 // http url stack size
#define HTTP_CODE_CONTINUE 100
#define HTTP_CODE_SWITCHING_PROTOCOL 101
#define HTTP_CODE_PROCESSING 102
#define HTTP_CODE_EARLY_HINTS 103
#define HTTP_CODE_OK 200
#define HTTP_CODE_CREATED 201
#define HTTP_CODE_ACCEPTED 202
#define HTTP_CODE_NON_AUTHORITATIVE_INFO 203
#define HTTP_CODE_NO_CONTENT 204
#define HTTP_CODE_RESET_CONTENT 205
#define HTTP_CODE_PARTIAL_CONTENT 206
#define HTTP_CODE_MULTI_STATUS 207
#define HTTP_CODE_ALREADY_REPORTED 208
#define HTTP_CODE_IM_USED 226
#define HTTP_CODE_MULTIPLE_CHOICE 300
#define HTTP_CODE_MOVED_PERMANENTLY 301
#define HTTP_CODE_FOUND 302
#define HTTP_CODE_SEE_OTHER 303
#define HTTP_CODE_NOT_MODIFIED 304
#define HTTP_CODE_USE_PROXY 305
#define HTTP_CODE_UNUSED 306
#define HTTP_CODE_TEMPORARY_REDIRECT 307
#define HTTP_CODE_PERMANENT_REDIRECT 308
#define HTTP_CODE_BAD_REQUEST 400
#define HTTP_CODE_UNAUTHORIZED 401
#define HTTP_CODE_PAYMENT_REQUIRED 402
#define HTTP_CODE_FORBIDDEN 403
#define HTTP_CODE_NOT_FOUND 404
#define HTTP_CODE_METHOD_NOT_ALLOWED 405
#define HTTP_CODE_NOT_ACCEPTABLE 406
#define HTTP_CODE_PROXY_AUTH_REQUIRED 407
#define HTTP_CODE_REQUEST_TIMEOUT 408
#define HTTP_CODE_CONFLICT 409
#define HTTP_CODE_GONE 410
#define HTTP_CODE_LENGTH_REQUIRED 411
#define HTTP_CODE_PRECONDITION_FAILED 412
#define HTTP_CODE_PAYLOAD_TOO_LARGE 413
#define HTTP_CODE_URI_TOO_LARGE 414
#define HTTP_CODE_UNSUPPORTED_MEDIA_TYPE 415
#define HTTP_CODE_RANGE_NOT_SATISFIABLE 416
#define HTTP_CODE_EXPECTATION_FAILED 417
#define HTTP_CODE_IM_A_TEAPOT 418
#define HTTP_CODE_MISDIRECTED_REQUEST 421
#define HTTP_CODE_UNPROCESSABLE_ENTITY 422
#define HTTP_CODE_LOCKED 423
#define HTTP_CODE_FAILED_DEPENDENCY 424
#define HTTP_CODE_TOO_EARLY 425
#define HTTP_CODE_UPGRADE_REQUIRED 426
#define HTTP_CODE_PRECONDITION_REQUIRED 428
#define HTTP_CODE_TOO_MANY_REQUESTS 429
#define HTTP_CODE_REQ_HDR_FIELDS_TOO_LARGE 431
#define HTTP_CODE_UNAVAIL_4_LEGAL_REASONS 451
#define HTTP_CODE_INTERNAL_SERVER_ERROR 500
#define HTTP_CODE_NOT_IMPLEMENTED 501
#define HTTP_CODE_BAD_GATEWAY 502
#define HTTP_CODE_SERVICE_UNAVAILABLE 503
#define HTTP_CODE_GATEWAY_TIMEOUT 504
#define HTTP_CODE_HTTP_VER_NOT_SUPPORTED 505
#define HTTP_CODE_VARIANT_ALSO_NEGOTIATES 506
#define HTTP_CODE_INSUFFICIENT_STORAGE 507
#define HTTP_CODE_LOOP_DETECTED 508
#define HTTP_CODE_NOT_EXTENDED 510
#define HTTP_CODE_NETWORK_AUTH_REQUIRED 511
typedef enum HTTP_PARSER_STATE { typedef enum HTTP_PARSER_STATE {
HTTP_PARSER_BEGIN, HTTP_PARSER_BEGIN,
HTTP_PARSER_REQUEST_OR_RESPONSE, HTTP_PARSER_REQUEST_OR_RESPONSE,
......
...@@ -133,6 +133,16 @@ bool gcBuildQueryJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result, ...@@ -133,6 +133,16 @@ bool gcBuildQueryJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result,
int32_t len; int32_t len;
len = snprintf(target, HTTP_GC_TARGET_SIZE, "%s{", aliasBuffer); len = snprintf(target, HTTP_GC_TARGET_SIZE, "%s{", aliasBuffer);
for (int32_t i = dataFields + 1; i < num_fields; i++) { for (int32_t i = dataFields + 1; i < num_fields; i++) {
if (row[i] == NULL) {
len += snprintf(target + len, HTTP_GC_TARGET_SIZE - len, "%s:nil", fields[i].name);
if (i < num_fields - 1) {
len += snprintf(target + len, HTTP_GC_TARGET_SIZE - len, ", ");
}
continue;
}
switch (fields[i].type) { switch (fields[i].type) {
case TSDB_DATA_TYPE_BOOL: case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
......
此差异已折叠。
...@@ -67,81 +67,81 @@ static void httpSendErrorRespImp(HttpContext *pContext, int32_t httpCode, char * ...@@ -67,81 +67,81 @@ static void httpSendErrorRespImp(HttpContext *pContext, int32_t httpCode, char *
} }
void httpSendErrorResp(HttpContext *pContext, int32_t errNo) { void httpSendErrorResp(HttpContext *pContext, int32_t errNo) {
int32_t httpCode = 500; int32_t httpCode = HTTP_CODE_INTERNAL_SERVER_ERROR;
if (errNo == TSDB_CODE_SUCCESS) if (errNo == TSDB_CODE_SUCCESS)
httpCode = 200; httpCode = HTTP_CODE_OK;
else if (errNo == TSDB_CODE_HTTP_SERVER_OFFLINE) else if (errNo == TSDB_CODE_HTTP_SERVER_OFFLINE)
httpCode = 404; httpCode = HTTP_CODE_NOT_FOUND;
else if (errNo == TSDB_CODE_HTTP_UNSUPPORT_URL) else if (errNo == TSDB_CODE_HTTP_UNSUPPORT_URL)
httpCode = 404; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_INVALID_URL) else if (errNo == TSDB_CODE_HTTP_INVALID_URL)
httpCode = 404; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_NO_ENOUGH_MEMORY) else if (errNo == TSDB_CODE_HTTP_NO_ENOUGH_MEMORY)
httpCode = 507; httpCode = HTTP_CODE_INSUFFICIENT_STORAGE;
else if (errNo == TSDB_CODE_HTTP_REQUSET_TOO_BIG) else if (errNo == TSDB_CODE_HTTP_REQUSET_TOO_BIG)
httpCode = 413; httpCode = HTTP_CODE_PAYLOAD_TOO_LARGE;
else if (errNo == TSDB_CODE_HTTP_NO_AUTH_INFO) else if (errNo == TSDB_CODE_HTTP_NO_AUTH_INFO)
httpCode = 401; httpCode = HTTP_CODE_UNAUTHORIZED;
else if (errNo == TSDB_CODE_HTTP_NO_MSG_INPUT) else if (errNo == TSDB_CODE_HTTP_NO_MSG_INPUT)
httpCode = 400; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_NO_SQL_INPUT) else if (errNo == TSDB_CODE_HTTP_NO_SQL_INPUT)
httpCode = 400; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_NO_EXEC_USEDB) else if (errNo == TSDB_CODE_HTTP_NO_EXEC_USEDB)
httpCode = 400; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_SESSION_FULL) else if (errNo == TSDB_CODE_HTTP_SESSION_FULL)
httpCode = 421; httpCode = HTTP_CODE_INTERNAL_SERVER_ERROR;
else if (errNo == TSDB_CODE_HTTP_GEN_TAOSD_TOKEN_ERR) else if (errNo == TSDB_CODE_HTTP_GEN_TAOSD_TOKEN_ERR)
httpCode = 507; httpCode = HTTP_CODE_INTERNAL_SERVER_ERROR;
else if (errNo == TSDB_CODE_HTTP_INVALID_MULTI_REQUEST) else if (errNo == TSDB_CODE_HTTP_INVALID_MULTI_REQUEST)
httpCode = 400; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_CREATE_GZIP_FAILED) else if (errNo == TSDB_CODE_HTTP_CREATE_GZIP_FAILED)
httpCode = 507; httpCode = HTTP_CODE_INTERNAL_SERVER_ERROR;
else if (errNo == TSDB_CODE_HTTP_FINISH_GZIP_FAILED) else if (errNo == TSDB_CODE_HTTP_FINISH_GZIP_FAILED)
httpCode = 507; httpCode = HTTP_CODE_INTERNAL_SERVER_ERROR;
else if (errNo == TSDB_CODE_HTTP_INVALID_VERSION) else if (errNo == TSDB_CODE_HTTP_INVALID_VERSION)
httpCode = 406; httpCode = HTTP_CODE_HTTP_VER_NOT_SUPPORTED;
else if (errNo == TSDB_CODE_HTTP_INVALID_CONTENT_LENGTH) else if (errNo == TSDB_CODE_HTTP_INVALID_CONTENT_LENGTH)
httpCode = 406; httpCode = HTTP_CODE_LENGTH_REQUIRED;
else if (errNo == TSDB_CODE_HTTP_INVALID_AUTH_TYPE) else if (errNo == TSDB_CODE_HTTP_INVALID_AUTH_TYPE)
httpCode = 406; httpCode = HTTP_CODE_UNAUTHORIZED;
else if (errNo == TSDB_CODE_HTTP_INVALID_AUTH_FORMAT) else if (errNo == TSDB_CODE_HTTP_INVALID_AUTH_FORMAT)
httpCode = 406; httpCode = HTTP_CODE_UNAUTHORIZED;
else if (errNo == TSDB_CODE_HTTP_INVALID_BASIC_AUTH) else if (errNo == TSDB_CODE_HTTP_INVALID_BASIC_AUTH)
httpCode = 406; httpCode = HTTP_CODE_UNAUTHORIZED;
else if (errNo == TSDB_CODE_HTTP_INVALID_TAOSD_AUTH) else if (errNo == TSDB_CODE_HTTP_INVALID_TAOSD_AUTH)
httpCode = 406; httpCode = HTTP_CODE_UNAUTHORIZED;
else if (errNo == TSDB_CODE_HTTP_PARSE_METHOD_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_METHOD_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_TARGET_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_TARGET_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_VERSION_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_VERSION_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_SP_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_SP_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_STATUS_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_STATUS_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_PHRASE_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_PHRASE_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_CRLF_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_CRLF_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_KEY_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_KEY_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_VAL_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_HEADER_VAL_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_CHUNK_SIZE_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_CHUNK_SIZE_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_CHUNK_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_CHUNK_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_END_FAILED) else if (errNo == TSDB_CODE_HTTP_PARSE_END_FAILED)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_INVALID_STATE) else if (errNo == TSDB_CODE_HTTP_PARSE_INVALID_STATE)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else if (errNo == TSDB_CODE_HTTP_PARSE_ERROR_STATE) else if (errNo == TSDB_CODE_HTTP_PARSE_ERROR_STATE)
httpCode = 406; httpCode = HTTP_CODE_BAD_REQUEST;
else else
httpCode = 400; httpCode = HTTP_CODE_BAD_REQUEST;
if (pContext->parser && pContext->parser->httpCode != 0) { if (pContext->parser && pContext->parser->httpCode != 0) {
httpCode = pContext->parser->httpCode; httpCode = pContext->parser->httpCode;
...@@ -152,7 +152,7 @@ void httpSendErrorResp(HttpContext *pContext, int32_t errNo) { ...@@ -152,7 +152,7 @@ void httpSendErrorResp(HttpContext *pContext, int32_t errNo) {
} }
void httpSendTaosdInvalidSqlErrorResp(HttpContext *pContext, char *errMsg) { void httpSendTaosdInvalidSqlErrorResp(HttpContext *pContext, char *errMsg) {
int32_t httpCode = 400; int32_t httpCode = HTTP_CODE_BAD_REQUEST;
char temp[512] = {0}; char temp[512] = {0};
int32_t len = sprintf(temp, "invalid SQL: %s", errMsg); int32_t len = sprintf(temp, "invalid SQL: %s", errMsg);
......
...@@ -138,7 +138,6 @@ typedef struct SQueryInfo { ...@@ -138,7 +138,6 @@ typedef struct SQueryInfo {
bool hasFilter; bool hasFilter;
bool onlyTagQuery; bool onlyTagQuery;
bool orderProjectQuery; bool orderProjectQuery;
// bool diffQuery;
bool stateWindow; bool stateWindow;
} SQueryInfo; } SQueryInfo;
......
...@@ -176,7 +176,7 @@ cmd ::= ALTER ACCOUNT ids(X) PASS ids(Y) acct_optr(Z). { setCreateAcctSql(p ...@@ -176,7 +176,7 @@ cmd ::= ALTER ACCOUNT ids(X) PASS ids(Y) acct_optr(Z). { setCreateAcctSql(p
////////////////////////////// COMPACT STATEMENT ////////////////////////////////////////////// ////////////////////////////// COMPACT STATEMENT //////////////////////////////////////////////
cmd ::= COMPACT VNODES IN LP exprlist(Y) RP. { setCompactVnodeSql(pInfo, TSDB_SQL_COMPACT_VNODE, Y);} cmd ::= COMPACT VNODES IN LP exprlist(Y) RP. { setCompactVnodeSql(pInfo, TSDB_SQL_COMPACT_VNODE, Y);}
// An IDENTIFIER can be a generic identifier, or one of several keywords. // An IDENTIFIER can be a generic identifier, or one of several keywords.
// Any non-standard keyword can also be an identifier. // Any non-standard keyword can also be an identifier.
......
...@@ -3613,16 +3613,7 @@ static void deriv_function(SQLFunctionCtx *pCtx) { ...@@ -3613,16 +3613,7 @@ static void deriv_function(SQLFunctionCtx *pCtx) {
qError("error input type"); qError("error input type");
} }
// initial value is not set yet, all data block are null GET_RES_INFO(pCtx)->numOfRes += notNullElems;
if (!pDerivInfo->valueSet || notNullElems <= 0) {
/*
* 1. current block and blocks before are full of null
* 2. current block may be null value
*/
assert(pCtx->hasNull);
} else {
GET_RES_INFO(pCtx)->numOfRes += notNullElems;
}
} }
#define DIFF_IMPL(ctx, d, type) \ #define DIFF_IMPL(ctx, d, type) \
...@@ -3662,7 +3653,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3662,7 +3653,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = (int32_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null *pOutput = (int32_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
...@@ -3684,7 +3675,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3684,7 +3675,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = pData[i] - pCtx->param[1].i64; // direct previous may be null *pOutput = pData[i] - pCtx->param[1].i64; // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
...@@ -3706,7 +3697,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3706,7 +3697,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = pData[i] - pCtx->param[1].dKey; // direct previous may be null *pOutput = pData[i] - pCtx->param[1].dKey; // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
...@@ -3728,7 +3719,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3728,7 +3719,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = (float)(pData[i] - pCtx->param[1].dKey); // direct previous may be null *pOutput = (float)(pData[i] - pCtx->param[1].dKey); // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
...@@ -3750,7 +3741,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3750,7 +3741,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = (int16_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null *pOutput = (int16_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
...@@ -3773,7 +3764,7 @@ static void diff_function(SQLFunctionCtx *pCtx) { ...@@ -3773,7 +3764,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet if (pCtx->param[1].nType != INITIAL_VALUE_NOT_ASSIGNED) { // initial value is not set yet
*pOutput = (int8_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null *pOutput = (int8_t)(pData[i] - pCtx->param[1].i64); // direct previous may be null
*pTimestamp = tsList[i]; *pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1; pOutput += 1;
pTimestamp += 1; pTimestamp += 1;
} }
......
...@@ -1312,6 +1312,8 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SGroupbyOperatorInfo *pIn ...@@ -1312,6 +1312,8 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SGroupbyOperatorInfo *pIn
} }
} }
// todo opt perf
for (int32_t k = 0; k < pOperator->numOfOutput; ++k) { for (int32_t k = 0; k < pOperator->numOfOutput; ++k) {
pInfo->binfo.pCtx[k].size = 1; pInfo->binfo.pCtx[k].size = 1;
int32_t functionId = pInfo->binfo.pCtx[k].functionId; int32_t functionId = pInfo->binfo.pCtx[k].functionId;
...@@ -7529,7 +7531,8 @@ static void doSetTagValueToResultBuf(char* output, const char* val, int16_t type ...@@ -7529,7 +7531,8 @@ static void doSetTagValueToResultBuf(char* output, const char* val, int16_t type
if (IS_VAR_DATA_TYPE(type)) { if (IS_VAR_DATA_TYPE(type)) {
// Binary data overflows for sort of unknown reasons. Let trim the overflow data // Binary data overflows for sort of unknown reasons. Let trim the overflow data
if (varDataTLen(val) > bytes) { if (varDataTLen(val) > bytes) {
int32_t len = bytes - VARSTR_HEADER_SIZE; // remain available space int32_t maxLen = bytes - VARSTR_HEADER_SIZE;
int32_t len = (varDataLen(val) > maxLen)? maxLen:varDataLen(val);
memcpy(varDataVal(output), varDataVal(val), len); memcpy(varDataVal(output), varDataVal(val), len);
varDataSetLen(output, len); varDataSetLen(output, len);
} else { } else {
......
...@@ -812,34 +812,31 @@ int32_t tsBufMerge(STSBuf* pDestBuf, const STSBuf* pSrcBuf) { ...@@ -812,34 +812,31 @@ int32_t tsBufMerge(STSBuf* pDestBuf, const STSBuf* pSrcBuf) {
int64_t offset = getDataStartOffset(); int64_t offset = getDataStartOffset();
int32_t size = (int32_t)pSrcBuf->fileSize - (int32_t)offset; int32_t size = (int32_t)pSrcBuf->fileSize - (int32_t)offset;
int64_t written = taosFSendFile(pDestBuf->f, pSrcBuf->f, &offset, size);
int64_t rc = taosFSendFile(pDestBuf->f, pSrcBuf->f, &offset, size);
if (rc == -1) {
// tscError("failed to merge tsBuf from:%s to %s, reason:%s\n", pSrcBuf->path, pDestBuf->path, strerror(errno));
return -1;
}
if (rc != size) { if (written == -1 || written != size) {
// tscError("failed to merge tsBuf from:%s to %s, reason:%s\n", pSrcBuf->path, pDestBuf->path, strerror(errno));
return -1; return -1;
} }
pDestBuf->numOfTotal += pSrcBuf->numOfTotal; pDestBuf->numOfTotal += pSrcBuf->numOfTotal;
int32_t oldSize = pDestBuf->fileSize; int32_t oldSize = pDestBuf->fileSize;
// file meta data may be cached, close and reopen the file for accurate file size.
fclose(pDestBuf->f);
pDestBuf->f = fopen(pDestBuf->path, "rb+");
if (pDestBuf->f == NULL) {
return -1;
}
struct stat fileStat; struct stat fileStat;
if (fstat(fileno(pDestBuf->f), &fileStat) != 0) { if (fstat(fileno(pDestBuf->f), &fileStat) != 0) {
return -1; return -1;
} }
pDestBuf->fileSize = (uint32_t)fileStat.st_size; pDestBuf->fileSize = (uint32_t)fileStat.st_size;
assert(pDestBuf->fileSize == oldSize + size); assert(pDestBuf->fileSize == oldSize + size);
// tscDebug("tsBuf merge success, %p, path:%s, fd:%d, file size:%d, numOfGroups:%d, autoDelete:%d", pDestBuf,
// pDestBuf->path, fileno(pDestBuf->f), pDestBuf->fileSize, pDestBuf->numOfGroups, pDestBuf->autoDelete);
return 0; return 0;
} }
......
...@@ -7,11 +7,17 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib) ...@@ -7,11 +7,17 @@ FIND_LIBRARY(LIB_GTEST_STATIC_DIR libgtest.a /usr/lib/ /usr/local/lib)
IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR) IF (HEADER_GTEST_INCLUDE_DIR AND LIB_GTEST_STATIC_DIR)
MESSAGE(STATUS "gTest library found, build unit test") MESSAGE(STATUS "gTest library found, build unit test")
# GoogleTest requires at least C++11
SET(CMAKE_CXX_STANDARD 11)
INCLUDE_DIRECTORIES(/usr/include /usr/local/include)
LINK_DIRECTORIES(/usr/lib /usr/local/lib)
INCLUDE_DIRECTORIES(${HEADER_GTEST_INCLUDE_DIR}) INCLUDE_DIRECTORIES(${HEADER_GTEST_INCLUDE_DIR})
AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST) AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST)
ADD_EXECUTABLE(queryTest ${SOURCE_LIST}) ADD_EXECUTABLE(queryTest ${SOURCE_LIST})
TARGET_LINK_LIBRARIES(queryTest taos query gtest pthread gcov) TARGET_LINK_LIBRARIES(queryTest taos query gtest pthread)
ENDIF() ENDIF()
SET_SOURCE_FILES_PROPERTIES(./astTest.cpp PROPERTIES COMPILE_FLAGS -w) SET_SOURCE_FILES_PROPERTIES(./astTest.cpp PROPERTIES COMPILE_FLAGS -w)
......
...@@ -812,6 +812,7 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) { ...@@ -812,6 +812,7 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) {
STable *pTable = pMeta->tables[i]; STable *pTable = pMeta->tables[i];
if (pTable == NULL) continue; if (pTable == NULL) continue;
pTable->restoreColumnNum = 0; pTable->restoreColumnNum = 0;
pTable->hasRestoreLastColumn = false;
} }
} }
...@@ -895,6 +896,7 @@ int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg) { ...@@ -895,6 +896,7 @@ int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg) {
maxTableIdx = i; maxTableIdx = i;
if (cacheLastCol) { if (cacheLastCol) {
pTable->restoreColumnNum = 0; pTable->restoreColumnNum = 0;
pTable->hasRestoreLastColumn = false;
} }
} }
...@@ -913,6 +915,7 @@ int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg) { ...@@ -913,6 +915,7 @@ int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg) {
} }
if (need_free_last_col) { if (need_free_last_col) {
tsdbFreeLastColumns(pTable); tsdbFreeLastColumns(pTable);
pTable->hasRestoreLastColumn = false;
} }
} }
} }
......
...@@ -148,7 +148,9 @@ int tsdbCreateTable(STsdbRepo *repo, STableCfg *pCfg) { ...@@ -148,7 +148,9 @@ int tsdbCreateTable(STsdbRepo *repo, STableCfg *pCfg) {
return 0; return 0;
_err: _err:
tsdbFreeTable(super); if (newSuper) {
tsdbFreeTable(super);
}
tsdbFreeTable(table); tsdbFreeTable(table);
return -1; return -1;
} }
...@@ -211,7 +213,7 @@ void *tsdbGetTableTagVal(const void* pTable, int32_t colId, int16_t type, int16_ ...@@ -211,7 +213,7 @@ void *tsdbGetTableTagVal(const void* pTable, int32_t colId, int16_t type, int16_
} }
char *val = tdGetKVRowValOfCol(((STable*)pTable)->tagVal, colId); char *val = tdGetKVRowValOfCol(((STable*)pTable)->tagVal, colId);
assert(type == pCol->type && bytes == pCol->bytes); assert(type == pCol->type && bytes >= pCol->bytes);
// if (val != NULL && IS_VAR_DATA_TYPE(type)) { // if (val != NULL && IS_VAR_DATA_TYPE(type)) {
// assert(varDataLen(val) < pCol->bytes); // assert(varDataLen(val) < pCol->bytes);
...@@ -607,6 +609,7 @@ void tsdbFreeLastColumns(STable* pTable) { ...@@ -607,6 +609,7 @@ void tsdbFreeLastColumns(STable* pTable) {
pTable->maxColNum = 0; pTable->maxColNum = 0;
pTable->lastColSVersion = -1; pTable->lastColSVersion = -1;
pTable->restoreColumnNum = 0; pTable->restoreColumnNum = 0;
pTable->hasRestoreLastColumn = false;
} }
int16_t tsdbGetLastColumnsIndexByColId(STable* pTable, int16_t colId) { int16_t tsdbGetLastColumnsIndexByColId(STable* pTable, int16_t colId) {
...@@ -643,6 +646,7 @@ int tsdbInitColIdCacheWithSchema(STable* pTable, STSchema* pSchema) { ...@@ -643,6 +646,7 @@ int tsdbInitColIdCacheWithSchema(STable* pTable, STSchema* pSchema) {
pTable->lastColSVersion = schemaVersion(pSchema); pTable->lastColSVersion = schemaVersion(pSchema);
pTable->maxColNum = numOfColumn; pTable->maxColNum = numOfColumn;
pTable->restoreColumnNum = 0; pTable->restoreColumnNum = 0;
pTable->hasRestoreLastColumn = false;
return 0; return 0;
} }
...@@ -655,7 +659,7 @@ int tsdbUpdateLastColSchema(STable *pTable, STSchema *pNewSchema) { ...@@ -655,7 +659,7 @@ int tsdbUpdateLastColSchema(STable *pTable, STSchema *pNewSchema) {
return 0; return 0;
} }
tsdbInfo("tsdbUpdateLastColSchema:%s,%d->%d", pTable->name->data, pTable->lastColSVersion, schemaVersion(pNewSchema)); tsdbDebug("tsdbUpdateLastColSchema:%s,%d->%d", pTable->name->data, pTable->lastColSVersion, schemaVersion(pNewSchema));
int16_t numOfCols = pNewSchema->numOfCols; int16_t numOfCols = pNewSchema->numOfCols;
SDataCol *lastCols = (SDataCol*)malloc(numOfCols * sizeof(SDataCol)); SDataCol *lastCols = (SDataCol*)malloc(numOfCols * sizeof(SDataCol));
...@@ -800,6 +804,7 @@ static STable *tsdbNewTable() { ...@@ -800,6 +804,7 @@ static STable *tsdbNewTable() {
pTable->lastCols = NULL; pTable->lastCols = NULL;
pTable->restoreColumnNum = 0; pTable->restoreColumnNum = 0;
pTable->maxColNum = 0; pTable->maxColNum = 0;
pTable->hasRestoreLastColumn = false;
pTable->lastColSVersion = -1; pTable->lastColSVersion = -1;
return pTable; return pTable;
} }
......
...@@ -2520,7 +2520,7 @@ static bool loadCachedLast(STsdbQueryHandle* pQueryHandle) { ...@@ -2520,7 +2520,7 @@ static bool loadCachedLast(STsdbQueryHandle* pQueryHandle) {
int32_t numOfCols = pTable->maxColNum; int32_t numOfCols = pTable->maxColNum;
if (pTable->lastCols == NULL || pTable->maxColNum <= 0) { if (pTable->lastCols == NULL || pTable->maxColNum <= 0) {
tsdbWarn("no last cached for table, uid:%" PRIu64 ",tid:%d", pTable->tableId.uid, pTable->tableId.tid); tsdbWarn("no last cached for table %s, uid:%" PRIu64 ",tid:%d", pTable->name->data, pTable->tableId.uid, pTable->tableId.tid);
continue; continue;
} }
......
...@@ -15,6 +15,7 @@ import sys ...@@ -15,6 +15,7 @@ import sys
from util.log import * from util.log import *
from util.cases import * from util.cases import *
from util.sql import * from util.sql import *
import time
class TDTestCase: class TDTestCase:
...@@ -129,9 +130,6 @@ class TDTestCase: ...@@ -129,9 +130,6 @@ class TDTestCase:
tdSql.prepare() tdSql.prepare()
##TODO: need to wait for TD-4445 to implement the following
## tests
## preset the keep ## preset the keep
tdSql.prepare() tdSql.prepare()
...@@ -176,20 +174,22 @@ class TDTestCase: ...@@ -176,20 +174,22 @@ class TDTestCase:
tdSql.error('insert into tb values (now-15d, 10)') tdSql.error('insert into tb values (now-15d, 10)')
tdSql.query('select * from tb') tdSql.query('select * from tb')
tdSql.checkRows(rowNum) tdSql.checkRows(rowNum)
tdLog.notice('testing keep will be altered if sudden change from small to big') tdLog.notice('testing keep will be altered if sudden change from small to big')
tdSql.execute('alter database db keep 14,14,14') for i in range(30):
tdSql.execute('alter database db keep 15,15,15') tdSql.execute('alter database db keep 14,14,14')
tdSql.execute('insert into tb values (now-15d, 10)') tdSql.execute('alter database db keep 16,16,16')
tdSql.query('select * from tb') tdSql.execute('insert into tb values (now-15d, 10)')
tdSql.checkRows(rowNum + 1) tdSql.query('select * from tb')
rowNum += 1
tdSql.checkRows(rowNum )
tdLog.notice('testing keep will be altered if sudden change from big to small') tdLog.notice('testing keep will be altered if sudden change from big to small')
tdSql.execute('alter database db keep 16,16,16') tdSql.execute('alter database db keep 16,16,16')
tdSql.execute('alter database db keep 14,14,14') tdSql.execute('alter database db keep 14,14,14')
tdSql.error('insert into tb values (now-15d, 10)') tdSql.error('insert into tb values (now-15d, 10)')
tdSql.query('select * from tb') tdSql.query('select * from tb')
tdSql.checkRows(rowNum + 1) tdSql.checkRows(rowNum)
......
...@@ -19,6 +19,7 @@ from util.pathFinding import * ...@@ -19,6 +19,7 @@ from util.pathFinding import *
from util.dnodes import tdDnodes from util.dnodes import tdDnodes
from datetime import datetime from datetime import datetime
import subprocess import subprocess
import time
##TODO: this is now automatic, but not sure if this will run through jenkins ##TODO: this is now automatic, but not sure if this will run through jenkins
class TDTestCase: class TDTestCase:
...@@ -34,24 +35,36 @@ class TDTestCase: ...@@ -34,24 +35,36 @@ class TDTestCase:
## change system time to 2020/10/20 ## change system time to 2020/10/20
os.system('sudo timedatectl set-ntp off') os.system('sudo timedatectl set-ntp off')
tdLog.sleep(10)
os.system('sudo timedatectl set-time 2020-10-20') os.system('sudo timedatectl set-time 2020-10-20')
#run taosdemo to insert data. one row per second from 2020/10/11 to 2020/10/20 #run taosdemo to insert data. one row per second from 2020/10/11 to 2020/10/20
#11 data files should be generated #11 data files should be generated
#vnode at TDinternal/community/sim/dnode1/data/vnode #vnode at TDinternal/community/sim/dnode1/data/vnode
os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_A.json") try:
commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data'] os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_A.json")
result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8') commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data']
print(result.count('data')) result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8')
except BaseException:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
if result.count('data') != 11: if result.count('data') != 11:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
tdLog.exit('wrong number of files') tdLog.exit('wrong number of files')
else: else:
tdLog.debug("data file number correct") tdLog.debug("data file number correct")
#move 5 days ahead to 2020/10/25. 4 oldest files should be removed during the new write #move 5 days ahead to 2020/10/25. 4 oldest files should be removed during the new write
#leaving 7 data files. #leaving 7 data files.
os.system ('timedatectl set-time 2020-10-25') try:
os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_B.json") os.system ('timedatectl set-time 2020-10-25')
os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_B.json")
except BaseException:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data'] commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data']
result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8') result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8')
print(result.count('data')) print(result.count('data'))
...@@ -61,9 +74,12 @@ class TDTestCase: ...@@ -61,9 +74,12 @@ class TDTestCase:
tdLog.debug("data file number correct") tdLog.debug("data file number correct")
tdSql.query('select first(ts) from stb_0') tdSql.query('select first(ts) from stb_0')
tdSql.checkData(0,0,datetime(2020,10,14,8,0,0,0)) #check the last data in the database tdSql.checkData(0,0,datetime(2020,10,14,8,0,0,0)) #check the last data in the database
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
def stop(self): def stop(self):
os.system('sudo timedatectl set-ntp on') os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
tdSql.close() tdSql.close()
tdLog.success("alter block manual check finish") tdLog.success("alter block manual check finish")
......
...@@ -34,34 +34,52 @@ class TDTestCase: ...@@ -34,34 +34,52 @@ class TDTestCase:
## change system time to 2020/10/20 ## change system time to 2020/10/20
os.system ('timedatectl set-ntp off') os.system ('timedatectl set-ntp off')
tdLog.sleep(10)
os.system ('timedatectl set-time 2020-10-20') os.system ('timedatectl set-time 2020-10-20')
#run taosdemo to insert data. one row per second from 2020/10/11 to 2020/10/20 #run taosdemo to insert data. one row per second from 2020/10/11 to 2020/10/20
#11 data files should be generated #11 data files should be generated
#vnode at TDinternal/community/sim/dnode1/data/vnode #vnode at TDinternal/community/sim/dnode1/data/vnode
os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_A.json") try:
commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data'] os.system(f"{binPath}taosdemo -f tools/taosdemoAllTest/manual_change_time_1_1_A.json")
result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8') commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data']
print(result.count('data')) result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8')
except BaseException:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
if result.count('data') != 11: if result.count('data') != 11:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
tdLog.exit('wrong number of files') tdLog.exit('wrong number of files')
else: else:
tdLog.debug("data file number correct") tdLog.debug("data file number correct")
tdSql.query('select first(ts) from stb_0') #check the last data in the database
tdSql.checkData(0,0,datetime(2020,10,11,0,0,0,0))
os.system ('timedatectl set-time 2020-10-25') try:
tdSql.query('select first(ts) from stb_0') #check the last data in the database
tdSql.checkData(0,0,datetime(2020,10,11,0,0,0,0))
except BaseException:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
#moves 5 days ahead to 2020/10/25 and restart taosd #moves 5 days ahead to 2020/10/25 and restart taosd
#4 oldest data file should be removed from tsdb/data #4 oldest data file should be removed from tsdb/data
#7 data file should be found #7 data file should be found
#vnode at TDinternal/community/sim/dnode1/data/vnode #vnode at TDinternal/community/sim/dnode1/data/vnode
os.system ('timedatectl set-time 2020-10-25')
tdDnodes.stop(1) try:
tdDnodes.start(1) os.system ('timedatectl set-time 2020-10-25')
tdSql.query('select first(ts) from stb_0') tdDnodes.stop(1)
tdSql.checkData(0,0,datetime(2020,10,14,8,0,0,0)) #check the last data in the database tdDnodes.start(1)
tdSql.query('select first(ts) from stb_0')
tdSql.checkData(0,0,datetime(2020,10,14,8,0,0,0)) #check the last data in the database
except BaseException:
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data'] commandArray = ['ls', '-l', f'{TDenginePath}/sim/dnode1/data/vnode/vnode2/tsdb/data']
result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8') result = subprocess.run(commandArray, stdout=subprocess.PIPE).stdout.decode('utf-8')
print(result.count('data')) print(result.count('data'))
...@@ -69,8 +87,8 @@ class TDTestCase: ...@@ -69,8 +87,8 @@ class TDTestCase:
tdLog.exit('wrong number of files') tdLog.exit('wrong number of files')
else: else:
tdLog.debug("data file number correct") tdLog.debug("data file number correct")
os.system('sudo timedatectl set-ntp on')
tdLog.sleep(10)
def stop(self): def stop(self):
os.system('sudo timedatectl set-ntp on') os.system('sudo timedatectl set-ntp on')
......
...@@ -25,6 +25,8 @@ python3 ./test.py -f insert/special_character_show.py ...@@ -25,6 +25,8 @@ python3 ./test.py -f insert/special_character_show.py
python3 bug2265.py python3 bug2265.py
python3 ./test.py -f insert/bug3654.py python3 ./test.py -f insert/bug3654.py
python3 ./test.py -f insert/insertDynamicColBeforeVal.py python3 ./test.py -f insert/insertDynamicColBeforeVal.py
python3 ./test.py -f insert/in_function.py
python3 ./test.py -f insert/modify_column.py
#table #table
python3 ./test.py -f table/alter_wal0.py python3 ./test.py -f table/alter_wal0.py
...@@ -258,8 +260,8 @@ python3 ./test.py -f client/client.py ...@@ -258,8 +260,8 @@ python3 ./test.py -f client/client.py
python3 ./test.py -f client/version.py python3 ./test.py -f client/version.py
python3 ./test.py -f client/alterDatabase.py python3 ./test.py -f client/alterDatabase.py
python3 ./test.py -f client/noConnectionErrorTest.py python3 ./test.py -f client/noConnectionErrorTest.py
#python3 test.py -f client/change_time_1_1.py # python3 test.py -f client/change_time_1_1.py
#python3 test.py -f client/change_time_1_2.py # python3 test.py -f client/change_time_1_2.py
# Misc # Misc
python3 testCompress.py python3 testCompress.py
...@@ -343,7 +345,7 @@ python3 test.py -f tools/taosdemoAllTest/taosdemoTestInsertWithJson.py ...@@ -343,7 +345,7 @@ python3 test.py -f tools/taosdemoAllTest/taosdemoTestInsertWithJson.py
python3 test.py -f tools/taosdemoAllTest/taosdemoTestQueryWithJson.py python3 test.py -f tools/taosdemoAllTest/taosdemoTestQueryWithJson.py
python3 ./test.py -f tag_lite/drop_auto_create.py python3 ./test.py -f tag_lite/drop_auto_create.py
python3 test.py -f insert/insert_before_use_db.py python3 test.py -f insert/insert_before_use_db.py
#python3 test.py -f alter/alter_keep.py python3 test.py -f alter/alter_keep.py
python3 test.py -f alter/alter_cacheLastRow.py python3 test.py -f alter/alter_cacheLastRow.py
python3 ./test.py -f query/querySession.py python3 ./test.py -f query/querySession.py
python3 test.py -f alter/alter_create_exception.py python3 test.py -f alter/alter_create_exception.py
......
...@@ -128,7 +128,14 @@ class TDTestCase: ...@@ -128,7 +128,14 @@ class TDTestCase:
def run(self): def run(self):
tdSql.prepare() tdSql.prepare()
self.insertAndCheckData() self.insertAndCheckData()
tdSql.execute("create table st(ts timestamp, c1 int, c2 int) tags(id int)")
tdSql.execute("insert into dev1(ts, c1) using st tags(1) values(now, 1)")
tdSql.error("select derivative(c1, 10s, 0) from (select c1 from st)")
tdSql.query("select diff(c1) from (select derivative(c1, 1s, 0) c1 from dev1)")
tdSql.checkRows(0)
def stop(self): def stop(self):
tdSql.close() tdSql.close()
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
...@@ -273,4 +273,18 @@ if $data03 != @20-09-15 00:00:00.000@ then ...@@ -273,4 +273,18 @@ if $data03 != @20-09-15 00:00:00.000@ then
return -1 return -1
endi endi
sql_error select derivative(val, 1s, 0) from (select c1 val from nest_tb0);
sql select diff(val) from (select c1 val from nest_tb0);
if $rows != 9999 then
return -1
endi
if $data00 != @70-01-01 08:00:00.000@ then
return -1
endi
if $data01 != 1 then
return -1
endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT system sh/exec.sh -n dnode1 -s stop -x SIGINT
\ No newline at end of file
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册