未验证 提交 520cae37 编写于 作者: H haojun Liao 提交者: GitHub

Merge branch 'develop' into release/s111

...@@ -34,6 +34,7 @@ matrix: ...@@ -34,6 +34,7 @@ matrix:
- psmisc - psmisc
- unixodbc - unixodbc
- unixodbc-dev - unixodbc-dev
- mono-complete
before_script: before_script:
- export TZ=Asia/Harbin - export TZ=Asia/Harbin
...@@ -59,6 +60,18 @@ matrix: ...@@ -59,6 +60,18 @@ matrix:
pip3 install guppy3 pip3 install guppy3
pip3 install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python3/ pip3 install --user ${TRAVIS_BUILD_DIR}/src/connector/python/linux/python3/
cd ${TRAVIS_BUILD_DIR}/tests/examples/C#/taosdemo
mcs -out:taosdemo *.cs || travis_terminate $?
pkill -TERM -x taosd
fuser -k -n tcp 6030
sleep 1
${TRAVIS_BUILD_DIR}/debug/build/bin/taosd -c ${TRAVIS_BUILD_DIR}/debug/test/cfg > /dev/null &
sleep 5
mono taosdemo -Q DEFAULT -y || travis_terminate $?
pkill -KILL -x taosd
fuser -k -n tcp 6030
sleep 1
cd ${TRAVIS_BUILD_DIR}/tests cd ${TRAVIS_BUILD_DIR}/tests
./test-all.sh smoke || travis_terminate $? ./test-all.sh smoke || travis_terminate $?
sleep 1 sleep 1
...@@ -74,6 +87,7 @@ matrix: ...@@ -74,6 +87,7 @@ matrix:
./valgrind-test.sh 2>&1 > mem-error-out.log ./valgrind-test.sh 2>&1 > mem-error-out.log
sleep 1 sleep 1
# Color setting # Color setting
RED='\033[0;31m' RED='\033[0;31m'
GREEN='\033[1;32m' GREEN='\033[1;32m'
...@@ -146,7 +160,7 @@ matrix: ...@@ -146,7 +160,7 @@ matrix:
branch_pattern: coverity_scan branch_pattern: coverity_scan
- os: linux - os: linux
dist: xenial dist: trusty
language: c language: c
git: git:
- depth: 1 - depth: 1
...@@ -156,8 +170,9 @@ matrix: ...@@ -156,8 +170,9 @@ matrix:
packages: packages:
- build-essential - build-essential
- cmake - cmake
- binutils-2.26
env: env:
- DESC="xenial build" - DESC="trusty/gcc-4.8/bintuils-2.26 build"
before_script: before_script:
- export TZ=Asia/Harbin - export TZ=Asia/Harbin
...@@ -168,7 +183,7 @@ matrix: ...@@ -168,7 +183,7 @@ matrix:
script: script:
- cmake .. > /dev/null - cmake .. > /dev/null
- make - export PATH=/usr/lib/binutils-2.26/bin:$PATH && make
- os: linux - os: linux
dist: bionic dist: bionic
...@@ -200,7 +215,7 @@ matrix: ...@@ -200,7 +215,7 @@ matrix:
dist: bionic dist: bionic
language: c language: c
compiler: clang compiler: clang
env: DESC="linux/clang build" env: DESC="arm64 linux/clang build"
git: git:
- depth: 1 - depth: 1
...@@ -238,7 +253,7 @@ matrix: ...@@ -238,7 +253,7 @@ matrix:
- build-essential - build-essential
- cmake - cmake
env: env:
- DESC="xenial build" - DESC="arm64 xenial build"
before_script: before_script:
- export TZ=Asia/Harbin - export TZ=Asia/Harbin
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
IF (CMAKE_VERSION VERSION_LESS 3.0) IF (CMAKE_VERSION VERSION_LESS 3.0)
PROJECT(TDengine CXX) PROJECT(TDengine CXX)
SET(PROJECT_VERSION_MAJOR "${LIB_MAJOR_VERSION}") SET(PROJECT_VERSION_MAJOR "${LIB_MAJOR_VERSION}")
......
...@@ -104,6 +104,10 @@ pipeline { ...@@ -104,6 +104,10 @@ pipeline {
find pytest -name '*'sql|xargs rm -rf find pytest -name '*'sql|xargs rm -rf
./test-all.sh p2 ./test-all.sh p2
date''' date'''
sh '''
cd ${WKC}/tests
./test-all.sh b4fq
'''
} }
} }
stage('test_b1') { stage('test_b1') {
......
...@@ -33,11 +33,17 @@ To build TDengine, use [CMake](https://cmake.org/) 3.5 or higher versions in the ...@@ -33,11 +33,17 @@ To build TDengine, use [CMake](https://cmake.org/) 3.5 or higher versions in the
## Install tools ## Install tools
### Ubuntu & Debian: ### Ubuntu 16.04 and above & Debian:
```bash ```bash
sudo apt-get install -y gcc cmake build-essential git sudo apt-get install -y gcc cmake build-essential git
``` ```
### Ubuntu 14.04:
```bash
sudo apt-get install -y gcc cmake3 build-essential git binutils-2.26
export PATH=/usr/lib/binutils-2.26/bin:$PATH
```
To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed. To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed.
To install openjdk-8: To install openjdk-8:
```bash ```bash
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
ADD_SUBDIRECTORY(zlib-1.2.11) ADD_SUBDIRECTORY(zlib-1.2.11)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
# MQTT-C build options # MQTT-C build options
option(MQTT_C_OpenSSL_SUPPORT "Build MQTT-C with OpenSSL support?" OFF) option(MQTT_C_OpenSSL_SUPPORT "Build MQTT-C with OpenSSL support?" OFF)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_WINDOWS) IF (TD_WINDOWS)
......
...@@ -4,6 +4,8 @@ ...@@ -4,6 +4,8 @@
TDengine采用关系型数据模型,需要建库、建表。因此对于一个具体的应用场景,需要考虑库的设计,超级表和普通表的设计。本节不讨论细致的语法规则,只介绍概念。 TDengine采用关系型数据模型,需要建库、建表。因此对于一个具体的应用场景,需要考虑库的设计,超级表和普通表的设计。本节不讨论细致的语法规则,只介绍概念。
关于数据建模请参考<a href="https://www.taosdata.com/blog/2020/11/11/1945.html">视频教程</a>
## 创建库 ## 创建库
不同类型的数据采集点往往具有不同的数据特征,包括数据采集频率的高低,数据保留时间的长短,副本的数目,数据块的大小,是否允许更新数据等等。为让各种场景下TDengine都能最大效率的工作,TDengine建议将不同数据特征的表创建在不同的库里,因为每个库可以配置不同的存储策略。创建一个库时,除SQL标准的选项外,应用还可以指定保留时长、副本数、内存块个数、时间精度、文件块里最大最小记录条数、是否压缩、一个数据文件覆盖的天数等多种参数。比如: 不同类型的数据采集点往往具有不同的数据特征,包括数据采集频率的高低,数据保留时间的长短,副本的数目,数据块的大小,是否允许更新数据等等。为让各种场景下TDengine都能最大效率的工作,TDengine建议将不同数据特征的表创建在不同的库里,因为每个库可以配置不同的存储策略。创建一个库时,除SQL标准的选项外,应用还可以指定保留时长、副本数、内存块个数、时间精度、文件块里最大最小记录条数、是否压缩、一个数据文件覆盖的天数等多种参数。比如:
...@@ -60,4 +62,3 @@ TDengine支持多列模型,只要物理量是一个数据采集点同时采集 ...@@ -60,4 +62,3 @@ TDengine支持多列模型,只要物理量是一个数据采集点同时采集
TDengine建议尽可能采用多列模型,因为插入效率以及存储效率更高。但对于有些场景,一个采集点的采集量的种类经常变化,这个时候,如果采用多列模型,就需要频繁修改超级表的结构定义,让应用变的复杂,这个时候,采用单列模型会显得简单。 TDengine建议尽可能采用多列模型,因为插入效率以及存储效率更高。但对于有些场景,一个采集点的采集量的种类经常变化,这个时候,如果采用多列模型,就需要频繁修改超级表的结构定义,让应用变的复杂,这个时候,采用单列模型会显得简单。
关于数据建模请参考<a href="https://www.taosdata.com/blog/2020/11/11/1945.html">视频教程</a>
...@@ -131,11 +131,26 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic ...@@ -131,11 +131,26 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic
``` ```
说明: 说明:
1) 表的第一个字段必须是TIMESTAMP,并且系统自动将其设为主键; 1) 表的第一个字段必须是TIMESTAMP,并且系统自动将其设为主键;
2) 表名最大长度为193 2) 表名最大长度为192
3) 表的每行长度不能超过16k个字符; 3) 表的每行长度不能超过16k个字符;
4) 子表名只能由字母、数字和下划线组成,且不能以数字开头 4) 子表名只能由字母、数字和下划线组成,且不能以数字开头
5) 使用数据类型binary或nchar,需指定其最长的字节数,如binary(20),表示20字节; 5) 使用数据类型binary或nchar,需指定其最长的字节数,如binary(20),表示20字节;
- **以超级表为模板创建数据表**
```mysql
CREATE TABLE [IF NOT EXISTS] tb_name USING stb_name TAGS (tag_value1, ...);
```
以指定的超级表为模板,指定 tags 的值来创建数据表。
- **批量创建数据表**
```mysql
CREATE TABLE [IF NOT EXISTS] tb_name1 USING stb_name TAGS (tag_value1, ...) tb_name2 USING stb_name TAGS (tag_value2, ...) ...;
```
以更快的速度批量创建大量数据表。(服务器端 2.0.14 及以上版本)
说明:批量建表方式要求数据表必须以超级表为模板。
- **删除数据表** - **删除数据表**
```mysql ```mysql
...@@ -148,7 +163,9 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic ...@@ -148,7 +163,9 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic
SHOW TABLES [LIKE tb_name_wildcar]; SHOW TABLES [LIKE tb_name_wildcar];
``` ```
显示当前数据库下的所有数据表信息。说明:可在like中使用通配符进行名称的匹配。 通配符匹配:1)’%’ (百分号)匹配0到任意个字符;2)’_’下划线匹配一个字符。 显示当前数据库下的所有数据表信息。
说明:可在like中使用通配符进行名称的匹配,这一通配符字符串最长不能超过24字节。
通配符匹配:1)’%’ (百分号)匹配0到任意个字符;2)’\_’下划线匹配一个字符。
- **在线修改显示字符宽度** - **在线修改显示字符宽度**
...@@ -169,7 +186,7 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic ...@@ -169,7 +186,7 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic
``` ```
说明: 说明:
1) 列的最大个数为1024,最小个数为2; 1) 列的最大个数为1024,最小个数为2;
2) 列名最大长度为65 2) 列名最大长度为64
- **表删除列** - **表删除列**
...@@ -190,7 +207,7 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic ...@@ -190,7 +207,7 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic
1) TAGS 列的数据类型不能是timestamp类型; 1) TAGS 列的数据类型不能是timestamp类型;
2) TAGS 列名不能与其他列名相同; 2) TAGS 列名不能与其他列名相同;
3) TAGS 列名不能为预留关键字; 3) TAGS 列名不能为预留关键字;
4) TAGS 最多允许128个,可以0个,总长度不超过16k个字符 4) TAGS 最多允许128个,至少1个,总长度不超过16k个字符。
- **删除超级表** - **删除超级表**
...@@ -263,33 +280,33 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic ...@@ -263,33 +280,33 @@ TDengine缺省的时间戳是毫秒精度,但通过修改配置参数enableMic
- **插入一条记录,数据对应到指定的列** - **插入一条记录,数据对应到指定的列**
```mysql ```mysql
INSERT INTO tb_name (field1_name, ...) VALUES(field1_value, ...) INSERT INTO tb_name (field1_name, ...) VALUES (field1_value, ...)
``` ```
向表tb_name中插入一条记录,数据对应到指定的列。SQL语句中没有出现的列,数据库将自动填充为NULL。主键(时间戳)不能为NULL。 向表tb_name中插入一条记录,数据对应到指定的列。SQL语句中没有出现的列,数据库将自动填充为NULL。主键(时间戳)不能为NULL。
- **插入多条记录** - **插入多条记录**
```mysql ```mysql
INSERT INTO tb_name VALUES (field1_value1, ...) (field1_value2, ...)...; INSERT INTO tb_name VALUES (field1_value1, ...) (field1_value2, ...) ...;
``` ```
向表tb_name中插入多条记录 向表tb_name中插入多条记录
- **按指定的列插入多条记录** - **按指定的列插入多条记录**
```mysql ```mysql
INSERT INTO tb_name (field1_name, ...) VALUES(field1_value1, ...) (field1_value2, ...) INSERT INTO tb_name (field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...) ...;
``` ```
向表tb_name中按指定的列插入多条记录 向表tb_name中按指定的列插入多条记录
- **向多个表插入多条记录** - **向多个表插入多条记录**
```mysql ```mysql
INSERT INTO tb1_name VALUES (field1_value1, ...)(field1_value2, ...)... INSERT INTO tb1_name VALUES (field1_value1, ...) (field1_value2, ...) ...
tb2_name VALUES (field1_value1, ...)(field1_value2, ...)...; tb2_name VALUES (field1_value1, ...) (field1_value2, ...) ...;
``` ```
同时向表tb1_name和tb2_name中分别插入多条记录 同时向表tb1_name和tb2_name中分别插入多条记录
- **同时向多个表按列插入多条记录** - **同时向多个表按列插入多条记录**
```mysql ```mysql
INSERT INTO tb1_name (tb1_field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...) INSERT INTO tb1_name (tb1_field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...) ...
tb2_name (tb2_field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...); tb2_name (tb2_field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...) ...;
``` ```
同时向表tb1_name和tb2_name中按列分别插入多条记录 同时向表tb1_name和tb2_name中按列分别插入多条记录
...@@ -318,23 +335,23 @@ SELECT select_expr [, select_expr ...] ...@@ -318,23 +335,23 @@ SELECT select_expr [, select_expr ...]
``` ```
说明:针对 insert 类型的 SQL 语句,我们采用的流式解析策略,在发现后面的错误之前,前面正确的部分SQL仍会执行。下面的sql中,insert语句是无效的,但是d1001仍会被创建。 说明:针对 insert 类型的 SQL 语句,我们采用的流式解析策略,在发现后面的错误之前,前面正确的部分SQL仍会执行。下面的sql中,insert语句是无效的,但是d1001仍会被创建。
```mysql ```mysql
taos> create table meters(ts timestamp, current float, voltage int, phase float) tags(location binary(30), groupId int); taos> CREATE TABLE meters(ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS(location BINARY(30), groupId INT);
Query OK, 0 row(s) affected (0.008245s) Query OK, 0 row(s) affected (0.008245s)
taos> show stables; taos> SHOW STABLES;
name | created_time | columns | tags | tables | name | created_time | columns | tags | tables |
============================================================================================ ============================================================================================
meters | 2020-08-06 17:50:27.831 | 4 | 2 | 0 | meters | 2020-08-06 17:50:27.831 | 4 | 2 | 0 |
Query OK, 1 row(s) in set (0.001029s) Query OK, 1 row(s) in set (0.001029s)
taos> show tables; taos> SHOW TABLES;
Query OK, 0 row(s) in set (0.000946s) Query OK, 0 row(s) in set (0.000946s)
taos> insert into d1001 using meters tags('Beijing.Chaoyang', 2); taos> INSERT INTO d1001 USING meters TAGS('Beijing.Chaoyang', 2);
DB error: invalid SQL: keyword VALUES or FILE required DB error: invalid SQL: keyword VALUES or FILE required
taos> show tables; taos> SHOW TABLES;
table_name | created_time | columns | stable_name | table_name | created_time | columns | stable_name |
====================================================================================================== ======================================================================================================
d1001 | 2020-08-06 17:52:02.097 | 4 | meters | d1001 | 2020-08-06 17:52:02.097 | 4 | meters |
...@@ -397,27 +414,41 @@ Query OK, 1 row(s) in set (0.020443s) ...@@ -397,27 +414,41 @@ Query OK, 1 row(s) in set (0.020443s)
在使用SQL函数来进行查询过程中,部分SQL函数支持通配符操作。其中的区别在于: 在使用SQL函数来进行查询过程中,部分SQL函数支持通配符操作。其中的区别在于:
```count(\*)```函数只返回一列。```first```、```last```、```last_row```函数则是返回全部列。 ```count(\*)```函数只返回一列。```first```、```last```、```last_row```函数则是返回全部列。
``` ```mysql
taos> select count(*) from d1001; taos> SELECT COUNT(*) FROM d1001;
count(*) | count(*) |
======================== ========================
3 | 3 |
Query OK, 1 row(s) in set (0.001035s) Query OK, 1 row(s) in set (0.001035s)
``` ```
``` ```mysql
taos> select first(*) from d1001; taos> SELECT FIRST(*) FROM d1001;
first(ts) | first(current) | first(voltage) | first(phase) | first(ts) | first(current) | first(voltage) | first(phase) |
========================================================================================= =========================================================================================
2018-10-03 14:38:05.000 | 10.30000 | 219 | 0.31000 | 2018-10-03 14:38:05.000 | 10.30000 | 219 | 0.31000 |
Query OK, 1 row(s) in set (0.000849s) Query OK, 1 row(s) in set (0.000849s)
``` ```
##### 标签列
从 2.0.14 版本开始,支持在普通表的查询中指定 _标签列_,且标签列的值会与普通列的数据一起返回。
```mysql
taos> SELECT location, groupid, current FROM d1001 LIMIT 2;
location | groupid | current |
======================================================================
Beijing.Chaoyang | 2 | 10.30000 |
Beijing.Chaoyang | 2 | 12.60000 |
Query OK, 2 row(s) in set (0.003112s)
```
注意:普通表的通配符 * 中并不包含 _标签列_。
#### 结果集列名 #### 结果集列名
```SELECT```子句中,如果不指定返回结果集合的列名,结果集列名称默认使用```SELECT```子句中的表达式名称作为列名称。此外,用户可使用```AS```来重命名返回结果集合中列的名称。例如: ```SELECT```子句中,如果不指定返回结果集合的列名,结果集列名称默认使用```SELECT```子句中的表达式名称作为列名称。此外,用户可使用```AS```来重命名返回结果集合中列的名称。例如:
``` ```mysql
taos> select ts, ts as primary_key_ts from d1001; taos> SELECT ts, ts AS primary_key_ts FROM d1001;
ts | primary_key_ts | ts | primary_key_ts |
==================================================== ====================================================
2018-10-03 14:38:05.000 | 2018-10-03 14:38:05.000 | 2018-10-03 14:38:05.000 | 2018-10-03 14:38:05.000 |
...@@ -434,53 +465,53 @@ Query OK, 3 row(s) in set (0.001191s) ...@@ -434,53 +465,53 @@ Query OK, 3 row(s) in set (0.001191s)
FROM关键字后面可以是若干个表(超级表)列表,也可以是子查询的结果。 FROM关键字后面可以是若干个表(超级表)列表,也可以是子查询的结果。
如果没有指定用户的当前数据库,可以在表名称之前使用数据库的名称来指定表所属的数据库。例如:```power.d1001``` 方式来跨库使用表。 如果没有指定用户的当前数据库,可以在表名称之前使用数据库的名称来指定表所属的数据库。例如:```power.d1001``` 方式来跨库使用表。
``` ```mysql
SELECT * FROM power.d1001; SELECT * FROM power.d1001;
------------------------------ ------------------------------
use power; USE power;
SELECT * FROM d1001; SELECT * FROM d1001;
``` ```
#### 特殊功能 #### 特殊功能
部分特殊的查询功能可以不使用FROM子句执行。获取当前所在的数据库 database() 部分特殊的查询功能可以不使用FROM子句执行。获取当前所在的数据库 database()
``` ```mysql
taos> SELECT database(); taos> SELECT DATABASE();
database() | database() |
================================= =================================
power | power |
Query OK, 1 row(s) in set (0.000079s) Query OK, 1 row(s) in set (0.000079s)
``` ```
如果登录的时候没有指定默认数据库,且没有使用```use```命令切换数据,则返回NULL。 如果登录的时候没有指定默认数据库,且没有使用```use```命令切换数据,则返回NULL。
``` ```mysql
taos> SELECT database(); taos> SELECT DATABASE();
database() | database() |
================================= =================================
NULL | NULL |
Query OK, 1 row(s) in set (0.000184s) Query OK, 1 row(s) in set (0.000184s)
``` ```
获取服务器和客户端版本号: 获取服务器和客户端版本号:
``` ```mysql
taos> SELECT client_version(); taos> SELECT CLIENT_VERSION();
client_version() | client_version() |
=================== ===================
2.0.0.0 | 2.0.0.0 |
Query OK, 1 row(s) in set (0.000070s) Query OK, 1 row(s) in set (0.000070s)
taos> SELECT server_version(); taos> SELECT SERVER_VERSION();
server_version() | server_version() |
=================== ===================
2.0.0.0 | 2.0.0.0 |
Query OK, 1 row(s) in set (0.000077s) Query OK, 1 row(s) in set (0.000077s)
``` ```
服务器状态检测语句。如果服务器正常,返回一个数字(例如 1)。如果服务器异常,返回error code。该SQL语法能兼容连接池对于TDengine状态的检查及第三方工具对于数据库服务器状态的检查。并可以避免出现使用了错误的心跳检测SQL语句导致的连接池连接丢失的问题。 服务器状态检测语句。如果服务器正常,返回一个数字(例如 1)。如果服务器异常,返回error code。该SQL语法能兼容连接池对于TDengine状态的检查及第三方工具对于数据库服务器状态的检查。并可以避免出现使用了错误的心跳检测SQL语句导致的连接池连接丢失的问题。
``` ```mysql
taos> SELECT server_status(); taos> SELECT SERVER_STATUS();
server_status() | server_status() |
================== ==================
1 | 1 |
Query OK, 1 row(s) in set (0.000074s) Query OK, 1 row(s) in set (0.000074s)
taos> SELECT server_status() as status; taos> SELECT SERVER_STATUS() AS status;
status | status |
============== ==============
1 | 1 |
...@@ -493,15 +524,15 @@ Query OK, 1 row(s) in set (0.000081s) ...@@ -493,15 +524,15 @@ Query OK, 1 row(s) in set (0.000081s)
#### 小技巧 #### 小技巧
获取一个超级表所有的子表名及相关的标签信息: 获取一个超级表所有的子表名及相关的标签信息:
``` ```mysql
SELECT TBNAME, location FROM meters; SELECT TBNAME, location FROM meters;
``` ```
统计超级表下辖子表数量: 统计超级表下辖子表数量:
``` ```mysql
SELECT COUNT(TBNAME) FROM meters; SELECT COUNT(TBNAME) FROM meters;
``` ```
以上两个查询均只支持在Where条件子句中添加针对标签(TAGS)的过滤条件。例如: 以上两个查询均只支持在Where条件子句中添加针对标签(TAGS)的过滤条件。例如:
``` ```mysql
taos> SELECT TBNAME, location FROM meters; taos> SELECT TBNAME, location FROM meters;
tbname | location | tbname | location |
================================================================== ==================================================================
...@@ -511,7 +542,7 @@ taos> SELECT TBNAME, location FROM meters; ...@@ -511,7 +542,7 @@ taos> SELECT TBNAME, location FROM meters;
d1001 | Beijing.Chaoyang | d1001 | Beijing.Chaoyang |
Query OK, 4 row(s) in set (0.000881s) Query OK, 4 row(s) in set (0.000881s)
taos> SELECT count(tbname) FROM meters WHERE groupId > 2; taos> SELECT COUNT(tbname) FROM meters WHERE groupId > 2;
count(tbname) | count(tbname) |
======================== ========================
2 | 2 |
...@@ -537,15 +568,15 @@ Query OK, 1 row(s) in set (0.001091s) ...@@ -537,15 +568,15 @@ Query OK, 1 row(s) in set (0.001091s)
| % | match with any char sequences | **`binary`** **`nchar`** | | % | match with any char sequences | **`binary`** **`nchar`** |
| _ | match with a single char | **`binary`** **`nchar`** | | _ | match with a single char | **`binary`** **`nchar`** |
1. 同时进行多个字段的范围过滤需要使用关键词AND进行连接不同的查询条件,暂不支持OR连接的不同列之间的查询过滤条件。 1. 同时进行多个字段的范围过滤,需要使用关键词 AND 来连接不同的查询条件,暂不支持 OR 连接的不同列之间的查询过滤条件。
2. 针对某一字段的过滤只支持单一时间区间过滤条件。但是针对其他的(普通)列或标签列,可以使用``` OR``` 条件进行组合条件的查询过滤。例如:((value > 20 and value < 30) OR (value < 12)) 。 2. 针对单一字段的过滤,如果是时间过滤条件,则一条语句中只支持设定一个;但针对其他的(普通)列或标签列,则可以使用``` OR``` 关键字进行组合条件的查询过滤。例如:((value > 20 and value < 30) OR (value < 12)) 。
### SQL 示例 ### SQL 示例
- 对于下面的例子,表tb1用以下语句创建 - 对于下面的例子,表tb1用以下语句创建
```mysql ```mysql
CREATE TABLE tb1 (ts timestamp, col1 int, col2 float, col3 binary(50)); CREATE TABLE tb1 (ts TIMESTAMP, col1 INT, col2 FLOAT, col3 BINARY(50));
``` ```
- 查询tb1刚过去的一个小时的所有记录 - 查询tb1刚过去的一个小时的所有记录
...@@ -563,7 +594,7 @@ Query OK, 1 row(s) in set (0.001091s) ...@@ -563,7 +594,7 @@ Query OK, 1 row(s) in set (0.001091s)
- 查询col1与col2的和,并取名complex, 时间大于2018-06-01 08:00:00.000, col2大于1.2,结果输出仅仅10条记录,从第5条开始 - 查询col1与col2的和,并取名complex, 时间大于2018-06-01 08:00:00.000, col2大于1.2,结果输出仅仅10条记录,从第5条开始
```mysql ```mysql
SELECT (col1 + col2) AS 'complex' FROM tb1 WHERE ts > '2018-06-01 08:00:00.000' and col2 > 1.2 LIMIT 10 OFFSET 5; SELECT (col1 + col2) AS 'complex' FROM tb1 WHERE ts > '2018-06-01 08:00:00.000' AND col2 > 1.2 LIMIT 10 OFFSET 5;
``` ```
- 查询过去10分钟的记录,col2的值大于3.14,并且将结果输出到文件 `/home/testoutpu.csv`. - 查询过去10分钟的记录,col2的值大于3.14,并且将结果输出到文件 `/home/testoutpu.csv`.
...@@ -590,13 +621,13 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -590,13 +621,13 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
示例: 示例:
```mysql ```mysql
taos> SELECT COUNT(*), COUNT(VOLTAGE) FROM meters; taos> SELECT COUNT(*), COUNT(voltage) FROM meters;
count(*) | count(voltage) | count(*) | count(voltage) |
================================================ ================================================
9 | 9 | 9 | 9 |
Query OK, 1 row(s) in set (0.004475s) Query OK, 1 row(s) in set (0.004475s)
taos> SELECT COUNT(*), COUNT(VOLTAGE) FROM d1001; taos> SELECT COUNT(*), COUNT(voltage) FROM d1001;
count(*) | count(voltage) | count(*) | count(voltage) |
================================================ ================================================
3 | 3 | 3 | 3 |
...@@ -620,7 +651,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -620,7 +651,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
11.466666751 | 220.444444444 | 0.293333333 | 11.466666751 | 220.444444444 | 0.293333333 |
Query OK, 1 row(s) in set (0.004135s) Query OK, 1 row(s) in set (0.004135s)
taos> SELECT AVG(current), AVG(voltage), AVG(phase) from d1001; taos> SELECT AVG(current), AVG(voltage), AVG(phase) FROM d1001;
avg(current) | avg(voltage) | avg(phase) | avg(current) | avg(voltage) | avg(phase) |
==================================================================================== ====================================================================================
11.733333588 | 219.333333333 | 0.316666673 | 11.733333588 | 219.333333333 | 0.316666673 |
...@@ -634,7 +665,6 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -634,7 +665,6 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
功能说明:时间加权平均函数。统计表/超级表中某列在一段时间内的时间加权平均。 功能说明:时间加权平均函数。统计表/超级表中某列在一段时间内的时间加权平均。
返回结果数据类型:双精度浮点数Double。 返回结果数据类型:双精度浮点数Double。
应用字段:不能应用在timestamp、binary、nchar、bool类型字段。 应用字段:不能应用在timestamp、binary、nchar、bool类型字段。
说明:时间加权平均(time weighted average, TWA)查询需要指定查询时间段的 _开始时间_ 和 _结束时间_ 。
适用于:表、超级表。 适用于:表、超级表。
- **SUM** - **SUM**
...@@ -648,13 +678,13 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -648,13 +678,13 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
示例: 示例:
```mysql ```mysql
taos> SELECT SUM(current), SUM(voltage), SUM(phase) from meters; taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM meters;
sum(current) | sum(voltage) | sum(phase) | sum(current) | sum(voltage) | sum(phase) |
================================================================================ ================================================================================
103.200000763 | 1984 | 2.640000001 | 103.200000763 | 1984 | 2.640000001 |
Query OK, 1 row(s) in set (0.001702s) Query OK, 1 row(s) in set (0.001702s)
taos> SELECT SUM(current), SUM(voltage), SUM(phase) from d1001; taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM d1001;
sum(current) | sum(voltage) | sum(phase) | sum(current) | sum(voltage) | sum(phase) |
================================================================================ ================================================================================
35.200000763 | 658 | 0.950000018 | 35.200000763 | 658 | 0.950000018 |
...@@ -753,7 +783,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -753,7 +783,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
功能说明:统计表/超级表中某列的值最先写入的非NULL值。 功能说明:统计表/超级表中某列的值最先写入的非NULL值。
返回结果数据类型:同应用的字段。 返回结果数据类型:同应用的字段。
应用字段:所有字段。 应用字段:所有字段。
说明:1)如果要返回各个列的首个(时间戳最小)非NULL值,可以使用FIRST(*);2) 如果结果集中的某列全部为NULL值,则该列的返回结果也是NULL;3) 如果结果集中所有列全部为NULL值,则不返回结果。 说明:1)如果要返回各个列的首个(时间戳最小)非NULL值,可以使用FIRST(\*);2) 如果结果集中的某列全部为NULL值,则该列的返回结果也是NULL;3) 如果结果集中所有列全部为NULL值,则不返回结果。
示例: 示例:
```mysql ```mysql
...@@ -777,7 +807,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ...@@ -777,7 +807,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
功能说明:统计表/超级表中某列的值最后写入的非NULL值。 功能说明:统计表/超级表中某列的值最后写入的非NULL值。
返回结果数据类型:同应用的字段。 返回结果数据类型:同应用的字段。
应用字段:所有字段。 应用字段:所有字段。
说明:1)如果要返回各个列的最后(时间戳最大)一个非NULL值,可以使用LAST(*);2)如果结果集中的某列全部为NULL值,则该列的返回结果也是NULL;如果结果集中所有列全部为NULL值,则不返回结果。 说明:1)如果要返回各个列的最后(时间戳最大)一个非NULL值,可以使用LAST(\*);2)如果结果集中的某列全部为NULL值,则该列的返回结果也是NULL;如果结果集中所有列全部为NULL值,则不返回结果。
示例: 示例:
```mysql ```mysql
...@@ -1004,15 +1034,15 @@ SELECT function_list FROM stb_name ...@@ -1004,15 +1034,15 @@ SELECT function_list FROM stb_name
**示例:** 智能电表的建表语句如下: **示例:** 智能电表的建表语句如下:
```mysql ```mysql
CREATE TABLE meters (ts timestamp, current float, voltage int, phase float) TAGS (location binary(64), groupId int); CREATE TABLE meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT);
``` ```
针对智能电表采集的数据,以10分钟为一个阶段,计算过去24小时的电流数据的平均值、最大值、电流的中位数、以及随着时间变化的电流走势拟合直线。如果没有计算值,用前一个非NULL值填充。 针对智能电表采集的数据,以10分钟为一个阶段,计算过去24小时的电流数据的平均值、最大值、电流的中位数、以及随着时间变化的电流走势拟合直线。如果没有计算值,用前一个非NULL值填充。
使用的查询语句如下: 使用的查询语句如下:
```mysql ```mysql
SELECT AVG(current),MAX(current),LEASTSQUARES(current, start_val, step_val), PERCENTILE(current, 50) FROM meters SELECT AVG(current), MAX(current), LEASTSQUARES(current, start_val, step_val), PERCENTILE(current, 50) FROM meters
WHERE TS>=NOW-1d WHERE ts>=NOW-1d
INTERVAL(10m) INTERVAL(10m)
FILL(PREV); FILL(PREV);
``` ```
......
...@@ -105,7 +105,7 @@ taosd -C ...@@ -105,7 +105,7 @@ taosd -C
- queryBufferSize: 为所有并发查询占用保留的内存大小。计算规则可以根据实际应用可能的最大并发数和表的数字相乘,再乘 170 。单位为字节。 - queryBufferSize: 为所有并发查询占用保留的内存大小。计算规则可以根据实际应用可能的最大并发数和表的数字相乘,再乘 170 。单位为字节。
- ratioOfQueryCores: 设置查询线程的最大数量。最小值0 表示只有1个查询线程;最大值2表示最大建立2倍CPU核数的查询线程。默认为1,表示最大和CPU核数相等的查询线程。该值可以为小数,即0.5表示最大建立CPU核数一半的查询线程。 - ratioOfQueryCores: 设置查询线程的最大数量。最小值0 表示只有1个查询线程;最大值2表示最大建立2倍CPU核数的查询线程。默认为1,表示最大和CPU核数相等的查询线程。该值可以为小数,即0.5表示最大建立CPU核数一半的查询线程。
**注意:**对于端口,TDengine会使用从serverPort起13个连续的TCP和UDP端口号,请务必在防火墙打开。因此如果是缺省配置,需要打开从60306042共13个端口,而且必须TCP和UDP都打开。 **注意:**对于端口,TDengine会使用从serverPort起13个连续的TCP和UDP端口号,请务必在防火墙打开。因此如果是缺省配置,需要打开从60306042共13个端口,而且必须TCP和UDP都打开。
不同应用场景的数据往往具有不同的数据特征,比如保留天数、副本数、采集频次、记录大小、采集点的数量、压缩等都可完全不同。为获得在存储上的最高效率,TDengine提供如下存储相关的系统配置参数: 不同应用场景的数据往往具有不同的数据特征,比如保留天数、副本数、采集频次、记录大小、采集点的数量、压缩等都可完全不同。为获得在存储上的最高效率,TDengine提供如下存储相关的系统配置参数:
......
...@@ -6,6 +6,8 @@ ...@@ -6,6 +6,8 @@
TDengine的集群管理极其简单,除添加和删除节点需要人工干预之外,其他全部是自动完成,最大程度的降低了运维的工作量。本章对集群管理的操作做详细的描述。 TDengine的集群管理极其简单,除添加和删除节点需要人工干预之外,其他全部是自动完成,最大程度的降低了运维的工作量。本章对集群管理的操作做详细的描述。
关于集群搭建请参考<a href="https://www.taosdata.com/blog/2020/11/11/1961.html">视频教程</a>
## 准备工作 ## 准备工作
**第零步**:规划集群所有物理节点的FQDN,将规划好的FQDN分别添加到每个物理节点的/etc/hostname;修改每个物理节点的/etc/hosts,将所有集群物理节点的IP与FQDN的对应添加好。【如部署了DNS,请联系网络管理员在DNS上做好相关配置】 **第零步**:规划集群所有物理节点的FQDN,将规划好的FQDN分别添加到每个物理节点的/etc/hostname;修改每个物理节点的/etc/hosts,将所有集群物理节点的IP与FQDN的对应添加好。【如部署了DNS,请联系网络管理员在DNS上做好相关配置】
...@@ -227,4 +229,3 @@ SHOW MNODES; ...@@ -227,4 +229,3 @@ SHOW MNODES;
TDengine提供一个执行程序tarbitrator, 找任何一台Linux服务器运行它即可。请点击[安装包下载](https://www.taosdata.com/cn/all-downloads/),在TDengine Arbitrator Linux一节中,选择适合的版本下载并安装。该程序对系统资源几乎没有要求,只需要保证有网络连接即可。该应用的命令行参数`-p`可以指定其对外服务的端口号,缺省是6042。配置每个taosd实例时,可以在配置文件taos.cfg里将参数arbitrator设置为arbitrator的End Point。如果该参数配置了,当副本数为偶数数,系统将自动连接配置的arbitrator。如果副本数为奇数,即使配置了arbitrator, 系统也不会去建立连接。 TDengine提供一个执行程序tarbitrator, 找任何一台Linux服务器运行它即可。请点击[安装包下载](https://www.taosdata.com/cn/all-downloads/),在TDengine Arbitrator Linux一节中,选择适合的版本下载并安装。该程序对系统资源几乎没有要求,只需要保证有网络连接即可。该应用的命令行参数`-p`可以指定其对外服务的端口号,缺省是6042。配置每个taosd实例时,可以在配置文件taos.cfg里将参数arbitrator设置为arbitrator的End Point。如果该参数配置了,当副本数为偶数数,系统将自动连接配置的arbitrator。如果副本数为奇数,即使配置了arbitrator, 系统也不会去建立连接。
关于集群搭建请参考<a href="https://www.taosdata.com/blog/2020/11/11/1961.html">视频教程</a>
...@@ -178,11 +178,11 @@ C/C++的API类似于MySQL的C API。应用程序使用时,需要包含TDengine ...@@ -178,11 +178,11 @@ C/C++的API类似于MySQL的C API。应用程序使用时,需要包含TDengine
获取客户端版本信息。 获取客户端版本信息。
- `TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, int port)` - `TAOS *taos_connect(const char *host, const char *user, const char *pass, const char *db, int port)`
创建数据库连接,初始化连接上下文。其中需要用户提供的参数包含: 创建数据库连接,初始化连接上下文。其中需要用户提供的参数包含:
- ip:TDengine管理主节点的IP地址 - host:TDengine管理主节点的FQDN
- user:用户名 - user:用户名
- pass:密码 - pass:密码
- db:数据库名字,如果用户没有提供,也可以正常连接,用户可以通过该连接创建新的数据库,如果用户提供了数据库名字,则说明该数据库用户已经创建好,缺省使用该数据库 - db:数据库名字,如果用户没有提供,也可以正常连接,用户可以通过该连接创建新的数据库,如果用户提供了数据库名字,则说明该数据库用户已经创建好,缺省使用该数据库
......
...@@ -72,9 +72,6 @@ ...@@ -72,9 +72,6 @@
# time interval of heart beat from shell to dnode, seconds # time interval of heart beat from shell to dnode, seconds
# shellActivityTimer 3 # shellActivityTimer 3
# time of keeping table meta data in cache, seconds
# tableMetaKeepTimer 7200
# minimum sliding window time, milli-second # minimum sliding window time, milli-second
# minSlidingTime 10 # minSlidingTime 10
......
...@@ -348,7 +348,7 @@ function set_ipAsFqdn() { ...@@ -348,7 +348,7 @@ function set_ipAsFqdn() {
} }
function local_fqdn_check() { function local_fqdn_check() {
#serverFqdn=$(hostname -f) #serverFqdn=$(hostname)
echo echo
echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}" echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}"
echo echo
...@@ -911,7 +911,7 @@ function install_TDengine() { ...@@ -911,7 +911,7 @@ function install_TDengine() {
## ==============================Main program starts from here============================ ## ==============================Main program starts from here============================
serverFqdn=$(hostname -f) serverFqdn=$(hostname)
if [ "$verType" == "server" ]; then if [ "$verType" == "server" ]; then
# Install server and client # Install server and client
if [ -x ${bin_dir}/taosd ]; then if [ -x ${bin_dir}/taosd ]; then
......
...@@ -345,7 +345,7 @@ function set_ipAsFqdn() { ...@@ -345,7 +345,7 @@ function set_ipAsFqdn() {
} }
function local_fqdn_check() { function local_fqdn_check() {
#serverFqdn=$(hostname -f) #serverFqdn=$(hostname)
echo echo
echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}" echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}"
echo echo
...@@ -881,7 +881,7 @@ function install_PowerDB() { ...@@ -881,7 +881,7 @@ function install_PowerDB() {
## ==============================Main program starts from here============================ ## ==============================Main program starts from here============================
serverFqdn=$(hostname -f) serverFqdn=$(hostname)
if [ "$verType" == "server" ]; then if [ "$verType" == "server" ]; then
# Install server and client # Install server and client
if [ -x ${bin_dir}/powerd ]; then if [ -x ${bin_dir}/powerd ]; then
......
...@@ -228,7 +228,7 @@ function set_ipAsFqdn() { ...@@ -228,7 +228,7 @@ function set_ipAsFqdn() {
} }
function local_fqdn_check() { function local_fqdn_check() {
#serverFqdn=$(hostname -f) #serverFqdn=$(hostname)
echo echo
echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}" echo -e -n "System hostname is: ${GREEN}$serverFqdn${NC}"
echo echo
...@@ -492,5 +492,5 @@ function install_TDengine() { ...@@ -492,5 +492,5 @@ function install_TDengine() {
## ==============================Main program starts from here============================ ## ==============================Main program starts from here============================
serverFqdn=$(hostname -f) serverFqdn=$(hostname)
install_TDengine install_TDengine
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
# Base compile # Base compile
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/mnode/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/mnode/inc)
......
...@@ -237,21 +237,21 @@ static bool bnCheckVgroupReady(SVgObj *pVgroup, SVnodeGid *pRmVnode) { ...@@ -237,21 +237,21 @@ static bool bnCheckVgroupReady(SVgObj *pVgroup, SVnodeGid *pRmVnode) {
bool isReady = false; bool isReady = false;
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SVnodeGid *pVnode = pVgroup->vnodeGid + i; SVnodeGid *pVnode = pVgroup->vnodeGid + i;
SDnodeObj *pDnode = pVnode->pDnode;
if (pVnode == pRmVnode) continue; if (pVnode == pRmVnode) continue;
int32_t vver = mnodeGetVgidVer(pVnode->vver); int32_t vver = mnodeGetVgidVer(pVnode->vver);
mTrace("vgId:%d, check vgroup status, vindex:%d dnode:%d status:%s role:%s vver:%d, rmvver:%d" , pVgroup->vgId, i, mTrace("vgId:%d, check vgroup status, vindex:%d dnode:%d status:%s role:%s vver:%d, rmvver:%d", pVgroup->vgId, i,
pVnode->dnodeId, dnodeStatus[pVnode->pDnode->status], syncRole[pVnode->role], vver, rmVnodeVer); pVnode->dnodeId, dnodeStatus[pDnode->status], syncRole[pVnode->role], vver, rmVnodeVer);
if (pVnode->pDnode->status == TAOS_DN_STATUS_DROPPING) continue; if (pDnode->status == TAOS_DN_STATUS_DROPPING) continue;
if (pVnode->pDnode->status == TAOS_DN_STATUS_OFFLINE) continue; if (pDnode->status == TAOS_DN_STATUS_OFFLINE) continue;
if (pVnode->role != TAOS_SYNC_ROLE_SLAVE && pVnode->role != TAOS_SYNC_ROLE_MASTER) continue; if (pVnode->role != TAOS_SYNC_ROLE_SLAVE && pVnode->role != TAOS_SYNC_ROLE_MASTER) continue;
if (rmVnodeVer == 0 || vver >= rmVnodeVer) { if (rmVnodeVer == 0 || vver >= rmVnodeVer) {
mInfo("vgId:%d, is ready for vindex:%d in dnode:%d status:%s role:%s vver:%d larger than rmvver:%d", pVgroup->vgId, i, mInfo("vgId:%d, is ready for vindex:%d in dnode:%d status:%s role:%s vver:%d larger than rmvver:%d",
pVnode->dnodeId, dnodeStatus[pVnode->pDnode->status], syncRole[pVnode->role], vver, rmVnodeVer); pVgroup->vgId, i, pVnode->dnodeId, dnodeStatus[pDnode->status], syncRole[pVnode->role], vver, rmVnodeVer);
isReady = true;
} }
isReady = true;
} }
return isReady; return isReady;
......
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
INCLUDE_DIRECTORIES(inc) INCLUDE_DIRECTORIES(inc)
......
...@@ -43,6 +43,11 @@ TAOS_ROW doSetResultRowData(SSqlObj *pSql); ...@@ -43,6 +43,11 @@ TAOS_ROW doSetResultRowData(SSqlObj *pSql);
char *getArithmeticInputSrc(void *param, const char *name, int32_t colId); char *getArithmeticInputSrc(void *param, const char *name, int32_t colId);
void tscLockByThread(int64_t *lockedBy);
void tscUnlockByThread(int64_t *lockedBy);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
......
...@@ -20,9 +20,6 @@ ...@@ -20,9 +20,6 @@
extern "C" { extern "C" {
#endif #endif
/*
* @date 2018/09/30
*/
#include "exception.h" #include "exception.h"
#include "os.h" #include "os.h"
#include "qExtbuffer.h" #include "qExtbuffer.h"
...@@ -216,7 +213,7 @@ STableMetaInfo* tscGetMetaInfo(SQueryInfo *pQueryInfo, int32_t tableIndex); ...@@ -216,7 +213,7 @@ STableMetaInfo* tscGetMetaInfo(SQueryInfo *pQueryInfo, int32_t tableIndex);
SQueryInfo *tscGetQueryInfoDetail(SSqlCmd* pCmd, int32_t subClauseIndex); SQueryInfo *tscGetQueryInfoDetail(SSqlCmd* pCmd, int32_t subClauseIndex);
SQueryInfo *tscGetQueryInfoDetailSafely(SSqlCmd *pCmd, int32_t subClauseIndex); SQueryInfo *tscGetQueryInfoDetailSafely(SSqlCmd *pCmd, int32_t subClauseIndex);
void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo, bool removeFromCache); void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo);
STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, const char* name, STableMeta* pTableMeta, STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, const char* name, STableMeta* pTableMeta,
SVgroupsInfo* vgroupList, SArray* pTagCols, SArray* pVgroupTables); SVgroupsInfo* vgroupList, SArray* pTagCols, SArray* pVgroupTables);
...@@ -276,7 +273,7 @@ void tscPrintSelectClause(SSqlObj* pSql, int32_t subClauseIndex); ...@@ -276,7 +273,7 @@ void tscPrintSelectClause(SSqlObj* pSql, int32_t subClauseIndex);
bool hasMoreVnodesToTry(SSqlObj *pSql); bool hasMoreVnodesToTry(SSqlObj *pSql);
bool hasMoreClauseToTry(SSqlObj* pSql); bool hasMoreClauseToTry(SSqlObj* pSql);
void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeFromCache); void tscFreeQueryInfo(SSqlCmd* pCmd);
void tscTryQueryNextVnode(SSqlObj *pSql, __async_cb_func_t fp); void tscTryQueryNextVnode(SSqlObj *pSql, __async_cb_func_t fp);
void tscAsyncQuerySingleRowForNextVnode(void *param, TAOS_RES *tres, int numOfRows); void tscAsyncQuerySingleRowForNextVnode(void *param, TAOS_RES *tres, int numOfRows);
...@@ -290,6 +287,14 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_ ...@@ -290,6 +287,14 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_
char* serializeTagData(STagData* pTagData, char* pMsg); char* serializeTagData(STagData* pTagData, char* pMsg);
int32_t copyTagData(STagData* dst, const STagData* src); int32_t copyTagData(STagData* dst, const STagData* src);
STableMeta* createSuperTableMeta(STableMetaMsg* pChild);
uint32_t tscGetTableMetaSize(STableMeta* pTableMeta);
CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta);
uint32_t tscGetTableMetaMaxSize();
int32_t tscCreateTableMetaFromCChildMeta(STableMeta* pChild, const char* name);
STableMeta* tscTableMetaClone(STableMeta* pTableMeta);
void* malloc_throw(size_t size); void* malloc_throw(size_t size);
void* calloc_throw(size_t nmemb, size_t size); void* calloc_throw(size_t nmemb, size_t size);
char* strdup_throw(const char* str); char* strdup_throw(const char* str);
......
...@@ -26,7 +26,7 @@ extern "C" { ...@@ -26,7 +26,7 @@ extern "C" {
#define VALIDNUMOFCOLS(x) ((x) >= TSDB_MIN_COLUMNS && (x) <= TSDB_MAX_COLUMNS) #define VALIDNUMOFCOLS(x) ((x) >= TSDB_MIN_COLUMNS && (x) <= TSDB_MAX_COLUMNS)
//struct SSchema; #define VALIDNUMOFTAGS(x) ((x) >= 0 && (x) <= TSDB_MAX_TAGS)
/** /**
* get the number of tags of this table * get the number of tags of this table
...@@ -91,7 +91,7 @@ SSchema* tscGetColumnSchemaById(STableMeta* pTableMeta, int16_t colId); ...@@ -91,7 +91,7 @@ SSchema* tscGetColumnSchemaById(STableMeta* pTableMeta, int16_t colId);
* @param numOfCols * @param numOfCols
* @return * @return
*/ */
bool isValidSchema(struct SSchema *pSchema, int32_t numOfCols); bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols, int32_t numOfTags);
/** /**
* get the schema for the "tbname" column. it is a built column * get the schema for the "tbname" column. it is a built column
...@@ -105,7 +105,10 @@ SSchema tscGetTbnameColumnSchema(); ...@@ -105,7 +105,10 @@ SSchema tscGetTbnameColumnSchema();
* @param size size of the table meta * @param size size of the table meta
* @return * @return
*/ */
STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size); STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg);
bool vgroupInfoIdentical(SNewVgroupInfo *pExisted, SVgroupMsg* src);
SNewVgroupInfo createNewVgroupInfo(SVgroupMsg *pVgroupMsg);
#ifdef __cplusplus #ifdef __cplusplus
} }
......
...@@ -56,23 +56,28 @@ typedef struct STableComInfo { ...@@ -56,23 +56,28 @@ typedef struct STableComInfo {
int32_t rowSize; int32_t rowSize;
} STableComInfo; } STableComInfo;
typedef struct SCorVgroupInfo { typedef struct SNewVgroupInfo {
int32_t version; int32_t vgId;
int8_t inUse; int8_t inUse;
int8_t numOfEps; int8_t numOfEps;
SEpAddr1 epAddr[TSDB_MAX_REPLICA]; SEpAddrMsg ep[TSDB_MAX_REPLICA];
} SCorVgroupInfo; } SNewVgroupInfo;
typedef struct CChildTableMeta {
int32_t vgId;
STableId id;
uint8_t tableType;
char sTableName[TSDB_TABLE_FNAME_LEN];
} CChildTableMeta;
typedef struct STableMeta { typedef struct STableMeta {
STableComInfo tableInfo; int32_t vgId;
STableId id;
uint8_t tableType; uint8_t tableType;
char sTableName[TSDB_TABLE_FNAME_LEN];
int16_t sversion; int16_t sversion;
int16_t tversion; int16_t tversion;
char sTableId[TSDB_TABLE_FNAME_LEN]; STableComInfo tableInfo;
int32_t vgId;
SCorVgroupInfo corVgroupInfo;
STableId id;
// union {int64_t stableUid; SSchema* schema;};
SSchema schema[]; // if the table is TSDB_CHILD_TABLE, schema is acquired by super table meta info SSchema schema[]; // if the table is TSDB_CHILD_TABLE, schema is acquired by super table meta info
} STableMeta; } STableMeta;
...@@ -171,7 +176,7 @@ typedef struct SParamInfo { ...@@ -171,7 +176,7 @@ typedef struct SParamInfo {
} SParamInfo; } SParamInfo;
typedef struct STableDataBlocks { typedef struct STableDataBlocks {
char tableId[TSDB_TABLE_FNAME_LEN]; char tableName[TSDB_TABLE_FNAME_LEN];
int8_t tsSource; // where does the UNIX timestamp come from, server or client int8_t tsSource; // where does the UNIX timestamp come from, server or client
bool ordered; // if current rows are ordered or not bool ordered; // if current rows are ordered or not
int64_t vgId; // virtual group id int64_t vgId; // virtual group id
...@@ -249,7 +254,7 @@ typedef struct { ...@@ -249,7 +254,7 @@ typedef struct {
int8_t submitSchema; // submit block is built with table schema int8_t submitSchema; // submit block is built with table schema
STagData tagData; // NOTE: pTagData->data is used as a variant length array STagData tagData; // NOTE: pTagData->data is used as a variant length array
STableMeta **pTableMetaList; // all involved tableMeta list of current insert sql statement. char **pTableNameList; // all involved tableMeta list of current insert sql statement.
int32_t numOfTables; int32_t numOfTables;
SHashObj *pTableBlockHashList; // data block for each table SHashObj *pTableBlockHashList; // data block for each table
...@@ -295,8 +300,8 @@ typedef struct STscObj { ...@@ -295,8 +300,8 @@ typedef struct STscObj {
void * pTimer; void * pTimer;
char user[TSDB_USER_LEN]; char user[TSDB_USER_LEN];
char pass[TSDB_KEY_LEN]; char pass[TSDB_KEY_LEN];
char acctId[TSDB_ACCT_LEN]; char acctId[TSDB_ACCT_ID_LEN];
char db[TSDB_ACCT_LEN + TSDB_DB_NAME_LEN]; char db[TSDB_ACCT_ID_LEN + TSDB_DB_NAME_LEN];
char sversion[TSDB_VERSION_LEN]; char sversion[TSDB_VERSION_LEN];
char writeAuth : 1; char writeAuth : 1;
char superAuth : 1; char superAuth : 1;
...@@ -342,6 +347,11 @@ typedef struct SSqlObj { ...@@ -342,6 +347,11 @@ typedef struct SSqlObj {
SSubqueryState subState; SSubqueryState subState;
struct SSqlObj **pSubs; struct SSqlObj **pSubs;
int64_t metaRid;
int64_t svgroupRid;
int64_t squeryLock;
struct SSqlObj *prev, *next; struct SSqlObj *prev, *next;
int64_t self; int64_t self;
} SSqlObj; } SSqlObj;
...@@ -386,7 +396,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet); ...@@ -386,7 +396,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet);
int tscProcessSql(SSqlObj *pSql); int tscProcessSql(SSqlObj *pSql);
int tscRenewTableMeta(SSqlObj *pSql, int32_t tableIndex); int tscRenewTableMeta(SSqlObj *pSql, int32_t tableIndex);
void tscQueueAsyncRes(SSqlObj *pSql); void tscAsyncResultOnError(SSqlObj *pSql);
void tscQueueAsyncError(void(*fp), void *param, int32_t code); void tscQueueAsyncError(void(*fp), void *param, int32_t code);
...@@ -400,7 +410,7 @@ void tscRestoreSQLFuncForSTableQuery(SQueryInfo *pQueryInfo); ...@@ -400,7 +410,7 @@ void tscRestoreSQLFuncForSTableQuery(SQueryInfo *pQueryInfo);
int32_t tscCreateResPointerInfo(SSqlRes *pRes, SQueryInfo *pQueryInfo); int32_t tscCreateResPointerInfo(SSqlRes *pRes, SQueryInfo *pQueryInfo);
void tscSetResRawPtr(SSqlRes* pRes, SQueryInfo* pQueryInfo); void tscSetResRawPtr(SSqlRes* pRes, SQueryInfo* pQueryInfo);
void tscResetSqlCmdObj(SSqlCmd *pCmd, bool removeFromCache); void tscResetSqlCmdObj(SSqlCmd *pCmd);
/** /**
* free query result of the sql object * free query result of the sql object
...@@ -414,7 +424,6 @@ void tscFreeSqlResult(SSqlObj *pSql); ...@@ -414,7 +424,6 @@ void tscFreeSqlResult(SSqlObj *pSql);
*/ */
void tscFreeSqlObj(SSqlObj *pSql); void tscFreeSqlObj(SSqlObj *pSql);
void tscFreeRegisteredSqlObj(void *pSql); void tscFreeRegisteredSqlObj(void *pSql);
void tscFreeTableMetaHelper(void *pTableMeta);
void tscCloseTscObj(void *pObj); void tscCloseTscObj(void *pObj);
...@@ -456,7 +465,7 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField ...@@ -456,7 +465,7 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField
} else { } else {
assert(bytes == tDataTypeDesc[type].nSize); assert(bytes == tDataTypeDesc[type].nSize);
pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : (unsigned char*)&pInfo->pSqlExpr->param[1].i64Key; pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : (unsigned char*)&pInfo->pSqlExpr->param[1].i64;
pRes->length[columnIndex] = bytes; pRes->length[columnIndex] = bytes;
} }
} else { } else {
...@@ -479,7 +488,9 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField ...@@ -479,7 +488,9 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField
} }
} }
extern SCacheObj *tscMetaCache; extern int32_t sentinel;
extern SHashObj *tscVgroupMap;
extern SHashObj *tscTableMetaInfo;
extern int tscObjRef; extern int tscObjRef;
extern void *tscTmr; extern void *tscTmr;
......
...@@ -18,7 +18,6 @@ ...@@ -18,7 +18,6 @@
#include "tnote.h" #include "tnote.h"
#include "trpc.h" #include "trpc.h"
#include "tcache.h"
#include "tscLog.h" #include "tscLog.h"
#include "tscSubquery.h" #include "tscSubquery.h"
#include "tscLocalMerge.h" #include "tscLocalMerge.h"
...@@ -57,7 +56,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para ...@@ -57,7 +56,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para
if (pSql->sqlstr == NULL) { if (pSql->sqlstr == NULL) {
tscError("%p failed to malloc sql string buffer", pSql); tscError("%p failed to malloc sql string buffer", pSql);
pSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY; pSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -71,7 +70,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para ...@@ -71,7 +70,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
pSql->res.code = code; pSql->res.code = code;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -96,7 +95,7 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa ...@@ -96,7 +95,7 @@ void taos_query_a(TAOS *taos, const char *sqlstr, __async_cb_func_t fp, void *pa
return; return;
} }
nPrintTsc(sqlstr); nPrintTsc("%s", sqlstr);
SSqlObj *pSql = (SSqlObj *)calloc(1, sizeof(SSqlObj)); SSqlObj *pSql = (SSqlObj *)calloc(1, sizeof(SSqlObj));
if (pSql == NULL) { if (pSql == NULL) {
...@@ -166,7 +165,7 @@ static void tscProcessAsyncRetrieveImpl(void *param, TAOS_RES *tres, int numOfRo ...@@ -166,7 +165,7 @@ static void tscProcessAsyncRetrieveImpl(void *param, TAOS_RES *tres, int numOfRo
pRes->code = numOfRows; pRes->code = numOfRows;
} }
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -217,7 +216,7 @@ void taos_fetch_rows_a(TAOS_RES *taosa, __async_cb_func_t fp, void *param) { ...@@ -217,7 +216,7 @@ void taos_fetch_rows_a(TAOS_RES *taosa, __async_cb_func_t fp, void *param) {
pRes->code = TSDB_CODE_TSC_INVALID_QHANDLE; pRes->code = TSDB_CODE_TSC_INVALID_QHANDLE;
pSql->param = param; pSql->param = param;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -280,7 +279,7 @@ void taos_fetch_row_a(TAOS_RES *taosa, void (*fp)(void *, TAOS_RES *, TAOS_ROW), ...@@ -280,7 +279,7 @@ void taos_fetch_row_a(TAOS_RES *taosa, void (*fp)(void *, TAOS_RES *, TAOS_ROW),
pSql->param = param; pSql->param = param;
pRes->code = TSDB_CODE_TSC_INVALID_QHANDLE; pRes->code = TSDB_CODE_TSC_INVALID_QHANDLE;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -366,7 +365,7 @@ static void tscProcessAsyncError(SSchedMsg *pMsg) { ...@@ -366,7 +365,7 @@ static void tscProcessAsyncError(SSchedMsg *pMsg) {
void (*fp)() = pMsg->ahandle; void (*fp)() = pMsg->ahandle;
terrno = *(int32_t*) pMsg->msg; terrno = *(int32_t*) pMsg->msg;
tfree(pMsg->msg); tfree(pMsg->msg);
(*fp)(pMsg->thandle, NULL, *(int32_t*)pMsg->msg); (*fp)(pMsg->thandle, NULL, terrno);
} }
void tscQueueAsyncError(void(*fp), void *param, int32_t code) { void tscQueueAsyncError(void(*fp), void *param, int32_t code) {
...@@ -382,14 +381,14 @@ void tscQueueAsyncError(void(*fp), void *param, int32_t code) { ...@@ -382,14 +381,14 @@ void tscQueueAsyncError(void(*fp), void *param, int32_t code) {
} }
void tscQueueAsyncRes(SSqlObj *pSql) { void tscAsyncResultOnError(SSqlObj *pSql) {
if (pSql == NULL || pSql->signature != pSql) { if (pSql == NULL || pSql->signature != pSql) {
tscDebug("%p SqlObj is freed, not add into queue async res", pSql); tscDebug("%p SqlObj is freed, not add into queue async res", pSql);
return; return;
} }
assert(pSql->res.code != TSDB_CODE_SUCCESS); assert(pSql->res.code != TSDB_CODE_SUCCESS);
tscError("%p add into queued async res, code:%s", pSql, tstrerror(pSql->res.code)); tscError("%p invoke user specified function due to error occured, code:%s", pSql, tstrerror(pSql->res.code));
SSqlRes *pRes = &pSql->res; SSqlRes *pRes = &pSql->res;
if (pSql->fp == NULL || pSql->fetchFp == NULL){ if (pSql->fp == NULL || pSql->fetchFp == NULL){
...@@ -403,8 +402,10 @@ void tscQueueAsyncRes(SSqlObj *pSql) { ...@@ -403,8 +402,10 @@ void tscQueueAsyncRes(SSqlObj *pSql) {
int tscSendMsgToServer(SSqlObj *pSql); int tscSendMsgToServer(SSqlObj *pSql);
void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
SSqlObj *pSql = (SSqlObj *)param; SSqlObj* pSql = (SSqlObj*)taosAcquireRef(tscObjRef, (int64_t)param);
if (pSql == NULL || pSql->signature != pSql) return; if (pSql == NULL) return;
assert(pSql->signature == pSql && (int64_t)param == pSql->self);
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
SSqlRes *pRes = &pSql->res; SSqlRes *pRes = &pSql->res;
...@@ -422,14 +423,15 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -422,14 +423,15 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
// check if it is a sub-query of super table query first, if true, enter another routine // check if it is a sub-query of super table query first, if true, enter another routine
if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY|TSDB_QUERY_TYPE_TAG_FILTER_QUERY))) { if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY|TSDB_QUERY_TYPE_SUBQUERY|TSDB_QUERY_TYPE_TAG_FILTER_QUERY))) {
tscDebug("%p update table meta in local cache, continue to process sql and send the corresponding query", pSql); tscDebug("%p update local table meta, continue to process sql and send the corresponding query", pSql);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
code = tscGetTableMeta(pSql, pTableMetaInfo); code = tscGetTableMeta(pSql, pTableMetaInfo);
assert(code == TSDB_CODE_TSC_ACTION_IN_PROGRESS || code == TSDB_CODE_SUCCESS); assert(code == TSDB_CODE_TSC_ACTION_IN_PROGRESS || code == TSDB_CODE_SUCCESS);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} }
...@@ -437,16 +439,18 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -437,16 +439,18 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
// tscProcessSql can add error into async res // tscProcessSql can add error into async res
tscProcessSql(pSql); tscProcessSql(pSql);
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else { // continue to process normal async query } else { // continue to process normal async query
if (pCmd->parseFinished) { if (pCmd->parseFinished) {
tscDebug("%p update table meta in local cache, continue to process sql and send corresponding query", pSql); tscDebug("%p update local table meta, continue to process sql and send corresponding query", pSql);
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
code = tscGetTableMeta(pSql, pTableMetaInfo); code = tscGetTableMeta(pSql, pTableMetaInfo);
assert(code == TSDB_CODE_TSC_ACTION_IN_PROGRESS || code == TSDB_CODE_SUCCESS); assert(code == TSDB_CODE_TSC_ACTION_IN_PROGRESS || code == TSDB_CODE_SUCCESS);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} }
...@@ -455,10 +459,11 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -455,10 +459,11 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
if (pCmd->command == TSDB_SQL_SELECT) { if (pCmd->command == TSDB_SQL_SELECT) {
tscDebug("%p redo parse sql string and proceed", pSql); tscDebug("%p redo parse sql string and proceed", pSql);
pCmd->parseFinished = false; pCmd->parseFinished = false;
tscResetSqlCmdObj(pCmd, false); tscResetSqlCmdObj(pCmd);
code = tsParseSql(pSql, true); code = tsParseSql(pSql, true);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else if (code != TSDB_CODE_SUCCESS) { } else if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
...@@ -469,12 +474,14 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -469,12 +474,14 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
tscProcessSql(pSql); tscProcessSql(pSql);
} }
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else { } else {
tscDebug("%p continue parse sql after get table meta", pSql); tscDebug("%p continue parse sql after get table meta", pSql);
code = tsParseSql(pSql, false); code = tsParseSql(pSql, false);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else if (code != TSDB_CODE_SUCCESS) { } else if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
...@@ -484,12 +491,14 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -484,12 +491,14 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
code = tscGetTableMeta(pSql, pTableMetaInfo); code = tscGetTableMeta(pSql, pTableMetaInfo);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else { } else {
assert(code == TSDB_CODE_SUCCESS); assert(code == TSDB_CODE_SUCCESS);
} }
(*pSql->fp)(pSql->param, pSql, code); (*pSql->fp)(pSql->param, pSql, code);
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} }
...@@ -502,6 +511,7 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -502,6 +511,7 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
code = tscGetTableMeta(pSql, pTableMetaInfo); code = tscGetTableMeta(pSql, pTableMetaInfo);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else if (code != TSDB_CODE_SUCCESS) { } else if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
...@@ -510,6 +520,7 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -510,6 +520,7 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
code = tscGetSTableVgroupInfo(pSql, pCmd->clauseIndex); code = tscGetSTableVgroupInfo(pSql, pCmd->clauseIndex);
if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} else if (code != TSDB_CODE_SUCCESS) { } else if (code != TSDB_CODE_SUCCESS) {
goto _error; goto _error;
...@@ -522,16 +533,23 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { ...@@ -522,16 +533,23 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
} }
(*pSql->fp)(pSql->param, pSql, code); (*pSql->fp)(pSql->param, pSql, code);
taosReleaseRef(tscObjRef, pSql->self);
return; return;
} }
tscDoQuery(pSql); tscDoQuery(pSql);
taosReleaseRef(tscObjRef, pSql->self);
return; return;
_error: _error:
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
pSql->res.code = code; pSql->res.code = code;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
taosReleaseRef(tscObjRef, pSql->self);
} }
...@@ -17,7 +17,6 @@ ...@@ -17,7 +17,6 @@
#include "taosmsg.h" #include "taosmsg.h"
#include "taosdef.h" #include "taosdef.h"
#include "tcache.h"
#include "tname.h" #include "tname.h"
#include "tscLog.h" #include "tscLog.h"
#include "tscUtil.h" #include "tscUtil.h"
...@@ -205,7 +204,7 @@ static int32_t tscProcessDescribeTable(SSqlObj *pSql) { ...@@ -205,7 +204,7 @@ static int32_t tscProcessDescribeTable(SSqlObj *pSql) {
assert(tscGetMetaInfo(pQueryInfo, 0)->pTableMeta != NULL); assert(tscGetMetaInfo(pQueryInfo, 0)->pTableMeta != NULL);
const int32_t NUM_OF_DESC_TABLE_COLUMNS = 4; const int32_t NUM_OF_DESC_TABLE_COLUMNS = 4;
const int32_t TYPE_COLUMN_LENGTH = 16; const int32_t TYPE_COLUMN_LENGTH = 20;
const int32_t NOTE_COLUMN_MIN_LENGTH = 8; const int32_t NOTE_COLUMN_MIN_LENGTH = 8;
int32_t noteFieldLen = NOTE_COLUMN_MIN_LENGTH; int32_t noteFieldLen = NOTE_COLUMN_MIN_LENGTH;
...@@ -273,7 +272,7 @@ void tscSCreateCallBack(void *param, TAOS_RES *tres, int code) { ...@@ -273,7 +272,7 @@ void tscSCreateCallBack(void *param, TAOS_RES *tres, int code) {
if (pRes->code != TSDB_CODE_SUCCESS) { if (pRes->code != TSDB_CODE_SUCCESS) {
taos_free_result(pSql); taos_free_result(pSql);
free(builder); free(builder);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -291,7 +290,7 @@ void tscSCreateCallBack(void *param, TAOS_RES *tres, int code) { ...@@ -291,7 +290,7 @@ void tscSCreateCallBack(void *param, TAOS_RES *tres, int code) {
if (pRes->code == TSDB_CODE_SUCCESS) { if (pRes->code == TSDB_CODE_SUCCESS) {
(*pParentSql->fp)(pParentSql->param, pParentSql, code); (*pParentSql->fp)(pParentSql->param, pParentSql, code);
} else { } else {
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
} }
} }
} }
...@@ -571,7 +570,7 @@ static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, ch ...@@ -571,7 +570,7 @@ static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, ch
char fullName[TSDB_TABLE_FNAME_LEN * 2] = {0}; char fullName[TSDB_TABLE_FNAME_LEN * 2] = {0};
extractDBName(pTableMetaInfo->name, fullName); extractDBName(pTableMetaInfo->name, fullName);
extractTableName(pMeta->sTableId, param->sTableName); extractTableName(pMeta->sTableName, param->sTableName);
snprintf(fullName + strlen(fullName), TSDB_TABLE_FNAME_LEN - strlen(fullName), ".%s", param->sTableName); snprintf(fullName + strlen(fullName), TSDB_TABLE_FNAME_LEN - strlen(fullName), ".%s", param->sTableName);
extractTableName(pTableMetaInfo->name, param->buf); extractTableName(pTableMetaInfo->name, param->buf);
...@@ -901,7 +900,7 @@ int tscProcessLocalCmd(SSqlObj *pSql) { ...@@ -901,7 +900,7 @@ int tscProcessLocalCmd(SSqlObj *pSql) {
} else if (pCmd->command == TSDB_SQL_SHOW_CREATE_DATABASE) { } else if (pCmd->command == TSDB_SQL_SHOW_CREATE_DATABASE) {
pRes->code = tscProcessShowCreateDatabase(pSql); pRes->code = tscProcessShowCreateDatabase(pSql);
} else if (pCmd->command == TSDB_SQL_RESET_CACHE) { } else if (pCmd->command == TSDB_SQL_RESET_CACHE) {
taosCacheEmpty(tscMetaCache); taosHashEmpty(tscTableMetaInfo);
pRes->code = TSDB_CODE_SUCCESS; pRes->code = TSDB_CODE_SUCCESS;
} else if (pCmd->command == TSDB_SQL_SERV_VERSION) { } else if (pCmd->command == TSDB_SQL_SERV_VERSION) {
pRes->code = tscProcessServerVer(pSql); pRes->code = tscProcessServerVer(pSql);
...@@ -925,7 +924,7 @@ int tscProcessLocalCmd(SSqlObj *pSql) { ...@@ -925,7 +924,7 @@ int tscProcessLocalCmd(SSqlObj *pSql) {
(*pSql->fp)(pSql->param, pSql, code); (*pSql->fp)(pSql->param, pSql, code);
} else if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS){ } else if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS){
} else { } else {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
return code; return code;
} }
...@@ -89,17 +89,17 @@ static void tscInitSqlContext(SSqlCmd *pCmd, SLocalReducer *pReducer, tOrderDesc ...@@ -89,17 +89,17 @@ static void tscInitSqlContext(SSqlCmd *pCmd, SLocalReducer *pReducer, tOrderDesc
pCtx->startOffset = 0; pCtx->startOffset = 0;
pCtx->size = 1; pCtx->size = 1;
pCtx->hasNull = true; pCtx->hasNull = true;
pCtx->currentStage = SECONDARY_STAGE_MERGE; pCtx->currentStage = MERGE_STAGE;
// for top/bottom function, the output of timestamp is the first column // for top/bottom function, the output of timestamp is the first column
int32_t functionId = pExpr->functionId; int32_t functionId = pExpr->functionId;
if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM || functionId == TSDB_FUNC_DIFF) { if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM || functionId == TSDB_FUNC_DIFF) {
pCtx->ptsOutputBuf = pReducer->pCtx[0].aOutputBuf; pCtx->ptsOutputBuf = pReducer->pCtx[0].aOutputBuf;
pCtx->param[2].i64Key = pQueryInfo->order.order; pCtx->param[2].i64 = pQueryInfo->order.order;
pCtx->param[2].nType = TSDB_DATA_TYPE_BIGINT; pCtx->param[2].nType = TSDB_DATA_TYPE_BIGINT;
pCtx->param[1].i64Key = pQueryInfo->order.orderColId; pCtx->param[1].i64 = pQueryInfo->order.orderColId;
} else if (functionId == TSDB_FUNC_APERCT) { } else if (functionId == TSDB_FUNC_APERCT) {
pCtx->param[0].i64Key = pExpr->param[0].i64Key; pCtx->param[0].i64 = pExpr->param[0].i64;
pCtx->param[0].nType = pExpr->param[0].nType; pCtx->param[0].nType = pExpr->param[0].nType;
} }
...@@ -1064,10 +1064,10 @@ static void doExecuteSecondaryMerge(SSqlCmd *pCmd, SLocalReducer *pLocalReducer, ...@@ -1064,10 +1064,10 @@ static void doExecuteSecondaryMerge(SSqlCmd *pCmd, SLocalReducer *pLocalReducer,
} }
} else if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM) { } else if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM) {
SSqlExpr *pExpr = tscSqlExprGet(pQueryInfo, j); SSqlExpr *pExpr = tscSqlExprGet(pQueryInfo, j);
pCtx->param[0].i64Key = pExpr->param[0].i64Key; pCtx->param[0].i64 = pExpr->param[0].i64;
} }
pCtx->currentStage = SECONDARY_STAGE_MERGE; pCtx->currentStage = MERGE_STAGE;
if (needInit) { if (needInit) {
aAggs[pCtx->functionId].init(pCtx); aAggs[pCtx->functionId].init(pCtx);
...@@ -1080,7 +1080,7 @@ static void doExecuteSecondaryMerge(SSqlCmd *pCmd, SLocalReducer *pLocalReducer, ...@@ -1080,7 +1080,7 @@ static void doExecuteSecondaryMerge(SSqlCmd *pCmd, SLocalReducer *pLocalReducer,
continue; continue;
} }
aAggs[functionId].distSecondaryMergeFunc(&pLocalReducer->pCtx[j]); aAggs[functionId].mergeFunc(&pLocalReducer->pCtx[j]);
} }
} }
...@@ -1647,7 +1647,7 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_ ...@@ -1647,7 +1647,7 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_
// calculate the result from several other columns // calculate the result from several other columns
if (pSup->pArithExprInfo != NULL) { if (pSup->pArithExprInfo != NULL) {
arithSup.pArithExpr = pSup->pArithExprInfo; arithSup.pArithExpr = pSup->pArithExprInfo;
tExprTreeCalcTraverse(arithSup.pArithExpr->pExpr, (int32_t) pOutput->num, pbuf + pOutput->num*offset, &arithSup, TSDB_ORDER_ASC, getArithmeticInputSrc); arithmeticTreeTraverse(arithSup.pArithExpr->pExpr, (int32_t) pOutput->num, pbuf + pOutput->num*offset, &arithSup, TSDB_ORDER_ASC, getArithmeticInputSrc);
} else { } else {
SSqlExpr* pExpr = pSup->pSqlExpr; SSqlExpr* pExpr = pSup->pSqlExpr;
memcpy(pbuf + pOutput->num * offset, pExpr->offset * pOutput->num + pOutput->data, (size_t)(pExpr->resBytes * pOutput->num)); memcpy(pbuf + pOutput->num * offset, pExpr->offset * pOutput->num + pOutput->data, (size_t)(pExpr->resBytes * pOutput->num));
......
...@@ -20,6 +20,7 @@ ...@@ -20,6 +20,7 @@
#include "os.h" #include "os.h"
#include "ttype.h"
#include "hash.h" #include "hash.h"
#include "tscUtil.h" #include "tscUtil.h"
#include "tschemautil.h" #include "tschemautil.h"
...@@ -40,46 +41,9 @@ enum { ...@@ -40,46 +41,9 @@ enum {
static int32_t tscAllocateMemIfNeed(STableDataBlocks *pDataBlock, int32_t rowSize, int32_t * numOfRows); static int32_t tscAllocateMemIfNeed(STableDataBlocks *pDataBlock, int32_t rowSize, int32_t * numOfRows);
static int32_t tscToInteger(SStrToken *pToken, int64_t *value, char **endPtr) {
if (pToken->n == 0) {
return TK_ILLEGAL;
}
int32_t radix = 10;
if (pToken->type == TK_HEX) {
radix = 16;
} else if (pToken->type == TK_BIN) {
radix = 2;
}
errno = 0;
*value = strtoll(pToken->z, endPtr, radix);
if (**endPtr == 'e' || **endPtr == 'E' || **endPtr == '.') {
errno = 0;
double v = round(strtod(pToken->z, endPtr));
if (v > INT64_MAX || v <= INT64_MIN) {
errno = ERANGE;
} else {
*value = (int64_t)v;
}
}
// not a valid integer number, return error
if (*endPtr - pToken->z != pToken->n) {
return TK_ILLEGAL;
}
return pToken->type;
}
static int32_t tscToDouble(SStrToken *pToken, double *value, char **endPtr) { static int32_t tscToDouble(SStrToken *pToken, double *value, char **endPtr) {
if (pToken->n == 0) {
return TK_ILLEGAL;
}
errno = 0; errno = 0;
*value = strtod(pToken->z, endPtr); *value = strtold(pToken->z, endPtr);
// not a valid integer number, return error // not a valid integer number, return error
if ((*endPtr - pToken->z) != pToken->n) { if ((*endPtr - pToken->z) != pToken->n) {
...@@ -163,86 +127,119 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1 ...@@ -163,86 +127,119 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
// todo extract the null value check
static bool isNullStr(SStrToken* pToken) {
return (pToken->type == TK_NULL) || ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0));
}
int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, char *msg, char **str, bool primaryKey, int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, char *msg, char **str, bool primaryKey,
int16_t timePrec) { int16_t timePrec) {
int64_t iv; int64_t iv;
int32_t numType; int32_t ret;
char * endptr = NULL; char *endptr = NULL;
errno = 0; // clear the previous existed error information
if (IS_NUMERIC_TYPE(pSchema->type) && pToken->n == 0) {
return tscInvalidSQLErrMsg(msg, "invalid numeric data", pToken->z);
}
switch (pSchema->type) { switch (pSchema->type) {
case TSDB_DATA_TYPE_BOOL: { // bool case TSDB_DATA_TYPE_BOOL: { // bool
if ((pToken->type == TK_BOOL || pToken->type == TK_STRING) && (pToken->n != 0)) { if (isNullStr(pToken)) {
if (strncmp(pToken->z, "true", pToken->n) == 0) { *((uint8_t *)payload) = TSDB_DATA_BOOL_NULL;
*(uint8_t *)payload = TSDB_TRUE; } else {
} else if (strncmp(pToken->z, "false", pToken->n) == 0) { if ((pToken->type == TK_BOOL || pToken->type == TK_STRING) && (pToken->n != 0)) {
*(uint8_t *)payload = TSDB_FALSE; if (strncmp(pToken->z, "true", pToken->n) == 0) {
} else if (strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0) { *(uint8_t *)payload = TSDB_TRUE;
*(uint8_t *)payload = TSDB_DATA_BOOL_NULL; } else if (strncmp(pToken->z, "false", pToken->n) == 0) {
*(uint8_t *)payload = TSDB_FALSE;
} else {
return tscSQLSyntaxErrMsg(msg, "invalid bool data", pToken->z);
}
} else if (pToken->type == TK_INTEGER) {
iv = strtoll(pToken->z, NULL, 10);
*(uint8_t *)payload = (int8_t)((iv == 0) ? TSDB_FALSE : TSDB_TRUE);
} else if (pToken->type == TK_FLOAT) {
double dv = strtod(pToken->z, NULL);
*(uint8_t *)payload = (int8_t)((dv == 0) ? TSDB_FALSE : TSDB_TRUE);
} else { } else {
return tscSQLSyntaxErrMsg(msg, "invalid bool data", pToken->z); return tscInvalidSQLErrMsg(msg, "invalid bool data", pToken->z);
} }
} else if (pToken->type == TK_INTEGER) {
iv = strtoll(pToken->z, NULL, 10);
*(uint8_t *)payload = (int8_t)((iv == 0) ? TSDB_FALSE : TSDB_TRUE);
} else if (pToken->type == TK_FLOAT) {
double dv = strtod(pToken->z, NULL);
*(uint8_t *)payload = (int8_t)((dv == 0) ? TSDB_FALSE : TSDB_TRUE);
} else if (pToken->type == TK_NULL) {
*(uint8_t *)payload = TSDB_DATA_BOOL_NULL;
} else {
return tscInvalidSQLErrMsg(msg, "invalid bool data", pToken->z);
} }
break; break;
} }
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int8_t *)payload) = TSDB_DATA_TINYINT_NULL; *((uint8_t *)payload) = TSDB_DATA_TINYINT_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int8_t *)payload) = TSDB_DATA_TINYINT_NULL;
} else { } else {
numType = tscToInteger(pToken, &iv, &endptr); ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, true);
if (TK_ILLEGAL == numType) { if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid tinyint data", pToken->z); return tscInvalidSQLErrMsg(msg, "invalid tinyint data", pToken->z);
} else if (errno == ERANGE || iv > INT8_MAX || iv <= INT8_MIN) { } else if (!IS_VALID_TINYINT(iv)) {
return tscInvalidSQLErrMsg(msg, "tinyint data overflow", pToken->z); return tscInvalidSQLErrMsg(msg, "data overflow", pToken->z);
} }
*((int8_t *)payload) = (int8_t)iv; *((uint8_t *)payload) = (uint8_t)iv;
}
break;
case TSDB_DATA_TYPE_UTINYINT:
if (isNullStr(pToken)) {
*((uint8_t *)payload) = TSDB_DATA_UTINYINT_NULL;
} else {
ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, false);
if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid unsigned tinyint data", pToken->z);
} else if (!IS_VALID_UTINYINT(iv)) {
return tscInvalidSQLErrMsg(msg, "unsigned tinyint data overflow", pToken->z);
}
*((uint8_t *)payload) = (uint8_t)iv;
} }
break; break;
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int16_t *)payload) = TSDB_DATA_SMALLINT_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int16_t *)payload) = TSDB_DATA_SMALLINT_NULL; *((int16_t *)payload) = TSDB_DATA_SMALLINT_NULL;
} else { } else {
numType = tscToInteger(pToken, &iv, &endptr); ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, true);
if (TK_ILLEGAL == numType) { if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid smallint data", pToken->z); return tscInvalidSQLErrMsg(msg, "invalid smallint data", pToken->z);
} else if (errno == ERANGE || iv > INT16_MAX || iv <= INT16_MIN) { } else if (!IS_VALID_SMALLINT(iv)) {
return tscInvalidSQLErrMsg(msg, "smallint data overflow", pToken->z); return tscInvalidSQLErrMsg(msg, "smallint data overflow", pToken->z);
} }
*((int16_t *)payload) = (int16_t)iv; *((int16_t *)payload) = (int16_t)iv;
} }
break;
case TSDB_DATA_TYPE_USMALLINT:
if (isNullStr(pToken)) {
*((uint16_t *)payload) = TSDB_DATA_USMALLINT_NULL;
} else {
ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, false);
if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid unsigned smallint data", pToken->z);
} else if (!IS_VALID_USMALLINT(iv)) {
return tscInvalidSQLErrMsg(msg, "unsigned smallint data overflow", pToken->z);
}
*((uint16_t *)payload) = (uint16_t)iv;
}
break; break;
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int32_t *)payload) = TSDB_DATA_INT_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int32_t *)payload) = TSDB_DATA_INT_NULL; *((int32_t *)payload) = TSDB_DATA_INT_NULL;
} else { } else {
numType = tscToInteger(pToken, &iv, &endptr); ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, true);
if (TK_ILLEGAL == numType) { if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid int data", pToken->z); return tscInvalidSQLErrMsg(msg, "invalid int data", pToken->z);
} else if (errno == ERANGE || iv > INT32_MAX || iv <= INT32_MIN) { } else if (!IS_VALID_INT(iv)) {
return tscInvalidSQLErrMsg(msg, "int data overflow", pToken->z); return tscInvalidSQLErrMsg(msg, "int data overflow", pToken->z);
} }
...@@ -251,17 +248,30 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -251,17 +248,30 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
break; break;
case TSDB_DATA_TYPE_UINT:
if (isNullStr(pToken)) {
*((uint32_t *)payload) = TSDB_DATA_UINT_NULL;
} else {
ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, false);
if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid unsigned int data", pToken->z);
} else if (!IS_VALID_UINT(iv)) {
return tscInvalidSQLErrMsg(msg, "unsigned int data overflow", pToken->z);
}
*((uint32_t *)payload) = (uint32_t)iv;
}
break;
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int64_t *)payload) = TSDB_DATA_BIGINT_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int64_t *)payload) = TSDB_DATA_BIGINT_NULL; *((int64_t *)payload) = TSDB_DATA_BIGINT_NULL;
} else { } else {
numType = tscToInteger(pToken, &iv, &endptr); ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, true);
if (TK_ILLEGAL == numType) { if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid bigint data", pToken->z); return tscInvalidSQLErrMsg(msg, "invalid bigint data", pToken->z);
} else if (errno == ERANGE || iv == INT64_MIN) { } else if (!IS_VALID_BIGINT(iv)) {
return tscInvalidSQLErrMsg(msg, "bigint data overflow", pToken->z); return tscInvalidSQLErrMsg(msg, "bigint data overflow", pToken->z);
} }
...@@ -269,11 +279,23 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -269,11 +279,23 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
} }
break; break;
case TSDB_DATA_TYPE_UBIGINT:
if (isNullStr(pToken)) {
*((uint64_t *)payload) = TSDB_DATA_UBIGINT_NULL;
} else {
ret = tStrToInteger(pToken->z, pToken->type, pToken->n, &iv, false);
if (ret != TSDB_CODE_SUCCESS) {
return tscInvalidSQLErrMsg(msg, "invalid unsigned bigint data", pToken->z);
} else if (!IS_VALID_UBIGINT((uint64_t)iv)) {
return tscInvalidSQLErrMsg(msg, "unsigned bigint data overflow", pToken->z);
}
*((uint64_t *)payload) = iv;
}
break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int32_t *)payload) = TSDB_DATA_FLOAT_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int32_t *)payload) = TSDB_DATA_FLOAT_NULL; *((int32_t *)payload) = TSDB_DATA_FLOAT_NULL;
} else { } else {
double dv; double dv;
...@@ -281,24 +303,16 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -281,24 +303,16 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z); return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z);
} }
float fv = (float)dv; if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || dv > FLT_MAX || dv < -FLT_MAX || isinf(dv) || isnan(dv)) {
if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || (fv > FLT_MAX || fv < -FLT_MAX)) {
return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z); return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z);
} }
if (isinf(fv) || isnan(fv)) { *((float *)payload) = (float)dv;
*((int32_t *)payload) = TSDB_DATA_FLOAT_NULL;
}
*((float *)payload) = fv;
} }
break; break;
case TSDB_DATA_TYPE_DOUBLE: case TSDB_DATA_TYPE_DOUBLE:
if (pToken->type == TK_NULL) { if (isNullStr(pToken)) {
*((int64_t *)payload) = TSDB_DATA_DOUBLE_NULL;
} else if ((pToken->type == TK_STRING) && (pToken->n != 0) &&
(strncasecmp(TSDB_DATA_NULL_STR_L, pToken->z, pToken->n) == 0)) {
*((int64_t *)payload) = TSDB_DATA_DOUBLE_NULL; *((int64_t *)payload) = TSDB_DATA_DOUBLE_NULL;
} else { } else {
double dv; double dv;
...@@ -306,15 +320,11 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -306,15 +320,11 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
return tscInvalidSQLErrMsg(msg, "illegal double data", pToken->z); return tscInvalidSQLErrMsg(msg, "illegal double data", pToken->z);
} }
if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || (dv > DBL_MAX || dv < -DBL_MAX)) { if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || isinf(dv) || isnan(dv)) {
return tscInvalidSQLErrMsg(msg, "illegal double data", pToken->z); return tscInvalidSQLErrMsg(msg, "illegal double data", pToken->z);
} }
if (isinf(dv) || isnan(dv)) { *((double *)payload) = dv;
*((int64_t *)payload) = TSDB_DATA_DOUBLE_NULL;
} else {
*((double *)payload) = dv;
}
} }
break; break;
...@@ -337,7 +347,7 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -337,7 +347,7 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
setVardataNull(payload, TSDB_DATA_TYPE_NCHAR); setVardataNull(payload, TSDB_DATA_TYPE_NCHAR);
} else { } else {
// if the converted output len is over than pColumnModel->bytes, return error: 'Argument list too long' // if the converted output len is over than pColumnModel->bytes, return error: 'Argument list too long'
size_t output = 0; int32_t output = 0;
if (!taosMbsToUcs4(pToken->z, pToken->n, varDataVal(payload), pSchema->bytes - VARSTR_HEADER_SIZE, &output)) { if (!taosMbsToUcs4(pToken->z, pToken->n, varDataVal(payload), pSchema->bytes - VARSTR_HEADER_SIZE, &output)) {
char buf[512] = {0}; char buf[512] = {0};
snprintf(buf, tListLen(buf), "%s", strerror(errno)); snprintf(buf, tListLen(buf), "%s", strerror(errno));
...@@ -752,6 +762,8 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql) { ...@@ -752,6 +762,8 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql) {
char *sql = *sqlstr; char *sql = *sqlstr;
pSql->cmd.autoCreated = false;
// get the token of specified table // get the token of specified table
index = 0; index = 0;
tableToken = tStrGetToken(sql, &index, false, 0, NULL); tableToken = tStrGetToken(sql, &index, false, 0, NULL);
...@@ -935,11 +947,15 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql) { ...@@ -935,11 +947,15 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql) {
SKVRow row = tdGetKVRowFromBuilder(&kvRowBuilder); SKVRow row = tdGetKVRowFromBuilder(&kvRowBuilder);
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
if (row == NULL) { if (row == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY; return tscInvalidSQLErrMsg(pCmd->payload, "tag value expected", NULL);
} }
tdSortKVRowByColIdx(row); tdSortKVRowByColIdx(row);
pCmd->tagData.dataLen = kvRowLen(row); pCmd->tagData.dataLen = kvRowLen(row);
if (pCmd->tagData.dataLen <= 0){
return tscInvalidSQLErrMsg(pCmd->payload, "tag value expected", NULL);
}
char* pTag = realloc(pCmd->tagData.data, pCmd->tagData.dataLen); char* pTag = realloc(pCmd->tagData.data, pCmd->tagData.dataLen);
if (pTag == NULL) { if (pTag == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY; return TSDB_CODE_TSC_OUT_OF_MEMORY;
...@@ -1339,7 +1355,7 @@ int tsParseSql(SSqlObj *pSql, bool initial) { ...@@ -1339,7 +1355,7 @@ int tsParseSql(SSqlObj *pSql, bool initial) {
if (sqlstr == NULL || pSql->parseRetry >= 1 || ret != TSDB_CODE_TSC_INVALID_SQL) { if (sqlstr == NULL || pSql->parseRetry >= 1 || ret != TSDB_CODE_TSC_INVALID_SQL) {
free(sqlstr); free(sqlstr);
} else { } else {
tscResetSqlCmdObj(pCmd, true); tscResetSqlCmdObj(pCmd);
free(pSql->sqlstr); free(pSql->sqlstr);
pSql->sqlstr = sqlstr; pSql->sqlstr = sqlstr;
pSql->parseRetry++; pSql->parseRetry++;
...@@ -1351,7 +1367,7 @@ int tsParseSql(SSqlObj *pSql, bool initial) { ...@@ -1351,7 +1367,7 @@ int tsParseSql(SSqlObj *pSql, bool initial) {
SSqlInfo SQLInfo = qSQLParse(pSql->sqlstr); SSqlInfo SQLInfo = qSQLParse(pSql->sqlstr);
ret = tscToSQLCmd(pSql, &SQLInfo); ret = tscToSQLCmd(pSql, &SQLInfo);
if (ret == TSDB_CODE_TSC_INVALID_SQL && pSql->parseRetry == 0 && SQLInfo.type == TSDB_SQL_NULL) { if (ret == TSDB_CODE_TSC_INVALID_SQL && pSql->parseRetry == 0 && SQLInfo.type == TSDB_SQL_NULL) {
tscResetSqlCmdObj(pCmd, true); tscResetSqlCmdObj(pCmd);
pSql->parseRetry++; pSql->parseRetry++;
ret = tscToSQLCmd(pSql, &SQLInfo); ret = tscToSQLCmd(pSql, &SQLInfo);
} }
...@@ -1429,7 +1445,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) { ...@@ -1429,7 +1445,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) {
fclose(fp); fclose(fp);
pParentSql->res.code = code; pParentSql->res.code = code;
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} while (0); } while (0);
} }
...@@ -1451,7 +1467,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) { ...@@ -1451,7 +1467,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) {
int32_t count = 0; int32_t count = 0;
int32_t maxRows = 0; int32_t maxRows = 0;
tfree(pCmd->pTableMetaList); tfree(pCmd->pTableNameList);
pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks);
if (pCmd->pTableBlockHashList == NULL) { if (pCmd->pTableBlockHashList == NULL) {
...@@ -1500,7 +1516,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) { ...@@ -1500,7 +1516,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int code) {
code = doPackSendDataBlock(pSql, count, pTableDataBlock); code = doPackSendDataBlock(pSql, count, pTableDataBlock);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
pParentSql->res.code = code; pParentSql->res.code = code;
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -1535,7 +1551,7 @@ void tscProcessMultiVnodesImportFromFile(SSqlObj *pSql) { ...@@ -1535,7 +1551,7 @@ void tscProcessMultiVnodesImportFromFile(SSqlObj *pSql) {
tscError("%p failed to open file %s to load data from file, code:%s", pSql, pCmd->payload, tstrerror(pSql->res.code)); tscError("%p failed to open file %s to load data from file, code:%s", pSql, pCmd->payload, tstrerror(pSql->res.code));
tfree(pSupporter); tfree(pSupporter);
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
......
...@@ -84,35 +84,35 @@ static int normalStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) { ...@@ -84,35 +84,35 @@ static int normalStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) {
var->nLen = 0; var->nLen = 0;
if (tb->is_null != NULL && *(tb->is_null)) { if (tb->is_null != NULL && *(tb->is_null)) {
var->nType = TSDB_DATA_TYPE_NULL; var->nType = TSDB_DATA_TYPE_NULL;
var->i64Key = 0; var->i64 = 0;
continue; continue;
} }
var->nType = tb->buffer_type; var->nType = tb->buffer_type;
switch (tb->buffer_type) { switch (tb->buffer_type) {
case TSDB_DATA_TYPE_NULL: case TSDB_DATA_TYPE_NULL:
var->i64Key = 0; var->i64 = 0;
break; break;
case TSDB_DATA_TYPE_BOOL: case TSDB_DATA_TYPE_BOOL:
var->i64Key = (*(int8_t*)tb->buffer) ? 1 : 0; var->i64 = (*(int8_t*)tb->buffer) ? 1 : 0;
break; break;
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
var->i64Key = *(int8_t*)tb->buffer; var->i64 = *(int8_t*)tb->buffer;
break; break;
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
var->i64Key = *(int16_t*)tb->buffer; var->i64 = *(int16_t*)tb->buffer;
break; break;
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
var->i64Key = *(int32_t*)tb->buffer; var->i64 = *(int32_t*)tb->buffer;
break; break;
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
case TSDB_DATA_TYPE_TIMESTAMP: case TSDB_DATA_TYPE_TIMESTAMP:
var->i64Key = *(int64_t*)tb->buffer; var->i64 = *(int64_t*)tb->buffer;
break; break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
...@@ -219,7 +219,7 @@ static char* normalStmtBuildSql(STscStmt* stmt) { ...@@ -219,7 +219,7 @@ static char* normalStmtBuildSql(STscStmt* stmt) {
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
taosStringBuilderAppendInteger(&sb, var->i64Key); taosStringBuilderAppendInteger(&sb, var->i64);
break; break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
...@@ -616,7 +616,7 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) { ...@@ -616,7 +616,7 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) {
case TSDB_DATA_TYPE_NCHAR: { case TSDB_DATA_TYPE_NCHAR: {
switch (bind->buffer_type) { switch (bind->buffer_type) {
case TSDB_DATA_TYPE_NCHAR: { case TSDB_DATA_TYPE_NCHAR: {
size_t output = 0; int32_t output = 0;
if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) { if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) {
return TSDB_CODE_TSC_INVALID_VALUE; return TSDB_CODE_TSC_INVALID_VALUE;
} }
...@@ -678,7 +678,7 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) { ...@@ -678,7 +678,7 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
case TSDB_DATA_TYPE_NCHAR: { case TSDB_DATA_TYPE_NCHAR: {
size_t output = 0; int32_t output = 0;
if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) { if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) {
return TSDB_CODE_TSC_INVALID_VALUE; return TSDB_CODE_TSC_INVALID_VALUE;
} }
......
此差异已折叠。
...@@ -66,30 +66,24 @@ STableComInfo tscGetTableInfo(const STableMeta* pTableMeta) { ...@@ -66,30 +66,24 @@ STableComInfo tscGetTableInfo(const STableMeta* pTableMeta) {
return pTableMeta->tableInfo; return pTableMeta->tableInfo;
} }
bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols) { static bool doValidateSchema(SSchema* pSchema, int32_t numOfCols, int32_t maxLen) {
if (!VALIDNUMOFCOLS(numOfCols)) {
return false;
}
/* first column must be the timestamp, which is a primary key */
if (pSchema[0].type != TSDB_DATA_TYPE_TIMESTAMP) {
return false;
}
/* type is valid, length is valid */
int32_t rowLen = 0; int32_t rowLen = 0;
for (int32_t i = 0; i < numOfCols; ++i) { for (int32_t i = 0; i < numOfCols; ++i) {
// 1. valid types // 1. valid types
if (pSchema[i].type > TSDB_DATA_TYPE_TIMESTAMP || pSchema[i].type < TSDB_DATA_TYPE_BOOL) { if (!isValidDataType(pSchema[i].type)) {
return false; return false;
} }
// 2. valid length for each type // 2. valid length for each type
if (pSchema[i].type == TSDB_DATA_TYPE_TIMESTAMP) { if (pSchema[i].type == TSDB_DATA_TYPE_BINARY) {
if (pSchema[i].bytes > TSDB_MAX_BINARY_LEN) { if (pSchema[i].bytes > TSDB_MAX_BINARY_LEN) {
return false; return false;
} }
} else if (pSchema[i].type == TSDB_DATA_TYPE_NCHAR) {
if (pSchema[i].bytes > TSDB_MAX_NCHAR_LEN) {
return false;
}
} else { } else {
if (pSchema[i].bytes != tDataTypeDesc[pSchema[i].type].nSize) { if (pSchema[i].bytes != tDataTypeDesc[pSchema[i].type].nSize) {
return false; return false;
...@@ -106,8 +100,32 @@ bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols) { ...@@ -106,8 +100,32 @@ bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols) {
rowLen += pSchema[i].bytes; rowLen += pSchema[i].bytes;
} }
// valid total length return rowLen <= maxLen;
return (rowLen <= TSDB_MAX_BYTES_PER_ROW); }
bool isValidSchema(struct SSchema* pSchema, int32_t numOfCols, int32_t numOfTags) {
if (!VALIDNUMOFCOLS(numOfCols)) {
return false;
}
if (!VALIDNUMOFTAGS(numOfTags)) {
return false;
}
/* first column must be the timestamp, which is a primary key */
if (pSchema[0].type != TSDB_DATA_TYPE_TIMESTAMP) {
return false;
}
if (!doValidateSchema(pSchema, numOfCols, TSDB_MAX_BYTES_PER_ROW)) {
return false;
}
if (!doValidateSchema(&pSchema[numOfCols], numOfTags, TSDB_MAX_TAGS_LEN)) {
return false;
}
return true;
} }
SSchema* tscGetTableColumnSchema(const STableMeta* pTableMeta, int32_t colIndex) { SSchema* tscGetTableColumnSchema(const STableMeta* pTableMeta, int32_t colIndex) {
...@@ -130,19 +148,8 @@ SSchema* tscGetColumnSchemaById(STableMeta* pTableMeta, int16_t colId) { ...@@ -130,19 +148,8 @@ SSchema* tscGetColumnSchemaById(STableMeta* pTableMeta, int16_t colId) {
return NULL; return NULL;
} }
static void tscInitCorVgroupInfo(SCorVgroupInfo *corVgroupInfo, SVgroupMsg *pVgroupMsg) { STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg) {
corVgroupInfo->version = 0; assert(pTableMetaMsg != NULL && pTableMetaMsg->numOfColumns >= 2 && pTableMetaMsg->numOfTags >= 0);
corVgroupInfo->inUse = 0;
corVgroupInfo->numOfEps = pVgroupMsg->numOfEps;
for (int32_t i = 0; i < pVgroupMsg->numOfEps; i++) {
corVgroupInfo->epAddr[i].fqdn = strndup(pVgroupMsg->epAddr[i].fqdn, tListLen(pVgroupMsg->epAddr[0].fqdn));
corVgroupInfo->epAddr[i].port = pVgroupMsg->epAddr[i].port;
}
}
STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size) {
assert(pTableMetaMsg != NULL);
int32_t schemaSize = (pTableMetaMsg->numOfColumns + pTableMetaMsg->numOfTags) * sizeof(SSchema); int32_t schemaSize = (pTableMetaMsg->numOfColumns + pTableMetaMsg->numOfTags) * sizeof(SSchema);
STableMeta* pTableMeta = calloc(1, sizeof(STableMeta) + schemaSize); STableMeta* pTableMeta = calloc(1, sizeof(STableMeta) + schemaSize);
...@@ -159,11 +166,9 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size ...@@ -159,11 +166,9 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size
pTableMeta->id.tid = pTableMetaMsg->tid; pTableMeta->id.tid = pTableMetaMsg->tid;
pTableMeta->id.uid = pTableMetaMsg->uid; pTableMeta->id.uid = pTableMetaMsg->uid;
tscInitCorVgroupInfo(&pTableMeta->corVgroupInfo, &pTableMetaMsg->vgroup);
pTableMeta->sversion = pTableMetaMsg->sversion; pTableMeta->sversion = pTableMetaMsg->sversion;
pTableMeta->tversion = pTableMetaMsg->tversion; pTableMeta->tversion = pTableMetaMsg->tversion;
tstrncpy(pTableMeta->sTableId, pTableMetaMsg->sTableId, TSDB_TABLE_FNAME_LEN); tstrncpy(pTableMeta->sTableName, pTableMetaMsg->sTableName, TSDB_TABLE_FNAME_LEN);
memcpy(pTableMeta->schema, pTableMetaMsg->schema, schemaSize); memcpy(pTableMeta->schema, pTableMetaMsg->schema, schemaSize);
...@@ -172,13 +177,44 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size ...@@ -172,13 +177,44 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size
pTableMeta->tableInfo.rowSize += pTableMeta->schema[i].bytes; pTableMeta->tableInfo.rowSize += pTableMeta->schema[i].bytes;
} }
if (size != NULL) {
*size = sizeof(STableMeta) + schemaSize;
}
return pTableMeta; return pTableMeta;
} }
bool vgroupInfoIdentical(SNewVgroupInfo *pExisted, SVgroupMsg* src) {
assert(pExisted != NULL && src != NULL);
if (pExisted->numOfEps != src->numOfEps) {
return false;
}
for(int32_t i = 0; i < pExisted->numOfEps; ++i) {
if (pExisted->ep[i].port != src->epAddr[i].port) {
return false;
}
if (strncmp(pExisted->ep[i].fqdn, src->epAddr[i].fqdn, tListLen(pExisted->ep[i].fqdn)) != 0) {
return false;
}
}
return true;
}
SNewVgroupInfo createNewVgroupInfo(SVgroupMsg *pVgroupMsg) {
assert(pVgroupMsg != NULL);
SNewVgroupInfo info = {0};
info.numOfEps = pVgroupMsg->numOfEps;
info.vgId = pVgroupMsg->vgId;
info.inUse = 0;
for(int32_t i = 0; i < pVgroupMsg->numOfEps; ++i) {
tstrncpy(info.ep[i].fqdn, pVgroupMsg->epAddr[i].fqdn, TSDB_FQDN_LEN);
info.ep[i].port = pVgroupMsg->epAddr[i].port;
}
return info;
}
// todo refactor // todo refactor
UNUSED_FUNC static FORCE_INLINE char* skipSegments(char* input, char delim, int32_t num) { UNUSED_FUNC static FORCE_INLINE char* skipSegments(char* input, char delim, int32_t num) {
for (int32_t i = 0; i < num; ++i) { for (int32_t i = 0; i < num; ++i) {
......
此差异已折叠。
...@@ -47,7 +47,7 @@ static bool validUserName(const char* user) { ...@@ -47,7 +47,7 @@ static bool validUserName(const char* user) {
} }
static bool validPassword(const char* passwd) { static bool validPassword(const char* passwd) {
return validImpl(passwd, TSDB_PASSWORD_LEN - 1); return validImpl(passwd, TSDB_KEY_LEN - 1);
} }
static SSqlObj *taosConnectImpl(const char *ip, const char *user, const char *pass, const char *auth, const char *db, static SSqlObj *taosConnectImpl(const char *ip, const char *user, const char *pass, const char *auth, const char *db,
...@@ -238,11 +238,11 @@ TAOS *taos_connect_c(const char *ip, uint8_t ipLen, const char *user, uint8_t us ...@@ -238,11 +238,11 @@ TAOS *taos_connect_c(const char *ip, uint8_t ipLen, const char *user, uint8_t us
uint8_t passLen, const char *db, uint8_t dbLen, uint16_t port) { uint8_t passLen, const char *db, uint8_t dbLen, uint16_t port) {
char ipBuf[TSDB_EP_LEN] = {0}; char ipBuf[TSDB_EP_LEN] = {0};
char userBuf[TSDB_USER_LEN] = {0}; char userBuf[TSDB_USER_LEN] = {0};
char passBuf[TSDB_PASSWORD_LEN] = {0}; char passBuf[TSDB_KEY_LEN] = {0};
char dbBuf[TSDB_DB_NAME_LEN] = {0}; char dbBuf[TSDB_DB_NAME_LEN] = {0};
strncpy(ipBuf, ip, MIN(TSDB_EP_LEN - 1, ipLen)); strncpy(ipBuf, ip, MIN(TSDB_EP_LEN - 1, ipLen));
strncpy(userBuf, user, MIN(TSDB_USER_LEN - 1, userLen)); strncpy(userBuf, user, MIN(TSDB_USER_LEN - 1, userLen));
strncpy(passBuf, pass, MIN(TSDB_PASSWORD_LEN - 1, passLen)); strncpy(passBuf, pass, MIN(TSDB_KEY_LEN - 1, passLen));
strncpy(dbBuf, db, MIN(TSDB_DB_NAME_LEN - 1, dbLen)); strncpy(dbBuf, db, MIN(TSDB_DB_NAME_LEN - 1, dbLen));
return taos_connect(ipBuf, userBuf, passBuf, dbBuf, port); return taos_connect(ipBuf, userBuf, passBuf, dbBuf, port);
} }
...@@ -327,7 +327,7 @@ TAOS_RES* taos_query_c(TAOS *taos, const char *sqlstr, uint32_t sqlLen, int64_t* ...@@ -327,7 +327,7 @@ TAOS_RES* taos_query_c(TAOS *taos, const char *sqlstr, uint32_t sqlLen, int64_t*
return NULL; return NULL;
} }
nPrintTsc(sqlstr); nPrintTsc("%s", sqlstr);
SSqlObj* pSql = calloc(1, sizeof(SSqlObj)); SSqlObj* pSql = calloc(1, sizeof(SSqlObj));
if (pSql == NULL) { if (pSql == NULL) {
...@@ -694,6 +694,8 @@ static void tscKillSTableQuery(SSqlObj *pSql) { ...@@ -694,6 +694,8 @@ static void tscKillSTableQuery(SSqlObj *pSql) {
// set the master sqlObj flag to cancel query // set the master sqlObj flag to cancel query
pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED; pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED;
tscLockByThread(&pSql->squeryLock);
for (int i = 0; i < pSql->subState.numOfSub; ++i) { for (int i = 0; i < pSql->subState.numOfSub; ++i) {
// NOTE: pSub may have been released already here // NOTE: pSub may have been released already here
SSqlObj *pSub = pSql->pSubs[i]; SSqlObj *pSub = pSql->pSubs[i];
...@@ -709,10 +711,16 @@ static void tscKillSTableQuery(SSqlObj *pSql) { ...@@ -709,10 +711,16 @@ static void tscKillSTableQuery(SSqlObj *pSql) {
pSubObj->rpcRid = -1; pSubObj->rpcRid = -1;
} }
tscQueueAsyncRes(pSubObj); tscAsyncResultOnError(pSubObj);
taosReleaseRef(tscObjRef, pSubObj->self); taosReleaseRef(tscObjRef, pSubObj->self);
} }
if (pSql->subState.numOfSub <= 0) {
tscAsyncResultOnError(pSql);
}
tscUnlockByThread(&pSql->squeryLock);
tscDebug("%p super table query cancelled", pSql); tscDebug("%p super table query cancelled", pSql);
} }
...@@ -745,7 +753,7 @@ void taos_stop_query(TAOS_RES *res) { ...@@ -745,7 +753,7 @@ void taos_stop_query(TAOS_RES *res) {
pSql->rpcRid = -1; pSql->rpcRid = -1;
} }
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
} }
...@@ -788,18 +796,34 @@ int taos_print_row(char *str, TAOS_ROW row, TAOS_FIELD *fields, int num_fields) ...@@ -788,18 +796,34 @@ int taos_print_row(char *str, TAOS_ROW row, TAOS_FIELD *fields, int num_fields)
len += sprintf(str + len, "%d", *((int8_t *)row[i])); len += sprintf(str + len, "%d", *((int8_t *)row[i]));
break; break;
case TSDB_DATA_TYPE_UTINYINT:
len += sprintf(str + len, "%u", *((uint8_t *)row[i]));
break;
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
len += sprintf(str + len, "%d", *((int16_t *)row[i])); len += sprintf(str + len, "%d", *((int16_t *)row[i]));
break; break;
case TSDB_DATA_TYPE_USMALLINT:
len += sprintf(str + len, "%u", *((uint16_t *)row[i]));
break;
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
len += sprintf(str + len, "%d", *((int32_t *)row[i])); len += sprintf(str + len, "%d", *((int32_t *)row[i]));
break; break;
case TSDB_DATA_TYPE_UINT:
len += sprintf(str + len, "%u", *((uint32_t *)row[i]));
break;
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
len += sprintf(str + len, "%" PRId64, *((int64_t *)row[i])); len += sprintf(str + len, "%" PRId64, *((int64_t *)row[i]));
break; break;
case TSDB_DATA_TYPE_UBIGINT:
len += sprintf(str + len, "%" PRIu64, *((uint64_t *)row[i]));
break;
case TSDB_DATA_TYPE_FLOAT: { case TSDB_DATA_TYPE_FLOAT: {
float fv = 0; float fv = 0;
fv = GET_FLOAT_VAL(row[i]); fv = GET_FLOAT_VAL(row[i]);
...@@ -909,7 +933,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) { ...@@ -909,7 +933,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
static int tscParseTblNameList(SSqlObj *pSql, const char *tblNameList, int32_t tblListLen) { static int tscParseTblNameList(SSqlObj *pSql, const char *tblNameList, int32_t tblListLen) {
// must before clean the sqlcmd object // must before clean the sqlcmd object
tscResetSqlCmdObj(&pSql->cmd, false); tscResetSqlCmdObj(&pSql->cmd);
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
...@@ -962,7 +986,7 @@ static int tscParseTblNameList(SSqlObj *pSql, const char *tblNameList, int32_t t ...@@ -962,7 +986,7 @@ static int tscParseTblNameList(SSqlObj *pSql, const char *tblNameList, int32_t t
return code; return code;
} }
if (++pCmd->count > TSDB_MULTI_METERMETA_MAX_NUM) { if (++pCmd->count > TSDB_MULTI_TABLEMETA_MAX_NUM) {
code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
sprintf(pCmd->payload, "tables over the max number"); sprintf(pCmd->payload, "tables over the max number");
return code; return code;
......
...@@ -65,44 +65,67 @@ static int64_t tscGetRetryDelayTime(SSqlStream* pStream, int64_t slidingTime, in ...@@ -65,44 +65,67 @@ static int64_t tscGetRetryDelayTime(SSqlStream* pStream, int64_t slidingTime, in
return retryDelta; return retryDelta;
} }
static void tscProcessStreamLaunchQuery(SSchedMsg *pMsg) { static void setRetryInfo(SSqlStream* pStream, int32_t code) {
SSqlStream *pStream = (SSqlStream *)pMsg->ahandle; SSqlObj* pSql = pStream->pSql;
SSqlObj * pSql = pStream->pSql;
pSql->fp = tscProcessStreamQueryCallback; pSql->res.code = code;
pSql->fetchFp = tscProcessStreamQueryCallback; int64_t retryDelayTime = tscGetRetryDelayTime(pStream, pStream->interval.sliding, pStream->precision);
pSql->param = pStream; tscDebug("%p stream:%p, get table Meta failed, retry in %" PRId64 "ms", pSql, pStream, retryDelayTime);
tscSetRetryTimer(pStream, pSql, retryDelayTime);
}
static void doLaunchQuery(void* param, TAOS_RES* tres, int32_t code) {
SSqlStream *pStream = (SSqlStream *)param;
assert(pStream->pSql == tres);
SSqlObj* pSql = (SSqlObj*) tres;
pSql->fp = doLaunchQuery;
pSql->fetchFp = doLaunchQuery;
pSql->res.completed = false; pSql->res.completed = false;
if (code != TSDB_CODE_SUCCESS) {
setRetryInfo(pStream, code);
return;
}
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0); SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0);
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
int code = tscGetTableMeta(pSql, pTableMetaInfo); code = tscGetTableMeta(pSql, pTableMetaInfo);
pSql->res.code = code;
if (code == 0 && UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { if (code == 0 && UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
code = tscGetSTableVgroupInfo(pSql, 0); code = tscGetSTableVgroupInfo(pSql, 0);
pSql->res.code = code;
} }
// failed to get meter/metric meta, retry in 10sec. if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
if (code != TSDB_CODE_SUCCESS) { return;
int64_t retryDelayTime = tscGetRetryDelayTime(pStream, pStream->interval.sliding, pStream->precision); }
tscDebug("%p stream:%p,get metermeta failed, retry in %" PRId64 "ms", pStream->pSql, pStream, retryDelayTime);
tscSetRetryTimer(pStream, pSql, retryDelayTime);
} else { // failed to get table Meta or vgroup list, retry in 10sec.
if (code == TSDB_CODE_SUCCESS) {
tscTansformSQLFuncForSTableQuery(pQueryInfo); tscTansformSQLFuncForSTableQuery(pQueryInfo);
tscDebug("%p stream:%p start stream query on:%s", pSql, pStream, pTableMetaInfo->name); tscDebug("%p stream:%p, start stream query on:%s", pSql, pStream, pTableMetaInfo->name);
tscDoQuery(pStream->pSql);
pSql->fp = tscProcessStreamQueryCallback;
pSql->fetchFp = tscProcessStreamQueryCallback;
tscDoQuery(pSql);
tscIncStreamExecutionCount(pStream); tscIncStreamExecutionCount(pStream);
} else {
setRetryInfo(pStream, code);
} }
} }
static void tscProcessStreamLaunchQuery(SSchedMsg *pMsg) {
SSqlStream *pStream = (SSqlStream *)pMsg->ahandle;
doLaunchQuery(pStream, pStream->pSql, 0);
}
static void tscProcessStreamTimer(void *handle, void *tmrId) { static void tscProcessStreamTimer(void *handle, void *tmrId) {
SSqlStream *pStream = (SSqlStream *)handle; SSqlStream *pStream = (SSqlStream *)handle;
if (pStream == NULL) return; if (pStream == NULL || pStream->pTimer != tmrId) {
if (pStream->pTimer != tmrId) return; return;
}
pStream->pTimer = NULL; pStream->pTimer = NULL;
pStream->numOfRes = 0; // reset the numOfRes. pStream->numOfRes = 0; // reset the numOfRes.
...@@ -167,7 +190,9 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf ...@@ -167,7 +190,9 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf
retryDelay); retryDelay);
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pStream->pSql->cmd, 0, 0); STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pStream->pSql->cmd, 0, 0);
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), true);
char* name = pTableMetaInfo->name;
taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList); pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList);
tscSetRetryTimer(pStream, pStream->pSql, retryDelay); tscSetRetryTimer(pStream, pStream->pSql, retryDelay);
...@@ -269,9 +294,8 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf ...@@ -269,9 +294,8 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf
tscDebug("%p stream:%p, query on:%s, fetch result completed, fetched rows:%" PRId64, pSql, pStream, pTableMetaInfo->name, tscDebug("%p stream:%p, query on:%s, fetch result completed, fetched rows:%" PRId64, pSql, pStream, pTableMetaInfo->name,
pStream->numOfRes); pStream->numOfRes);
// release the metric/meter meta information reference, so data in cache can be updated tfree(pTableMetaInfo->pTableMeta);
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), false);
tscFreeSqlResult(pSql); tscFreeSqlResult(pSql);
tfree(pSql->pSubs); tfree(pSql->pSubs);
pSql->subState.numOfSub = 0; pSql->subState.numOfSub = 0;
...@@ -391,11 +415,16 @@ static void tscSetNextLaunchTimer(SSqlStream *pStream, SSqlObj *pSql) { ...@@ -391,11 +415,16 @@ static void tscSetNextLaunchTimer(SSqlStream *pStream, SSqlObj *pSql) {
tscSetRetryTimer(pStream, pSql, timer); tscSetRetryTimer(pStream, pSql, timer);
} }
static void tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) { static int32_t tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) {
int64_t minIntervalTime = int64_t minIntervalTime =
(pStream->precision == TSDB_TIME_PRECISION_MICRO) ? tsMinIntervalTime * 1000L : tsMinIntervalTime; (pStream->precision == TSDB_TIME_PRECISION_MICRO) ? tsMinIntervalTime * 1000L : tsMinIntervalTime;
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0);
if (!pStream->isProject && pQueryInfo->interval.interval == 0) {
sprintf(pSql->cmd.payload, "the interval value is 0");
return -1;
}
if (pQueryInfo->interval.intervalUnit != 'n' && pQueryInfo->interval.intervalUnit!= 'y' && pQueryInfo->interval.interval < minIntervalTime) { if (pQueryInfo->interval.intervalUnit != 'n' && pQueryInfo->interval.intervalUnit!= 'y' && pQueryInfo->interval.interval < minIntervalTime) {
tscWarn("%p stream:%p, original sample interval:%" PRId64 " too small, reset to:%" PRId64, pSql, pStream, tscWarn("%p stream:%p, original sample interval:%" PRId64 " too small, reset to:%" PRId64, pSql, pStream,
...@@ -435,6 +464,8 @@ static void tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) { ...@@ -435,6 +464,8 @@ static void tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) {
pQueryInfo->interval.interval = 0; // clear the interval value to avoid the force time window split by query processor pQueryInfo->interval.interval = 0; // clear the interval value to avoid the force time window split by query processor
pQueryInfo->interval.sliding = 0; pQueryInfo->interval.sliding = 0;
} }
return TSDB_CODE_SUCCESS;
} }
static int64_t tscGetStreamStartTimestamp(SSqlObj *pSql, SSqlStream *pStream, int64_t stime) { static int64_t tscGetStreamStartTimestamp(SSqlObj *pSql, SSqlStream *pStream, int64_t stime) {
...@@ -484,34 +515,19 @@ static int64_t tscGetLaunchTimestamp(const SSqlStream *pStream) { ...@@ -484,34 +515,19 @@ static int64_t tscGetLaunchTimestamp(const SSqlStream *pStream) {
return (pStream->precision == TSDB_TIME_PRECISION_MICRO) ? timer / 1000L : timer; return (pStream->precision == TSDB_TIME_PRECISION_MICRO) ? timer / 1000L : timer;
} }
static void setErrorInfo(SSqlObj* pSql, int32_t code, char* info) {
if (pSql == NULL) {
return;
}
SSqlCmd* pCmd = &pSql->cmd;
pSql->res.code = code;
if (info != NULL) {
strncpy(pCmd->payload, info, pCmd->payloadLen);
}
}
static void tscCreateStream(void *param, TAOS_RES *res, int code) { static void tscCreateStream(void *param, TAOS_RES *res, int code) {
SSqlStream* pStream = (SSqlStream*)param; SSqlStream* pStream = (SSqlStream*)param;
SSqlObj* pSql = pStream->pSql; SSqlObj* pSql = pStream->pSql;
SSqlCmd* pCmd = &pSql->cmd; SSqlCmd* pCmd = &pSql->cmd;
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
setErrorInfo(pSql, code, pCmd->payload); pSql->res.code = code;
tscError("%p open stream failed, sql:%s, reason:%s, code:0x%08x", pSql, pSql->sqlstr, pCmd->payload, code); tscError("%p open stream failed, sql:%s, reason:%s, code:%s", pSql, pSql->sqlstr, pCmd->payload, tstrerror(code));
pStream->fp(pStream->param, NULL, NULL); pStream->fp(pStream->param, NULL, NULL);
return; return;
} }
registerSqlObj(pSql);
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
...@@ -522,13 +538,21 @@ static void tscCreateStream(void *param, TAOS_RES *res, int code) { ...@@ -522,13 +538,21 @@ static void tscCreateStream(void *param, TAOS_RES *res, int code) {
pStream->ctime = taosGetTimestamp(pStream->precision); pStream->ctime = taosGetTimestamp(pStream->precision);
pStream->etime = pQueryInfo->window.ekey; pStream->etime = pQueryInfo->window.ekey;
tscAddIntoStreamList(pStream); if (tscSetSlidingWindowInfo(pSql, pStream) != TSDB_CODE_SUCCESS) {
pSql->res.code = code;
tscError("%p stream %p open failed, since the interval value is incorrect", pSql, pStream);
pStream->fp(pStream->param, NULL, NULL);
return;
}
tscSetSlidingWindowInfo(pSql, pStream);
pStream->stime = tscGetStreamStartTimestamp(pSql, pStream, pStream->stime); pStream->stime = tscGetStreamStartTimestamp(pSql, pStream, pStream->stime);
int64_t starttime = tscGetLaunchTimestamp(pStream); int64_t starttime = tscGetLaunchTimestamp(pStream);
pCmd->command = TSDB_SQL_SELECT; pCmd->command = TSDB_SQL_SELECT;
tscAddIntoStreamList(pStream);
taosTmrReset(tscProcessStreamTimer, (int32_t)starttime, pStream, tscTmr, &pStream->pTimer); taosTmrReset(tscProcessStreamTimer, (int32_t)starttime, pStream, tscTmr, &pStream->pTimer);
tscDebug("%p stream:%p is opened, query on:%s, interval:%" PRId64 ", sliding:%" PRId64 ", first launched in:%" PRId64 ", sql:%s", pSql, tscDebug("%p stream:%p is opened, query on:%s, interval:%" PRId64 ", sliding:%" PRId64 ", first launched in:%" PRId64 ", sql:%s", pSql,
...@@ -585,12 +609,15 @@ TAOS_STREAM *taos_open_stream(TAOS *taos, const char *sqlstr, void (*fp)(void *p ...@@ -585,12 +609,15 @@ TAOS_STREAM *taos_open_stream(TAOS *taos, const char *sqlstr, void (*fp)(void *p
pSql->fp = tscCreateStream; pSql->fp = tscCreateStream;
pSql->fetchFp = tscCreateStream; pSql->fetchFp = tscCreateStream;
registerSqlObj(pSql);
int32_t code = tsParseSql(pSql, true); int32_t code = tsParseSql(pSql, true);
if (code == TSDB_CODE_SUCCESS) { if (code == TSDB_CODE_SUCCESS) {
tscCreateStream(pStream, pSql, code); tscCreateStream(pStream, pSql, code);
} else if (code != TSDB_CODE_TSC_ACTION_IN_PROGRESS) { } else if (code != TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
tscError("%p open stream failed, sql:%s, code:%s", pSql, sqlstr, tstrerror(pRes->code)); tscError("%p open stream failed, sql:%s, code:%s", pSql, sqlstr, tstrerror(pRes->code));
tscFreeSqlObj(pSql); taosReleaseRef(tscObjRef, pSql->self);
free(pStream); free(pStream);
return NULL; return NULL;
} }
......
...@@ -188,8 +188,10 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, SJoinSupporter* pSupporter1, SJ ...@@ -188,8 +188,10 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, SJoinSupporter* pSupporter1, SJ
tsBufFlush(output2); tsBufFlush(output2);
tsBufDestroy(pSupporter1->pTSBuf); tsBufDestroy(pSupporter1->pTSBuf);
pSupporter1->pTSBuf = NULL;
tsBufDestroy(pSupporter2->pTSBuf); tsBufDestroy(pSupporter2->pTSBuf);
pSupporter2->pTSBuf = NULL;
TSKEY et = taosGetTimestampUs(); TSKEY et = taosGetTimestampUs();
tscDebug("%p input1:%" PRId64 ", input2:%" PRId64 ", final:%" PRId64 " in %d vnodes for secondary query after ts blocks " tscDebug("%p input1:%" PRId64 ", input2:%" PRId64 ", final:%" PRId64 " in %d vnodes for secondary query after ts blocks "
"intersecting, skey:%" PRId64 ", ekey:%" PRId64 ", numOfVnode:%d, elapsed time:%" PRId64 " us", "intersecting, skey:%" PRId64 ", ekey:%" PRId64 ", numOfVnode:%d, elapsed time:%" PRId64 " us",
...@@ -219,12 +221,9 @@ SJoinSupporter* tscCreateJoinSupporter(SSqlObj* pSql, int32_t index) { ...@@ -219,12 +221,9 @@ SJoinSupporter* tscCreateJoinSupporter(SSqlObj* pSql, int32_t index) {
assert (pSupporter->uid != 0); assert (pSupporter->uid != 0);
taosGetTmpfilePath("join-", pSupporter->path); taosGetTmpfilePath("join-", pSupporter->path);
pSupporter->f = fopen(pSupporter->path, "w");
// todo handle error // do NOT create file here to reduce crash generated file left issue
if (pSupporter->f == NULL) { pSupporter->f = NULL;
tscError("%p failed to create tmp file:%s, reason:%s", pSql, pSupporter->path, strerror(errno));
}
return pSupporter; return pSupporter;
} }
...@@ -244,12 +243,19 @@ static void tscDestroyJoinSupporter(SJoinSupporter* pSupporter) { ...@@ -244,12 +243,19 @@ static void tscDestroyJoinSupporter(SJoinSupporter* pSupporter) {
tscFieldInfoClear(&pSupporter->fieldsInfo); tscFieldInfoClear(&pSupporter->fieldsInfo);
if (pSupporter->pTSBuf != NULL) {
tsBufDestroy(pSupporter->pTSBuf);
pSupporter->pTSBuf = NULL;
}
unlink(pSupporter->path);
if (pSupporter->f != NULL) { if (pSupporter->f != NULL) {
fclose(pSupporter->f); fclose(pSupporter->f);
unlink(pSupporter->path);
pSupporter->f = NULL; pSupporter->f = NULL;
} }
if (pSupporter->pVgroupTables != NULL) { if (pSupporter->pVgroupTables != NULL) {
taosArrayDestroy(pSupporter->pVgroupTables); taosArrayDestroy(pSupporter->pVgroupTables);
pSupporter->pVgroupTables = NULL; pSupporter->pVgroupTables = NULL;
...@@ -459,7 +465,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { ...@@ -459,7 +465,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid); int16_t colId = tscGetJoinTagColIdByUid(&pQueryInfo->tagCond, pTableMetaInfo->pTableMeta->id.uid);
// set the tag column id for executor to extract correct tag value // set the tag column id for executor to extract correct tag value
pExpr->param[0] = (tVariant) {.i64Key = colId, .nType = TSDB_DATA_TYPE_BIGINT, .nLen = sizeof(int64_t)}; pExpr->param[0] = (tVariant) {.i64 = colId, .nType = TSDB_DATA_TYPE_BIGINT, .nLen = sizeof(int64_t)};
pExpr->numOfParams = 1; pExpr->numOfParams = 1;
} }
...@@ -526,6 +532,8 @@ static void quitAllSubquery(SSqlObj* pSqlObj, SJoinSupporter* pSupporter) { ...@@ -526,6 +532,8 @@ static void quitAllSubquery(SSqlObj* pSqlObj, SJoinSupporter* pSupporter) {
tscError("%p all subquery return and query failed, global code:%s", pSqlObj, tstrerror(pSqlObj->res.code)); tscError("%p all subquery return and query failed, global code:%s", pSqlObj, tstrerror(pSqlObj->res.code));
freeJoinSubqueryObj(pSqlObj); freeJoinSubqueryObj(pSqlObj);
} }
//tscDestroyJoinSupporter(pSupporter);
} }
// update the query time range according to the join results on timestamp // update the query time range according to the join results on timestamp
...@@ -642,7 +650,7 @@ static void issueTSCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj* ...@@ -642,7 +650,7 @@ static void issueTSCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj*
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
SSqlExpr *pExpr = tscSqlExprGet(pQueryInfo, 0); SSqlExpr *pExpr = tscSqlExprGet(pQueryInfo, 0);
int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid); int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid);
pExpr->param->i64Key = tagColId; pExpr->param->i64 = tagColId;
pExpr->numOfParams = 1; pExpr->numOfParams = 1;
} }
...@@ -779,7 +787,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -779,7 +787,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
pParentSql->res.code = numOfRows; pParentSql->res.code = numOfRows;
quitAllSubquery(pParentSql, pSupporter); quitAllSubquery(pParentSql, pSupporter);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -796,7 +804,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -796,7 +804,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
pParentSql->res.code = TAOS_SYSTEM_ERROR(errno); pParentSql->res.code = TAOS_SYSTEM_ERROR(errno);
quitAllSubquery(pParentSql, pSupporter); quitAllSubquery(pParentSql, pSupporter);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -845,7 +853,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -845,7 +853,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
freeJoinSubqueryObj(pParentSql); freeJoinSubqueryObj(pParentSql);
pParentSql->res.code = code; pParentSql->res.code = code;
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
taosArrayDestroy(s1); taosArrayDestroy(s1);
taosArrayDestroy(s2); taosArrayDestroy(s2);
...@@ -916,11 +924,27 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -916,11 +924,27 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
pParentSql->res.code = numOfRows; pParentSql->res.code = numOfRows;
quitAllSubquery(pParentSql, pSupporter); quitAllSubquery(pParentSql, pSupporter);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
if (numOfRows > 0) { // write the compressed timestamp to disk file if (numOfRows > 0) { // write the compressed timestamp to disk file
if(pSupporter->f == NULL) {
pSupporter->f = fopen(pSupporter->path, "w");
if (pSupporter->f == NULL) {
tscError("%p failed to create tmp file:%s, reason:%s", pSql, pSupporter->path, strerror(errno));
pParentSql->res.code = TAOS_SYSTEM_ERROR(errno);
quitAllSubquery(pParentSql, pSupporter);
tscAsyncResultOnError(pParentSql);
return;
}
}
fwrite(pRes->data, (size_t)pRes->numOfRows, 1, pSupporter->f); fwrite(pRes->data, (size_t)pRes->numOfRows, 1, pSupporter->f);
fclose(pSupporter->f); fclose(pSupporter->f);
pSupporter->f = NULL; pSupporter->f = NULL;
...@@ -930,7 +954,10 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -930,7 +954,10 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
tscError("%p invalid ts comp file from vnode, abort subquery, file size:%d", pSql, numOfRows); tscError("%p invalid ts comp file from vnode, abort subquery, file size:%d", pSql, numOfRows);
pParentSql->res.code = TAOS_SYSTEM_ERROR(errno); pParentSql->res.code = TAOS_SYSTEM_ERROR(errno);
tscQueueAsyncRes(pParentSql);
quitAllSubquery(pParentSql, pSupporter);
tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -1028,7 +1055,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR ...@@ -1028,7 +1055,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
pParentSql->res.code = numOfRows; pParentSql->res.code = numOfRows;
tscError("%p retrieve failed, index:%d, code:%s", pSql, pSupporter->subqueryIndex, tstrerror(numOfRows)); tscError("%p retrieve failed, index:%d, code:%s", pSql, pSupporter->subqueryIndex, tstrerror(numOfRows));
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -1155,7 +1182,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { ...@@ -1155,7 +1182,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
if (pSql->res.code == TSDB_CODE_SUCCESS) { if (pSql->res.code == TSDB_CODE_SUCCESS) {
(*pSql->fp)(pSql->param, pSql, 0); (*pSql->fp)(pSql->param, pSql, 0);
} else { } else {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
return; return;
...@@ -1233,7 +1260,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { ...@@ -1233,7 +1260,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
if (pSql->res.code == TSDB_CODE_SUCCESS) { if (pSql->res.code == TSDB_CODE_SUCCESS) {
(*pSql->fp)(pSql->param, pSql, 0); (*pSql->fp)(pSql->param, pSql, 0);
} else { } else {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
return; return;
...@@ -1335,16 +1362,25 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { ...@@ -1335,16 +1362,25 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) {
SJoinSupporter* pSupporter = (SJoinSupporter*)param; SJoinSupporter* pSupporter = (SJoinSupporter*)param;
SSqlObj* pParentSql = pSupporter->pObj; SSqlObj* pParentSql = pSupporter->pObj;
// There is only one subquery and table for each subquery. // There is only one subquery and table for each subquery.
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(&pSql->cmd, 0);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
assert(pQueryInfo->numOfTables == 1 && pSql->cmd.numOfClause == 1); assert(pQueryInfo->numOfTables == 1 && pSql->cmd.numOfClause == 1);
// retrieve actual query results from vnode during the second stage join subquery // retrieve actual query results from vnode during the second stage join subquery
if (pParentSql->res.code != TSDB_CODE_SUCCESS) { if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
tscError("%p abort query due to other subquery failure. code:%d, global code:%d", pSql, code, pParentSql->res.code); tscError("%p abort query due to other subquery failure. code:%d, global code:%d", pSql, code, pParentSql->res.code);
if (!(pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0))) {
if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) {
return;
}
}
quitAllSubquery(pParentSql, pSupporter); quitAllSubquery(pParentSql, pSupporter);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -1356,8 +1392,14 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { ...@@ -1356,8 +1392,14 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) {
tscError("%p abort query, code:%s, global code:%s", pSql, tstrerror(code), tstrerror(pParentSql->res.code)); tscError("%p abort query, code:%s, global code:%s", pSql, tstrerror(code), tstrerror(pParentSql->res.code));
pParentSql->res.code = code; pParentSql->res.code = code;
if (!(pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0))) {
if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) {
return;
}
}
quitAllSubquery(pParentSql, pSupporter); quitAllSubquery(pParentSql, pSupporter);
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
return; return;
} }
...@@ -1378,9 +1420,6 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { ...@@ -1378,9 +1420,6 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) {
return; return;
} }
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
// In case of consequence query from other vnode, do not wait for other query response here. // In case of consequence query from other vnode, do not wait for other query response here.
if (!(pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0))) { if (!(pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0))) {
if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) { if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) {
...@@ -1403,7 +1442,7 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { ...@@ -1403,7 +1442,7 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) {
if (pParentSql->res.code == TSDB_CODE_SUCCESS) { if (pParentSql->res.code == TSDB_CODE_SUCCESS) {
(*pParentSql->fp)(pParentSql->param, pParentSql, 0); (*pParentSql->fp)(pParentSql->param, pParentSql, 0);
} else { } else {
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
} }
} }
} }
...@@ -1520,7 +1559,7 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter ...@@ -1520,7 +1559,7 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid); int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid);
pExpr->param->i64Key = tagColId; pExpr->param->i64 = tagColId;
pExpr->numOfParams = 1; pExpr->numOfParams = 1;
} }
...@@ -1612,7 +1651,7 @@ void tscHandleMasterJoinQuery(SSqlObj* pSql) { ...@@ -1612,7 +1651,7 @@ void tscHandleMasterJoinQuery(SSqlObj* pSql) {
_error: _error:
pRes->code = code; pRes->code = code;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
static void doCleanupSubqueries(SSqlObj *pSql, int32_t numOfSubs) { static void doCleanupSubqueries(SSqlObj *pSql, int32_t numOfSubs) {
...@@ -1631,6 +1670,25 @@ static void doCleanupSubqueries(SSqlObj *pSql, int32_t numOfSubs) { ...@@ -1631,6 +1670,25 @@ static void doCleanupSubqueries(SSqlObj *pSql, int32_t numOfSubs) {
} }
} }
void tscLockByThread(int64_t *lockedBy) {
int64_t tid = taosGetSelfPthreadId();
int i = 0;
while (atomic_val_compare_exchange_64(lockedBy, 0, tid) != 0) {
if (++i % 100 == 0) {
sched_yield();
}
}
}
void tscUnlockByThread(int64_t *lockedBy) {
int64_t tid = taosGetSelfPthreadId();
if (atomic_val_compare_exchange_64(lockedBy, tid, 0) != tid) {
assert(false);
}
}
int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) {
SSqlRes *pRes = &pSql->res; SSqlRes *pRes = &pSql->res;
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
...@@ -1666,7 +1724,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { ...@@ -1666,7 +1724,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) {
int32_t ret = tscLocalReducerEnvCreate(pSql, &pMemoryBuf, &pDesc, &pModel, &pFinalModel, nBufferSize); int32_t ret = tscLocalReducerEnvCreate(pSql, &pMemoryBuf, &pDesc, &pModel, &pFinalModel, nBufferSize);
if (ret != 0) { if (ret != 0) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
tfree(pMemoryBuf); tfree(pMemoryBuf);
return ret; return ret;
} }
...@@ -1680,7 +1738,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { ...@@ -1680,7 +1738,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pModel, pFinalModel,pState->numOfSub); tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pModel, pFinalModel,pState->numOfSub);
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return ret; return ret;
} }
...@@ -1890,7 +1948,7 @@ void tscHandleSubqueryError(SRetrieveSupport *trsupport, SSqlObj *pSql, int numO ...@@ -1890,7 +1948,7 @@ void tscHandleSubqueryError(SRetrieveSupport *trsupport, SSqlObj *pSql, int numO
(*pParentSql->fp)(pParentSql->param, pParentSql, pParentSql->res.code); (*pParentSql->fp)(pParentSql->param, pParentSql, pParentSql->res.code);
} else { // regular super table query } else { // regular super table query
if (pParentSql->res.code != TSDB_CODE_SUCCESS) { if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
} }
} }
} }
...@@ -1968,7 +2026,7 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p ...@@ -1968,7 +2026,7 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p
if (pParentSql->res.code == TSDB_CODE_SUCCESS) { if (pParentSql->res.code == TSDB_CODE_SUCCESS) {
(*pParentSql->fp)(pParentSql->param, pParentSql, 0); (*pParentSql->fp)(pParentSql->param, pParentSql, 0);
} else { } else {
tscQueueAsyncRes(pParentSql); tscAsyncResultOnError(pParentSql);
} }
} }
...@@ -2093,7 +2151,15 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo ...@@ -2093,7 +2151,15 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo
return pNew; return pNew;
} }
// todo there is are race condition in this function, while cancel is called by user.
void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) { void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) {
// the param may be null, since it may be done by other query threads. and the asyncOnError may enter in this
// function while kill query by a user.
if (param == NULL) {
assert(code != TSDB_CODE_SUCCESS);
return;
}
SRetrieveSupport *trsupport = (SRetrieveSupport *) param; SRetrieveSupport *trsupport = (SRetrieveSupport *) param;
SSqlObj* pParentSql = trsupport->pParentSql; SSqlObj* pParentSql = trsupport->pParentSql;
...@@ -2220,7 +2286,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) ...@@ -2220,7 +2286,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows)
(*pParentObj->fp)(pParentObj->param, pParentObj, v); (*pParentObj->fp)(pParentObj->param, pParentObj, v);
} else { } else {
if (!needRetryInsert(pParentObj, numOfSub)) { if (!needRetryInsert(pParentObj, numOfSub)) {
tscQueueAsyncRes(pParentObj); tscAsyncResultOnError(pParentObj);
return; return;
} }
...@@ -2231,7 +2297,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) ...@@ -2231,7 +2297,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows)
numOfFailed += 1; numOfFailed += 1;
// clean up tableMeta in cache // clean up tableMeta in cache
tscFreeQueryInfo(&pSql->cmd, true); tscFreeQueryInfo(&pSql->cmd);
SQueryInfo* pQueryInfo = tscGetQueryInfoDetailSafely(&pSql->cmd, 0); SQueryInfo* pQueryInfo = tscGetQueryInfoDetailSafely(&pSql->cmd, 0);
STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, pSql->cmd.clauseIndex, 0); STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, pSql->cmd.clauseIndex, 0);
tscAddTableMetaInfo(pQueryInfo, pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL); tscAddTableMetaInfo(pQueryInfo, pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL);
...@@ -2243,15 +2309,16 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) ...@@ -2243,15 +2309,16 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows)
tscError("%p Async insertion completed, total inserted:%d rows, numOfFailed:%d, numOfTotal:%d", pParentObj, tscError("%p Async insertion completed, total inserted:%d rows, numOfFailed:%d, numOfTotal:%d", pParentObj,
pParentObj->res.numOfRows, numOfFailed, numOfSub); pParentObj->res.numOfRows, numOfFailed, numOfSub);
tscDebug("%p cleanup %d tableMeta in cache", pParentObj, pParentObj->cmd.numOfTables); tscDebug("%p cleanup %d tableMeta in hashTable", pParentObj, pParentObj->cmd.numOfTables);
for(int32_t i = 0; i < pParentObj->cmd.numOfTables; ++i) { for(int32_t i = 0; i < pParentObj->cmd.numOfTables; ++i) {
taosCacheRelease(tscMetaCache, (void**)&(pParentObj->cmd.pTableMetaList[i]), true); char* name = pParentObj->cmd.pTableNameList[i];
taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
} }
pParentObj->cmd.parseFinished = false; pParentObj->cmd.parseFinished = false;
pParentObj->subState.numOfRemain = numOfFailed; pParentObj->subState.numOfRemain = numOfFailed;
tscResetSqlCmdObj(&pParentObj->cmd, false); tscResetSqlCmdObj(&pParentObj->cmd);
// in case of insert, redo parsing the sql string and build new submit data block for two reasons: // in case of insert, redo parsing the sql string and build new submit data block for two reasons:
// 1. the table Id(tid & uid) may have been update, the submit block needs to be updated accordingly. // 1. the table Id(tid & uid) may have been update, the submit block needs to be updated accordingly.
...@@ -2264,7 +2331,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) ...@@ -2264,7 +2331,7 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows)
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
pParentObj->res.code = code; pParentObj->res.code = code;
tscQueueAsyncRes(pParentObj); tscAsyncResultOnError(pParentObj);
return; return;
} }
...@@ -2288,7 +2355,7 @@ int32_t tscHandleInsertRetry(SSqlObj* pParent, SSqlObj* pSql) { ...@@ -2288,7 +2355,7 @@ int32_t tscHandleInsertRetry(SSqlObj* pParent, SSqlObj* pSql) {
int32_t code = tscCopyDataBlockToPayload(pSql, pTableDataBlock); int32_t code = tscCopyDataBlockToPayload(pSql, pTableDataBlock);
if ((pRes->code = code)!= TSDB_CODE_SUCCESS) { if ((pRes->code = code)!= TSDB_CODE_SUCCESS) {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return code; // here the pSql may have been released already. return code; // here the pSql may have been released already.
} }
...@@ -2481,7 +2548,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { ...@@ -2481,7 +2548,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) {
SSqlRes* pRes = &pSql->res; SSqlRes* pRes = &pSql->res;
if (pRes->code != TSDB_CODE_SUCCESS) { if (pRes->code != TSDB_CODE_SUCCESS) {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -2496,7 +2563,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { ...@@ -2496,7 +2563,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) {
if (pRes->tsrow == NULL || pRes->buffer == NULL || pRes->length == NULL) { if (pRes->tsrow == NULL || pRes->buffer == NULL || pRes->length == NULL) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
return; return;
} }
...@@ -2508,7 +2575,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { ...@@ -2508,7 +2575,7 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) {
if (pRes->code == TSDB_CODE_SUCCESS) { if (pRes->code == TSDB_CODE_SUCCESS) {
(*pSql->fp)(pSql->param, pSql, pRes->numOfRows); (*pSql->fp)(pSql->param, pSql, pRes->numOfRows);
} else { } else {
tscQueueAsyncRes(pSql); tscAsyncResultOnError(pSql);
} }
} }
......
...@@ -31,15 +31,20 @@ ...@@ -31,15 +31,20 @@
#include "tlocale.h" #include "tlocale.h"
// global, not configurable // global, not configurable
SCacheObj *tscMetaCache; // table meta cache #define TSC_VAR_NOT_RELEASE 1
SHashObj *tscHashMap; // hash map to keep the global vgroup info #define TSC_VAR_RELEASED 0
int tscObjRef = -1;
void *tscTmr;
void *tscQhandle;
void *tscCheckDiskUsageTmr;
int tscRefId = -1;
int tscNumOfObj = 0; // number of sqlObj in current process.
int32_t sentinel = TSC_VAR_NOT_RELEASE;
SHashObj *tscVgroupMap; // hash map to keep the global vgroup info
SHashObj *tscTableMetaInfo; // table meta info
int32_t tscObjRef = -1;
void *tscTmr;
void *tscQhandle;
int32_t tscRefId = -1;
int32_t tscNumOfObj = 0; // number of sqlObj in current process.
static void *tscCheckDiskUsageTmr;
static pthread_once_t tscinit = PTHREAD_ONCE_INIT; static pthread_once_t tscinit = PTHREAD_ONCE_INIT;
void tscCheckDiskUsage(void *UNUSED_PARAM(para), void* UNUSED_PARAM(param)) { void tscCheckDiskUsage(void *UNUSED_PARAM(para), void* UNUSED_PARAM(param)) {
...@@ -129,11 +134,11 @@ void taos_init_imp(void) { ...@@ -129,11 +134,11 @@ void taos_init_imp(void) {
taosTmrReset(tscCheckDiskUsage, 10, NULL, tscTmr, &tscCheckDiskUsageTmr); taosTmrReset(tscCheckDiskUsage, 10, NULL, tscTmr, &tscCheckDiskUsageTmr);
} }
int64_t refreshTime = 10; // 10 seconds by default if (tscTableMetaInfo == NULL) {
if (tscMetaCache == NULL) {
tscMetaCache = taosCacheInit(TSDB_DATA_TYPE_BINARY, refreshTime, false, tscFreeTableMetaHelper, "tableMeta");
tscObjRef = taosOpenRef(40960, tscFreeRegisteredSqlObj); tscObjRef = taosOpenRef(40960, tscFreeRegisteredSqlObj);
tscHashMap = taosHashInit(1024, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK); tscVgroupMap = taosHashInit(256, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK);
tscTableMetaInfo = taosHashInit(1024, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
tscDebug("TableMeta:%p", tscTableMetaInfo);
} }
tscRefId = taosOpenRef(200, tscCloseTscObj); tscRefId = taosOpenRef(200, tscCloseTscObj);
...@@ -151,30 +156,38 @@ void taos_init() { pthread_once(&tscinit, taos_init_imp); } ...@@ -151,30 +156,38 @@ void taos_init() { pthread_once(&tscinit, taos_init_imp); }
void taos_cleanup(void) { void taos_cleanup(void) {
tscDebug("start to cleanup client environment"); tscDebug("start to cleanup client environment");
void* m = tscMetaCache; if (atomic_val_compare_exchange_32(&sentinel, TSC_VAR_NOT_RELEASE, TSC_VAR_RELEASED) != TSC_VAR_NOT_RELEASE) {
if (m != NULL && atomic_val_compare_exchange_ptr(&tscMetaCache, m, 0) == m) { return;
taosCacheCleanup(m);
} }
int refId = atomic_exchange_32(&tscObjRef, -1); taosHashCleanup(tscTableMetaInfo);
if (refId != -1) { tscTableMetaInfo = NULL;
taosCloseRef(refId);
}
m = tscQhandle; taosHashCleanup(tscVgroupMap);
if (m != NULL && atomic_val_compare_exchange_ptr(&tscQhandle, m, 0) == m) { tscVgroupMap = NULL;
taosCleanUpScheduler(m);
} int32_t id = tscObjRef;
tscObjRef = -1;
taosCloseRef(id);
void* p = tscQhandle;
tscQhandle = NULL;
taosCleanUpScheduler(p);
id = tscRefId;
tscRefId = -1;
taosCloseRef(id);
taosCloseRef(tscRefId);
taosCleanupKeywordsTable(); taosCleanupKeywordsTable();
taosCloseLog(); taosCloseLog();
if (tscEmbedded == 0) rpcCleanup();
m = tscTmr; if (tscEmbedded == 0) {
if (m != NULL && atomic_val_compare_exchange_ptr(&tscTmr, m, 0) == m) { rpcCleanup();
taosTmrCleanUp(m);
} }
p = tscTmr;
tscTmr = NULL;
taosTmrCleanUp(p);
} }
static int taos_options_imp(TSDB_OPTION option, const char *pStr) { static int taos_options_imp(TSDB_OPTION option, const char *pStr) {
......
...@@ -18,7 +18,6 @@ ...@@ -18,7 +18,6 @@
#include "os.h" #include "os.h"
#include "qAst.h" #include "qAst.h"
#include "taosmsg.h" #include "taosmsg.h"
#include "tcache.h"
#include "tkey.h" #include "tkey.h"
#include "tmd5.h" #include "tmd5.h"
#include "tscLocalMerge.h" #include "tscLocalMerge.h"
...@@ -31,7 +30,7 @@ ...@@ -31,7 +30,7 @@
#include "ttokendef.h" #include "ttokendef.h"
static void freeQueryInfoImpl(SQueryInfo* pQueryInfo); static void freeQueryInfoImpl(SQueryInfo* pQueryInfo);
static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, const char* address, bool removeFromCache); static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo);
SCond* tsGetSTableQueryCond(STagCond* pTagCond, uint64_t uid) { SCond* tsGetSTableQueryCond(STagCond* pTagCond, uint64_t uid) {
if (pTagCond->pCond == NULL) { if (pTagCond->pCond == NULL) {
...@@ -72,6 +71,7 @@ void tsSetSTableQueryCond(STagCond* pTagCond, uint64_t uid, SBufferWriter* bw) { ...@@ -72,6 +71,7 @@ void tsSetSTableQueryCond(STagCond* pTagCond, uint64_t uid, SBufferWriter* bw) {
bool tscQueryTags(SQueryInfo* pQueryInfo) { bool tscQueryTags(SQueryInfo* pQueryInfo) {
int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < numOfCols; ++i) { for (int32_t i = 0; i < numOfCols; ++i) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i); SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
int32_t functId = pExpr->functionId; int32_t functId = pExpr->functionId;
...@@ -315,7 +315,7 @@ void tscSetResRawPtr(SSqlRes* pRes, SQueryInfo* pQueryInfo) { ...@@ -315,7 +315,7 @@ void tscSetResRawPtr(SSqlRes* pRes, SQueryInfo* pQueryInfo) {
} else { } else {
for (int32_t k = 0; k < pRes->numOfRows; ++k) { for (int32_t k = 0; k < pRes->numOfRows; ++k) {
char* p = ((char**)pRes->urow)[i] + k * pInfo->field.bytes; char* p = ((char**)pRes->urow)[i] + k * pInfo->field.bytes;
memcpy(p, &pInfo->pSqlExpr->param[1].i64Key, pInfo->field.bytes); memcpy(p, &pInfo->pSqlExpr->param[1].i64, pInfo->field.bytes);
} }
} }
} }
...@@ -379,17 +379,16 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) { ...@@ -379,17 +379,16 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) {
pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free pRes->data = NULL; // pRes->data points to the buffer of pRsp, no need to free
} }
void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeFromCache) { void tscFreeQueryInfo(SSqlCmd* pCmd) {
if (pCmd == NULL || pCmd->numOfClause == 0) { if (pCmd == NULL || pCmd->numOfClause == 0) {
return; return;
} }
for (int32_t i = 0; i < pCmd->numOfClause; ++i) { for (int32_t i = 0; i < pCmd->numOfClause; ++i) {
char* addr = (char*)pCmd - offsetof(SSqlObj, cmd);
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, i); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, i);
freeQueryInfoImpl(pQueryInfo); freeQueryInfoImpl(pQueryInfo);
clearAllTableMetaInfo(pQueryInfo, (const char*)addr, removeFromCache); clearAllTableMetaInfo(pQueryInfo);
tfree(pQueryInfo); tfree(pQueryInfo);
} }
...@@ -397,7 +396,7 @@ void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeFromCache) { ...@@ -397,7 +396,7 @@ void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeFromCache) {
tfree(pCmd->pQueryInfo); tfree(pCmd->pQueryInfo);
} }
void tscResetSqlCmdObj(SSqlCmd* pCmd, bool removeFromCache) { void tscResetSqlCmdObj(SSqlCmd* pCmd) {
pCmd->command = 0; pCmd->command = 0;
pCmd->numOfCols = 0; pCmd->numOfCols = 0;
pCmd->count = 0; pCmd->count = 0;
...@@ -407,17 +406,17 @@ void tscResetSqlCmdObj(SSqlCmd* pCmd, bool removeFromCache) { ...@@ -407,17 +406,17 @@ void tscResetSqlCmdObj(SSqlCmd* pCmd, bool removeFromCache) {
pCmd->autoCreated = 0; pCmd->autoCreated = 0;
for(int32_t i = 0; i < pCmd->numOfTables; ++i) { for(int32_t i = 0; i < pCmd->numOfTables; ++i) {
if (pCmd->pTableMetaList && pCmd->pTableMetaList[i]) { if (pCmd->pTableNameList && pCmd->pTableNameList[i]) {
taosCacheRelease(tscMetaCache, (void**)&(pCmd->pTableMetaList[i]), false); tfree(pCmd->pTableNameList[i]);
} }
} }
pCmd->numOfTables = 0; pCmd->numOfTables = 0;
tfree(pCmd->pTableMetaList); tfree(pCmd->pTableNameList);
pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList); pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList);
pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks);
tscFreeQueryInfo(pCmd, removeFromCache); tscFreeQueryInfo(pCmd);
} }
void tscFreeSqlResult(SSqlObj* pSql) { void tscFreeSqlResult(SSqlObj* pSql) {
...@@ -460,22 +459,24 @@ void tscFreeRegisteredSqlObj(void *pSql) { ...@@ -460,22 +459,24 @@ void tscFreeRegisteredSqlObj(void *pSql) {
assert(RID_VALID(p->self)); assert(RID_VALID(p->self));
tscFreeSqlObj(p);
taosReleaseRef(tscRefId, pTscObj->rid);
int32_t num = atomic_sub_fetch_32(&pTscObj->numOfObj, 1); int32_t num = atomic_sub_fetch_32(&pTscObj->numOfObj, 1);
int32_t total = atomic_sub_fetch_32(&tscNumOfObj, 1); int32_t total = atomic_sub_fetch_32(&tscNumOfObj, 1);
tscDebug("%p free SqlObj, total in tscObj:%d, total:%d", pSql, num, total); tscDebug("%p free SqlObj, total in tscObj:%d, total:%d", pSql, num, total);
} tscFreeSqlObj(p);
taosReleaseRef(tscRefId, pTscObj->rid);
void tscFreeTableMetaHelper(void *pTableMeta) { }
STableMeta* p = (STableMeta*) pTableMeta;
int32_t numOfEps1 = p->corVgroupInfo.numOfEps; void tscFreeMetaSqlObj(int64_t *rid){
assert(numOfEps1 >= 0 && numOfEps1 <= TSDB_MAX_REPLICA); if (RID_VALID(*rid)) {
SSqlObj* pSql = (SSqlObj*)taosAcquireRef(tscObjRef, *rid);
if (pSql) {
taosRemoveRef(tscObjRef, *rid);
taosReleaseRef(tscObjRef, *rid);
}
for(int32_t i = 0; i < numOfEps1; ++i) { *rid = 0;
tfree(p->corVgroupInfo.epAddr[i].fqdn);
} }
} }
...@@ -488,6 +489,9 @@ void tscFreeSqlObj(SSqlObj* pSql) { ...@@ -488,6 +489,9 @@ void tscFreeSqlObj(SSqlObj* pSql) {
pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED; pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED;
tscFreeMetaSqlObj(&pSql->metaRid);
tscFreeMetaSqlObj(&pSql->svgroupRid);
tscFreeSubobj(pSql); tscFreeSubobj(pSql);
SSqlCmd* pCmd = &pSql->cmd; SSqlCmd* pCmd = &pSql->cmd;
...@@ -506,7 +510,7 @@ void tscFreeSqlObj(SSqlObj* pSql) { ...@@ -506,7 +510,7 @@ void tscFreeSqlObj(SSqlObj* pSql) {
pSql->self = 0; pSql->self = 0;
tscFreeSqlResult(pSql); tscFreeSqlResult(pSql);
tscResetSqlCmdObj(pCmd, false); tscResetSqlCmdObj(pCmd);
tfree(pCmd->tagData.data); tfree(pCmd->tagData.data);
pCmd->tagData.dataLen = 0; pCmd->tagData.dataLen = 0;
...@@ -516,6 +520,7 @@ void tscFreeSqlObj(SSqlObj* pSql) { ...@@ -516,6 +520,7 @@ void tscFreeSqlObj(SSqlObj* pSql) {
pCmd->allocSize = 0; pCmd->allocSize = 0;
tsem_destroy(&pSql->rspSem); tsem_destroy(&pSql->rspSem);
memset(pSql, 0, sizeof(*pSql));
free(pSql); free(pSql);
} }
...@@ -529,7 +534,7 @@ void tscDestroyDataBlock(STableDataBlocks* pDataBlock) { ...@@ -529,7 +534,7 @@ void tscDestroyDataBlock(STableDataBlocks* pDataBlock) {
// free the refcount for metermeta // free the refcount for metermeta
if (pDataBlock->pTableMeta != NULL) { if (pDataBlock->pTableMeta != NULL) {
taosCacheRelease(tscMetaCache, (void**)&(pDataBlock->pTableMeta), false); tfree(pDataBlock->pTableMeta);
} }
tfree(pDataBlock); tfree(pDataBlock);
...@@ -600,15 +605,15 @@ int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) { ...@@ -600,15 +605,15 @@ int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) {
// set the correct table meta object, the table meta has been locked in pDataBlocks, so it must be in the cache // set the correct table meta object, the table meta has been locked in pDataBlocks, so it must be in the cache
if (pTableMetaInfo->pTableMeta != pDataBlock->pTableMeta) { if (pTableMetaInfo->pTableMeta != pDataBlock->pTableMeta) {
tstrncpy(pTableMetaInfo->name, pDataBlock->tableId, sizeof(pTableMetaInfo->name)); tstrncpy(pTableMetaInfo->name, pDataBlock->tableName, sizeof(pTableMetaInfo->name));
if (pTableMetaInfo->pTableMeta != NULL) { if (pTableMetaInfo->pTableMeta != NULL) {
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), false); tfree(pTableMetaInfo->pTableMeta);
} }
pTableMetaInfo->pTableMeta = taosCacheTransfer(tscMetaCache, (void**)&pDataBlock->pTableMeta); pTableMetaInfo->pTableMeta = tscTableMetaClone(pDataBlock->pTableMeta);
} else { } else {
assert(strncmp(pTableMetaInfo->name, pDataBlock->tableId, tListLen(pDataBlock->tableId)) == 0); assert(strncmp(pTableMetaInfo->name, pDataBlock->tableName, tListLen(pDataBlock->tableName)) == 0);
} }
/* /*
...@@ -671,14 +676,10 @@ int32_t tscCreateDataBlock(size_t initialSize, int32_t rowSize, int32_t startOff ...@@ -671,14 +676,10 @@ int32_t tscCreateDataBlock(size_t initialSize, int32_t rowSize, int32_t startOff
dataBuf->size = startOffset; dataBuf->size = startOffset;
dataBuf->tsSource = -1; dataBuf->tsSource = -1;
tstrncpy(dataBuf->tableId, name, sizeof(dataBuf->tableId)); tstrncpy(dataBuf->tableName, name, sizeof(dataBuf->tableName));
/* //Here we keep the tableMeta to avoid it to be remove by other threads.
* The table meta may be released since the table meta cache are completed clean by other thread dataBuf->pTableMeta = tscTableMetaClone(pTableMeta);
* due to operation such as drop database. So here we add the reference count directly instead of invoke
* taosGetDataFromCache, which may return NULL value.
*/
dataBuf->pTableMeta = taosCacheAcquireByData(tscMetaCache, pTableMeta);
assert(initialSize > 0 && pTableMeta != NULL && dataBuf->pTableMeta != NULL); assert(initialSize > 0 && pTableMeta != NULL && dataBuf->pTableMeta != NULL);
*dataBlocks = dataBuf; *dataBlocks = dataBuf;
...@@ -784,15 +785,15 @@ static int32_t getRowExpandSize(STableMeta* pTableMeta) { ...@@ -784,15 +785,15 @@ static int32_t getRowExpandSize(STableMeta* pTableMeta) {
return result; return result;
} }
static void extractTableMeta(SSqlCmd* pCmd) { static void extractTableNameList(SSqlCmd* pCmd) {
pCmd->numOfTables = (int32_t) taosHashGetSize(pCmd->pTableBlockHashList); pCmd->numOfTables = (int32_t) taosHashGetSize(pCmd->pTableBlockHashList);
pCmd->pTableMetaList = calloc(pCmd->numOfTables, POINTER_BYTES); pCmd->pTableNameList = calloc(pCmd->numOfTables, POINTER_BYTES);
STableDataBlocks **p1 = taosHashIterate(pCmd->pTableBlockHashList, NULL); STableDataBlocks **p1 = taosHashIterate(pCmd->pTableBlockHashList, NULL);
int32_t i = 0; int32_t i = 0;
while(p1) { while(p1) {
STableDataBlocks* pBlocks = *p1; STableDataBlocks* pBlocks = *p1;
pCmd->pTableMetaList[i++] = taosCacheTransfer(tscMetaCache, (void**) &pBlocks->pTableMeta); pCmd->pTableNameList[i++] = strndup(pBlocks->tableName, TSDB_TABLE_FNAME_LEN);
p1 = taosHashIterate(pCmd->pTableBlockHashList, p1); p1 = taosHashIterate(pCmd->pTableBlockHashList, p1);
} }
...@@ -815,7 +816,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) { ...@@ -815,7 +816,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) {
STableDataBlocks* dataBuf = NULL; STableDataBlocks* dataBuf = NULL;
int32_t ret = tscGetDataBlockFromList(pVnodeDataBlockHashList, pOneTableBlock->vgId, TSDB_PAYLOAD_SIZE, int32_t ret = tscGetDataBlockFromList(pVnodeDataBlockHashList, pOneTableBlock->vgId, TSDB_PAYLOAD_SIZE,
INSERT_HEAD_SIZE, 0, pOneTableBlock->tableId, pOneTableBlock->pTableMeta, &dataBuf, pVnodeDataBlockList); INSERT_HEAD_SIZE, 0, pOneTableBlock->tableName, pOneTableBlock->pTableMeta, &dataBuf, pVnodeDataBlockList);
if (ret != TSDB_CODE_SUCCESS) { if (ret != TSDB_CODE_SUCCESS) {
tscError("%p failed to prepare the data block buffer for merging table data, code:%d", pSql, ret); tscError("%p failed to prepare the data block buffer for merging table data, code:%d", pSql, ret);
taosHashCleanup(pVnodeDataBlockHashList); taosHashCleanup(pVnodeDataBlockHashList);
...@@ -849,7 +850,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) { ...@@ -849,7 +850,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) {
tscSortRemoveDataBlockDupRows(pOneTableBlock); tscSortRemoveDataBlockDupRows(pOneTableBlock);
char* ekey = (char*)pBlocks->data + pOneTableBlock->rowSize*(pBlocks->numOfRows-1); char* ekey = (char*)pBlocks->data + pOneTableBlock->rowSize*(pBlocks->numOfRows-1);
tscDebug("%p tableId:%s, sid:%d rows:%d sversion:%d skey:%" PRId64 ", ekey:%" PRId64, pSql, pOneTableBlock->tableId, tscDebug("%p name:%s, sid:%d rows:%d sversion:%d skey:%" PRId64 ", ekey:%" PRId64, pSql, pOneTableBlock->tableName,
pBlocks->tid, pBlocks->numOfRows, pBlocks->sversion, GET_INT64_VAL(pBlocks->data), GET_INT64_VAL(ekey)); pBlocks->tid, pBlocks->numOfRows, pBlocks->sversion, GET_INT64_VAL(pBlocks->data), GET_INT64_VAL(ekey));
int32_t len = pBlocks->numOfRows * (pOneTableBlock->rowSize + expandSize) + sizeof(STColumn) * tscGetNumOfColumns(pOneTableBlock->pTableMeta); int32_t len = pBlocks->numOfRows * (pOneTableBlock->rowSize + expandSize) + sizeof(STColumn) * tscGetNumOfColumns(pOneTableBlock->pTableMeta);
...@@ -879,7 +880,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) { ...@@ -879,7 +880,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql) {
pOneTableBlock = *p; pOneTableBlock = *p;
} }
extractTableMeta(pCmd); extractTableNameList(pCmd);
// free the table data blocks; // free the table data blocks;
pCmd->pDataBlocks = pVnodeDataBlockList; pCmd->pDataBlocks = pVnodeDataBlockList;
...@@ -900,6 +901,7 @@ void tscCloseTscObj(void *param) { ...@@ -900,6 +901,7 @@ void tscCloseTscObj(void *param) {
rpcClose(pObj->pDnodeConn); rpcClose(pObj->pDnodeConn);
pObj->pDnodeConn = NULL; pObj->pDnodeConn = NULL;
} }
tfree(pObj->tscCorMgmtEpSet); tfree(pObj->tscCorMgmtEpSet);
pthread_mutex_destroy(&pObj->mutex); pthread_mutex_destroy(&pObj->mutex);
...@@ -1528,6 +1530,7 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) { ...@@ -1528,6 +1530,7 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
} }
dest->tbnameCond.uid = src->tbnameCond.uid; dest->tbnameCond.uid = src->tbnameCond.uid;
dest->tbnameCond.len = src->tbnameCond.len;
memcpy(&dest->joinInfo, &src->joinInfo, sizeof(SJoinInfo)); memcpy(&dest->joinInfo, &src->joinInfo, sizeof(SJoinInfo));
dest->relType = src->relType; dest->relType = src->relType;
...@@ -1823,14 +1826,12 @@ SArray* tscVgroupTableInfoClone(SArray* pVgroupTables) { ...@@ -1823,14 +1826,12 @@ SArray* tscVgroupTableInfoClone(SArray* pVgroupTables) {
return pa; return pa;
} }
void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, const char* address, bool removeFromCache) { void clearAllTableMetaInfo(SQueryInfo* pQueryInfo) {
tscDebug("%p unref %d tables in the tableMeta cache", address, pQueryInfo->numOfTables);
for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i);
tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables); tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables);
tscClearTableMetaInfo(pTableMetaInfo, removeFromCache); tscClearTableMetaInfo(pTableMetaInfo);
free(pTableMetaInfo); free(pTableMetaInfo);
} }
...@@ -1884,14 +1885,12 @@ STableMetaInfo* tscAddEmptyMetaInfo(SQueryInfo* pQueryInfo) { ...@@ -1884,14 +1885,12 @@ STableMetaInfo* tscAddEmptyMetaInfo(SQueryInfo* pQueryInfo) {
return tscAddTableMetaInfo(pQueryInfo, NULL, NULL, NULL, NULL, NULL); return tscAddTableMetaInfo(pQueryInfo, NULL, NULL, NULL, NULL, NULL);
} }
void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo, bool removeFromCache) { void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo) {
if (pTableMetaInfo == NULL) { if (pTableMetaInfo == NULL) {
return; return;
} }
if (pTableMetaInfo->pTableMeta != NULL) { tfree(pTableMetaInfo->pTableMeta);
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), removeFromCache);
}
pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList); pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList);
tscColumnListDestroy(pTableMetaInfo->tagColList); tscColumnListDestroy(pTableMetaInfo->tagColList);
...@@ -2015,7 +2014,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void ...@@ -2015,7 +2014,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
pNew->pTscObj = pSql->pTscObj; pNew->pTscObj = pSql->pTscObj;
pNew->signature = pNew; pNew->signature = pNew;
pNew->sqlstr = NULL; pNew->sqlstr = strdup(pSql->sqlstr);
SSqlCmd* pnCmd = &pNew->cmd; SSqlCmd* pnCmd = &pNew->cmd;
memcpy(pnCmd, pCmd, sizeof(SSqlCmd)); memcpy(pnCmd, pCmd, sizeof(SSqlCmd));
...@@ -2031,7 +2030,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void ...@@ -2031,7 +2030,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
pnCmd->numOfTables = 0; pnCmd->numOfTables = 0;
pnCmd->parseFinished = 1; pnCmd->parseFinished = 1;
pnCmd->pTableMetaList = NULL; pnCmd->pTableNameList = NULL;
pnCmd->pTableBlockHashList = NULL; pnCmd->pTableBlockHashList = NULL;
if (tscAddSubqueryInfo(pnCmd) != TSDB_CODE_SUCCESS) { if (tscAddSubqueryInfo(pnCmd) != TSDB_CODE_SUCCESS) {
...@@ -2113,8 +2112,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void ...@@ -2113,8 +2112,8 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
char* name = pTableMetaInfo->name; char* name = pTableMetaInfo->name;
STableMetaInfo* pFinalInfo = NULL; STableMetaInfo* pFinalInfo = NULL;
if (pPrevSql == NULL) { // get by name may failed due to the cache cleanup if (pPrevSql == NULL) {
STableMeta* pTableMeta = taosCacheAcquireByData(tscMetaCache, pTableMetaInfo->pTableMeta); STableMeta* pTableMeta = tscTableMetaClone(pTableMetaInfo->pTableMeta);
assert(pTableMeta != NULL); assert(pTableMeta != NULL);
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pTableMeta, pTableMetaInfo->vgroupList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pTableMeta, pTableMetaInfo->vgroupList,
...@@ -2122,15 +2121,15 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void ...@@ -2122,15 +2121,15 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void
} else { // transfer the ownership of pTableMeta to the newly create sql object. } else { // transfer the ownership of pTableMeta to the newly create sql object.
STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex, 0); STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex, 0);
STableMeta* pPrevTableMeta = taosCacheTransfer(tscMetaCache, (void**)&pPrevInfo->pTableMeta); STableMeta* pPrevTableMeta = tscTableMetaClone(pPrevInfo->pTableMeta);
SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList; SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList;
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pPrevTableMeta, pVgroupsInfo, pTableMetaInfo->tagColList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pPrevTableMeta, pVgroupsInfo, pTableMetaInfo->tagColList,
pTableMetaInfo->pVgroupTables); pTableMetaInfo->pVgroupTables);
} }
// this case cannot be happened
if (pFinalInfo->pTableMeta == NULL) { if (pFinalInfo->pTableMeta == NULL) {
tscError("%p new subquery failed since no tableMeta in cache, name:%s", pSql, name); tscError("%p new subquery failed since no tableMeta, name:%s", pSql, name);
if (pPrevSql != NULL) { // pass the previous error to client if (pPrevSql != NULL) { // pass the previous error to client
assert(pPrevSql->res.code != TSDB_CODE_SUCCESS); assert(pPrevSql->res.code != TSDB_CODE_SUCCESS);
...@@ -2210,7 +2209,9 @@ void tscDoQuery(SSqlObj* pSql) { ...@@ -2210,7 +2209,9 @@ void tscDoQuery(SSqlObj* pSql) {
tscProcessSql(pSql); tscProcessSql(pSql);
} else { // secondary stage join query. } else { // secondary stage join query.
if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query
tscLockByThread(&pSql->squeryLock);
tscHandleMasterSTableQuery(pSql); tscHandleMasterSTableQuery(pSql);
tscUnlockByThread(&pSql->squeryLock);
} else { } else {
tscProcessSql(pSql); tscProcessSql(pSql);
} }
...@@ -2219,7 +2220,9 @@ void tscDoQuery(SSqlObj* pSql) { ...@@ -2219,7 +2220,9 @@ void tscDoQuery(SSqlObj* pSql) {
return; return;
} else if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query } else if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query
tscLockByThread(&pSql->squeryLock);
tscHandleMasterSTableQuery(pSql); tscHandleMasterSTableQuery(pSql);
tscUnlockByThread(&pSql->squeryLock);
return; return;
} }
...@@ -2557,6 +2560,7 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) { ...@@ -2557,6 +2560,7 @@ void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) {
for(int32_t j = 0; j < pVgroupInfo->numOfEps; ++j) { for(int32_t j = 0; j < pVgroupInfo->numOfEps; ++j) {
tfree(pVgroupInfo->epAddr[j].fqdn); tfree(pVgroupInfo->epAddr[j].fqdn);
} }
for(int32_t j = pVgroupInfo->numOfEps; j < TSDB_MAX_REPLICA; j++) { for(int32_t j = pVgroupInfo->numOfEps; j < TSDB_MAX_REPLICA; j++) {
assert( pVgroupInfo->epAddr[j].fqdn == NULL ); assert( pVgroupInfo->epAddr[j].fqdn == NULL );
} }
...@@ -2610,3 +2614,87 @@ int32_t copyTagData(STagData* dst, const STagData* src) { ...@@ -2610,3 +2614,87 @@ int32_t copyTagData(STagData* dst, const STagData* src) {
return 0; return 0;
} }
STableMeta* createSuperTableMeta(STableMetaMsg* pChild) {
assert(pChild != NULL);
int32_t total = pChild->numOfColumns + pChild->numOfTags;
STableMeta* pTableMeta = calloc(1, sizeof(STableMeta) + sizeof(SSchema) * total);
pTableMeta->tableType = TSDB_SUPER_TABLE;
pTableMeta->tableInfo.numOfTags = pChild->numOfTags;
pTableMeta->tableInfo.numOfColumns = pChild->numOfColumns;
pTableMeta->tableInfo.precision = pChild->precision;
pTableMeta->id.tid = 0;
pTableMeta->id.uid = pChild->suid;
pTableMeta->tversion = pChild->tversion;
pTableMeta->sversion = pChild->sversion;
memcpy(pTableMeta->schema, pChild->schema, sizeof(SSchema) * total);
int32_t num = pTableMeta->tableInfo.numOfColumns;
for(int32_t i = 0; i < num; ++i) {
pTableMeta->tableInfo.rowSize += pTableMeta->schema[i].bytes;
}
return pTableMeta;
}
uint32_t tscGetTableMetaSize(STableMeta* pTableMeta) {
assert(pTableMeta != NULL);
int32_t totalCols = pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags;
return sizeof(STableMeta) + totalCols * sizeof(SSchema);
}
CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta) {
assert(pTableMeta != NULL);
CChildTableMeta* cMeta = calloc(1, sizeof(CChildTableMeta));
cMeta->tableType = TSDB_CHILD_TABLE;
cMeta->vgId = pTableMeta->vgId;
cMeta->id = pTableMeta->id;
tstrncpy(cMeta->sTableName, pTableMeta->sTableName, TSDB_TABLE_FNAME_LEN);
return cMeta;
}
int32_t tscCreateTableMetaFromCChildMeta(STableMeta* pChild, const char* name) {
assert(pChild != NULL);
uint32_t size = tscGetTableMetaMaxSize();
STableMeta* p = calloc(1, size);
taosHashGetClone(tscTableMetaInfo, pChild->sTableName, strnlen(pChild->sTableName, TSDB_TABLE_FNAME_LEN), NULL, p, -1);
if (p->id.uid > 0) { // tableMeta exists, build child table meta and return
pChild->sversion = p->sversion;
pChild->tversion = p->tversion;
memcpy(&pChild->tableInfo, &p->tableInfo, sizeof(STableInfo));
int32_t total = pChild->tableInfo.numOfColumns + pChild->tableInfo.numOfTags;
memcpy(pChild->schema, p->schema, sizeof(SSchema) *total);
tfree(p);
return TSDB_CODE_SUCCESS;
} else { // super table has been removed, current tableMeta is also expired. remove it here
taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
tfree(p);
return -1;
}
}
uint32_t tscGetTableMetaMaxSize() {
return sizeof(STableMeta) + TSDB_MAX_COLUMNS * sizeof(SSchema);
}
STableMeta* tscTableMetaClone(STableMeta* pTableMeta) {
assert(pTableMeta != NULL);
uint32_t size = tscGetTableMetaSize(pTableMeta);
STableMeta* p = calloc(1, size);
memcpy(p, pTableMeta, size);
return p;
}
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
FIND_PATH(HEADER_GTEST_INCLUDE_DIR gtest.h /usr/include/gtest /usr/local/include/gtest) FIND_PATH(HEADER_GTEST_INCLUDE_DIR gtest.h /usr/include/gtest /usr/local/include/gtest)
......
...@@ -162,6 +162,10 @@ TEST(testCase, parse_time) { ...@@ -162,6 +162,10 @@ TEST(testCase, parse_time) {
taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0); taosParseTime(t13, &time, strlen(t13), TSDB_TIME_PRECISION_MILLI, 0);
EXPECT_EQ(time, -28800 * MILLISECOND_PER_SECOND); EXPECT_EQ(time, -28800 * MILLISECOND_PER_SECOND);
char* t = "2021-01-08T02:11:40.000+00:00";
taosParseTime(t, &time, strlen(t), TSDB_TIME_PRECISION_MILLI, 0);
printf("%ld\n", time);
} }
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
INCLUDE_DIRECTORIES(inc) INCLUDE_DIRECTORIES(inc)
......
...@@ -36,6 +36,7 @@ extern int8_t tsEnableVnodeBak; ...@@ -36,6 +36,7 @@ extern int8_t tsEnableVnodeBak;
extern int8_t tsEnableTelemetryReporting; extern int8_t tsEnableTelemetryReporting;
extern char tsEmail[]; extern char tsEmail[];
extern char tsArbitrator[]; extern char tsArbitrator[];
extern int8_t tsArbOnline;
// common // common
extern int tsRpcTimer; extern int tsRpcTimer;
...@@ -62,7 +63,6 @@ extern int32_t tsRetrieveBlockingModel;// retrieve threads will be blocked ...@@ -62,7 +63,6 @@ extern int32_t tsRetrieveBlockingModel;// retrieve threads will be blocked
extern int8_t tsKeepOriginalColumnName; extern int8_t tsKeepOriginalColumnName;
// client // client
extern int32_t tsTableMetaKeepTimer;
extern int32_t tsMaxSQLStringLen; extern int32_t tsMaxSQLStringLen;
extern int8_t tsTscEnableRecordSql; extern int8_t tsTscEnableRecordSql;
extern int32_t tsMaxNumOfOrderedResults; extern int32_t tsMaxNumOfOrderedResults;
...@@ -190,7 +190,7 @@ extern int32_t monDebugFlag; ...@@ -190,7 +190,7 @@ extern int32_t monDebugFlag;
extern int32_t uDebugFlag; extern int32_t uDebugFlag;
extern int32_t rpcDebugFlag; extern int32_t rpcDebugFlag;
extern int32_t odbcDebugFlag; extern int32_t odbcDebugFlag;
extern int32_t qDebugFlag; extern uint32_t qDebugFlag;
extern int32_t wDebugFlag; extern int32_t wDebugFlag;
extern int32_t cqDebugFlag; extern int32_t cqDebugFlag;
extern int32_t debugFlag; extern int32_t debugFlag;
......
...@@ -28,7 +28,8 @@ typedef struct tVariant { ...@@ -28,7 +28,8 @@ typedef struct tVariant {
uint32_t nType; uint32_t nType;
int32_t nLen; // only used for string, for number, it is useless int32_t nLen; // only used for string, for number, it is useless
union { union {
int64_t i64Key; int64_t i64;
uint64_t u64;
double dKey; double dKey;
char * pz; char * pz;
wchar_t *wpz; wchar_t *wpz;
...@@ -36,9 +37,9 @@ typedef struct tVariant { ...@@ -36,9 +37,9 @@ typedef struct tVariant {
}; };
} tVariant; } tVariant;
void tVariantCreate(tVariant *pVar, SStrToken *token); bool tVariantIsValid(tVariant *pVar);
void tVariantCreateFromString(tVariant *pVar, char *pz, uint32_t len, uint32_t type); void tVariantCreate(tVariant *pVar, SStrToken *token);
void tVariantCreateFromBinary(tVariant *pVar, const char *pz, size_t len, uint32_t type); void tVariantCreateFromBinary(tVariant *pVar, const char *pz, size_t len, uint32_t type);
......
...@@ -41,6 +41,7 @@ int32_t tsStatusInterval = 1; // second ...@@ -41,6 +41,7 @@ int32_t tsStatusInterval = 1; // second
int32_t tsNumOfMnodes = 3; int32_t tsNumOfMnodes = 3;
int8_t tsEnableVnodeBak = 1; int8_t tsEnableVnodeBak = 1;
int8_t tsEnableTelemetryReporting = 1; int8_t tsEnableTelemetryReporting = 1;
int8_t tsArbOnline = 0;
char tsEmail[TSDB_FQDN_LEN] = {0}; char tsEmail[TSDB_FQDN_LEN] = {0};
// common // common
...@@ -71,7 +72,6 @@ char tsTempDir[TSDB_FILENAME_LEN] = "/tmp/"; ...@@ -71,7 +72,6 @@ char tsTempDir[TSDB_FILENAME_LEN] = "/tmp/";
int32_t tsCompressMsgSize = -1; int32_t tsCompressMsgSize = -1;
// client // client
int32_t tsTableMetaKeepTimer = 7200; // second
int32_t tsMaxSQLStringLen = TSDB_MAX_SQL_LEN; int32_t tsMaxSQLStringLen = TSDB_MAX_SQL_LEN;
int8_t tsTscEnableRecordSql = 0; int8_t tsTscEnableRecordSql = 0;
...@@ -218,7 +218,7 @@ int32_t odbcDebugFlag = 131; ...@@ -218,7 +218,7 @@ int32_t odbcDebugFlag = 131;
int32_t httpDebugFlag = 131; int32_t httpDebugFlag = 131;
int32_t mqttDebugFlag = 131; int32_t mqttDebugFlag = 131;
int32_t monDebugFlag = 131; int32_t monDebugFlag = 131;
int32_t qDebugFlag = 131; uint32_t qDebugFlag = 131;
int32_t rpcDebugFlag = 131; int32_t rpcDebugFlag = 131;
int32_t uDebugFlag = 131; int32_t uDebugFlag = 131;
int32_t debugFlag = 0; int32_t debugFlag = 0;
...@@ -595,16 +595,6 @@ static void doInitGlobalConfig(void) { ...@@ -595,16 +595,6 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_SECOND; cfg.unitType = TAOS_CFG_UTYPE_SECOND;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
cfg.option = "tableMetaKeepTimer";
cfg.ptr = &tsTableMetaKeepTimer;
cfg.valType = TAOS_CFG_VTYPE_INT32;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 1;
cfg.maxValue = 8640000;
cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_SECOND;
taosInitConfigOption(cfg);
cfg.option = "minSlidingTime"; cfg.option = "minSlidingTime";
cfg.ptr = &tsMinSlidingTime; cfg.ptr = &tsMinSlidingTime;
cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.valType = TAOS_CFG_VTYPE_INT32;
...@@ -810,8 +800,8 @@ static void doInitGlobalConfig(void) { ...@@ -810,8 +800,8 @@ static void doInitGlobalConfig(void) {
cfg.ptr = &tsQuorum; cfg.ptr = &tsQuorum;
cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.valType = TAOS_CFG_VTYPE_INT32;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW;
cfg.minValue = TSDB_MIN_DB_REPLICA_OPTION; cfg.minValue = TSDB_MIN_DB_QUORUM_OPTION;
cfg.maxValue = TSDB_MAX_DB_REPLICA_OPTION; cfg.maxValue = TSDB_MAX_DB_QUORUM_OPTION;
cfg.ptrLength = 0; cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
......
...@@ -56,6 +56,12 @@ ...@@ -56,6 +56,12 @@
<scope>test</scope> <scope>test</scope>
</dependency> </dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.47</version>
</dependency>
<!-- for restful --> <!-- for restful -->
<dependency> <dependency>
<groupId>org.apache.httpcomponents</groupId> <groupId>org.apache.httpcomponents</groupId>
...@@ -73,7 +79,14 @@ ...@@ -73,7 +79,14 @@
<version>1.2.58</version> <version>1.2.58</version>
</dependency> </dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.49</version>
</dependency>
</dependencies> </dependencies>
<build> <build>
<plugins> <plugins>
<plugin> <plugin>
......
...@@ -33,7 +33,7 @@ public class RestfulDriver extends AbstractTaosDriver { ...@@ -33,7 +33,7 @@ public class RestfulDriver extends AbstractTaosDriver {
return null; return null;
Properties props = parseURL(url, info); Properties props = parseURL(url, info);
String host = props.getProperty(TSDBDriver.PROPERTY_KEY_HOST, "localhost"); String host = props.getProperty(TSDBDriver.PROPERTY_KEY_HOST);
String port = props.getProperty(TSDBDriver.PROPERTY_KEY_PORT, "6041"); String port = props.getProperty(TSDBDriver.PROPERTY_KEY_PORT, "6041");
String database = props.containsKey(TSDBDriver.PROPERTY_KEY_DBNAME) ? props.getProperty(TSDBDriver.PROPERTY_KEY_DBNAME) : null; String database = props.containsKey(TSDBDriver.PROPERTY_KEY_DBNAME) ? props.getProperty(TSDBDriver.PROPERTY_KEY_DBNAME) : null;
......
...@@ -2,13 +2,15 @@ package com.taosdata.jdbc.rs; ...@@ -2,13 +2,15 @@ package com.taosdata.jdbc.rs;
import java.sql.ResultSetMetaData; import java.sql.ResultSetMetaData;
import java.sql.SQLException; import java.sql.SQLException;
import java.util.List; import java.util.ArrayList;
public class RestfulResultSetMetaData implements ResultSetMetaData { public class RestfulResultSetMetaData implements ResultSetMetaData {
private List<String> fields; private final String database;
private ArrayList<RestfulResultSet.Field> fields;
public RestfulResultSetMetaData(List<String> fields) { public RestfulResultSetMetaData(String database, ArrayList<RestfulResultSet.Field> fields) {
this.database = database;
this.fields = fields; this.fields = fields;
} }
...@@ -24,6 +26,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -24,6 +26,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public boolean isCaseSensitive(int column) throws SQLException { public boolean isCaseSensitive(int column) throws SQLException {
//TODO
return false; return false;
} }
...@@ -39,7 +42,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -39,7 +42,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public int isNullable(int column) throws SQLException { public int isNullable(int column) throws SQLException {
return 0; return ResultSetMetaData.columnNullable;
} }
@Override @Override
...@@ -54,7 +57,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -54,7 +57,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public String getColumnLabel(int column) throws SQLException { public String getColumnLabel(int column) throws SQLException {
return fields.get(column - 1); return fields.get(column - 1).name;
} }
@Override @Override
...@@ -64,7 +67,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -64,7 +67,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public String getSchemaName(int column) throws SQLException { public String getSchemaName(int column) throws SQLException {
return null; return this.database;
} }
@Override @Override
...@@ -84,7 +87,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData { ...@@ -84,7 +87,7 @@ public class RestfulResultSetMetaData implements ResultSetMetaData {
@Override @Override
public String getCatalogName(int column) throws SQLException { public String getCatalogName(int column) throws SQLException {
return null; return this.database;
} }
@Override @Override
......
...@@ -19,7 +19,7 @@ public class AppMemoryLeakTest { ...@@ -19,7 +19,7 @@ public class AppMemoryLeakTest {
} }
} }
@Test @Test(expected = Exception.class)
public void testCreateTooManyStatement() throws ClassNotFoundException, SQLException { public void testCreateTooManyStatement() throws ClassNotFoundException, SQLException {
Class.forName("com.taosdata.jdbc.TSDBDriver"); Class.forName("com.taosdata.jdbc.TSDBDriver");
int stmtCnt = 0; int stmtCnt = 0;
...@@ -30,15 +30,4 @@ public class AppMemoryLeakTest { ...@@ -30,15 +30,4 @@ public class AppMemoryLeakTest {
} }
} }
public static void main(String[] args) throws ClassNotFoundException, SQLException {
Class.forName("com.taosdata.jdbc.TSDBDriver");
int stmtCnt = 0;
Connection conn = DriverManager.getConnection("jdbc:TAOS://localhost:6030/?user=root&password=taosdata");
while (true) {
Statement stmt = conn.createStatement();
System.out.println(++stmtCnt + " : " + stmt);
}
}
} }
package com.taosdata.jdbc.rs; package com.taosdata.jdbc.rs;
import org.junit.*; import org.junit.*;
import org.junit.runners.MethodSorters; import org.junit.runners.MethodSorters;
......
此差异已折叠。
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) CMAKE_MINIMUM_REQUIRED(VERSION 3.5)
PROJECT(TDengine) PROJECT(TDengine)
IF (TD_LINUX_64) IF (TD_LINUX_64)
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册