未验证 提交 6fec4b9f 编写于 作者: G Ganlin Zhao 提交者: GitHub

Merge branch 'develop' into enhance/TD-10532

......@@ -177,6 +177,7 @@ taos> select avg(current), max(voltage), min(phase) from test.d10 interval(10s);
```
**Note:** taosdemo 命令本身带有很多选项,配置表的数目、记录条数等等,请执行 `taosdemo --help` 详细列出。您可以设置不同参数进行体验。
taosdemo 详细使用方法请参照 [如何使用taosdemo对TDengine进行性能测试?](https://www.taosdata.com/cn/documentation/getting-started/taosdemo )
## 客户端和报警模块
......
# TDengine数据建模
# TDengine 数据建模
TDengine采用关系型数据模型,需要建库、建表。因此对于一个具体的应用场景,需要考虑库、超级表和普通表的设计。本节不讨论细致的语法规则,只介绍概念。
TDengine 采用关系型数据模型,需要建库、建表。因此对于一个具体的应用场景,需要考虑库、超级表和普通表的设计。本节不讨论细致的语法规则,只介绍概念。
关于数据建模请参考[视频教程](https://www.taosdata.com/blog/2020/11/11/1945.html)
## <a class="anchor" id="create-db"></a>创建库
## <a class="anchor" id="create-db"></a> 创建库
不同类型的数据采集点往往具有不同的数据特征,包括数据采集频率的高低,数据保留时间的长短,副本的数目,数据块的大小,是否允许更新数据等等。为了在各种场景下TDengine都能最大效率的工作,TDengine建议将不同数据特征的表创建在不同的库里,因为每个库可以配置不同的存储策略。创建一个库时,除SQL标准的选项外,应用还可以指定保留时长、副本数、内存块个数、时间精度、文件块里最大最小记录条数、是否压缩、一个数据文件覆盖的天数等多种参数。比如:
不同类型的数据采集点往往具有不同的数据特征,包括数据采集频率的高低,数据保留时间的长短,副本的数目,数据块的大小,是否允许更新数据等等。为了在各种场景下 TDengine 都能最大效率的工作,TDengine 建议将不同数据特征的表创建在不同的库里,因为每个库可以配置不同的存储策略。创建一个库时,除SQL标准的选项外,应用还可以指定保留时长、副本数、内存块个数、时间精度、文件块里最大最小记录条数、是否压缩、一个数据文件覆盖的天数等多种参数。比如:
```mysql
CREATE DATABASE power KEEP 365 DAYS 10 BLOCKS 6 UPDATE 1;
```
上述语句将创建一个名为power的库,这个库的数据将保留365天(超过365天将被自动删除),每10天一个数据文件,内存块数为6,允许更新数据。详细的语法及参数请见 [TAOS SQL 的数据管理](https://www.taosdata.com/cn/documentation/taos-sql#management) 章节。
上述语句将创建一个名为 power 的库,这个库的数据将保留 365 天(超过 365 天将被自动删除),每 10 天一个数据文件,内存块数为 6,允许更新数据。详细的语法及参数请见 [TAOS SQL 的数据管理](https://www.taosdata.com/cn/documentation/taos-sql#management) 章节。
创建库之后,需要使用SQL命令USE将当前库切换过来,例如:
创建库之后,需要使用 SQL 命令 USE 将当前库切换过来,例如:
```mysql
USE power;
```
将当前连接里操作的库换为power,否则对具体表操作前,需要使用“库名.表名”来指定库的名字。
将当前连接里操作的库换为 power,否则对具体表操作前,需要使用“库名.表名”来指定库的名字。
**注意:**
- 任何一张表或超级表是属于一个库的,在创建表之前,必须先创建库。
- 处于两个不同库的表是不能进行JOIN操作的。
- 处于两个不同库的表是不能进行 JOIN 操作的。
- 创建并插入记录、查询历史记录的时候,均需要指定时间戳。
## <a class="anchor" id="create-stable"></a>创建超级表
## <a class="anchor" id="create-stable"></a> 创建超级表
一个物联网系统,往往存在多种类型的设备,比如对于电网,存在智能电表、变压器、母线、开关等等。为便于多表之间的聚合,使用TDengine, 需要对每个类型的数据采集点创建一个超级表。以[表1](https://www.taosdata.com/cn/documentation/architecture#model_table1)中的智能电表为例,可以使用如下的SQL命令创建超级表:
一个物联网系统,往往存在多种类型的设备,比如对于电网,存在智能电表、变压器、母线、开关等等。为便于多表之间的聚合,使用 TDengine, 需要对每个类型的数据采集点创建一个超级表。以[表1](https://www.taosdata.com/cn/documentation/architecture#model_table1) 中的智能电表为例,可以使用如下的 SQL 命令创建超级表:
```mysql
CREATE STABLE meters (ts timestamp, current float, voltage int, phase float) TAGS (location binary(64), groupId int);
......@@ -39,25 +39,25 @@ CREATE STABLE meters (ts timestamp, current float, voltage int, phase float) TAG
**注意:**这一指令中的 STABLE 关键字,在 2.0.15 之前的版本中需写作 TABLE 。
与创建普通表一样,创建表时,需要提供表名(示例中为meters),表结构Schema,即数据列的定义。第一列必须为时间戳(示例中为ts),其他列为采集的物理量(示例中为current, voltage, phase),数据类型可以为整型、浮点型、字符串等。除此之外,还需要提供标签的schema (示例中为location, groupId),标签的数据类型可以为整型、浮点型、字符串等。采集点的静态属性往往可以作为标签,比如采集点的地理位置、设备型号、设备组ID、管理员ID等等。标签的schema可以事后增加、删除、修改。具体定义以及细节请见 [TAOS SQL 的超级表管理](https://www.taosdata.com/cn/documentation/taos-sql#super-table) 章节。
与创建普通表一样,创建表时,需要提供表名(示例中为 meters),表结构 Schema,即数据列的定义。第一列必须为时间戳(示例中为 ts),其他列为采集的物理量(示例中为 current, voltage, phase),数据类型可以为整型、浮点型、字符串等。除此之外,还需要提供标签的 schema (示例中为 location, groupId),标签的数据类型可以为整型、浮点型、字符串等。采集点的静态属性往往可以作为标签,比如采集点的地理位置、设备型号、设备组 ID、管理员 ID 等等。标签的 schema 可以事后增加、删除、修改。具体定义以及细节请见 [TAOS SQL 的超级表管理](https://www.taosdata.com/cn/documentation/taos-sql#super-table) 章节。
每一种类型的数据采集点需要建立一个超级表,因此一个物联网系统,往往会有多个超级表。对于电网,我们就需要对智能电表、变压器、母线、开关等都建立一个超级表。在物联网中,一个设备就可能有多个数据采集点(比如一台风力发电的风机,有的采集点采集电流、电压等电参数,有的采集点采集温度、湿度、风向等环境参数),这个时候,对这一类型的设备,需要建立多张超级表。一张超级表里包含的采集物理量必须是同时采集的(时间戳是一致的)。
一张超级表最多容许 1024 列,如果一个采集点采集的物理量个数超过 1024,需要建多张超级表来处理。一个系统可以有多个 DB,一个 DB 里可以有一到多个超级表。(从 2.1.7.0 版本开始,列数限制由 1024 列放宽到了 4096 列。)
## <a class="anchor" id="create-table"></a>创建表
## <a class="anchor" id="create-table"></a> 创建表
TDengine对每个数据采集点需要独立建表。与标准的关系型数据库一样,一张表有表名,Schema,但除此之外,还可以带有一到多个标签。创建时,需要使用超级表做模板,同时指定标签的具体值。以[表1](https://www.taosdata.com/cn/documentation/architecture#model_table1)中的智能电表为例,可以使用如下的SQL命令建表:
TDengine 对每个数据采集点需要独立建表。与标准的关系型数据库一样,一张表有表名,Schema,但除此之外,还可以带有一到多个标签。创建时,需要使用超级表做模板,同时指定标签的具体值。以[表1](https://www.taosdata.com/cn/documentation/architecture#model_table1)中的智能电表为例,可以使用如下的SQL命令建表:
```mysql
CREATE TABLE d1001 USING meters TAGS ("Beijing.Chaoyang", 2);
```
其中d1001是表名,meters是超级表的表名,后面紧跟标签Location的具体标签值”Beijing.Chaoyang",标签groupId的具体标签值2。虽然在创建表时,需要指定标签值,但可以事后修改。详细细则请见 [TAOS SQL 的表管理](https://www.taosdata.com/cn/documentation/taos-sql#table) 章节。
其中 d1001 是表名,meters 是超级表的表名,后面紧跟标签 Location 的具体标签值 ”Beijing.Chaoyang",标签 groupId 的具体标签值 2。虽然在创建表时,需要指定标签值,但可以事后修改。详细细则请见 [TAOS SQL 的表管理](https://www.taosdata.com/cn/documentation/taos-sql#table) 章节。
**注意:**目前 TDengine 没有从技术层面限制使用一个 database (dbA)的超级表作为模板建立另一个 database (dbB)的子表,后续会禁止这种用法,不建议使用这种方法建表。
**注意:**目前 TDengine 没有从技术层面限制使用一个 database (dbA)的超级表作为模板建立另一个 database (dbB)的子表,后续会禁止这种用法,不建议使用这种方法建表。
TDengine建议将数据采集点的全局唯一ID作为表名(比如设备序列号)。但对于有的场景,并没有唯一的ID,可以将多个ID组合成一个唯一的ID。不建议将具有唯一性的ID作为标签值。
TDengine 建议将数据采集点的全局唯一 ID 作为表名(比如设备序列号)。但对于有的场景,并没有唯一的 ID,可以将多个 ID 组合成一个唯一的 ID。不建议将具有唯一性的 ID 作为标签值。
**自动建表**:在某些特殊场景中,用户在写数据时并不确定某个数据采集点的表是否存在,此时可在写入数据时使用自动建表语法来创建不存在的表,若该表已存在则不会建立新表。比如:
......@@ -65,13 +65,13 @@ TDengine建议将数据采集点的全局唯一ID作为表名(比如设备序列
INSERT INTO d1001 USING meters TAGS ("Beijng.Chaoyang", 2) VALUES (now, 10.2, 219, 0.32);
```
上述SQL语句将记录 (now, 10.2, 219, 0.32) 插入表d1001。如果表d1001还未创建,则使用超级表meters做模板自动创建,同时打上标签值 `“Beijing.Chaoyang", 2`
上述 SQL 语句将记录(now, 10.2, 219, 0.32)插入表 d1001。如果表 d1001 还未创建,则使用超级表 meters 做模板自动创建,同时打上标签值 `“Beijing.Chaoyang", 2`
关于自动建表的详细语法请参见 [插入记录时自动建表](https://www.taosdata.com/cn/documentation/taos-sql#auto_create_table) 章节。
## 多列模型 vs 单列模型
TDengine支持多列模型,只要物理量是一个数据采集点同时采集的(时间戳一致),这些量就可以作为不同列放在一张超级表里。但还有一种极限的设计,单列模型,每个采集的物理量都单独建表,因此每种类型的物理量都单独建立一超级表。比如电流、电压、相位,就建三张超级表。
TDengine 支持多列模型,只要物理量是一个数据采集点同时采集的(时间戳一致),这些量就可以作为不同列放在一张超级表里。但还有一种极限的设计,单列模型,每个采集的物理量都单独建表,因此每种类型的物理量都单独建立一超级表。比如电流、电压、相位,就建三张超级表。
TDengine建议尽可能采用多列模型,因为插入效率以及存储效率更高。但对于有些场景,一个采集点的采集量的种类经常变化,这个时候,如果采用多列模型,就需要频繁修改超级表的结构定义,让应用变的复杂,这个时候,采用单列模型会显得更简单。
TDengine 建议尽可能采用多列模型,因为插入效率以及存储效率更高。但对于有些场景,一个采集点的采集量的种类经常变化,这个时候,如果采用多列模型,就需要频繁修改超级表的结构定义,让应用变的复杂,这个时候,采用单列模型会显得更简单。
......@@ -107,7 +107,7 @@ gcc -g -O0 -fPIC -shared add_one.c -o add_one.so
例如,如下语句可以把 add_one.so 创建为系统中可用的 UDF:
```sql
CREATE FUNCTION add_one AS "/home/taos/udf_example/add_one.so" OUTPUTTYPE INT bufsize 128;
CREATE FUNCTION add_one AS "/home/taos/udf_example/add_one.so" OUTPUTTYPE INT;
```
- 创建聚合函数:`CREATE AGGREGATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) bufsize B;`
......@@ -116,9 +116,9 @@ gcc -g -O0 -fPIC -shared add_one.c -o add_one.so
* typename(Z):此函数计算结果的数据类型,与上文中 udfNormalFunc 的 itype 参数不同,这里不是使用数字表示法,而是直接写类型名称即可;
* B:系统使用的中间临时缓冲区大小,单位是字节,最小 0,最大 512,通常可以设置为 128。
例如,如下语句可以把 add_one.so 创建为系统中可用的 UDF:
例如,如下语句可以把 abs_max.so 创建为系统中可用的 UDF:
```sql
CREATE FUNCTION abs_max AS "/home/taos/udf_example/abs_max.so" OUTPUTTYPE BIGINT bufsize 128;
CREATE AGGREGATE FUNCTION abs_max AS "/home/taos/udf_example/abs_max.so" OUTPUTTYPE BIGINT bufsize 128;
```
### 管理 UDF
......
......@@ -71,6 +71,14 @@ TDengine 缺省的时间戳是毫秒精度,但通过在 CREATE DATABASE 时传
1) KEEP是该数据库的数据保留多长天数,缺省是3650天(10年),数据库会自动删除超过时限的数据;<!-- REPLACE_OPEN_TO_ENTERPRISE__KEEP_PARAM_DESCRIPTION -->
2) UPDATE 标志数据库支持更新相同时间戳数据;(从 2.1.7.0 版本开始此参数支持设为 2,表示允许部分列更新,也即更新数据行时未被设置的列会保留原值。)(从 2.0.8.0 版本开始支持此参数。注意此参数不能通过 `ALTER DATABASE` 指令进行修改。)
1) UPDATE设为0时,表示不允许更新数据,后发送的相同时间戳的数据会被直接丢弃;
2) UPDATE设为1时,表示更新全部列数据,即如果更新一个数据行,其中某些列没有提供取值,那么这些列会被设为 NULL;
3) UPDATE设为2时,表示支持更新部分列数据,即如果更新一个数据行,其中某些列没有提供取值,那么这些列会保持原有数据行中的对应值;
4) 更多关于UPDATE参数的用法,请参考[FAQ](https://www.taosdata.com/cn/documentation/faq)。
3) 数据库名最大长度为33;
......
......@@ -181,6 +181,7 @@ taos> select avg(f1), max(f2), min(f3) from test.t10 interval(10s);
```
**Note**: you can run command `taosdemo` with many options, like number of tables, rows of records and so on. To know more about these options, you can execute `taosdemo --help` and then take a try using different options.
Please refer to [How to use taosdemo to test the performance of TDengine](https://www.taosdata.com/en/documentation/getting-started/taosdemo) for detail.
## Client and Alarm Module
......
......@@ -192,6 +192,11 @@ else
allocator_macro=""
fi
if [[ "$dbName" == "pro" ]]; then
sed -i "s/taos config/prodb config/g" ${top_dir}/src/util/src/tconfig.c
fi
# check support cpu type
if [[ "$cpuType" == "x64" ]] || [[ "$cpuType" == "aarch64" ]] || [[ "$cpuType" == "aarch32" ]] || [[ "$cpuType" == "mips64" ]] ; then
if [ "$verMode" != "cluster" ]; then
......
......@@ -1002,6 +1002,7 @@ function install_TDengine() {
echo
echo -e "\033[44;32;1mTDengine client is installed successfully!${NC}"
fi
touch ~/.taos_history
rm -rf $(tar -tf taos.tar.gz)
}
......
......@@ -154,9 +154,9 @@ function install_main_path() {
${csudo} mkdir -p ${install_main_dir}
${csudo} mkdir -p ${install_main_dir}/cfg
${csudo} mkdir -p ${install_main_dir}/bin
${csudo} mkdir -p ${install_main_dir}/connector
# ${csudo} mkdir -p ${install_main_dir}/connector
${csudo} mkdir -p ${install_main_dir}/driver
${csudo} mkdir -p ${install_main_dir}/examples
# ${csudo} mkdir -p ${install_main_dir}/examples
${csudo} mkdir -p ${install_main_dir}/include
${csudo} mkdir -p ${install_main_dir}/init.d
if [ "$verMode" == "cluster" ]; then
......@@ -779,10 +779,10 @@ function update_prodb() {
install_log
install_header
install_lib
if [ "$pagMode" != "lite" ]; then
install_connector
fi
install_examples
# if [ "$pagMode" != "lite" ]; then
# install_connector
# fi
# install_examples
if [ -z $1 ]; then
install_bin
install_service
......@@ -853,10 +853,10 @@ function install_prodb() {
install_header
install_lib
install_jemalloc
if [ "$pagMode" != "lite" ]; then
install_connector
fi
install_examples
# if [ "$pagMode" != "lite" ]; then
# install_connector
# fi
# install_examples
if [ -z $1 ]; then # install service and client
# For installing new
......
文件模式从 100644 更改为 100755
......@@ -24,7 +24,7 @@ build_dir="${compile_dir}/build"
code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
# package_name='linux'
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/ProDB-enterprise-server-${version}"
else
......@@ -45,19 +45,13 @@ nginx_dir="${code_dir}/../../enterprise/src/plugins/web"
mkdir -p ${install_dir}
mkdir -p ${install_dir}/inc && cp ${header_files} ${install_dir}/inc
mkdir -p ${install_dir}/cfg && cp ${cfg_dir}/taos.cfg ${install_dir}/cfg/taos.cfg
#mkdir -p ${install_dir}/bin && cp ${bin_files} ${install_dir}/bin && chmod a+x ${install_dir}/bin/* || :
mkdir -p ${install_dir}/bin
# bin
if [ "$pagMode" == "lite" ]; then
strip ${build_dir}/bin/taosd
strip ${build_dir}/bin/taos
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${build_dir}/bin/taosd ${install_dir}/bin/prodbs
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
else
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${build_dir}/bin/taosd ${install_dir}/bin/prodbs
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
cp ${build_dir}/bin/taosdemo ${install_dir}/bin/prodemo
cp ${build_dir}/bin/taosdump ${install_dir}/bin/prodump
cp ${build_dir}/bin/tarbitrator ${install_dir}/bin
......@@ -66,13 +60,16 @@ else
cp ${script_dir}/startPre.sh ${install_dir}/bin
cp ${script_dir}/taosd-dump-cfg.gdb ${install_dir}/bin
fi
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${build_dir}/bin/taosd ${install_dir}/bin/prodbs
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
chmod a+x ${install_dir}/bin/* || :
# cluster
if [ "$verMode" == "cluster" ]; then
sed 's/verMode=edge/verMode=cluster/g' ${install_dir}/bin/remove_pro.sh >> remove_prodb_temp.sh
mv remove_prodb_temp.sh ${install_dir}/bin/remove_pro.sh
mkdir -p ${install_dir}/nginxd && cp -r ${nginx_dir}/* ${install_dir}/nginxd
cp ${nginx_dir}/png/taos.png ${install_dir}/nginxd/admin/images/taos.png
rm -rf ${install_dir}/nginxd/png
......@@ -85,14 +82,9 @@ if [ "$verMode" == "cluster" ]; then
sed -i -e 's/<th style="font-weight: normal">taosd<\/th>/<th style="font-weight: normal">prodbs<\/th>/g' ${install_dir}/nginxd/admin/monitor.html
sed -i -e "s/data:\['taosd', 'system'\],/data:\['prodbs', 'system'\],/g" ${install_dir}/nginxd/admin/monitor.html
sed -i -e "s/name: 'taosd',/name: 'prodbs',/g" ${install_dir}/nginxd/admin/monitor.html
sed -i "s/TDengine/ProDB/g" ${install_dir}/nginxd/admin/*.html
sed -i "s/TDengine/ProDB/g" ${install_dir}/nginxd/admin/js/*.js
sed -i '/dataDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i '/logDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i "s/TDengine/ProDB/g" ${install_dir}/cfg/taos.cfg
if [ "$cpuType" == "aarch64" ]; then
cp -f ${install_dir}/nginxd/sbin/arm/64bit/nginx ${install_dir}/nginxd/sbin/
elif [ "$cpuType" == "aarch32" ]; then
......@@ -101,6 +93,13 @@ if [ "$verMode" == "cluster" ]; then
rm -rf ${install_dir}/nginxd/sbin/arm
fi
sed -i '/dataDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i '/logDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i "s/TDengine/ProDB/g" ${install_dir}/cfg/taos.cfg
sed -i "s/support@taosdata.com/support@hanatech.com.cn/g" ${install_dir}/cfg/taos.cfg
sed -i "s/taos client/prodbc/g" ${install_dir}/cfg/taos.cfg
sed -i "s/taosd/prodbs/g" ${install_dir}/cfg/taos.cfg
cd ${install_dir}
tar -zcv -f prodb.tar.gz * --remove-files || :
exitcode=$?
......@@ -116,58 +115,62 @@ if [ "$verMode" == "cluster" ]; then
mv install_prodb_temp.sh ${install_dir}/install_pro.sh
fi
if [ "$pagMode" == "lite" ]; then
sed 's/pagMode=full/pagMode=lite/g' ${install_dir}/install.sh >> install_prodb_temp.sh
sed -e "s/pagMode=full/pagMode=lite/g" -e "s/taos_history/prodb_history/g" ${install_dir}/install.sh >> install_prodb_temp.sh
mv install_prodb_temp.sh ${install_dir}/install_pro.sh
fi
sed -i "/install_connector$/d" ${install_dir}/install_pro.sh
sed -i "/install_examples$/d" ${install_dir}/install_pro.sh
chmod a+x ${install_dir}/install_pro.sh
# Copy example code
mkdir -p ${install_dir}/examples
examples_dir="${top_dir}/tests/examples"
cp -r ${examples_dir}/c ${install_dir}/examples
sed -i '/passwd/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp -r ${examples_dir}/JDBC ${install_dir}/examples
cp -r ${examples_dir}/matlab ${install_dir}/examples
mv ${install_dir}/examples/matlab/TDengineDemo.m ${install_dir}/examples/matlab/ProDBDemo.m
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/matlab/ProDBDemo.m
cp -r ${examples_dir}/python ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/python/read_example.py
cp -r ${examples_dir}/R ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/R/command.txt
cp -r ${examples_dir}/go ${install_dir}/examples
mv ${install_dir}/examples/go/taosdemo.go ${install_dir}/examples/go/prodemo.go
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/go/prodemo.go
fi
#mkdir -p ${install_dir}/examples
#examples_dir="${top_dir}/tests/examples"
#cp -r ${examples_dir}/c ${install_dir}/examples
#sed -i '/passwd/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
#sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
#
#if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
# cp -r ${examples_dir}/JDBC ${install_dir}/examples
# cp -r ${examples_dir}/matlab ${install_dir}/examples
# mv ${install_dir}/examples/matlab/TDengineDemo.m ${install_dir}/examples/matlab/ProDBDemo.m
# sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/matlab/ProDBDemo.m
# cp -r ${examples_dir}/python ${install_dir}/examples
# sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/python/read_example.py
# cp -r ${examples_dir}/R ${install_dir}/examples
# sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/R/command.txt
# cp -r ${examples_dir}/go ${install_dir}/examples
# mv ${install_dir}/examples/go/taosdemo.go ${install_dir}/examples/go/prodemo.go
# sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/go/prodemo.go
#fi
# Copy driver
mkdir -p ${install_dir}/driver && cp ${lib_files} ${install_dir}/driver && echo "${versionComp}" > ${install_dir}/driver/vercomp.txt
# Copy connector
connector_dir="${code_dir}/connector"
mkdir -p ${install_dir}/connector
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
if [ -d "${connector_dir}/grafanaplugin/dist" ]; then
cp -r ${connector_dir}/grafanaplugin/dist ${install_dir}/connector/grafanaplugin
else
echo "WARNING: grafanaplugin bundled dir not found, please check if want to use it!"
fi
if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
cp -r ${connector_dir}/go ${install_dir}/connector
else
echo "WARNING: go connector not found, please check if want to use it!"
fi
cp -r ${connector_dir}/python ${install_dir}/connector/
mv ${install_dir}/connector/python/taos ${install_dir}/connector/python/prodb
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/cinterface.py
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/subscription.py
sed -i '/self._password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/connection.py
fi
#connector_dir="${code_dir}/connector"
#mkdir -p ${install_dir}/connector
#if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
# cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
# if [ -d "${connector_dir}/grafanaplugin/dist" ]; then
# cp -r ${connector_dir}/grafanaplugin/dist ${install_dir}/connector/grafanaplugin
# else
# echo "WARNING: grafanaplugin bundled dir not found, please check if want to use it!"
# fi
# if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
# cp -r ${connector_dir}/go ${install_dir}/connector
# else
# echo "WARNING: go connector not found, please check if want to use it!"
# fi
# cp -r ${connector_dir}/python ${install_dir}/connector/
# mv ${install_dir}/connector/python/taos ${install_dir}/connector/python/prodb
# sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/cinterface.py
# sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/subscription.py
# sed -i '/self._password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/connection.py
#fi
cd ${release_dir}
......
......@@ -180,7 +180,7 @@ bool tscQueryBlockInfo(SQueryInfo* pQueryInfo);
SExprInfo* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId,
SColumnIndex* pIndex, SSchema* pColSchema, int16_t colType, int16_t colId);
int32_t tscSetTableFullName(SName* pName, SStrToken* pzTableName, SSqlObj* pSql);
int32_t tscSetTableFullName(SName* pName, SStrToken* pzTableName, SSqlObj* pSql, bool dbIncluded);
void tscClearInterpInfo(SQueryInfo* pQueryInfo);
bool tscIsInsertData(char* sqlstr);
......@@ -251,7 +251,7 @@ void tscColumnListCopyAll(SArray* dst, const SArray* src);
void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, bool convertNchar);
void tscDequoteAndTrimToken(SStrToken* pToken);
int32_t tscValidateName(SStrToken* pToken);
int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded);
void tscIncStreamExecutionCount(void* pStream);
......
......@@ -398,7 +398,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const
TAOS_FIELD f;
if (type == SCREATE_BUILD_TABLE) {
f.type = TSDB_DATA_TYPE_BINARY;
f.bytes = (TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE;
f.bytes = (TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE;
tstrncpy(f.name, "Table", sizeof(f.name));
} else {
f.type = TSDB_DATA_TYPE_BINARY;
......@@ -465,7 +465,7 @@ int32_t tscRebuildCreateTableStatement(void *param,char *result) {
code = tscGetTableTagValue(builder, buf);
if (code == TSDB_CODE_SUCCESS) {
snprintf(result + strlen(result), TSDB_MAX_BINARY_LEN - strlen(result), "CREATE TABLE %s USING %s TAGS %s", builder->buf, builder->sTableName, buf);
snprintf(result + strlen(result), TSDB_MAX_BINARY_LEN - strlen(result), "CREATE TABLE `%s` USING `%s` TAGS %s", builder->buf, builder->sTableName, buf);
code = tscSCreateBuildResult(builder->pParentSql, SCREATE_BUILD_TABLE, builder->buf, result);
}
free(buf);
......@@ -574,12 +574,14 @@ static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, ch
}
char fullName[TSDB_TABLE_FNAME_LEN * 2] = {0};
char tblName[TSDB_TABLE_NAME_LEN + 1] = {0};
tNameGetDbName(&pTableMetaInfo->name, fullName);
extractTableName(pMeta->sTableName, param->sTableName);
snprintf(fullName + strlen(fullName), TSDB_TABLE_FNAME_LEN - strlen(fullName), ".%s", param->sTableName);
snprintf(fullName + strlen(fullName), TSDB_TABLE_FNAME_LEN - strlen(fullName), ".`%s`", param->sTableName);
strncpy(param->buf, tNameGetTableName(&pTableMetaInfo->name), TSDB_TABLE_NAME_LEN);
tableNameToStr(tblName, param->buf, '\'');
param->pParentSql = pSql;
param->pInterSql = pInterSql;
......@@ -602,7 +604,7 @@ static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, ch
return code;
}
snprintf(query + strlen(query), TSDB_MAX_BINARY_LEN - strlen(query), "SELECT %s FROM %s WHERE TBNAME IN(\'%s\')", columns, fullName, param->buf);
snprintf(query + strlen(query), TSDB_MAX_BINARY_LEN - strlen(query), "SELECT %s FROM %s WHERE TBNAME IN(\'%s\')", columns, fullName, tblName);
doAsyncQuery(pSql->pTscObj, pInterSql, tscSCreateCallBack, param, query, strlen(query));
free(query);
free(columns);
......@@ -619,7 +621,7 @@ static int32_t tscRebuildDDLForNormalTable(SSqlObj *pSql, const char *tableName,
SSchema *pSchema = tscGetTableSchema(pMeta);
char *result = ddl;
sprintf(result, "create table %s (", tableName);
sprintf(result, "create table `%s` (", tableName);
for (int32_t i = 0; i < numOfRows; ++i) {
uint8_t type = pSchema[i].type;
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
......@@ -646,7 +648,7 @@ static int32_t tscRebuildDDLForSuperTable(SSqlObj *pSql, const char *tableName,
int32_t totalRows = numOfRows + tscGetNumOfTags(pMeta);
SSchema *pSchema = tscGetTableSchema(pMeta);
sprintf(result, "create table %s (", tableName);
sprintf(result, "create table `%s` (", tableName);
for (int32_t i = 0; i < numOfRows; ++i) {
uint8_t type = pSchema[i].type;
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
......
......@@ -906,6 +906,18 @@ static int32_t doParseInsertStatement(SInsertStatementParam *pInsertParam, char
return TSDB_CODE_SUCCESS;
}
int validateTableName(char *tblName, int len, SStrToken* psTblToken, bool *dbIncluded) {
tstrncpy(psTblToken->z, tblName, TSDB_TABLE_FNAME_LEN);
psTblToken->n = len;
psTblToken->type = TK_ID;
tGetToken(psTblToken->z, &psTblToken->type);
return tscValidateName(psTblToken, true, dbIncluded);
}
static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundColumn) {
int32_t index = 0;
SStrToken sToken = {0};
......@@ -968,13 +980,27 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
sToken = tStrGetToken(sql, &index, false);
sql += index;
if (sToken.type == TK_ILLEGAL) {
return tscSQLSyntaxErrMsg(pCmd->payload, NULL, sql);
}
//the source super table is moved to the secondary position of the pTableMetaInfo list
if (pQueryInfo->numOfTables < 2) {
tscAddEmptyMetaInfo(pQueryInfo);
}
bool dbIncluded1 = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
code = validateTableName(sToken.z, sToken.n, &sTblToken, &dbIncluded1);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
STableMetaInfo *pSTableMetaInfo = tscGetMetaInfo(pQueryInfo, STABLE_INDEX);
code = tscSetTableFullName(&pSTableMetaInfo->name, &sToken, pSql);
code = tscSetTableFullName(&pSTableMetaInfo->name, &sTblToken, pSql, dbIncluded1);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -988,7 +1014,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
}
if (!UTIL_TABLE_IS_SUPER_TABLE(pSTableMetaInfo)) {
return tscInvalidOperationMsg(pInsertParam->msg, "create table only from super table is allowed", sToken.z);
return tscInvalidOperationMsg(pInsertParam->msg, "create table only from super table is allowed", sTblToken.z);
}
SSchema *pTagSchema = tscGetTableTagSchema(pSTableMetaInfo->pTableMeta);
......@@ -1144,12 +1170,16 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
}
sql = sToken.z;
bool dbIncluded2 = false;
if (tscValidateName(&tableToken) != TSDB_CODE_SUCCESS) {
sTblToken.z = buf;
code = validateTableName(tableToken.z, tableToken.n, &sTblToken, &dbIncluded2);
if (code != TSDB_CODE_SUCCESS) {
return tscInvalidOperationMsg(pInsertParam->msg, "invalid table name", *sqlstr);
}
int32_t ret = tscSetTableFullName(&pTableMetaInfo->name, &tableToken, pSql);
int32_t ret = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded2);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
......@@ -1179,16 +1209,6 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
return code;
}
int validateTableName(char *tblName, int len, SStrToken* psTblToken) {
tstrncpy(psTblToken->z, tblName, TSDB_TABLE_FNAME_LEN);
psTblToken->n = len;
psTblToken->type = TK_ID;
tGetToken(psTblToken->z, &psTblToken->type);
return tscValidateName(psTblToken);
}
static int32_t validateDataSource(SInsertStatementParam *pInsertParam, int32_t type, const char *sql) {
uint32_t *insertType = &pInsertParam->insertType;
if (*insertType == TSDB_QUERY_TYPE_STMT_INSERT && type == TSDB_QUERY_TYPE_INSERT) {
......@@ -1409,13 +1429,14 @@ int tsParseInsertSql(SSqlObj *pSql) {
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
bool dbIncluded = false;
// Check if the table name available or not
if (validateTableName(sToken.z, sToken.n, &sTblToken) != TSDB_CODE_SUCCESS) {
if (validateTableName(sToken.z, sToken.n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS) {
code = tscInvalidOperationMsg(pInsertParam->msg, "table name invalid", sToken.z);
goto _clean;
}
if ((code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql)) != TSDB_CODE_SUCCESS) {
if ((code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded)) != TSDB_CODE_SUCCESS) {
goto _clean;
}
......
......@@ -558,8 +558,10 @@ static int32_t retrieveTableMeta(TAOS* taos, char* tableName, STableMeta** pTabl
registerSqlObj(pSql);
SStrToken tableToken = {.z = tableNameLowerCase, .n = (uint32_t)strlen(tableNameLowerCase), .type = TK_ID};
tGetToken(tableNameLowerCase, &tableToken.type);
bool dbIncluded = false;
// Check if the table name available or not
if (tscValidateName(&tableToken) != TSDB_CODE_SUCCESS) {
if (tscValidateName(&tableToken, true, &dbIncluded) != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
sprintf(pSql->cmd.payload, "table name is invalid");
tscFreeRegisteredSqlObj(pSql);
......@@ -567,7 +569,7 @@ static int32_t retrieveTableMeta(TAOS* taos, char* tableName, STableMeta** pTabl
}
SName sname = {0};
if ((code = tscSetTableFullName(&sname, &tableToken, pSql)) != TSDB_CODE_SUCCESS) {
if ((code = tscSetTableFullName(&sname, &tableToken, pSql, dbIncluded)) != TSDB_CODE_SUCCESS) {
tscFreeRegisteredSqlObj(pSql);
return code;
}
......
......@@ -63,12 +63,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **index,
}
}
//convert dot to underscore for now, will be removed once dot is allowed in tbname.
if (*cur == '.') {
pSml->stableName[len] = '_';
} else {
pSml->stableName[len] = tolower(*cur);
}
pSml->stableName[len] = *cur;
cur++;
len++;
......@@ -486,13 +481,6 @@ static int32_t parseMetricFromJSON(cJSON *root, TAOS_SML_DATA_POINT* pSml, SSmlL
return TSDB_CODE_TSC_INVALID_JSON;
}
//convert dot to underscore for now, will be removed once dot is allowed in tbname.
for (int i = 0; i < stableLen; ++i) {
if (metric->valuestring[i] == '.') {
metric->valuestring[i] = '_';
}
}
tstrncpy(pSml->stableName, metric->valuestring, stableLen + 1);
strntolower_s(pSml->stableName, pSml->stableName, (int32_t)stableLen);
......
......@@ -1585,9 +1585,10 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
SSqlObj* pSql = pStmt->pSql;
SSqlCmd* pCmd = &pSql->cmd;
uint32_t nameLen = (uint32_t)strlen(name);
if (name == NULL) {
tscError("0x%"PRIx64" name is NULL", pSql->self);
if (name == NULL || nameLen <= 0) {
tscError("0x%"PRIx64" tbname is NULL", pSql->self);
STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "name is NULL"));
}
......@@ -1603,6 +1604,20 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
pStmt->last = STMT_SETTBNAME;
SStrToken tname = {0};
tname.type = TK_STRING;
tname.z = (char *)strdup(name);
tname.n = (uint32_t)strlen(name);
bool dbIncluded = false;
// Check if the table name available or not
if (tscValidateName(&tname, true, &dbIncluded) != TSDB_CODE_SUCCESS) {
tscError("0x%"PRIx64" tbname[%s] is invalid", pSql->self, name);
free(tname.z);
STMT_RET(invalidOperationMsg(tscGetErrorMsgPayload(&pStmt->pSql->cmd), "name is invalid"));
}
uint64_t* uid = (uint64_t*)taosHashGet(pStmt->mtb.pTableHash, name, strlen(name));
if (uid != NULL) {
pStmt->mtb.currentUid = *uid;
......@@ -1610,6 +1625,7 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pStmt->mtb.pTableBlockHashList, (const char*)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid));
if (t1 == NULL) {
tscError("0x%"PRIx64" no table data block in hash list, uid:%" PRId64 , pSql->self, pStmt->mtb.currentUid);
free(tname.z);
STMT_RET(TSDB_CODE_TSC_APP_ERROR);
}
......@@ -1624,6 +1640,7 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
taosHashPut(pCmd->insertParam.pTableBlockHashList, (void *)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid), (void*)t1, POINTER_BYTES);
tscDebug("0x%"PRIx64" table:%s is already prepared, uid:%" PRIu64, pSql->self, name, pStmt->mtb.currentUid);
free(tname.z);
STMT_RET(TSDB_CODE_SUCCESS);
}
......@@ -1632,13 +1649,10 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
STableMeta* pTableMeta = pTableMetaInfo->pTableMeta;
char sTableName[TSDB_TABLE_FNAME_LEN] = {0};
tstrncpy(sTableName, pTableMeta->sTableName, sizeof(sTableName));
SStrToken tname = {0};
tname.type = TK_STRING;
tname.z = (char *)name;
tname.n = (uint32_t)strlen(name);
SName fullname = {0};
tscSetTableFullName(&fullname, &tname, pSql);
tscSetTableFullName(&fullname, &tname, pSql, dbIncluded);
free(tname.z);
memcpy(&pTableMetaInfo->name, &fullname, sizeof(fullname));
......@@ -1672,6 +1686,8 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags
STMT_RET(TSDB_CODE_SUCCESS);
}
free(tname.z);
if (pStmt->mtb.tagSet) {
pStmt->mtb.tbname = tscReplaceStrToken(&pSql->sqlstr, &pStmt->mtb.tbname, name);
} else {
......
......@@ -142,6 +142,8 @@ static int32_t checkQueryRangeForFill(SSqlCmd* pCmd, SQueryInfo* pQueryInfo);
static int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo);
static tSqlExpr* extractExprForSTable(SSqlCmd* pCmd, tSqlExpr** pExpr, SQueryInfo* pQueryInfo, int32_t tableIndex);
int validateTableName(char *tblName, int len, SStrToken* psTblToken, bool *dbIncluded);
static bool isTimeWindowQuery(SQueryInfo* pQueryInfo) {
return pQueryInfo->interval.interval > 0 || pQueryInfo->sessionWindow.gap > 0;
}
......@@ -567,8 +569,18 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg3 = "param name too long";
SStrToken* pzName = taosArrayGet(pInfo->pMiscInfo->a, 0);
if ((pInfo->type != TSDB_SQL_DROP_DNODE) && (tscValidateName(pzName) != TSDB_CODE_SUCCESS)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
bool escapeEnabled = (pInfo->type == TSDB_SQL_DROP_TABLE) ? true: false;
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
if (pInfo->type != TSDB_SQL_DROP_DNODE) {
if ((escapeEnabled && (validateTableName(pzName->z, pzName->n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS)) ||
((!escapeEnabled) && (tscValidateName(pzName, escapeEnabled, &dbIncluded) != TSDB_CODE_SUCCESS))){
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
}
if (pInfo->type == TSDB_SQL_DROP_DB) {
......@@ -581,7 +593,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
} else if (pInfo->type == TSDB_SQL_DROP_TABLE) {
assert(taosArrayGetSize(pInfo->pMiscInfo->a) == 1);
code = tscSetTableFullName(&pTableMetaInfo->name, pzName, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded);
if(code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -605,7 +617,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg = "invalid db name";
SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0);
if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pToken, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg);
}
......@@ -652,7 +664,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
char buf[TSDB_DB_NAME_LEN] = {0};
SStrToken token = taosTokenDup(&pCreateDB->dbname, buf, tListLen(buf));
if (tscValidateName(&token) != TSDB_CODE_SUCCESS) {
if (tscValidateName(&token, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
......@@ -699,7 +711,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
if (tscValidateName(pName) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pName, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
......@@ -721,11 +733,17 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg1 = "invalid table name";
SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0);
if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
if (validateTableName(pToken->z, pToken->n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
// additional msg has been attached already
code = tscSetTableFullName(&pTableMetaInfo->name, pToken, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -737,11 +755,17 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg1 = "invalid table name";
SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0);
if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
if (validateTableName(pToken->z, pToken->n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
code = tscSetTableFullName(&pTableMetaInfo->name, pToken, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -752,7 +776,8 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg1 = "invalid database name";
SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0);
if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pToken, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
......@@ -816,7 +841,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
if (tscValidateName(pName) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pName, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
......@@ -1336,7 +1361,7 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* pSl
return TSDB_CODE_SUCCESS;
}
int32_t tscSetTableFullName(SName* pName, SStrToken* pTableName, SSqlObj* pSql) {
int32_t tscSetTableFullName(SName* pName, SStrToken* pTableName, SSqlObj* pSql, bool dbIncluded) {
const char* msg1 = "name too long";
const char* msg2 = "acctId too long";
const char* msg3 = "no acctId";
......@@ -1345,7 +1370,12 @@ int32_t tscSetTableFullName(SName* pName, SStrToken* pTableName, SSqlObj* pSql)
SSqlCmd* pCmd = &pSql->cmd;
int32_t code = TSDB_CODE_SUCCESS;
int32_t idx = getDelimiterIndex(pTableName);
int32_t idx = -1;
if (dbIncluded) {
idx = getDelimiterIndex(pTableName);
}
if (idx != -1) { // db has been specified in sql string so we ignore current db path
char* acctId = getAccountId(pSql);
if (acctId == NULL || strlen(acctId) <= 0) {
......@@ -3216,6 +3246,7 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg3 = "database name too long";
const char* msg5 = "database name is empty";
const char* msg6 = "pattern string is empty";
const char* msg7 = "pattern is invalid";
/*
* database prefix in pInfo->pMiscInfo->a[0]
......@@ -3235,8 +3266,7 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (pDbPrefixToken->n <= 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
}
if (tscValidateName(pDbPrefixToken) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pDbPrefixToken, false, NULL) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
......@@ -3249,6 +3279,10 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
// show table/stable like 'xxxx', set the like pattern for show tables
SStrToken* pPattern = &pShowInfo->pattern;
if (pPattern->type != 0) {
if (pPattern->type == TK_ID && pPattern->z[0] == TS_ESCAPE_CHAR) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
}
pPattern->n = strdequote(pPattern->z);
if (pPattern->n <= 0) {
......@@ -5898,12 +5932,13 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd);
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, DEFAULT_TABLE_INDEX);
bool dbIncluded = false;
if (tscValidateName(&(pAlterSQL->name)) != TSDB_CODE_SUCCESS) {
if (tscValidateName(&(pAlterSQL->name), true, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
code = tscSetTableFullName(&pTableMetaInfo->name, &(pAlterSQL->name), pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &(pAlterSQL->name), pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -7456,12 +7491,13 @@ int32_t doCheckForCreateTable(SSqlObj* pSql, int32_t subClauseIndex, SSqlInfo* p
// if sql specifies db, use it, otherwise use default db
SStrToken* pzTableName = &(pCreateTable->name);
if (tscValidateName(pzTableName) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
if (tscValidateName(pzTableName, true, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
int32_t code = tscSetTableFullName(&pTableMetaInfo->name, pzTableName, pSql);
int32_t code = tscSetTableFullName(&pTableMetaInfo->name, pzTableName, pSql, dbIncluded);
if(code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -7521,11 +7557,18 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
SCreatedTableInfo* pCreateTableInfo = taosArrayGet(pCreateTable->childTableInfo, j);
SStrToken* pToken = &pCreateTableInfo->stableName;
if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
int32_t code = validateTableName(pToken->z, pToken->n, &sTblToken, &dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
int32_t code = tscSetTableFullName(&pStableMetaInfo->name, pToken, pSql);
code = tscSetTableFullName(&pStableMetaInfo->name, &sTblToken, pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -7694,14 +7737,15 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
kvRowCpy(pTag->data, row);
free(row);
bool dbIncluded2 = false;
// table name
if (tscValidateName(&(pCreateTableInfo->name)) != TSDB_CODE_SUCCESS) {
if (tscValidateName(&(pCreateTableInfo->name), true, &dbIncluded2) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, TABLE_INDEX);
ret = tscSetTableFullName(&pTableMetaInfo->name, &pCreateTableInfo->name, pSql);
ret = tscSetTableFullName(&pTableMetaInfo->name, &pCreateTableInfo->name, pSql, dbIncluded2);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
......@@ -7738,8 +7782,9 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) {
// if sql specifies db, use it, otherwise use default db
SStrToken* pName = &(pCreateTable->name);
SSqlNode* pSqlNode = pCreateTable->pSelect;
bool dbIncluded1 = false;
if (tscValidateName(pName) != TSDB_CODE_SUCCESS) {
if (tscValidateName(pName, true, &dbIncluded1) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
......@@ -7753,12 +7798,19 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) {
}
SRelElementPair* p1 = taosArrayGet(pFromInfo->list, 0);
SStrToken srcToken = {.z = p1->tableName.z, .n = p1->tableName.n, .type = TK_STRING};
if (tscValidateName(&srcToken) != TSDB_CODE_SUCCESS) {
SStrToken srcToken = {.z = p1->tableName.z, .n = p1->tableName.n, .type = p1->tableName.type};
bool dbIncluded2 = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
int32_t code = validateTableName(srcToken.z, srcToken.n, &sTblToken, &dbIncluded2);
if (code != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
int32_t code = tscSetTableFullName(&pTableMetaInfo->name, &srcToken, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded2);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -7800,7 +7852,7 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) {
}
// set the created table[stream] name
code = tscSetTableFullName(&pTableMetaInfo->name, pName, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, pName, pSql, dbIncluded1);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -8191,14 +8243,18 @@ static int32_t getTableNameFromSqlNode(SSqlNode* pSqlNode, SArray* tableNameList
if (t->type == TK_INTEGER || t->type == TK_FLOAT) {
return invalidOperationMsg(msgBuf, msg1);
}
tscDequoteAndTrimToken(t);
if (tscValidateName(t) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
if (validateTableName(t->z, t->n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(msgBuf, msg1);
}
SName name = {0};
int32_t code = tscSetTableFullName(&name, t, pSql);
int32_t code = tscSetTableFullName(&name, &sTblToken, pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -8453,12 +8509,18 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
}
tscDequoteAndTrimToken(oriName);
if (tscValidateName(oriName) != TSDB_CODE_SUCCESS) {
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
sTblToken.z = buf;
if (validateTableName(oriName->z, oriName->n, &sTblToken, &dbIncluded) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i);
code = tscSetTableFullName(&pTableMetaInfo->name, oriName, pSql);
code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql, dbIncluded);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
......@@ -8470,7 +8532,7 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
}
tscDequoteAndTrimToken(aliasName);
if (tscValidateName(aliasName) != TSDB_CODE_SUCCESS || aliasName->n >= TSDB_TABLE_NAME_LEN) {
if (tscValidateName(aliasName, false, NULL) != TSDB_CODE_SUCCESS || aliasName->n >= TSDB_TABLE_NAME_LEN) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
......@@ -9004,6 +9066,7 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS
if (colSize > 0) {
SColIndex* idx = taosArrayGet(pCols, colSize - 1);
SSchema* pSchema = tscGetTableColumnSchema(pTableMeta, idx->colIndex);
// convert time by precision
if (pSchema != NULL && TSDB_DATA_TYPE_TIMESTAMP == pSchema->type && TSDB_DATA_TYPE_BINARY == (*pExpr)->pVal->nType) {
......
......@@ -2870,7 +2870,7 @@ int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVg
for(int32_t i = 0; i < numOfTable; ++i) {
char* name = taosArrayGetP(pNameList, i);
if (i < numOfTable - 1 || numOfVgroupList > 0 || numOfUdf > 0) {
len = sprintf(start, "%s,", name);
len = sprintf(start, "%s`", name);
} else {
len = sprintf(start, "%s", name);
}
......@@ -2881,7 +2881,7 @@ int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVg
for(int32_t i = 0; i < numOfVgroupList; ++i) {
char* name = taosArrayGetP(pVgroupNameList, i);
if (i < numOfVgroupList - 1 || numOfUdf > 0) {
len = sprintf(start, "%s,", name);
len = sprintf(start, "%s`", name);
} else {
len = sprintf(start, "%s", name);
}
......@@ -2892,7 +2892,7 @@ int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVg
for(int32_t i = 0; i < numOfUdf; ++i) {
SUdfInfo * u = taosArrayGet(pUdfList, i);
if (i < numOfUdf - 1) {
len = sprintf(start, "%s,", u->name);
len = sprintf(start, "%s`", u->name);
} else {
len = sprintf(start, "%s", u->name);
}
......
......@@ -2759,13 +2759,13 @@ void tscColumnListDestroy(SArray* pColumnList) {
* 'first_part.second_part'
*
*/
static int32_t validateQuoteToken(SStrToken* pToken) {
static int32_t validateQuoteToken(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded) {
tscDequoteAndTrimToken(pToken);
int32_t k = tGetToken(pToken->z, &pToken->type);
if (pToken->type == TK_STRING) {
return tscValidateName(pToken);
return tscValidateName(pToken, escapeEnabled, dbIncluded);
}
if (k != pToken->n || pToken->type != TK_ID) {
......@@ -2817,14 +2817,74 @@ void tscDequoteAndTrimToken(SStrToken* pToken) {
pToken->n = last - first;
}
int32_t tscValidateName(SStrToken* pToken) {
if (pToken == NULL || pToken->z == NULL ||
(pToken->type != TK_STRING && pToken->type != TK_ID)) {
void tscRmEscapeAndTrimToken(SStrToken* pToken) {
uint32_t first = 0, last = pToken->n;
// trim leading spaces
while (first < last) {
char c = pToken->z[first];
if (c != ' ' && c != '\t') {
break;
}
first++;
}
// trim ending spaces
while (first < last) {
char c = pToken->z[last - 1];
if (c != ' ' && c != '\t') {
break;
}
last--;
}
// there are still at least two characters
if (first < last - 1) {
char c = pToken->z[first];
// dequote
if ((c == '`') && c == pToken->z[last - 1]) {
first++;
last--;
}
}
// left shift the string and pad spaces
for (uint32_t i = 0; i + first < last; i++) {
pToken->z[i] = pToken->z[first + i];
}
for (uint32_t i = last - first; i < pToken->n; i++) {
pToken->z[i] = ' ';
}
// adjust token length
pToken->n = last - first;
}
int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded) {
if (pToken == NULL || pToken->z == NULL
|| (pToken->type != TK_STRING && pToken->type != TK_ID)) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
char* sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true);
if ((!escapeEnabled) && pToken->type == TK_ID) {
if (pToken->z[0] == TS_ESCAPE_CHAR) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
}
char* sep = NULL;
if (escapeEnabled) {
sep = tableNameGetPosition(pToken, TS_PATH_DELIMITER[0]);
} else {
sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true);
}
if (sep == NULL) { // single part
if (dbIncluded) *dbIncluded = false;
if (pToken->type == TK_STRING) {
tscDequoteAndTrimToken(pToken);
......@@ -2835,23 +2895,30 @@ int32_t tscValidateName(SStrToken* pToken) {
// single token, validate it
if (len == pToken->n) {
return validateQuoteToken(pToken);
return validateQuoteToken(pToken, escapeEnabled, NULL);
} else {
sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true);
if (sep == NULL) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
*dbIncluded = true;
return tscValidateName(pToken);
return tscValidateName(pToken, escapeEnabled, NULL);
}
} else if (pToken->type == TK_ID) {
tscRmEscapeAndTrimToken(pToken);
return TSDB_CODE_SUCCESS;
} else {
if (isNumber(pToken)) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
}
}
} else { // two part
int32_t oldLen = pToken->n;
char* pStr = pToken->z;
bool firstPartQuote = false;
if (dbIncluded) *dbIncluded = true;
if (pToken->type == TK_SPACE) {
pToken->n = (uint32_t)strtrim(pToken->z);
......@@ -2866,8 +2933,13 @@ int32_t tscValidateName(SStrToken* pToken) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
if (pToken->type == TK_STRING) {
if (validateQuoteToken(pToken, escapeEnabled, NULL) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
} else {
tscStrToLower(pToken->z,pToken->n);
firstPartQuote = true;
}
}
int32_t firstPartLen = pToken->n;
......@@ -2879,12 +2951,20 @@ int32_t tscValidateName(SStrToken* pToken) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
if (pToken->type == TK_STRING) {
if (validateQuoteToken(pToken, escapeEnabled, NULL) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
} else {
tscStrToLower(pToken->z,pToken->n);
}
}
if (escapeEnabled && pToken->type == TK_ID) {
tscRmEscapeAndTrimToken(pToken);
}
// re-build the whole name string
if (pStr[firstPartLen] == TS_PATH_DELIMITER[0]) {
if (!firstPartQuote) {
// first part do not have quote do nothing
} else {
pStr[firstPartLen] = TS_PATH_DELIMITER[0];
......@@ -2894,8 +2974,6 @@ int32_t tscValidateName(SStrToken* pToken) {
}
pToken->n += (firstPartLen + sizeof(TS_PATH_DELIMITER[0]));
pToken->z = pStr;
tscStrToLower(pToken->z,pToken->n);
}
return TSDB_CODE_SUCCESS;
......@@ -5009,14 +5087,16 @@ static int32_t doAddTableName(char* nextStr, char** str, SArray* pNameArray, SSq
SStrToken sToken = {.n = len, .type = TK_ID, .z = tablename};
tGetToken(tablename, &sToken.type);
bool dbIncluded = false;
// Check if the table name available or not
if (tscValidateName(&sToken) != TSDB_CODE_SUCCESS) {
if (tscValidateName(&sToken, true, &dbIncluded) != TSDB_CODE_SUCCESS) {
sprintf(pCmd->payload, "table name is invalid");
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
}
SName name = {0};
if ((code = tscSetTableFullName(&name, &sToken, pSql)) != TSDB_CODE_SUCCESS) {
if ((code = tscSetTableFullName(&name, &sToken, pSql, dbIncluded)) != TSDB_CODE_SUCCESS) {
return code;
}
......
......@@ -92,6 +92,10 @@ size_t tableIdPrefix(const char* name, char* prefix, int32_t len);
void extractTableNameFromToken(SStrToken *pToken, SStrToken* pTable);
char *tableNameGetPosition(SStrToken* pToken, char target);
char *tableNameToStr(char *dst, char *src, char quote);
SSchema tGetUserSpecifiedColumnSchema(tVariant* pVal, SStrToken* exprStr, const char* name);
bool tscValidateTableNameLength(size_t len);
......
......@@ -151,6 +151,63 @@ int64_t taosGetIntervalStartTimestamp(int64_t startTime, int64_t slidingTime, in
#endif
char *tableNameGetPosition(SStrToken* pToken, char target) {
bool inEscape = false;
bool inQuote = false;
char quotaStr = 0;
for (uint32_t i = 0; i < pToken->n; ++i) {
if (*(pToken->z + i) == target && (!inEscape) && (!inQuote)) {
return pToken->z + i;
}
if (*(pToken->z + i) == TS_ESCAPE_CHAR) {
if (!inQuote) {
inEscape = !inEscape;
}
}
if (*(pToken->z + i) == '\'' || *(pToken->z + i) == '"') {
if (!inEscape) {
if (!inQuote) {
quotaStr = *(pToken->z + i);
inQuote = !inQuote;
} else if (quotaStr == *(pToken->z + i)) {
inQuote = !inQuote;
}
}
}
}
return NULL;
}
char *tableNameToStr(char *dst, char *src, char quote) {
*dst = 0;
if (src == NULL) {
return NULL;
}
int32_t len = (int32_t)strlen(src);
if (len <= 0) {
return NULL;
}
int32_t j = 0;
for (int32_t i = 0; i < len; ++i) {
if (*(src + i) == quote) {
*(dst + j++) = '\\';
}
*(dst + j++) = *(src + i);
}
return dst;
}
/*
* tablePrefix.columnName
* extract table name and save it in pTable, with only column name in pToken
......@@ -162,12 +219,17 @@ void extractTableNameFromToken(SStrToken* pToken, SStrToken* pTable) {
return;
}
char* r = strnchr(pToken->z, sep, pToken->n, false);
if (r != NULL) { // record the table name token
pTable->n = (uint32_t)(r - pToken->z);
pTable->z = pToken->z;
char* r = tableNameGetPosition(pToken, sep);
if (r != NULL) { // record the table name token
if (pToken->z[0] == TS_ESCAPE_CHAR && *(r - 1) == TS_ESCAPE_CHAR) {
pTable->n = (uint32_t)(r - pToken->z - 2);
pTable->z = pToken->z + 1;
} else {
pTable->n = (uint32_t)(r - pToken->z);
pTable->z = pToken->z;
}
r += 1;
pToken->n -= (uint32_t)(r - pToken->z);
pToken->z = r;
......
......@@ -81,7 +81,7 @@ void tVariantCreate(tVariant *pVar, SStrToken *token) {
case TSDB_DATA_TYPE_BINARY: {
pVar->pz = strndup(token->z, token->n);
pVar->nLen = strRmquote(pVar->pz, token->n);
pVar->nLen = strRmquoteEscape(pVar->pz, token->n);
break;
}
case TSDB_DATA_TYPE_TIMESTAMP: {
......
Subproject commit 016d8e82a24d72779be0ab0090580a372b4fffca
Subproject commit 72ecc6d636453c6a9b71d78d3edce385165cc35f
......@@ -98,6 +98,7 @@ extern const int32_t TYPE_BYTES[15];
#define TSDB_ERR -1
#define TS_PATH_DELIMITER "."
#define TS_ESCAPE_CHAR '`'
#define TSDB_TIME_PRECISION_MILLI 0
#define TSDB_TIME_PRECISION_MICRO 1
......
......@@ -214,6 +214,8 @@
#define TK_VALUES 196
#define TK_SPACE 300
#define TK_COMMENT 301
#define TK_ILLEGAL 302
......
......@@ -27,7 +27,12 @@
#define MAX_IP_SIZE 20
#define MAX_HISTORY_SIZE 1000
#define MAX_COMMAND_SIZE 1048586
#define HISTORY_FILE ".taos_history"
#ifdef _TD_PRO_
#define HISTORY_FILE ".prodb_history"
#else
#define HISTORY_FILE ".taos_history"
#endif
#define DEFAULT_RES_SHOW_NUM 100
......
......@@ -250,6 +250,7 @@ int32_t shellRunCommand(TAOS* con, char* command) {
break;
case '\'':
case '"':
case '`':
if (quote) {
*p++ = '\\';
}
......@@ -271,7 +272,7 @@ int32_t shellRunCommand(TAOS* con, char* command) {
if (quote == c) {
quote = 0;
} else if (quote == 0 && (c == '\'' || c == '"')) {
} else if (quote == 0 && (c == '\'' || c == '"' || c == '`')) {
quote = c;
}
......@@ -308,8 +309,8 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
char * fname = NULL;
bool printMode = false;
if ((sptr = strstr(command, ">>")) != NULL) {
cptr = strstr(command, ";");
if ((sptr = tstrstr(command, ">>", true)) != NULL) {
cptr = tstrstr(command, ";", true);
if (cptr != NULL) {
*cptr = '\0';
}
......@@ -322,8 +323,8 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
fname = full_path.we_wordv[0];
}
if ((sptr = strstr(command, "\\G")) != NULL) {
cptr = strstr(command, ";");
if ((sptr = tstrstr(command, "\\G", true)) != NULL) {
cptr = tstrstr(command, ";", true);
if (cptr != NULL) {
*cptr = '\0';
}
......
......@@ -1020,25 +1020,25 @@ static void dumpCreateMTableClause(
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, ", \'%s\'", tableDes->cols[counter].note);
if (tableDes->cols[counter].var_value) {
pstr += sprintf(pstr, ", %s",
pstr += sprintf(pstr, ", \'%s\'",
tableDes->cols[counter].var_value);
} else {
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].value);
pstr += sprintf(pstr, ", \'%s\'", tableDes->cols[counter].value);
}
} else {
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].value);
pstr += sprintf(pstr, ", \'%s\'", tableDes->cols[counter].value);
}
} else {
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].note);
if (tableDes->cols[counter].var_value) {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].var_value);
pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].var_value);
} else {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].value);
pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].value);
}
} else {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].value);
pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].value);
}
/* pstr += sprintf(pstr, "%s", tableDes->cols[counter].note); */
}
......@@ -1149,6 +1149,10 @@ static int64_t dumpNormalTable(
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
errorPrint("%s() LN%d, failed to get table[%s] schema\n",
__func__,
__LINE__,
tbName);
free(tableDes);
return -1;
}
......@@ -1160,6 +1164,10 @@ static int64_t dumpNormalTable(
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
errorPrint("%s() LN%d, failed to get table[%s] schema\n",
__func__,
__LINE__,
tbName);
free(tableDes);
return -1;
}
......@@ -1172,19 +1180,21 @@ static int64_t dumpNormalTable(
if (g_args.avro) {
if (0 != convertTbDesToAvroSchema(
dbName, tbName, tableDes, colCount, &jsonAvroSchema)) {
errorPrint("%s() LN%d, convertTbDesToAvroSchema failed\n",
__func__,
__LINE__);
freeTbDes(tableDes);
return -1;
}
}
free(tableDes);
int64_t ret = 0;
if (!g_args.schemaonly) {
ret = dumpTableData(fp, tbName, dbName, precision,
jsonAvroSchema);
}
freeTbDes(tableDes);
return ret;
}
......@@ -1281,20 +1291,23 @@ static void *dumpNtbOfDb(void *arg) {
return NULL;
}
int64_t count;
for (int64_t i = 0; i < pThreadInfo->tablesOfDumpOut; i++) {
debugPrint("[%d] No.\t%"PRId64" table name: %s\n",
pThreadInfo->threadIndex, i,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name);
dumpNormalTable(
count = dumpNormalTable(
pThreadInfo->dbName,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->stable,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name,
pThreadInfo->precision,
fp);
if (count < 0) {
break;
}
}
fclose(fp);
return NULL;
}
......@@ -1340,16 +1353,20 @@ static void *dumpNormalTablesOfStb(void *arg) {
TAOS_ROW row = NULL;
int64_t i = 0;
int64_t count;
while((row = taos_fetch_row(res)) != NULL) {
debugPrint("[%d] sub table %"PRId64": name: %s\n",
pThreadInfo->threadIndex, i++, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX]);
dumpNormalTable(
count = dumpNormalTable(
pThreadInfo->dbName,
pThreadInfo->stbName,
(char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
pThreadInfo->precision,
fp);
if (count < 0) {
break;
}
}
fclose(fp);
......@@ -2006,9 +2023,9 @@ static int getTableDes(
if (row[TSDB_SHOW_TABLES_NAME_INDEX] == NULL) {
sprintf(tableDes->cols[i].note, "%s", "NUL");
sprintf(tableDes->cols[i].value, "%s", "NULL");
taos_free_result(res);
res = NULL;
taos_close(taos);
continue;
}
......@@ -2050,26 +2067,22 @@ static int getTableDes(
int len = strlen((char *)row[0]);
// FIXME for long value
if (len < (COL_VALUEBUF_LEN - 2)) {
tableDes->cols[i].value[0] = '\'';
converStringToReadable(
(char *)row[0],
length[0],
tableDes->cols[i].value + 1,
tableDes->cols[i].value,
len);
tableDes->cols[i].value[len+1] = '\'';
} else {
tableDes->cols[i].var_value = calloc(1, len + 2);
tableDes->cols[i].var_value = calloc(1, len * 2);
if (tableDes->cols[i].var_value == NULL) {
errorPrint("%s() LN%d, memory alalocation failed!\n",
__func__, __LINE__);
taos_free_result(res);
return -1;
}
tableDes->cols[i].var_value[0] = '\'';
converStringToReadable((char *)row[0],
length[0],
(char *)(tableDes->cols[i].var_value + 1), len);
tableDes->cols[i].var_value[len+1] = '\'';
(char *)(tableDes->cols[i].var_value), len);
}
break;
......
......@@ -2975,7 +2975,7 @@ static int32_t mnodeProcessMultiTableMetaMsg(SMnodeMsg *pMsg) {
int32_t num = 0;
int32_t code = TSDB_CODE_SUCCESS;
char* str = strndup(pInfo->tableNames, contLen);
char** nameList = strsplit(str, ",", &num);
char** nameList = strsplit(str, "`", &num);
SArray* pList = taosArrayInit(4, POINTER_BYTES);
SMultiTableMeta *pMultiMeta = NULL;
......
......@@ -94,7 +94,7 @@ cpxName(A) ::= DOT ids(Y). {A = Y; A.n += 1; }
cmd ::= SHOW CREATE TABLE ids(X) cpxName(Y). {
X.n += Y.n;
setDCLSqlElems(pInfo, TSDB_SQL_SHOW_CREATE_TABLE, 1, &X);
}
}
cmd ::= SHOW CREATE STABLE ids(X) cpxName(Y). {
X.n += Y.n;
setDCLSqlElems(pInfo, TSDB_SQL_SHOW_CREATE_STABLE, 1, &X);
......@@ -162,6 +162,7 @@ cmd ::= DESCRIBE ids(X) cpxName(Y). {
X.n += Y.n;
setDCLSqlElems(pInfo, TSDB_SQL_DESCRIBE_TABLE, 1, &X);
}
cmd ::= DESC ids(X) cpxName(Y). {
X.n += Y.n;
setDCLSqlElems(pInfo, TSDB_SQL_DESCRIBE_TABLE, 1, &X);
......
......@@ -1802,10 +1802,7 @@ int32_t filterInitValFieldData(SFilterInfo *info) {
}
if (unit->compare.optr == TSDB_RELATION_IN) {
SSchema *sch = FILTER_UNIT_COL_DESC(info, unit);
bool tolower = (sch->colId == -1) ? true : false;
filterConvertSetFromBinary((void **)&fi->data, var->pz, var->nLen, type, tolower);
filterConvertSetFromBinary((void **)&fi->data, var->pz, var->nLen, type, false);
CHK_LRET(fi->data == NULL, TSDB_CODE_QRY_APP_ERROR, "failed to convert in param");
FILTER_SET_FLAG(fi->flag, FLD_DATA_IS_HASH);
......
此差异已折叠。
......@@ -23,7 +23,7 @@ int32_t testValidateName(char* name) {
token.type = 0;
tGetToken(name, &token.type);
return tscValidateName(&token);
return tscValidateName(&token, false, NULL);
}
}
......
......@@ -27,7 +27,9 @@ extern "C" {
int32_t strdequote(char *src);
int32_t strRmquote(char *z, int32_t len);
int32_t strRmquoteEscape(char *z, int32_t len);
size_t strtrim(char *src);
char * tstrstr(char *src, char *dst, bool ignoreInEsc);
char * strnchr(char *haystack, char needle, int32_t len, bool skipquote);
char ** strsplit(char *src, const char *delim, int32_t *num);
char * strtolower(char *dst, const char *src);
......
......@@ -228,7 +228,7 @@ static SKeyword keywordTable[] = {
{"FUNCTIONS", TK_FUNCTIONS},
{"OUTPUTTYPE", TK_OUTPUTTYPE},
{"AGGREGATE", TK_AGGREGATE},
{"BUFSIZE", TK_BUFSIZE},
{"BUFSIZE", TK_BUFSIZE}
};
static const char isIdChar[] = {
......@@ -442,6 +442,17 @@ uint32_t tGetToken(char* z, uint32_t* tokenId) {
break;
}
case '`': {
for (i = 1; z[i]; i++) {
if (z[i] == '`') {
i++;
*tokenId = TK_ID;
return i;
}
}
break;
}
case '.': {
/*
* handle the the float number with out integer part
......
......@@ -84,6 +84,20 @@ int32_t strRmquote(char *z, int32_t len){
return len - 2 - cnt;
}
int32_t strRmquoteEscape(char *z, int32_t len) {
if (len <= 0) return len;
if (z[0] == '\'' || z[0] == '\"') {
return strRmquote(z, len);
} else if (len > 1 && z[0] == TS_ESCAPE_CHAR && z[len - 1] == TS_ESCAPE_CHAR) {
memmove(z, z + 1, len - 2);
return len - 2;
}
return len;
}
size_t strtrim(char *z) {
int32_t i = 0;
......@@ -165,6 +179,43 @@ char *strnchr(char *haystack, char needle, int32_t len, bool skipquote) {
return NULL;
}
char *tstrstr(char *src, char *dst, bool ignoreInEsc) {
if (!ignoreInEsc) {
return strstr(src, dst);
}
int32_t len = (int32_t)strlen(src);
bool inEsc = false;
char escChar = 0;
char *str = src, *res = NULL;
for (int32_t i = 0; i < len; ++i) {
if (src[i] == TS_ESCAPE_CHAR || src[i] == '\'' || src[i] == '\"') {
if (!inEsc) {
escChar = src[i];
src[i] = 0;
res = strstr(str, dst);
src[i] = escChar;
if (res) {
return res;
}
str = NULL;
} else {
if (src[i] != escChar) {
continue;
}
str = src + i + 1;
}
inEsc = !inEsc;
continue;
}
}
return str ? strstr(str, dst) : NULL;
}
char* strtolower(char *dst, const char *src) {
......@@ -195,7 +246,7 @@ char* strtolower(char *dst, const char *src) {
}
char* strntolower(char *dst, const char *src, int32_t n) {
int esc = 0;
int esc = 0, inEsc = 0;
char quote = 0, *p = dst, c;
assert(dst != NULL);
......@@ -212,10 +263,16 @@ char* strntolower(char *dst, const char *src, int32_t n) {
} else if (c == quote) {
quote = 0;
}
} else if (inEsc) {
if (c == '`') {
inEsc = 0;
}
} else if (c >= 'A' && c <= 'Z') {
c -= 'A' - 'a';
} else if (c == '\'' || c == '"') {
} else if (inEsc == 0 && (c == '\'' || c == '"')) {
quote = c;
} else if (c == '`' && quote == 0) {
inEsc = 1;
}
*p++ = c;
}
......
......@@ -17,6 +17,7 @@ sys.path.insert(0, os.getcwd())
from util.log import *
from util.sql import *
from util.dnodes import *
import multiprocessing as mp
import taos
......@@ -25,7 +26,6 @@ class TwoClients:
self.host = "127.0.0.1"
self.user = "root"
self.password = "taosdata"
self.config = "/home/xp/git/TDengine/sim/dnode1/cfg"
def run(self):
tdDnodes.init("")
......@@ -37,7 +37,7 @@ class TwoClients:
tdDnodes.start(1)
# first client create a stable and insert data
conn1 = taos.connect(self.host, self.user, self.password, self.config)
conn1 = taos.connect(host=self.host, user=self.user, password=self.password, config=tdDnodes.getSimCfgPath())
cursor1 = conn1.cursor()
cursor1.execute("drop database if exists db")
cursor1.execute("create database db")
......@@ -46,7 +46,7 @@ class TwoClients:
cursor1.execute("insert into t0 using tb tags('beijing') values(now, 1)")
# second client alter the table created by cleint
conn2 = taos.connect(self.host, self.user, self.password, self.config)
conn2 = taos.connect(host=self.host, user=self.user, password=self.password, config=tdDnodes.getSimCfgPath())
cursor2 = conn2.cursor()
cursor2.execute("use db")
cursor2.execute("alter table tb add column name nchar(30)")
......
......@@ -198,7 +198,7 @@ python3 ./test.py -f update/merge_commit_data2.py
python3 ./test.py -f update/merge_commit_data2_update0.py
python3 ./test.py -f update/merge_commit_last-0.py
python3 ./test.py -f update/merge_commit_last.py
python3 ./test.py -f update/bug_td2279.py
python3 ./test.py -f update/update_options.py
#======================p2-end===============
#======================p3-start===============
......
......@@ -49,7 +49,7 @@ class TDTestCase:
code = self._conn.schemaless_insert(payload, 2, None)
print("schemaless_insert result {}".format(code))
tdSql.query("describe _stb_0_")
tdSql.query("describe `.stb.0.`")
tdSql.checkRows(6)
### metric value ###
......
......@@ -36,7 +36,7 @@ class TDTestCase:
"stb0_0 1626006833639000000ns 4i8 host=\"host0\" interface=\"eth0\"",
"stb0_1 1626006833639000000ns 4i8 host=\"host0\" interface=\"eth0\"",
"stb0_2 1626006833639000000ns 4i8 host=\"host0\" interface=\"eth0\"",
".stb0.3. 1626006833639000000ns 4i8 host=\"host0\" interface=\"eth0\"",
"`.stb0.3.` 1626006833639000000ns 4i8 host=\"host0\" interface=\"eth0\"",
]
code = self._conn.schemaless_insert(lines0, 1, None)
......@@ -54,7 +54,7 @@ class TDTestCase:
tdSql.query("describe stb0_2")
tdSql.checkRows(4)
tdSql.query("describe _stb0_3_")
tdSql.query("describe `.stb0.3.`")
tdSql.checkRows(4)
### timestamp ###
......
......@@ -46,17 +46,17 @@ class TDTestCase:
## query where tbname in single
tdSql.query(f'select * from {table_name} where tbname in ("{table_name_sub}1")')
tdSql.checkRows(1)
tdSql.checkRows(0)
tdSql.query(f'select * from {table_name} where tbname in ("{table_name_sub.upper()}1")')
tdSql.checkRows(1)
tdSql.checkRows(0)
tdSql.query(f'select * from {table_name} where tbname in ("{table_name_sub.lower()}1")')
tdSql.checkRows(1)
tdSql.query(f'select * from {table_name} where tbname in ("{tb_name_lower}2")')
tdSql.checkRows(2)
tdSql.query(f'select * from {table_name} where tbname in ("{tb_name_lower.upper()}2")')
tdSql.checkRows(2)
tdSql.checkRows(0)
tdSql.query(f'select * from {table_name} where tbname in ("{tb_name_upper}3")')
tdSql.checkRows(3)
tdSql.checkRows(0)
tdSql.query(f'select * from {table_name} where tbname in ("{tb_name_upper.lower()}3")')
tdSql.checkRows(3)
......@@ -64,7 +64,7 @@ class TDTestCase:
tdSql.query(f'select * from {table_name} where id=5 and tbname in ("{table_name_sub}1", "{tb_name_lower.upper()}2", "{tb_name_upper.lower()}3")')
tdSql.checkRows(1)
tdSql.query(f'select * from {table_name} where tbname in ("{table_name_sub}1", "{tb_name_lower.upper()}2", "{tb_name_upper.lower()}3")')
tdSql.checkRows(6)
tdSql.checkRows(3)
def run(self):
tdSql.prepare()
......@@ -75,4 +75,4 @@ class TDTestCase:
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
此差异已折叠。
......@@ -34,6 +34,7 @@ class TDTestCase:
def run(self):
tdSql.prepare()
# test case for TD-2279
print("==============step1")
tdSql.execute("create table t (ts timestamp, a int)")
......@@ -58,6 +59,24 @@ class TDTestCase:
tdSql.checkRows(6612)
tdDnodes.stop(1)
tdDnodes.startWithoutSleep(1)
tdLog.sleep(3)
# test case for https://jira.taosdata.com:18080/browse/TS-402
tdSql.execute("create database test update 1")
tdSql.execute("use test")
tdSql.execute("create table tb (ts timestamp, c1 int, c2 int, c3 int)")
tdSql.execute("insert into tb values(%d, 1, 2, 3)(%d, null, null, 9)" % (self.ts, self.ts))
tdSql.query("select * from tb")
tdSql.checkRows(1)
tdSql.checkData(0, 1, None)
tdSql.checkData(0, 2, None)
tdSql.checkData(0, 3, 9)
def stop(self):
tdSql.close()
......
......@@ -227,3 +227,4 @@ run general/db/show_create_db.sim
run general/db/show_create_table.sim
run general/parser/like.sim
run general/parser/regex.sim
run general/parser/tbname_escape.sim
......@@ -101,9 +101,9 @@ if $data11 != 2 then
return -1
endi
## tbname in can accpet Upper case table name
## no support tbname in Upper case
sql select count(*) from $stb where tbname in ('ti_tb0', 'TI_tb1', 'TI_TB2') group by t1 order by t1
if $rows != 3 then
if $rows != 1 then
return -1
endi
if $data00 != 10 then
......@@ -112,18 +112,6 @@ endi
if $data01 != 0 then
return -1
endi
if $data10 != 10 then
return -1
endi
if $data11 != 1 then
return -1
endi
if $data20 != 10 then
return -1
endi
if $data21 != 2 then
return -1
endi
sql select count(*) from $stb where tbname in ('ti_tb1', 'ti_tb300') and tbname in ('ti_tb5', 'ti_tb1000') group by t1 order by t1 asc
if $rows != 0 then
......
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册