Skip to content

TDengine
TDengine

taosdata / TDengine
接近 2 年 前同步成功

通知 1191
Star 22018
Fork 4786
TDengine
TDengine
未验证 提交 adf429f8 编写于 作者: G Ganlin Zhao 提交者： GitHub
浏览文件
操作

Merge branch 'develop' into fix/TD-12945

上级 cfe9962c fa94faeb
展开全部 隐藏空白更改
内联 并排
Showing with 9406 addition and 1240 deletion
Jenkinsfile
浏览文件 @ adf429f8
......@@ -381,7 +381,7 @@ pipeline {
println gitlog
if (!(gitlog =~ /\((.*?)\)/)){
autoCancelled = true
error('Aborting the build.')
error('Please fill in the scope information correctly.\neg. [TD-xxxx]<fix>(query,insert):xxxxxxxxxxxxxxxxxx ')
}
temp = (gitlog =~ /\((.*?)\)/)
temp = temp[0].remove(1)
......
cmake/define.inc
浏览文件 @ adf429f8
......@@ -130,10 +130,10 @@ IF (TD_MIPS_32)
ENDIF ()
IF (TD_ALPINE)
SET(COMMON_FLAGS "${COMMON_FLAGS} -largp")
link_libraries(/usr/lib/libargp.a)
SET(COMMON_FLAGS "${COMMON_FLAGS} -Wl,-z,stack-size=2097152")
link_libraries(argp)
ADD_DEFINITIONS(-D_ALPINE)
MESSAGE(STATUS "aplhine is defined")
MESSAGE(STATUS "alpine is defined")
ENDIF ()
IF ("${BUILD_HTTP}" STREQUAL "")
......
cmake/install.inc
浏览文件 @ adf429f8
......@@ -46,7 +46,7 @@ ELSEIF (TD_WINDOWS)
#INSTALL(TARGETS taos RUNTIME DESTINATION driver)
#INSTALL(TARGETS shell RUNTIME DESTINATION .)
IF (TD_MVN_INSTALLED)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.36-dist.jar DESTINATION connector/jdbc)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.37-dist.jar DESTINATION connector/jdbc)
ENDIF ()
ELSEIF (TD_DARWIN)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
......
documentation20/cn/00.index/docs.md
浏览文件 @ adf429f8
......@@ -83,10 +83,11 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台，专
* [Windows客户端](https://www.taosdata.com/blog/2019/07/26/514.html)：自行编译windows客户端，Windows环境的各种连接器都需要它
* [Rust Connector](/connector/rust): Rust语言下通过libtaos客户端或RESTful接口，连接TDengine服务器。
## [TDengine 组件与工具](/cn/documentation/)
## TDengine 组件与工具
* [taosAdapter 用户手册](/tools/adapter)
* [TDinsight 用户手册](/tools/insight)
* [taoTools 用户手册](/tools/taos-tools)
## [与其他工具的连接](/connections)
......
documentation20/cn/05.insert/docs.md
浏览文件 @ adf429f8
......@@ -20,7 +20,7 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6,
详细的SQL INSERT语法规则请见 [TAOS SQL 的数据写入](https://www.taosdata.com/cn/documentation/taos-sql#insert) 章节。
**Tips:**
**Tips:**
- 要提高写入效率，需要批量写入。一批写入的记录条数越多，插入效率就越高。但一条记录不能超过16K，一条SQL语句总长度不能超过1M 。
- TDengine支持多线程同时写入，要进一步提高写入速度，一个客户端需要打开20个以上的线程同时写。但线程数达到一定数量后，无法再提高，甚至还会下降，因为线程频繁切换，带来额外开销。
......@@ -56,7 +56,7 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型，并不需要
* 对空格、等号（=）、逗号（,）、双引号（"），前面需要使用反斜杠（\）进行转义。（都指的是英文半角符号）
* 数值类型将通过后缀来区分数据类型：
| **序号** | **后缀** | **映射类型** | **大小(字节)** |
| **序号** | **后缀** | **映射类型** | **大小(字节)** |
| -- | ------- | ---------| ------ |
| 1 | 无或f64 | double | 8 |
| 2 | f32 | float | 4 |
......@@ -231,16 +231,16 @@ prometheus产生的数据格式如下：
```json
{
Timestamp: 1576466279341,
Value: 37.000000,
Value: 37.000000,
apiserver_request_latencies_bucket {
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes",
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes",
scope="cluster",
verb="LIST",
version="v1"
verb="LIST",
version="v1"
}
}
```
......@@ -251,6 +251,7 @@ select * from apiserver_request_latencies_bucket;
```
## <a class="anchor" id="telegraf"></a> Telegraf 直接写入(通过 taosAdapter)
安装 Telegraf 请参考[官方文档](https://portal.influxdata.com/downloads/)。
TDengine 新版本（2.3.0.0+）包含一个 taosAdapter 独立程序，负责接收包括 Telegraf 的多种应用的数据写入。
......@@ -276,6 +277,7 @@ sudo systemctl start telegraf
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter)
安装 collectd，请参考[官方文档](https://collectd.org/download.shtml)。
TDengine 新版本（2.3.0.0+）包含一个 taosAdapter 独立程序，负责接收包括 collectd 的多种应用的数据写入。
......@@ -294,6 +296,7 @@ sudo systemctl start collectd
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter)
安装 StatsD
请参考[官方文档](https://github.com/statsd/statsd)。
......@@ -316,6 +319,30 @@ port: 8125
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
icinga2 可以收集监控和性能数据并写入 OpenTSDB，taosAdapter 可以支持接收 icinga2 的数据并写入到 TDengine 中。
## <a class="anchor" id="icinga2"></a> icinga2 直接写入(通过 taosAdapter)
* 参考链接 https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer 使能 opentsdb-writer
* 使能 taosAdapter 配置项 opentsdb_telnet.enable
* 修改配置文件 /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="tcollector"></a> TCollector 直接写入(通过 taosAdapter)
TCollector 是一个在客户侧收集本地收集器并发送数据到 OpenTSDB 的进程，taosAdaapter 可以支持接收 TCollector 的数据并写入到 TDengine 中。
使能 taosAdapter 配置项 opentsdb_telnet.enable
修改 TCollector 配置文件，修改 OpenTSDB 宿主机地址为 taosAdapter 被部署的地址，并修改端口号为 taosAdapter 使用的端口（默认6049）。
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="taosadapter2-telegraf"></a> 使用 Bailongma 2.0 接入 Telegraf 数据写入
......
documentation20/cn/12.taos-sql/docs.md
浏览文件 @ adf429f8
......@@ -1669,7 +1669,7 @@ SELECT COUNT(*), FIRST(ts), status FROM temp_tb_1 STATE_WINDOW(status)
```mysql
SELECT COUNT(*), FIRST(ts) FROM temp_tb_1 SESSION_WINDOW(ts, tol_val)
SELECT COUNT(*), FIRST(ts) FROM temp_tb_1 SESSION(ts, tol_val)
```
这种类型的查询语法如下：
......@@ -1851,3 +1851,24 @@ TDengine 中的表（列）名命名规则如下：
```mysql
select jtag->'key' from (select jtag from stable) where jtag->'key'>0
```
## 转义字符说明
- 转义字符表
| 字符序列 | **代表的字符** |
| :--------: | ------- |
| `\'` | 单引号' |
| `\"` | 双引号" |
| \n | 换行符 |
| \r | 回车符 |
| \t | tab符 |
| `\\` | 斜杠\ |
| `\%` | % 规则见下 |
| `\%` | _ 规则见下 |
- 转义字符使用规则
1. 标识符里有转义字符（数据库名、表名、列名）
1. 普通标识符： 直接提示错误的标识符，因为标识符规定必须是数字、字母和下划线，并且不能以数字开头。
2. 反引号``标识符： 保持原样，不转义
2. 数据里有转义字符
1. 遇到上面定义的转义字符会转义（%和_见下面说明），如果没有匹配的转义字符会忽略掉转义符\。
2. 对于%和_，因为在like里这两个字符是通配符，所以在模式匹配like里用`\%`%和`\_`表示字符里本身的%和_，如果在like模式匹配上下文之外使用`\%`或`\_`，则它们的计算结果为字符串`\%`和`\_`，而不是%和_。
\ No newline at end of file
documentation20/en/00.index/docs.md
浏览文件 @ adf429f8
......@@ -83,6 +83,7 @@ TDengine is a highly efficient platform to store, query, and analyze time-series
* [taosAdapter User Manual](/tools/adapter)
* [TDinsight User Manual](/tools/insight)
* [taos-tools User Manual](/tools/taos-tools)
## [Connections with Other Tools](/connections)
......
documentation20/en/02.getting-started/01.docker/docs.md
浏览文件 @ adf429f8
......@@ -8,14 +8,14 @@ The following article explains how to quickly build a single-node TDengine runti
The Docker tools themselves can be downloaded from [Docker official site](https://docs.docker.com/get-docker/).
After installation, you can check the Docker version in the command line terminal. If the version number is output properly, the Docker environment has been installed successfully.
After installation, you can check the Docker version in the command-line terminal. If the version number is output properly, the Docker environment has been installed successfully.
```bash
$ docker -v
Docker version 20.10.3, build 48d30b5
```
## Using Docker to run TDengine
## How to use Docker to run TDengine
### running TDengine server inside Docker
......@@ -215,7 +215,7 @@ column[0]:FLOAT column[1]:INT column[2]:FLOAT
Press enter key to continue or Ctrl-C to stop
```
After enter, this command will automatically create a super table meters under the database test, there are 10,000 tables under this super table, the table name is "d0" to "d9999", each table has 10,000 records, each record has four fields (ts, current, voltage, phase), the time stamp is from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999", each table has a tag location and groupId, groupId is set from 1 to 10 and location is set to "beijing" or "shanghai".
After enter, this command will automatically create a super table `meters` under the database test, there are 10,000 tables under this super table, the table name is "d0" to "d9999", each table has 10,000 records, each record has four fields (ts, current, voltage, phase), the time stamp is from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999", each table has a tag location and groupId, groupId is set from 1 to 10 and location is set to "beijing" or "shanghai".
It takes about a few minutes to execute this command and ends up inserting a total of 100 million records.
......
documentation20/en/05.insert/docs.md
浏览文件 @ adf429f8
......@@ -2,7 +2,7 @@
TDengine supports multiple ways to write data, including SQL, Prometheus, Telegraf, collectd, StatsD, EMQ MQTT Broker, HiveMQ Broker, CSV file, etc. Kafka, OPC and other interfaces will be provided in the future. Data can be inserted in one single record or in batches, data from one or multiple data collection points can be inserted at the same time. TDengine supports multi-thread insertion, out-of-order data insertion, and also historical data insertion.
## <a class="anchor" id="sql"></a> Data Writing via SQL
## <a class="anchor" id="sql"></a> Data Writing via SQL
Applications insert data by executing SQL insert statements through C/C++, Java, Go, C#, Python, Node.js Connectors, and users can manually enter SQL insert statements to insert data through TAOS Shell. For example, the following insert writes a record to table d1001:
......@@ -119,16 +119,16 @@ The format of generated data by Prometheus is as follows:
```json
{
Timestamp: 1576466279341,
Value: 37.000000,
Value: 37.000000,
apiserver_request_latencies_bucket {
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes", s
cope="cluster",
verb="LIST",
version=“v1"
verb="LIST",
version=“v1"
}
}
```
......@@ -167,13 +167,13 @@ Now you can query the metrics data of Telegraf from TDengine.
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter)
## <a class="anchor" id="collectd"></a> Data Writing via collectd and taosAdapter
Please refer to [official document](https://collectd.org/download.shtml) for collectd installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from collectd.
Configuration:
Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using.
Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using.
```
LoadPlugin network
<Plugin network>
......@@ -186,12 +186,12 @@ sudo systemctl start collectd
```
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter)
## <a class="anchor" id="statsd"></a> Data Writting via StatsD and taosAdapter
Please refer to [official document](https://github.com/statsd/statsd) for StatsD installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from StatsD.
Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using.
Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using.
```
add "./backends/repeater" to backends section.
add { host:'<TDengine server/cluster host>', port: <port for StatsD>} to repeater section.
......@@ -206,8 +206,30 @@ port: 8125
}
```
## <a class="anchor" id="cinga2"></a> Data Writting via icinga2 and taosAdapter
Use icinga2 to collect check result metrics and performance data
* Follow the doc to enable opentsdb-writer https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the configuration file /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="tcollector"></a> Data Writting via TCollector and taosAdapter
TCollector is a client-side process that gathers data from local collectors and pushes the data to OpenTSDB. You run it on all your hosts, and it does the work of sending each host’s data to the TSD (OpenTSDB backend process).
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the TCollector configuration file, modify the OpenTSDB host to the host where taosAdapter is deployed, and modify the port to 6049
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="taosadapter2-telegraf"></a> Insert data via Bailongma 2.0 and Telegraf
......
documentation20/en/12.taos-sql/docs.md
浏览文件 @ adf429f8
......@@ -1335,3 +1335,24 @@ Is not null supports all types of columns. Non-null expression is < > "" and onl
select jtag->'key' from (select jtag from stable) where jtag->'key'>0
```
## Escape character description
- Special Character Escape Sequences
| Escape Sequence | **Character Represented by Sequence** |
| :--------: | ------------------- |
| `\'` | A single quote (') character |
| `\"` | A double quote (") character |
| \n | A newline (linefeed) character |
| \r | A carriage return character |
| \t | A tab character |
| `\\` | A backslash (\) character |
| `\%` | A % character; see note following the table |
| `\_` | A _ character; see note following the table |
- Escape character usage rules
- The escape characters that in a identifier (database name, table name, column name)
1. Normal identifier： The wrong identifier is prompted directly, because the identifier must be numbers, letters and underscores, and cannot start with a number.
2. Backquote`` identifier： Keep it as it is.
- The escape characters that in a data
3. The escape character defined above will be escaped (% and _ see the description below). If there is no matching escape character, the escape character will be ignored.
4. The `\%` and `\_` sequences are used to search for literal instances of % and _ in pattern-matching contexts where they would otherwise be interpreted as wildcard characters.If you use `\%` or `\_` outside of pattern-matching contexts, they evaluate to the strings `\%` and `\_`, not to % and _.
\ No newline at end of file
packaging/deb/makedeb.sh
浏览文件 @ adf429f8
......@@ -36,11 +36,11 @@ install_home_path="/usr/local/taos"
mkdir -p ${pkg_dir}${install_home_path}
mkdir -p ${pkg_dir}${install_home_path}/bin
mkdir -p ${pkg_dir}${install_home_path}/cfg
mkdir -p ${pkg_dir}${install_home_path}/connector
#mkdir -p ${pkg_dir}${install_home_path}/connector
mkdir -p ${pkg_dir}${install_home_path}/driver
mkdir -p ${pkg_dir}${install_home_path}/examples
mkdir -p ${pkg_dir}${install_home_path}/include
mkdir -p ${pkg_dir}${install_home_path}/init.d
#mkdir -p ${pkg_dir}${install_home_path}/init.d
mkdir -p ${pkg_dir}${install_home_path}/script
cp ${compile_dir}/../packaging/cfg/taos.cfg ${pkg_dir}${install_home_path}/cfg
......@@ -51,7 +51,7 @@ if [ -f "${compile_dir}/test/cfg/taosadapter.service" ]; then
cp ${compile_dir}/test/cfg/taosadapter.service ${pkg_dir}${install_home_path}/cfg || :
fi
cp ${compile_dir}/../packaging/deb/taosd ${pkg_dir}${install_home_path}/init.d
#cp ${compile_dir}/../packaging/deb/taosd ${pkg_dir}${install_home_path}/init.d
cp ${compile_dir}/../packaging/tools/post.sh ${pkg_dir}${install_home_path}/script
cp ${compile_dir}/../packaging/tools/preun.sh ${pkg_dir}${install_home_path}/script
cp ${compile_dir}/../packaging/tools/startPre.sh ${pkg_dir}${install_home_path}/bin
......@@ -70,10 +70,10 @@ cp ${compile_dir}/../src/inc/taos.h ${pkg_dir}${install_home_pat
cp ${compile_dir}/../src/inc/taosdef.h ${pkg_dir}${install_home_path}/include
cp ${compile_dir}/../src/inc/taoserror.h ${pkg_dir}${install_home_path}/include
cp -r ${top_dir}/tests/examples/* ${pkg_dir}${install_home_path}/examples
cp -r ${top_dir}/src/connector/python ${pkg_dir}${install_home_path}/connector
cp -r ${top_dir}/src/connector/go ${pkg_dir}${install_home_path}/connector
cp -r ${top_dir}/src/connector/nodejs ${pkg_dir}${install_home_path}/connector
cp ${compile_dir}/build/lib/taos-jdbcdriver*.* ${pkg_dir}${install_home_path}/connector ||:
#cp -r ${top_dir}/src/connector/python ${pkg_dir}${install_home_path}/connector
#cp -r ${top_dir}/src/connector/go ${pkg_dir}${install_home_path}/connector
#cp -r ${top_dir}/src/connector/nodejs ${pkg_dir}${install_home_path}/connector
#cp ${compile_dir}/build/lib/taos-jdbcdriver*.* ${pkg_dir}${install_home_path}/connector ||:
install_user_local_path="/usr/local"
......
packaging/rpm/tdengine.spec
浏览文件 @ adf429f8
......@@ -46,11 +46,11 @@ libfile="libtaos.so.%{_version}"
# create install path, and cp file
mkdir -p %{buildroot}%{homepath}/bin
mkdir -p %{buildroot}%{homepath}/cfg
mkdir -p %{buildroot}%{homepath}/connector
#mkdir -p %{buildroot}%{homepath}/connector
mkdir -p %{buildroot}%{homepath}/driver
mkdir -p %{buildroot}%{homepath}/examples
mkdir -p %{buildroot}%{homepath}/include
mkdir -p %{buildroot}%{homepath}/init.d
#mkdir -p %{buildroot}%{homepath}/init.d
mkdir -p %{buildroot}%{homepath}/script
cp %{_compiledir}/../packaging/cfg/taos.cfg %{buildroot}%{homepath}/cfg
......@@ -60,7 +60,7 @@ fi
if [ -f %{_compiledir}/test/cfg/taosadapter.service ]; then
cp %{_compiledir}/test/cfg/taosadapter.service %{buildroot}%{homepath}/cfg
fi
cp %{_compiledir}/../packaging/rpm/taosd %{buildroot}%{homepath}/init.d
#cp %{_compiledir}/../packaging/rpm/taosd %{buildroot}%{homepath}/init.d
cp %{_compiledir}/../packaging/tools/post.sh %{buildroot}%{homepath}/script
cp %{_compiledir}/../packaging/tools/preun.sh %{buildroot}%{homepath}/script
cp %{_compiledir}/../packaging/tools/startPre.sh %{buildroot}%{homepath}/bin
......@@ -75,10 +75,10 @@ cp %{_compiledir}/build/lib/${libfile} %{buildroot}%{homepath}/driv
cp %{_compiledir}/../src/inc/taos.h %{buildroot}%{homepath}/include
cp %{_compiledir}/../src/inc/taosdef.h %{buildroot}%{homepath}/include
cp %{_compiledir}/../src/inc/taoserror.h %{buildroot}%{homepath}/include
cp -r %{_compiledir}/../src/connector/python %{buildroot}%{homepath}/connector
cp -r %{_compiledir}/../src/connector/go %{buildroot}%{homepath}/connector
cp -r %{_compiledir}/../src/connector/nodejs %{buildroot}%{homepath}/connector
cp %{_compiledir}/build/lib/taos-jdbcdriver*.* %{buildroot}%{homepath}/connector ||:
#cp -r %{_compiledir}/../src/connector/python %{buildroot}%{homepath}/connector
#cp -r %{_compiledir}/../src/connector/go %{buildroot}%{homepath}/connector
#cp -r %{_compiledir}/../src/connector/nodejs %{buildroot}%{homepath}/connector
#cp %{_compiledir}/build/lib/taos-jdbcdriver*.* %{buildroot}%{homepath}/connector ||:
cp -r %{_compiledir}/../tests/examples/* %{buildroot}%{homepath}/examples
if [ -f %{_compiledir}/build/bin/jemalloc-config ]; then
......
packaging/tools/install.sh
浏览文件 @ adf429f8
......@@ -167,11 +167,11 @@ function install_main_path() {
${csudo}mkdir -p ${install_main_dir}
${csudo}mkdir -p ${install_main_dir}/cfg
${csudo}mkdir -p ${install_main_dir}/bin
${csudo}mkdir -p ${install_main_dir}/connector
# ${csudo}mkdir -p ${install_main_dir}/connector
${csudo}mkdir -p ${install_main_dir}/driver
${csudo}mkdir -p ${install_main_dir}/examples
${csudo}mkdir -p ${install_main_dir}/include
${csudo}mkdir -p ${install_main_dir}/init.d
# ${csudo}mkdir -p ${install_main_dir}/init.d
if [ "$verMode" == "cluster" ]; then
${csudo}mkdir -p ${nginx_dir}
fi
......@@ -199,7 +199,7 @@ function install_bin() {
[ -x ${install_main_dir}/bin/taos ] && ${csudo}ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || :
[ -x ${install_main_dir}/bin/taosd ] && ${csudo}ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || :
[ -x ${install_main_dir}/bin/taosadapter ] && ${csudo}ln -s ${install_main_dir}/bin/taosadapter ${bin_link_dir}/taosadapter || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo}ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/taosBenchmark ] && ${csudo}ln -s ${install_main_dir}/bin/taosBenchmark ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo}ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo}ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || :
[ -x ${install_main_dir}/bin/set_core.sh ] && ${csudo}ln -s ${install_main_dir}/bin/set_core.sh ${bin_link_dir}/set_core || :
......@@ -639,14 +639,14 @@ function install_service_on_sysvinit() {
# Install taosd service
if ((${os_type}==1)); then
${csudo}cp -f ${script_dir}/init.d/taosd.deb ${install_main_dir}/init.d/taosd
# ${csudo}cp -f ${script_dir}/init.d/taosd.deb ${install_main_dir}/init.d/taosd
${csudo}cp ${script_dir}/init.d/taosd.deb ${service_config_dir}/taosd && ${csudo}chmod a+x ${service_config_dir}/taosd
${csudo}cp -f ${script_dir}/init.d/tarbitratord.deb ${install_main_dir}/init.d/tarbitratord
# ${csudo}cp -f ${script_dir}/init.d/tarbitratord.deb ${install_main_dir}/init.d/tarbitratord
${csudo}cp ${script_dir}/init.d/tarbitratord.deb ${service_config_dir}/tarbitratord && ${csudo}chmod a+x ${service_config_dir}/tarbitratord
elif ((${os_type}==2)); then
${csudo}cp -f ${script_dir}/init.d/taosd.rpm ${install_main_dir}/init.d/taosd
# ${csudo}cp -f ${script_dir}/init.d/taosd.rpm ${install_main_dir}/init.d/taosd
${csudo}cp ${script_dir}/init.d/taosd.rpm ${service_config_dir}/taosd && ${csudo}chmod a+x ${service_config_dir}/taosd
${csudo}cp -f ${script_dir}/init.d/tarbitratord.rpm ${install_main_dir}/init.d/tarbitratord
# ${csudo}cp -f ${script_dir}/init.d/tarbitratord.rpm ${install_main_dir}/init.d/tarbitratord
${csudo}cp ${script_dir}/init.d/tarbitratord.rpm ${service_config_dir}/tarbitratord && ${csudo}chmod a+x ${service_config_dir}/tarbitratord
fi
......@@ -706,7 +706,7 @@ function install_service_on_systemd() {
${csudo}cp ${script_dir}/cfg/taosd.service \
${service_config_dir}/ || :
${csudo}systemctl daemon-reload
#taosd_service_config="${service_config_dir}/taosd.service"
#${csudo}bash -c "echo '[Unit]' >> ${taosd_service_config}"
#${csudo}bash -c "echo 'Description=TDengine server service' >> ${taosd_service_config}"
......@@ -736,7 +736,7 @@ function install_service_on_systemd() {
${csudo}cp ${script_dir}/cfg/tarbitratord.service \
${service_config_dir}/ || :
${csudo}systemctl daemon-reload
#tarbitratord_service_config="${service_config_dir}/tarbitratord.service"
#${csudo}bash -c "echo '[Unit]' >> ${tarbitratord_service_config}"
#${csudo}bash -c "echo 'Description=TDengine arbitrator service' >> ${tarbitratord_service_config}"
......@@ -923,15 +923,14 @@ function update_TDengine() {
install_log
install_header
install_lib
if [ "$pagMode" != "lite" ]; then
install_connector
fi
# if [ "$pagMode" != "lite" ]; then
# install_connector
# fi
install_examples
if [ -z $1 ]; then
install_bin
install_service
install_taosadapter_service
install_config
install_taosadapter_config
openresty_work=false
......@@ -1008,9 +1007,9 @@ function install_TDengine() {
#install_avro lib
#install_avro lib64
if [ "$pagMode" != "lite" ]; then
install_connector
fi
# if [ "$pagMode" != "lite" ]; then
# install_connector
# fi
install_examples
if [ -z $1 ]; then # install service and client
......@@ -1018,6 +1017,7 @@ function install_TDengine() {
install_bin
install_service
install_taosadapter_service
install_taosadapter_config
openresty_work=false
if [ "$verMode" == "cluster" ]; then
......
packaging/tools/make_install.sh
浏览文件 @ adf429f8
......@@ -137,17 +137,17 @@ function install_main_path() {
${csudo}mkdir -p ${install_main_dir}
${csudo}mkdir -p ${install_main_dir}/cfg
${csudo}mkdir -p ${install_main_dir}/bin
${csudo}mkdir -p ${install_main_dir}/connector
# ${csudo}mkdir -p ${install_main_dir}/connector
${csudo}mkdir -p ${install_main_dir}/driver
${csudo}mkdir -p ${install_main_dir}/examples
${csudo}mkdir -p ${install_main_dir}/include
${csudo}mkdir -p ${install_main_dir}/init.d
# ${csudo}mkdir -p ${install_main_dir}/init.d
else
${csudo}rm -rf ${install_main_dir} || ${csudo}rm -rf ${install_main_2_dir} || :
${csudo}mkdir -p ${install_main_dir} || ${csudo}mkdir -p ${install_main_2_dir}
${csudo}mkdir -p ${install_main_dir}/cfg || ${csudo}mkdir -p ${install_main_2_dir}/cfg
${csudo}mkdir -p ${install_main_dir}/bin || ${csudo}mkdir -p ${install_main_2_dir}/bin
${csudo}mkdir -p ${install_main_dir}/connector || ${csudo}mkdir -p ${install_main_2_dir}/connector
# ${csudo}mkdir -p ${install_main_dir}/connector || ${csudo}mkdir -p ${install_main_2_dir}/connector
${csudo}mkdir -p ${install_main_dir}/driver || ${csudo}mkdir -p ${install_main_2_dir}/driver
${csudo}mkdir -p ${install_main_dir}/examples || ${csudo}mkdir -p ${install_main_2_dir}/examples
${csudo}mkdir -p ${install_main_dir}/include || ${csudo}mkdir -p ${install_main_2_dir}/include
......@@ -168,9 +168,15 @@ function install_bin() {
${csudo}rm -f ${bin_link_dir}/run_taosd.sh || :
${csudo}rm -f ${bin_link_dir}/rmtaos || :
${csudo}cp -r ${binary_dir}/build/bin/* ${install_main_dir}/bin
${csudo}cp -r ${script_dir}/taosd-dump-cfg.gdb ${install_main_dir}/bin
${csudo}cp -r ${binary_dir}/build/bin/taos ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosBenchmark ] && ${csudo}cp -r ${binary_dir}/build/bin/taosBenchmark ${install_main_dir}/bin || :
[ -f ${install_main_dir}/bin/taosBenchmark ] && ${csudo}ln -sf ${install_main_dir}/bin/taosBenchmark ${install_main_dir}/bin/taosdemo || :
[ -f ${binary_dir}/build/bin/taosdump ] && ${csudo}cp -r ${binary_dir}/build/bin/taosdump ${install_main_dir}/bin || :
[ -f ${binary_dir}/build/bin/taosadapter ] && ${csudo}cp -r ${binary_dir}/build/bin/taosadapter ${install_main_dir}/bin || :
${csudo}cp -r ${binary_dir}/build/bin/taosd ${install_main_dir}/bin || :
${csudo}cp -r ${binary_dir}/build/bin/tarbitrator ${install_main_dir}/bin || :
${csudo}cp -r ${script_dir}/taosd-dump-cfg.gdb ${install_main_dir}/bin
${csudo}cp -r ${script_dir}/remove.sh ${install_main_dir}/bin
${csudo}cp -r ${script_dir}/set_core.sh ${install_main_dir}/bin
${csudo}cp -r ${script_dir}/run_taosd.sh ${install_main_dir}/bin
......@@ -458,10 +464,10 @@ function install_service_on_sysvinit() {
# Install taosd service
if ((${os_type}==1)); then
${csudo}cp -f ${script_dir}/../deb/taosd ${install_main_dir}/init.d
# ${csudo}cp -f ${script_dir}/../deb/taosd ${install_main_dir}/init.d
${csudo}cp ${script_dir}/../deb/taosd ${service_config_dir} && ${csudo}chmod a+x ${service_config_dir}/taosd
elif ((${os_type}==2)); then
${csudo}cp -f ${script_dir}/../rpm/taosd ${install_main_dir}/init.d
# ${csudo}cp -f ${script_dir}/../rpm/taosd ${install_main_dir}/init.d
${csudo}cp ${script_dir}/../rpm/taosd ${service_config_dir} && ${csudo}chmod a+x ${service_config_dir}/taosd
fi
......@@ -563,7 +569,7 @@ function update_TDengine() {
install_log
install_header
install_lib
install_connector
# install_connector
install_examples
install_bin
......@@ -603,7 +609,7 @@ function install_TDengine() {
install_log
install_header
install_lib
install_connector
# install_connector
install_examples
install_bin
......
packaging/tools/makepkg.sh
浏览文件 @ adf429f8
......@@ -123,9 +123,9 @@ if [ -n "${taostools_bin_files}" ]; then
mkdir -p ${taostools_install_dir}/bin \
&& cp ${taostools_bin_files} ${taostools_install_dir}/bin \
&& chmod a+x ${taostools_install_dir}/bin/* || :
[ -f ${taostools_install_dir}/bin/taosBenchmark ] && \
ln -sf ${taostools_install_dir}/bin/taosBenchmark \
${taostools_install_dir}/bin/taosdemo
# [ -f ${taostools_install_dir}/bin/taosBenchmark ] && \
# ln -sf ${taostools_install_dir}/bin/taosBenchmark \
# ${taostools_install_dir}/bin/taosdemo
if [ -f ${top_dir}/src/kit/taos-tools/packaging/tools/install-taostools.sh ]; then
cp ${top_dir}/src/kit/taos-tools/packaging/tools/install-taostools.sh \
......@@ -135,6 +135,15 @@ if [ -n "${taostools_bin_files}" ]; then
else
echo -e "install-taostools.sh not found"
fi
if [ -f ${top_dir}/src/kit/taos-tools/packaging/tools/uninstall-taostools.sh ]; then
cp ${top_dir}/src/kit/taos-tools/packaging/tools/uninstall-taostools.sh \
${taostools_install_dir}/ > /dev/null \
&& chmod a+x ${taostools_install_dir}/uninstall-taostools.sh \
|| echo -e "failed to copy uninstall-taostools.sh"
else
echo -e "uninstall-taostools.sh not found"
fi
if [ -f ${build_dir}/lib/libavro.so.23.0.0 ]; then
mkdir -p ${taostools_install_dir}/avro/{lib,lib/pkgconfig} || echo -e "failed to create ${taostools_install_dir}/avro"
......@@ -248,18 +257,18 @@ fi
mkdir -p ${install_dir}/driver && cp ${lib_files} ${install_dir}/driver && echo "${versionComp}" > ${install_dir}/driver/vercomp.txt
# Copy connector
connector_dir="${code_dir}/connector"
mkdir -p ${install_dir}/connector
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
cp -r ${connector_dir}/go ${install_dir}/connector
else
echo "WARNING: go connector not found, please check if want to use it!"
fi
cp -r ${connector_dir}/python ${install_dir}/connector
cp -r ${connector_dir}/nodejs ${install_dir}/connector
fi
#connector_dir="${code_dir}/connector"
#mkdir -p ${install_dir}/connector
#if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
# cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
# if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
# cp -r ${connector_dir}/go ${install_dir}/connector
# else
# echo "WARNING: go connector not found, please check if want to use it!"
# fi
# cp -r ${connector_dir}/python ${install_dir}/connector
# cp -r ${connector_dir}/nodejs ${install_dir}/connector
#fi
# Copy release note
# cp ${script_dir}/release_note ${install_dir}
......
src/client/inc/tscUtil.h
浏览文件 @ adf429f8
......@@ -207,7 +207,7 @@ TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index);
void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo);
int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index);
int32_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index);
void tscFieldInfoClear(SFieldInfo* pFieldInfo);
void tscFieldInfoCopy(SFieldInfo* pFieldInfo, const SFieldInfo* pSrc, const SArray* pExprList);
......@@ -258,8 +258,6 @@ void tscColumnListCopyAll(SArray* dst, const SArray* src);
void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, bool convertNchar, bool convertJson);
void tscDequoteAndTrimToken(SStrToken* pToken);
void tscRmEscapeAndTrimToken(SStrToken* pToken);
int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded);
void tscIncStreamExecutionCount(void* pStream);
......
src/client/jni/com_taosdata_jdbc_TSDBJNIConnector.h
浏览文件 @ adf429f8
......@@ -209,6 +209,15 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_prepareStmtImp
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameImp
(JNIEnv *, jobject, jlong, jstring, jlong);
/**
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: setTableNameTagsImp
* Signature: (JLjava/lang/String;I[B[B[B[BJ)I
*/
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp
(JNIEnv *, jobject, jlong, jstring, jint, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: bindColDataImp
......@@ -217,6 +226,14 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameI
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp
(JNIEnv *, jobject, jlong, jbyteArray, jbyteArray, jbyteArray, jint, jint, jint, jint, jlong);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: stmt_add_batch
* Signature: (JJ)I
*/
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_addBatchImp(JNIEnv *env, jobject jobj, jlong stmt, jlong con);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: executeBatchImp
......@@ -231,13 +248,12 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(J
*/
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv *env, jobject jobj, jlong stmt, jlong con);
/**
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: setTableNameTagsImp
* Signature: (JLjava/lang/String;I[B[B[B[BJ)I
* Method: stmt_errstr
* Signature: (JJ)I
*/
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp
(JNIEnv *, jobject, jlong, jstring, jint, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong);
JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_stmtErrorMsgImp(JNIEnv *env, jobject jobj, jlong stmt, jlong con);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
......
src/client/src/TSDBJNIConnector.c
浏览文件 @ adf429f8
......@@ -805,6 +805,78 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameI
return JNI_SUCCESS;
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp(
JNIEnv *env, jobject jobj, jlong stmt, jstring tableName, jint numOfTags, jbyteArray tags, jbyteArray typeList,
jbyteArray lengthList, jbyteArray nullList, jlong conn) {
TAOS *tsconn = (TAOS *)conn;
if (tsconn == NULL) {
jniError("jobj:%p, connection already closed", jobj);
return JNI_CONNECTION_NULL;
}
TAOS_STMT *pStmt = (TAOS_STMT *)stmt;
if (pStmt == NULL) {
jniError("jobj:%p, conn:%p, invalid stmt handle", jobj, tsconn);
return JNI_SQL_NULL;
}
jsize len = (*env)->GetArrayLength(env, tags);
char *tagsData = (char *)calloc(1, len);
(*env)->GetByteArrayRegion(env, tags, 0, len, (jbyte *)tagsData);
if ((*env)->ExceptionCheck(env)) {
// todo handle error
}
len = (*env)->GetArrayLength(env, lengthList);
int64_t *lengthArray = (int64_t *)calloc(1, len);
(*env)->GetByteArrayRegion(env, lengthList, 0, len, (jbyte *)lengthArray);
if ((*env)->ExceptionCheck(env)) {
}
len = (*env)->GetArrayLength(env, typeList);
char *typeArray = (char *)calloc(1, len);
(*env)->GetByteArrayRegion(env, typeList, 0, len, (jbyte *)typeArray);
if ((*env)->ExceptionCheck(env)) {
}
len = (*env)->GetArrayLength(env, nullList);
int32_t *nullArray = (int32_t *)calloc(1, len);
(*env)->GetByteArrayRegion(env, nullList, 0, len, (jbyte *)nullArray);
if ((*env)->ExceptionCheck(env)) {
}
const char *name = (*env)->GetStringUTFChars(env, tableName, NULL);
char *curTags = tagsData;
TAOS_BIND *tagsBind = calloc(numOfTags, sizeof(TAOS_BIND));
for (int32_t i = 0; i < numOfTags; ++i) {
tagsBind[i].buffer_type = typeArray[i];
tagsBind[i].buffer = curTags;
tagsBind[i].is_null = &nullArray[i];
tagsBind[i].length = (uintptr_t *)&lengthArray[i];
curTags += lengthArray[i];
}
int32_t code = taos_stmt_set_tbname_tags((void *)stmt, name, tagsBind);
int32_t nTags = (int32_t)numOfTags;
jniDebug("jobj:%p, conn:%p, set table name:%s, numOfTags:%d", jobj, tsconn, name, nTags);
tfree(tagsData);
tfree(lengthArray);
tfree(typeArray);
tfree(nullArray);
tfree(tagsBind);
(*env)->ReleaseStringUTFChars(env, tableName, name);
if (code != TSDB_CODE_SUCCESS) {
jniError("jobj:%p, conn:%p, code:%s", jobj, tsconn, tstrerror(code));
return JNI_TDENGINE_ERROR;
}
return JNI_SUCCESS;
}
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp(
JNIEnv *env, jobject jobj, jlong stmt, jbyteArray colDataList, jbyteArray lengthList, jbyteArray nullList,
jint dataType, jint dataBytes, jint numOfRows, jint colIndex, jlong con) {
......@@ -872,8 +944,8 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp(
return JNI_SUCCESS;
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_addBatchImp(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
TAOS *tscon = (TAOS *)con;
if (tscon == NULL) {
jniError("jobj:%p, connection already closed", jobj);
......@@ -886,19 +958,18 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(J
return JNI_SQL_NULL;
}
taos_stmt_add_batch(pStmt);
int32_t code = taos_stmt_execute(pStmt);
int32_t code = taos_stmt_add_batch(pStmt);
if (code != TSDB_CODE_SUCCESS) {
jniError("jobj:%p, conn:%p, code:%s", jobj, tscon, tstrerror(code));
return JNI_TDENGINE_ERROR;
}
jniDebug("jobj:%p, conn:%p, batch execute", jobj, tscon);
jniDebug("jobj:%p, conn:%p, stmt closed", jobj, tscon);
return JNI_SUCCESS;
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
TAOS *tscon = (TAOS *)con;
if (tscon == NULL) {
jniError("jobj:%p, connection already closed", jobj);
......@@ -911,91 +982,63 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv
return JNI_SQL_NULL;
}
int32_t code = taos_stmt_close(pStmt);
int32_t code = taos_stmt_execute(pStmt);
if (code != TSDB_CODE_SUCCESS) {
jniError("jobj:%p, conn:%p, code:%s", jobj, tscon, tstrerror(code));
return JNI_TDENGINE_ERROR;
}
jniDebug("jobj:%p, conn:%p, stmt closed", jobj, tscon);
jniDebug("jobj:%p, conn:%p, batch execute", jobj, tscon);
return JNI_SUCCESS;
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp(
JNIEnv *env, jobject jobj, jlong stmt, jstring tableName, jint numOfTags, jbyteArray tags, jbyteArray typeList,
jbyteArray lengthList, jbyteArray nullList, jlong conn) {
TAOS *tsconn = (TAOS *)conn;
if (tsconn == NULL) {
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
TAOS *tscon = (TAOS *)con;
if (tscon == NULL) {
jniError("jobj:%p, connection already closed", jobj);
return JNI_CONNECTION_NULL;
}
TAOS_STMT *pStmt = (TAOS_STMT *)stmt;
if (pStmt == NULL) {
jniError("jobj:%p, conn:%p, invalid stmt handle", jobj, tsconn);
jniError("jobj:%p, conn:%p, invalid stmt", jobj, tscon);
return JNI_SQL_NULL;
}
jsize len = (*env)->GetArrayLength(env, tags);
char *tagsData = (char *)calloc(1, len);
(*env)->GetByteArrayRegion(env, tags, 0, len, (jbyte *)tagsData);
if ((*env)->ExceptionCheck(env)) {
// todo handle error
}
len = (*env)->GetArrayLength(env, lengthList);
int64_t *lengthArray = (int64_t *)calloc(1, len);
(*env)->GetByteArrayRegion(env, lengthList, 0, len, (jbyte *)lengthArray);
if ((*env)->ExceptionCheck(env)) {
int32_t code = taos_stmt_close(pStmt);
if (code != TSDB_CODE_SUCCESS) {
jniError("jobj:%p, conn:%p, code:%s", jobj, tscon, tstrerror(code));
return JNI_TDENGINE_ERROR;
}
len = (*env)->GetArrayLength(env, typeList);
char *typeArray = (char *)calloc(1, len);
(*env)->GetByteArrayRegion(env, typeList, 0, len, (jbyte *)typeArray);
if ((*env)->ExceptionCheck(env)) {
}
jniDebug("jobj:%p, conn:%p, stmt closed", jobj, tscon);
return JNI_SUCCESS;
}
len = (*env)->GetArrayLength(env, nullList);
int32_t *nullArray = (int32_t *)calloc(1, len);
(*env)->GetByteArrayRegion(env, nullList, 0, len, (jbyte *)nullArray);
if ((*env)->ExceptionCheck(env)) {
JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_stmtErrorMsgImp(JNIEnv *env, jobject jobj, jlong stmt,
jlong con) {
char errMsg[128];
TAOS *tscon = (TAOS *)con;
if (tscon == NULL) {
jniError("jobj:%p, connection already closed", jobj);
sprintf(errMsg, "jobj:%p, connection already closed", jobj);
return (*env)->NewStringUTF(env, errMsg);
}
const char *name = (*env)->GetStringUTFChars(env, tableName, NULL);
char *curTags = tagsData;
TAOS_BIND *tagsBind = calloc(numOfTags, sizeof(TAOS_BIND));
for (int32_t i = 0; i < numOfTags; ++i) {
tagsBind[i].buffer_type = typeArray[i];
tagsBind[i].buffer = curTags;
tagsBind[i].is_null = &nullArray[i];
tagsBind[i].length = (uintptr_t *)&lengthArray[i];
curTags += lengthArray[i];
TAOS_STMT *pStmt = (TAOS_STMT *)stmt;
if (pStmt == NULL) {
jniError("jobj:%p, conn:%p, invalid stmt", jobj, tscon);
sprintf(errMsg, "jobj:%p, conn:%p, invalid stmt", jobj, tscon);
return (*env)->NewStringUTF(env, errMsg);
}
int32_t code = taos_stmt_set_tbname_tags((void *)stmt, name, tagsBind);
int32_t nTags = (int32_t)numOfTags;
jniDebug("jobj:%p, conn:%p, set table name:%s, numOfTags:%d", jobj, tsconn, name, nTags);
tfree(tagsData);
tfree(lengthArray);
tfree(typeArray);
tfree(nullArray);
tfree(tagsBind);
(*env)->ReleaseStringUTFChars(env, tableName, name);
if (code != TSDB_CODE_SUCCESS) {
jniError("jobj:%p, conn:%p, code:%s", jobj, tsconn, tstrerror(code));
return JNI_TDENGINE_ERROR;
}
return JNI_SUCCESS;
return (*env)->NewStringUTF(env, taos_stmt_errstr((TAOS_STMT *)stmt));
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_insertLinesImp(JNIEnv *env, jobject jobj,
jobjectArray lines, jlong conn,
jint protocol, jint precision) {
jobjectArray lines, jlong conn,
jint protocol, jint precision) {
TAOS *taos = (TAOS *)conn;
if (taos == NULL) {
jniError("jobj:%p, connection already closed", jobj);
......@@ -1013,8 +1056,8 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_insertLinesImp(JN
c_lines[i] = (char *)(*env)->GetStringUTFChars(env, line, 0);
}
SSqlObj* result = (SSqlObj*)taos_schemaless_insert(taos, c_lines, numLines, protocol, precision);
int code = taos_errno(result);
SSqlObj *result = (SSqlObj *)taos_schemaless_insert(taos, c_lines, numLines, protocol, precision);
int code = taos_errno(result);
for (int i = 0; i < numLines; ++i) {
jstring line = (jstring)((*env)->GetObjectArrayElement(env, lines, i));
......
src/client/src/taos.def
浏览文件 @ adf429f8
......@@ -52,3 +52,4 @@ taos_stmt_bind_single_param_batch
taos_is_null
taos_insert_lines
taos_schemaless_insert
taos_result_block
src/client/src/tscParseInsert.c
浏览文件 @ adf429f8
......@@ -481,32 +481,12 @@ int tsParseOneRow(char **str, STableDataBlocks *pDataBlocks, int16_t timePrec, i
// Remove quotation marks
if (TK_STRING == sToken.type) {
// delete escape character: \\, \', \"
char delim = sToken.z[0];
int32_t cnt = 0;
int32_t j = 0;
if (sToken.n >= TSDB_MAX_BYTES_PER_ROW) {
return tscSQLSyntaxErrMsg(pInsertParam->msg, "too long string", sToken.z);
}
for (uint32_t k = 1; k < sToken.n - 1; ++k) {
if (sToken.z[k] == '\\' || (sToken.z[k] == delim && sToken.z[k + 1] == delim)) {
tmpTokenBuf[j] = sToken.z[k + 1];
cnt++;
j++;
k++;
continue;
}
tmpTokenBuf[j] = sToken.z[k];
j++;
}
tmpTokenBuf[j] = 0;
strncpy(tmpTokenBuf, sToken.z, sToken.n);
sToken.n = stringProcess(tmpTokenBuf, sToken.n);
sToken.z = tmpTokenBuf;
sToken.n -= 2 + cnt;
}
bool isPrimaryKey = (colIndex == PRIMARYKEY_TIMESTAMP_COL_INDEX);
......@@ -1057,10 +1037,12 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
break;
}
char* tmp = NULL;
// Remove quotation marks
if (TK_STRING == sToken.type) {
sToken.z++;
sToken.n -= 2;
tmp = strndup(sToken.z, sToken.n);
sToken.n = stringProcess(tmp, sToken.n);
sToken.z = tmp;
}
char tagVal[TSDB_MAX_TAGS_LEN] = {0};
......@@ -1068,6 +1050,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
if (code != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder);
tscDestroyBoundColumnInfo(&spd);
tfree(tmp);
return code;
}
......@@ -1078,18 +1061,18 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
if(sToken.n > TSDB_MAX_JSON_TAGS_LEN/TSDB_NCHAR_SIZE){
tdDestroyKVRowBuilder(&kvRowBuilder);
tscDestroyBoundColumnInfo(&spd);
tfree(tmp);
return tscSQLSyntaxErrMsg(pInsertParam->msg, "json tag too long", NULL);
}
char* json = strndup(sToken.z, sToken.n);
code = parseJsontoTagData(json, &kvRowBuilder, pInsertParam->msg, pTagSchema[spd.boundedColumns[0]].colId);
code = parseJsontoTagData(sToken.z, &kvRowBuilder, pInsertParam->msg, pTagSchema[spd.boundedColumns[0]].colId);
if (code != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder);
tscDestroyBoundColumnInfo(&spd);
tfree(json);
tfree(tmp);
return code;
}
tfree(json);
}
tfree(tmp);
}
tscDestroyBoundColumnInfo(&spd);
......@@ -1246,12 +1229,8 @@ static int32_t parseBoundColumns(SInsertStatementParam *pInsertParam, SParsedDat
strncpy(tmpTokenBuf, sToken.z, sToken.n);
sToken.z = tmpTokenBuf;
if (TK_STRING == sToken.type) {
tscDequoteAndTrimToken(&sToken);
}
if (TK_ID == sToken.type) {
tscRmEscapeAndTrimToken(&sToken);
if (TK_STRING == sToken.type || TK_ID == sToken.type) {
sToken.n = stringProcess(sToken.z, sToken.n);
}
if (sToken.type == TK_RP) {
......@@ -1371,7 +1350,7 @@ _clean:
static int32_t getFileFullPath(SStrToken* pToken, char* output) {
char path[PATH_MAX] = {0};
strncpy(path, pToken->z, pToken->n);
strdequote(path);
stringProcess(path, (int32_t)strlen(path));
wordexp_t full_path;
if (wordexp(path, &full_path, 0) != 0) {
......
src/client/src/tscParseLineProtocol.c
浏览文件 @ adf429f8
......@@ -20,7 +20,7 @@
#include "tscParseLine.h"
typedef struct {
char sTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE];
char sTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE];
SHashObj* tagHash;
SHashObj* fieldHash;
SArray* tags; //SArray<SSchema>
......@@ -68,13 +68,13 @@ typedef enum {
} ESchemaAction;
typedef struct {
char sTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE];
char sTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE];
SArray* tags; //SArray<SSchema>
SArray* fields; //SArray<SSchema>
} SCreateSTableActionInfo;
typedef struct {
char sTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE];
char sTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE];
SSchema* field;
} SAlterSTableActionInfo;
......@@ -161,14 +161,14 @@ static int32_t getSmlMd5ChildTableName(TAOS_SML_DATA_POINT* point, char* tableNa
}
SStringBuilder sb; memset(&sb, 0, sizeof(sb));
char sTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char sTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
strncpy(sTableName, point->stableName, strlen(point->stableName));
//strtolower(sTableName, point->stableName);
taosStringBuilderAppendString(&sb, sTableName);
for (int j = 0; j < point->tagNum; ++j) {
taosStringBuilderAppendChar(&sb, ',');
TAOS_SML_KV* tagKv = point->tags + j;
char tagName[TSDB_COL_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char tagName[TSDB_COL_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
strncpy(tagName, tagKv->key, strlen(tagKv->key));
//strtolower(tagName, tagKv->key);
taosStringBuilderAppendString(&sb, tagName);
......@@ -192,8 +192,8 @@ static int32_t getSmlMd5ChildTableName(TAOS_SML_DATA_POINT* point, char* tableNa
static int32_t buildSmlChildTableName(TAOS_SML_DATA_POINT* point, SSmlLinesInfo* info) {
tscDebug("SML:0x%"PRIx64" taos_sml_insert build child table name", info->id);
char childTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE];
int32_t tableNameLen = TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE;
char childTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE];
int32_t tableNameLen = TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE;
getSmlMd5ChildTableName(point, childTableName, &tableNameLen, info);
point->childTableName = calloc(1, tableNameLen+1);
strncpy(point->childTableName, childTableName, tableNameLen);
......@@ -251,15 +251,15 @@ static int32_t buildDataPointSchemas(TAOS_SML_DATA_POINT* points, int numPoint,
size_t nameLen = strlen(tsSmlTagNullName);
strncpy(tagNullName, tsSmlTagNullName, nameLen);
addEscapeCharToString(tagNullName, (int32_t)nameLen);
size_t* pTagNullIdx = taosHashGet(pStableSchema->tagHash, tagNullName, nameLen + TS_ESCAPE_CHAR_SIZE);
size_t* pTagNullIdx = taosHashGet(pStableSchema->tagHash, tagNullName, nameLen + TS_BACKQUOTE_CHAR_SIZE);
if (!pTagNullIdx) {
SSchema tagNull = {0};
tagNull.type = TSDB_DATA_TYPE_NCHAR;
tagNull.bytes = TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE;
strncpy(tagNull.name, tagNullName, nameLen + TS_ESCAPE_CHAR_SIZE);
strncpy(tagNull.name, tagNullName, nameLen + TS_BACKQUOTE_CHAR_SIZE);
taosArrayPush(pStableSchema->tags, &tagNull);
size_t tagNullIdx = taosArrayGetSize(pStableSchema->tags) - 1;
taosHashPut(pStableSchema->tagHash, tagNull.name, nameLen + TS_ESCAPE_CHAR_SIZE, &tagNullIdx, sizeof(tagNullIdx));
taosHashPut(pStableSchema->tagHash, tagNull.name, nameLen + TS_BACKQUOTE_CHAR_SIZE, &tagNullIdx, sizeof(tagNullIdx));
}
}
......@@ -295,7 +295,7 @@ static int32_t buildDataPointSchemas(TAOS_SML_DATA_POINT* points, int numPoint,
static int32_t generateSchemaAction(SSchema* pointColField, SHashObj* dbAttrHash, SArray* dbAttrArray, bool isTag, char sTableName[],
SSchemaAction* action, bool* actionNeeded, SSmlLinesInfo* info) {
char fieldName[TSDB_COL_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char fieldName[TSDB_COL_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
strcpy(fieldName, pointColField->name);
size_t* pDbIndex = taosHashGet(dbAttrHash, fieldName, strlen(fieldName));
......@@ -315,7 +315,7 @@ static int32_t generateSchemaAction(SSchema* pointColField, SHashObj* dbAttrHash
action->action = SCHEMA_ACTION_CHANGE_COLUMN_SIZE;
}
memset(&action->alterSTable, 0, sizeof(SAlterSTableActionInfo));
memcpy(action->alterSTable.sTableName, sTableName, TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE);
memcpy(action->alterSTable.sTableName, sTableName, TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE);
action->alterSTable.field = pointColField;
*actionNeeded = true;
}
......@@ -326,7 +326,7 @@ static int32_t generateSchemaAction(SSchema* pointColField, SHashObj* dbAttrHash
action->action = SCHEMA_ACTION_ADD_COLUMN;
}
memset(&action->alterSTable, 0, sizeof(SAlterSTableActionInfo));
memcpy(action->alterSTable.sTableName, sTableName, TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE);
memcpy(action->alterSTable.sTableName, sTableName, TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE);
action->alterSTable.field = pointColField;
*actionNeeded = true;
}
......@@ -572,7 +572,7 @@ static int32_t getSuperTableMetaFromLocalCache(TAOS* taos, char* tableName, STab
pSql->fp = NULL;
registerSqlObj(pSql);
char tableNameBuf[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char tableNameBuf[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
memcpy(tableNameBuf, tableName, strlen(tableName));
SStrToken tableToken = {.z = tableNameBuf, .n = (uint32_t)strlen(tableName), .type = TK_ID};
tGetToken(tableNameBuf, &tableToken.type);
......@@ -689,7 +689,7 @@ static int32_t modifyDBSchemas(TAOS* taos, SArray* stableSchemas, SSmlLinesInfo*
SSchemaAction schemaAction = {0};
schemaAction.action = SCHEMA_ACTION_CREATE_STABLE;
memset(&schemaAction.createSTable, 0, sizeof(SCreateSTableActionInfo));
memcpy(schemaAction.createSTable.sTableName, pointSchema->sTableName, TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE);
memcpy(schemaAction.createSTable.sTableName, pointSchema->sTableName, TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE);
schemaAction.createSTable.tags = pointSchema->tags;
schemaAction.createSTable.fields = pointSchema->fields;
applySchemaAction(taos, &schemaAction, info);
......@@ -726,7 +726,7 @@ static int32_t modifyDBSchemas(TAOS* taos, SArray* stableSchemas, SSmlLinesInfo*
SSchema* pointColTs = taosArrayGet(pointSchema->fields, 0);
SSchema* dbColTs = taosArrayGet(dbSchema.fields, 0);
memcpy(pointColTs->name, dbColTs->name, TSDB_COL_NAME_LEN + TS_ESCAPE_CHAR_SIZE);
memcpy(pointColTs->name, dbColTs->name, TSDB_COL_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE);
for (int j = 1; j < pointFieldSize; ++j) {
SSchema* pointCol = taosArrayGet(pointSchema->fields, j);
......@@ -1398,7 +1398,7 @@ char* addEscapeCharToString(char *str, int32_t len) {
return NULL;
}
memmove(str + 1, str, len);
str[0] = str[len + 1] = TS_ESCAPE_CHAR;
str[0] = str[len + 1] = TS_BACKQUOTE_CHAR;
str[len + 2] = '\0';
return str;
}
......@@ -2129,7 +2129,7 @@ static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **index, SHashObj *pHash
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
}
pKV->key = calloc(len + TS_ESCAPE_CHAR_SIZE + 1, 1);
pKV->key = calloc(len + TS_BACKQUOTE_CHAR_SIZE + 1, 1);
memcpy(pKV->key, key, len + 1);
addEscapeCharToString(pKV->key, len);
tscDebug("SML:0x%"PRIx64" Key:%s|len:%d", info->id, pKV->key, len);
......@@ -2227,7 +2227,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index
const char *cur = *index;
int16_t len = 0;
pSml->stableName = calloc(TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE, 1);
pSml->stableName = calloc(TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE, 1);
if (pSml->stableName == NULL){
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
......@@ -2313,7 +2313,7 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
}
size_t childTableNameLen = strlen(tsSmlChildTableName);
char childTableName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char childTableName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
if (childTableNameLen != 0) {
memcpy(childTableName, tsSmlChildTableName, childTableNameLen);
addEscapeCharToString(childTableName, (int32_t)(childTableNameLen));
......@@ -2332,7 +2332,7 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
}
if (!isField && childTableNameLen != 0 && strcasecmp(pkv->key, childTableName) == 0) {
smlData->childTableName = malloc(pkv->length + TS_ESCAPE_CHAR_SIZE + 1);
smlData->childTableName = malloc(pkv->length + TS_BACKQUOTE_CHAR_SIZE + 1);
memcpy(smlData->childTableName, pkv->value, pkv->length);
addEscapeCharToString(smlData->childTableName, (int32_t)pkv->length);
free(pkv->key);
......
src/client/src/tscParseOpenTSDB.c
浏览文件 @ adf429f8
......@@ -37,7 +37,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **index,
const char *cur = *index;
uint16_t len = 0;
pSml->stableName = tcalloc(TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE, 1);
pSml->stableName = tcalloc(TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE, 1);
if (pSml->stableName == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
......@@ -125,7 +125,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
}
tfree(value);
(*pTS)->key = tcalloc(sizeof(key) + TS_ESCAPE_CHAR_SIZE, 1);
(*pTS)->key = tcalloc(sizeof(key) + TS_BACKQUOTE_CHAR_SIZE, 1);
memcpy((*pTS)->key, key, sizeof(key));
addEscapeCharToString((*pTS)->key, (int32_t)strlen(key));
......@@ -196,7 +196,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
}
tfree(value);
pVal->key = tcalloc(sizeof(key) + TS_ESCAPE_CHAR_SIZE, 1);
pVal->key = tcalloc(sizeof(key) + TS_BACKQUOTE_CHAR_SIZE, 1);
memcpy(pVal->key, key, sizeof(key));
addEscapeCharToString(pVal->key, (int32_t)strlen(pVal->key));
*num_kvs += 1;
......@@ -240,7 +240,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **index, SHashObj
return TSDB_CODE_TSC_DUP_TAG_NAMES;
}
pKV->key = tcalloc(len + TS_ESCAPE_CHAR_SIZE + 1, 1);
pKV->key = tcalloc(len + TS_BACKQUOTE_CHAR_SIZE + 1, 1);
memcpy(pKV->key, key, len + 1);
addEscapeCharToString(pKV->key, len);
//tscDebug("OTD:0x%"PRIx64" Key:%s|len:%d", info->id, pKV->key, len);
......@@ -307,7 +307,7 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
pkv = *pKVs;
size_t childTableNameLen = strlen(tsSmlChildTableName);
char childTbName[TSDB_TABLE_NAME_LEN + TS_ESCAPE_CHAR_SIZE] = {0};
char childTbName[TSDB_TABLE_NAME_LEN + TS_BACKQUOTE_CHAR_SIZE] = {0};
if (childTableNameLen != 0) {
memcpy(childTbName, tsSmlChildTableName, childTableNameLen);
addEscapeCharToString(childTbName, (int32_t)(childTableNameLen));
......@@ -324,7 +324,7 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
return ret;
}
if (childTableNameLen != 0 && strcasecmp(pkv->key, childTbName) == 0) {
*childTableName = tcalloc(pkv->length + TS_ESCAPE_CHAR_SIZE + 1, 1);
*childTableName = tcalloc(pkv->length + TS_BACKQUOTE_CHAR_SIZE + 1, 1);
memcpy(*childTableName, pkv->value, pkv->length);
(*childTableName)[pkv->length] = '\0';
addEscapeCharToString(*childTableName, pkv->length);
......@@ -500,7 +500,7 @@ static int32_t parseMetricFromJSON(cJSON *root, TAOS_SML_DATA_POINT* pSml, SSmlL
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
}
pSml->stableName = tcalloc(stableLen + TS_ESCAPE_CHAR_SIZE + 1, sizeof(char));
pSml->stableName = tcalloc(stableLen + TS_BACKQUOTE_CHAR_SIZE + 1, sizeof(char));
if (pSml->stableName == NULL){
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
......@@ -879,7 +879,7 @@ static int32_t parseMetricValueFromJSON(cJSON *root, TAOS_SML_KV **pKVs, int *nu
return ret;
}
pVal->key = tcalloc(sizeof(key) + TS_ESCAPE_CHAR_SIZE, 1);
pVal->key = tcalloc(sizeof(key) + TS_BACKQUOTE_CHAR_SIZE, 1);
memcpy(pVal->key, key, sizeof(key));
addEscapeCharToString(pVal->key, (int32_t)strlen(pVal->key));
......@@ -910,7 +910,7 @@ static int32_t parseTagsFromJSON(cJSON *root, TAOS_SML_KV **pKVs, int *num_kvs,
return TSDB_CODE_TSC_INVALID_JSON;
}
size_t idLen = strlen(id->valuestring);
*childTableName = tcalloc(idLen + TS_ESCAPE_CHAR_SIZE + 1, sizeof(char));
*childTableName = tcalloc(idLen + TS_BACKQUOTE_CHAR_SIZE + 1, sizeof(char));
memcpy(*childTableName, id->valuestring, idLen);
addEscapeCharToString(*childTableName, (int32_t)idLen);
......@@ -948,7 +948,7 @@ static int32_t parseTagsFromJSON(cJSON *root, TAOS_SML_KV **pKVs, int *num_kvs,
tscError("OTD:0x%"PRIx64" Tag key cannot exceeds %d characters in JSON", info->id, TSDB_COL_NAME_LEN - 1);
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
}
pkv->key = tcalloc(keyLen + TS_ESCAPE_CHAR_SIZE + 1, sizeof(char));
pkv->key = tcalloc(keyLen + TS_BACKQUOTE_CHAR_SIZE + 1, sizeof(char));
strncpy(pkv->key, tag->string, keyLen);
addEscapeCharToString(pkv->key, (int32_t)keyLen);
//value
......
src/client/src/tscPrepare.c
浏览文件 @ adf429f8
......@@ -1534,10 +1534,10 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO
}
int32_t stmtValidateValuesFields(SSqlCmd *pCmd, char * sql) {
int32_t loopCont = 1, index0 = 0, values = 0;
int32_t index0 = 0, values = 0;
SStrToken sToken;
while (loopCont) {
while (1) {
sToken = tStrGetToken(sql, &index0, false);
if (sToken.n <= 0) {
return TSDB_CODE_SUCCESS;
......
src/client/src/tscSQLParser.c
浏览文件 @ adf429f8
......@@ -322,7 +322,7 @@ static int32_t invalidOperationMsg(char* dstBuffer, const char* errMsg) {
static int convertTimestampStrToInt64(tVariant *pVar, int32_t precision) {
int64_t time = 0;
strdequote(pVar->pz);
stringProcess(pVar->pz, pVar->nLen);
char* seg = strnchr(pVar->pz, '-', pVar->nLen, false);
if (seg != NULL) {
......@@ -359,7 +359,7 @@ static int32_t handlePassword(SSqlCmd* pCmd, SStrToken* pPwd) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
strdequote(pPwd->z);
stringProcess(pPwd->z, pPwd->n);
pPwd->n = (uint32_t)strtrim(pPwd->z); // trim space before and after passwords
if (pPwd->n <= 0) {
......@@ -477,7 +477,7 @@ int32_t handleUserDefinedFunc(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (validateColumnName(createInfo->name.z) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
strdequote(createInfo->name.z);
stringProcess(createInfo->name.z, createInfo->name.n);
if (strlen(createInfo->name.z) >= TSDB_FUNC_NAME_LEN) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
......@@ -485,7 +485,7 @@ int32_t handleUserDefinedFunc(SSqlObj* pSql, struct SSqlInfo* pInfo) {
createInfo->path.z[createInfo->path.n] = 0;
strdequote(createInfo->path.z);
stringProcess(createInfo->path.z, createInfo->path.n);
if (strlen(createInfo->path.z) >= PATH_MAX) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
......@@ -543,7 +543,7 @@ int32_t handleUserDefinedFunc(SSqlObj* pSql, struct SSqlInfo* pInfo) {
t0->z[t0->n] = 0;
strdequote(t0->z);
stringProcess(t0->z, t0->n);
if (strlen(t0->z) >= TSDB_FUNC_NAME_LEN) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
......@@ -628,7 +628,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
}
} else if (pInfo->type == TSDB_SQL_DROP_DNODE) {
if (pzName->type == TK_STRING) {
pzName->n = strdequote(pzName->z);
pzName->n = stringProcess(pzName->z, pzName->n);
}
strncpy(pCmd->payload, pzName->z, pzName->n);
} else { // drop user/account
......@@ -718,7 +718,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SStrToken* id = taosArrayGet(pInfo->pMiscInfo->a, 0);
if (id->type == TK_STRING) {
id->n = strdequote(id->z);
id->n = stringProcess(id->z, id->n);
}
break;
}
......@@ -834,7 +834,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SStrToken* t0 = taosArrayGet(pMiscInfo->a, 0);
SStrToken* t1 = taosArrayGet(pMiscInfo->a, 1);
t0->n = strdequote(t0->z);
t0->n = stringProcess(t0->z, t0->n);
strncpy(pCfg->ep, t0->z, t0->n);
if (validateEp(pCfg->ep) != TSDB_CODE_SUCCESS) {
......@@ -2674,6 +2674,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
const char* msg15 = "parameter is out of range [1, 1000]";
const char* msg16 = "elapsed duration should be greater than or equal to database precision";
const char* msg17 = "elapsed/twa should not be used in nested query if inner query has group by clause";
const char* msg18 = "the second parameter is not an integer";
switch (functionId) {
case TSDB_FUNC_COUNT: {
......@@ -3156,6 +3157,11 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
char* endptr = NULL;
strtoll(pParamElem[1].pNode->exprToken.z, &endptr, 10);
if ((endptr-pParamElem[1].pNode->exprToken.z != pParamElem[1].pNode->exprToken.n) || errno == ERANGE) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg18);
}
tVariantDump(pVariant, val, TSDB_DATA_TYPE_BIGINT, true);
int64_t numRowsSelected = GET_INT64_VAL(val);
......@@ -3435,7 +3441,7 @@ static int16_t doGetColumnIndex(SQueryInfo* pQueryInfo, int32_t index, SStrToken
pToken->z = tmpTokenBuf;
if (pToken->type == TK_ID) {
tscRmEscapeAndTrimToken(pToken);
pToken->n = stringProcess(pToken->z, pToken->n);
}
for (int16_t i = 0; i < numOfCols; ++i) {
......@@ -3563,7 +3569,6 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
const char* msg3 = "database name too long";
const char* msg5 = "database name is empty";
const char* msg6 = "pattern string is empty";
const char* msg7 = "pattern is invalid";
/*
* database prefix in pInfo->pMiscInfo->a[0]
......@@ -3596,11 +3601,7 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
// show table/stable like 'xxxx', set the like pattern for show tables
SStrToken* pPattern = &pShowInfo->pattern;
if (pPattern->type != 0) {
if (pPattern->type == TK_ID && pPattern->z[0] == TS_ESCAPE_CHAR) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
}
pPattern->n = strdequote(pPattern->z);
pPattern->n = stringProcess(pPattern->z, pPattern->n);
if (pPattern->n <= 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6);
......@@ -3618,7 +3619,7 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
}
if (pShowInfo->prefix.type == TK_STRING) {
pShowInfo->prefix.n = strdequote(pShowInfo->prefix.z);
pShowInfo->prefix.n = stringProcess(pShowInfo->prefix.z, pShowInfo->prefix.n);
}
}
return TSDB_CODE_SUCCESS;
......@@ -4947,7 +4948,7 @@ static int32_t validateNullExpr(tSqlExpr* pExpr, STableMeta* pTableMeta, int32_t
}
char *v = strndup(pRight->exprToken.z, pRight->exprToken.n);
int32_t len = strRmquote(v, pRight->exprToken.n);
int32_t len = stringProcess(v, pRight->exprToken.n);
if (len > 0) {
uint32_t type = 0;
tGetToken(v, &type);
......@@ -5061,20 +5062,10 @@ static int32_t validateMatchExpr(tSqlExpr* pExpr, STableMeta* pTableMeta, int32_
regex_t regex;
char regErrBuf[256] = {0};
//remove the quote at the begin end of original sql string.
uint32_t lenPattern = pRight->exprToken.n - 2;
char* pattern = malloc(lenPattern + 1);
strncpy(pattern, pRight->exprToken.z+1, lenPattern);
pattern[lenPattern] = '\0';
tfree(pRight->value.pz);
pRight->value.pz = pattern;
pRight->value.nLen = lenPattern;
int cflags = REG_EXTENDED;
if ((errCode = regcomp(&regex, pattern, cflags)) != 0) {
if ((errCode = regcomp(&regex, pRight->value.pz, cflags)) != 0) {
regerror(errCode, &regex, regErrBuf, sizeof(regErrBuf));
tscError("Failed to compile regex pattern %s. reason %s", pattern, regErrBuf);
tscError("Failed to compile regex pattern %s. reason %s", pRight->value.pz, regErrBuf);
return invalidOperationMsg(msgBuf, msg3);
}
regfree(&regex);
......@@ -6074,7 +6065,7 @@ int32_t getTimeRange(STimeWindow* win, tSqlExpr* pRight, int32_t optr, int16_t t
int64_t val = 0;
bool parsed = false;
if (pRight->value.nType == TSDB_DATA_TYPE_BINARY) {
pRight->value.nLen = strdequote(pRight->value.pz);
pRight->value.nLen = stringProcess(pRight->value.pz, pRight->value.nLen);
char* seg = strnchr(pRight->value.pz, '-', pRight->value.nLen, false);
if (seg != NULL) {
......@@ -7030,7 +7021,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
//handle Escape character backstick
bool inEscape = false;
if (name.z[0] == TS_ESCAPE_CHAR && name.z[name.n - 1] == TS_ESCAPE_CHAR) {
if (name.z[0] == TS_BACKQUOTE_CHAR && name.z[name.n - 1] == TS_BACKQUOTE_CHAR) {
inEscape = true;
name.type = TK_ID;
}
......@@ -7049,7 +7040,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
int32_t nameLen = pItem->pVar.nLen;
if (inEscape) {
memmove(name1, name1 + 1, nameLen);
name1[nameLen - TS_ESCAPE_CHAR_SIZE] = '\0';
name1[nameLen - TS_BACKQUOTE_CHAR_SIZE] = '\0';
}
TAOS_FIELD f = tscCreateField(TSDB_DATA_TYPE_INT, name1, tDataTypes[TSDB_DATA_TYPE_INT].bytes);
......@@ -7070,7 +7061,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
//handle Escape character backstick
bool inEscape = false;
if (name.z[0] == TS_ESCAPE_CHAR && name.z[name.n - 1] == TS_ESCAPE_CHAR) {
if (name.z[0] == TS_BACKQUOTE_CHAR && name.z[name.n - 1] == TS_BACKQUOTE_CHAR) {
inEscape = true;
name.type = TK_ID;
}
......@@ -7111,8 +7102,8 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
if (inEscape) {
memmove(name.z, name.z + 1, name.n);
name.z[name.n - TS_ESCAPE_CHAR_SIZE] = '\0';
name.n -= TS_ESCAPE_CHAR_SIZE;
name.z[name.n - TS_BACKQUOTE_CHAR_SIZE] = '\0';
name.n -= TS_BACKQUOTE_CHAR_SIZE;
}
TAOS_FIELD f = tscCreateField(pColSchema->type, name.z, pItem->bytes);
......@@ -7130,10 +7121,10 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER;
SStrToken name = {.type = TK_STRING, .z = pItem->name, .n = (uint32_t)strlen(pItem->name)};
//handle Escape character backstick
if (name.z[0] == TS_ESCAPE_CHAR && name.z[name.n - 1] == TS_ESCAPE_CHAR) {
if (name.z[0] == TS_BACKQUOTE_CHAR && name.z[name.n - 1] == TS_BACKQUOTE_CHAR) {
memmove(name.z, name.z + 1, name.n);
name.z[name.n - TS_ESCAPE_CHAR_SIZE] = '\0';
name.n -= TS_ESCAPE_CHAR_SIZE;
name.z[name.n - TS_BACKQUOTE_CHAR_SIZE] = '\0';
name.n -= TS_BACKQUOTE_CHAR_SIZE;
}
if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) {
return invalidOperationMsg(pMsg, msg17);
......@@ -7330,7 +7321,7 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) {
SStrToken* pValToken = taosArrayGet(pOptions->a, 2);
int32_t vnodeId = 0;
int32_t dnodeId = 0;
strdequote(pValToken->z);
stringProcess(pValToken->z, pValToken->n);
bool parseOk = taosCheckBalanceCfgOptions(pValToken->z, &vnodeId, &dnodeId);
if (!parseOk) {
return TSDB_CODE_TSC_INVALID_OPERATION; // options value is invalid
......@@ -7432,7 +7423,7 @@ int32_t validateColumnName(char* name) {
}
if (token.type == TK_STRING) {
strdequote(token.z);
token.n = stringProcess(token.z, token.n);
strntolower(token.z, token.z, token.n);
token.n = (uint32_t)strtrim(token.z);
......@@ -7443,7 +7434,7 @@ int32_t validateColumnName(char* name) {
return validateColumnName(token.z);
} else if (token.type == TK_ID) {
strRmquoteEscape(name, token.n);
stringProcess(name, token.n);
return TSDB_CODE_SUCCESS;
} else {
if (isNumber(&token)) {
......@@ -7594,7 +7585,7 @@ static int32_t setTimePrecision(SSqlCmd* pCmd, SCreateDbMsg* pMsg, SCreateDbInfo
SStrToken* pToken = &pCreateDbInfo->precision;
if (pToken->n > 0) {
pToken->n = strdequote(pToken->z);
pToken->n = stringProcess(pToken->z, pToken->n);
if (strncmp(pToken->z, TSDB_TIME_PRECISION_MILLI_STR, pToken->n) == 0 &&
strlen(TSDB_TIME_PRECISION_MILLI_STR) == pToken->n) {
......@@ -8649,12 +8640,8 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
strncpy(tmpTokenBuf, sToken->z, sToken->n);
sToken->z = tmpTokenBuf;
if (TK_STRING == sToken->type) {
tscDequoteAndTrimToken(sToken);
}
if (TK_ID == sToken->type) {
tscRmEscapeAndTrimToken(sToken);
if (TK_STRING == sToken->type || TK_ID == sToken->type) {
sToken->n = stringProcess(sToken->z, sToken->n);
}
tVariantListItem* pItem = taosArrayGet(pValList, i);
......@@ -9593,8 +9580,6 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
}
tscDequoteAndTrimToken(oriName);
bool dbIncluded = false;
char buf[TSDB_TABLE_FNAME_LEN];
SStrToken sTblToken;
......@@ -9616,7 +9601,6 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
tscDequoteAndTrimToken(aliasName);
if (tscValidateName(aliasName, false, NULL) != TSDB_CODE_SUCCESS || aliasName->n >= TSDB_TABLE_NAME_LEN) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
......
src/client/src/tscServer.c
浏览文件 @ adf429f8
......@@ -363,6 +363,41 @@ void tscSetFqdnErrorMsg(SSqlObj* pSql, SRpcEpSet* pEpSet) {
}
}
bool shouldRewTableMeta(SSqlObj* pSql, SRpcMsg* rpcMsg) {
SSqlCmd *pCmd = &pSql->cmd;
SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd);
int32_t cmd = pCmd->command;
if ((cmd != TSDB_SQL_SELECT && cmd != TSDB_SQL_UPDATE_TAGS_VAL)) {
return false;
}
if (rpcMsg->code != TSDB_CODE_TDB_INVALID_TABLE_ID &&
rpcMsg->code != TSDB_CODE_VND_INVALID_VGROUP_ID &&
rpcMsg->code != TSDB_CODE_QRY_INVALID_SCHEMA_VERSION &&
rpcMsg->code != TSDB_CODE_RPC_NETWORK_UNAVAIL &&
rpcMsg->code != TSDB_CODE_APP_NOT_READY ) {
return false;
}
if (rpcMsg->code == TSDB_CODE_QRY_INVALID_SCHEMA_VERSION) {
return true;
}
// 1. super table subquery
// 2. nest queries are all not updated the tablemeta and retry parse the sql after cleanup local tablemeta/vgroup id buffer
if ((TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY | TSDB_QUERY_TYPE_SUBQUERY | TSDB_QUERY_TYPE_TAG_FILTER_QUERY)) &&
!TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_PROJECTION_QUERY)) ||
(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_NEST_SUBQUERY)) ||
(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_STABLE_SUBQUERY) && pQueryInfo->distinct)
|| (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_JOIN_QUERY))) {
return false;
}
// single table query error need to renew table meta.
return true;
}
void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet) {
TSDB_CACHE_PTR_TYPE handle = (TSDB_CACHE_PTR_TYPE) rpcMsg->ahandle;
SSqlObj* pSql = (SSqlObj*)taosAcquireRef(tscObjRef, handle);
......@@ -415,42 +450,29 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet) {
pSql->cmd.insertParam.schemaAttached = 1;
}
// single table query error need to be handled here.
if ((cmd == TSDB_SQL_SELECT || cmd == TSDB_SQL_UPDATE_TAGS_VAL) &&
(((rpcMsg->code == TSDB_CODE_TDB_INVALID_TABLE_ID || rpcMsg->code == TSDB_CODE_VND_INVALID_VGROUP_ID)) ||
rpcMsg->code == TSDB_CODE_RPC_NETWORK_UNAVAIL || rpcMsg->code == TSDB_CODE_APP_NOT_READY)) {
// 1. super table subquery
// 2. nest queries are all not updated the tablemeta and retry parse the sql after cleanup local tablemeta/vgroup id buffer
if ((TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY | TSDB_QUERY_TYPE_SUBQUERY |
TSDB_QUERY_TYPE_TAG_FILTER_QUERY)) &&
!TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_PROJECTION_QUERY)) ||
(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_NEST_SUBQUERY)) || (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_STABLE_SUBQUERY) && pQueryInfo->distinct)
|| (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_JOIN_QUERY))) {
// do nothing in case of super table subquery
} else {
pSql->retry += 1;
tscWarn("0x%" PRIx64 " it shall renew table meta, code:%s, retry:%d", pSql->self, tstrerror(rpcMsg->code), pSql->retry);
pSql->res.code = rpcMsg->code; // keep the previous error code
if (pSql->retry > pSql->maxRetry) {
tscError("0x%" PRIx64 " max retry %d reached, give up", pSql->self, pSql->maxRetry);
} else {
// wait for a little bit moment and then retry
// todo do not sleep in rpc callback thread, add this process into queue to process
if (rpcMsg->code == TSDB_CODE_APP_NOT_READY || rpcMsg->code == TSDB_CODE_VND_INVALID_VGROUP_ID) {
int32_t duration = getWaitingTimeInterval(pSql->retry);
taosMsleep(duration);
}
pSql->retryReason = rpcMsg->code;
rpcMsg->code = tscRenewTableMeta(pSql);
// if there is an error occurring, proceed to the following error handling procedure.
if (rpcMsg->code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, handle);
rpcFreeCont(rpcMsg->pCont);
return;
}
bool renewTableMeta = shouldRewTableMeta(pSql, rpcMsg);
if (renewTableMeta) {
pSql->retry += 1;
tscWarn("0x%" PRIx64 " it shall renew table meta, code:%s, retry:%d", pSql->self, tstrerror(rpcMsg->code), pSql->retry);
pSql->res.code = rpcMsg->code; // keep the previous error code
if (pSql->retry > pSql->maxRetry) {
tscError("0x%" PRIx64 " max retry %d reached, give up", pSql->self, pSql->maxRetry);
} else {
// wait for a little bit moment and then retry
// todo do not sleep in rpc callback thread, add this process into queue to process
if (rpcMsg->code == TSDB_CODE_APP_NOT_READY || rpcMsg->code == TSDB_CODE_VND_INVALID_VGROUP_ID) {
int32_t duration = getWaitingTimeInterval(pSql->retry);
taosMsleep(duration);
}
pSql->retryReason = rpcMsg->code;
rpcMsg->code = tscRenewTableMeta(pSql);
// if there is an error occurring, proceed to the following error handling procedure.
if (rpcMsg->code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
taosReleaseRef(tscObjRef, handle);
rpcFreeCont(rpcMsg->pCont);
return;
}
}
}
......@@ -511,6 +533,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet) {
}
bool shouldFree = tscShouldBeFreed(pSql);
if (rpcMsg->code != TSDB_CODE_TSC_ACTION_IN_PROGRESS) {
if (rpcMsg->code != TSDB_CODE_SUCCESS) {
pRes->code = rpcMsg->code;
......@@ -962,7 +985,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->numOfGroupCols = htons(pQueryInfo->groupbyExpr.numOfGroupCols);
pQueryMsg->queryType = htonl(pQueryInfo->type);
pQueryMsg->prevResultLen = htonl(pQueryInfo->bufLen);
// set column list ids
size_t numOfCols = taosArrayGetSize(pQueryInfo->colList);
char *pMsg = (char *)(pQueryMsg->tableCols) + numOfCols * sizeof(SColumnInfo);
......@@ -1148,21 +1171,21 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += sqlLen;
/*
//MSG EXTEND DEMO
pQueryMsg->extend = 1;
STLV *tlv = (STLV *)pMsg;
tlv->type = htons(TLV_TYPE_DUMMY);
tlv->len = htonl(sizeof(int16_t));
*(int16_t *)tlv->value = htons(12345);
tlv->type = htons(TLV_TYPE_META_VERSION);
tlv->len = htonl(sizeof(int16_t) * 2);
*(int16_t*)tlv->value = htons(pTableMeta->sversion);
*(int16_t*)(tlv->value+sizeof(int16_t)) = htons(pTableMeta->tversion);
pMsg += sizeof(*tlv) + ntohl(tlv->len);
tlv = (STLV *)pMsg;
tlv->type = htons(TLV_TYPE_END_MARK);
tlv->len = 0;
pMsg += sizeof(*tlv);
*/
int32_t msgLen = (int32_t)(pMsg - pCmd->payload);
......@@ -1859,6 +1882,13 @@ int tscProcessRetrieveGlobalMergeRsp(SSqlObj *pSql) {
tscDebug("0x%"PRIx64" create QInfo 0x%"PRIx64" to execute query processing", pSql->self, pSql->self);
pQueryInfo->pQInfo = createQInfoFromQueryNode(pQueryInfo, &tableGroupInfo, NULL, NULL, pRes->pMerger, MERGE_STAGE, pSql->self);
if (pQueryInfo->pQInfo == NULL) {
taosHashCleanup(tableGroupInfo.map);
taosArrayDestroy(&group);
tscAsyncResultOnError(pSql);
pRes->code = TSDB_CODE_QRY_OUT_OF_MEMORY;
return pRes->code;
}
}
uint64_t localQueryId = pSql->self;
......@@ -2691,7 +2721,9 @@ int tscProcessQueryRsp(SSqlObj *pSql) {
pRes->data = NULL;
tscResetForNextRetrieve(pRes);
tscDebug("0x%"PRIx64" query rsp received, qId:0x%"PRIx64, pSql->self, pRes->qId);
return 0;
}
......@@ -2703,7 +2735,7 @@ static void decompressQueryColData(SSqlObj *pSql, SSqlRes *pRes, SQueryInfo* pQu
compSizes = (int32_t *)(pData + compLen);
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, numOfCols - 1);
int16_t offset = tscFieldInfoGetOffset(pQueryInfo, numOfCols - 1);
int32_t offset = tscFieldInfoGetOffset(pQueryInfo, numOfCols - 1);
char *outputBuf = tcalloc(pRes->numOfRows, (pField->bytes + offset));
char *p = outputBuf;
......@@ -2804,11 +2836,12 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) {
pRes->row = 0;
tscDebug("0x%"PRIx64" numOfRows:%d, offset:%" PRId64 ", complete:%d, qId:0x%"PRIx64, pSql->self, pRes->numOfRows, pRes->offset,
pRes->completed, pRes->qId);
pRes->completed, pRes->qId);
return 0;
}
void tscTableMetaCallBack(void *param, TAOS_RES *res, int code);
static int32_t getTableMetaFromMnode(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo, bool autocreate) {
......
src/client/src/tscSql.c
浏览文件 @ adf429f8
......@@ -137,7 +137,7 @@ static SSqlObj *taosConnectImpl(const char *ip, const char *user, const char *pa
char tmp[TSDB_DB_NAME_LEN] = {0};
tstrncpy(tmp, db, sizeof(tmp));
strdequote(tmp);
stringProcess(tmp, (int32_t)strlen(tmp));
strtolower(pObj->db, tmp);
}
......@@ -547,6 +547,28 @@ int taos_fetch_block(TAOS_RES *res, TAOS_ROW *rows) {
return pRes->numOfRows;
}
TAOS_ROW *taos_result_block(TAOS_RES *res) {
SSqlObj *pSql = (SSqlObj *)res;
if (pSql == NULL || pSql->signature != pSql) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return NULL;
}
SSqlCmd *pCmd = &pSql->cmd;
SSqlRes *pRes = &pSql->res;
if (pCmd == NULL ||
pRes == NULL ||
pRes->qId == 0 ||
pRes->code == TSDB_CODE_TSC_QUERY_CANCELLED ||
pCmd->command == TSDB_SQL_RETRIEVE_EMPTY_RESULT ||
pCmd->command == TSDB_SQL_INSERT) {
return NULL;
}
return &pRes->urow;
}
int taos_select_db(TAOS *taos, const char *db) {
char sql[256] = {0};
......
src/client/src/tscStream.c
浏览文件 @ adf429f8
......@@ -211,6 +211,7 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf
tfree(pSql->pSubs);
pSql->subState.numOfSub = 0;
pSql->parseRetry = 0;
int32_t code = tsParseSql(pSql, true);
if (code == TSDB_CODE_SUCCESS) {
cbParseSql(pStream, pSql, code);
......@@ -220,6 +221,7 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf
tscError("0x%"PRIx64" open stream failed, code:%s", pSql->self, tstrerror(code));
taosReleaseRef(tscObjRef, pSql->self);
free(pStream);
return;
}
// tscSetRetryTimer(pStream, pStream->pSql, retryDelay);
......
src/client/src/tscSubquery.c
浏览文件 @ adf429f8
......@@ -3902,8 +3902,11 @@ void* createQInfoFromQueryNode(SQueryInfo* pQueryInfo, STableGroupInfo* pTableGr
STsBufInfo bufInfo = {0};
SQueryParam param = {.pOperator = pa};
/*int32_t code = */initQInfo(&bufInfo, NULL, pSourceOperator, pQInfo, &param, NULL, 0, merger);
int32_t code = initQInfo(&bufInfo, NULL, pSourceOperator, pQInfo, &param, NULL, 0, merger);
taosArrayDestroy(&pa);
if (code != TSDB_CODE_SUCCESS) {
goto _cleanup;
}
return pQInfo;
......
src/client/src/tscUtil.c
浏览文件 @ adf429f8
......@@ -2369,7 +2369,7 @@ TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index) {
return &((SInternalField*)TARRAY_GET_ELEM(pFieldInfo->internalField, index))->field;
}
int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index) {
int32_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index) {
SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, index);
assert(pInfo != NULL && pInfo->pExpr->pExpr == NULL);
......@@ -2918,7 +2918,7 @@ void tscColumnListDestroy(SArray* pColumnList) {
*
*/
static int32_t validateQuoteToken(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded) {
tscDequoteAndTrimToken(pToken);
if(pToken->z[0] != TS_BACKQUOTE_CHAR) pToken->n = stringProcess(pToken->z, pToken->n);
int32_t k = tGetToken(pToken->z, &pToken->type);
......@@ -2932,94 +2932,6 @@ static int32_t validateQuoteToken(SStrToken* pToken, bool escapeEnabled, bool *d
return TSDB_CODE_SUCCESS;
}
void tscDequoteAndTrimToken(SStrToken* pToken) {
uint32_t first = 0, last = pToken->n;
// trim leading spaces
while (first < last) {
char c = pToken->z[first];
if (c != ' ' && c != '\t') {
break;
}
first++;
}
// trim ending spaces
while (first < last) {
char c = pToken->z[last - 1];
if (c != ' ' && c != '\t') {
break;
}
last--;
}
// there are still at least two characters
if (first < last - 1) {
char c = pToken->z[first];
// dequote
if ((c == '\'' || c == '"') && c == pToken->z[last - 1]) {
first++;
last--;
}
}
// left shift the string and pad spaces
for (uint32_t i = 0; i + first < last; i++) {
pToken->z[i] = pToken->z[first + i];
}
for (uint32_t i = last - first; i < pToken->n; i++) {
pToken->z[i] = ' ';
}
// adjust token length
pToken->n = last - first;
}
void tscRmEscapeAndTrimToken(SStrToken* pToken) {
uint32_t first = 0, last = pToken->n;
// trim leading spaces
while (first < last) {
char c = pToken->z[first];
if (c != ' ' && c != '\t') {
break;
}
first++;
}
// trim ending spaces
while (first < last) {
char c = pToken->z[last - 1];
if (c != ' ' && c != '\t') {
break;
}
last--;
}
// there are still at least two characters
if (first < last - 1) {
char c = pToken->z[first];
// dequote
if ((c == '`') && c == pToken->z[last - 1]) {
first++;
last--;
}
}
// left shift the string and pad spaces
for (uint32_t i = 0; i + first < last; i++) {
pToken->z[i] = pToken->z[first + i];
}
for (uint32_t i = last - first; i < pToken->n; i++) {
pToken->z[i] = ' ';
}
// adjust token length
pToken->n = last - first;
}
int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded) {
if (pToken == NULL || pToken->z == NULL
|| (pToken->type != TK_STRING && pToken->type != TK_ID)) {
......@@ -3027,7 +2939,7 @@ int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded)
}
if ((!escapeEnabled) && pToken->type == TK_ID) {
if (pToken->z[0] == TS_ESCAPE_CHAR) {
if (pToken->z[0] == TS_BACKQUOTE_CHAR) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
}
......@@ -3045,7 +2957,7 @@ int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded)
if (pToken->type == TK_STRING) {
tscDequoteAndTrimToken(pToken);
if(pToken->z[0] != TS_BACKQUOTE_CHAR) pToken->n = stringProcess(pToken->z, pToken->n);
// tscStrToLower(pToken->z, pToken->n);
strntolower(pToken->z, pToken->z, pToken->n);
//pToken->n = (uint32_t)strtrim(pToken->z);
......@@ -3065,7 +2977,7 @@ int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded)
return tscValidateName(pToken, escapeEnabled, NULL);
}
} else if (pToken->type == TK_ID) {
tscRmEscapeAndTrimToken(pToken);
if(pToken->z[0] == TS_BACKQUOTE_CHAR) pToken->n = stringProcess(pToken->z, pToken->n);
if (pToken->n == 0) {
return TSDB_CODE_TSC_INVALID_OPERATION;
......@@ -3126,7 +3038,7 @@ int32_t tscValidateName(SStrToken* pToken, bool escapeEnabled, bool *dbIncluded)
}
if (escapeEnabled && pToken->type == TK_ID) {
tscRmEscapeAndTrimToken(pToken);
if(pToken->z[0] == TS_BACKQUOTE_CHAR) pToken->n = stringProcess(pToken->z, pToken->n);
}
// re-build the whole name string
......
src/common/src/tglobal.c
浏览文件 @ adf429f8
......@@ -289,7 +289,7 @@ char Compressor[32] = "ZSTD_COMPRESSOR"; // ZSTD_COMPRESSOR or GZIP_COMPRES
#endif
// long query death-lock
int8_t tsDeadLockKillQuery = 0;
int8_t tsDeadLockKillQuery = 1;
// default JSON string type
char tsDefaultJSONStrType[7] = "nchar";
......
src/common/src/tname.c
浏览文件 @ adf429f8
......@@ -50,7 +50,7 @@ SSchema tGetUserSpecifiedColumnSchema(tVariant* pVal, SStrToken* exprStr, const
} else {
size_t tlen = MIN(sizeof(s.name), exprStr->n + 1);
tstrncpy(s.name, exprStr->z, tlen);
strdequote(s.name);
stringProcess(s.name, (int32_t)strlen(s.name));
}
return s;
......@@ -163,7 +163,7 @@ char *tableNameGetPosition(SStrToken* pToken, char target) {
return pToken->z + i;
}
if (*(pToken->z + i) == TS_ESCAPE_CHAR) {
if (*(pToken->z + i) == TS_BACKQUOTE_CHAR) {
if (!inQuote) {
inEscape = !inEscape;
}
......@@ -223,7 +223,7 @@ void extractTableNameFromToken(SStrToken* pToken, SStrToken* pTable) {
char* r = tableNameGetPosition(pToken, sep);
if (r != NULL) { // record the table name token
if (pToken->z[0] == TS_ESCAPE_CHAR && *(r - 1) == TS_ESCAPE_CHAR) {
if (pToken->z[0] == TS_BACKQUOTE_CHAR && *(r - 1) == TS_BACKQUOTE_CHAR) {
pTable->n = (uint32_t)(r - pToken->z - 2);
pTable->z = pToken->z + 1;
} else {
......
src/common/src/tvariant.c
浏览文件 @ adf429f8
......@@ -87,7 +87,7 @@ void tVariantCreateExt(tVariant *pVar, SStrToken *token, int32_t optrType, bool
case TSDB_DATA_TYPE_BINARY: {
pVar->pz = strndup(token->z, token->n);
pVar->nLen = needRmquoteEscape ? strRmquoteEscape(pVar->pz, token->n) : token->n;
pVar->nLen = needRmquoteEscape ? stringProcess(pVar->pz, token->n) : token->n;
break;
}
case TSDB_DATA_TYPE_TIMESTAMP: {
......
src/connector/C#/src/TDengineDriver/TDengineDriver.cs
浏览文件 @ adf429f8
......@@ -173,7 +173,17 @@ namespace TDengineDriver
static extern public int ErrorNo(IntPtr res);
[DllImport("taos", EntryPoint = "taos_query", CallingConvention = CallingConvention.Cdecl)]
static extern public IntPtr Query(IntPtr conn, string sqlstr);
// static extern public IntPtr Query(IntPtr conn, string sqlstr);
static extern private IntPtr Query(IntPtr conn, IntPtr byteArr);
static public IntPtr Query(IntPtr conn,string command)
{
IntPtr res = IntPtr.Zero;
IntPtr commandBuffer = Marshal.StringToCoTaskMemUTF8(command);
res = Query(conn,commandBuffer);
return res;
}
[DllImport("taos", EntryPoint = "taos_affected_rows", CallingConvention = CallingConvention.Cdecl)]
static extern public int AffectRows(IntPtr res);
......
src/connector/C#/src/TDengineDriver/TaosBind.cs
浏览文件 @ adf429f8
......@@ -2,7 +2,6 @@ using System;
using System.Runtime.InteropServices;
using System.Text;
namespace TDengineDriver
{
/// <summary>
......@@ -248,9 +247,10 @@ namespace TDengineDriver
{
TAOS_BIND bind = new TAOS_BIND();
IntPtr umanageBinary = Marshal.StringToHGlobalAnsi(val);
// IntPtr umanageBinary = Marshal.StringToHGlobalAnsi(val);
IntPtr umanageBinary = Marshal.StringToCoTaskMemUTF8(val);
var strToBytes = System.Text.Encoding.Default.GetBytes(val);
var strToBytes = System.Text.Encoding.UTF8.GetBytes(val);
int leng = strToBytes.Length;
IntPtr lenPtr = Marshal.AllocHGlobal(sizeof(ulong));
Marshal.WriteInt64(lenPtr, leng);
......@@ -266,8 +266,9 @@ namespace TDengineDriver
public static TAOS_BIND BindNchar(String val)
{
TAOS_BIND bind = new TAOS_BIND();
var strToBytes = System.Text.Encoding.Default.GetBytes(val);
IntPtr umanageNchar = (IntPtr)Marshal.StringToHGlobalAnsi(val);
var strToBytes = System.Text.Encoding.UTF8.GetBytes(val);
// IntPtr umanageNchar = (IntPtr)Marshal.StringToHGlobalAnsi(val);
IntPtr umanageNchar = (IntPtr)Marshal.StringToCoTaskMemUTF8(val);
int leng = strToBytes.Length;
......
src/connector/C#/src/TDengineDriver/TaosMultiBind.cs
浏览文件 @ adf429f8
......@@ -2,7 +2,6 @@ using System;
using System.Text;
using System.Runtime.InteropServices;
namespace TDengineDriver
{
public class TaosMultiBind
......
src/connector/C#/src/test/FunctionTest/InsertCn.cs 0 → 100644
浏览文件 @ adf429f8
using System;
using Test.UtilsTools;
using TDengineDriver;
using Test.UtilsTools.DataSource;
using Xunit;
using System.Collections.Generic;
using Test.UtilsTools.ResultSet;
namespace Cases
{
public class InsertCnCharacterCases
{
/// <author>xiaolei</author>
/// <Name>InsertCnCharacterCases.TestInsertCnToNtable</Name>
/// <describe>test insert Chinese character into normal table's nchar column</describe>
/// <filename>InsertCn.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "InsertCnCharacterCases.TestInsertCnToNtable()")]
public void TestInsertCnToNtable()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "cn_insert_nchar_ntable";
// var expectResData = new List<String> { "1637064040000", "true", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "XI", "XII", "{\"k1\": \"v1\"}" };
var colData = new List<Object>{1637064040000,1,"涛思数据",
1637064041000,2,"涛思数据taosdata",
1637064042000,3,"TDegnine涛思数据",
1637064043000,4,"4涛思数据",
1637064044000,5,"涛思数据5",
1637064045000,6,"taos涛思数据6",
1637064046000,7,"7涛思数据taos",
1637064047000,8,"8&涛思数据taos",
1637064048000,9,"&涛思数据taos9"
};
String dropTb = "drop table if exists " + tableName;
String createTb = $"create table if not exists {tableName} (ts timestamp,v4 int,blob nchar(200));";
String insertSql = UtilsTools.ConstructInsertSql(tableName, "", colData, null, 9);
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
List<TDengineMeta> expectResMeta = DataSource.GetMetaFromDLL(createTb);
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
List<String> actualResData = actualResult.GetResultData();
//Assert Meta data
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
// Assert retrieve data
for (int i = 0; i < actualResData.Count; i++)
{
Assert.Equal(colData[i].ToString(), actualResData[i]);
}
}
/// <author>xiaolei</author>
/// <Name>InsertCnCharacterCases.TestInsertCnToStable</Name>
/// <describe>test insert Chinese character into stable's nchar column,both tag and column</describe>
/// <filename>InsertCn.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "InsertCnCharacterCases.TestInsertCnToStable()")]
public void TestInsertCnToStable()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "cn_insert_nchar_stable";
var colData = new List<Object>{1637064040000,1,"涛思数据",
1637064041000,2,"涛思数据taosdata",
1637064042000,3,"TDegnine涛思数据",
1637064043000,4,"4涛思数据",
1637064044000,5,"涛思数据5",
1637064045000,6,"taos涛思数据6",
1637064046000,7,"7涛思数据taos",
1637064047000,8,"8&涛思数据taos",
1637064048000,9,"&涛思数据taos9"
};
var tagData = new List<Object>{1,"涛思数据",};
String dropTb = "drop table if exists " + tableName;
String createTb = $"create table {tableName} (ts timestamp,v4 int,blob nchar(200))tags(id int,name nchar(50));";
String insertSql = UtilsTools.ConstructInsertSql(tableName+"_sub1", tableName, colData, tagData, 9);
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
List<TDengineMeta> expectResMeta = DataSource.GetMetaFromDLL(createTb);
List<Object> expectResData = UtilsTools.CombineColAndTagData(colData,tagData,9);
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
List<String> actualResData = actualResult.GetResultData();
//Assert Meta data
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
// Assert retrieve data
for (int i = 0; i < actualResData.Count; i++)
{
Assert.Equal(expectResData[i].ToString(), actualResData[i]);
}
}
/// <author>xiaolei</author>
/// <Name>InsertCnCharacterCases.TestInsertMutilCnToNtable</Name>
/// <describe>test insert Chinese character into normal table's multiple nchar columns</describe>
/// <filename>InsertCn.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "InsertCnCharacterCases.TestInsertMutilCnToNtable()")]
public void TestInsertMutilCnToNtable()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "cn_multi_insert_nchar_ntable";
var colData = new List<Object>{1637064040000,1,"涛思数据","保利广场","Beijing","China",
1637064041000,2,"涛思数据taosdata","保利广场baoli","Beijing","China",
1637064042000,3,"TDegnine涛思数据","time广场","NewYork","US",
1637064043000,4,"4涛思数据","4广场南部","London","UK",
1637064044000,5,"涛思数据5","!广场路中部123","Tokyo","JP",
1637064045000,6,"taos涛思数据6","青年广场123号！","Washin","DC",
1637064046000,7,"7涛思数据taos","asdf#壮年广场%#endregion","NewYork","US",
1637064047000,8,"8&涛思数据taos","incluse阿斯顿发","NewYork","US",
1637064048000,9,"&涛思数据taos9","123黑化肥werq会挥……&¥%发！afsdfa","NewYork","US",
};
String dropTb = "drop table if exists " + tableName;
String createTb = $"create table if not exists {tableName} (ts timestamp,v4 int,blob nchar(200),location nchar(200),city binary(100),coutry binary(200));";
String insertSql = UtilsTools.ConstructInsertSql(tableName, "", colData, null, 9);
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
List<TDengineMeta> expectResMeta = DataSource.GetMetaFromDLL(createTb);
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
List<String> actualResData = actualResult.GetResultData();
//Assert Meta data
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
// Assert retrieve data
for (int i = 0; i < actualResData.Count; i++)
{
Assert.Equal(colData[i].ToString(), actualResData[i]);
}
}
/// <author>xiaolei</author>
/// <Name>InsertCnCharacterCases.TestInsertMutilCnToStable</Name>
/// <describe>test insert Chinese character into stable's multiple nchar columns</describe>
/// <filename>InsertCn.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "InsertCnCharacterCases.TestInsertMutilCnToStable()")]
public void TestInsertMutilCnToStable()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "cn_multi_insert_nchar_stable";
var colData = new List<Object>{1637064040000,1,"涛思数据","保利广场","Beijing","China",
1637064041000,2,"涛思数据taosdata","保利广场baoli","Beijing","China",
1637064042000,3,"TDegnine涛思数据","time广场","NewYork","US",
1637064043000,4,"4涛思数据","4广场南部","London","UK",
1637064044000,5,"涛思数据5","!广场路中部123","Tokyo","JP",
1637064045000,6,"taos涛思数据6","青年广场123号！","Washin","DC",
1637064046000,7,"7涛思数据taos","asdf#壮年广场%#endregion","NewYork","US",
1637064047000,8,"8&涛思数据taos","incluse阿斯顿发","NewYork","US",
1637064048000,9,"&涛思数据taos9","123黑化肥werq会挥……&¥%发！afsdfa","NewYork","US",
};
var tagData = new List<Object>{1,"涛思数据","中国北方&南方长江黄河！49wq","tdengine"};
String dropTb = "drop table if exists " + tableName;
String createTb = $"create table if not exists {tableName} (ts timestamp," +
$"v4 int," +
$"blob nchar(200)," +
$"locate nchar(200)," +
$"country nchar(200)," +
$"city nchar(50)" +
$")tags(" +
$"id int," +
$"name nchar(50)," +
$"addr nchar(200)," +
$"en_name binary(200));";
String insertSql = UtilsTools.ConstructInsertSql(tableName+"_sub1", tableName, colData, tagData, 9);
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
List<TDengineMeta> expectResMeta = DataSource.GetMetaFromDLL(createTb);
List<Object> expectResData = UtilsTools.CombineColAndTagData(colData,tagData,9);
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
List<String> actualResData = actualResult.GetResultData();
//Assert Meta data
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
// Assert retrieve data
for (int i = 0; i < actualResData.Count; i++)
{
Assert.Equal(expectResData[i].ToString(), actualResData[i]);
}
}
}
}
\ No newline at end of file
src/connector/C#/src/test/FunctionTest/TaosFeild.cs
浏览文件 @ adf429f8
using System;
using Test.UtilsTools;
using TDengineDriver;
using System.Collections.Generic;
using Xunit;
using Test.UtilsTools.ResultSet;
namespace Cases
{
public class FetchFieldCases
{
/// <author>xiaolei</author>
/// <Name>FetchFieldCases.TestFetchFieldJsonTag</Name>
/// <describe>test taos_fetch_fields(), check the meta data</describe>
/// <filename>TaosFeild.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "FetchFieldCases.TestFetchFieldJsonTag()")]
public void TestFetchFieldJsonTag()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "fetchfeilds";
var expectResMeta = new List<TDengineMeta> {
UtilsTools.ConstructTDengineMeta("ts", "timestamp"),
UtilsTools.ConstructTDengineMeta("b", "bool"),
UtilsTools.ConstructTDengineMeta("v1", "tinyint"),
UtilsTools.ConstructTDengineMeta("v2", "smallint"),
UtilsTools.ConstructTDengineMeta("v4", "int"),
UtilsTools.ConstructTDengineMeta("v8", "bigint"),
UtilsTools.ConstructTDengineMeta("f4", "float"),
UtilsTools.ConstructTDengineMeta("f8", "double"),
UtilsTools.ConstructTDengineMeta("u1", "tinyint unsigned"),
UtilsTools.ConstructTDengineMeta("u2", "smallint unsigned"),
UtilsTools.ConstructTDengineMeta("u4", "int unsigned"),
UtilsTools.ConstructTDengineMeta("u8", "bigint unsigned"),
UtilsTools.ConstructTDengineMeta("bin", "binary(200)"),
UtilsTools.ConstructTDengineMeta("blob", "nchar(200)"),
UtilsTools.ConstructTDengineMeta("jsontag", "json"),
};
var expectResData = new List<String> { "1637064040000", "true", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "XI", "XII", "{\"k1\": \"v1\"}" };
String dropTb = "drop table if exists " + tableName;
String createTb = "create stable " + tableName
+ " (ts timestamp" +
",b bool" +
",v1 tinyint" +
",v2 smallint" +
",v4 int" +
",v8 bigint" +
",f4 float" +
",f8 double" +
",u1 tinyint unsigned" +
",u2 smallint unsigned" +
",u4 int unsigned" +
",u8 bigint unsigned" +
",bin binary(200)" +
",blob nchar(200)" +
")" +
"tags" +
"(jsontag json);";
String insertSql = "insert into " + tableName + "_t1 using " + tableName +
" tags('{\"k1\": \"v1\"}') " +
"values(1637064040000,true,1,2,3,4,5,6,7,8,9,10,'XI','XII')";
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
}
}
}
using System;
using Test.UtilsTools;
using TDengineDriver;
using System.Collections.Generic;
using Xunit;
using Test.UtilsTools.ResultSet;
namespace Cases
{
public class FetchFieldCases
{
/// <author>xiaolei</author>
/// <Name>FetchFieldCases.TestFetchFieldJsonTag</Name>
/// <describe>test taos_fetch_fields(), check the meta data</describe>
/// <filename>TaosFeild.cs</filename>
/// <result>pass or failed </result>
[Fact(DisplayName = "FetchFieldCases.TestFetchFieldJsonTag()")]
public void TestFetchFieldJsonTag()
{
IntPtr conn = UtilsTools.TDConnection();
IntPtr _res = IntPtr.Zero;
string tableName = "fetchfeilds";
var expectResMeta = new List<TDengineMeta> {
UtilsTools.ConstructTDengineMeta("ts", "timestamp"),
UtilsTools.ConstructTDengineMeta("b", "bool"),
UtilsTools.ConstructTDengineMeta("v1", "tinyint"),
UtilsTools.ConstructTDengineMeta("v2", "smallint"),
UtilsTools.ConstructTDengineMeta("v4", "int"),
UtilsTools.ConstructTDengineMeta("v8", "bigint"),
UtilsTools.ConstructTDengineMeta("f4", "float"),
UtilsTools.ConstructTDengineMeta("f8", "double"),
UtilsTools.ConstructTDengineMeta("u1", "tinyint unsigned"),
UtilsTools.ConstructTDengineMeta("u2", "smallint unsigned"),
UtilsTools.ConstructTDengineMeta("u4", "int unsigned"),
UtilsTools.ConstructTDengineMeta("u8", "bigint unsigned"),
UtilsTools.ConstructTDengineMeta("bin", "binary(200)"),
UtilsTools.ConstructTDengineMeta("blob", "nchar(200)"),
UtilsTools.ConstructTDengineMeta("jsontag", "json"),
};
var expectResData = new List<String> { "1637064040000", "true", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "XI", "XII", "{\"k1\": \"v1\"}" };
String dropTb = "drop table if exists " + tableName;
String createTb = "create stable " + tableName
+ " (ts timestamp" +
",b bool" +
",v1 tinyint" +
",v2 smallint" +
",v4 int" +
",v8 bigint" +
",f4 float" +
",f8 double" +
",u1 tinyint unsigned" +
",u2 smallint unsigned" +
",u4 int unsigned" +
",u8 bigint unsigned" +
",bin binary(200)" +
",blob nchar(200)" +
")" +
"tags" +
"(jsontag json);";
String insertSql = "insert into " + tableName + "_t1 using " + tableName +
" tags('{\"k1\": \"v1\"}') " +
"values(1637064040000,true,1,2,3,4,5,6,7,8,9,10,'XI','XII')";
String selectSql = "select * from " + tableName;
String dropSql = "drop table " + tableName;
UtilsTools.ExecuteUpdate(conn, dropTb);
UtilsTools.ExecuteUpdate(conn, createTb);
UtilsTools.ExecuteUpdate(conn, insertSql);
_res = UtilsTools.ExecuteQuery(conn, selectSql);
ResultSet actualResult = new ResultSet(_res);
List<TDengineMeta> actualMeta = actualResult.GetResultMeta();
for (int i = 0; i < actualMeta.Count; i++)
{
Assert.Equal(expectResMeta[i].name, actualMeta[i].name);
Assert.Equal(expectResMeta[i].type, actualMeta[i].type);
Assert.Equal(expectResMeta[i].size, actualMeta[i].size);
}
}
}
}
src/connector/C#/src/test/FunctionTest/Utils.cs → src/connector/C#/src/test/FunctionTest/lib/Utils.cs
浏览文件 @ adf429f8
......@@ -13,6 +13,7 @@ namespace Test.UtilsTools
static string password = "taosdata";
static string db = "";
static short port = 0;
//get a tdengine connection
public static IntPtr TDConnection()
{
TDengine.Options((int)TDengineInitOption.TDDB_OPTION_CONFIGDIR, GetConfigPath());
......@@ -24,6 +25,7 @@ namespace Test.UtilsTools
UtilsTools.ExecuteUpdate(conn, "use csharp");
return conn;
}
//get taos.cfg file based on different os
public static string GetConfigPath()
{
string configDir = "" ;
......@@ -340,7 +342,8 @@ namespace Test.UtilsTools
dataRaw.Add(v7.ToString());
break;
case TDengineDataType.TSDB_DATA_TYPE_BINARY:
string v8 = Marshal.PtrToStringAnsi(data, colLengthArr[fields]);
// string v8 = Marshal.PtrToStringAnsi(data, colLengthArr[fields]);
string v8 = Marshal.PtrToStringUTF8(data, colLengthArr[fields]);
dataRaw.Add(v8);
break;
case TDengineDataType.TSDB_DATA_TYPE_TIMESTAMP:
......@@ -348,7 +351,8 @@ namespace Test.UtilsTools
dataRaw.Add(v9.ToString());
break;
case TDengineDataType.TSDB_DATA_TYPE_NCHAR:
string v10 = Marshal.PtrToStringAnsi(data, colLengthArr[fields]);
// string v10 = Marshal.PtrToStringAnsi(data, colLengthArr[fields]);
string v10 = Marshal.PtrToStringUTF8(data, colLengthArr[fields]);
dataRaw.Add(v10);
break;
case TDengineDataType.TSDB_DATA_TYPE_UTINYINT:
......@@ -383,6 +387,89 @@ namespace Test.UtilsTools
return dataRaw;
}
// Generate insert sql for the with the coldata and tag data
public static string ConstructInsertSql(string table,string stable,List<Object> colData,List<Object> tagData,int numOfRows)
{
int numofFileds = colData.Count / numOfRows;
StringBuilder insertSql;
if (stable == "")
{
insertSql = new StringBuilder($"insert into {table} values(");
}
else
{
insertSql = new StringBuilder($"insert into {table} using {stable} tags(");
for (int j = 0; j < tagData.Count; j++)
{
if (tagData[j] is String)
{
insertSql.Append('\'');
insertSql.Append(tagData[j]);
insertSql.Append('\'');
}
else
{
insertSql.Append(tagData[j]);
}
if (j + 1 != tagData.Count)
{
insertSql.Append(',');
}
}
insertSql.Append(")values(");
}
for (int i = 0; i < colData.Count; i++)
{
if (colData[i] is String)
{
insertSql.Append('\'');
insertSql.Append(colData[i]);
insertSql.Append('\'');
}
else
{
insertSql.Append(colData[i]);
}
if ((i + 1) % numofFileds == 0 && (i + 1) != colData.Count)
{
insertSql.Append(")(");
}
else if ((i + 1) == colData.Count)
{
insertSql.Append(')');
}
else
{
insertSql.Append(',');
}
}
insertSql.Append(';');
//Console.WriteLine(insertSql.ToString());
return insertSql.ToString();
}
public static List<object> CombineColAndTagData(List<object> colData,List<object> tagData, int numOfRows)
{
var list = new List<Object>();
for (int i = 0; i < colData.Count; i++)
{
list.Add(colData[i]);
if ((i + 1) % (colData.Count / numOfRows) == 0)
{
for (int j = 0; j < tagData.Count; j++)
{
list.Add(tagData[j]);
}
}
}
return list;
}
}
}
src/connector/C#/src/test/XUnitTest/TestTaosBind.cs
浏览文件 @ adf429f8
......@@ -763,12 +763,12 @@ namespace TDengineDriver.Test
{
int bufferType = 8;
String buffer = "qwertyuiopasdghjklzxcvbnm<>?:\"{}+_)(*&^%$#@!~QWERTYUIOP[]\\ASDFGHJKL;'ZXCVBNM,./`1234567890-=";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindBinary("qwertyuiopasdghjklzxcvbnm<>?:\"{}+_)(*&^%$#@!~QWERTYUIOP[]\\ASDFGHJKL;'ZXCVBNM,./`1234567890-=");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......@@ -789,12 +789,12 @@ namespace TDengineDriver.Test
{
int bufferType = 8;
String buffer = "一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindBinary("一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......@@ -815,12 +815,12 @@ namespace TDengineDriver.Test
{
int bufferType = 8;
String buffer = "一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindBinary("一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......@@ -841,12 +841,12 @@ namespace TDengineDriver.Test
{
int bufferType = 10;
String buffer = "qwertyuiopasdghjklzxcvbnm<>?:\"{}+_)(*&^%$#@!~QWERTYUIOP[]\\ASDFGHJKL;'ZXCVBNM,./`1234567890-=";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindNchar("qwertyuiopasdghjklzxcvbnm<>?:\"{}+_)(*&^%$#@!~QWERTYUIOP[]\\ASDFGHJKL;'ZXCVBNM,./`1234567890-=");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......@@ -867,12 +867,12 @@ namespace TDengineDriver.Test
{
int bufferType = 10;
String buffer = "一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindNchar("一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......@@ -893,12 +893,12 @@ namespace TDengineDriver.Test
{
int bufferType = 10;
String buffer = "一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM";
int bufferLength = System.Text.Encoding.Default.GetBytes(buffer).Length;
int length = System.Text.Encoding.Default.GetBytes(buffer).Length;
int bufferLength = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
int length = System.Text.Encoding.UTF8.GetBytes(buffer).Length;
TDengineDriver.TAOS_BIND bind = TaosBind.BindNchar("一二两三四五六七八九十廿毛另壹贰叁肆伍陆柒捌玖拾佰仟万亿元角分零整1234567890`~!@#$%^&*()_+[]{};':<>?,./qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM");
int BindLengPtr = Marshal.ReadInt32(bind.length);
string bindBuffer = Marshal.PtrToStringAnsi(bind.buffer);
string bindBuffer = Marshal.PtrToStringUTF8(bind.buffer);
Assert.Equal(bind.buffer_type, bufferType);
Assert.Equal(bindBuffer, buffer);
......
src/connector/jdbc/CMakeLists.txt
浏览文件 @ adf429f8
......@@ -5,11 +5,21 @@ PROJECT(TDengine)
IF (TD_MVN_INSTALLED)
SET(JDBC_CMD_NAME "jdbc_cmd")
SET(JDBC_TARGET_NAME "jdbc_target")
ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME}
POST_BUILD
SET(_output "${CMAKE_CURRENT_BINARY_DIR}/${JDBC_CMD_NAME}")
file(GLOB_RECURSE _depends "${CMAKE_CURRENT_SOURCE_DIR}/src/*")
ADD_CUSTOM_COMMAND(OUTPUT ${_output}
DEPENDS taos taos_static
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/deploy-pom.xml
DEPENDS ${_depends}
COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.36-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.37-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E touch "${_output}"
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
VERBATIM
COMMENT "build jdbc driver")
ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME})
UNSET(_depends)
UNSET(_output)
ENDIF ()
src/connector/jdbc/deploy-pom.xml
浏览文件 @ adf429f8
......@@ -5,7 +5,7 @@
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.36</version>
<version>2.0.37</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
......
src/connector/jdbc/pom.xml
浏览文件 @ adf429f8
......@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.36</version>
<version>2.0.37</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBConnection.java
浏览文件 @ adf429f8
......@@ -63,7 +63,6 @@ public class TSDBConnection extends AbstractConnection {
if (isClosed()) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_CONNECTION_CLOSED);
}
return new TSDBPreparedStatement(this, sql);
}
......@@ -71,7 +70,6 @@ public class TSDBConnection extends AbstractConnection {
if (isClosed) {
return;
}
this.connector.closeConnection();
this.isClosed = true;
}
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java
浏览文件 @ adf429f8
......@@ -28,6 +28,8 @@ public class TSDBJNIConnector {
System.loadLibrary("taos");
}
/***********************************************************************/
//NOTE: JDBC
public static void init(Properties props) throws SQLWarning {
synchronized (LOCK) {
if (!isInitialized) {
......@@ -242,6 +244,9 @@ public class TSDBJNIConnector {
private native int closeConnectionImp(long connection);
/*****************************************************************************************/
// NOTE: subscribe
/**
* Create a subscription
*/
......@@ -269,6 +274,8 @@ public class TSDBJNIConnector {
private native void unsubscribeImp(long subscription, boolean isKeep);
/******************************************************************************************************/
// NOTE: parameter binding
public long prepareStmt(String sql) throws SQLException {
long stmt = prepareStmtImp(sql.getBytes(), this.taos);
......@@ -293,16 +300,19 @@ public class TSDBJNIConnector {
public void setBindTableName(long stmt, String tableName) throws SQLException {
int code = setBindTableNameImp(stmt, tableName, this.taos);
if (code != TSDBConstants.JNI_SUCCESS) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "failed to set table name");
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN,
"failed to set table name, reason: " + stmtErrorMsgImp(stmt, this.taos));
}
}
private native int setBindTableNameImp(long stmt, String name, long conn);
public void setBindTableNameAndTags(long stmt, String tableName, int numOfTags, ByteBuffer tags, ByteBuffer typeList, ByteBuffer lengthList, ByteBuffer nullList) throws SQLException {
public void setBindTableNameAndTags(long stmt, String tableName, int numOfTags, ByteBuffer tags,
ByteBuffer typeList, ByteBuffer lengthList, ByteBuffer nullList) throws SQLException {
int code = setTableNameTagsImp(stmt, tableName, numOfTags, tags.array(), typeList.array(), lengthList.array(), nullList.array(), this.taos);
if (code != TSDBConstants.JNI_SUCCESS) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "failed to bind table name and corresponding tags");
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN,
"failed to bind table name and corresponding tags, reason: " + stmtErrorMsgImp(stmt, this.taos));
}
}
......@@ -311,7 +321,8 @@ public class TSDBJNIConnector {
public void bindColumnDataArray(long stmt, ByteBuffer colDataList, ByteBuffer lengthList, ByteBuffer isNullList, int type, int bytes, int numOfRows, int columnIndex) throws SQLException {
int code = bindColDataImp(stmt, colDataList.array(), lengthList.array(), isNullList.array(), type, bytes, numOfRows, columnIndex, this.taos);
if (code != TSDBConstants.JNI_SUCCESS) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "failed to bind column data");
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN,
"failed to bind column data, reason: " + stmtErrorMsgImp(stmt, this.taos));
}
}
......@@ -320,10 +331,20 @@ public class TSDBJNIConnector {
public void executeBatch(long stmt) throws SQLException {
int code = executeBatchImp(stmt, this.taos);
if (code != TSDBConstants.JNI_SUCCESS) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "failed to execute batch bind");
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN,
"failed to execute batch bind, reason: " + stmtErrorMsgImp(stmt, this.taos));
}
}
public void addBatch(long stmt) throws SQLException {
int code = addBatchImp(stmt, this.taos);
if (code != TSDBConstants.JNI_SUCCESS){
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, stmtErrorMsgImp(stmt, this.taos));
}
}
private native int addBatchImp(long stmt, long con);
private native int executeBatchImp(long stmt, long con);
public void closeBatch(long stmt) throws SQLException {
......@@ -335,6 +356,10 @@ public class TSDBJNIConnector {
private native int closeStmt(long stmt, long con);
private native String stmtErrorMsgImp(long stmt, long con);
/*************************************************************************************************/
// NOTE: schemaless-lines
public void insertLines(String[] lines, SchemalessProtocolType protocolType, SchemalessTimestampType timestampType) throws SQLException {
int code = insertLinesImp(lines, this.taos, protocolType.ordinal(), timestampType.ordinal());
if (code != TSDBConstants.JNI_SUCCESS) {
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBPreparedStatement.java
浏览文件 @ adf429f8
......@@ -40,25 +40,27 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
private String rawSql;
private Object[] parameters;
// for parameter binding
private long nativeStmtHandle = 0;
private long nativeStmtHandle;
private String tableName;
private ArrayList<TableTagInfo> tableTags;
private int tagValueLength;
private ArrayList<ColumnInfo> colData;
TSDBPreparedStatement(TSDBConnection connection, String sql) {
TSDBPreparedStatement(TSDBConnection connection, String sql) throws SQLException {
super(connection);
init(sql);
int parameterCnt = 0;
if (sql.contains("?")) {
for (int i = 0; i < sql.length(); i++) {
if ('?' == sql.charAt(i)) {
parameterCnt++;
}
if (!sql.contains("?"))
return;
for (int i = 0; i < sql.length(); i++) {
if ('?' == sql.charAt(i)) {
parameterCnt++;
}
}
parameters = new Object[parameterCnt];
// for parameter-binding
// TSDBJNIConnector connector = ((TSDBConnection) this.getConnection()).getConnector();
// this.nativeStmtHandle = connector.prepareStmt(rawSql);
if (parameterCnt > 1) {
// the table name is also a parameter, so ignore it.
......@@ -530,8 +532,14 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
}
public void setTableName(String name) throws SQLException {
if (this.nativeStmtHandle == 0) {
TSDBJNIConnector connector = ((TSDBConnection) this.getConnection()).getConnector();
this.nativeStmtHandle = connector.prepareStmt(rawSql);
}
if (this.tableName != null) {
this.columnDataExecuteBatch();
this.columnDataAddBatch();
this.columnDataClearBatchInternal();
}
this.tableName = name;
......@@ -693,7 +701,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
if (rawSql == null) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "sql statement not set yet");
}
// table name is not set yet, abort
if (this.tableName == null) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "table name not set yet");
......@@ -703,24 +710,25 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
if (numOfCols == 0) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "column data not bind");
}
if (nativeStmtHandle == 0) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "stmt is null");
}
TSDBJNIConnector connector = ((TSDBConnection) this.getConnection()).getConnector();
this.nativeStmtHandle = connector.prepareStmt(rawSql);
if (this.tableTags == null) {
connector.setBindTableName(this.nativeStmtHandle, this.tableName);
} else {
int num = this.tableTags.size();
int tagSize = this.tableTags.size();
ByteBuffer tagDataList = ByteBuffer.allocate(this.tagValueLength);
tagDataList.order(ByteOrder.LITTLE_ENDIAN);
ByteBuffer typeList = ByteBuffer.allocate(num);
ByteBuffer typeList = ByteBuffer.allocate(tagSize);
typeList.order(ByteOrder.LITTLE_ENDIAN);
ByteBuffer lengthList = ByteBuffer.allocate(num * Long.BYTES);
ByteBuffer lengthList = ByteBuffer.allocate(tagSize * Long.BYTES);
lengthList.order(ByteOrder.LITTLE_ENDIAN);
ByteBuffer isNullList = ByteBuffer.allocate(num * Integer.BYTES);
ByteBuffer isNullList = ByteBuffer.allocate(tagSize * Integer.BYTES);
isNullList.order(ByteOrder.LITTLE_ENDIAN);
for (TableTagInfo tag : this.tableTags) {
......@@ -744,54 +752,43 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
lengthList.putLong(Byte.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_BOOL: {
Boolean val = (Boolean) tag.value;
tagDataList.put((byte) (val ? 1 : 0));
lengthList.putLong(Byte.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_SMALLINT: {
Short val = (Short) tag.value;
tagDataList.putShort(val);
lengthList.putLong(Short.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP:
case TSDBConstants.TSDB_DATA_TYPE_BIGINT: {
Long val = (Long) tag.value;
tagDataList.putLong(val == null ? 0 : val);
lengthList.putLong(Long.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_FLOAT: {
Float val = (Float) tag.value;
tagDataList.putFloat(val == null ? 0 : val);
lengthList.putLong(Float.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_DOUBLE: {
Double val = (Double) tag.value;
tagDataList.putDouble(val == null ? 0 : val);
lengthList.putLong(Double.BYTES);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_NCHAR:
case TSDBConstants.TSDB_DATA_TYPE_JSON:
case TSDBConstants.TSDB_DATA_TYPE_BINARY: {
String charset = TaosGlobalConfig.getCharset();
String val = (String) tag.value;
byte[] b = null;
byte[] b;
try {
if (tag.type == TSDBConstants.TSDB_DATA_TYPE_BINARY) {
b = val.getBytes();
......@@ -801,12 +798,10 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e.getMessage());
}
tagDataList.put(b);
lengthList.putLong(b.length);
break;
}
case TSDBConstants.TSDB_DATA_TYPE_UTINYINT:
case TSDBConstants.TSDB_DATA_TYPE_USMALLINT:
case TSDBConstants.TSDB_DATA_TYPE_UINT:
......@@ -814,13 +809,12 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "not support data types");
}
}
typeList.put((byte) tag.type);
isNullList.putInt(tag.isNull ? 1 : 0);
}
connector.setBindTableNameAndTags(this.nativeStmtHandle, this.tableName, this.tableTags.size(), tagDataList,
typeList, lengthList, isNullList);
connector.setBindTableNameAndTags(this.nativeStmtHandle, this.tableName, this.tableTags.size(),
tagDataList, typeList, lengthList, isNullList);
}
ColumnInfo colInfo = this.colData.get(0);
......@@ -834,7 +828,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
if (col1 == null || !col1.isTypeSet()) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "column data not bind");
}
if (rows != col1.data.size()) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "the rows in column data not identical");
}
......@@ -951,7 +944,6 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
}
break;
}
case TSDBConstants.TSDB_DATA_TYPE_UTINYINT:
case TSDBConstants.TSDB_DATA_TYPE_USMALLINT:
case TSDBConstants.TSDB_DATA_TYPE_UINT:
......@@ -962,6 +954,8 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
connector.bindColumnDataArray(this.nativeStmtHandle, colDataList, lengthList, isNullList, col1.type, col1.bytes, rows, i);
}
connector.addBatch(this.nativeStmtHandle);
this.columnDataClearBatchInternal();
}
public void columnDataExecuteBatch() throws SQLException {
......@@ -976,13 +970,14 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
}
private void columnDataClearBatchInternal() {
int size = this.colData.size();
this.colData.clear();
this.colData.addAll(Collections.nCopies(size, null));
this.tableName = null; // clear the table name
this.tableName = null;
if (this.tableTags != null)
this.tableTags.clear();
tagValueLength = 0;
if (this.colData != null)
this.colData.clear();
}
public void columnDataCloseBatch() throws SQLException {
TSDBJNIConnector connector = ((TSDBConnection) this.getConnection()).getConnector();
connector.closeBatch(this.nativeStmtHandle);
......
src/connector/jdbc/src/test/java/com/taosdata/jdbc/ParameterBindTest.java
浏览文件 @ adf429f8
package com.taosdata.jdbc;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.*;
import java.sql.*;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
......@@ -21,13 +20,17 @@ public class ParameterBindTest {
private final Random random = new Random(System.currentTimeMillis());
@Test
public void test() {
public void one_batch_multi_table() throws SQLException {
// given
String[] tbnames = {"t1", "t2", "t3"};
int rows = 10;
// when
insertIntoTables(tbnames, 10);
String sql = "insert into ? using " + stable + " tags(?, ?) values(?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(sql).unwrap(TSDBPreparedStatement.class)) {
long current = System.currentTimeMillis();
insertIntoTables(pstmt, tbnames, current, 10);
}
// then
assertRows(stable, tbnames.length * rows);
......@@ -37,13 +40,48 @@ public class ParameterBindTest {
}
@Test
public void testMultiThreads() {
public void multi_batch_multi_table() throws SQLException {
// given
int rows = 10;
int batchSize = 10;
String[] tbnames = {"t1", "t2", "t3"};
// when
String sql = "insert into ? using " + stable + " tags(?, ?) values(?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(sql).unwrap(TSDBPreparedStatement.class)) {
long current = System.currentTimeMillis();
for (int i = 0; i < batchSize; i++) {
insertIntoTables(pstmt, tbnames, current + 1000 * i * rows, rows);
}
}
// then
assertRows(stable, tbnames.length * batchSize * rows);
for (String t : tbnames) {
assertRows(t, rows * batchSize);
}
}
@Test
public void multiThreads() {
// given
String[][] tables = {{"t1", "t2", "t3"}, {"t4", "t5", "t6"}, {"t7", "t8", "t9"}, {"t10"}};
int rows = 10;
// when
List<Thread> threads = Arrays.stream(tables).map(tbnames -> new Thread(() -> insertIntoTables(tbnames, rows))).collect(Collectors.toList());
List<Thread> threads = Arrays.stream(tables).map(tbnames -> new Thread(() -> {
String sql = "insert into ? using " + stable + " tags(?, ?) values(?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(sql).unwrap(TSDBPreparedStatement.class)) {
long current = System.currentTimeMillis();
insertIntoTables(pstmt, tbnames, current, 10);
} catch (SQLException throwables) {
throwables.printStackTrace();
}
})).collect(Collectors.toList());
threads.forEach(Thread::start);
for (Thread thread : threads) {
try {
......@@ -59,9 +97,26 @@ public class ParameterBindTest {
assertRows(t, rows);
}
}
}
@Ignore
@Test
public void testOOM() throws SQLException {
String[] tbnames = {"t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9", "t10"};
String sql = "insert into ? using " + stable + " tags(?, ?) values(?, ?, ?)";
int rows = 1000;
try (TSDBPreparedStatement pstmt = conn.prepareStatement(sql).unwrap(TSDBPreparedStatement.class)) {
long ts = Instant.now().minus(5 * 365, ChronoUnit.DAYS).getEpochSecond() * 1000;
while (true) {
insertIntoTables(pstmt, tbnames, ts, rows);
ts += 1000 * rows;
}
}
}
private void assertRows(String tbname, int rows) {
try (Statement stmt = conn.createStatement()) {
ResultSet rs = stmt.executeQuery("select count(*) from " + tbname);
......@@ -74,40 +129,36 @@ public class ParameterBindTest {
}
}
private void insertIntoTables(String[] tbnames, int rowsEachTable) {
long current = System.currentTimeMillis();
String sql = "insert into ? using " + stable + " tags(?, ?) values(?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(sql).unwrap(TSDBPreparedStatement.class)) {
for (int i = 0; i < tbnames.length; i++) {
pstmt.setTableName(tbnames[i]);
pstmt.setTagInt(0, random.nextInt(100));
pstmt.setTagInt(1, random.nextInt(100));
ArrayList<Long> timestampList = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
timestampList.add(current + i * 1000 + j);
}
pstmt.setTimestamp(0, timestampList);
ArrayList<Integer> f1List = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
f1List.add(random.nextInt(100));
}
pstmt.setInt(1, f1List);
ArrayList<Integer> f2List = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
f2List.add(random.nextInt(100));
}
pstmt.setInt(2, f2List);
pstmt.columnDataAddBatch();
private void insertIntoTables(TSDBPreparedStatement pstmt, String[] tbnames, long ts_start, int rowsEachTable) throws SQLException {
for (int i = 0; i < tbnames.length; i++) {
// set table name
pstmt.setTableName(tbnames[i]);
// set tags
pstmt.setTagInt(0, random.nextInt(100));
pstmt.setTagInt(1, random.nextInt(100));
// set column: ts
ArrayList<Long> timestampList = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
timestampList.add(ts_start + j * 1000L);
}
pstmt.columnDataExecuteBatch();
} catch (SQLException e) {
e.printStackTrace();
pstmt.setTimestamp(0, timestampList);
// set column: f1
ArrayList<Integer> f1List = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
f1List.add(random.nextInt(100));
}
pstmt.setInt(1, f1List);
// set column: f2
ArrayList<Integer> f2List = new ArrayList<>();
for (int j = 0; j < rowsEachTable; j++) {
f2List.add(random.nextInt(100));
}
pstmt.setInt(2, f2List);
// add batch
pstmt.columnDataAddBatch();
}
// execute batch
pstmt.columnDataExecuteBatch();
}
@Before
......
src/connector/jdbc/src/test/java/com/taosdata/jdbc/TSDBJNIConnectorTest.java
浏览文件 @ adf429f8
......@@ -4,38 +4,25 @@ import com.taosdata.jdbc.enums.SchemalessProtocolType;
import com.taosdata.jdbc.enums.SchemalessTimestampType;
import org.junit.Test;
import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import java.util.Random;
import java.util.stream.IntStream;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public class TSDBJNIConnectorTest {
private static final String host = "127.0.0.1";
private static TSDBResultSetRowData rowData;
@Test
public void test() throws SQLException {
try {
//change sleepSeconds when debugging with attach to process to find PID
int sleepSeconds = -1;
if (sleepSeconds > 0) {
RuntimeMXBean runtimeBean = ManagementFactory.getRuntimeMXBean();
String jvmName = runtimeBean.getName();
long pid = Long.valueOf(jvmName.split("@")[0]);
System.out.println("JVM PID = " + pid);
Thread.sleep(sleepSeconds * 1000);
}
} catch (Exception e) {
e.printStackTrace();
}
// init
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_CONFIG_DIR, "/etc/taos");
......@@ -43,7 +30,7 @@ public class TSDBJNIConnectorTest {
// connect
TSDBJNIConnector connector = new TSDBJNIConnector();
connector.connect("127.0.0.1", 6030, null, "root", "taosdata");
connector.connect(host, 6030, null, "root", "taosdata");
// setup
String setupSqlStrs[] = {"create database if not exists d precision \"us\"",
......@@ -141,4 +128,128 @@ public class TSDBJNIConnectorTest {
} else return code != TSDBConstants.JNI_FETCH_END;
}
@Test
public void param_bind_one_batch_multi_table() throws SQLException {
TSDBJNIConnector connector = new TSDBJNIConnector();
connector.connect(host, 6030, null, "root", "taosdata");
connector.executeQuery("drop database if exists test");
connector.executeQuery("create database if not exists test");
connector.executeQuery("use test");
connector.executeQuery("create table weather(ts timestamp, f1 int) tags(t1 int)");
// 1. init + prepare
long stmt = connector.prepareStmt("insert into ? using weather tags(?) values(?,?)");
for (int i = 0; i < 10; i++) {
// 2. set_tbname_tags
stmt_set_table_tags(connector, stmt, "t" + i);
// 3. bind_single_param_batch
// bind timestamp
long ts = System.currentTimeMillis();
bind_col_timestamp(connector, stmt, ts, 100);
// bind int
bind_col_integer(connector, stmt, 100);
// 4. add_batch
connector.addBatch(stmt);
}
connector.executeBatch(stmt);
connector.closeBatch(stmt);
connector.executeQuery("drop database if exists test");
connector.closeConnection();
}
@Test
public void param_bind_multi_batch_multi_table() throws SQLException {
TSDBJNIConnector connector = new TSDBJNIConnector();
connector.connect(host, 6030, null, "root", "taosdata");
connector.executeQuery("drop database if exists test");
connector.executeQuery("create database if not exists test");
connector.executeQuery("use test");
connector.executeQuery("create table weather(ts timestamp, f1 int) tags(t1 int)");
// 1. init + prepare
long stmt = connector.prepareStmt("insert into ? using weather tags(?) values(?,?)");
long ts = System.currentTimeMillis();
for (int ind_batch = 0; ind_batch < 10; ind_batch++) {
ts += ind_batch * 1000 * 1000;
System.out.println("batch: " + ind_batch + ", ts: " + ts);
for (int i = 0; i < 10; i++) {
// 2. set_tbname_tags
stmt_set_table_tags(connector, stmt, "t" + i);
// 3. bind_single_param_batch
// bind timestamp
bind_col_timestamp(connector, stmt, ts, 100);
// bind int
bind_col_integer(connector, stmt, 100);
// 4. add_batch
connector.addBatch(stmt);
}
connector.executeBatch(stmt);
}
connector.closeBatch(stmt);
connector.executeQuery("drop database if exists test");
connector.closeConnection();
}
private void bind_col_timestamp(TSDBJNIConnector connector, long stmt, long ts_start, int numOfRows) throws SQLException {
ByteBuffer colDataList = ByteBuffer.allocate(numOfRows * Long.BYTES);
colDataList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> colDataList.putLong(ts_start + ind * 1000L));
ByteBuffer lengthList = ByteBuffer.allocate(numOfRows * Long.BYTES);
lengthList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> lengthList.putLong(Integer.BYTES));
ByteBuffer isNullList = ByteBuffer.allocate(numOfRows * Integer.BYTES);
isNullList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> isNullList.putInt(0));
connector.bindColumnDataArray(stmt, colDataList, lengthList, isNullList, TSDBConstants.TSDB_DATA_TYPE_TIMESTAMP, Long.BYTES, numOfRows, 0);
}
private void bind_col_integer(TSDBJNIConnector connector, long stmt, int numOfRows) throws SQLException {
ByteBuffer colDataList = ByteBuffer.allocate(numOfRows * Integer.BYTES);
colDataList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> colDataList.putInt(new Random().nextInt(100)));
ByteBuffer lengthList = ByteBuffer.allocate(numOfRows * Long.BYTES);
lengthList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> lengthList.putLong(Integer.BYTES));
ByteBuffer isNullList = ByteBuffer.allocate(numOfRows * Integer.BYTES);
isNullList.order(ByteOrder.LITTLE_ENDIAN);
IntStream.range(0, numOfRows).forEach(ind -> isNullList.putInt(0));
connector.bindColumnDataArray(stmt, colDataList, lengthList, isNullList, TSDBConstants.TSDB_DATA_TYPE_INT, Integer.BYTES, numOfRows, 1);
}
private void stmt_set_table_tags(TSDBJNIConnector connector, long stmt, String tbname) throws SQLException {
ByteBuffer tagDataList = ByteBuffer.allocate(Integer.BYTES);
tagDataList.order(ByteOrder.LITTLE_ENDIAN);
tagDataList.putInt(new Random().nextInt(100));
ByteBuffer typeList = ByteBuffer.allocate(1);
typeList.order(ByteOrder.LITTLE_ENDIAN);
typeList.put((byte) TSDBConstants.TSDB_DATA_TYPE_INT);
ByteBuffer lengthList = ByteBuffer.allocate(1 * Long.BYTES);
lengthList.order(ByteOrder.LITTLE_ENDIAN);
lengthList.putLong(Integer.BYTES);
ByteBuffer isNullList = ByteBuffer.allocate(1 * Integer.BYTES);
isNullList.order(ByteOrder.LITTLE_ENDIAN);
isNullList.putInt(0);
connector.setBindTableNameAndTags(stmt, tbname, 1, tagDataList, typeList, lengthList, isNullList);
}
}
src/connector/node-red-contrib-tdengine/package-lock.json
浏览文件 @ adf429f8
......@@ -1420,9 +1420,9 @@
"integrity": "sha512-b6suED+5/3rTpUBdG1gupIl8MPFCAMA0QXwmljLhvCUKcUvdE4gWky9zpuGCcXHOsz4J9wPGNWq6OKpmIzz3hQ=="
},
"follow-redirects": {
"version": "1.14.6",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.6.tgz",
"integrity": "sha512-fhUl5EwSJbbl8AR+uYL2KQDxLkdSjZGR36xy46AO7cOMTrCMON6Sa28FmAnC2tRTDbd/Uuzz3aJBv7EBN7JH8A=="
"version": "1.14.7",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.7.tgz",
"integrity": "sha512-+hbxoLbFMbRKDwohX8GkTataGqO6Jb7jGwpAlwgy2bIz25XtRm7KEzJM76R1WiNT5SwZkX4Y75SwBolkpmE7iQ=="
},
"form-data": {
"version": "4.0.0",
......
src/connector/nodejs/jest.config.js 0 → 100644
浏览文件 @ adf429f8
/*
* For a detailed explanation regarding each configuration property, visit:
* https://jestjs.io/docs/configuration
*/
module.exports = {
// All imported modules in your tests should be mocked automatically
// automock: false,
// Stop running tests after `n` failures
// bail: 0,
// The directory where Jest should store its cached dependency information
// cacheDirectory: "/tmp/jest_rt",
// Automatically clear mock calls, instances and results before every test
// clearMocks: true,
// Indicates whether the coverage information should be collected while executing the test
// collectCoverage: true,
// An array of glob patterns indicating a set of files for which coverage information should be collected
// collectCoverageFrom: undefined,
// The directory where Jest should output its coverage files
// coverageDirectory: "coverage",
// An array of regexp pattern strings used to skip coverage collection
// coveragePathIgnorePatterns: [
// "/node_modules/"
// ],
// Indicates which provider should be used to instrument code for coverage
// coverageProvider: "v8",
// A list of reporter names that Jest uses when writing coverage reports
// coverageReporters: [
// "json",
// "text",
// "lcov",
// "clover"
// ],
// An object that configures minimum threshold enforcement for coverage results
// coverageThreshold: undefined,
// A path to a custom dependency extractor
// dependencyExtractor: undefined,
// Make calling deprecated APIs throw helpful error messages
// errorOnDeprecated: false,
// Force coverage collection from ignored files using an array of glob patterns
// forceCoverageMatch: [],
// A path to a module which exports an async function that is triggered once before all test suites
// globalSetup: undefined,
// A path to a module which exports an async function that is triggered once after all test suites
// globalTeardown: undefined,
// A set of global variables that need to be available in all test environments
// globals: {},
// The maximum amount of workers used to run your tests. Can be specified as % or a number. E.g. maxWorkers: 10% will use 10% of your CPU amount + 1 as the maximum worker number. maxWorkers: 2 will use a maximum of 2 workers.
// maxWorkers: "50%",
// An array of directory names to be searched recursively up from the requiring module's location
// moduleDirectories: [
// "node_modules"
// ],
// An array of file extensions your modules use
// moduleFileExtensions: [
// "js",
// "jsx",
// "ts",
// "tsx",
// "json",
// "node"
// ],
// A map from regular expressions to module names or to arrays of module names that allow to stub out resources with a single module
// moduleNameMapper: {},
// An array of regexp pattern strings, matched against all module paths before considered 'visible' to the module loader
// modulePathIgnorePatterns: [],
// Activates notifications for test results
// notify: false,
// An enum that specifies notification mode. Requires { notify: true }
// notifyMode: "failure-change",
// A preset that is used as a base for Jest's configuration
// preset: undefined,
// Run tests from one or more projects
// projects: undefined,
// Use this configuration option to add custom reporters to Jest
// reporters: undefined,
// Automatically reset mock state before every test
// resetMocks: false,
// Reset the module registry before running each individual test
// resetModules: false,
// A path to a custom resolver
// resolver: undefined,
// Automatically restore mock state and implementation before every test
// restoreMocks: false,
// The root directory that Jest should scan for tests and modules within
// rootDir: undefined,
// A list of paths to directories that Jest should use to search for files in
// roots: [
// "<rootDir>"
// ],
// Allows you to use a custom runner instead of Jest's default test runner
// runner: "jest-runner",
// The paths to modules that run some code to configure or set up the testing environment before each test
// setupFiles: [],
// A list of paths to modules that run some code to configure or set up the testing framework before each test
// setupFilesAfterEnv: [],
// The number of seconds after which a test is considered as slow and reported as such in the results.
// slowTestThreshold: 5,
// A list of paths to snapshot serializer modules Jest should use for snapshot testing
// snapshotSerializers: [],
// The test environment that will be used for testing
// testEnvironment: "jest-environment-node",
// Options that will be passed to the testEnvironment
// testEnvironmentOptions: {},
// Adds a location field to test results
// testLocationInResults: false,
// The glob patterns Jest uses to detect test files
// testMatch: [
// "**/__tests__/**/*.[jt]s?(x)",
// "**/?(*.)+(spec|test).[tj]s?(x)"
// ],
testMatch: [
"**/test/cases/*.[jt]s?(x)"
],
// An array of regexp pattern strings that are matched against all test paths, matched tests are skipped
// testPathIgnorePatterns: [
// "/node_modules/"
// ],
// The regexp pattern or array of patterns that Jest uses to detect test files
// testRegex: [],
// This option allows the use of a custom results processor
// testResultsProcessor: undefined,
// This option allows use of a custom test runner
// testRunner: "jest-circus/runner",
// This option sets the URL for the jsdom environment. It is reflected in properties such as location.href
// testURL: "http://localhost",
// Setting this value to "fake" allows the use of fake timers for functions such as "setTimeout"
// timers: "real",
// A map from regular expressions to paths to transformers
// transform: undefined,
// An array of regexp pattern strings that are matched against all source file paths, matched files will skip transformation
// transformIgnorePatterns: [
// "/node_modules/",
// "\\.pnp\\.[^\\/]+$"
// ],
// An array of regexp pattern strings that are matched against all modules before the module loader will automatically return a mock for them
// unmockedModulePathPatterns: undefined,
// Indicates whether each individual test should be reported during the run
// verbose: undefined,
// An array of regexp patterns that are matched against all source file paths before re-running tests in watch mode
// watchPathIgnorePatterns: [],
// Whether to use watchman for file crawling
// watchman: true,
};
src/connector/nodejs/nodetaos/taosquery.js
浏览文件 @ adf429f8
......@@ -46,7 +46,6 @@ TaosQuery.prototype.execute = async function execute() {
reject(err);
}
resolve(result)
});
return executionPromise;
}
......
src/connector/nodejs/package.json
浏览文件 @ adf429f8
......@@ -7,7 +7,8 @@
"test": "test"
},
"scripts": {
"test": "node test/test.js && node test/testMicroseconds.js && node test/testNanoseconds.js && node test/testUnsignedType.js && node test/testSchemalessInsert.js && node test/testJsonTag.js"
"test": "jest",
"catalog": "jest --json"
},
"repository": {
"type": "git",
......@@ -31,5 +32,8 @@
"ref-array-napi": "^1.2.1",
"ref-napi": "^1.5.2",
"ref-struct-napi": "^1.1.1"
},
"devDependencies": {
"jest": "^27.4.7"
}
}
src/connector/nodejs/test/cases/test.cases.js 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
src/connector/nodejs/test/utils/utilTools.js 0 → 100644
浏览文件 @ adf429f8
/**
* This is an util function will return the column info based on the create sql.
* @param {*} sql
* @returns Return an Array about the column names and column type.
*
*/
function getFeildsFromDll(sql) {
let fields = [];
let firstBracket = sql.indexOf('(');
let lastBracket = sql.lastIndexOf(')')
let metaStr = sql.slice(firstBracket, lastBracket + 1);
let splitTags = metaStr.split("tags");
splitTags.forEach((item, index, arr) => {
arr[index] = item.slice(1, item.length - 1)
})
splitTags.forEach((item) => {
let tmp = item.split(",");
tmp.forEach((item) => {
let newItem = item.trim();
let spaceInd = newItem.indexOf(' ', 1)
fields.push(newItem.slice(0, spaceInd));
fields.push(newItem.slice(spaceInd + 1, newItem.length))
})
})
return fields;
}
/**
* Based on the input array, it will generate sql that could be used to insert the data of array into the db.
* @param {*} tableName It could be the table name that you want to insert data.
* @param {*} stable If you want to using stable as template to create table automatically,
* set this to your stable name. Deault if '';
* @param {*} dataArr An Array of data that you want insert (it could be mutilple lines)
* @param {*} tagArr An Array used to store one sub table's tag info
* @param {*} numOfColumn The number of columns that the target table has.
* @returns Return an insert sql string.
*/
function buildInsertSql(tableName, stable = '', dataArr, tagArr = [], numOfColumn) {
let insertSql = "";
let dataPartial = "(";
let tagPart = "(";
dataArr.forEach((item, index) => {
// let item = dataArr[index];
if (typeof item == "string") {
dataPartial += '\'' + item + '\'';
} else {
dataPartial += item;
}
if ((index + 1) % numOfColumn == 0 && (index + 1) != dataArr.length) {
dataPartial += ")("
} else if ((index + 1) % numOfColumn == 0 && (index + 1) == dataArr.length) {
dataPartial += ")"
} else {
dataPartial += ","
}
})
if (stable != '') {
tagArr.forEach((item, index) => {
if (typeof item == "string") {
tagPart += '\'' + item + '\'';
} else {
tagPart += item;
}
if (index != tagArr.length - 1) {
tagPart += ",";
} else {
tagPart += ")";
}
})
}
if (stable == '') {
insertSql += `insert into ${tableName} values ${dataPartial};`
} else {
insertSql += `insert into ${tableName} using ${stable} tags ${tagPart} values ${dataPartial};`
}
return insertSql;
}
/**
* used to mapping the data type of an create clause into TDengine's datatype code
*/
const TDengineTypeCode = {
'null': 0,
'bool': 1,
'tinyint': 2,
'smallint': 3,
'int': 4,
'bigint': 5,
'float': 6,
'double': 7,
'binary': 8,
'timestamp': 9,
'nchar': 10,
'tinyint unsigned': 11,
'smallint unsigned': 12,
'int unsigned': 13,
'bigint unsigned': 14,
'json': 15,
}
/**
* Mapping the data type with corresponing size that has defined in tdengine
*/
const TDengineTypeBytes = {
'null': 0,
'bool': 1,
'tinyint': 1,
'smallint': 2,
'int': 4,
'bigint': 8,
'float': 4,
'double': 8,
'timestamp': 8,
'tinyint unsigned': 1,
'smallint unsigned': 2,
'int unsigned': 4,
'bigint unsigned': 8,
'json': 4096,
}
/**
* Used to create an array of taos feilds object.
* @param {*} arr This should be the return array from the method getFeildsFromDll()
* @returns Return an array of taosFeild Object
*/
function getFieldArr(arr) {
let feild = [];
for (let i = 0; i < arr.length;) {
let bracetPosi = arr[i + 1].indexOf('(');
let type = '';
let size = -1;
if (bracetPosi == -1) {
type = TDengineTypeCode[arr[i + 1]];
size = TDengineTypeBytes[arr[i + 1]];
}else{
type = TDengineTypeCode[arr[i + 1].slice(0, bracetPosi)];
size = Number(arr[i + 1].slice(bracetPosi + 1, arr[i + 1].indexOf(')')));
}
let fieldObj = {
name: arr[i].toLowerCase(),
type: type,
bytes: size
}
feild.push(fieldObj);
i = i + 2;
}
return feild;
}
/**
* Conbine arrays of data info and tag info together, and return a new array. This array construction is simmilar with query result
* from the tdengine by taos shell.This method only can be used by a subtable.
* @param {*} dataArr An array holds columns' data that will be insert into the db.
* @param {*} tagArr An array holds tags' data that is belong to a sub table.
* @param {*} numOfcolumn
* @returns return the an array of column data and tag data.
*/
function getResData(dataArr, tagArr, numOfcolumn) {
let resData = [];
dataArr.forEach((item, index) => {
resData.push(item);
if ((index + 1) % numOfcolumn == 0) {
tagArr.forEach((element) => {
resData.push(element);
}) ;
}
});
return resData;
}
module.exports = { getFeildsFromDll, buildInsertSql, getFieldArr, getResData };
\ No newline at end of file
src/dnode/src/dnodeVnodes.c
浏览文件 @ adf429f8
......@@ -287,7 +287,7 @@ static void dnodeSendStatusMsg(void *handle, void *tmrId) {
dnodeGetCfg(&pStatus->dnodeId, pStatus->clusterId);
pStatus->dnodeId = htonl(dnodeGetDnodeId());
pStatus->version = htonl(tsVersion);
pStatus->version = htonl(tsVersion >> 8);
pStatus->lastReboot = htonl(tsRebootTime);
pStatus->numOfCores = htons((uint16_t) tsNumOfCores);
pStatus->diskAvailable = tsAvailDataDirGB;
......
src/inc/taos.h
浏览文件 @ adf429f8
......@@ -179,6 +179,7 @@ DLL_EXPORT bool taos_is_null(TAOS_RES *res, int32_t row, int32_t col);
DLL_EXPORT bool taos_is_update_query(TAOS_RES *res);
DLL_EXPORT int taos_fetch_block(TAOS_RES *res, TAOS_ROW *rows);
DLL_EXPORT int* taos_fetch_lengths(TAOS_RES *res);
DLL_EXPORT TAOS_ROW *taos_result_block(TAOS_RES *res);
DLL_EXPORT int taos_validate_sql(TAOS *taos, const char *sql);
DLL_EXPORT void taos_reset_current_db(TAOS *taos);
......
src/inc/taosdef.h
浏览文件 @ adf429f8
......@@ -108,8 +108,8 @@ extern const int32_t TYPE_BYTES[16];
#define TSDB_ERR -1
#define TS_PATH_DELIMITER "."
#define TS_ESCAPE_CHAR '`'
#define TS_ESCAPE_CHAR_SIZE 2
#define TS_BACKQUOTE_CHAR '`'
#define TS_BACKQUOTE_CHAR_SIZE 2
#define TSDB_TIME_PRECISION_MILLI 0
#define TSDB_TIME_PRECISION_MICRO 1
......
src/inc/taoserror.h
浏览文件 @ adf429f8
......@@ -115,6 +115,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_TSC_INVALID_PROTOCOL_TYPE TAOS_DEF_ERROR_CODE(0, 0x0225) //"Invalid line protocol type")
#define TSDB_CODE_TSC_INVALID_PRECISION_TYPE TAOS_DEF_ERROR_CODE(0, 0x0226) //"Invalid timestamp precision type")
#define TSDB_CODE_TSC_RES_TOO_MANY TAOS_DEF_ERROR_CODE(0, 0x0227) //"Result set too large to be output")
#define TSDB_CODE_TSC_INVALID_SCHEMA_VERSION TAOS_DEF_ERROR_CODE(0, 0x0228) //"invalid table schema version")
// mnode
#define TSDB_CODE_MND_MSG_NOT_PROCESSED TAOS_DEF_ERROR_CODE(0, 0x0300) //"Message not processed"
......@@ -291,6 +292,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_QRY_INCONSISTAN TAOS_DEF_ERROR_CODE(0, 0x070C) //"File inconsistency in replica")
#define TSDB_CODE_QRY_SYS_ERROR TAOS_DEF_ERROR_CODE(0, 0x070D) //"System error")
#define TSDB_CODE_QRY_INVALID_TIME_CONDITION TAOS_DEF_ERROR_CODE(0, 0x070E) //"invalid time condition")
#define TSDB_CODE_QRY_INVALID_SCHEMA_VERSION TAOS_DEF_ERROR_CODE(0, 0x0710) //"invalid schema version")
// grant
#define TSDB_CODE_GRANT_EXPIRED TAOS_DEF_ERROR_CODE(0, 0x0800) //"License expired"
......
src/inc/taosmsg.h
浏览文件 @ adf429f8
......@@ -978,7 +978,9 @@ typedef struct {
} STLV;
enum {
TLV_TYPE_DUMMY = 1,
TLV_TYPE_END_MARK = -1,
//TLV_TYPE_DUMMY = 1,
TLV_TYPE_META_VERSION = 1,
};
#pragma pack(pop)
......
src/inc/tsdb.h
浏览文件 @ adf429f8
......@@ -229,6 +229,8 @@ typedef struct {
uint32_t numOfTables;
SArray *pGroupList;
SHashObj *map; // speedup acquire the tableQueryInfo by table uid
int32_t sVersion;
int32_t tVersion;
} STableGroupInfo;
#define TSDB_BLOCK_DIST_STEP_ROWS 16
......
src/kit/shell/src/shellCommand.c
浏览文件 @ adf429f8
......@@ -51,8 +51,8 @@ void getPrevCharSize(const char *str, int pos, int *size, int *width) {
if (str[pos] > 0 || countPrefixOnes((unsigned char )str[pos]) > 1) break;
}
int rc = mbtowc(&wc, str + pos, MB_CUR_MAX);
assert(rc == *size);
mbtowc(&wc, str + pos, MB_CUR_MAX);
// assert(rc == *size); // it will be core, if str is encode by utf8 and taos charset is gbk
*width = wcwidth(wc);
}
......
src/kit/shell/src/shellEngine.c
浏览文件 @ adf429f8
......@@ -81,9 +81,9 @@ extern TAOS *taos_connect_auth(const char *ip, const char *user, const char *aut
TAOS *shellInit(SShellArguments *_args) {
printf("\n");
if (!_args->is_use_passwd) {
#ifdef TD_WINDOWS
#ifdef WINDOWS
strcpy(tsOsName, "Windows");
#elif defined(TD_DARWIN)
#elif defined(DARWIN)
strcpy(tsOsName, "Darwin");
#endif
printf(CLIENT_VERSION, tsOsName, taos_get_client_info());
......@@ -239,64 +239,27 @@ int32_t shellRunCommand(TAOS* con, char* command) {
}
}
bool esc = false;
char quote = 0, *cmd = command, *p = command;
char quote = 0, *cmd = command;
for (char c = *command++; c != 0; c = *command++) {
if (esc) {
switch (c) {
case 'n':
c = '\n';
break;
case 'r':
c = '\r';
break;
case 't':
c = '\t';
break;
case 'G':
*p++ = '\\';
break;
case '\'':
case '"':
case '`':
if (quote) {
*p++ = '\\';
}
break;
}
*p++ = c;
esc = false;
if (c == '\\' && (*command == '\'' || *command == '"' || *command == '`')) {
command ++;
continue;
}
if (c == '\\') {
if (quote != 0 && (*command == '_' || *command == '%' || *command == '\\')) {
//DO nothing
} else {
esc = true;
continue;
}
}
if (quote == c) {
quote = 0;
} else if (quote == 0 && (c == '\'' || c == '"' || c == '`')) {
quote = c;
}
*p++ = c;
if (c == ';' && quote == 0) {
c = *p;
*p = 0;
} else if (c == ';' && quote == 0) {
c = *command;
*command = 0;
if (shellRunSingleCommand(con, cmd) < 0) {
return -1;
}
*p = c;
p = cmd;
*command = c;
cmd = command;
}
}
*p = 0;
return shellRunSingleCommand(con, cmd);
}
......@@ -411,7 +374,14 @@ int regex_match(const char *s, const char *reg, int cflags) {
} else if (reti == REG_NOMATCH) {
regfree(&regex);
return 0;
} else {
}
#ifdef DARWIN
else if (reti == REG_ILLSEQ){
regfree(&regex);
return 0;
}
#endif
else {
regerror(reti, &regex, msgbuf, sizeof(msgbuf));
fprintf(stderr, "Regex match failed: %s\n", msgbuf);
regfree(&regex);
......@@ -512,15 +482,27 @@ static void dumpFieldToFile(FILE* fp, const char* val, TAOS_FIELD* field, int32_
case TSDB_DATA_TYPE_TINYINT:
fprintf(fp, "%d", *((int8_t *)val));
break;
case TSDB_DATA_TYPE_UTINYINT:
fprintf(fp, "%u", *((uint8_t *)val));
break;
case TSDB_DATA_TYPE_SMALLINT:
fprintf(fp, "%d", *((int16_t *)val));
break;
case TSDB_DATA_TYPE_USMALLINT:
fprintf(fp, "%u", *((uint16_t *)val));
break;
case TSDB_DATA_TYPE_INT:
fprintf(fp, "%d", *((int32_t *)val));
break;
case TSDB_DATA_TYPE_UINT:
fprintf(fp, "%u", *((uint32_t *)val));
break;
case TSDB_DATA_TYPE_BIGINT:
fprintf(fp, "%" PRId64, *((int64_t *)val));
break;
case TSDB_DATA_TYPE_UBIGINT:
fprintf(fp, "%" PRIu64, *((uint64_t *)val));
break;
case TSDB_DATA_TYPE_FLOAT:
fprintf(fp, "%.5f", GET_FLOAT_VAL(val));
break;
......@@ -609,20 +591,25 @@ static void shellPrintNChar(const char *str, int length, int width) {
if (bytes <= 0) {
break;
}
pos += bytes;
if (pos > length) {
break;
}
int w = 0;
#ifdef WINDOWS
int w = bytes;
w = bytes;
#else
int w = wcwidth(wc);
if(*(str + pos) == '\t' || *(str + pos) == '\n' || *(str + pos) == '\r'){
w = bytes;
}else{
w = wcwidth(wc);
}
#endif
if (w <= 0) {
continue;
}
pos += bytes;
if (pos > length) {
break;
}
if (width <= 0) {
printf("%lc", wc);
continue;
......
taos-tools @ dbc67af5
比较 59f00a69...dbc67af5
Subproject commit 59f00a69f36b08cea86a70a22c29b2c27ef506ae
Subproject commit dbc67af5f5a764e4c7950923e4e08741a7874168
src/mnode/src/mnodeDnode.c
浏览文件 @ adf429f8
......@@ -530,7 +530,7 @@ static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
pStatus->numOfCores = htons(pStatus->numOfCores);
uint32_t _version = htonl(pStatus->version);
if (_version != tsVersion) {
if (_version != tsVersion >> 8) {
pDnode = mnodeGetDnodeByEp(pStatus->dnodeEp);
if (pDnode != NULL && pDnode->status != TAOS_DN_STATUS_READY) {
pDnode->offlineReason = TAOS_DN_OFF_VERSION_NOT_MATCH;
......
src/mnode/src/mnodeVgroup.c
浏览文件 @ adf429f8
......@@ -903,6 +903,8 @@ static SCreateVnodeMsg *mnodeBuildVnodeMsg(SVgObj *pVgroup) {
SDbObj *pDb = pVgroup->pDb;
if (pDb == NULL) return NULL;
if (pVgroup->idPool == NULL) return NULL;
SCreateVnodeMsg *pVnode = rpcMallocCont(sizeof(SCreateVnodeMsg));
if (pVnode == NULL) return NULL;
......@@ -1057,6 +1059,11 @@ void mnodeSendCompactVgroupMsg(SVgObj *pVgroup) {
}
static void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcEpSet *epSet, void *ahandle) {
SCreateVnodeMsg *pCreate = mnodeBuildVnodeMsg(pVgroup);
if (pCreate == NULL) {
mError("vgId: %d, can not create vnode msg for send create vnode", pVgroup->vgId);
return;
}
SRpcMsg rpcMsg = {
.ahandle = ahandle,
.pCont = pCreate,
......
src/os/src/detail/osFile.c
浏览文件 @ adf429f8
......@@ -370,8 +370,11 @@ int32_t taosFsync(FileFd fd) {
}
HANDLE h = (HANDLE)_get_osfhandle(fd);
return FlushFileBuffers(h);
//If the function succeeds, the return value is nonzero.
//If the function fails, the return value is zero. To get extended error information, call GetLastError.
//The function fails if hFile is a handle to the console output. That is because the console output is not buffered. The function returns FALSE, and GetLastError returns ERROR_INVALID_HANDLE.
return FlushFileBuffers(h)-1;
}
int32_t taosRename(char *oldName, char *newName) {
......
src/os/src/windows/wSysinfo.c
浏览文件 @ adf429f8
......@@ -120,7 +120,7 @@ static void taosGetSystemLocale() {
SGlobalCfg *cfg_charset = taosGetConfigOption("charset");
if (cfg_charset && cfg_charset->cfgStatus < TAOS_CFG_CSTATUS_DEFAULT) {
strcpy(tsCharset, "cp936");
strcpy(tsCharset, "UTF-8");
cfg_charset->cfgStatus = TAOS_CFG_CSTATUS_DEFAULT;
uInfo("charset not configured, set to default:%s", tsCharset);
}
......
src/plugins/CMakeLists.txt
浏览文件 @ adf429f8
......@@ -13,6 +13,22 @@ ELSEIF(TD_BUILD_TAOSA_INTERNAL)
ELSE ()
MESSAGE("")
MESSAGE("${Green} use taosadapter as httpd ${ColourReset}")
EXECUTE_PROCESS(
COMMAND git rev-parse --abbrev-ref HEAD
RESULT_VARIABLE result_taos_version
OUTPUT_VARIABLE taos_version
)
STRING(FIND ${taos_version} release is_release_branch)
IF ("${is_release_branch}" STREQUAL "0")
STRING(SUBSTRING "${taos_version}" 12 -1 taos_version)
STRING(STRIP "${taos_version}" taos_version)
ELSE ()
STRING(CONCAT taos_version "branch_" "${taos_version}")
STRING(STRIP "${taos_version}" taos_version)
ENDIF ()
EXECUTE_PROCESS(
COMMAND cd ${CMAKE_CURRENT_SOURCE_DIR}/taosadapter
)
......@@ -22,12 +38,12 @@ ELSE ()
OUTPUT_VARIABLE taosadapter_commit_sha1
)
IF ("${taosadapter_commit_sha1}" STREQUAL "")
SET(taosadapter_commit_sha1 "unknown")
SET(taosadapter_commit_sha1 "unknown")
ELSE ()
STRING(SUBSTRING "${taosadapter_commit_sha1}" 0 7 taosadapter_commit_sha1)
STRING(STRIP "${taosadapter_commit_sha1}" taosadapter_commit_sha1)
STRING(SUBSTRING "${taosadapter_commit_sha1}" 0 7 taosadapter_commit_sha1)
STRING(STRIP "${taosadapter_commit_sha1}" taosadapter_commit_sha1)
ENDIF ()
MESSAGE("${Green} taosadapter commit: ${taosadapter_commit_sha1} ${ColourReset}")
MESSAGE("${Green} taosAdapter will use ${taos_version} and commit ${taosadapter_commit_sha1} as version ${ColourReset}")
EXECUTE_PROCESS(
COMMAND cd ..
)
......@@ -43,7 +59,7 @@ ELSE ()
CONFIGURE_COMMAND cmake -E echo "taosadapter no need cmake to config"
PATCH_COMMAND
COMMAND git clean -f -d
BUILD_COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../inc CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -ldflags "-s -w -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}"
BUILD_COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../inc CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}"
INSTALL_COMMAND
COMMAND curl -sL https://github.com/upx/upx/releases/download/v3.96/upx-3.96-amd64_linux.tar.xz -o upx.tar.xz && tar -xvJf upx.tar.xz -C ${CMAKE_BINARY_DIR} --strip-components 1 > /dev/null && ${CMAKE_BINARY_DIR}/upx taosadapter || :
COMMAND cmake -E copy taosadapter ${CMAKE_BINARY_DIR}/build/bin
......@@ -62,7 +78,7 @@ ELSE ()
CONFIGURE_COMMAND cmake -E echo "taosadapter no need cmake to config"
PATCH_COMMAND
COMMAND git clean -f -d
BUILD_COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../inc CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -ldflags "-s -w -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}"
BUILD_COMMAND CGO_CFLAGS=-I${CMAKE_CURRENT_SOURCE_DIR}/../inc CGO_LDFLAGS=-L${CMAKE_BINARY_DIR}/build/lib go build -ldflags "-s -w -X github.com/taosdata/taosadapter/version.Version=${taos_version} -X github.com/taosdata/taosadapter/version.CommitID=${taosadapter_commit_sha1}"
INSTALL_COMMAND
COMMAND cmake -E copy taosadapter ${CMAKE_BINARY_DIR}/build/bin
COMMAND cmake -E make_directory ${CMAKE_BINARY_DIR}/test/cfg/
......
taosadapter @ 8f9501a3
比较 47fb0b3e...8f9501a3
Subproject commit 47fb0b3e627ddadf1ca983c1d75b9a4e44cd98fd
Subproject commit 8f9501a30b1893c6616d644a924c995aa21ad957
src/query/inc/qExecutor.h
浏览文件 @ adf429f8
......@@ -428,6 +428,8 @@ typedef struct SQueryParam {
int32_t tableScanOperator;
SArray *pOperator;
SUdfInfo *pUdfInfo;
int16_t schemaVersion;
int16_t tagVersion;
} SQueryParam;
typedef struct SColumnDataParam{
......
src/query/inc/qScript.h
浏览文件 @ adf429f8
......@@ -25,10 +25,11 @@
#include "tlist.h"
#include "qUdf.h"
#define MAX_FUNC_NAME 64
#define USER_FUNC_NAME "funcName"
#define USER_FUNC_NAME_LIMIT 48
/* define in this way to let others know that these two macros are logically related */
#define MAX_FUNC_NAME (USER_FUNC_NAME_LIMIT + 16)
enum ScriptState {
SCRIPT_STATE_INIT,
......@@ -44,7 +45,9 @@ typedef struct {
} ScriptEnv;
typedef struct ScriptCtx {
char funcName[USER_FUNC_NAME_LIMIT];
// one-more-space-for-null-terminator to support function name
// at most USER_FUNC_NAME_LIMIT bytes long actually
char funcName[USER_FUNC_NAME_LIMIT+1];
int8_t state;
ScriptEnv *pEnv;
int8_t isAgg; // agg function or not
......
src/query/inc/sql.y
浏览文件 @ adf429f8
......@@ -107,7 +107,7 @@ cmd ::= SHOW dbPrefix(X) TABLES. {
setShowOptions(pInfo, TSDB_MGMT_TABLE_TABLE, &X, 0);
}
cmd ::= SHOW dbPrefix(X) TABLES LIKE ids(Y). {
cmd ::= SHOW dbPrefix(X) TABLES LIKE STRING(Y). {
setShowOptions(pInfo, TSDB_MGMT_TABLE_TABLE, &X, &Y);
}
......@@ -115,7 +115,7 @@ cmd ::= SHOW dbPrefix(X) STABLES. {
setShowOptions(pInfo, TSDB_MGMT_TABLE_METRIC, &X, 0);
}
cmd ::= SHOW dbPrefix(X) STABLES LIKE ids(Y). {
cmd ::= SHOW dbPrefix(X) STABLES LIKE STRING(Y). {
SStrToken token;
tSetDbName(&token, &X);
setShowOptions(pInfo, TSDB_MGMT_TABLE_METRIC, &token, &Y);
......
src/query/src/qAggMain.c
浏览文件 @ adf429f8
......@@ -4618,9 +4618,7 @@ static void mavg_function(SQLFunctionCtx *pCtx) {
}
}
if (notNullElems <= 0) {
assert(pCtx->hasNull);
} else {
{
for (int t = 0; t < pCtx->tagInfo.numOfTagCols; ++t) {
SQLFunctionCtx* tagCtx = pCtx->tagInfo.pTagCtxList[t];
if (tagCtx->functionId == TSDB_FUNC_TAG_DUMMY) {
......@@ -5119,7 +5117,7 @@ SAggFunctionInfo aAggs[40] = {{
"twa",
TSDB_FUNC_TWA,
TSDB_FUNC_TWA,
TSDB_BASE_FUNC_SO | TSDB_FUNCSTATE_NEED_TS,
TSDB_FUNCSTATE_SO | TSDB_FUNCSTATE_STABLE | TSDB_FUNCSTATE_NEED_TS,
twa_function_setup,
twa_function,
twa_function_finalizer,
......@@ -5395,7 +5393,7 @@ SAggFunctionInfo aAggs[40] = {{
"elapsed",
TSDB_FUNC_ELAPSED,
TSDB_FUNC_ELAPSED,
TSDB_BASE_FUNC_SO,
TSDB_FUNCSTATE_SO | TSDB_FUNCSTATE_STABLE,
elapsedSetup,
elapsedFunction,
elapsedFinalizer,
......
src/query/src/qExecutor.c
浏览文件 @ adf429f8
此差异已折叠。
src/query/src/qFill.c
浏览文件 @ adf429f8
......@@ -354,6 +354,10 @@ SFillInfo* taosCreateFillInfo(int32_t order, TSKEY skey, int32_t numOfTags, int3
}
SFillInfo* pFillInfo = calloc(1, sizeof(SFillInfo));
if (pFillInfo == NULL) {
return NULL;
}
taosResetFillInfo(pFillInfo, skey);
pFillInfo->order = order;
......@@ -371,6 +375,10 @@ SFillInfo* taosCreateFillInfo(int32_t order, TSKEY skey, int32_t numOfTags, int3
pFillInfo->interval.slidingUnit = slidingUnit;
pFillInfo->pData = malloc(POINTER_BYTES * numOfCols);
if (pFillInfo->pData == NULL) {
tfree(pFillInfo);
return NULL;
}
// if (numOfTags > 0) {
pFillInfo->pTags = calloc(numOfCols, sizeof(SFillTagColInfo));
......
src/query/src/qFilter.c
浏览文件 @ adf429f8
......@@ -3585,6 +3585,10 @@ _return:
int32_t filterConverNcharColumns(SFilterInfo* info, int32_t rows, bool *gotNchar) {
if (FILTER_EMPTY_RES(info) || FILTER_ALL_RES(info)) {
return TSDB_CODE_SUCCESS;
}
for (uint32_t i = 0; i < info->fields[FLD_TYPE_COLUMN].num; ++i) {
SFilterField* fi = &info->fields[FLD_TYPE_COLUMN].fields[i];
int32_t type = FILTER_GET_COL_FIELD_TYPE(fi);
......
src/query/src/qScript.c
浏览文件 @ adf429f8
此差异已折叠。
src/query/src/qSqlParser.c
浏览文件 @ adf429f8
......@@ -97,7 +97,7 @@ SArray *tSqlExprListAppend(SArray *pList, tSqlExpr *pNode, SStrToken *pDistinct,
strncpy(item.aliasName, pToken->z, pToken->n);
item.aliasName[pToken->n] = 0;
strdequote(item.aliasName);
stringProcess(item.aliasName, (int32_t)strlen(item.aliasName));
}
taosArrayPush(pList, &item);
......
src/query/src/queryMain.c
浏览文件 @ adf429f8
此差异已折叠。
src/query/src/sql.c
浏览文件 @ adf429f8
此差异已折叠。
src/rpc/src/rpcMain.c
浏览文件 @ adf429f8
......@@ -963,9 +963,14 @@ static SRpcConn *rpcProcessMsgHead(SRpcInfo *pRpc, SRecvInfo *pRecv, SRpcReqCont
terrno = TSDB_CODE_RPC_INVALID_SESSION_ID; return NULL;
}
if (rpcIsReq(pHead->msgType) && htonl(pHead->msgVer) != tsVersion >> 8) {
tDebug("%s sid:%d, invalid client version:%x/%x %s", pRpc->label, sid, htonl(pHead->msgVer), tsVersion, taosMsg[pHead->msgType]);
terrno = TSDB_CODE_RPC_INVALID_VERSION; return NULL;
// compatibility between old version client and new version server, since 2.4.0.0
if (rpcIsReq(pHead->msgType)){
if((htonl(pHead->msgVer) >> 16 != tsVersion >> 24) ||
((htonl(pHead->msgVer) >> 16 == tsVersion >> 24) && htonl(pHead->msgVer) < ((2 << 16) | (4 << 8)))){
tError("%s sid:%d, invalid client version:%x/%x %s", pRpc->label, sid, htonl(pHead->msgVer), tsVersion, taosMsg[pHead->msgType]);
terrno = TSDB_CODE_RPC_INVALID_VERSION;
return NULL;
}
}
pConn = rpcGetConnObj(pRpc, sid, pRecv);
......
src/tsdb/src/tsdbRead.c
浏览文件 @ adf429f8
此差异已折叠。
src/util/inc/tutil.h
浏览文件 @ adf429f8
此差异已折叠。
src/util/src/tcompare.c
浏览文件 @ adf429f8
此差异已折叠。
src/util/src/ttokenizer.c
浏览文件 @ adf429f8
此差异已折叠。
src/util/src/tutil.c
浏览文件 @ adf429f8
此差异已折叠。
src/util/tests/stringTest.cpp
浏览文件 @ adf429f8
此差异已折叠。
tests/develop-test/2-query/TD-6347.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/develop-test/2-query/escape.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/develop-test/2-query/function_mavg.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/develop-test/test.py
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/.gitignore
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/insertCn/Program.cs 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/insertCn/TDengineDriver.cs 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/insertCn/insertCn.csproj 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/insertCn/lib/ResultSetUtils.cs 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/examples/C#/insertCn/lib/Utils.cs 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/dockerCluster/basic.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/fulltest-tools.sh
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/query/queryBase.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/query/queryRegex.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/test.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/util/cases.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/util/dnodes-default.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/util/dnodes.py
浏览文件 @ adf429f8
此差异已折叠。
tests/pytest/wal/addOldWalTest.py
浏览文件 @ adf429f8
此差异已折叠。
tests/requirements.txt 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/script/api/makefile
浏览文件 @ adf429f8
此差异已折叠。
tests/script/api/resultBlock.c 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/TD-12275.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/TD-12861.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/TD-12863.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/TD-12909.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/TD-12926.py 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/2-query/td_12191.json 0 → 100644
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/fulltest-query.sh
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/fulltest-tools.sh
浏览文件 @ adf429f8
此差异已折叠。
tests/system-test/test.py
浏览文件 @ adf429f8
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或