diff --git a/documentation20/cn/08.connector/01.java/docs.md b/documentation20/cn/08.connector/01.java/docs.md index a5f0d21ef82bad7e89d74ed3ac68b378558fbe3f..edd81a49cd334859fd9581a601822ee782bad45a 100644 --- a/documentation20/cn/08.connector/01.java/docs.md +++ b/documentation20/cn/08.connector/01.java/docs.md @@ -46,7 +46,7 @@ TDengine 的 JDBC 驱动实现尽可能与关系型数据库驱动保持一致 -注意：与 JNI 方式不同，RESTful 接口是无状态的。在使用JDBC-RESTful时，需要在sql中指定表、超级表的数据库名称。例如： +注意：与 JNI 方式不同，RESTful 接口是无状态的。在使用JDBC-RESTful时，需要在sql中指定表、超级表的数据库名称。（从 TDengine 2.1.8.0 版本开始，也可以在 RESTful url 中指定当前 SQL 语句所使用的默认数据库名。）例如： ```sql INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(now, 24.6); ``` diff --git a/documentation20/cn/08.connector/docs.md b/documentation20/cn/08.connector/docs.md index 0ac5a91b507b451d33ea692e79e20aaa205677cc..ea3f9a4d55181e319848e1a5e14abb1409a3a5e9 100644 --- a/documentation20/cn/08.connector/docs.md +++ b/documentation20/cn/08.connector/docs.md @@ -654,22 +654,23 @@ conn.close() 为支持各种不同类型平台的开发，TDengine 提供符合 REST 设计标准的 API，即 RESTful API。为最大程度降低学习成本，不同于其他数据库 RESTful API 的设计方法，TDengine 直接通过 HTTP POST 请求 BODY 中包含的 SQL 语句来操作数据库，仅需要一个 URL。RESTful 连接器的使用参见[视频教程](https://www.taosdata.com/blog/2020/11/11/1965.html)。 -注意：与标准连接器的一个区别是，RESTful 接口是无状态的，因此 `USE db_name` 指令没有效果，所有对表名、超级表名的引用都需要指定数据库名前缀。 +注意：与标准连接器的一个区别是，RESTful 接口是无状态的，因此 `USE db_name` 指令没有效果，所有对表名、超级表名的引用都需要指定数据库名前缀。（从 2.1.8.0 版本开始，支持在 RESTful url 中指定 db_name，这时如果 SQL 语句中没有指定数据库名前缀的话，会使用 url 中指定的这个 db_name。） ### 安装 -RESTful接口不依赖于任何TDengine的库，因此客户端不需要安装任何TDengine的库，只要客户端的开发语言支持HTTP协议即可。 +RESTful 接口不依赖于任何 TDengine 的库，因此客户端不需要安装任何 TDengine 的库，只要客户端的开发语言支持 HTTP 协议即可。 ### 验证 -在已经安装TDengine服务器端的情况下，可以按照如下方式进行验证。 +在已经安装 TDengine 服务器端的情况下，可以按照如下方式进行验证。 -下面以Ubuntu环境中使用curl工具（确认已经安装）来验证RESTful接口的正常。 +下面以 Ubuntu 环境中使用 curl 工具（确认已经安装）来验证 RESTful 接口的正常。 -下面示例是列出所有的数据库，请把h1.taosdata.com和6041（缺省值）替换为实际运行的TDengine服务fqdn和端口号： +下面示例是列出所有的数据库，请把 h1.taosdata.com 和 6041（缺省值）替换为实际运行的 TDengine 服务 fqdn 和端口号： ```html curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'show databases;' h1.taosdata.com:6041/rest/sql ``` + 返回值结果如下表示验证通过： ```json { @@ -682,22 +683,23 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'show databases;' h1.taos } ``` -### RESTful连接器的使用 +### RESTful 连接器的使用 -#### HTTP请求格式 +#### HTTP 请求格式 ``` -http://:/rest/sql +http://:/rest/sql/[db_name] ``` 参数说明： -- fqnd: 集群中的任一台主机FQDN或IP地址 -- port: 配置文件中httpPort配置项，缺省为6041 +- fqnd: 集群中的任一台主机 FQDN 或 IP 地址 +- port: 配置文件中 httpPort 配置项，缺省为 6041 +- db_name: 可选参数，指定本次所执行的 SQL 语句的默认数据库库名。（从 2.1.8.0 版本开始支持） -例如：http://h1.taos.com:6041/rest/sql 是指向地址为h1.taos.com:6041的url。 +例如：http://h1.taos.com:6041/rest/sql/test 是指向地址为 h1.taos.com:6041 的 url，并将默认使用的数据库库名设置为 test。 -HTTP请求的Header里需带有身份认证信息，TDengine支持Basic认证与自定义认证两种机制，后续版本将提供标准安全的数字签名机制来做身份验证。 +HTTP 请求的 Header 里需带有身份认证信息，TDengine 支持 Basic 认证与自定义认证两种机制，后续版本将提供标准安全的数字签名机制来做身份验证。 - 自定义身份认证信息如下所示（稍后介绍） @@ -711,25 +713,25 @@ Authorization: Taosd Authorization: Basic ``` -HTTP请求的BODY里就是一个完整的SQL语句，SQL语句中的数据表应提供数据库前缀，例如\.\。如果表名不带数据库前缀，系统会返回错误。因为HTTP模块只是一个简单的转发，没有当前DB的概念。 +HTTP 请求的 BODY 里就是一个完整的 SQL 语句，SQL 语句中的数据表应提供数据库前缀，例如 \.\。如果表名不带数据库前缀，又没有在 url 中指定数据库名的话，系统会返回错误。因为 HTTP 模块只是一个简单的转发，没有当前 DB 的概念。 -使用curl通过自定义身份认证方式来发起一个HTTP Request，语法如下： +使用 curl 通过自定义身份认证方式来发起一个 HTTP Request，语法如下： ```bash -curl -H 'Authorization: Basic ' -d '' :/rest/sql +curl -H 'Authorization: Basic ' -d '' :/rest/sql/[db_name] ``` 或者 ```bash -curl -u username:password -d '' :/rest/sql +curl -u username:password -d '' :/rest/sql/[db_name] ``` -其中，`TOKEN`为`{username}:{password}`经过Base64编码之后的字符串，例如`root:taosdata`编码后为`cm9vdDp0YW9zZGF0YQ==` +其中，`TOKEN` 为 `{username}:{password}` 经过 Base64 编码之后的字符串，例如 `root:taosdata` 编码后为 `cm9vdDp0YW9zZGF0YQ==` -### HTTP返回格式 +### HTTP 返回格式 -返回值为JSON格式，如下: +返回值为 JSON 格式，如下: ```json { @@ -747,9 +749,9 @@ curl -u username:password -d '' :/rest/sql 说明： - status: 告知操作结果是成功还是失败。 -- head: 表的定义，如果不返回结果集，则仅有一列“affected_rows”。（从 2.0.17.0 版本开始，建议不要依赖 head 返回值来判断数据列类型，而推荐使用 column_meta。在未来版本中，有可能会从返回值中去掉 head 这一项。） +- head: 表的定义，如果不返回结果集，则仅有一列 “affected_rows”。（从 2.0.17.0 版本开始，建议不要依赖 head 返回值来判断数据列类型，而推荐使用 column_meta。在未来版本中，有可能会从返回值中去掉 head 这一项。） - column_meta: 从 2.0.17.0 版本开始，返回值中增加这一项来说明 data 里每一列的数据类型。具体每个列会用三个值来说明，分别为：列名、列类型、类型长度。例如`["current",6,4]`表示列名为“current”；列类型为 6，也即 float 类型；类型长度为 4，也即对应 4 个字节表示的 float。如果列类型为 binary 或 nchar，则类型长度表示该列最多可以保存的内容长度，而不是本次返回值中的具体数据长度。当列类型是 nchar 的时候，其类型长度表示可以保存的 unicode 字符数量，而不是 bytes。 -- data: 具体返回的数据，一行一行的呈现，如果不返回结果集，那么就仅有[[affected_rows]]。data 中每一行的数据列顺序，与 column_meta 中描述数据列的顺序完全一致。 +- data: 具体返回的数据，一行一行的呈现，如果不返回结果集，那么就仅有 [[affected_rows]]。data 中每一行的数据列顺序，与 column_meta 中描述数据列的顺序完全一致。 - rows: 表明总共多少行数据。 column_meta 中的列类型说明： @@ -766,13 +768,13 @@ column_meta 中的列类型说明： ### 自定义授权码 -HTTP请求中需要带有授权码``，用于身份识别。授权码通常由管理员提供，可简单的通过发送`HTTP GET`请求来获取授权码，操作如下： +HTTP 请求中需要带有授权码 ``，用于身份识别。授权码通常由管理员提供，可简单的通过发送 `HTTP GET` 请求来获取授权码，操作如下： ```bash curl http://:/rest/login// ``` -其中，`fqdn`是TDengine数据库的fqdn或ip地址，port是TDengine服务的端口号，`username`为数据库用户名，`password`为数据库密码，返回值为`JSON`格式，各字段含义如下： +其中，`fqdn` 是 TDengine 数据库的 fqdn 或 ip 地址，port 是 TDengine 服务的端口号，`username` 为数据库用户名，`password` 为数据库密码，返回值为 `JSON` 格式，各字段含义如下： - status：请求结果的标志位 @@ -798,7 +800,7 @@ curl http://192.168.0.1:6041/rest/login/root/taosdata ### 使用示例 -- 在demo库里查询表d1001的所有记录： +- 在 demo 库里查询表 d1001 的所有记录： ```bash curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001' 192.168.0.1:6041/rest/sql @@ -818,7 +820,7 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001 } ``` -- 创建库demo： +- 创建库 demo： ```bash curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'create database demo' 192.168.0.1:6041/rest/sql @@ -837,9 +839,9 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'create database demo' 19 ### 其他用法 -#### 结果集采用Unix时间戳 +#### 结果集采用 Unix 时间戳 -HTTP请求URL采用`sqlt`时，返回结果集的时间戳将采用Unix时间戳格式表示，例如 +HTTP 请求 URL 采用 `sqlt` 时，返回结果集的时间戳将采用 Unix 时间戳格式表示，例如 ```bash curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001' 192.168.0.1:6041/rest/sqlt @@ -860,9 +862,9 @@ curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.d1001 } ``` -#### 结果集采用UTC时间字符串 +#### 结果集采用 UTC 时间字符串 -HTTP请求URL采用`sqlutc`时，返回结果集的时间戳将采用UTC时间字符串表示，例如 +HTTP 请求 URL 采用 `sqlutc` 时，返回结果集的时间戳将采用 UTC 时间字符串表示，例如 ```bash curl -H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' -d 'select * from demo.t1' 192.168.0.1:6041/rest/sqlutc ``` @@ -884,13 +886,14 @@ HTTP请求URL采用`sqlutc`时，返回结果集的时间戳将采用UTC时间 ### 重要配置项 -下面仅列出一些与RESTful接口有关的配置参数，其他系统参数请看配置文件里的说明。（注意：配置修改后，需要重启taosd服务才能生效） +下面仅列出一些与 RESTful 接口有关的配置参数，其他系统参数请看配置文件里的说明。（注意：配置修改后，需要重启 taosd 服务才能生效） -- 对外提供RESTful服务的端口号，默认绑定到 6041（实际取值是 serverPort + 11，因此可以通过修改 serverPort 参数的设置来修改） -- httpMaxThreads: 启动的线程数量，默认为2（2.0.17.0版本开始，默认值改为CPU核数的一半向下取整） -- restfulRowLimit: 返回结果集（JSON格式）的最大条数，默认值为10240 -- httpEnableCompress: 是否支持压缩，默认不支持，目前TDengine仅支持gzip压缩格式 -- httpDebugFlag: 日志开关，默认131。131：仅错误和报警信息，135：调试信息，143：非常详细的调试信息，默认131 +- 对外提供 RESTful 服务的端口号，默认绑定到 6041（实际取值是 serverPort + 11，因此可以通过修改 serverPort 参数的设置来修改）。 +- httpMaxThreads: 启动的线程数量，默认为 2（2.0.17.0 版本开始，默认值改为 CPU 核数的一半向下取整）。 +- restfulRowLimit: 返回结果集（JSON 格式）的最大条数，默认值为 10240。 +- httpEnableCompress: 是否支持压缩，默认不支持，目前 TDengine 仅支持 gzip 压缩格式。 +- httpDebugFlag: 日志开关，默认 131。131：仅错误和报警信息，135：调试信息，143：非常详细的调试信息，默认 131。 +- httpDbNameMandatory: 是否必须在 RESTful url 中指定默认的数据库名。默认为 0，即关闭此检查。如果设置为 1，那么每个 RESTful url 中都必须设置一个默认数据库名，否则无论此时执行的 SQL 语句是否需要指定数据库，都会返回一个执行错误，拒绝执行此 SQL 语句。 ## CSharp Connector diff --git a/documentation20/cn/10.cluster/docs.md b/documentation20/cn/10.cluster/docs.md index f995597db016ed693ef6c3a37d15163a00160fde..f39138d61de35a09f5988014e95f284cb0f9e26f 100644 --- a/documentation20/cn/10.cluster/docs.md +++ b/documentation20/cn/10.cluster/docs.md @@ -14,7 +14,7 @@ TDengine的集群管理极其简单，除添加和删除节点需要人工干预 **第一步**：如果搭建集群的物理节点中，存有之前的测试数据、装过1.X的版本，或者装过其他版本的TDengine，请先将其删除，并清空所有数据（如果需要保留原有数据，请联系涛思交付团队进行旧版本升级、数据迁移），具体步骤请参考博客[《TDengine多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html)。 **注意1：**因为FQDN的信息会写进文件，如果之前没有配置或者更改FQDN，且启动了TDengine。请一定在确保数据无用或者备份的前提下，清理一下之前的数据（`rm -rf /var/lib/taos/*`）； -**注意2：**客户端也需要配置，确保它可以正确解析每个节点的FQDN配置，不管是通过DNS服务，还是 Host 文件。 +**注意2：**客户端也需要配置，确保它可以正确解析每个节点的FQDN配置，不管是通过DNS服务，还是修改 hosts 文件。 **第二步**：建议关闭所有物理节点的防火墙，至少保证端口：6030 - 6042的TCP和UDP端口都是开放的。**强烈建议**先关闭防火墙，集群搭建完毕之后，再来配置端口； diff --git a/documentation20/cn/12.taos-sql/docs.md b/documentation20/cn/12.taos-sql/docs.md index b183b6e4193b994b79eee691f2d231e429898f12..9981f1b7f83b416ec2fec6594b8cfa350a3e6348 100644 --- a/documentation20/cn/12.taos-sql/docs.md +++ b/documentation20/cn/12.taos-sql/docs.md @@ -35,7 +35,7 @@ taos> DESCRIBE meters; - 内部函数 now 是客户端的当前时间 - 插入记录时，如果时间戳为 now，插入数据时使用提交这条记录的客户端的当前时间 - Epoch Time：时间戳也可以是一个长整数，表示从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的毫秒数（相应地，如果所在 Database 的时间精度设置为“微秒”，则长整型格式的时间戳含义也就对应于从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的微秒数；纳秒精度的逻辑也是类似的。） -- 时间可以加减，比如 now-2h，表明查询时刻向前推 2 个小时（最近 2 小时）。数字后面的时间单位可以是 b(纳秒)、u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。 比如 `select * from t1 where ts > now-2w and ts <= now-1w`，表示查询两周前整整一周的数据。在指定降频操作（down sampling）的时间窗口（interval）时，时间单位还可以使用 n(自然月) 和 y(自然年)。 +- 时间可以加减，比如 now-2h，表明查询时刻向前推 2 个小时（最近 2 小时）。数字后面的时间单位可以是 b(纳秒)、u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。 比如 `select * from t1 where ts > now-2w and ts <= now-1w`，表示查询两周前整整一周的数据。在指定降采样操作（down sampling）的时间窗口（interval）时，时间单位还可以使用 n(自然月) 和 y(自然年)。 TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传递的 PRECISION 参数就可以支持微秒和纳秒。（从 2.1.5.0 版本开始支持纳秒精度） @@ -722,6 +722,7 @@ Query OK, 1 row(s) in set (0.001091s) 1. <> 算子也可以写为 != ，请注意，这个算子不能用于数据表第一列的 timestamp 字段。 2. like 算子使用通配符字符串进行匹配检查。 * 在通配符字符串中：'%'（百分号）匹配 0 到任意个字符；'\_'（下划线）匹配单个任意字符。 + * 如果希望匹配字符串中原本就带有的 \_（下划线）字符，那么可以在通配符字符串中写作 `\_`，也即加一个反斜线来进行转义。（从 2.1.8.0 版本开始支持） * 通配符字符串最长不能超过 20 字节。（从 2.1.6.1 版本开始，通配符字符串的长度放宽到了 100 字节，并可以通过 taos.cfg 中的 maxWildCardsLength 参数来配置这一长度限制。但不建议使用太长的通配符字符串，将有可能严重影响 LIKE 操作的执行性能。） 3. 同时进行多个字段的范围过滤，需要使用关键词 AND 来连接不同的查询条件，暂不支持 OR 连接的不同列之间的查询过滤条件。 4. 针对单一字段的过滤，如果是时间过滤条件，则一条语句中只支持设定一个；但针对其他的（普通）列或标签列，则可以使用 `OR` 关键字进行组合条件的查询过滤。例如： `((value > 20 AND value < 30) OR (value < 12))`。 @@ -1220,27 +1221,36 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数 ``` 功能说明：返回表/超级表的指定时间截面、指定字段的记录。 - 返回结果数据类型：同应用的字段。 + 返回结果数据类型：同字段类型。 - 应用字段：所有字段。 + 应用字段：数值型字段。 适用于：**表、超级表**。 - 说明：（从 2.0.15.0 版本开始新增此函数）INTERP 必须指定时间断面，如果该时间断面不存在直接对应的数据，那么会根据 FILL 参数的设定进行插值。其中，条件语句里面可以附带更多的筛选条件，例如标签、tbname。 + 说明：（从 2.0.15.0 版本开始新增此函数）INTERP 必须指定时间断面，如果该时间断面不存在直接对应的数据，那么会根据 FILL 参数的设定进行插值。此外，条件语句里面可附带筛选条件，例如标签、tbname。 + + INTERP 查询要求查询的时间区间必须位于数据集合（表）的所有记录的时间范围之内。如果给定的时间戳位于时间范围之外，即使有插值指令，仍然不返回结果。 示例： - ```mysql - taos> select interp(*) from meters where ts='2017-7-14 10:42:00.005' fill(prev); - interp(ts) | interp(f1) | interp(f2) | interp(f3) | - ==================================================================== - 2017-07-14 10:42:00.005 | 5 | 9 | 6 | - Query OK, 1 row(s) in set (0.002912s) + ```sql + taos> SELECT INTERP(*) FROM meters WHERE ts='2017-7-14 18:40:00.004'; + interp(ts) | interp(current) | interp(voltage) | interp(phase) | + ========================================================================================== + 2017-07-14 18:40:00.004 | 9.84020 | 216 | 0.32222 | + Query OK, 1 row(s) in set (0.002652s) + ``` + + 如果给定的时间戳无对应的数据，在不指定插值生成策略的情况下，不会返回结果，如果指定了插值策略，会根据插值策略返回结果。 + + ```sql + taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005'; + Query OK, 0 row(s) in set (0.004022s) - taos> select interp(*) from meters where tbname in ('t1') and ts='2017-7-14 10:42:00.005' fill(prev); - interp(ts) | interp(f1) | interp(f2) | interp(f3) | - ==================================================================== - 2017-07-14 10:42:00.005 | 5 | 6 | 7 | - Query OK, 1 row(s) in set (0.002005s) + taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005' FILL(PREV);; + interp(ts) | interp(current) | interp(voltage) | interp(phase) | + ========================================================================================== + 2017-07-14 18:40:00.005 | 9.88150 | 217 | 0.32500 | + Query OK, 1 row(s) in set (0.003056s) ``` ### 计算函数 diff --git a/documentation20/en/00.index/docs.md b/documentation20/en/00.index/docs.md index 1672c70b3cb57b413808fed0ebd68dc91a1f80c9..0ac4a06ef49ab99f78d39c81cb301ff925445acf 100644 --- a/documentation20/en/00.index/docs.md +++ b/documentation20/en/00.index/docs.md @@ -6,7 +6,7 @@ TDengine is a highly efficient platform to store, query, and analyze time-series * [TDengine Introduction and Features](/evaluation#intro) * [TDengine Use Scenes](/evaluation#scenes) -* [TDengine Performance Metrics and Verification]((/evaluation#)) +* [TDengine Performance Metrics and Verification](/evaluation#) ## [Getting Started](/getting-started) diff --git a/documentation20/en/02.getting-started/01.docker/docs.md b/documentation20/en/02.getting-started/01.docker/docs.md new file mode 100644 index 0000000000000000000000000000000000000000..daa89ef1016179e7860e4178c52481aef2760243 --- /dev/null +++ b/documentation20/en/02.getting-started/01.docker/docs.md @@ -0,0 +1,243 @@ +# Quickly experience TDengine through Docker + +While it is not recommended to deploy TDengine services via Docker in a production environment, Docker tools do a good job of shielding the environmental differences in the underlying operating system and are well suited for use in development testing or first-time experience with the toolset for installing and running TDengine. In particular, Docker makes it relatively easy to try TDengine on Mac OSX and Windows systems without having to install a virtual machine or rent an additional Linux server. In addition, starting from version 2.0.14.0, TDengine provides images that support both X86-64, X86, arm64, and arm32 platforms, so non-mainstream computers that can run docker, such as NAS, Raspberry Pi, and embedded development boards, can also easily experience TDengine based on this document. + +The following article explains how to quickly build a single-node TDengine runtime environment via Docker to support development and testing through a Step by Step style introduction. + +## Docker download + +The Docker tools themselves can be downloaded from [Docker official site](https://docs.docker.com/get-docker/). + +After installation, you can check the Docker version in the command line terminal. If the version number is output properly, the Docker environment has been installed successfully. + +```bash +$ docker -v +Docker version 20.10.3, build 48d30b5 +``` + +## Running TDengine in a Docker container + +1, Use the command to pull the TDengine image and make it run in the background. + +```bash +$ docker run -d --name tdengine tdengine/tdengine +7760c955f225d72e9c1ec5a4cef66149a7b94dae7598b11eb392138877e7d292 +``` + +- **docker run**: Running a container via Docker +- **--name tdengine**: Set the container name, we can see the corresponding container by the container name +- **-d**: Keeping containers running in the background +- **tdengine/tdengine**: Pulled from the official TDengine published application image +- **7760c955f225d72e9c1ec5a4cef66149a7b94dae7598b11eb392138877e7d292**: The long character returned is the container ID, and we can also view the corresponding container by its container ID + +2, Verify that the container is running correctly. + +```bash +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS ··· +c452519b0f9b tdengine/tdengine "taosd" 14 minutes ago Up 14 minutes ··· +``` + +- **docker ps**: Lists information about all containers that are in running state. +- **CONTAINER ID**: Container ID. +- **IMAGE**: The mirror used. +- **COMMAND**: The command to run when starting the container. +- **CREATED**: The time when the container was created. +- **STATUS**: The container status. Up means running. + +3, Go inside the Docker container and use TDengine. + +```bash +$ docker exec -it tdengine /bin/bash +root@c452519b0f9b:~/TDengine-server-2.0.20.13# +``` + +- **docker exec**: Enter the container via the docker exec command; if you exit, the container will not stop. +- **-i**: Enter the interactive mode. +- **-t**: Specify a terminal. +- **c452519b0f9b**: The container ID, which needs to be modified according to the value returned by the docker ps command. +- **/bin/bash**: Load the container and run bash to interact with it. + +4, After entering the container, execute the taos shell client program. + +```bash +$ root@c452519b0f9b:~/TDengine-server-2.0.20.13# taos + +Welcome to the TDengine shell from Linux, Client Version:2.0.20.13 +Copyright (c) 2020 by TAOS Data, Inc. All rights reserved. + +taos> +``` + +The TDengine terminal successfully connects to the server and prints out a welcome message and version information. If it fails, an error message is printed. + +In the TDengine terminal, you can create/delete databases, tables, super tables, etc., and perform insert and query operations via SQL commands. For details, you can refer to [TAOS SQL guide](https://www.taosdata.com/en/documentation/taos-sql). + +## Learn more about TDengine with taosdemo + +1, Following the above steps, exit the TDengine terminal program first. + +```bash +$ taos> q +root@c452519b0f9b:~/TDengine-server-2.0.20.13# +``` + +2, Execute taosdemo from the command line interface. + +```bash +root@c452519b0f9b:~/TDengine-server-2.0.20.13# taosdemo + +taosdemo is simulating data generated by power equipments monitoring... + +host: 127.0.0.1:6030 +user: root +password: taosdata +configDir: +resultFile: ./output.txt +thread num of insert data: 10 +thread num of create table: 10 +top insert interval: 0 +number of records per req: 30000 +max sql length: 1048576 +database count: 1 +database[0]: + database[0] name: test + drop: yes + replica: 1 + precision: ms + super table count: 1 + super table[0]: + stbName: meters + autoCreateTable: no + childTblExists: no + childTblCount: 10000 + childTblPrefix: d + dataSource: rand + iface: taosc + insertRows: 10000 + interlaceRows: 0 + disorderRange: 1000 + disorderRatio: 0 + maxSqlLen: 1048576 + timeStampStep: 1 + startTimestamp: 2017-07-14 10:40:00.000 + sampleFormat: + sampleFile: + tagsFile: + columnCount: 3 +column[0]:FLOAT column[1]:INT column[2]:FLOAT + tagCount: 2 + tag[0]:INT tag[1]:BINARY(16) + + Press enter key to continue or Ctrl-C to stop +``` + +After enter, this command will automatically create a super table meters under the database test, there are 10,000 tables under this super table, the table name is "d0" to "d9999", each table has 10,000 records, each record has four fields (ts, current, voltage, phase), the time stamp is from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999", each table has a tag location and groupId, groupId is set from 1 to 10 and location is set to "beijing" or "shanghai". + +It takes about a few minutes to execute this command and ends up inserting a total of 100 million records. + +3, Go to the TDengine terminal and view the data generated by taosdemo. + +- **Go to the terminal interface.** + +```bash +$ root@c452519b0f9b:~/TDengine-server-2.0.20.13# taos + +Welcome to the TDengine shell from Linux, Client Version:2.0.20.13 +Copyright (c) 2020 by TAOS Data, Inc. All rights reserved. + +taos> +``` + +- **View the database.** + +```bash +$ taos> show databases; + name | created_time | ntables | vgroups | ··· + test | 2021-08-18 06:01:11.021 | 10000 | 6 | ··· + log | 2021-08-18 05:51:51.065 | 4 | 1 | ··· + +``` + +- **View Super Tables.** + +```bash +$ taos> use test; +Database changed. + +$ taos> show stables; + name | created_time | columns | tags | tables | +============================================================================================ + meters | 2021-08-18 06:01:11.116 | 4 | 2 | 10000 | +Query OK, 1 row(s) in set (0.003259s) + +``` + +- **View the table and limit the output to 10 entries.** + +```bash +$ taos> select * from test.t0 limit 10; + +DB error: Table does not exist (0.002857s) +taos> select * from test.d0 limit 10; + ts | current | voltage | phase | +====================================================================================== + 2017-07-14 10:40:00.000 | 10.12072 | 223 | 0.34167 | + 2017-07-14 10:40:00.001 | 10.16103 | 224 | 0.34445 | + 2017-07-14 10:40:00.002 | 10.00204 | 220 | 0.33334 | + 2017-07-14 10:40:00.003 | 10.00030 | 220 | 0.33333 | + 2017-07-14 10:40:00.004 | 9.84029 | 216 | 0.32222 | + 2017-07-14 10:40:00.005 | 9.88028 | 217 | 0.32500 | + 2017-07-14 10:40:00.006 | 9.88110 | 217 | 0.32500 | + 2017-07-14 10:40:00.007 | 10.08137 | 222 | 0.33889 | + 2017-07-14 10:40:00.008 | 10.12063 | 223 | 0.34167 | + 2017-07-14 10:40:00.009 | 10.16086 | 224 | 0.34445 | +Query OK, 10 row(s) in set (0.016791s) + +``` + +- **View the tag values for the d0 table.** + +```bash +$ taos> select groupid, location from test.d0; + groupid | location | +================================= + 0 | shanghai | +Query OK, 1 row(s) in set (0.003490s) + +``` + +## Stop the TDengine service that is running in Docker + +```bash +$ docker stop tdengine +tdengine +``` + +- **docker stop**: Stop the specified running docker image with docker stop. +- **tdengine**: The name of the container. + +## TDengine connected in Docker during programming development + +There are two ideas for connecting from outside of Docker to use TDengine services running inside a Docker container: + +1, By port mapping (-p), the open network port inside the container is mapped to the specified port of the host. By mounting the local directory (-v), you can synchronize the data inside the host and the container to prevent data loss after the container is deleted. + +```bash +$ docker run -d -v /etc/taos:/etc/taos -P 6041:6041 tdengine/tdengine +526aa188da767ae94b244226a2b2eec2b5f17dd8eff592893d9ec0cd0f3a1ccd + +$ curl -u root:taosdata -d 'show databases' 127.0.0.1:6041/rest/sql +{"status":"succ","head":["name","created_time","ntables","vgroups","replica","quorum","days","keep0,keep1,keep(D)","cache(MB)","blocks","minrows","maxrows","wallevel","fsync","comp","cachelast","precision","update","status"],"column_meta":[["name",8,32],["created_time",9,8],["ntables",4,4],["vgroups",4,4],["replica",3,2],["quorum",3,2],["days",3,2],["keep0,keep1,keep(D)",8,24],["cache(MB)",4,4],["blocks",4,4],["minrows",4,4],["maxrows",4,4],["wallevel",2,1],["fsync",4,4],["comp",2,1],["cachelast",2,1],["precision",8,3],["update",2,1],["status",8,10]],"data":[["test","2021-08-18 06:01:11.021",10000,4,1,1,10,"3650,3650,3650",16,6,100,4096,1,3000,2,0,"ms",0,"ready"],["log","2021-08-18 05:51:51.065",4,1,1,1,10,"30,30,30",1,3,100,4096,1,3000,2,0,"us",0,"ready"]],"rows":2} +``` + +- The first command starts a docker container with TDengine running and maps the 6041 port of the container to port 6041 of the host. +- The second command, accessing TDengine through the RESTful interface, connects to port 6041 on the local machine, so the connection is successful. + +Note: In this example, for convenience reasons, only port 6041 is mapped, which is required for RESTful. If you wish to connect to the TDengine service in a non-RESTful manner, you will need to map a total of 11 ports starting at 6030. In the example, mounting the local directory also only deals with the /etc/taos directory where the configuration files are located, but not the data storage directory. + +2, Go directly to the docker container to do development via the exec command. That is, put the program code in the same Docker container where the TDengine server is located and connect to the TDengine service local to the container. + +```bash +$ docker exec -it tdengine /bin/bash +``` diff --git a/documentation20/en/02.getting-started/docs.md b/documentation20/en/02.getting-started/docs.md index 7b7de202b791173cca8eda9a36bd5128ad11f8a4..6941e44cf54fc03f40685ca85d593e54b475b1a4 100644 --- a/documentation20/en/02.getting-started/docs.md +++ b/documentation20/en/02.getting-started/docs.md @@ -2,7 +2,7 @@ ## Quick Install -TDengine software consists of 3 parts: server, client, and alarm module. At the moment, TDengine server only runs on Linux (Windows, mac OS and more OS supports will come soon), but client can run on either Windows or Linux. TDengine client can be installed and run on Windows or Linux. Applications based-on any OSes can all connect to server taosd via a RESTful interface. About CPU, TDengine supports X64/ARM64/MIPS64/Alpha64, and ARM32、RISC-V, other more CPU architectures will be supported soon. You can set up and install TDengine server either from the [source code](https://www.taosdata.com/en/getting-started/#Install-from-Source) or the [packages](https://www.taosdata.com/en/getting-started/#Install-from-Package). +TDengine software consists of 3 components: server, client, and alarm module. At the moment, TDengine server only runs on Linux (Windows, mac OS and more OS supports will come soon), but client can run on either Windows or Linux. TDengine client can be installed and run on Windows or Linux. Applications based-on any OSes can all connect to server taosd via a RESTful interface. About CPU, TDengine supports X64/ARM64/MIPS64/Alpha64, and ARM32、RISC-V, other more CPU architectures will be supported soon. You can set up and install TDengine server either from the [source code](https://www.taosdata.com/en/getting-started/#Install-from-Source) or the [packages](https://www.taosdata.com/en/getting-started/#Install-from-Package). ### Install from Source @@ -10,7 +10,9 @@ Please visit our [TDengine github page](https://github.com/taosdata/TDengine) fo ### Install from Docker Container -Please visit our [TDengine Official Docker Image: Distribution, Downloading, and Usage](https://www.taosdata.com/blog/2020/05/13/1509.html). +For the time being, it is not recommended to use Docker to deploy the client or server side of TDengine in production environments, but it is convenient to use Docker to deploy in development environments or when trying it for the first time. In particular, with Docker, it is easy to try TDengine in Mac OS X and Windows environments. + +Please refer to the detailed operation in [Quickly experience TDengine through Docker](https://www.taosdata.com/en/documentation/getting-started/docker). ### Install from Package diff --git a/documentation20/en/03.architecture/docs.md b/documentation20/en/03.architecture/docs.md index ce8dd6c8be75ae87afcd51fbbecbaf97a274ba3e..ac6c94fe40f0283d31ee85d8f1c3c09c82605e2a 100644 --- a/documentation20/en/03.architecture/docs.md +++ b/documentation20/en/03.architecture/docs.md @@ -119,7 +119,7 @@ As the data points are a series of data points over time, the data points genera 9. in addition to storage and query operations, various statistical and real-time calculation operations are also required; 10. data volume is huge, a system may generate over 10 billion data points in a day. -By utilizing the above characteristics, TDengine designs the storage and computing engine in a special and optimized way for time-series data, resulting in massive improvements in system efficiency. +In light of the characteristics mentioned above, TDengine designs the storage and computing engine in a special and optimized way for time-series data, resulting in massive improvements in system efficiency. ### Relational Database Model @@ -139,7 +139,7 @@ TDengine suggests using collection point ID as the table name (like D1001 in the ### STable: A Collection of Data Points in the Same Type -The method of one table for each point will bring a greatly increasing number of tables, which is difficult to manage. Moreover, applications often need to take aggregation operations between collection points, thus aggregation operations will become complicated. To support aggregation over multiple tables efficiently, the [STable(Super Table)](https://www.taosdata.com/en/documentation/super-table) concept is introduced by TDengine. +The method of one table for each point will bring a greatly increasing number of tables, which is difficult to manage. Moreover, applications often need to take aggregation operations between collection points, thus aggregation operations will become complicated. To support aggregation over multiple tables efficiently, the STable (Super Table) concept is introduced by TDengine. STable is an abstract collection for a type of data point. A STable contains a set of points (tables) that have the same schema or data structure, but with different static attributes (tags). To describe a STable (a combination of data collection points of a specific type), in addition to defining the table structure of the collected metrics, it is also necessary to define the schema of its tag. The data type of tags can be int, float, string, and there can be multiple tags, which can be added, deleted, or modified afterward. If the whole system has N different types of data collection points, N STables need to be established. diff --git a/documentation20/en/04.model/docs.md b/documentation20/en/04.model/docs.md index 1bf6c6787846b65ec9b56b65b80470cad2224756..08d952d317e30ef91dad32333c9abb0e646b8e48 100644 --- a/documentation20/en/04.model/docs.md +++ b/documentation20/en/04.model/docs.md @@ -26,6 +26,7 @@ Replace the database operating in the current connection with “power”, other - Any table or STable belongs to a database. Before creating a table, a database must be created first. - Tables in two different databases cannot be JOIN. +- You need to specify a timestamp when creating and inserting records, or querying history records. ## Create a STable diff --git a/documentation20/en/10.cluster/docs.md b/documentation20/en/10.cluster/docs.md index d7d908ff424270d9aa33f89eefd36e73f6ab68b2..05d0a463aa1bcf5bf56e9ca4032be9ddbaa59b4d 100644 --- a/documentation20/en/10.cluster/docs.md +++ b/documentation20/en/10.cluster/docs.md @@ -16,7 +16,7 @@ Please refer to the [video tutorial](https://www.taosdata.com/blog/2020/11/11/19 **Note 1:** Because the information of FQDN will be written into a file, if FQDN has not been configured or changed before, and TDengine has been started, be sure to clean up the previous data （`rm -rf /var/lib/taos/*`）on the premise of ensuring that the data is useless or backed up; -**Note 2:** The client also needs to be configured to ensure that it can correctly parse the FQDN configuration of each node, whether through DNS service or Host file. +**Note 2:** The client also needs to be configured to ensure that it can correctly parse the FQDN configuration of each node, whether through DNS service or modify hosts file. **Step 2:** It is recommended to close the firewall of all physical nodes, and at least ensure that the TCP and UDP ports of ports 6030-6042 are open. It is **strongly recommended** to close the firewall first and configure the ports after the cluster is built; diff --git a/packaging/cfg/taos.cfg b/packaging/cfg/taos.cfg index 06e7cb7da01d50b613754b40c56a3cb335cffb37..310369aa14ad5e9e6ccb49843605a92fdc333563 100644 --- a/packaging/cfg/taos.cfg +++ b/packaging/cfg/taos.cfg @@ -194,6 +194,9 @@ keepColumnName 1 # maximum number of rows returned by the restful interface # restfulRowLimit 10240 +# database name must be specified in restful interface if the following parameter is set, off by default +# httpDbNameMandatory 1 + # The following parameter is used to limit the maximum number of lines in log files. # max number of lines per log filters # numOfLogLines 10000000 diff --git a/packaging/tools/make_install.sh b/packaging/tools/make_install.sh index 55ca1174c91dd2a27049cc82733836344e73b90a..0849a76e31a631bae7a6ba0d6ef7ffcb58b8480b 100755 --- a/packaging/tools/make_install.sh +++ b/packaging/tools/make_install.sh @@ -19,18 +19,21 @@ else fi # Dynamic directory -data_dir="/var/lib/taos" + if [ "$osType" != "Darwin" ]; then + data_dir="/var/lib/taos" log_dir="/var/log/taos" else - log_dir=~/TDengine/log + data_dir="/usr/local/var/lib/taos" + log_dir="/usr/local/var/log/taos" fi -data_link_dir="/usr/local/taos/data" -log_link_dir="/usr/local/taos/log" - -cfg_install_dir="/etc/taos" +if [ "$osType" != "Darwin" ]; then + cfg_install_dir="/etc/taos" +else + cfg_install_dir="/usr/local/etc/taos" +fi if [ "$osType" != "Darwin" ]; then bin_link_dir="/usr/bin" @@ -44,10 +47,18 @@ else fi #install main path -install_main_dir="/usr/local/taos" +if [ "$osType" != "Darwin" ]; then + install_main_dir="/usr/local/taos" +else + install_main_dir="/usr/local/Cellar/tdengine/${verNumber}" +fi # old bin dir -bin_dir="/usr/local/taos/bin" +if [ "$osType" != "Darwin" ]; then + bin_dir="/usr/local/taos/bin" +else + bin_dir="/usr/local/Cellar/tdengine/${verNumber}/bin" +fi service_config_dir="/etc/systemd/system" @@ -59,12 +70,11 @@ GREEN_UNDERLINE='\033[4;32m' NC='\033[0m' csudo="" -if command -v sudo > /dev/null; then - csudo="sudo" -fi if [ "$osType" != "Darwin" ]; then - + if command -v sudo > /dev/null; then + csudo="sudo" + fi initd_mod=0 service_mod=2 if pidof systemd &> /dev/null; then @@ -137,18 +147,17 @@ function install_main_path() { function install_bin() { # Remove links - ${csudo} rm -f ${bin_link_dir}/taos || : + ${csudo} rm -f ${bin_link_dir}/taos || : + ${csudo} rm -f ${bin_link_dir}/taosd || : + ${csudo} rm -f ${bin_link_dir}/taosdemo || : + ${csudo} rm -f ${bin_link_dir}/taosdump || : if [ "$osType" != "Darwin" ]; then - ${csudo} rm -f ${bin_link_dir}/taosd || : - ${csudo} rm -f ${bin_link_dir}/taosdemo || : ${csudo} rm -f ${bin_link_dir}/perfMonitor || : - ${csudo} rm -f ${bin_link_dir}/taosdump || : ${csudo} rm -f ${bin_link_dir}/set_core || : + ${csudo} rm -f ${bin_link_dir}/rmtaos || : fi - - ${csudo} rm -f ${bin_link_dir}/rmtaos || : - + ${csudo} cp -r ${binary_dir}/build/bin/* ${install_main_dir}/bin ${csudo} cp -r ${script_dir}/taosd-dump-cfg.gdb ${install_main_dir}/bin @@ -162,20 +171,18 @@ function install_bin() { ${csudo} chmod 0555 ${install_main_dir}/bin/* #Make link - [ -x ${install_main_dir}/bin/taos ] && ${csudo} ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || : + [ -x ${install_main_dir}/bin/taos ] && ${csudo} ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || : + [ -x ${install_main_dir}/bin/taosd ] && ${csudo} ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || : + [ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || : + [ -x ${install_main_dir}/bin/taosdemo ] && ${csudo} ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || : if [ "$osType" != "Darwin" ]; then - [ -x ${install_main_dir}/bin/taosd ] && ${csudo} ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || : - [ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || : - [ -x ${install_main_dir}/bin/taosdemo ] && ${csudo} ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || : [ -x ${install_main_dir}/bin/perfMonitor ] && ${csudo} ln -s ${install_main_dir}/bin/perfMonitor ${bin_link_dir}/perfMonitor || : [ -x ${install_main_dir}/set_core.sh ] && ${csudo} ln -s ${install_main_dir}/bin/set_core.sh ${bin_link_dir}/set_core || : fi - + if [ "$osType" != "Darwin" ]; then - [ -x ${install_main_dir}/bin/remove.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || : - else - [ -x ${install_main_dir}/bin/remove_client.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_client.sh ${bin_link_dir}/rmtaos || : + [ -x ${install_main_dir}/bin/remove.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || : fi } @@ -222,7 +229,7 @@ function install_jemalloc() { fi if [ -d /etc/ld.so.conf.d ]; then - ${csudo} echo "/usr/local/lib" > /etc/ld.so.conf.d/jemalloc.conf + echo "/usr/local/lib" | ${csudo} tee /etc/ld.so.conf.d/jemalloc.conf ${csudo} ldconfig else echo "/etc/ld.so.conf.d not found!" @@ -248,10 +255,8 @@ function install_lib() { fi else ${csudo} cp -Rf ${binary_dir}/build/lib/libtaos.* ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/* - ${csudo} ln -sf ${install_main_dir}/driver/libtaos.1.dylib ${lib_link_dir}/libtaos.1.dylib - ${csudo} ln -sf ${lib_link_dir}/libtaos.1.dylib ${lib_link_dir}/libtaos.dylib fi - + install_jemalloc if [ "$osType" != "Darwin" ]; then @@ -261,10 +266,14 @@ function install_lib() { function install_header() { - ${csudo} rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taoserror.h || : + if [ "$osType" != "Darwin" ]; then + ${csudo} rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taoserror.h || : + fi ${csudo} cp -f ${source_dir}/src/inc/taos.h ${source_dir}/src/inc/taoserror.h ${install_main_dir}/include && ${csudo} chmod 644 ${install_main_dir}/include/* - ${csudo} ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h - ${csudo} ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h + if [ "$osType" != "Darwin" ]; then + ${csudo} ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h + ${csudo} ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h + fi } function install_config() { @@ -272,23 +281,20 @@ function install_config() { if [ ! -f ${cfg_install_dir}/taos.cfg ]; then ${csudo} mkdir -p ${cfg_install_dir} - [ -f ${script_dir}/../cfg/taos.cfg ] && ${csudo} cp ${script_dir}/../cfg/taos.cfg ${cfg_install_dir} + [ -f ${script_dir}/../cfg/taos.cfg ] && + ${csudo} cp ${script_dir}/../cfg/taos.cfg ${cfg_install_dir} ${csudo} chmod 644 ${cfg_install_dir}/* fi ${csudo} cp -f ${script_dir}/../cfg/taos.cfg ${install_main_dir}/cfg/taos.cfg.org - ${csudo} ln -s ${cfg_install_dir}/taos.cfg ${install_main_dir}/cfg + + if [ "$osType" != "Darwin" ]; then ${csudo} ln -s ${cfg_install_dir}/taos.cfg ${install_main_dir}/cfg + fi } function install_log() { ${csudo} rm -rf ${log_dir} || : - - if [ "$osType" != "Darwin" ]; then - ${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir} - else - mkdir -p ${log_dir} && chmod 777 ${log_dir} - fi - + ${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir} ${csudo} ln -s ${log_dir} ${install_main_dir}/log } @@ -309,7 +315,6 @@ function install_connector() { echo "WARNING: go connector not found, please check if want to use it!" fi ${csudo} cp -rf ${source_dir}/src/connector/python ${install_main_dir}/connector - ${csudo} cp ${binary_dir}/build/lib/*.jar ${install_main_dir}/connector &> /dev/null && ${csudo} chmod 777 ${install_main_dir}/connector/*.jar || echo &> /dev/null } @@ -489,24 +494,21 @@ function install_TDengine() { else echo -e "${GREEN}Start to install TDEngine Client ...${NC}" fi - + install_main_path - if [ "$osType" != "Darwin" ]; then - install_data - fi + install_data install_log install_header install_lib install_connector install_examples - install_bin - + if [ "$osType" != "Darwin" ]; then install_service fi - + install_config if [ "$osType" != "Darwin" ]; then diff --git a/src/client/inc/tscUtil.h b/src/client/inc/tscUtil.h index 5c3b2903eeb3577c9a57c3dfd4b481f9bf3616c7..c59ec3e624ec84a76ae805d969a6b319667fb1af 100644 --- a/src/client/inc/tscUtil.h +++ b/src/client/inc/tscUtil.h @@ -365,6 +365,8 @@ STblCond* tsGetTableFilter(SArray* filters, uint64_t uid, int16_t idx); void tscRemoveCachedTableMeta(STableMetaInfo* pTableMetaInfo, uint64_t id); +char* cloneCurrentDBName(SSqlObj* pSql); + #ifdef __cplusplus } #endif diff --git a/src/client/src/TSDBJNIConnector.c b/src/client/src/TSDBJNIConnector.c index 7ba613de88f2d358e4a359cfa5fb0d5f32a1071e..506c8d64b9f4213713656ecd08612a103e0b1b2d 100644 --- a/src/client/src/TSDBJNIConnector.c +++ b/src/client/src/TSDBJNIConnector.c @@ -20,12 +20,42 @@ #include "com_taosdata_jdbc_TSDBJNIConnector.h" -#define jniFatal(...) { if (jniDebugFlag & DEBUG_FATAL) { taosPrintLog("JNI FATAL ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); }} -#define jniError(...) { if (jniDebugFlag & DEBUG_ERROR) { taosPrintLog("JNI ERROR ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); }} -#define jniWarn(...) { if (jniDebugFlag & DEBUG_WARN) { taosPrintLog("JNI WARN ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); }} -#define jniInfo(...) { if (jniDebugFlag & DEBUG_INFO) { taosPrintLog("JNI ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); }} -#define jniDebug(...) { if (jniDebugFlag & DEBUG_DEBUG) { taosPrintLog("JNI ", jniDebugFlag, __VA_ARGS__); }} -#define jniTrace(...) { if (jniDebugFlag & DEBUG_TRACE) { taosPrintLog("JNI ", jniDebugFlag, __VA_ARGS__); }} +#define jniFatal(...) \ + { \ + if (jniDebugFlag & DEBUG_FATAL) { \ + taosPrintLog("JNI FATAL ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); \ + } \ + } +#define jniError(...) \ + { \ + if (jniDebugFlag & DEBUG_ERROR) { \ + taosPrintLog("JNI ERROR ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); \ + } \ + } +#define jniWarn(...) \ + { \ + if (jniDebugFlag & DEBUG_WARN) { \ + taosPrintLog("JNI WARN ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); \ + } \ + } +#define jniInfo(...) \ + { \ + if (jniDebugFlag & DEBUG_INFO) { \ + taosPrintLog("JNI ", tscEmbedded ? 255 : jniDebugFlag, __VA_ARGS__); \ + } \ + } +#define jniDebug(...) \ + { \ + if (jniDebugFlag & DEBUG_DEBUG) { \ + taosPrintLog("JNI ", jniDebugFlag, __VA_ARGS__); \ + } \ + } +#define jniTrace(...) \ + { \ + if (jniDebugFlag & DEBUG_TRACE) { \ + taosPrintLog("JNI ", jniDebugFlag, __VA_ARGS__); \ + } \ + } int __init = 0; @@ -60,14 +90,14 @@ jmethodID g_blockdataSetByteArrayFp; jmethodID g_blockdataSetNumOfRowsFp; jmethodID g_blockdataSetNumOfColsFp; -#define JNI_SUCCESS 0 -#define JNI_TDENGINE_ERROR -1 +#define JNI_SUCCESS 0 +#define JNI_TDENGINE_ERROR -1 #define JNI_CONNECTION_NULL -2 #define JNI_RESULT_SET_NULL -3 #define JNI_NUM_OF_FIELDS_0 -4 -#define JNI_SQL_NULL -5 -#define JNI_FETCH_END -6 -#define JNI_OUT_OF_MEMORY -7 +#define JNI_SQL_NULL -5 +#define JNI_FETCH_END -6 +#define JNI_OUT_OF_MEMORY -7 static void jniGetGlobalMethod(JNIEnv *env) { // make sure init function executed once @@ -129,13 +159,13 @@ static void jniGetGlobalMethod(JNIEnv *env) { } static int32_t check_for_params(jobject jobj, jlong conn, jlong res) { - if ((TAOS*) conn == NULL) { + if ((TAOS *)conn == NULL) { jniError("jobj:%p, connection is closed", jobj); return JNI_CONNECTION_NULL; } - if ((TAOS_RES *) res == NULL) { - jniError("jobj:%p, conn:%p, res is null", jobj, (TAOS*) conn); + if ((TAOS_RES *)res == NULL) { + jniError("jobj:%p, conn:%p, res is null", jobj, (TAOS *)conn); return JNI_RESULT_SET_NULL; } @@ -216,7 +246,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setOptions(JNIEnv JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_connectImp(JNIEnv *env, jobject jobj, jstring jhost, jint jport, jstring jdbName, jstring juser, jstring jpass) { - jlong ret = 0; + jlong ret = 0; const char *host = NULL; const char *user = NULL; const char *pass = NULL; @@ -246,7 +276,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_connectImp(JNIEn jniDebug("jobj:%p, pass not specified, use default password", jobj); } - ret = (jlong) taos_connect((char *)host, (char *)user, (char *)pass, (char *)dbname, (uint16_t)jport); + ret = (jlong)taos_connect((char *)host, (char *)user, (char *)pass, (char *)dbname, (uint16_t)jport); if (ret == 0) { jniError("jobj:%p, conn:%p, connect to database failed, host=%s, user=%s, dbname=%s, port=%d", jobj, (void *)ret, (char *)host, (char *)user, (char *)dbname, (int32_t)jport); @@ -289,7 +319,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeQueryImp( jsize len = (*env)->GetArrayLength(env, jsql); - char *str = (char *) calloc(1, sizeof(char) * (len + 1)); + char *str = (char *)calloc(1, sizeof(char) * (len + 1)); if (str == NULL) { jniError("jobj:%p, conn:%p, alloc memory failed", jobj, tscon); return JNI_OUT_OF_MEMORY; @@ -315,16 +345,17 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeQueryImp( } free(str); - return (jlong) pSql; + return (jlong)pSql; } -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getErrCodeImp(JNIEnv *env, jobject jobj, jlong con, jlong tres) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getErrCodeImp(JNIEnv *env, jobject jobj, jlong con, + jlong tres) { int32_t code = check_for_params(jobj, con, tres); if (code != JNI_SUCCESS) { return code; } - return (jint)taos_errno((TAOS_RES*) tres); + return (jint)taos_errno((TAOS_RES *)tres); } JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getErrMsgImp(JNIEnv *env, jobject jobj, jlong tres) { @@ -334,7 +365,7 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getErrMsgImp(J JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultSetImp(JNIEnv *env, jobject jobj, jlong con, jlong tres) { - TAOS *tscon = (TAOS *)con; + TAOS * tscon = (TAOS *)con; int32_t code = check_for_params(jobj, con, tres); if (code != JNI_SUCCESS) { return code; @@ -359,7 +390,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_isUpdateQueryImp( SSqlObj *pSql = (TAOS_RES *)tres; - return (tscIsUpdateQuery(pSql)? 1:0); + return (tscIsUpdateQuery(pSql) ? 1 : 0); } JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_freeResultSetImp(JNIEnv *env, jobject jobj, jlong con, @@ -370,21 +401,22 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_freeResultSetImp( } taos_free_result((void *)res); - jniDebug("jobj:%p, conn:%p, free resultset:%p", jobj, (TAOS*) con, (void *)res); + jniDebug("jobj:%p, conn:%p, free resultset:%p", jobj, (TAOS *)con, (void *)res); return JNI_SUCCESS; } JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getAffectedRowsImp(JNIEnv *env, jobject jobj, jlong con, jlong res) { - TAOS *tscon = (TAOS *)con; + TAOS * tscon = (TAOS *)con; int32_t code = check_for_params(jobj, con, res); if (code != JNI_SUCCESS) { return code; } jint ret = taos_affected_rows((SSqlObj *)res); - jniDebug("jobj:%p, conn:%p, sql:%p, res: %p, affect rows:%d", jobj, tscon, (TAOS *)con, (TAOS_RES *)res, (int32_t)ret); + jniDebug("jobj:%p, conn:%p, sql:%p, res: %p, affect rows:%d", jobj, tscon, (TAOS *)con, (TAOS_RES *)res, + (int32_t)ret); return ret; } @@ -392,13 +424,13 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getAffectedRowsIm JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getSchemaMetaDataImp(JNIEnv *env, jobject jobj, jlong con, jlong res, jobject arrayListObj) { - TAOS *tscon = (TAOS *)con; + TAOS * tscon = (TAOS *)con; int32_t code = check_for_params(jobj, con, res); if (code != JNI_SUCCESS) { return code; } - TAOS_RES* tres = (TAOS_RES*) res; + TAOS_RES * tres = (TAOS_RES *)res; TAOS_FIELD *fields = taos_fetch_fields(tres); int32_t num_fields = taos_num_fields(tres); @@ -452,7 +484,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn int32_t numOfFields = taos_num_fields(result); if (numOfFields == 0) { - jniError("jobj:%p, conn:%p, resultset:%p, fields size %d", jobj, tscon, (void*)res, numOfFields); + jniError("jobj:%p, conn:%p, resultset:%p, fields size %d", jobj, tscon, (void *)res, numOfFields); return JNI_NUM_OF_FIELDS_0; } @@ -460,7 +492,8 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn if (row == NULL) { int code = taos_errno(result); if (code == TSDB_CODE_SUCCESS) { - jniDebug("jobj:%p, conn:%p, resultset:%p, fields size is %d, fetch row to the end", jobj, tscon, (void*)res, numOfFields); + jniDebug("jobj:%p, conn:%p, resultset:%p, fields size is %d, fetch row to the end", jobj, tscon, (void *)res, + numOfFields); return JNI_FETCH_END; } else { jniDebug("jobj:%p, conn:%p, interrupted query", jobj, tscon); @@ -468,7 +501,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn } } - int32_t* length = taos_fetch_lengths(result); + int32_t *length = taos_fetch_lengths(result); char tmp[TSDB_MAX_BYTES_PER_ROW] = {0}; @@ -533,7 +566,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchRowImp(JNIEn } JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchBlockImp(JNIEnv *env, jobject jobj, jlong con, - jlong res, jobject rowobj) { + jlong res, jobject rowobj) { TAOS * tscon = (TAOS *)con; int32_t code = check_for_params(jobj, con, res); if (code != JNI_SUCCESS) { @@ -564,8 +597,13 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_fetchBlockImp(JNI (*env)->CallVoidMethod(env, rowobj, g_blockdataSetNumOfColsFp, (jint)numOfFields); for (int i = 0; i < numOfFields; i++) { - (*env)->CallVoidMethod(env, rowobj, g_blockdataSetByteArrayFp, i, fields[i].bytes * numOfRows, - jniFromNCharToByteArray(env, (char *)row[i], fields[i].bytes * numOfRows)); + int bytes = fields[i].bytes; + + if (fields[i].type == TSDB_DATA_TYPE_BINARY || fields[i].type == TSDB_DATA_TYPE_NCHAR) { + bytes += 2; + } + (*env)->CallVoidMethod(env, rowobj, g_blockdataSetByteArrayFp, i, bytes * numOfRows, + jniFromNCharToByteArray(env, (char *)row[i], bytes * numOfRows)); } return JNI_SUCCESS; @@ -585,7 +623,8 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeConnectionIm } JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp(JNIEnv *env, jobject jobj, jlong con, - jboolean restart, jstring jtopic, jstring jsql, jint jinterval) { + jboolean restart, jstring jtopic, + jstring jsql, jint jinterval) { jlong sub = 0; TAOS *taos = (TAOS *)con; char *topic = NULL; @@ -682,8 +721,8 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getTsCharset(J * @param res the TAOS_RES object, i.e. the SSqlObject * @return precision 0:ms 1:us 2:ns */ -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultTimePrecisionImp(JNIEnv *env, jobject jobj, jlong con, - jlong res) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultTimePrecisionImp(JNIEnv *env, jobject jobj, + jlong con, jlong res) { TAOS *tscon = (TAOS *)con; if (tscon == NULL) { jniError("jobj:%p, connection is closed", jobj); @@ -699,7 +738,8 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_getResultTimePrec return taos_result_precision(result); } -JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_prepareStmtImp(JNIEnv *env, jobject jobj, jbyteArray jsql, jlong con) { +JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_prepareStmtImp(JNIEnv *env, jobject jobj, + jbyteArray jsql, jlong con) { TAOS *tscon = (TAOS *)con; if (tscon == NULL) { jniError("jobj:%p, connection already closed", jobj); @@ -713,7 +753,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_prepareStmtImp(J jsize len = (*env)->GetArrayLength(env, jsql); - char *str = (char *) calloc(1, sizeof(char) * (len + 1)); + char *str = (char *)calloc(1, sizeof(char) * (len + 1)); if (str == NULL) { jniError("jobj:%p, conn:%p, alloc memory failed", jobj, tscon); return JNI_OUT_OF_MEMORY; @@ -724,25 +764,27 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_prepareStmtImp(J // todo handle error } - TAOS_STMT* pStmt = taos_stmt_init(tscon); - int32_t code = taos_stmt_prepare(pStmt, str, len); + TAOS_STMT *pStmt = taos_stmt_init(tscon); + int32_t code = taos_stmt_prepare(pStmt, str, len); tfree(str); if (code != TSDB_CODE_SUCCESS) { jniError("jobj:%p, conn:%p, code:%s", jobj, tscon, tstrerror(code)); return JNI_TDENGINE_ERROR; } - return (jlong) pStmt; + return (jlong)pStmt; } -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameImp(JNIEnv *env, jobject jobj, jlong stmt, jstring jname, jlong conn) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameImp(JNIEnv *env, jobject jobj, + jlong stmt, jstring jname, + jlong conn) { TAOS *tsconn = (TAOS *)conn; if (tsconn == NULL) { jniError("jobj:%p, connection already closed", jobj); return JNI_CONNECTION_NULL; } - TAOS_STMT* pStmt = (TAOS_STMT*) stmt; + TAOS_STMT *pStmt = (TAOS_STMT *)stmt; if (pStmt == NULL) { jniError("jobj:%p, conn:%p, invalid stmt handle", jobj, tsconn); return JNI_SQL_NULL; @@ -750,7 +792,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameI const char *name = (*env)->GetStringUTFChars(env, jname, NULL); - int32_t code = taos_stmt_set_tbname((void*)stmt, name); + int32_t code = taos_stmt_set_tbname((void *)stmt, name); if (code != TSDB_CODE_SUCCESS) { (*env)->ReleaseStringUTFChars(env, jname, name); @@ -763,8 +805,9 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setBindTableNameI return JNI_SUCCESS; } -JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp(JNIEnv *env, jobject jobj, jlong stmt, - jbyteArray colDataList, jbyteArray lengthList, jbyteArray nullList, jint dataType, jint dataBytes, jint numOfRows, jint colIndex, jlong con) { +JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp( + JNIEnv *env, jobject jobj, jlong stmt, jbyteArray colDataList, jbyteArray lengthList, jbyteArray nullList, + jint dataType, jint dataBytes, jint numOfRows, jint colIndex, jlong con) { TAOS *tscon = (TAOS *)con; if (tscon == NULL) { jniError("jobj:%p, connection already closed", jobj); @@ -798,14 +841,14 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp(J } // bind multi-rows with only one invoke. - TAOS_MULTI_BIND* b = calloc(1, sizeof(TAOS_MULTI_BIND)); + TAOS_MULTI_BIND *b = calloc(1, sizeof(TAOS_MULTI_BIND)); - b->num = numOfRows; - b->buffer_type = dataType; // todo check data type - b->buffer_length = IS_VAR_DATA_TYPE(dataType)? dataBytes:tDataTypes[dataType].bytes; - b->is_null = nullArray; - b->buffer = colBuf; - b->length = (int32_t*)lengthArray; + b->num = numOfRows; + b->buffer_type = dataType; // todo check data type + b->buffer_length = IS_VAR_DATA_TYPE(dataType) ? dataBytes : tDataTypes[dataType].bytes; + b->is_null = nullArray; + b->buffer = colBuf; + b->length = (int32_t *)lengthArray; // set the length and is_null array if (!IS_VAR_DATA_TYPE(dataType)) { @@ -829,14 +872,15 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_bindColDataImp(J return JNI_SUCCESS; } -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(JNIEnv *env, jobject jobj, jlong stmt, jlong con) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(JNIEnv *env, jobject jobj, jlong stmt, + jlong con) { TAOS *tscon = (TAOS *)con; if (tscon == NULL) { jniError("jobj:%p, connection already closed", jobj); return JNI_CONNECTION_NULL; } - TAOS_STMT *pStmt = (TAOS_STMT*) stmt; + TAOS_STMT *pStmt = (TAOS_STMT *)stmt; if (pStmt == NULL) { jniError("jobj:%p, conn:%p, invalid stmt", jobj, tscon); return JNI_SQL_NULL; @@ -853,14 +897,15 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_executeBatchImp(J return JNI_SUCCESS; } -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv *env, jobject jobj, jlong stmt, jlong con) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv *env, jobject jobj, jlong stmt, + jlong con) { TAOS *tscon = (TAOS *)con; if (tscon == NULL) { jniError("jobj:%p, connection already closed", jobj); return JNI_CONNECTION_NULL; } - TAOS_STMT *pStmt = (TAOS_STMT*) stmt; + TAOS_STMT *pStmt = (TAOS_STMT *)stmt; if (pStmt == NULL) { jniError("jobj:%p, conn:%p, invalid stmt", jobj, tscon); return JNI_SQL_NULL; @@ -876,15 +921,16 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeStmt(JNIEnv return JNI_SUCCESS; } -JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp(JNIEnv *env, jobject jobj, - jlong stmt, jstring tableName, jint numOfTags, jbyteArray tags, jbyteArray typeList, jbyteArray lengthList, jbyteArray nullList, jlong conn) { +JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsImp( + JNIEnv *env, jobject jobj, jlong stmt, jstring tableName, jint numOfTags, jbyteArray tags, jbyteArray typeList, + jbyteArray lengthList, jbyteArray nullList, jlong conn) { TAOS *tsconn = (TAOS *)conn; if (tsconn == NULL) { jniError("jobj:%p, connection already closed", jobj); return JNI_CONNECTION_NULL; } - TAOS_STMT* pStmt = (TAOS_STMT*) stmt; + TAOS_STMT *pStmt = (TAOS_STMT *)stmt; if (pStmt == NULL) { jniError("jobj:%p, conn:%p, invalid stmt handle", jobj, tsconn); return JNI_SQL_NULL; @@ -898,39 +944,39 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsI } len = (*env)->GetArrayLength(env, lengthList); - int64_t *lengthArray = (int64_t*) calloc(1, len); - (*env)->GetByteArrayRegion(env, lengthList, 0, len, (jbyte*) lengthArray); + int64_t *lengthArray = (int64_t *)calloc(1, len); + (*env)->GetByteArrayRegion(env, lengthList, 0, len, (jbyte *)lengthArray); if ((*env)->ExceptionCheck(env)) { } len = (*env)->GetArrayLength(env, typeList); - char *typeArray = (char*) calloc(1, len); - (*env)->GetByteArrayRegion(env, typeList, 0, len, (jbyte*) typeArray); + char *typeArray = (char *)calloc(1, len); + (*env)->GetByteArrayRegion(env, typeList, 0, len, (jbyte *)typeArray); if ((*env)->ExceptionCheck(env)) { } len = (*env)->GetArrayLength(env, nullList); - int32_t *nullArray = (int32_t*) calloc(1, len); - (*env)->GetByteArrayRegion(env, nullList, 0, len, (jbyte*) nullArray); + int32_t *nullArray = (int32_t *)calloc(1, len); + (*env)->GetByteArrayRegion(env, nullList, 0, len, (jbyte *)nullArray); if ((*env)->ExceptionCheck(env)) { } const char *name = (*env)->GetStringUTFChars(env, tableName, NULL); - char* curTags = tagsData; + char * curTags = tagsData; TAOS_BIND *tagsBind = calloc(numOfTags, sizeof(TAOS_BIND)); - for(int32_t i = 0; i < numOfTags; ++i) { + for (int32_t i = 0; i < numOfTags; ++i) { tagsBind[i].buffer_type = typeArray[i]; - tagsBind[i].buffer = curTags; + tagsBind[i].buffer = curTags; tagsBind[i].is_null = &nullArray[i]; - tagsBind[i].length = (uintptr_t*) &lengthArray[i]; + tagsBind[i].length = (uintptr_t *)&lengthArray[i]; curTags += lengthArray[i]; } - int32_t code = taos_stmt_set_tbname_tags((void*)stmt, name, tagsBind); + int32_t code = taos_stmt_set_tbname_tags((void *)stmt, name, tagsBind); - int32_t nTags = (int32_t) numOfTags; + int32_t nTags = (int32_t)numOfTags; jniDebug("jobj:%p, conn:%p, set table name:%s, numOfTags:%d", jobj, tsconn, name, nTags); tfree(tagsData); @@ -948,28 +994,28 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setTableNameTagsI } JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_insertLinesImp(JNIEnv *env, jobject jobj, - jobjectArray lines, jlong conn) { + jobjectArray lines, jlong conn) { TAOS *taos = (TAOS *)conn; if (taos == NULL) { jniError("jobj:%p, connection already closed", jobj); return JNI_CONNECTION_NULL; } - int numLines = (*env)->GetArrayLength(env, lines); - char** c_lines = calloc(numLines, sizeof(char*)); + int numLines = (*env)->GetArrayLength(env, lines); + char **c_lines = calloc(numLines, sizeof(char *)); if (c_lines == NULL) { jniError("c_lines:%p, alloc memory failed", c_lines); return JNI_OUT_OF_MEMORY; } for (int i = 0; i < numLines; ++i) { - jstring line = (jstring) ((*env)->GetObjectArrayElement(env, lines, i)); - c_lines[i] = (char*)(*env)->GetStringUTFChars(env, line, 0); + jstring line = (jstring)((*env)->GetObjectArrayElement(env, lines, i)); + c_lines[i] = (char *)(*env)->GetStringUTFChars(env, line, 0); } int code = taos_insert_lines(taos, c_lines, numLines); for (int i = 0; i < numLines; ++i) { - jstring line = (jstring) ((*env)->GetObjectArrayElement(env, lines, i)); + jstring line = (jstring)((*env)->GetObjectArrayElement(env, lines, i)); (*env)->ReleaseStringUTFChars(env, line, c_lines[i]); } diff --git a/src/client/src/tscParseLineProtocol.c b/src/client/src/tscParseLineProtocol.c index 6c30c786435e3c81b87632fac0d2b14430c0a2c9..778c0cfb47ad255663b403412931be71a3200d2a 100644 --- a/src/client/src/tscParseLineProtocol.c +++ b/src/client/src/tscParseLineProtocol.c @@ -371,12 +371,15 @@ static int32_t applySchemaAction(TAOS* taos, SSchemaAction* action, SSmlLinesInf buildColumnDescription(action->alterSTable.field, result+n, capacity-n, &outBytes); TAOS_RES* res = taos_query(taos, result); //TODO async doAsyncQuery code = taos_errno(res); + char* errStr = taos_errstr(res); + char* begin = strstr(errStr, "duplicated column names"); + bool tscDupColNames = (begin != NULL); if (code != TSDB_CODE_SUCCESS) { - tscError("SML:0x%"PRIx64" apply schema action. error: %s", info->id, taos_errstr(res)); + tscError("SML:0x%"PRIx64" apply schema action. error: %s", info->id, errStr); } taos_free_result(res); - if (code == TSDB_CODE_MND_FIELD_ALREAY_EXIST || code == TSDB_CODE_TSC_DUP_COL_NAMES) { + if (code == TSDB_CODE_MND_FIELD_ALREAY_EXIST || code == TSDB_CODE_MND_TAG_ALREAY_EXIST || tscDupColNames) { TAOS_RES* res2 = taos_query(taos, "RESET QUERY CACHE"); code = taos_errno(res2); if (code != TSDB_CODE_SUCCESS) { @@ -392,12 +395,15 @@ static int32_t applySchemaAction(TAOS* taos, SSchemaAction* action, SSmlLinesInf result+n, capacity-n, &outBytes); TAOS_RES* res = taos_query(taos, result); //TODO async doAsyncQuery code = taos_errno(res); + char* errStr = taos_errstr(res); + char* begin = strstr(errStr, "duplicated column names"); + bool tscDupColNames = (begin != NULL); if (code != TSDB_CODE_SUCCESS) { tscError("SML:0x%"PRIx64" apply schema action. error : %s", info->id, taos_errstr(res)); } taos_free_result(res); - if (code == TSDB_CODE_MND_TAG_ALREAY_EXIST || code == TSDB_CODE_TSC_DUP_COL_NAMES) { + if (code == TSDB_CODE_MND_TAG_ALREAY_EXIST || code == TSDB_CODE_MND_FIELD_ALREAY_EXIST || tscDupColNames) { TAOS_RES* res2 = taos_query(taos, "RESET QUERY CACHE"); code = taos_errno(res2); if (code != TSDB_CODE_SUCCESS) { diff --git a/src/client/src/tscSQLParser.c b/src/client/src/tscSQLParser.c index 264d8a1a5121e9698b4b59184cb8a78d558ccf3e..5a2e35045018a4780be8dff09c13454a039d4ab7 100644 --- a/src/client/src/tscSQLParser.c +++ b/src/client/src/tscSQLParser.c @@ -40,7 +40,6 @@ #include "qScript.h" #include "ttype.h" #include "qFilter.h" -#include "httpInt.h" #define DEFAULT_PRIMARY_TIMESTAMP_COL_NAME "_c0" @@ -72,7 +71,6 @@ static int convertTimestampStrToInt64(tVariant *pVar, int32_t precision); static bool serializeExprListToVariant(SArray* pList, tVariant **dst, int16_t colType, uint8_t precision); static bool has(SArray* pFieldList, int32_t startIdx, const char* name); -static char* cloneCurrentDBName(SSqlObj* pSql); static int32_t getDelimiterIndex(SStrToken* pTableName); static bool validateTableColumnInfo(SArray* pFieldList, SSqlCmd* pCmd); static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pCmd); @@ -432,7 +430,7 @@ int32_t readFromFile(char *name, uint32_t *len, void **buf) { int32_t handleUserDefinedFunc(SSqlObj* pSql, struct SSqlInfo* pInfo) { - const char *msg1 = "function name is too long"; + const char *msg1 = "invalidate function name"; const char *msg2 = "path is too long"; const char *msg3 = "invalid outputtype"; const char *msg4 = "invalid script"; @@ -449,7 +447,10 @@ int32_t handleUserDefinedFunc(SSqlObj* pSql, struct SSqlInfo* pInfo) { } createInfo->name.z[createInfo->name.n] = 0; - + // funcname's naming rule is same to column + if (validateColumnName(createInfo->name.z) != TSDB_CODE_SUCCESS) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); + } strdequote(createInfo->name.z); if (strlen(createInfo->name.z) >= TSDB_FUNC_NAME_LEN) { @@ -1607,7 +1608,8 @@ int32_t validateOneTag(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { for (int32_t i = 0; i < numOfTags + numOfCols; ++i) { if (strncasecmp(pTagField->name, pSchema[i].name, sizeof(pTagField->name) - 1) == 0) { - return tscErrorMsgWithCode(TSDB_CODE_TSC_DUP_COL_NAMES, tscGetErrorMsgPayload(pCmd), pTagField->name, NULL); + //return tscErrorMsgWithCode(TSDB_CODE_TSC_DUP_COL_NAMES, tscGetErrorMsgPayload(pCmd), pTagField->name, NULL); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "duplicated column names"); } } @@ -1660,7 +1662,8 @@ int32_t validateOneColumn(SSqlCmd* pCmd, TAOS_FIELD* pColField) { // field name must be unique for (int32_t i = 0; i < numOfTags + numOfCols; ++i) { if (strncasecmp(pColField->name, pSchema[i].name, sizeof(pColField->name) - 1) == 0) { - return tscErrorMsgWithCode(TSDB_CODE_TSC_DUP_COL_NAMES, tscGetErrorMsgPayload(pCmd), pColField->name, NULL); + //return tscErrorMsgWithCode(TSDB_CODE_TSC_DUP_COL_NAMES, tscGetErrorMsgPayload(pCmd), pColField->name, NULL); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "duplicated column names"); } } @@ -1680,34 +1683,6 @@ static bool has(SArray* pFieldList, int32_t startIdx, const char* name) { static char* getAccountId(SSqlObj* pSql) { return pSql->pTscObj->acctId; } -static char* cloneCurrentDBName(SSqlObj* pSql) { - char *p = NULL; - HttpContext *pCtx = NULL; - - pthread_mutex_lock(&pSql->pTscObj->mutex); - STscObj *pTscObj = pSql->pTscObj; - switch (pTscObj->from) { - case TAOS_REQ_FROM_HTTP: - pCtx = pSql->param; - if (pCtx && pCtx->db[0] != '\0') { - char db[TSDB_ACCT_ID_LEN + TSDB_DB_NAME_LEN] = {0}; - int32_t len = sprintf(db, "%s%s%s", pTscObj->acctId, TS_PATH_DELIMITER, pCtx->db); - assert(len <= sizeof(db)); - - p = strdup(db); - } - break; - default: - break; - } - if (p == NULL) { - p = strdup(pSql->pTscObj->db); - } - pthread_mutex_unlock(&pSql->pTscObj->mutex); - - return p; -} - /* length limitation, strstr cannot be applied */ static int32_t getDelimiterIndex(SStrToken* pTableName) { for (uint32_t i = 0; i < pTableName->n; ++i) { @@ -8812,7 +8787,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf * select server_status(); * select server_version(); * select client_version(); - * select current_database(); + * select database(); */ if (pSqlNode->from == NULL) { assert(pSqlNode->fillType == NULL && pSqlNode->pGroupby == NULL && pSqlNode->pWhere == NULL && diff --git a/src/client/src/tscServer.c b/src/client/src/tscServer.c index ef686fe9553871cbb7cce4bff1ca990b0ad3dcd1..9d523f273016e258940c67eaa1596153de0998eb 100644 --- a/src/client/src/tscServer.c +++ b/src/client/src/tscServer.c @@ -1403,7 +1403,6 @@ int32_t tscBuildSyncDbReplicaMsg(SSqlObj* pSql, SSqlInfo *pInfo) { } int32_t tscBuildShowMsg(SSqlObj *pSql, SSqlInfo *pInfo) { - STscObj *pObj = pSql->pTscObj; SSqlCmd *pCmd = &pSql->cmd; pCmd->msgType = TSDB_MSG_TYPE_CM_SHOW; pCmd->payloadLen = sizeof(SShowMsg) + 100; @@ -1426,9 +1425,9 @@ int32_t tscBuildShowMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } if (tNameIsEmpty(&pTableMetaInfo->name)) { - pthread_mutex_lock(&pObj->mutex); - tstrncpy(pShowMsg->db, pObj->db, sizeof(pShowMsg->db)); - pthread_mutex_unlock(&pObj->mutex); + char *p = cloneCurrentDBName(pSql); + tstrncpy(pShowMsg->db, p, sizeof(pShowMsg->db)); + tfree(p); } else { tNameGetFullDbName(&pTableMetaInfo->name, pShowMsg->db); } @@ -2924,13 +2923,15 @@ int32_t tscGetTableMetaImpl(SSqlObj* pSql, STableMetaInfo *pTableMetaInfo, bool // just make runtime happy if (pTableMetaInfo->tableMetaCapacity != 0 && pTableMetaInfo->pTableMeta != NULL) { memset(pTableMetaInfo->pTableMeta, 0, pTableMetaInfo->tableMetaCapacity); - } + } + if (NULL == taosHashGetCloneExt(tscTableMetaMap, name, len, NULL, (void **)&(pTableMetaInfo->pTableMeta), &pTableMetaInfo->tableMetaCapacity)) { tfree(pTableMetaInfo->pTableMeta); } STableMeta* pMeta = pTableMetaInfo->pTableMeta; STableMeta* pSTMeta = (STableMeta *)(pSql->pBuf); + if (pMeta && pMeta->id.uid > 0) { // in case of child table, here only get the if (pMeta->tableType == TSDB_CHILD_TABLE) { @@ -2940,6 +2941,8 @@ int32_t tscGetTableMetaImpl(SSqlObj* pSql, STableMetaInfo *pTableMetaInfo, bool return getTableMetaFromMnode(pSql, pTableMetaInfo, autocreate); } } + + tscDebug("0x%"PRIx64 " %s retrieve tableMeta from cache, numOfCols:%d, numOfTags:%d", pSql->self, name, pMeta->tableInfo.numOfColumns, pMeta->tableInfo.numOfTags); return TSDB_CODE_SUCCESS; } diff --git a/src/client/src/tscUtil.c b/src/client/src/tscUtil.c index b96fdabd5cc87a57f3e9720ffe74dcf35bd6a310..31631560af12a01e27d5a71acaecb724cc822b5d 100644 --- a/src/client/src/tscUtil.c +++ b/src/client/src/tscUtil.c @@ -29,6 +29,7 @@ #include "tsclient.h" #include "ttimer.h" #include "ttokendef.h" +#include "httpInt.h" static void freeQueryInfoImpl(SQueryInfo* pQueryInfo); @@ -5112,3 +5113,31 @@ void tscRemoveCachedTableMeta(STableMetaInfo* pTableMetaInfo, uint64_t id) { taosHashRemove(tscTableMetaMap, fname, len); tscDebug("0x%"PRIx64" remove table meta %s, numOfRemain:%d", id, fname, (int32_t) taosHashGetSize(tscTableMetaMap)); } + +char* cloneCurrentDBName(SSqlObj* pSql) { + char *p = NULL; + HttpContext *pCtx = NULL; + + pthread_mutex_lock(&pSql->pTscObj->mutex); + STscObj *pTscObj = pSql->pTscObj; + switch (pTscObj->from) { + case TAOS_REQ_FROM_HTTP: + pCtx = pSql->param; + if (pCtx && pCtx->db[0] != '\0') { + char db[TSDB_ACCT_ID_LEN + TSDB_DB_NAME_LEN] = {0}; + int32_t len = sprintf(db, "%s%s%s", pTscObj->acctId, TS_PATH_DELIMITER, pCtx->db); + assert(len <= sizeof(db)); + + p = strdup(db); + } + break; + default: + break; + } + if (p == NULL) { + p = strdup(pSql->pTscObj->db); + } + pthread_mutex_unlock(&pSql->pTscObj->mutex); + + return p; +} diff --git a/src/common/inc/tglobal.h b/src/common/inc/tglobal.h index 30ae6faf1c87b30bf6d87900f30662cff08f0029..285145f8f63693e193817a83e079f978ce4ebe6a 100644 --- a/src/common/inc/tglobal.h +++ b/src/common/inc/tglobal.h @@ -130,6 +130,7 @@ extern int32_t tsHttpMaxThreads; extern int8_t tsHttpEnableCompress; extern int8_t tsHttpEnableRecordSql; extern int8_t tsTelegrafUseFieldNum; +extern int8_t tsHttpDbNameMandatory; // mqtt extern int8_t tsEnableMqttModule; diff --git a/src/common/src/tglobal.c b/src/common/src/tglobal.c index 2d1c6780d198da8a025c3ad7f976b32dcf0bad0b..0f1fe5e2b6761629509a82b29fd3e742341c6e90 100644 --- a/src/common/src/tglobal.c +++ b/src/common/src/tglobal.c @@ -175,6 +175,7 @@ int32_t tsHttpMaxThreads = 2; int8_t tsHttpEnableCompress = 1; int8_t tsHttpEnableRecordSql = 0; int8_t tsTelegrafUseFieldNum = 0; +int8_t tsHttpDbNameMandatory = 0; // mqtt int8_t tsEnableMqttModule = 0; // not finished yet, not started it by default @@ -1287,6 +1288,16 @@ static void doInitGlobalConfig(void) { cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); + cfg.option = "httpDbNameMandatory"; + cfg.ptr = &tsHttpDbNameMandatory; + cfg.valType = TAOS_CFG_VTYPE_INT8; + cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG; + cfg.minValue = 0; + cfg.maxValue = 1; + cfg.ptrLength = 0; + cfg.unitType = TAOS_CFG_UTYPE_NONE; + taosInitConfigOption(cfg); + // debug flag cfg.option = "numOfLogLines"; cfg.ptr = &tsNumOfLogLines; diff --git a/src/connector/jdbc/pom.xml b/src/connector/jdbc/pom.xml index fbeeeb56d30357294663a5ebf64608c57e066a7d..6b9fc9d96ce16700ee1243ef7c148a423a965d0b 100644 --- a/src/connector/jdbc/pom.xml +++ b/src/connector/jdbc/pom.xml @@ -117,7 +117,6 @@ **/DatetimeBefore1970Test.java **/FailOverTest.java **/InvalidResultSetPointerTest.java - **/RestfulConnectionTest.java **/TSDBJNIConnectorTest.java **/TaosInfoMonitorTest.java **/UnsignedNumberJniTest.java diff --git a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBError.java b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBError.java index 977ae665152fd1627cf25005061a4ca2aaa4e488..bdb3ea410005cadd865de1d9e080dd5b9f20834f 100644 --- a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBError.java +++ b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBError.java @@ -40,13 +40,13 @@ public class TSDBError { TSDBErrorMap.put(TSDBErrorNumbers.ERROR_SUBSCRIBE_FAILED, "failed to create subscription"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNSUPPORTED_ENCODING, "Unsupported encoding"); - TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_TDENGINE_ERROR, "internal error of database"); + TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_TDENGINE_ERROR, "internal error of database, please see taoslog for more details"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL, "JNI connection is NULL"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL, "JNI result set is NULL"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0, "invalid num of fields"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_SQL_NULL, "empty sql string"); TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_FETCH_END, "fetch to the end of resultSet"); - TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_OUT_OF_MEMORY, "JNI alloc memory failed"); + TSDBErrorMap.put(TSDBErrorNumbers.ERROR_JNI_OUT_OF_MEMORY, "JNI alloc memory failed, please see taoslog for more details"); } public static SQLException createSQLException(int errorCode) { diff --git a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java index c634fe2e9503ff19afae85f285d921f330562612..4a9e80ba53b096f057840eab67e61418332dbf81 100755 --- a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java +++ b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java @@ -278,25 +278,20 @@ public class TSDBJNIConnector { private native int validateCreateTableSqlImp(long connection, byte[] sqlBytes); public long prepareStmt(String sql) throws SQLException { - long stmt; - try { - stmt = prepareStmtImp(sql.getBytes(), this.taos); - } catch (Exception e) { - e.printStackTrace(); - throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNSUPPORTED_ENCODING); - } + long stmt = prepareStmtImp(sql.getBytes(), this.taos); if (stmt == TSDBConstants.JNI_CONNECTION_NULL) { - throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL); + throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL, "connection already closed"); } - if (stmt == TSDBConstants.JNI_SQL_NULL) { throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_SQL_NULL); } - if (stmt == TSDBConstants.JNI_OUT_OF_MEMORY) { throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_OUT_OF_MEMORY); } + if (stmt == TSDBConstants.JNI_TDENGINE_ERROR) { + throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_TDENGINE_ERROR); + } return stmt; } @@ -313,8 +308,7 @@ public class TSDBJNIConnector { private native int setBindTableNameImp(long stmt, String name, long conn); public void setBindTableNameAndTags(long stmt, String tableName, int numOfTags, ByteBuffer tags, ByteBuffer typeList, ByteBuffer lengthList, ByteBuffer nullList) throws SQLException { - int code = setTableNameTagsImp(stmt, tableName, numOfTags, tags.array(), typeList.array(), lengthList.array(), - nullList.array(), this.taos); + int code = setTableNameTagsImp(stmt, tableName, numOfTags, tags.array(), typeList.array(), lengthList.array(), nullList.array(), this.taos); if (code != TSDBConstants.JNI_SUCCESS) { throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_UNKNOWN, "failed to bind table name and corresponding tags"); } diff --git a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBResultSetBlockData.java b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBResultSetBlockData.java index 6211f61dc505d2ccba5f11f3aacc980771b1a110..ff49677b01fa1c3a4d482cebd51269d5f1589e43 100644 --- a/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBResultSetBlockData.java +++ b/src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBResultSetBlockData.java @@ -32,6 +32,7 @@ import java.util.List; import com.taosdata.jdbc.utils.NullType; public class TSDBResultSetBlockData { + private static final int BINARY_LENGTH_OFFSET = 2; private int numOfRows = 0; private int rowIndex = 0; @@ -404,10 +405,8 @@ public class TSDBResultSetBlockData { case TSDBConstants.TSDB_DATA_TYPE_BINARY: { ByteBuffer bb = (ByteBuffer) this.colData.get(col); - bb.position(fieldSize * this.rowIndex); - + bb.position((fieldSize + BINARY_LENGTH_OFFSET) * this.rowIndex); int length = bb.getShort(); - byte[] dest = new byte[length]; bb.get(dest, 0, length); if (NullType.isBinaryNull(dest, length)) { @@ -419,16 +418,13 @@ public class TSDBResultSetBlockData { case TSDBConstants.TSDB_DATA_TYPE_NCHAR: { ByteBuffer bb = (ByteBuffer) this.colData.get(col); - bb.position(fieldSize * this.rowIndex); - + bb.position((fieldSize + BINARY_LENGTH_OFFSET) * this.rowIndex); int length = bb.getShort(); - byte[] dest = new byte[length]; bb.get(dest, 0, length); if (NullType.isNcharNull(dest, length)) { return null; } - try { String charset = TaosGlobalConfig.getCharset(); return new String(dest, charset); diff --git a/src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/UseNowInsertTimestampTest.java b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/UseNowInsertTimestampTest.java new file mode 100644 index 0000000000000000000000000000000000000000..fbce021d1bff3655eedcf487dbcbf4747d5f9897 --- /dev/null +++ b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/UseNowInsertTimestampTest.java @@ -0,0 +1,84 @@ +package com.taosdata.jdbc.cases; + +import org.junit.Before; +import org.junit.Test; + +import java.sql.*; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +public class UseNowInsertTimestampTest { + String url = "jdbc:TAOS://127.0.0.1:6030/?user=root&password=taosdata"; + + @Test + public void millisec() { + try (Connection conn = DriverManager.getConnection(url)) { + Statement stmt = conn.createStatement(); + stmt.execute("drop database if exists test"); + stmt.execute("create database if not exists test precision 'ms'"); + stmt.execute("use test"); + stmt.execute("create table weather(ts timestamp, f1 int)"); + stmt.execute("insert into weather values(now, 1)"); + + ResultSet rs = stmt.executeQuery("select * from weather"); + rs.next(); + Timestamp ts = rs.getTimestamp("ts"); + assertEquals(13, Long.toString(ts.getTime()).length()); + + int nanos = ts.getNanos(); + assertEquals(0, nanos % 1000_000); + + stmt.execute("drop database if exists test"); + } catch (SQLException e) { + e.printStackTrace(); + } + } + + @Test + public void microsec() { + try (Connection conn = DriverManager.getConnection(url)) { + Statement stmt = conn.createStatement(); + stmt.execute("drop database if exists test"); + stmt.execute("create database if not exists test precision 'us'"); + stmt.execute("use test"); + stmt.execute("create table weather(ts timestamp, f1 int)"); + stmt.execute("insert into weather values(now, 1)"); + + ResultSet rs = stmt.executeQuery("select * from weather"); + rs.next(); + Timestamp ts = rs.getTimestamp("ts"); + int nanos = ts.getNanos(); + + assertEquals(0, nanos % 1000); + + stmt.execute("drop database if exists test"); + } catch (SQLException e) { + e.printStackTrace(); + } + } + + @Test + public void nanosec() { + try (Connection conn = DriverManager.getConnection(url)) { + Statement stmt = conn.createStatement(); + stmt.execute("drop database if exists test"); + stmt.execute("create database if not exists test precision 'ns'"); + stmt.execute("use test"); + stmt.execute("create table weather(ts timestamp, f1 int)"); + stmt.execute("insert into weather values(now, 1)"); + + ResultSet rs = stmt.executeQuery("select * from weather"); + rs.next(); + + Timestamp ts = rs.getTimestamp("ts"); + + int nanos = ts.getNanos(); + assertTrue(nanos % 1000 != 0); + + stmt.execute("drop database if exists test"); + } catch (SQLException e) { + e.printStackTrace(); + } + } +} diff --git a/src/connector/jdbc/src/test/java/com/taosdata/jdbc/rs/RestfulConnectionTest.java b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/rs/RestfulConnectionTest.java index abd60f5b63d46b406f19b6be9dcbbab6b786de12..1c5c03aacb5e7ed5683c75414975224a67d49e21 100644 --- a/src/connector/jdbc/src/test/java/com/taosdata/jdbc/rs/RestfulConnectionTest.java +++ b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/rs/RestfulConnectionTest.java @@ -9,6 +9,8 @@ import org.junit.Test; import java.sql.*; import java.util.Properties; +import static org.junit.Assert.assertEquals; + public class RestfulConnectionTest { private static final String host = "127.0.0.1"; @@ -26,7 +28,7 @@ public class RestfulConnectionTest { ResultSet rs = stmt.executeQuery("select server_status()"); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); } catch (SQLException e) { e.printStackTrace(); } @@ -38,7 +40,7 @@ public class RestfulConnectionTest { ResultSet rs = pstmt.executeQuery(); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); } @Test(expected = SQLFeatureNotSupportedException.class) @@ -49,7 +51,7 @@ public class RestfulConnectionTest { @Test public void nativeSQL() throws SQLException { String nativeSQL = conn.nativeSQL("select * from log.log"); - Assert.assertEquals("select * from log.log", nativeSQL); + assertEquals("select * from log.log", nativeSQL); } @Test @@ -87,7 +89,7 @@ public class RestfulConnectionTest { public void getMetaData() throws SQLException { DatabaseMetaData meta = conn.getMetaData(); Assert.assertNotNull(meta); - Assert.assertEquals("com.taosdata.jdbc.rs.RestfulDriver", meta.getDriverName()); + assertEquals("com.taosdata.jdbc.rs.RestfulDriver", meta.getDriverName()); } @Test @@ -103,25 +105,25 @@ public class RestfulConnectionTest { @Test public void setCatalog() throws SQLException { conn.setCatalog("test"); - Assert.assertEquals("test", conn.getCatalog()); + assertEquals("test", conn.getCatalog()); } @Test public void getCatalog() throws SQLException { conn.setCatalog("log"); - Assert.assertEquals("log", conn.getCatalog()); + assertEquals("log", conn.getCatalog()); } @Test(expected = SQLFeatureNotSupportedException.class) public void setTransactionIsolation() throws SQLException { conn.setTransactionIsolation(Connection.TRANSACTION_NONE); - Assert.assertEquals(Connection.TRANSACTION_NONE, conn.getTransactionIsolation()); + assertEquals(Connection.TRANSACTION_NONE, conn.getTransactionIsolation()); conn.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED); } @Test public void getTransactionIsolation() throws SQLException { - Assert.assertEquals(Connection.TRANSACTION_NONE, conn.getTransactionIsolation()); + assertEquals(Connection.TRANSACTION_NONE, conn.getTransactionIsolation()); } @Test @@ -140,7 +142,7 @@ public class RestfulConnectionTest { ResultSet rs = stmt.executeQuery("select server_status()"); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY); } @@ -152,7 +154,7 @@ public class RestfulConnectionTest { ResultSet rs = pstmt.executeQuery(); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); conn.prepareStatement("select server_status", ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY); } @@ -175,13 +177,13 @@ public class RestfulConnectionTest { @Test(expected = SQLFeatureNotSupportedException.class) public void setHoldability() throws SQLException { conn.setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT); - Assert.assertEquals(ResultSet.HOLD_CURSORS_OVER_COMMIT, conn.getHoldability()); + assertEquals(ResultSet.HOLD_CURSORS_OVER_COMMIT, conn.getHoldability()); conn.setHoldability(ResultSet.CLOSE_CURSORS_AT_COMMIT); } @Test public void getHoldability() throws SQLException { - Assert.assertEquals(ResultSet.HOLD_CURSORS_OVER_COMMIT, conn.getHoldability()); + assertEquals(ResultSet.HOLD_CURSORS_OVER_COMMIT, conn.getHoldability()); } @Test(expected = SQLFeatureNotSupportedException.class) @@ -210,7 +212,7 @@ public class RestfulConnectionTest { ResultSet rs = stmt.executeQuery("select server_status()"); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY, ResultSet.HOLD_CURSORS_OVER_COMMIT); } @@ -222,7 +224,7 @@ public class RestfulConnectionTest { ResultSet rs = pstmt.executeQuery(); rs.next(); int status = rs.getInt("server_status()"); - Assert.assertEquals(1, status); + assertEquals(1, status); conn.prepareStatement("select server_status", ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY, ResultSet.HOLD_CURSORS_OVER_COMMIT); } @@ -299,11 +301,11 @@ public class RestfulConnectionTest { Properties info = conn.getClientInfo(); String charset = info.getProperty(TSDBDriver.PROPERTY_KEY_CHARSET); - Assert.assertEquals("UTF-8", charset); + assertEquals("UTF-8", charset); String locale = info.getProperty(TSDBDriver.PROPERTY_KEY_LOCALE); - Assert.assertEquals("en_US.UTF-8", locale); + assertEquals("en_US.UTF-8", locale); String timezone = info.getProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE); - Assert.assertEquals("UTC-8", timezone); + assertEquals("UTC-8", timezone); } @Test @@ -313,11 +315,11 @@ public class RestfulConnectionTest { conn.setClientInfo(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); String charset = conn.getClientInfo(TSDBDriver.PROPERTY_KEY_CHARSET); - Assert.assertEquals("UTF-8", charset); + assertEquals("UTF-8", charset); String locale = conn.getClientInfo(TSDBDriver.PROPERTY_KEY_LOCALE); - Assert.assertEquals("en_US.UTF-8", locale); + assertEquals("en_US.UTF-8", locale); String timezone = conn.getClientInfo(TSDBDriver.PROPERTY_KEY_TIME_ZONE); - Assert.assertEquals("UTC-8", timezone); + assertEquals("UTC-8", timezone); } @Test(expected = SQLFeatureNotSupportedException.class) @@ -345,14 +347,15 @@ public class RestfulConnectionTest { conn.abort(null); } - @Test(expected = SQLFeatureNotSupportedException.class) + @Test public void setNetworkTimeout() throws SQLException { conn.setNetworkTimeout(null, 1000); } - @Test(expected = SQLFeatureNotSupportedException.class) + @Test public void getNetworkTimeout() throws SQLException { - conn.getNetworkTimeout(); + int timeout = conn.getNetworkTimeout(); + assertEquals(0, timeout); } @Test diff --git a/src/connector/python/taos/cinterface.py b/src/connector/python/taos/cinterface.py index 42dac3c2e8f2647193d88b250db083b64cac3b6b..a1b6fe312b5725b8bf030701608d93c3e0c85706 100644 --- a/src/connector/python/taos/cinterface.py +++ b/src/connector/python/taos/cinterface.py @@ -102,9 +102,7 @@ _libtaos.taos_get_client_info.restype = c_char_p def taos_get_client_info(): # type: () -> str - """Get client version info. - 获取客户端版本信息。 - """ + """Get client version info.""" return _libtaos.taos_get_client_info().decode() @@ -114,6 +112,7 @@ _libtaos.taos_get_server_info.argtypes = (c_void_p,) def taos_get_server_info(connection): # type: (c_void_p) -> str + """Get server version as string.""" return _libtaos.taos_get_server_info(connection).decode() @@ -134,11 +133,10 @@ _libtaos.taos_connect.argtypes = c_char_p, c_char_p, c_char_p, c_char_p, c_uint1 def taos_connect(host=None, user="root", password="taosdata", db=None, port=0): # type: (None|str, str, str, None|str, int) -> c_void_p """Create TDengine database connection. - 创建数据库连接，初始化连接上下文。其中需要用户提供的参数包含： - - host: server hostname/FQDN, TDengine管理主节点的FQDN - - user: user name/用户名 - - password: user password / 用户密码 + - host: server hostname/FQDN + - user: user name + - password: user password - db: database name (optional) - port: server port @@ -187,11 +185,10 @@ _libtaos.taos_connect_auth.argtypes = c_char_p, c_char_p, c_char_p, c_char_p, c_ def taos_connect_auth(host=None, user="root", auth="", db=None, port=0): # type: (None|str, str, str, None|str, int) -> c_void_p - """ - 创建数据库连接，初始化连接上下文。其中需要用户提供的参数包含： + """Connect server with auth token. - - host: server hostname/FQDN, TDengine管理主节点的FQDN - - user: user name/用户名 + - host: server hostname/FQDN + - user: user name - auth: base64 encoded auth token - db: database name (optional) - port: server port diff --git a/src/kit/taosdemo/taosdemo.c b/src/kit/taosdemo/taosdemo.c index df3a741c10d9080d77efe3180cbb497ca4c1093e..9ae70b384bcd98b3bbbb4cd65b6152c72b9c3cb1 100644 --- a/src/kit/taosdemo/taosdemo.c +++ b/src/kit/taosdemo/taosdemo.c @@ -291,7 +291,6 @@ typedef struct SSuperTable_S { uint64_t lenOfTagOfOneRow; char* sampleDataBuf; - char* sampleBindArray; //int sampleRowCount; //int sampleUsePos; @@ -438,7 +437,8 @@ typedef struct SQueryMetaInfo_S { typedef struct SThreadInfo_S { TAOS * taos; TAOS_STMT *stmt; - int64_t *bind_ts; + char* sampleBindArray; + int64_t *bind_ts; int threadID; char db_name[TSDB_DB_NAME_LEN]; uint32_t time_precision; @@ -754,12 +754,11 @@ static void printHelp() { "Set the replica parameters of the database, Default 1, min: 1, max: 3."); printf("%s%s%s%s\n", indent, "-m, --table-prefix=TABLEPREFIX", "\t", "Table prefix name. Default is 'd'."); - printf("%s%s%s%s\n", indent, "-s, --sql-file=FILE", "\t\t", "The select sql file."); + printf("%s%s%s%s\n", indent, "-s, --sql-file=FILE", "\t\t", + "The select sql file."); printf("%s%s%s%s\n", indent, "-N, --normal-table", "\t\t", "Use normal table flag."); printf("%s%s%s%s\n", indent, "-o, --output=FILE", "\t\t", "Direct output to the named file. Default is './output.txt'."); - printf("%s%s%s%s\n", indent, "-s, --sql-file=FILE", "\t\t", - "The select sql file."); printf("%s%s%s%s\n", indent, "-q, --query-mode=MODE", "\t\t", "Query mode -- 0: SYNC, 1: ASYNC. Default is SYNC."); printf("%s%s%s%s\n", indent, "-b, --data-type=DATATYPE", "\t", @@ -806,10 +805,14 @@ static void printHelp() { "Give this help list"); printf("%s%s%s%s\n", indent, " --usage\t", "\t\t", "Give a short usage message"); - printf("%s%s\n", indent, "-V, --version\t\t\tPrint version info."); + printf("%s%s\n", indent, "-V, --version\t\t\tPrint program version."); /* printf("%s%s%s%s\n", indent, "-D", indent, "Delete database if exists. 0: no, 1: yes, default is 1"); */ + printf("\nMandatory or optional arguments to long options are also mandatory or optional\n\ +for any corresponding short options.\n\ +\n\ +Report bugs to .\n"); } static bool isStringNumber(char *input) @@ -827,6 +830,18 @@ static bool isStringNumber(char *input) return true; } +static void errorWrongValue(char *program, char *wrong_arg, char *wrong_value) +{ + fprintf(stderr, "%s %s: %s is an invalid value\n", program, wrong_arg, wrong_value); + fprintf(stderr, "Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); +} + +static void errorUnreconized(char *program, char *wrong_arg) +{ + fprintf(stderr, "%s: unrecognized options '%s'\n", program, wrong_arg); + fprintf(stderr, "Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); +} + static void errorPrintReqArg(char *program, char *wrong_arg) { fprintf(stderr, @@ -857,23 +872,31 @@ static void errorPrintReqArg3(char *program, char *wrong_arg) static void parse_args(int argc, char *argv[], SArguments *arguments) { for (int i = 1; i < argc; i++) { - if ((strcmp(argv[i], "-f") == 0) + if ((0 == strncmp(argv[i], "-f", strlen("-f"))) || (0 == strncmp(argv[i], "--file", strlen("--file")))) { - if (2 == strlen(argv[i])) { - arguments->demo_mode = false; + arguments->demo_mode = false; - if (NULL == argv[i+1]) { - errorPrintReqArg3(argv[0], "f"); + if (2 == strlen(argv[i])) { + if (i+1 == argc) { + errorPrintReqArg(argv[0], "f"); exit(EXIT_FAILURE); } arguments->metaFile = argv[++i]; + } else if (0 == strncmp(argv[i], "-f", strlen("-f"))) { + arguments->metaFile = (char *)(argv[i] + strlen("-f")); } else if (strlen("--file") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--file"); - exit(EXIT_FAILURE); + if (i+1 == argc) { + errorPrintReqArg3(argv[0], "--file"); + exit(EXIT_FAILURE); + } + arguments->metaFile = argv[++i]; } else if (0 == strncmp(argv[i], "--file=", strlen("--file="))) { arguments->metaFile = (char *)(argv[i] + strlen("--file=")); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-c") == 0) + } else if ((0 == strncmp(argv[i], "-c", strlen("-c"))) || (0 == strncmp(argv[i], "--config-dir", strlen("--config-dir")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -881,13 +904,21 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } tstrncpy(configDir, argv[++i], TSDB_FILENAME_LEN); + } else if (0 == strncmp(argv[i], "-c", strlen("-c"))) { + tstrncpy(configDir, (char *)(argv[i] + strlen("-c")), TSDB_FILENAME_LEN); } else if (strlen("--config-dir") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--config-dir"); - exit(EXIT_FAILURE); + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--config-dir"); + exit(EXIT_FAILURE); + } + tstrncpy(configDir, argv[++i], TSDB_FILENAME_LEN); } else if (0 == strncmp(argv[i], "--config-dir=", strlen("--config-dir="))) { tstrncpy(configDir, (char *)(argv[i] + strlen("--config-dir=")), TSDB_FILENAME_LEN); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-h") == 0) + } else if ((0 == strncmp(argv[i], "-h", strlen("-h"))) || (0 == strncmp(argv[i], "--host", strlen("--host")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -895,13 +926,23 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->host = argv[++i]; + } else if (0 == strncmp(argv[i], "-h", strlen("-h"))) { + arguments->host = (char *)(argv[i] + strlen("-h")); } else if (strlen("--host") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--host"); - exit(EXIT_FAILURE); + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--host"); + exit(EXIT_FAILURE); + } + arguments->host = argv[++i]; } else if (0 == strncmp(argv[i], "--host=", strlen("--host="))) { arguments->host = (char *)(argv[i] + strlen("--host=")); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-P") == 0) + } else if (strcmp(argv[i], "-PP") == 0) { + arguments->performance_print = true; + } else if ((0 == strncmp(argv[i], "-P", strlen("-P"))) || (0 == strncmp(argv[i], "--port", strlen("--port")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -912,15 +953,28 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->port = atoi(argv[++i]); - } else if (strlen("--port") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--port"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--port=", strlen("--port="))) { if (isStringNumber((char *)(argv[i] + strlen("--port=")))) { arguments->port = atoi((char *)(argv[i]+strlen("--port="))); } + } else if (0 == strncmp(argv[i], "-P", strlen("-P"))) { + if (isStringNumber((char *)(argv[i] + strlen("-P")))) { + arguments->port = atoi((char *)(argv[i]+strlen("-P"))); + } + } else if (strlen("--port") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--port"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--port"); + exit(EXIT_FAILURE); + } + arguments->port = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-I") == 0) + } else if ((0 == strncmp(argv[i], "-I", strlen("-I"))) || (0 == strncmp(argv[i], "--interface", strlen("--interface")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -934,13 +988,10 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } else if (0 == strcasecmp(argv[i+1], "stmt")) { arguments->iface = STMT_IFACE; } else { - errorPrintReqArg(argv[0], "I"); + errorWrongValue(argv[0], "-I", argv[i+1]); exit(EXIT_FAILURE); } i++; - } else if (strlen("--interface") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--interface"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--interface=", strlen("--interface="))) { if (0 == strcasecmp((char *)(argv[i] + strlen("--interface=")), "taosc")) { arguments->iface = TAOSC_IFACE; @@ -952,8 +1003,39 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg3(argv[0], "--interface"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-I", strlen("-I"))) { + if (0 == strcasecmp((char *)(argv[i] + strlen("-I")), "taosc")) { + arguments->iface = TAOSC_IFACE; + } else if (0 == strcasecmp((char *)(argv[i] + strlen("-I")), "rest")) { + arguments->iface = REST_IFACE; + } else if (0 == strcasecmp((char *)(argv[i] + strlen("-I")), "stmt")) { + arguments->iface = STMT_IFACE; + } else { + errorWrongValue(argv[0], "-I", + (char *)(argv[i] + strlen("-I"))); + exit(EXIT_FAILURE); + } + } else if (strlen("--interface") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--interface"); + exit(EXIT_FAILURE); + } + if (0 == strcasecmp(argv[i+1], "taosc")) { + arguments->iface = TAOSC_IFACE; + } else if (0 == strcasecmp(argv[i+1], "rest")) { + arguments->iface = REST_IFACE; + } else if (0 == strcasecmp(argv[i+1], "stmt")) { + arguments->iface = STMT_IFACE; + } else { + errorWrongValue(argv[0], "--interface", argv[i+1]); + exit(EXIT_FAILURE); + } + i++; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-u") == 0) + } else if ((0 == strncmp(argv[i], "-u", strlen("-u"))) || (0 == strncmp(argv[i], "--user", strlen("--user")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -961,13 +1043,21 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->user = argv[++i]; - } else if (strlen("--user") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--user"); - exit(EXIT_FAILURE); + } else if (0 == strncmp(argv[i], "-u", strlen("-u"))) { + arguments->user = (char *)(argv[i++] + strlen("-u")); } else if (0 == strncmp(argv[i], "--user=", strlen("--user="))) { arguments->user = (char *)(argv[i++] + strlen("--user=")); + } else if (strlen("--user") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--user"); + exit(EXIT_FAILURE); + } + arguments->user = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strncmp(argv[i], "-p", 2) == 0) + } else if ((0 == strncmp(argv[i], "-p", strlen("-p"))) || (0 == strcmp(argv[i], "--password"))) { if ((strlen(argv[i]) == 2) || (0 == strcmp(argv[i], "--password"))) { printf("Enter password: "); @@ -979,21 +1069,29 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } else { tstrncpy(arguments->password, (char *)(argv[i] + 2), SHELL_MAX_PASSWORD_LEN); } - } else if ((strcmp(argv[i], "-o") == 0) + } else if ((0 == strncmp(argv[i], "-o", strlen("-o"))) || (0 == strncmp(argv[i], "--output", strlen("--output")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { - errorPrintReqArg3(argv[0], "o"); + errorPrintReqArg3(argv[0], "--output"); exit(EXIT_FAILURE); } arguments->output_file = argv[++i]; - } else if (strlen("--output") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--output"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--output=", strlen("--output="))) { arguments->output_file = (char *)(argv[i++] + strlen("--output=")); + } else if (0 == strncmp(argv[i], "-o", strlen("-o"))) { + arguments->output_file = (char *)(argv[i++] + strlen("-o")); + } else if (strlen("--output") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--output"); + exit(EXIT_FAILURE); + } + arguments->output_file = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-s") == 0) + } else if ((0 == strncmp(argv[i], "-s", strlen("-s"))) || (0 == strncmp(argv[i], "--sql-file", strlen("--sql-file")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1001,13 +1099,21 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->sqlFile = argv[++i]; + } else if (0 == strncmp(argv[i], "--sql-file=", strlen("--sql-file="))) { + arguments->sqlFile = (char *)(argv[i++] + strlen("--sql-file=")); + } else if (0 == strncmp(argv[i], "-s", strlen("-s"))) { + arguments->sqlFile = (char *)(argv[i++] + strlen("-s")); } else if (strlen("--sql-file") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--sql-file"); + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--sql-file"); + exit(EXIT_FAILURE); + } + arguments->sqlFile = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); exit(EXIT_FAILURE); - } else if (0 == strncmp(argv[i], "--sql-file=", strlen("--sql-file="))) { - arguments->host = (char *)(argv[i++] + strlen("--sql-file=")); } - } else if ((strcmp(argv[i], "-q") == 0) + } else if ((0 == strncmp(argv[i], "-q", strlen("-q"))) || (0 == strncmp(argv[i], "--query-mode", strlen("--query-mode")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1018,9 +1124,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->async_mode = atoi(argv[++i]); - } else if (strlen("--query-mode") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--query-mode"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--query-mode=", strlen("--query-mode="))) { if (isStringNumber((char *)(argv[i] + strlen("--query-mode=")))) { arguments->async_mode = atoi((char *)(argv[i]+strlen("--query-mode="))); @@ -1028,8 +1131,27 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--query-mode"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-q", strlen("-q"))) { + if (isStringNumber((char *)(argv[i] + strlen("-q")))) { + arguments->async_mode = atoi((char *)(argv[i]+strlen("-q"))); + } else { + errorPrintReqArg2(argv[0], "-q"); + exit(EXIT_FAILURE); + } + } else if (strlen("--query-mode") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--query-mode"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--query-mode"); + exit(EXIT_FAILURE); + } + arguments->async_mode = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-T") == 0) + } else if ((0 == strncmp(argv[i], "-T", strlen("-T"))) || (0 == strncmp(argv[i], "--threads", strlen("--threads")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1040,9 +1162,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->num_of_threads = atoi(argv[++i]); - } else if (strlen("--threads") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--threads"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--threads=", strlen("--threads="))) { if (isStringNumber((char *)(argv[i] + strlen("--threads=")))) { arguments->num_of_threads = atoi((char *)(argv[i]+strlen("--threads="))); @@ -1050,8 +1169,27 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--threads"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-T", strlen("-T"))) { + if (isStringNumber((char *)(argv[i] + strlen("-T")))) { + arguments->num_of_threads = atoi((char *)(argv[i]+strlen("-T"))); + } else { + errorPrintReqArg2(argv[0], "-T"); + exit(EXIT_FAILURE); + } + } else if (strlen("--threads") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--threads"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--threads"); + exit(EXIT_FAILURE); + } + arguments->num_of_threads = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-i") == 0) + } else if ((0 == strncmp(argv[i], "-i", strlen("-i"))) || (0 == strncmp(argv[i], "--insert-interval", strlen("--insert-interval")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1062,25 +1200,34 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->insert_interval = atoi(argv[++i]); - } else if (strlen("--insert-interval")== strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--insert-interval"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--insert-interval=", strlen("--insert-interval="))) { - if (isStringNumber((char *)(argv[i] + 18))) { + if (isStringNumber((char *)(argv[i] + strlen("--insert-interval=")))) { arguments->insert_interval = atoi((char *)(argv[i]+strlen("--insert-interval="))); } else { errorPrintReqArg3(argv[0], "--insert-innterval"); exit(EXIT_FAILURE); } - } - if ((argc == i+1) || - (!isStringNumber(argv[i+1]))) { - printHelp(); - errorPrint("%s", "\n\t-i need a number following!\n"); + } else if (0 == strncmp(argv[i], "-i", strlen("-i"))) { + if (isStringNumber((char *)(argv[i] + strlen("-i")))) { + arguments->insert_interval = atoi((char *)(argv[i]+strlen("-i"))); + } else { + errorPrintReqArg3(argv[0], "-i"); + exit(EXIT_FAILURE); + } + } else if (strlen("--insert-interval")== strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--insert-interval"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--insert-interval"); + exit(EXIT_FAILURE); + } + arguments->insert_interval = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); exit(EXIT_FAILURE); } - arguments->insert_interval = atoi(argv[++i]); - } else if ((strcmp(argv[i], "-S") == 0) + } else if ((0 == strncmp(argv[i], "-S", strlen("-S"))) || (0 == strncmp(argv[i], "--time-step", strlen("--time-step")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1091,9 +1238,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->async_mode = atoi(argv[++i]); - } else if (strlen("--time-step") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--time-step"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--time-step=", strlen("--time-step="))) { if (isStringNumber((char *)(argv[i] + strlen("--time-step=")))) { arguments->async_mode = atoi((char *)(argv[i]+strlen("--time-step="))); @@ -1101,6 +1245,25 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--time-step"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-S", strlen("-S"))) { + if (isStringNumber((char *)(argv[i] + strlen("-S")))) { + arguments->async_mode = atoi((char *)(argv[i]+strlen("-S"))); + } else { + errorPrintReqArg2(argv[0], "-S"); + exit(EXIT_FAILURE); + } + } else if (strlen("--time-step") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--time-step"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--time-step"); + exit(EXIT_FAILURE); + } + arguments->async_mode = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } } else if (strcmp(argv[i], "-qt") == 0) { if ((argc == i+1) @@ -1110,9 +1273,9 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->query_times = atoi(argv[++i]); - } else if ((strcmp(argv[i], "-B") == 0) + } else if ((0 == strncmp(argv[i], "-B", strlen("-B"))) || (0 == strncmp(argv[i], "--interlace-rows", strlen("--interlace-rows")))) { - if (2 == strlen(argv[i])) { + if (strlen("-B") == strlen(argv[i])) { if (argc == i+1) { errorPrintReqArg(argv[0], "B"); exit(EXIT_FAILURE); @@ -1121,9 +1284,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->interlace_rows = atoi(argv[++i]); - } else if (strlen("--interlace-rows")== strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--interlace-rows"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--interlace-rows=", strlen("--interlace-rows="))) { if (isStringNumber((char *)(argv[i] + strlen("--interlace-rows=")))) { arguments->interlace_rows = atoi((char *)(argv[i]+strlen("--interlace-rows="))); @@ -1131,10 +1291,29 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--interlace-rows"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-B", strlen("-B"))) { + if (isStringNumber((char *)(argv[i] + strlen("-B")))) { + arguments->interlace_rows = atoi((char *)(argv[i]+strlen("-B"))); + } else { + errorPrintReqArg2(argv[0], "-B"); + exit(EXIT_FAILURE); + } + } else if (strlen("--interlace-rows")== strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--interlace-rows"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--interlace-rows"); + exit(EXIT_FAILURE); + } + arguments->interlace_rows = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-r") == 0) + } else if ((0 == strncmp(argv[i], "-r", strlen("-r"))) || (0 == strncmp(argv[i], "--rec-per-req", 13))) { - if (2 == strlen(argv[i])) { + if (strlen("-r") == strlen(argv[i])) { if (argc == i+1) { errorPrintReqArg(argv[0], "r"); exit(EXIT_FAILURE); @@ -1143,9 +1322,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->num_of_RPR = atoi(argv[++i]); - } else if (strlen("--rec-per-req")== strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--rec-per-req"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--rec-per-req=", strlen("--rec-per-req="))) { if (isStringNumber((char *)(argv[i] + strlen("--rec-per-req=")))) { arguments->num_of_RPR = atoi((char *)(argv[i]+strlen("--rec-per-req="))); @@ -1153,8 +1329,27 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--rec-per-req"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-r", strlen("-r"))) { + if (isStringNumber((char *)(argv[i] + strlen("-r")))) { + arguments->num_of_RPR = atoi((char *)(argv[i]+strlen("-r"))); + } else { + errorPrintReqArg2(argv[0], "-r"); + exit(EXIT_FAILURE); + } + } else if (strlen("--rec-per-req")== strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--rec-per-req"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--rec-per-req"); + exit(EXIT_FAILURE); + } + arguments->num_of_RPR = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-t") == 0) + } else if ((0 == strncmp(argv[i], "-t", strlen("-t"))) || (0 == strncmp(argv[i], "--tables", strlen("--tables")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1165,9 +1360,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->num_of_tables = atoi(argv[++i]); - } else if (strlen("--tables") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--tables"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--tables=", strlen("--tables="))) { if (isStringNumber((char *)(argv[i] + strlen("--tables=")))) { arguments->num_of_tables = atoi((char *)(argv[i]+strlen("--tables="))); @@ -1175,8 +1367,29 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--tables"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-t", strlen("-t"))) { + if (isStringNumber((char *)(argv[i] + strlen("-t")))) { + arguments->num_of_tables = atoi((char *)(argv[i]+strlen("-t"))); + } else { + errorPrintReqArg2(argv[0], "-t"); + exit(EXIT_FAILURE); + } + } else if (strlen("--tables") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--tables"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--tables"); + exit(EXIT_FAILURE); + } + arguments->num_of_tables = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-n") == 0) + + g_totalChildTables = arguments->num_of_tables; + } else if ((0 == strncmp(argv[i], "-n", strlen("-n"))) || (0 == strncmp(argv[i], "--records", strlen("--records")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1187,9 +1400,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->num_of_DPT = atoi(argv[++i]); - } else if (strlen("--records") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--records"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--records=", strlen("--records="))) { if (isStringNumber((char *)(argv[i] + strlen("--records=")))) { arguments->num_of_DPT = atoi((char *)(argv[i]+strlen("--records="))); @@ -1197,25 +1407,49 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--records"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-n", strlen("-n"))) { + if (isStringNumber((char *)(argv[i] + strlen("-n")))) { + arguments->num_of_DPT = atoi((char *)(argv[i]+strlen("-n"))); + } else { + errorPrintReqArg2(argv[0], "-n"); + exit(EXIT_FAILURE); + } + } else if (strlen("--records") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--records"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--records"); + exit(EXIT_FAILURE); + } + arguments->num_of_DPT = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - - g_totalChildTables = arguments->num_of_DPT; - } else if ((strcmp(argv[i], "-d") == 0) - + } else if ((0 == strncmp(argv[i], "-d", strlen("-d"))) || (0 == strncmp(argv[i], "--database", strlen("--database")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { - errorPrintReqArg3(argv[0], "d"); + errorPrintReqArg(argv[0], "d"); exit(EXIT_FAILURE); } arguments->database = argv[++i]; - } else if (strlen("--database") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--database"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--database=", strlen("--database="))) { arguments->output_file = (char *)(argv[i] + strlen("--database=")); + } else if (0 == strncmp(argv[i], "-d", strlen("-d"))) { + arguments->output_file = (char *)(argv[i] + strlen("-d")); + } else if (strlen("--database") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--database"); + exit(EXIT_FAILURE); + } + arguments->database = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-l") == 0) + } else if ((0 == strncmp(argv[i], "-l", strlen("-l"))) || (0 == strncmp(argv[i], "--columns", strlen("--columns")))) { arguments->demo_mode = false; if (2 == strlen(argv[i])) { @@ -1227,16 +1461,32 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->num_of_CPR = atoi(argv[++i]); - } else if (strlen("--columns")== strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--columns"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--columns=", strlen("--columns="))) { if (isStringNumber((char *)(argv[i] + strlen("--columns=")))) { - arguments->num_of_CPR = atoi((char *)(argv[i]+strlen("--columns"))); + arguments->num_of_CPR = atoi((char *)(argv[i]+strlen("--columns="))); } else { errorPrintReqArg2(argv[0], "--columns"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-l", strlen("-l"))) { + if (isStringNumber((char *)(argv[i] + strlen("-l")))) { + arguments->num_of_CPR = atoi((char *)(argv[i]+strlen("-l"))); + } else { + errorPrintReqArg2(argv[0], "-l"); + exit(EXIT_FAILURE); + } + } else if (strlen("--columns")== strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--columns"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--columns"); + exit(EXIT_FAILURE); + } + arguments->num_of_CPR = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } if (arguments->num_of_CPR > MAX_NUM_COLUMNS) { @@ -1251,7 +1501,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { for (int col = arguments->num_of_CPR; col < MAX_NUM_COLUMNS; col++) { arguments->datatype[col] = NULL; } - } else if ((strcmp(argv[i], "-b") == 0) + } else if ((0 == strncmp(argv[i], "-b", strlen("-b"))) || (0 == strncmp(argv[i], "--data-type", strlen("--data-type")))) { arguments->demo_mode = false; @@ -1262,11 +1512,19 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } dataType = argv[++i]; - } else if (strlen("--data-type") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--data-type"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--data-type=", strlen("--data-type="))) { dataType = (char *)(argv[i] + strlen("--data-type=")); + } else if (0 == strncmp(argv[i], "-b", strlen("-b"))) { + dataType = (char *)(argv[i] + strlen("-b")); + } else if (strlen("--data-type") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--data-type"); + exit(EXIT_FAILURE); + } + dataType = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } if (strstr(dataType, ",") == NULL) { @@ -1315,7 +1573,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } arguments->datatype[index] = NULL; } - } else if ((strcmp(argv[i], "-w") == 0) + } else if ((0 == strncmp(argv[i], "-w", strlen("-w"))) || (0 == strncmp(argv[i], "--binwidth", strlen("--binwidth")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1326,9 +1584,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->binwidth = atoi(argv[++i]); - } else if (strlen("--binwidth") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--binwidth"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--binwidth=", strlen("--binwidth="))) { if (isStringNumber((char *)(argv[i] + strlen("--binwidth=")))) { arguments->binwidth = atoi((char *)(argv[i]+strlen("--binwidth="))); @@ -1336,21 +1591,47 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--binwidth"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-w", strlen("-w"))) { + if (isStringNumber((char *)(argv[i] + strlen("-w")))) { + arguments->binwidth = atoi((char *)(argv[i]+strlen("-w"))); + } else { + errorPrintReqArg2(argv[0], "-w"); + exit(EXIT_FAILURE); + } + } else if (strlen("--binwidth") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--binwidth"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--binwidth"); + exit(EXIT_FAILURE); + } + arguments->binwidth = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } - } else if ((strcmp(argv[i], "-m") == 0) + } else if ((0 == strncmp(argv[i], "-m", strlen("-m"))) || (0 == strncmp(argv[i], "--table-prefix", strlen("--table-prefix")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { - errorPrintReqArg3(argv[0], "m"); + errorPrintReqArg(argv[0], "m"); exit(EXIT_FAILURE); } arguments->tb_prefix = argv[++i]; - } else if (strlen("--table-prefix") == strlen(argv[i]) - || (strlen("--table-prefix=") == strlen(argv[i]))) { - errorPrintReqArg3(argv[0], "--table-prefix"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--table-prefix=", strlen("--table-prefix="))) { arguments->tb_prefix = (char *)(argv[i] + strlen("--table-prefix=")); + } else if (0 == strncmp(argv[i], "-m", strlen("-m"))) { + arguments->tb_prefix = (char *)(argv[i] + strlen("-m")); + } else if (strlen("--table-prefix") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--table-prefix"); + exit(EXIT_FAILURE); + } + arguments->tb_prefix = argv[++i]; + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } } else if ((strcmp(argv[i], "-N") == 0) || (0 == strcmp(argv[i], "--normal-table"))) { @@ -1369,9 +1650,55 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { arguments->debug_print = true; } else if (strcmp(argv[i], "-gg") == 0) { arguments->verbose_print = true; - } else if (strcmp(argv[i], "-PP") == 0) { - arguments->performance_print = true; - } else if ((strcmp(argv[i], "-O") == 0) + } else if ((0 == strncmp(argv[i], "-R", strlen("-R"))) + || (0 == strncmp(argv[i], "--disorder-range", + strlen("--disorder-range")))) { + if (strlen("-R") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg(argv[0], "R"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "R"); + exit(EXIT_FAILURE); + } + arguments->disorderRange = atoi(argv[++i]); + } else if (0 == strncmp(argv[i], "--disorder-range=", + strlen("--disorder-range="))) { + if (isStringNumber((char *)(argv[i] + strlen("--disorder-range=")))) { + arguments->disorderRange = + atoi((char *)(argv[i]+strlen("--disorder-range="))); + } else { + errorPrintReqArg2(argv[0], "--disorder-range"); + exit(EXIT_FAILURE); + } + } else if (0 == strncmp(argv[i], "-R", strlen("-R"))) { + if (isStringNumber((char *)(argv[i] + strlen("-R")))) { + arguments->disorderRange = + atoi((char *)(argv[i]+strlen("-R"))); + } else { + errorPrintReqArg2(argv[0], "-R"); + exit(EXIT_FAILURE); + } + + if (arguments->disorderRange < 0) { + errorPrint("Invalid disorder range %d, will be set to %d\n", + arguments->disorderRange, 1000); + arguments->disorderRange = 1000; + } + } else if (strlen("--disorder-range") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--disorder-range"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--disorder-range"); + exit(EXIT_FAILURE); + } + arguments->disorderRange = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); + } + } else if ((0 == strncmp(argv[i], "-O", strlen("-O"))) || (0 == strncmp(argv[i], "--disorder", strlen("--disorder")))) { if (2 == strlen(argv[i])) { if (argc == i+1) { @@ -1382,9 +1709,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->disorderRatio = atoi(argv[++i]); - } else if (strlen("--disorder") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--disorder"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--disorder=", strlen("--disorder="))) { if (isStringNumber((char *)(argv[i] + strlen("--disorder=")))) { arguments->disorderRatio = atoi((char *)(argv[i]+strlen("--disorder="))); @@ -1392,6 +1716,25 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--disorder"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-O", strlen("-O"))) { + if (isStringNumber((char *)(argv[i] + strlen("-O")))) { + arguments->disorderRatio = atoi((char *)(argv[i]+strlen("-O"))); + } else { + errorPrintReqArg2(argv[0], "-O"); + exit(EXIT_FAILURE); + } + } else if (strlen("--disorder") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--disorder"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--disorder"); + exit(EXIT_FAILURE); + } + arguments->disorderRatio = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } if (arguments->disorderRatio > 50) { @@ -1405,38 +1748,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { arguments->disorderRatio, 0); arguments->disorderRatio = 0; } - } else if ((strcmp(argv[i], "-R") == 0) - || (0 == strncmp(argv[i], "--disorder-range", - strlen("--disorder-range")))) { - if (2 == strlen(argv[i])) { - if (argc == i+1) { - errorPrintReqArg(argv[0], "R"); - exit(EXIT_FAILURE); - } else if (!isStringNumber(argv[i+1])) { - errorPrintReqArg2(argv[0], "R"); - exit(EXIT_FAILURE); - } - arguments->disorderRange = atoi(argv[++i]); - } else if (strlen("--disorder-range") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--disorder-range"); - exit(EXIT_FAILURE); - } else if (0 == strncmp(argv[i], "--disorder-range=", - strlen("--disorder-range="))) { - if (isStringNumber((char *)(argv[i] + strlen("--disorder-range=")))) { - arguments->disorderRange = - atoi((char *)(argv[i]+strlen("--disorder-rnage="))); - } else { - errorPrintReqArg2(argv[0], "--disorder-range"); - exit(EXIT_FAILURE); - } - } - - if (arguments->disorderRange < 0) { - errorPrint("Invalid disorder range %d, will be set to %d\n", - arguments->disorderRange, 1000); - arguments->disorderRange = 1000; - } - } else if ((strcmp(argv[i], "-a") == 0) + } else if ((0 == strncmp(argv[i], "-a", strlen("-a"))) || (0 == strncmp(argv[i], "--replica", strlen("--replica")))) { if (2 == strlen(argv[i])) { @@ -1448,9 +1760,6 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { exit(EXIT_FAILURE); } arguments->replica = atoi(argv[++i]); - } else if (strlen("--replica") == strlen(argv[i])) { - errorPrintReqArg3(argv[0], "--replica"); - exit(EXIT_FAILURE); } else if (0 == strncmp(argv[i], "--replica=", strlen("--replica="))) { if (isStringNumber((char *)(argv[i] + strlen("--replica=")))) { @@ -1460,7 +1769,28 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { errorPrintReqArg2(argv[0], "--replica"); exit(EXIT_FAILURE); } + } else if (0 == strncmp(argv[i], "-a", strlen("-a"))) { + if (isStringNumber((char *)(argv[i] + strlen("-a")))) { + arguments->replica = + atoi((char *)(argv[i]+strlen("-a"))); + } else { + errorPrintReqArg2(argv[0], "-a"); + exit(EXIT_FAILURE); + } + } else if (strlen("--replica") == strlen(argv[i])) { + if (argc == i+1) { + errorPrintReqArg3(argv[0], "--replica"); + exit(EXIT_FAILURE); + } else if (!isStringNumber(argv[i+1])) { + errorPrintReqArg2(argv[0], "--replica"); + exit(EXIT_FAILURE); + } + arguments->replica = atoi(argv[++i]); + } else { + errorUnreconized(argv[0], argv[i]); + exit(EXIT_FAILURE); } + if (arguments->replica > 3 || arguments->replica < 1) { errorPrint("Invalid replica value %d, will be set to %d\n", arguments->replica, 1); @@ -1492,7 +1822,15 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } else { // to simulate argp_option output if (strlen(argv[i]) > 2) { - fprintf(stderr, "%s unrecognized options '%s'\n", argv[0], argv[i]); + if (0 == strncmp(argv[i], "--", 2)) { + fprintf(stderr, "%s: unrecognized options '%s'\n", argv[0], argv[i]); + } else if (0 == strncmp(argv[i], "-", 1)) { + char tmp[2] = {0}; + tstrncpy(tmp, argv[i]+1, 2); + fprintf(stderr, "%s: invalid options -- '%s'\n", argv[0], tmp); + } else { + fprintf(stderr, "%s: Too many arguments\n", argv[0]); + } } else { fprintf(stderr, "%s invalid options -- '%s'\n", argv[0], (char *)((char *)argv[i])+1); @@ -1527,9 +1865,9 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { arguments->use_metric ? "true" : "false"); if (*(arguments->datatype)) { printf("# Specified data type: "); - for (int i = 0; i < MAX_NUM_COLUMNS; i++) - if (arguments->datatype[i]) - printf("%s,", arguments->datatype[i]); + for (int c = 0; c < MAX_NUM_COLUMNS; c++) + if (arguments->datatype[c]) + printf("%s,", arguments->datatype[c]); else break; printf("\n"); @@ -5406,20 +5744,6 @@ static void postFreeResource() { free(g_Dbs.db[i].superTbls[j].sampleDataBuf); g_Dbs.db[i].superTbls[j].sampleDataBuf = NULL; } - if (g_Dbs.db[i].superTbls[j].sampleBindArray) { - for (int k = 0; k < MAX_SAMPLES_ONCE_FROM_FILE; k++) { - uintptr_t *tmp = (uintptr_t *)(*(uintptr_t *)( - g_Dbs.db[i].superTbls[j].sampleBindArray - + sizeof(uintptr_t *) * k)); - for (int c = 1; c < g_Dbs.db[i].superTbls[j].columnCount + 1; c++) { - TAOS_BIND *bind = (TAOS_BIND *)((char *)tmp + (sizeof(TAOS_BIND) * c)); - if (bind) - tmfree(bind->buffer); - } - tmfree((char *)tmp); - } - } - tmfree((char *)g_Dbs.db[i].superTbls[j].sampleBindArray); if (0 != g_Dbs.db[i].superTbls[j].tagDataBuf) { free(g_Dbs.db[i].superTbls[j].tagDataBuf); @@ -5645,12 +5969,85 @@ static int64_t generateData(char *recBuf, char **data_type, return (int32_t)strlen(recBuf); } +static int generateSampleMemoryFromRand(SSuperTable *stbInfo) +{ + char data[MAX_DATA_SIZE]; + memset(data, 0, MAX_DATA_SIZE); + + char *buff = malloc(stbInfo->lenOfOneRow); + if (NULL == buff) { + errorPrint2("%s() LN%d, memory allocation %"PRId64" bytes failed\n", + __func__, __LINE__, stbInfo->lenOfOneRow); + exit(EXIT_FAILURE); + } + + for (int i=0; i < MAX_SAMPLES_ONCE_FROM_FILE; i++) { + uint64_t pos = 0; + memset(buff, 0, stbInfo->lenOfOneRow); + + for (int c = 0; c < stbInfo->columnCount; c++) { + char *tmp; + if (0 == strncasecmp(stbInfo->columns[c].dataType, + "BINARY", strlen("BINARY"))) { + rand_string(data, stbInfo->columns[c].dataLen); + pos += sprintf(buff + pos, "%s,", data); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "NCHAR", strlen("NCHAR"))) { + rand_string(data, stbInfo->columns[c].dataLen); + pos += sprintf(buff + pos, "%s,", data); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "INT", strlen("INT"))) { + if ((g_args.demo_mode) && (c == 1)) { + tmp = demo_voltage_int_str(); + } else { + tmp = rand_int_str(); + } + pos += sprintf(buff + pos, "%s,", tmp); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "BIGINT", strlen("BIGINT"))) { + pos += sprintf(buff + pos, "%s,", rand_bigint_str()); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "FLOAT", strlen("FLOAT"))) { + if (g_args.demo_mode) { + if (c == 0) { + tmp = demo_current_float_str(); + } else { + tmp = demo_phase_float_str(); + } + } else { + tmp = rand_float_str(); + } + pos += sprintf(buff + pos, "%s,", tmp); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "DOUBLE", strlen("DOUBLE"))) { + pos += sprintf(buff + pos, "%s,", rand_double_str()); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "SMALLINT", strlen("SMALLINT"))) { + pos += sprintf(buff + pos, "%s,", rand_smallint_str()); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "TINYINT", strlen("TINYINT"))) { + pos += sprintf(buff + pos, "%s,", rand_tinyint_str()); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "BOOL", strlen("BOOL"))) { + pos += sprintf(buff + pos, "%s,", rand_bool_str()); + } else if (0 == strncasecmp(stbInfo->columns[c].dataType, + "TIMESTAMP", strlen("TIMESTAMP"))) { + pos += sprintf(buff + pos, "%s,", rand_bigint_str()); + } + } + *(buff + pos - 1) = 0; + memcpy(stbInfo->sampleDataBuf + i * stbInfo->lenOfOneRow, buff, pos); + } + + free(buff); + return 0; +} + static int prepareSampleDataForSTable(SSuperTable *stbInfo) { - char* sampleDataBuf = NULL; - sampleDataBuf = calloc( + stbInfo->sampleDataBuf = calloc( stbInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE, 1); - if (sampleDataBuf == NULL) { + if (NULL == stbInfo->sampleDataBuf) { errorPrint2("%s() LN%d, Failed to calloc %"PRIu64" Bytes, reason:%s\n", __func__, __LINE__, stbInfo->lenOfOneRow * MAX_SAMPLES_ONCE_FROM_FILE, @@ -5658,13 +6055,16 @@ static int prepareSampleDataForSTable(SSuperTable *stbInfo) { return -1; } - stbInfo->sampleDataBuf = sampleDataBuf; - int ret = readSampleFromCsvFileToMem(stbInfo); + int ret; + if (0 == strncasecmp(stbInfo->dataSource, "sample", strlen("sample"))) + ret = readSampleFromCsvFileToMem(stbInfo); + else + ret = generateSampleMemoryFromRand(stbInfo); if (0 != ret) { errorPrint2("%s() LN%d, read sample from csv file failed.\n", __func__, __LINE__); - tmfree(sampleDataBuf); + tmfree(stbInfo->sampleDataBuf); stbInfo->sampleDataBuf = NULL; return -1; } @@ -5677,9 +6077,6 @@ static int32_t execInsert(threadInfo *pThreadInfo, uint32_t k) int32_t affectedRows; SSuperTable* stbInfo = pThreadInfo->stbInfo; - verbosePrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, - __func__, __LINE__, pThreadInfo->buffer); - uint16_t iface; if (stbInfo) iface = stbInfo->iface; @@ -5697,12 +6094,18 @@ static int32_t execInsert(threadInfo *pThreadInfo, uint32_t k) switch(iface) { case TAOSC_IFACE: + verbosePrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, + __func__, __LINE__, pThreadInfo->buffer); + affectedRows = queryDbExec( pThreadInfo->taos, pThreadInfo->buffer, INSERT_TYPE, false); break; case REST_IFACE: + verbosePrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, + __func__, __LINE__, pThreadInfo->buffer); + if (0 != postProceSql(g_Dbs.host, &g_Dbs.serv_addr, g_Dbs.port, pThreadInfo->buffer, pThreadInfo)) { affectedRows = -1; @@ -6680,12 +7083,12 @@ static int32_t prepareStbStmtBindRand( return 0; } -static int32_t prepareStbStmtBindWithSample( +static int32_t prepareStbStmtBindStartTime( + char *tableName, int64_t *ts, char *bindArray, SSuperTable *stbInfo, int64_t startTime, int32_t recSeq, - int32_t timePrec, - int64_t samplePos) + int32_t timePrec) { TAOS_BIND *bind; @@ -6702,6 +7105,10 @@ static int32_t prepareStbStmtBindWithSample( } else { *bind_ts = startTime + stbInfo->timeStampStep * recSeq; } + + verbosePrint("%s() LN%d, tableName: %s, bind_ts=%"PRId64"\n", + __func__, __LINE__, tableName, *bind_ts); + bind->buffer_length = sizeof(int64_t); bind->buffer = bind_ts; bind->length = &bind->buffer_length; @@ -6710,7 +7117,7 @@ static int32_t prepareStbStmtBindWithSample( return 0; } -static int32_t prepareStbStmtRand( +UNUSED_FUNC static int32_t prepareStbStmtRand( threadInfo *pThreadInfo, char *tableName, int64_t tableSeq, @@ -6891,14 +7298,14 @@ static int32_t prepareStbStmtWithSample( uint32_t k; for (k = 0; k < batch;) { char *bindArray = (char *)(*((uintptr_t *) - (stbInfo->sampleBindArray + (sizeof(char *)) * (*pSamplePos)))); + (pThreadInfo->sampleBindArray + (sizeof(char *)) * (*pSamplePos)))); /* columnCount + 1 (ts) */ - if (-1 == prepareStbStmtBindWithSample( + if (-1 == prepareStbStmtBindStartTime( + tableName, pThreadInfo->bind_ts, bindArray, stbInfo, startTime, k, - pThreadInfo->time_precision, - *pSamplePos + pThreadInfo->time_precision /* is column */)) { return -1; } @@ -7019,8 +7426,6 @@ static void* syncWriteInterlace(threadInfo *pThreadInfo) { int64_t nTimeStampStep; uint64_t insert_interval; - bool sourceRand; - SSuperTable* stbInfo = pThreadInfo->stbInfo; if (stbInfo) { @@ -7035,18 +7440,12 @@ static void* syncWriteInterlace(threadInfo *pThreadInfo) { maxSqlLen = stbInfo->maxSqlLen; nTimeStampStep = stbInfo->timeStampStep; insert_interval = stbInfo->insertInterval; - if (0 == strncasecmp(stbInfo->dataSource, "rand", 4)) { - sourceRand = true; - } else { - sourceRand = false; // from sample data file - } } else { insertRows = g_args.num_of_DPT; interlaceRows = g_args.interlace_rows; maxSqlLen = g_args.max_sql_len; nTimeStampStep = g_args.timestamp_step; insert_interval = g_args.insert_interval; - sourceRand = true; } debugPrint("[%d] %s() LN%d: start_table_from=%"PRIu64" ntables=%"PRId64" insertRows=%"PRIu64"\n", @@ -7131,25 +7530,14 @@ static void* syncWriteInterlace(threadInfo *pThreadInfo) { int32_t generated; if (stbInfo) { if (stbInfo->iface == STMT_IFACE) { - if (sourceRand) { - generated = prepareStbStmtRand( - pThreadInfo, - tableName, - tableSeq, - batchPerTbl, - insertRows, 0, - startTime - ); - } else { - generated = prepareStbStmtWithSample( - pThreadInfo, - tableName, - tableSeq, - batchPerTbl, - insertRows, 0, - startTime, - &(pThreadInfo->samplePos)); - } + generated = prepareStbStmtWithSample( + pThreadInfo, + tableName, + tableSeq, + batchPerTbl, + insertRows, 0, + startTime, + &(pThreadInfo->samplePos)); } else { generated = generateStbInterlaceData( pThreadInfo, @@ -7339,17 +7727,6 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { pThreadInfo->totalInsertRows = 0; pThreadInfo->totalAffectedRows = 0; - bool sourceRand; - if (stbInfo) { - if (0 == strncasecmp(stbInfo->dataSource, "rand", 4)) { - sourceRand = true; - } else { - sourceRand = false; // from sample data file - } - } else { - sourceRand = true; - } - pThreadInfo->samplePos = 0; int percentComplete = 0; @@ -7382,27 +7759,19 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { pstr += len; remainderBufLen -= len; + // measure prepare + insert + startTs = taosGetTimestampUs(); + int32_t generated; if (stbInfo) { if (stbInfo->iface == STMT_IFACE) { - if (sourceRand) { - generated = prepareStbStmtRand( - pThreadInfo, - tableName, - tableSeq, - g_args.num_of_RPR, - insertRows, - i, start_time - ); - } else { - generated = prepareStbStmtWithSample( - pThreadInfo, - tableName, - tableSeq, - g_args.num_of_RPR, - insertRows, i, start_time, - &(pThreadInfo->samplePos)); - } + generated = prepareStbStmtWithSample( + pThreadInfo, + tableName, + tableSeq, + g_args.num_of_RPR, + insertRows, i, start_time, + &(pThreadInfo->samplePos)); } else { generated = generateStbProgressiveData( stbInfo, @@ -7430,6 +7799,11 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { &remainderBufLen); } } + + verbosePrint("[%d] %s() LN%d generated=%d\n", + pThreadInfo->threadID, + __func__, __LINE__, generated); + if (generated > 0) i += generated; else @@ -7438,7 +7812,8 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { start_time += generated * timeStampStep; pThreadInfo->totalInsertRows += generated; - startTs = taosGetTimestampUs(); + // only measure insert + // startTs = taosGetTimestampUs(); int32_t affectedRows = execInsert(pThreadInfo, generated); @@ -7639,25 +8014,28 @@ static int convertHostToServAddr(char *host, uint16_t port, struct sockaddr_in * return 0; } -static int parseSampleFileToStmt(SSuperTable *stbInfo, uint32_t timePrec) +static int parseSampleFileToStmt( + threadInfo *pThreadInfo, + SSuperTable *stbInfo, uint32_t timePrec) { - stbInfo->sampleBindArray = calloc(1, sizeof(char *) * MAX_SAMPLES_ONCE_FROM_FILE); - if (stbInfo->sampleBindArray == NULL) { + pThreadInfo->sampleBindArray = + calloc(1, sizeof(char *) * MAX_SAMPLES_ONCE_FROM_FILE); + if (pThreadInfo->sampleBindArray == NULL) { errorPrint2("%s() LN%d, Failed to allocate %"PRIu64" bind array buffer\n", - __func__, __LINE__, (uint64_t)sizeof(char *) * MAX_SAMPLES_ONCE_FROM_FILE); + __func__, __LINE__, + (uint64_t)sizeof(char *) * MAX_SAMPLES_ONCE_FROM_FILE); return -1; } - for (int i=0; i < MAX_SAMPLES_ONCE_FROM_FILE; i++) { - char *bindArray = calloc(1, sizeof(TAOS_BIND) * (stbInfo->columnCount + 1)); + char *bindArray = + calloc(1, sizeof(TAOS_BIND) * (stbInfo->columnCount + 1)); if (bindArray == NULL) { errorPrint2("%s() LN%d, Failed to allocate %d bind params\n", __func__, __LINE__, (stbInfo->columnCount + 1)); return -1; } - TAOS_BIND *bind; int cursor = 0; @@ -7704,7 +8082,8 @@ static int parseSampleFileToStmt(SSuperTable *stbInfo, uint32_t timePrec) free(bindBuffer); } } - *((uintptr_t *)(stbInfo->sampleBindArray + (sizeof(char *)) * i)) = (uintptr_t)bindArray; + *((uintptr_t *)(pThreadInfo->sampleBindArray + (sizeof(char *)) * i)) = + (uintptr_t)bindArray; } return 0; @@ -7746,11 +8125,8 @@ static void startMultiThreadInsertData(int threads, char* db_name, debugPrint("%s() LN%d, start_time= %"PRId64"\n", __func__, __LINE__, start_time); - int64_t start = taosGetTimestampMs(); - // read sample data from file first - if ((stbInfo) && (0 == strncasecmp(stbInfo->dataSource, - "sample", strlen("sample")))) { + if (stbInfo) { if (0 != prepareSampleDataForSTable(stbInfo)) { errorPrint2("%s() LN%d, prepare sample data for stable failed!\n", __func__, __LINE__); @@ -7897,11 +8273,6 @@ static void startMultiThreadInsertData(int threads, char* db_name, pstr += sprintf(pstr, ")"); debugPrint("%s() LN%d, stmtBuffer: %s", __func__, __LINE__, stmtBuffer); - - if ((stbInfo) && (0 == strncasecmp(stbInfo->dataSource, - "sample", strlen("sample")))) { - parseSampleFileToStmt(stbInfo, timePrec); - } } for (int i = 0; i < threads; i++) { @@ -7934,7 +8305,6 @@ static void startMultiThreadInsertData(int threads, char* db_name, || ((stbInfo) && (stbInfo->iface == STMT_IFACE))) { - pThreadInfo->stmt = taos_stmt_init(pThreadInfo->taos); if (NULL == pThreadInfo->stmt) { free(pids); @@ -7956,6 +8326,10 @@ static void startMultiThreadInsertData(int threads, char* db_name, exit(EXIT_FAILURE); } pThreadInfo->bind_ts = malloc(sizeof(int64_t)); + + if (stbInfo) { + parseSampleFileToStmt(pThreadInfo, stbInfo, timePrec); + } } } else { pThreadInfo->taos = NULL; @@ -7984,6 +8358,8 @@ static void startMultiThreadInsertData(int threads, char* db_name, free(stmtBuffer); + int64_t start = taosGetTimestampUs(); + for (int i = 0; i < threads; i++) { pthread_join(pids[i], NULL); } @@ -8004,6 +8380,21 @@ static void startMultiThreadInsertData(int threads, char* db_name, tsem_destroy(&(pThreadInfo->lock_sem)); taos_close(pThreadInfo->taos); + if (pThreadInfo->sampleBindArray) { + for (int k = 0; k < MAX_SAMPLES_ONCE_FROM_FILE; k++) { + uintptr_t *tmp = (uintptr_t *)(*(uintptr_t *)( + pThreadInfo->sampleBindArray + + sizeof(uintptr_t *) * k)); + for (int c = 1; c < pThreadInfo->stbInfo->columnCount + 1; c++) { + TAOS_BIND *bind = (TAOS_BIND *)((char *)tmp + (sizeof(TAOS_BIND) * c)); + if (bind) + tmfree(bind->buffer); + } + tmfree((char *)tmp); + } + tmfree(pThreadInfo->sampleBindArray); + } + debugPrint("%s() LN%d, [%d] totalInsert=%"PRIu64" totalAffected=%"PRIu64"\n", __func__, __LINE__, pThreadInfo->threadID, pThreadInfo->totalInsertRows, @@ -8026,43 +8417,40 @@ static void startMultiThreadInsertData(int threads, char* db_name, if (cntDelay == 0) cntDelay = 1; avgDelay = (double)totalDelay / cntDelay; - int64_t end = taosGetTimestampMs(); + int64_t end = taosGetTimestampUs(); int64_t t = end - start; + if (0 == t) t = 1; - double tInMs = t/1000.0; + double tInMs = (double) t / 1000000.0; if (stbInfo) { - fprintf(stderr, "Spent %.2f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s.%s. %.2f records/second\n\n", + fprintf(stderr, "Spent %.4f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s.%s. %.2f records/second\n\n", tInMs, stbInfo->totalInsertRows, stbInfo->totalAffectedRows, threads, db_name, stbInfo->sTblName, - (tInMs)? - (double)(stbInfo->totalInsertRows/tInMs):FLT_MAX); + (double)(stbInfo->totalInsertRows/tInMs)); if (g_fpOfInsertResult) { fprintf(g_fpOfInsertResult, - "Spent %.2f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s.%s. %.2f records/second\n\n", + "Spent %.4f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s.%s. %.2f records/second\n\n", tInMs, stbInfo->totalInsertRows, stbInfo->totalAffectedRows, threads, db_name, stbInfo->sTblName, - (tInMs)? - (double)(stbInfo->totalInsertRows/tInMs):FLT_MAX); + (double)(stbInfo->totalInsertRows/tInMs)); } } else { - fprintf(stderr, "Spent %.2f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s %.2f records/second\n\n", + fprintf(stderr, "Spent %.4f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s %.2f records/second\n\n", tInMs, g_args.totalInsertRows, g_args.totalAffectedRows, threads, db_name, - (tInMs)? - (double)(g_args.totalInsertRows/tInMs):FLT_MAX); + (double)(g_args.totalInsertRows/tInMs)); if (g_fpOfInsertResult) { fprintf(g_fpOfInsertResult, - "Spent %.2f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s %.2f records/second\n\n", + "Spent %.4f seconds to insert rows: %"PRIu64", affected rows: %"PRIu64" with %d thread(s) into %s %.2f records/second\n\n", tInMs, g_args.totalInsertRows, g_args.totalAffectedRows, threads, db_name, - (tInMs)? - (double)(g_args.totalInsertRows/tInMs):FLT_MAX); + (double)(g_args.totalInsertRows/tInMs)); } } diff --git a/src/mnode/src/mnodeTable.c b/src/mnode/src/mnodeTable.c index a8a26156726a327bf72b4bc74fc465b2127500b8..68529ab8a240c2313ae9417bef9f4112759b0c9f 100644 --- a/src/mnode/src/mnodeTable.c +++ b/src/mnode/src/mnodeTable.c @@ -1246,13 +1246,13 @@ static int32_t mnodeAddSuperTableTag(SMnodeMsg *pMsg, SSchema schema[], int32_t if (mnodeFindSuperTableColumnIndex(pStable, schema[i].name) > 0) { mError("msg:%p, app:%p stable:%s, add tag, column:%s already exist", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, schema[i].name); - return TSDB_CODE_MND_FIELD_ALREAY_EXIST; + return TSDB_CODE_MND_TAG_ALREAY_EXIST; } if (mnodeFindSuperTableTagIndex(pStable, schema[i].name) > 0) { mError("msg:%p, app:%p stable:%s, add tag, tag:%s already exist", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, schema[i].name); - return TSDB_CODE_MND_TAG_ALREAY_EXIST; + return TSDB_CODE_MND_FIELD_ALREAY_EXIST; } } diff --git a/src/plugins/http/CMakeLists.txt b/src/plugins/http/CMakeLists.txt index 57fc2ee3a2692c239d7fa36d6e55ddae738a2720..89fdc141b66adafb9f882dd6f59eca54053aff6c 100644 --- a/src/plugins/http/CMakeLists.txt +++ b/src/plugins/http/CMakeLists.txt @@ -6,6 +6,7 @@ INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/cJson/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/lz4/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/client/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/query/inc) +INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/common/inc) INCLUDE_DIRECTORIES(inc) AUX_SOURCE_DIRECTORY(src SRC) diff --git a/src/plugins/http/src/httpRestHandle.c b/src/plugins/http/src/httpRestHandle.c index a029adec0c792be16e5288893ef0a2ef26c81f9c..24e4f90244cf52799fde1a45054875a4bf3d1850 100644 --- a/src/plugins/http/src/httpRestHandle.c +++ b/src/plugins/http/src/httpRestHandle.c @@ -19,6 +19,7 @@ #include "httpLog.h" #include "httpRestHandle.h" #include "httpRestJson.h" +#include "tglobal.h" static HttpDecodeMethod restDecodeMethod = {"rest", restProcessRequest}; static HttpDecodeMethod restDecodeMethod2 = {"restful", restProcessRequest}; @@ -111,6 +112,14 @@ bool restProcessSqlRequest(HttpContext* pContext, int32_t timestampFmt) { pContext->db[0] = '\0'; HttpString *path = &pContext->parser->path[REST_USER_USEDB_URL_POS]; + if (tsHttpDbNameMandatory) { + if (path->pos == 0) { + httpError("context:%p, fd:%d, user:%s, database name is mandatory", pContext, pContext->fd, pContext->user); + httpSendErrorResp(pContext, TSDB_CODE_HTTP_INVALID_URL); + return false; + } + } + if (path->pos > 0 && !(strlen(sql) > 4 && (sql[0] == 'u' || sql[0] == 'U') && (sql[1] == 's' || sql[1] == 'S') && (sql[2] == 'e' || sql[2] == 'E') && sql[3] == ' ')) { diff --git a/src/query/inc/sql.y b/src/query/inc/sql.y index 3d14c0f55988d5eab01ec7f10e8abb0f2d5a3c33..8466b7262ab43e49496b078dd3ca25985fe6605a 100644 --- a/src/query/inc/sql.y +++ b/src/query/inc/sql.y @@ -496,7 +496,7 @@ union(Y) ::= union(Z) UNION ALL select(X). { Y = appendSelectClause(Z, X); } cmd ::= union(X). { setSqlInfo(pInfo, X, NULL, TSDB_SQL_SELECT); } // Support for the SQL exprssion without from & where subclauses, e.g., -// select current_database() +// select database() // select server_version() // select client_version() // select server_state() diff --git a/src/tsdb/src/tsdbRead.c b/src/tsdb/src/tsdbRead.c index 71a9009c4581dae86861a67de850e275cae352f7..e3ad254bbb1a4ce0a3c504e9a799a109b7e1159a 100644 --- a/src/tsdb/src/tsdbRead.c +++ b/src/tsdb/src/tsdbRead.c @@ -3474,19 +3474,19 @@ void filterPrepare(void* expr, void* param) { if (pInfo->optr == TSDB_RELATION_IN) { int dummy = -1; - SHashObj *pObj = NULL; + SHashObj *pObj = NULL; if (pInfo->sch.colId == TSDB_TBNAME_COLUMN_INDEX) { pObj = taosHashInit(256, taosGetDefaultHashFunction(pInfo->sch.type), true, false); SArray *arr = (SArray *)(pCond->arr); for (size_t i = 0; i < taosArrayGetSize(arr); i++) { char* p = taosArrayGetP(arr, i); - strtolower(varDataVal(p), varDataVal(p)); - taosHashPut(pObj, varDataVal(p),varDataLen(p), &dummy, sizeof(dummy)); + strntolower_s(varDataVal(p), varDataVal(p), varDataLen(p)); + taosHashPut(pObj, varDataVal(p), varDataLen(p), &dummy, sizeof(dummy)); } } else { buildFilterSetFromBinary((void **)&pObj, pCond->pz, pCond->nLen); } - pInfo->q = (char *)pObj; + pInfo->q = (char *)pObj; } else if (pCond != NULL) { uint32_t size = pCond->nLen * TSDB_NCHAR_SIZE; if (size < (uint32_t)pSchema->bytes) { diff --git a/src/util/inc/tutil.h b/src/util/inc/tutil.h index 7c8fd2ed8353a086ec9f145aaaa30f4db57113cb..6bcfb5de295c5719032b81c23d16ec2b1476349e 100644 --- a/src/util/inc/tutil.h +++ b/src/util/inc/tutil.h @@ -32,6 +32,7 @@ char * strnchr(char *haystack, char needle, int32_t len, bool skipquote); char ** strsplit(char *src, const char *delim, int32_t *num); char * strtolower(char *dst, const char *src); char * strntolower(char *dst, const char *src, int32_t n); +char * strntolower_s(char *dst, const char *src, int32_t n); int64_t strnatoi(char *num, int32_t len); char * strbetween(char *string, char *begin, char *end); char * paGetToken(char *src, char **token, int32_t *tokenLen); diff --git a/src/util/src/tutil.c b/src/util/src/tutil.c index 02ea7091a65cc0decd3030b8805de9a7a1fed419..5f8c92898fc5f0abc4c733c0558befd68ac3cac7 100644 --- a/src/util/src/tutil.c +++ b/src/util/src/tutil.c @@ -202,7 +202,7 @@ char* strntolower(char *dst, const char *src, int32_t n) { if (n == 0) { *p = 0; return dst; - } + } for (c = *src++; n-- > 0; c = *src++) { if (esc) { esc = 0; @@ -224,6 +224,26 @@ char* strntolower(char *dst, const char *src, int32_t n) { return dst; } +char* strntolower_s(char *dst, const char *src, int32_t n) { + char *p = dst, c; + + assert(dst != NULL); + if (n == 0) { + return NULL; + } + + while (n-- > 0) { + c = *src; + if (c >= 'A' && c <= 'Z') { + c -= 'A' - 'a'; + } + *p++ = c; + src++; + } + + return dst; +} + char *paGetToken(char *string, char **token, int32_t *tokenLen) { char quote = 0; diff --git a/tests/pytest/fulltest.sh b/tests/pytest/fulltest.sh index f1894966a240a0470a8ffad8cb2ec621b0c3b45c..886e2a365e26dca4912ad2b52a5cd01e5385faa6 100755 --- a/tests/pytest/fulltest.sh +++ b/tests/pytest/fulltest.sh @@ -172,6 +172,10 @@ python3 test.py -f tools/taosdemoTestSampleData.py python3 test.py -f tools/taosdemoTestInterlace.py python3 test.py -f tools/taosdemoTestQuery.py +# restful test for python +python3 test.py -f restful/restful_bind_db1.py +python3 test.py -f restful/restful_bind_db2.py + # nano support python3 test.py -f tools/taosdemoAllTest/NanoTestCase/taosdemoTestSupportNanoInsert.py python3 test.py -f tools/taosdemoAllTest/NanoTestCase/taosdemoTestSupportNanoQuery.py @@ -385,6 +389,7 @@ python3 ./test.py -f insert/schemalessInsert.py python3 ./test.py -f alter/alterColMultiTimes.py python3 ./test.py -f query/queryWildcardLength.py python3 ./test.py -f query/queryTbnameUpperLower.py +python3 ./test.py -f query/query.py #======================p4-end=============== diff --git a/tests/pytest/query/last_row_cache.py b/tests/pytest/query/last_row_cache.py index 83153841496b9ef4e0b7d919b44cf0a9e05b02c6..0e11e3d60c08362c0114166dcf071990893689f8 100644 --- a/tests/pytest/query/last_row_cache.py +++ b/tests/pytest/query/last_row_cache.py @@ -25,7 +25,7 @@ class TDTestCase: self.tables = 10 self.rows = 20 - self.columns = 5 + self.columns = 100 self.perfix = 't' self.ts = 1601481600000 @@ -33,8 +33,8 @@ class TDTestCase: print("==============step1") sql = "create table st(ts timestamp, " for i in range(self.columns - 1): - sql += "c%d int, " % (i + 1) - sql += "c5 int) tags(t1 int)" + sql += "c%d bigint, " % (i + 1) + sql += "c100 bigint) tags(t1 int)" tdSql.execute(sql) for i in range(self.tables): diff --git a/tests/pytest/query/query.py b/tests/pytest/query/query.py index 8cec38780e5e873cb07d8ade45ef56b5d4ee43f7..c759e7766827e9b8e30f1b9ceb812c755fb057ae 100644 --- a/tests/pytest/query/query.py +++ b/tests/pytest/query/query.py @@ -25,6 +25,18 @@ class TDTestCase: self.ts = 1538548685000 + def bug_6387(self): + tdSql.execute("create database bug6387 ") + tdSql.execute("use bug6387 ") + tdSql.execute("create table test(ts timestamp, c1 int) tags(t1 int)") + for i in range(5000): + sql = "insert into t%d using test tags(1) values " % i + for j in range(21): + sql = sql + "(now+%ds,%d)" % (j ,j ) + tdSql.execute(sql) + tdSql.query("select count(*) from test interval(1s) group by tbname") + tdSql.checkData(0,1,1) + def run(self): tdSql.prepare() @@ -121,6 +133,9 @@ class TDTestCase: tdSql.query("select * from tb0") tdSql.checkRows(1) + + #For jira: https://jira.taosdata.com:18080/browse/TD-6387 + self.bug_6387() def stop(self): diff --git a/tests/pytest/query/queryLike.py b/tests/pytest/query/queryLike.py index 3c3b030f8f718371867f323f2496fb17b6b962e1..2bcd5906a8eaa505e1702cefce7f8b2594f53f43 100644 --- a/tests/pytest/query/queryLike.py +++ b/tests/pytest/query/queryLike.py @@ -17,7 +17,6 @@ from util.log import * from util.cases import * from util.sql import * - class TDTestCase: def init(self, conn, logSql): tdLog.debug("start to execute %s" % __file__) @@ -25,7 +24,8 @@ class TDTestCase: def run(self): tdSql.prepare() - + tdSql.execute("drop database db ") + tdSql.execute("create database if not exists db ") tdSql.execute("create table cars(ts timestamp, c nchar(2)) tags(t1 nchar(2))") tdSql.execute("insert into car0 using cars tags('aa') values(now, 'bb');") tdSql.query("select count(*) from cars where t1 like '%50 90 30 04 00 00%'") @@ -35,6 +35,132 @@ class TDTestCase: tdSql.execute("insert into car1 using test_cars tags('150 90 30 04 00 002') values(now, 'bb');") tdSql.query("select * from test_cars where t1 like '%50 90 30 04 00 00%'") tdSql.checkRows(1) + + tdSql.execute("create stable st (ts timestamp , id int , name binary(20), data double ) tags (ind int , tagg binary(20)) ") + + # check escape about tbname by show tables like + + tdSql.execute("create table tb_ using st tags (1 , 'tag_' ) ") + tdSql.execute("insert into tb_ values( now , 1 , 'tbname_' , 1.0)") + tdSql.execute("create table tba using st tags (1 , 'taga' ) ") + tdSql.execute("insert into tba values( now , 1 , 'tbnamea' , 1.0)") + tdSql.query("show tables like 'tb_'") + tdSql.checkRows(2) + tdSql.query("show tables like 'tb\_'") + tdSql.checkRows(1) + + # check escape about tbname by show tables like + tdSql.query("select * from st where tbname like 'tb\_'") + tdSql.checkRows(1) + + # check escape about regular cols + tdSql.query("select * from st where name like 'tbname\_';") + tdSql.checkRows(1) + + # check escape about tags + tdSql.query("select * from st where tagg like 'tag\_';") + tdSql.checkRows(1) + + # ======================= check multi escape =================== + + tdSql.execute("create table tb_1 using st tags (1 , 'tag_1' ) ") + tdSql.execute("insert into tb_1 values( now , 1 , 'tbname_1' , 1.0)") + tdSql.execute("create table tb_2 using st tags (2 , 'tag_2' )") + tdSql.execute("insert into tb_2 values( now , 2 , 'tbname_2' , 2.0)") + tdSql.execute("create table tb__ using st tags (3 , 'tag__' )") + tdSql.execute("insert into tb__ values( now , 3 , 'tbname__' , 2.0)") + tdSql.execute("create table tb__1 using st tags (3 , 'tag__1' )") + tdSql.execute("insert into tb__1 values( now , 1 , 'tbname__1' , 1.0)") + tdSql.execute("create table tb__2 using st tags (4 , 'tag__2' )") + tdSql.execute("create table tb___ using st tags (5 , 'tag___' )") + tdSql.execute("insert into tb___ values( now , 1 , 'tbname___' , 1.0)") + tdSql.execute("create table tb_d_ using st tags (5 , 'tag_d_' )") + tdSql.execute("insert into tb_d_ values( now , 1 , 'tbname_d_' , 1.0)") + + tdSql.execute("create table tb_d__ using st tags (5 , 'tag_d__' )") + tdSql.execute("insert into tb_d__ values( now , 1 , 'tbname_d__' , 1.0)") + tdSql.execute("create table tb____ using st tags (5 , 'tag____' )") + tdSql.execute("insert into tb____ values( now , 1 , 'tbname____' , 1.0)") + tdSql.execute("create table tb__a_ using st tags (5 , 'tag__a_' )") + tdSql.execute("insert into tb__a_ values( now , 1 , 'tbname__a_' , 1.0)") + tdSql.execute("create table tb__ab__ using st tags (5 , 'tag__ab__' )") + tdSql.execute("insert into tb__ab__ values( now , 1 , 'tbname__ab__' , 1.0)") + + # check escape about tbname by show tables like + tdSql.query("select * from st where tbname like 'tb__'") + tdSql.checkRows(3) + tdSql.query("select * from st where tbname like 'tb_\_'") + tdSql.checkRows(1) + tdSql.query("select * from st where tbname like 'tb___'") + tdSql.checkRows(4) + tdSql.query("select * from st where tbname like 'tb_\__'") + tdSql.checkRows(3) + tdSql.query("select * from st where tbname like 'tb_\_\_'") + tdSql.checkRows(1) + tdSql.query("select * from st where tbname like 'tb\__\_'") + tdSql.checkRows(1) + tdSql.query("select * from st where tbname like 'tb\__\__'") + tdSql.checkRows(2) + tdSql.query("select * from st where tbname like 'tb\__\_\_'") + tdSql.checkRows(2) + tdSql.query("select * from st where tbname like 'tb\____'") + tdSql.checkRows(3) + tdSql.query("select * from st where tbname like 'tb\_\__\_'") + tdSql.checkRows(2) + tdSql.query("select * from st where tbname like 'tb\_\_\_\_'") + tdSql.checkRows(1) + + # check escape about regular cols + tdSql.query("select * from st where name like 'tbname\_\_';") + tdSql.checkRows(1) + tdSql.query("select * from st where name like 'tbname\__';") + tdSql.checkRows(3) + tdSql.query("select * from st where name like 'tbname___';") + tdSql.checkRows(4) + tdSql.query("select * from st where name like 'tbname_\__';") + tdSql.checkRows(3) + tdSql.query("select * from st where name like 'tbname_\_\_';") + tdSql.checkRows(1) + tdSql.query("select * from st where name like 'tbname\_\__';") + tdSql.checkRows(2) + tdSql.query("select * from st where name like 'tbname____';") + tdSql.checkRows(3) + tdSql.query("select * from st where name like 'tbname\_\___';") + tdSql.checkRows(2) + tdSql.query("select * from st where name like 'tbname\_\_\__';") + tdSql.checkRows(1) + tdSql.query("select * from st where name like 'tbname\_\__\_';") + tdSql.checkRows(2) + tdSql.query("select name from st where name like 'tbname\_\_\__';") + tdSql.checkData(0,0 "tbname____") + + # check escape about tags + tdSql.query("select * from st where tagg like 'tag\_';") + tdSql.checkRows(1) + tdSql.query("select * from st where tagg like 'tag\_\_';") + tdSql.checkRows(1) + tdSql.query("select * from st where tagg like 'tag\__';") + tdSql.checkRows(3) + tdSql.query("select * from st where tagg like 'tag___';") + tdSql.checkRows(4) + tdSql.query("select * from st where tagg like 'tag_\__';") + tdSql.checkRows(3) + tdSql.query("select * from st where tagg like 'tag_\_\_';") + tdSql.checkRows(1) + tdSql.query("select * from st where tagg like 'tag\_\__';") + tdSql.checkRows(2) + tdSql.query("select * from st where tagg like 'tag____';") + tdSql.checkRows(3) + tdSql.query("select * from st where tagg like 'tag\_\___';") + tdSql.checkRows(2) + tdSql.query("select * from st where tagg like 'tag\_\_\__';") + tdSql.checkRows(1) + tdSql.query("select * from st where tagg like 'tag\_\__\_';") + tdSql.checkRows(2) + tdSql.query("select * from st where tagg like 'tag\_\__\_';") + tdSql.checkData(0,0 "tag__a_") + + os.system("rm -rf ./*.py.sql") def stop(self): tdSql.close() diff --git a/tests/pytest/restful/restful_bind_db1.py b/tests/pytest/restful/restful_bind_db1.py new file mode 100644 index 0000000000000000000000000000000000000000..9620535fd7c9388d08d0ebae9237b243fc0a099c --- /dev/null +++ b/tests/pytest/restful/restful_bind_db1.py @@ -0,0 +1,123 @@ +# ################################################################# +# Copyright (c) 2016 by TAOS Technologies, Inc. +# All rights reserved. + +# This file is proprietary and confidential to TAOS Technologies. +# No part of this file may be reproduced, stored, transmitted, +# disclosed or used in any form or by any means other than as +# expressly provided by the written permission from Jianhui Tao + +# ################################################################# + +# -*- coding: utf-8 -*- + +# TODO: after TD-4518 and TD-4510 is resolved, add the exception test case for these situations + +from distutils.log import error +import sys + +from requests.api import head +from requests.models import Response +from util.log import * +from util.cases import * +from util.sql import * +import time, datetime +import requests, json +import threading +import string +import random + + +def check_unbind_db(url, data, header): + resp = requests.post(url, data, headers = header ) + resp.encoding='utf-8' + resp = eval(resp.text) + status = resp['status'] + desc = resp['desc'] + sqls = data + if status=="error" and desc == "invalid url format": + print(" %s : check pass" %sqls) + else: + printf(" error occured , ") + sys.exit() + +def check_bind_db(url, data, header): + resp = requests.post(url, data, headers = header ) + resp.encoding='utf-8' + resp_dict = eval(resp.text) + status = resp_dict['status'] + if status =="succ": + print("%s run success!"%data) + # print(resp.text) + else : + print("%s run failed !"%data) + print(resp.text) + sys.exit() + +class TDTestCase(): + updatecfgDict={'httpDbNameMandatory':1} + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor(), logSql) + + def run(self): + tdSql.prepare() + tdSql.execute('reset query cache') + tdSql.execute('drop database if exists test') + tdSql.execute('drop database if exists db') + tdSql.execute('drop database if exists des') + tdSql.execute('create database test') + tdSql.execute('create database des') + + header = {'Authorization': 'Basic cm9vdDp0YW9zZGF0YQ=='} + url = "http://127.0.0.1:6041/rest/sql/" + + # test with no bind databases + + sqls = ["show databases;", + "use test", + "show tables;", + "show dnodes;", + "show vgroups;", + "create database db;", + "drop database db;", + "select client_version();" , + "use test", + "ALTER DATABASE test COMP 2;", + "create table tb (ts timestamp, id int , data double)", + "insert into tb values (now , 1, 1.0) ", + "select * from tb", + "show test.tables", + "show tables", + "insert into tb values (now , 2, 2.0) ", + "create table test.tb (ts timestamp, id int , data double)", + "insert into test.tb values (now , 2, 2.0) ", + "select * from tb", + "select * from test.tb"] + + for sql in sqls: + print("===================") + check_unbind_db(url,sql,header) + + print("==================="*5) + print(" check bind db about restful ") + print("==================="*5) + url = "http://127.0.0.1:6041/rest/sql/des" + for sql in sqls: + print("===================") + check_bind_db(url,sql,header) + # check data + tdSql.query("select * from test.tb") + tdSql.checkRows(1) + tdSql.query("select * from des.tb") + tdSql.checkRows(2) + + os.system('sudo timedatectl set-ntp on') + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/pytest/restful/restful_bind_db2.py b/tests/pytest/restful/restful_bind_db2.py new file mode 100644 index 0000000000000000000000000000000000000000..35aa1408a7690798d66bc0303e64fe4843dab0d8 --- /dev/null +++ b/tests/pytest/restful/restful_bind_db2.py @@ -0,0 +1,133 @@ +# ################################################################# +# Copyright (c) 2016 by TAOS Technologies, Inc. +# All rights reserved. + +# This file is proprietary and confidential to TAOS Technologies. +# No part of this file may be reproduced, stored, transmitted, +# disclosed or used in any form or by any means other than as +# expressly provided by the written permission from Jianhui Tao + +# ################################################################# + +# -*- coding: utf-8 -*- + +# TODO: after TD-4518 and TD-4510 is resolved, add the exception test case for these situations + +from distutils.log import error +import sys + +from requests.api import head +from requests.models import Response +from util.log import * +from util.cases import * +from util.sql import * +import time, datetime +import requests, json +import threading +import string +import random + +def check_res(url, data, header): + resp = requests.post(url, data, headers = header ) + resp.encoding='utf-8' + resp_dict = eval(resp.text) + status = resp_dict['status'] + if status =="succ": + print("%s run success!"%data) + # print(resp.text) + else : + print("%s run failed !"%data) + print(resp.text) + sys.exit() + +class TDTestCase(): + # updatecfgDict={'httpDbNameMandatory':0} + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor(), logSql) + + def run(self): + tdSql.prepare() + tdSql.execute('reset query cache') + tdSql.execute('drop database if exists test') + tdSql.execute('drop database if exists db') + tdSql.execute('drop database if exists des') + tdSql.execute('create database test') + tdSql.execute('create database des') + header = {'Authorization': 'Basic cm9vdDp0YW9zZGF0YQ=='} + url = "http://127.0.0.1:6041/rest/sql/" + + # test with no bind databases + + sqls = ["show databases;", + "use test", + "show tables;", + "show dnodes;", + "show vgroups;", + "create database db;", + "drop database db;", + "select client_version();" , + "use test", + "ALTER DATABASE test COMP 2;", + "create table tb (ts timestamp, id int , data double)", + "insert into tb values (now , 1, 1.0) ", + "select * from tb", + "show test.tables", + "show tables", + "insert into tb values (now , 2, 2.0) ", + "create table test.tb (ts timestamp, id int , data double)", + "insert into test.tb values (now , 3, 3.0) ", + "select * from tb", + "select * from test.tb", + "create table des.tb (ts timestamp, id int , data double)", + "insert into des.tb values (now , 3, 3.0)"] + for sql in sqls: + print("===================") + if sql == "create table test.tb (ts timestamp, id int , data double)": + resp = requests.post(url, sql, headers = header ) + print(resp.text) + print ("%s run occur error as expect ,check pass!" %(sql)) + else: + check_res(url,sql,header) + + tdSql.query("select * from test.tb") + tdSql.checkRows(3) + tdSql.query("select * from des.tb") + tdSql.checkRows(1) + + print("==================="*5) + print(" check bind db about restful ") + print("==================="*5) + + tdSql.execute('reset query cache') + tdSql.execute('drop database if exists test') + tdSql.execute('drop database if exists db') + tdSql.execute('drop database if exists des') + tdSql.execute('create database test') + tdSql.execute('create database des') + + url = "http://127.0.0.1:6041/rest/sql/des" + for sql in sqls: + print("===================") + if sql in ["create table des.tb (ts timestamp, id int , data double)"]: + resp = requests.post(url, sql, headers = header ) + print(resp.text) + print ("%s run occur error as expect ,check pass!" %(sql)) + else: + check_res(url,sql,header) + # check data + tdSql.query("select * from test.tb") + tdSql.checkRows(1) + tdSql.query("select * from des.tb") + tdSql.checkRows(3) + + os.system('sudo timedatectl set-ntp on') + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/pytest/util/sql.py b/tests/pytest/util/sql.py index dfe1e4a582c80f82a982ca270c6a3b2d144fb52f..2b654a379369c67cf906be0dde2f0cc4a309e1ea 100644 --- a/tests/pytest/util/sql.py +++ b/tests/pytest/util/sql.py @@ -21,7 +21,15 @@ import shutil import pandas as pd from util.log import * - +def _parse_datetime(timestr): + try: + return datetime.datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S.%f') + except ValueError: + pass + try: + return datetime.datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S') + except ValueError: + pass class TDSql: def __init__(self): @@ -181,7 +189,7 @@ class TDSql: tdLog.info("sql:%s, row:%d col:%d data:%d == expect:%s" % (self.sql, row, col, self.queryResult[row][col], data)) else: - if self.queryResult[row][col] == datetime.datetime.fromisoformat(data): + if self.queryResult[row][col] == _parse_datetime(data): tdLog.info("sql:%s, row:%d col:%d data:%s == expect:%s" % (self.sql, row, col, self.queryResult[row][col], data)) return diff --git a/tests/script/general/http/restful_dbname.sim b/tests/script/general/http/restful_dbname.sim new file mode 100644 index 0000000000000000000000000000000000000000..75cd198507dda306c736cbcdeef4a044f66da9de --- /dev/null +++ b/tests/script/general/http/restful_dbname.sim @@ -0,0 +1,124 @@ +system sh/stop_dnodes.sh +sleep 2000 +system sh/deploy.sh -n dnode1 -i 1 +system sh/cfg.sh -n dnode1 -c walLevel -v 1 +system sh/cfg.sh -n dnode1 -c http -v 1 +system sh/cfg.sh -n dnode1 -c httpDbNameMandatory -v 1 +system sh/exec.sh -n dnode1 -s start + +sleep 2000 +sql connect +sql drop database if exists db + +print ============================ dnode1 start + +print =============== step1 - login + +system_content curl 127.0.0.1:7111/rest/login/root/taosdata +print curl 127.0.0.1:7111/rest/login/root/taosdata -----> $system_content + +if $system_content != @{"status":"succ","code":0,"desc":"/KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04"}@ then + return -1 +endi + +print =============== step2 - execute sql without db_name + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database if not exists db' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database if not exists db' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create table table_rest (ts timestamp, i int)' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create table table_rest (ts timestamp, i int)' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'insert into table_rest (now, 1)' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'insert into table_rest values (now, 1)' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from table_rest' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from table_rest' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'drop database if exists db' 127.0.0.1:7111/rest/sql + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'drop database if exists db' 127.0.0.1:7111/rest/sql + +if $system_content != @{"status":"error","code":4354,"desc":"invalid url format"}@ then + return -1 +endi + +print =============== step3 - execute sql with db_name + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql/databases + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'show databases' 127.0.0.1:7111/rest/sql/databases + +if $system_content != @{"status":"succ","head":["name","created_time","ntables","vgroups","replica","quorum","days","keep","cache(MB)","blocks","minrows","maxrows","wallevel","fsync","comp","cachelast","precision","update","status"],"column_meta":[["name",8,32],["created_time",9,8],["ntables",4,4],["vgroups",4,4],["replica",3,2],["quorum",3,2],["days",3,2],["keep",8,24],["cache(MB)",4,4],["blocks",4,4],["minrows",4,4],["maxrows",4,4],["wallevel",2,1],["fsync",4,4],["comp",2,1],["cachelast",2,1],["precision",8,3],["update",2,1],["status",8,10]],"data":[],"rows":0}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database if not exists db' 127.0.0.1:7111/rest/sql/db + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create database if not exists db' 127.0.0.1:7111/rest/sql/db + +if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create table table_rest (ts timestamp, i int)' 127.0.0.1:7111/rest/sql/db + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'create table table_rest (ts timestamp, i int)' 127.0.0.1:7111/rest/sql/db + +if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'insert into table_rest (now, 1)' 127.0.0.1:7111/rest/sql/db + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'insert into table_rest values (1629904789233, 1)' 127.0.0.1:7111/rest/sql/db + +if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[1]],"rows":1}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from table_rest' 127.0.0.1:7111/rest/sql/db + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'select * from table_rest' 127.0.0.1:7111/rest/sql/db + +if $system_content != @{"status":"succ","head":["ts","i"],"column_meta":[["ts",9,8],["i",4,4]],"data":[["2021-08-25 23:19:49.233",1]],"rows":1}@ then + return -1 +endi + +print curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'drop database if exists db' 127.0.0.1:7111/rest/sql/db + +system_content curl -H 'Authorization: Taosd /KfeAzX/f9na8qdtNZmtONryp201ma04bEl8LcvLUd7a8qdtNZmtONryp201ma04' -d 'drop database if exists db' 127.0.0.1:7111/rest/sql/db + +if $system_content != @{"status":"succ","head":["affected_rows"],"column_meta":[["affected_rows",4,4]],"data":[[0]],"rows":1}@ then + return -1 +endi + +system sh/exec.sh -n dnode1 -s stop -x SIGINT