提交 6562e725 编写于 作者: C Cary Xu

Merge branch 'develop' into feature/TD-6117

[submodule "src/connector/go"]
path = src/connector/go
url = git@github.com:taosdata/driver-go.git
url = https://github.com/taosdata/driver-go.git
[submodule "src/connector/grafanaplugin"]
path = src/connector/grafanaplugin
url = git@github.com:taosdata/grafanaplugin.git
url = https://github.com/taosdata/grafanaplugin.git
[submodule "src/connector/hivemq-tdengine-extension"]
path = src/connector/hivemq-tdengine-extension
url = git@github.com:taosdata/hivemq-tdengine-extension.git
url = https://github.com/taosdata/hivemq-tdengine-extension.git
[submodule "tests/examples/rust"]
path = tests/examples/rust
url = https://github.com/songtianyi/tdengine-rust-bindings.git
......
......@@ -271,12 +271,12 @@ pipeline {
'''
}
timeout(time: 60, unit: 'MINUTES'){
// sh '''
// cd ${WKC}/tests/pytest
// rm -rf /var/lib/taos/*
// rm -rf /var/log/taos/*
// ./handle_crash_gen_val_log.sh
// '''
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
rm -rf /var/log/taos/*
./handle_crash_gen_val_log.sh
'''
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
......
......@@ -7,6 +7,7 @@
[![TDengine](TDenginelogo.png)](https://www.taosdata.com)
简体中文 | [English](./README.md)
很多职位正在热招中,请看[这里](https://www.taosdata.com/cn/careers/)
# TDengine 简介
......@@ -107,6 +108,12 @@ Go 连接器和 Grafana 插件在其他独立仓库,如果安装它们的话
git submodule update --init --recursive
```
如果使用 https 协议下载比较慢,可以通过修改 ~/.gitconfig 文件添加以下两行设置使用 ssh 协议下载。需要首先上传 ssh 密钥到 GitHub,详细方法请参考 GitHub 官方文档。
```
[url "git@github.com:"]
insteadOf = https://github.com/
```
## 构建 TDengine
### Linux 系统
......
......@@ -7,6 +7,7 @@
[![TDengine](TDenginelogo.png)](https://www.taosdata.com)
English | [简体中文](./README-CN.md)
We are hiring, check [here](https://www.taosdata.com/en/careers/)
# What is TDengine?
......@@ -101,6 +102,12 @@ so you should run this command in the TDengine directory to install them:
git submodule update --init --recursive
```
You can modify the file ~/.gitconfig to use ssh protocol instead of https for better download speed. You need to upload ssh public key to GitHub first. Please refer to GitHub official documentation for detail.
```
[url "git@github.com:"]
insteadOf = https://github.com/
```
## Build TDengine
### On Linux platform
......
......@@ -45,6 +45,10 @@ IF (TD_TQ)
ADD_DEFINITIONS(-D_TD_TQ_)
ENDIF ()
IF (TD_PRO)
ADD_DEFINITIONS(-D_TD_PRO_)
ENDIF ()
IF (TD_MEM_CHECK)
ADD_DEFINITIONS(-DTAOS_MEM_CHECK)
ENDIF ()
......
......@@ -49,6 +49,9 @@ IF (${DBNAME} MATCHES "power")
ELSEIF (${DBNAME} MATCHES "tq")
SET(TD_TQ TRUE)
MESSAGE(STATUS "tq is true")
ELSEIF (${DBNAME} MATCHES "pro")
SET(TD_PRO TRUE)
MESSAGE(STATUS "pro is true")
ENDIF ()
IF (${DLLTYPE} MATCHES "go")
......
......@@ -32,7 +32,7 @@ ELSEIF (TD_WINDOWS)
#INSTALL(TARGETS taos RUNTIME DESTINATION driver)
#INSTALL(TARGETS shell RUNTIME DESTINATION .)
IF (TD_MVN_INSTALLED)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.34-dist.jar DESTINATION connector/jdbc)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.35-dist.jar DESTINATION connector/jdbc)
ENDIF ()
ELSEIF (TD_DARWIN)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
......
......@@ -16,7 +16,7 @@ TDengine 的模块之一是时序数据库。但除此之外,为减少研发
采用 TDengine,可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。但需要指出的是,因充分利用了物联网时序数据的特点,它无法用来处理网络爬虫、微博、微信、电商、ERP、CRM 等通用型数据。
![TDengine技术生态图](page://images/eco_system.png)
![TDengine技术生态图](../images/eco_system.png)
<center>图 1. TDengine技术生态图</center>
## <a class="anchor" id="scenes"></a>TDengine 总体适用场景
......
......@@ -6,7 +6,7 @@
taosd包含rpc, dnode, vnode, tsdb, query, cq, sync, wal, mnode, http, monitor等模块,具体如下图:
![modules.png](page://images/architecture/modules.png)
![modules.png](../../images/architecture/modules.png)
taosd的启动入口是dnode模块,dnode然后启动其他模块,包括可选配置的http, monitor模块。taosc或dnode之间交互的消息都是通过rpc模块进行,dnode模块根据接收到的消息类型,将消息分发到vnode或mnode的消息队列,或由dnode模块自己消费。dnode的工作线程(worker)消费消息队列里的消息,交给mnode或vnode进行处理。下面对各个模块做简要说明。
......@@ -41,13 +41,13 @@ RPC模块还提供数据压缩功能,如果数据包的字节数超过系统
taosd的消息消费由dnode通过读写线程池进行控制,是系统的中枢。该模块内的结构体图如下:
![dnode.png](page://images/architecture/dnode.png)
![dnode.png](../../images/architecture/dnode.png)
## VNODE模块
vnode是一独立的数据存储查询逻辑单元,但因为一个vnode只能容许一个DB,因此vnode内部没有account, DB, user等概念。为实现更好的模块化、封装以及未来的扩展,它有很多子模块,包括负责存储的TSDB,负责查询的Query, 负责数据复制的sync,负责数据库日志的的wal, 负责连续查询的cq(continuous query), 负责事件触发的流计算的event等模块,这些子模块只与vnode模块发生关系,与其他模块没有任何调用关系。模块图如下:
![vnode.png](page://images/architecture/vnode.png)
![vnode.png](../../images/architecture/vnode.png)
vnode模块向下,与dnodeVRead,dnodeVWrite发生互动,向上,与子模块发生互动。它主要的功能有:
......
......@@ -90,7 +90,7 @@ TDengine采取的是Master-Slave模式进行同步,与流行的RAFT一致性
具体的流程图如下:
![replica-master.png](page://images/architecture/replica-master.png)
![replica-master.png](../../images/architecture/replica-master.png)
选择Master的具体规则如下:
......@@ -105,7 +105,7 @@ TDengine采取的是Master-Slave模式进行同步,与流行的RAFT一致性
如果vnode A是master, vnode B是slave, vnode A能接受客户端的写请求,而vnode B不能。当vnode A收到写的请求后,遵循下面的流程:
![replica-forward.png](page://images/architecture/replica-forward.png)
![replica-forward.png](../../images/architecture/replica-forward.png)
1. 应用对写请求做基本的合法性检查,通过,则给该请求包打上一个版本号(version, 单调递增)
2. 应用将打上版本号的写请求封装一个WAL Head, 写入WAL(Write Ahead Log)
......@@ -140,7 +140,7 @@ TDengine采取的是Master-Slave模式进行同步,与流行的RAFT一致性
整个数据恢复流程分为两大步骤,第一步,先恢复archived data(file), 然后恢复wal。具体流程如下:
![replica-restore.png](page://images/architecture/replica-restore.png)
![replica-restore.png](../../images/architecture/replica-restore.png)
1. 通过已经建立的TCP连接,发送sync req给master节点
2. master收到sync req后,以client的身份,向vnode B主动建立一新的专用于同步的TCP连接(syncFd)
......
......@@ -156,7 +156,7 @@ TDengine 的设计是基于单个硬件、软件系统不可靠,基于任何
TDengine 分布式架构的逻辑结构图如下:
![TDengine架构示意图](page://images/architecture/structure.png)
![TDengine架构示意图](../images/architecture/structure.png)
<center> 图 1 TDengine架构示意图 </center>
一个完整的 TDengine 系统是运行在一到多个物理节点上的,逻辑上,它包含数据节点(dnode)、TDengine应用驱动(taosc)以及应用(app)。系统中存在一到多个数据节点,这些数据节点组成一个集群(cluster)。应用通过taosc的API与TDengine集群进行互动。下面对每个逻辑单元进行简要介绍。
......@@ -207,7 +207,7 @@ TDengine 分布式架构的逻辑结构图如下:
为解释vnode、mnode、taosc和应用之间的关系以及各自扮演的角色,下面对写入数据这个典型操作的流程进行剖析。
![TDengine典型的操作流程](page://images/architecture/message.png)
![TDengine典型的操作流程](../images/architecture/message.png)
<center> 图 2 TDengine典型的操作流程 </center>
1. 应用通过JDBC、ODBC或其他API接口发起插入数据的请求。
......@@ -278,7 +278,7 @@ TDengine除vnode分片之外,还对时序数据按照时间段进行分区。
Master Vnode遵循下面的写入流程:
![TDengine Master写入流程](page://images/architecture/write_master.png)
![TDengine Master写入流程](../images/architecture/write_master.png)
<center> 图 3 TDengine Master写入流程 </center>
1. master vnode收到应用的数据插入请求,验证OK,进入下一步;
......@@ -292,7 +292,7 @@ Master Vnode遵循下面的写入流程:
对于slave vnode,写入流程是:
![TDengine Slave写入流程](page://images/architecture/write_slave.png)
![TDengine Slave写入流程](../images/architecture/write_slave.png)
<center> 图 4 TDengine Slave写入流程 </center>
1. slave vnode收到Master vnode转发了的数据插入请求。检查last version是否与master一致,如果一致,进入下一步。如果不一致,需要进入同步状态。
......@@ -434,7 +434,7 @@ SELECT COUNT(*) FROM d1001 WHERE ts >= '2017-7-14 00:00:00' AND ts < '2017-7-14
TDengine对每个数据采集点单独建表,但在实际应用中经常需要对不同的采集点数据进行聚合。为高效的进行聚合操作,TDengine引入超级表(STable)的概念。超级表用来代表一特定类型的数据采集点,它是包含多张表的表集合,集合里每张表的模式(schema)完全一致,但每张表都带有自己的静态标签,标签可以有多个,可以随时增加、删除和修改。应用可通过指定标签的过滤条件,对一个STable下的全部或部分表进行聚合或统计操作,这样大大简化应用的开发。其具体流程如下图所示:
![多表聚合查询原理图](page://images/architecture/multi_tables.png)
![多表聚合查询原理图](../images/architecture/multi_tables.png)
<center> 图 5 多表聚合查询原理图 </center>
1. 应用将一个查询条件发往系统;
......
......@@ -3,7 +3,7 @@
## <a class="anchor" id="queries"></a>主要查询功能
TDengine 采用 SQL 作为查询语言。应用程序可以通过 C/C++, Java, Go, Python 连接器发送 SQL 语句,用户可以通过 TDengine 提供的命令行(Command Line Interface, CLI)工具 TAOS Shell 手动执行 SQL 即席查询(Ad-Hoc Query)。TDengine 支持如下查询功能:
TDengine 采用 SQL 作为查询语言。应用程序可以通过 C/C++, Java, Go, C#, Python, Node.js 连接器发送 SQL 语句,用户可以通过 TDengine 提供的命令行(Command Line Interface, CLI)工具 TAOS Shell 手动执行 SQL 即席查询(Ad-Hoc Query)。TDengine 支持如下查询功能:
- 单列、多列数据查询
- 标签和数值的多种过滤条件:>, <, =, <>, like 等
......
......@@ -4,7 +4,7 @@
`taos-jdbcdriver` 的实现包括 2 种形式: JDBC-JNI 和 JDBC-RESTful(taos-jdbcdriver-2.0.18 开始支持 JDBC-RESTful)。 JDBC-JNI 通过调用客户端 libtaos.so(或 taos.dll )的本地方法实现, JDBC-RESTful 则在内部封装了 RESTful 接口实现。
![tdengine-connector](page://images/tdengine-jdbc-connector.png)
![tdengine-connector](../../images/tdengine-jdbc-connector.png)
上图显示了 3 种 Java 应用使用连接器访问 TDengine 的方式:
......@@ -68,18 +68,18 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
TDengine 目前支持时间戳、数字、字符、布尔类型,与 Java 对应类型转换如下:
| TDengine DataType | Java DataType |
| ----------------- | ------------------ |
| TIMESTAMP | java.sql.Timestamp |
| INT | java.lang.Integer |
| BIGINT | java.lang.Long |
| FLOAT | java.lang.Float |
| DOUBLE | java.lang.Double |
| SMALLINT | java.lang.Short |
| TINYINT | java.lang.Byte |
| BOOL | java.lang.Boolean |
| BINARY | byte array |
| NCHAR | java.lang.String |
| TDengine DataType | JDBCType (driver 版本 < 2.0.24) | JDBCType driver 版本 >= 2.0.24) |
| ----------------- | ------------------ | ------------------ |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long |
| FLOAT | java.lang.Float | java.lang.Float |
| DOUBLE | java.lang.Double | java.lang.Double |
| SMALLINT | java.lang.Short | java.lang.Short |
| TINYINT | java.lang.Byte | java.lang.Byte |
| BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String |
## 安装Java Connector
......
......@@ -2,7 +2,7 @@
TDengine提供了丰富的应用程序开发接口,其中包括C/C++、Java、Python、Go、Node.js、C# 、RESTful 等,便于用户快速开发应用。
![image-connecotr](page://images/connector.png)
![image-connecotr](../images/connector.png)
目前TDengine的连接器可支持的平台广泛,包括:X64/X86/ARM64/ARM32/MIPS/Alpha等硬件平台,以及Linux/Win64/Win32等开发环境。对照矩阵如下:
......@@ -64,8 +64,7 @@ TDengine提供了丰富的应用程序开发接口,其中包括C/C++、Java、
编辑taos.cfg文件(默认路径/etc/taos/taos.cfg),将firstEP修改为TDengine服务器的End Point,例如:h1.taos.com:6030
**提示: **
**提示:**
1. **如本机没有部署TDengine服务,仅安装了应用驱动,则taos.cfg中仅需配置firstEP,无需配置FQDN。**
2. **为防止与服务器端连接时出现“unable to resolve FQDN”错误,建议确认客户端的hosts文件已经配置正确的FQDN值。**
......@@ -1166,7 +1165,7 @@ var affectRows = cursor.execute('insert into test.weather values(now, 22.3, 34);
execute方法的返回值为该语句影响的行数,上面的sql向test库的weather表中,插入了一条数据,则返回值affectRows为1。
TDengine目前还不支持update和delete语句
TDengine 目前还不支持 delete 语句。但从 2.0.8.0 版本开始,可以通过 `CREATE DATABASE` 时指定的 UPDATE 参数来启用对数据行的 update
#### 查询
......
......@@ -32,15 +32,15 @@ allow_loading_unsigned_plugins = taosdata-tdengine-datasource
用户可以直接通过 localhost:3000 的网址,登录 Grafana 服务器(用户名/密码:admin/admin),通过左侧 `Configuration -> Data Sources` 可以添加数据源,如下图所示:
![img](page://images/connections/add_datasource1.jpg)
![img](../images/connections/add_datasource1.jpg)
点击 `Add data source` 可进入新增数据源页面,在查询框中输入 TDengine 可选择添加,如下图所示:
![img](page://images/connections/add_datasource2.jpg)
![img](../images/connections/add_datasource2.jpg)
进入数据源配置页面,按照默认提示修改相应配置即可:
![img](page://images/connections/add_datasource3.jpg)
![img](../images/connections/add_datasource3.jpg)
* Host: TDengine 集群的中任意一台服务器的 IP 地址与 TDengine RESTful 接口的端口号(6041),默认 http://localhost:6041 。
* User:TDengine 用户名。
......@@ -48,13 +48,13 @@ allow_loading_unsigned_plugins = taosdata-tdengine-datasource
点击 `Save & Test` 进行测试,成功会有如下提示:
![img](page://images/connections/add_datasource4.jpg)
![img](../images/connections/add_datasource4.jpg)
#### 创建 Dashboard
回到主界面创建 Dashboard,点击 Add Query 进入面板查询页面:
![img](page://images/connections/create_dashboard1.jpg)
![img](../images/connections/create_dashboard1.jpg)
如上图所示,在 Query 中选中 `TDengine` 数据源,在下方查询框可输入相应 sql 进行查询,具体说明如下:
......@@ -65,7 +65,7 @@ allow_loading_unsigned_plugins = taosdata-tdengine-datasource
按照默认提示查询当前 TDengine 部署所在服务器指定间隔系统内存平均使用量如下:
![img](page://images/connections/create_dashboard2.jpg)
![img](../images/connections/create_dashboard2.jpg)
> 关于如何使用Grafana创建相应的监测界面以及更多有关使用Grafana的信息,请参考Grafana官方的[文档](https://grafana.com/docs/)。
......@@ -75,11 +75,11 @@ allow_loading_unsigned_plugins = taosdata-tdengine-datasource
点击左侧 `Import` 按钮,并上传 `tdengine-grafana.json` 文件:
![img](page://images/connections/import_dashboard1.jpg)
![img](../images/connections/import_dashboard1.jpg)
导入完成之后可看到如下效果:
![img](page://images/connections/import_dashboard2.jpg)
![img](../images/connections/import_dashboard2.jpg)
## <a class="anchor" id="matlab"></a>MATLAB
......
......@@ -216,7 +216,7 @@ taosd -C
| 98 | maxBinaryDisplayWidth | | **C** | | Taos shell中binary 和 nchar字段的显示宽度上限,超过此限制的部分将被隐藏 | 5 - | 30 | 实际上限按以下规则计算:如果字段值的长度大于 maxBinaryDisplayWidth,则显示上限为 **字段名长度****maxBinaryDisplayWidth** 的较大者。否则,上限为 **字段名长度****字段值长度** 的较大者。可在 shell 中通过命令 set max_binary_display_width nn动态修改此选项 |
| 99 | queryBufferSize | | **S** | MB | 为所有并发查询占用保留的内存大小。 | | | 计算规则可以根据实际应用可能的最大并发数和表的数字相乘,再乘 170 。(2.0.15 以前的版本中,此参数的单位是字节) |
| 100 | ratioOfQueryCores | | **S** | | 设置查询线程的最大数量。 | | | 最小值0 表示只有1个查询线程;最大值2表示最大建立2倍CPU核数的查询线程。默认为1,表示最大和CPU核数相等的查询线程。该值可以为小数,即0.5表示最大建立CPU核数一半的查询线程。 |
| 101 | update | | **S** | | 允许更新已存在的数据行 | 0 \| 1 | 0 | 从 2.0.8.0 版本开始 |
| 101 | update | | **S** | | 允许更新已存在的数据行 | 0:不允许更新;1:允许整行更新;2:允许部分列更新。(2.1.7.0 版本开始此参数支持设为 2,在此之前取值只能是 [0, 1]) | 0 | 2.0.8.0 版本之前,不支持此参数。 |
| 102 | cacheLast | | **S** | | 是否在内存中缓存子表的最近数据 | 0:关闭;1:缓存子表最近一行数据;2:缓存子表每一列的最近的非NULL值;3:同时打开缓存最近行和列功能。(2.1.2.0 版本开始此参数支持 0~3 的取值范围,在此之前取值只能是 [0, 1]) | 0 | 2.1.2.0 版本之前、2.0.20.7 版本之前在 taos.cfg 文件中不支持此参数。 |
| 103 | numOfCommitThreads | YES | **S** | | 设置写入线程的最大数量 | | | |
| 104 | maxWildCardsLength | | **C** | bytes | 设定 LIKE 算子的通配符字符串允许的最大长度 | 0-16384 | 100 | 2.1.6.1 版本新增。 |
......@@ -239,7 +239,7 @@ taosd -C
| 10 | fsync | 毫秒 | 当wal设置为2时,执行fsync的周期。设置为0,表示每次写入,立即执行fsync。 | | 3000 |
| 11 | replica | | (可通过 alter database 修改)副本个数 | 1-3 | 1 |
| 12 | precision | | 时间戳精度标识(2.1.2.0 版本之前、2.0.20.7 版本之前在 taos.cfg 文件中不支持此参数。)(从 2.1.5.0 版本开始,新增对纳秒时间精度的支持) | ms 表示毫秒,us 表示微秒,ns 表示纳秒 | ms |
| 13 | update | | 是否允许更新 | 0:不允许;1:允许 | 0 |
| 13 | update | | 是否允许数据更新(从 2.1.7.0 版本开始此参数支持 0~2 的取值范围,在此之前取值只能是 [0, 1];而 2.0.8.0 之前的版本在 SQL 指令中不支持此参数。) | 0:不允许;1:允许更新整行;2:允许部分列更新。 | 0 |
| 14 | cacheLast | | (可通过 alter database 修改)是否在内存中缓存子表的最近数据(从 2.1.2.0 版本开始此参数支持 0~3 的取值范围,在此之前取值只能是 [0, 1];而 2.0.11.0 之前的版本在 SQL 指令中不支持此参数。)(2.1.2.0 版本之前、2.0.20.7 版本之前在 taos.cfg 文件中不支持此参数。) | 0:关闭;1:缓存子表最近一行数据;2:缓存子表每一列的最近的非NULL值;3:同时打开缓存最近行和列功能 | 0 |
对于一个应用场景,可能有多种数据特征的数据并存,最佳的设计是将具有相同数据特征的表放在一个库里,这样一个应用有多个库,而每个库可以配置不同的存储参数,从而保证系统有最优的性能。TDengine允许应用在创建库时指定上述存储参数,如果指定,该参数就将覆盖对应的系统配置参数。举例,有下述SQL:
......
......@@ -99,22 +99,32 @@ gcc -g -O0 -fPIC -shared add_one.c -o add_one.so
在创建 UDF 时,需要区分标量函数和聚合函数。如果创建时声明了错误的函数类别,则可能导致通过 SQL 指令调用函数时出错。
- 创建标量函数:`CREATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) bufsize(B);`
* X:标量函数未来在 SQL 指令中被调用时的函数名,必须与函数实现中 udfNormalFunc 的实际名称一致;
* Y:包含 UDF 函数实现的动态链接库的库文件路径(指的是库文件在当前客户端所在主机上的保存路径,通常是指向一个 .so 文件)
* Z:此函数计算结果的数据类型,使用数字表示,含义与上文中 udfNormalFunc 的 itype 参数一致
- 创建标量函数:`CREATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) bufsize B;`
* ids(X):标量函数未来在 SQL 指令中被调用时的函数名,必须与函数实现中 udfNormalFunc 的实际名称一致;
* ids(Y):包含 UDF 函数实现的动态链接库的库文件路径(指的是库文件在当前客户端所在主机上的保存路径,通常是指向一个 .so 文件),这个路径需要用英文单引号或英文双引号括起来
* typename(Z):此函数计算结果的数据类型,与上文中 udfNormalFunc 的 itype 参数不同,这里不是使用数字表示法,而是直接写类型名称即可
* B:系统使用的中间临时缓冲区大小,单位是字节,最小 0,最大 512,通常可以设置为 128。
- 创建聚合函数:`CREATE AGGREGATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) bufsize(B);`
* X:标量函数未来在 SQL 指令中被调用时的函数名,必须与函数实现中 udfNormalFunc 的实际名称一致;
* Y:包含 UDF 函数实现的动态链接库的库文件路径(指的是库文件在当前客户端所在主机上的保存路径,通常是指向一个 .so 文件);
* Z:此函数计算结果的数据类型,使用数字表示,含义与上文中 udfNormalFunc 的 itype 参数一致;
例如,如下语句可以把 add_one.so 创建为系统中可用的 UDF:
```sql
CREATE FUNCTION add_one AS "/home/taos/udf_example/add_one.so" OUTPUTTYPE INT bufsize 128;
```
- 创建聚合函数:`CREATE AGGREGATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) bufsize B;`
* ids(X):聚合函数未来在 SQL 指令中被调用时的函数名,必须与函数实现中 udfNormalFunc 的实际名称一致;
* ids(Y):包含 UDF 函数实现的动态链接库的库文件路径(指的是库文件在当前客户端所在主机上的保存路径,通常是指向一个 .so 文件),这个路径需要用英文单引号或英文双引号括起来;
* typename(Z):此函数计算结果的数据类型,与上文中 udfNormalFunc 的 itype 参数不同,这里不是使用数字表示法,而是直接写类型名称即可;
* B:系统使用的中间临时缓冲区大小,单位是字节,最小 0,最大 512,通常可以设置为 128。
例如,如下语句可以把 add_one.so 创建为系统中可用的 UDF:
```sql
CREATE FUNCTION abs_max AS "/home/taos/udf_example/abs_max.so" OUTPUTTYPE BIGINT bufsize 128;
```
### 管理 UDF
- 删除指定名称的用户定义函数:`DROP FUNCTION ids(X);`
* X:此参数的含义与 CREATE 指令中的 X 参数一致
* ids(X):此参数的含义与 CREATE 指令中的 ids(X) 参数一致,也即要删除的函数的名字,例如 `DROP FUNCTION add_one;`
- 显示系统中当前可用的所有 UDF:`SHOW FUNCTIONS;`
### 调用 UDF
......
......@@ -70,7 +70,7 @@ TDengine 缺省的时间戳是毫秒精度,但通过在 CREATE DATABASE 时传
1) KEEP是该数据库的数据保留多长天数,缺省是3650天(10年),数据库会自动删除超过时限的数据;<!-- REPLACE_OPEN_TO_ENTERPRISE__KEEP_PARAM_DESCRIPTION -->
2) UPDATE 标志数据库支持更新相同时间戳数据;
2) UPDATE 标志数据库支持更新相同时间戳数据;(从 2.1.7.0 版本开始此参数支持设为 2,表示允许部分列更新,也即更新数据行时未被设置的列会保留原值。)(从 2.0.8.0 版本开始支持此参数。注意此参数不能通过 `ALTER DATABASE` 指令进行修改。)
3) 数据库名最大长度为33;
......@@ -573,16 +573,24 @@ Query OK, 2 row(s) in set (0.003112s)
注意:普通表的通配符 * 中并不包含 _标签列_。
##### 获取标签列的去重取值
#### 获取标签列或普通列的去重取值
从 2.0.15 版本开始,支持在超级表查询标签列时,指定 DISTINCT 关键字,这样将返回指定标签列的所有不重复取值
```mysql
SELECT DISTINCT tag_name FROM stb_name;
从 2.0.15.0 版本开始,支持在超级表查询标签列时,指定 DISTINCT 关键字,这样将返回指定标签列的所有不重复取值。注意,在 2.1.6.0 版本之前,DISTINCT 只支持处理单个标签列,而从 2.1.6.0 版本开始,DISTINCT 可以对多个标签列进行处理,输出这些标签列取值不重复的组合
```sql
SELECT DISTINCT tag_name [, tag_name ...] FROM stb_name;
```
注意:目前 DISTINCT 关键字只支持对超级表的标签列进行去重,而不能用于普通列。
从 2.1.7.0 版本开始,DISTINCT 也支持对数据子表或普通表进行处理,也即支持获取单个普通列的不重复取值,或多个普通列取值的不重复组合。
```sql
SELECT DISTINCT col_name [, col_name ...] FROM tb_name;
```
需要注意的是,DISTINCT 目前不支持对超级表中的普通列进行处理。如果需要进行此类操作,那么需要把超级表放在子查询中,再对子查询的计算结果执行 DISTINCT。
说明:
1. cfg 文件中的配置参数 maxNumOfDistinctRes 将对 DISTINCT 能够输出的数据行数进行限制。其最小值是 100000,最大值是 100000000,默认值是 10000000。如果实际计算结果超出了这个限制,那么会仅输出这个数量范围内的部分。
2. 由于浮点数天然的精度机制原因,在特定情况下,对 FLOAT 和 DOUBLE 列使用 DISTINCT 并不能保证输出值的完全唯一性。
3. 在当前版本下,DISTINCT 不能在嵌套查询的子查询中使用,也不能与聚合函数、GROUP BY、或 JOIN 在同一条语句中混用。
#### 结果集列名
......@@ -1453,8 +1461,6 @@ SELECT function_list FROM tb_name
SELECT function_list FROM stb_name
[WHERE where_condition]
[SESSION(ts_col, tol_val)]
[STATE_WINDOW(col)]
[INTERVAL(interval [, offset]) [SLIDING sliding]]
[FILL({NONE | VALUE | PREV | NULL | LINEAR | NEXT})]
[GROUP BY tags]
......@@ -1465,8 +1471,8 @@ SELECT function_list FROM stb_name
1. 时间窗口:聚合时间段的窗口宽度由关键词 INTERVAL 指定,最短时间间隔 10 毫秒(10a);并且支持偏移 offset(偏移必须小于间隔),也即时间窗口划分与“UTC 时刻 0”相比的偏移量。SLIDING 语句用于指定聚合时间段的前向增量,也即每次窗口向前滑动的时长。当 SLIDING 与 INTERVAL 取值相等的时候,滑动窗口即为翻转窗口。
* 从 2.1.5.0 版本开始,INTERVAL 语句允许的最短时间间隔调整为 1 微秒(1u),当然如果所查询的 DATABASE 的时间精度设置为毫秒级,那么允许的最短时间间隔为 1 毫秒(1a)。
* **注意:**用到 INTERVAL 语句时,除非极特殊的情况,都要求把客户端和服务端的 taos.cfg 配置文件中的 timezone 参数配置为相同的取值,以避免时间处理函数频繁进行跨时区转换而导致的严重性能影响。
2. 状态窗口:使用整数或布尔值来标识产生记录时设备的状态量,产生的记录如果具有相同的状态量取值则归属于同一个状态窗口,数值改变后该窗口关闭。状态量所对应的列作为 STATE_WINDOW 语句的参数来指定。
3. 会话窗口:时间戳所在的列由 SESSION 语句的 ts_col 参数指定,会话窗口根据相邻两条记录的时间戳差值来确定是否属于同一个会话——如果时间戳差异在 tol_val 以内,则认为记录仍属于同一个窗口;如果时间变化超过 tol_val,则自动开启下一个窗口。
2. 状态窗口:使用整数或布尔值来标识产生记录时设备的状态量,产生的记录如果具有相同的状态量取值则归属于同一个状态窗口,数值改变后该窗口关闭。状态量所对应的列作为 STATE_WINDOW 语句的参数来指定。(状态窗口暂不支持对超级表使用)
3. 会话窗口:时间戳所在的列由 SESSION 语句的 ts_col 参数指定,会话窗口根据相邻两条记录的时间戳差值来确定是否属于同一个会话——如果时间戳差异在 tol_val 以内,则认为记录仍属于同一个窗口;如果时间变化超过 tol_val,则自动开启下一个窗口。(会话窗口暂不支持对超级表使用)
- WHERE 语句可以指定查询的起止时间和其他过滤条件。
- FILL 语句指定某一窗口区间数据缺失的情况下的填充模式。填充模式包括以下几种:
1. 不进行填充:NONE(默认填充模式)。
......
......@@ -96,6 +96,8 @@ TDengine 目前尚不支持删除功能,未来根据用户需求可能会支
另需注意,在 UPDATE 设置为 0 时,后发送的相同时间戳的数据会被直接丢弃,但并不会报错,而且仍然会被计入 affected rows (所以不能利用 INSERT 指令的返回信息进行时间戳查重)。这样设计的主要原因是,TDengine 把写入的数据看做一个数据流,无论时间戳是否出现冲突,TDengine 都认为产生数据的原始设备真实地产生了这样的数据。UPDATE 参数只是控制这样的流数据在进行持久化时要怎样处理——UPDATE 为 0 时,表示先写入的数据覆盖后写入的数据;而 UPDATE 为 1 时,表示后写入的数据覆盖先写入的数据。这种覆盖关系如何选择,取决于对数据的后续使用和统计中,希望以先还是后生成的数据为准。
此外,从 2.1.7.0 版本开始,支持将 UPDATE 参数设为 2,表示“支持部分列更新”。也即,当 UPDATE 设为 1 时,如果更新一个数据行,其中某些列没有提供取值,那么这些列会被设为 NULL;而当 UPDATE 设为 2 时,如果更新一个数据行,其中某些列没有提供取值,那么这些列会保持原有数据行中的对应值。
## 10. 我怎么创建超过1024列的表?
使用 2.0 及其以上版本,默认支持 1024 列;2.0 之前的版本,TDengine 最大允许创建 250 列的表。但是如果确实超过限值,建议按照数据特性,逻辑地将这个宽表分解成几个小表。(从 2.1.7.0 版本开始,表的最大列数增加到了 4096 列。)
......
......@@ -15,7 +15,8 @@ One of the modules of TDengine is the time-series database. However, in addition
With TDengine, the total cost of ownership of typical IoT, Internet of Vehicles, and Industrial Internet Big Data platforms can be greatly reduced. However, since it makes full use of the characteristics of IoT time-series data, TDengine cannot be used to process general data from web crawlers, microblogs, WeChat, e-commerce, ERP, CRM, and other sources.
![TDengine Technology Ecosystem](page://images/eco_system.png)
![TDengine Technology Ecosystem](../images/eco_system.png)
<center>Figure 1. TDengine Technology Ecosystem</center>
## <a class="anchor" id="scenes"></a>Overall Scenarios of TDengine
......
......@@ -154,10 +154,8 @@ The design of TDengine is based on the assumption that one single node or softwa
Logical structure diagram of TDengine distributed architecture as following:
![TDengine architecture diagram](page://images/architecture/structure.png)
<center> Picture 1: TDengine architecture diagram </center>
![TDengine architecture diagram](../images/architecture/structure.png)
<center> Figure 1: TDengine architecture diagram </center>
A complete TDengine system runs on one or more physical nodes. Logically, it includes data node (dnode), TDEngine application driver (TAOSC) and application (app). There are one or more data nodes in the system, which form a cluster. The application interacts with the TDengine cluster through TAOSC's API. The following is a brief introduction to each logical unit.
......@@ -199,8 +197,8 @@ A complete TDengine system runs on one or more physical nodes. Logically, it inc
To explain the relationship between vnode, mnode, TAOSC and application and their respective roles, the following is an analysis of a typical data writing process.
![ typical process of TDengine](page://images/architecture/message.png)
<center> Picture 2 typical process of TDengine </center>
![typical process of TDengine](../images/architecture/message.png)
<center> Figure 2: Typical process of TDengine </center>
1. Application initiates a request to insert data through JDBC, ODBC, or other APIs.
2. TAOSC checks if meta data existing for the table in the cache. If so, go straight to Step 4. If not, TAOSC sends a get meta-data request to mnode.
......@@ -268,7 +266,8 @@ If a database has N replicas, thus a virtual node group has N virtual nodes, but
Master Vnode uses a writing process as follows:
Figure 3: TDengine Master writing process
![TDengine Master Writing Process](../images/architecture/write_master.png)
<center> Figure 3: TDengine Master writing process </center>
1. Master vnode receives the application data insertion request, verifies, and moves to next step;
2. If the system configuration parameter `walLevel` is greater than 0, vnode will write the original request packet into database log file WAL. If walLevel is set to 2 and fsync is set to 0, TDengine will make WAL data written immediately to ensure that even system goes down, all data can be recovered from database log file;
......@@ -281,8 +280,8 @@ Figure 3: TDengine Master writing process
For a slave vnode, the write process as follows:
![TDengine Slave Writing Process](page://images/architecture/write_master.png)
<center> Picture 3 TDengine Slave Writing Process </center>
![TDengine Slave Writing Process](../images/architecture/write_slave.png)
<center> Figure 4: TDengine Slave Writing Process </center>
1. Slave vnode receives a data insertion request forwarded by Master vnode;
2. If the system configuration parameter `walLevel` is greater than 0, vnode will write the original request packet into database log file WAL. If walLevel is set to 2 and fsync is set to 0, TDengine will make WAL data written immediately to ensure that even system goes down, all data can be recovered from database log file;
......@@ -355,8 +354,6 @@ When data is written to disk, it is decided whether to compress the data accordi
By default, TDengine saves all data in /var/lib/taos directory, and the data files of each vnode are saved in a different directory under this directory. In order to expand the storage space, minimize the bottleneck of file reading and improve the data throughput rate, TDengine can configure the system parameter “dataDir” to allow multiple mounted hard disks to be used by system at the same time. In addition, TDengine also provides the function of tiered data storage, i.e. storage on different storage media according to the time stamps of data files. For example, the latest data is stored on SSD, the data for more than one week is stored on local hard disk, and the data for more than four weeks is stored on network storage device, thus reducing the storage cost and ensuring efficient data access. The movement of data on different storage media is automatically done by the system and completely transparent to applications. Tiered storage of data is also configured through the system parameter “dataDir”.
dataDir format is as follows:
```
dataDir data_path [tier_level]
......@@ -364,8 +361,6 @@ dataDir data_path [tier_level]
Where data_path is the folder path of mount point and tier_level is the media storage-tier. The higher the media storage-tier, means the older the data file. Multiple hard disks can be mounted at the same storage-tier, and data files on the same storage-tier are distributed on all hard disks within the tier. TDengine supports up to 3 tiers of storage, so tier_level values are 0, 1, and 2. When configuring dataDir, there must be only one mount path without specifying tier_level, which is called special mount disk (path). The mount path defaults to level 0 storage media and contains special file links, which cannot be removed, otherwise it will have a devastating impact on the written data.
Suppose a physical node with six mountable hard disks/mnt/disk1,/mnt/disk2, …,/mnt/disk6, where disk1 and disk2 need to be designated as level 0 storage media, disk3 and disk4 are level 1 storage media, and disk5 and disk6 are level 2 storage media. Disk1 is a special mount disk, you can configure it in/etc/taos/taos.cfg as follows:
```
......@@ -379,7 +374,6 @@ dataDir /mnt/disk6/taos 2
Mounted disks can also be a non-local network disk, as long as the system can access it.
Note: Tiered Storage is only supported in Enterprise Edition
## <a class="anchor" id="query"></a>Data Query
......@@ -418,8 +412,8 @@ For the data collected by device D1001, the number of records per hour is counte
TDengine creates a separate table for each data collection point, but in practical applications, it is often necessary to aggregate data from different data collection points. In order to perform aggregation operations efficiently, TDengine introduces the concept of STable. STable is used to represent a specific type of data collection point. It is a table set containing multiple tables. The schema of each table in the set is the same, but each table has its own static tag. The tags can be multiple and be added, deleted and modified at any time. Applications can aggregate or statistically operate all or a subset of tables under a STABLE by specifying tag filters, thus greatly simplifying the development of applications. The process is shown in the following figure:
![Diagram of multi-table aggregation query](page://images/architecture/multi_tables.png)
<center> Picture 4 Diagram of multi-table aggregation query </center>
![Diagram of multi-table aggregation query](../images/architecture/multi_tables.png)
<center> Figure 5: Diagram of multi-table aggregation query </center>
1. Application sends a query condition to system;
2. TAOSC sends the STable name to Meta Node(management node);
......
......@@ -2,7 +2,7 @@
## <a class="anchor" id="queries"></a> Main Query Features
TDengine uses SQL as the query language. Applications can send SQL statements through C/C++, Java, Go, Python connectors, and users can manually execute SQL Ad-Hoc Query through the Command Line Interface (CLI) tool TAOS Shell provided by TDengine. TDengine supports the following query functions:
TDengine uses SQL as the query language. Applications can send SQL statements through C/C++, Java, Go, C#, Python, Node.js connectors, and users can manually execute SQL Ad-Hoc Query through the Command Line Interface (CLI) tool TAOS Shell provided by TDengine. TDengine supports the following query functions:
- Single-column and multi-column data query
- Multiple filters for tags and numeric values: >, <, =, < >, like, etc
......@@ -96,4 +96,4 @@ Query OK, 5 row(s) in set (0.001521s)
In IoT scenario, it is difficult to synchronize the time stamp of collected data at each point, but many analysis algorithms (such as FFT) need to align the collected data strictly at equal intervals of time. In many systems, it’s required to write their own programs to process, but the down sampling operation of TDengine can be used to solve the problem easily. If there is no collected data in an interval, TDengine also provides interpolation calculation function.
For details of syntax rules, please refer to the [Time-dimension Aggregation section of TAOS SQL](https://www.taosdata.com/en/documentation/taos-sql#aggregation).
\ No newline at end of file
For details of syntax rules, please refer to the [Time-dimension Aggregation section of TAOS SQL](https://www.taosdata.com/en/documentation/taos-sql#aggregation).
......@@ -4,7 +4,7 @@
The taos-jdbcdriver is implemented in two forms: JDBC-JNI and JDBC-RESTful (supported from taos-jdbcdriver-2.0.18). JDBC-JNI is implemented by calling the local methods of libtaos.so (or taos.dll) on the client, while JDBC-RESTful encapsulates the RESTful interface implementation internally.
![tdengine-connector](page://images/tdengine-jdbc-connector.png)
![tdengine-connector](../../images/tdengine-jdbc-connector.png)
The figure above shows the three ways Java applications can access the TDengine:
......@@ -69,18 +69,18 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
The TDengine supports the following data types and Java data types:
| TDengine DataType | Java DataType |
| ----------------- | ------------------ |
| TIMESTAMP | java.sql.Timestamp |
| INT | java.lang.Integer |
| BIGINT | java.lang.Long |
| FLOAT | java.lang.Float |
| DOUBLE | java.lang.Double |
| SMALLINT | java.lang.Short |
| TINYINT | java.lang.Byte |
| BOOL | java.lang.Boolean |
| BINARY | byte[] |
| NCHAR | java.lang.String |
| TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version >= 2.0.24) |
| ----------------- | ------------------ | ------------------ |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long |
| FLOAT | java.lang.Float | java.lang.Float |
| DOUBLE | java.lang.Double | java.lang.Double |
| SMALLINT | java.lang.Short | java.lang.Short |
| TINYINT | java.lang.Byte | java.lang.Byte |
| BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String |
## Install Java connector
......
......@@ -2,7 +2,7 @@
TDengine provides many connectors for development, including C/C++, JAVA, Python, RESTful, Go, Node.JS, etc.
![image-connector](page://images/connector.png)
![image-connector](../images/connector.png)
At present, TDengine connectors support a wide range of platforms, including hardware platforms such as X64/X86/ARM64/ARM32/MIPS/Alpha, and development environments such as Linux/Win64/Win32. The comparison matrix is as follows:
......
......@@ -26,15 +26,15 @@ sudo cp -rf /usr/local/taos/connector/grafanaplugin /var/lib/grafana/plugins/tde
You can log in the Grafana server (username/password:admin/admin) through localhost:3000, and add data sources through `Configuration -> Data Sources` on the left panel, as shown in the following figure:
![img](page://images/connections/add_datasource1.jpg)
![img](../images/connections/add_datasource1.jpg)
Click `Add data source` to enter the Add Data Source page, and enter TDengine in the query box to select Add, as shown in the following figure:
![img](page://images/connections/add_datasource2.jpg)
![img](../images/connections/add_datasource2.jpg)
Enter the data source configuration page and modify the corresponding configuration according to the default prompt:
![img](page://images/connections/add_datasource3.jpg)
![img](../images/connections/add_datasource3.jpg)
- Host: IP address of any server in TDengine cluster and port number of TDengine RESTful interface (6041), default [http://localhost:6041](http://localhost:6041/)
- User: TDengine username.
......@@ -42,13 +42,13 @@ Enter the data source configuration page and modify the corresponding configurat
Click `Save & Test` to test. Success will be prompted as follows:
![img](page://images/connections/add_datasource4.jpg)
![img](../images/connections/add_datasource4.jpg)
#### Create Dashboard
Go back to the home to create Dashboard, and click `Add Query` to enter the panel query page:
![img](page://images/connections/create_dashboard1.jpg)
![img](../images/connections/create_dashboard1.jpg)
As shown in the figure above, select the TDengine data source in Query, and enter the corresponding sql in the query box below to query. Details are as follows:
......@@ -58,7 +58,7 @@ As shown in the figure above, select the TDengine data source in Query, and ente
According to the default prompt, query the average system memory usage at the specified interval of the server where the current TDengine deployed in as follows:
![img](page://images/connections/create_dashboard2.jpg)
![img](../images/connections/create_dashboard2.jpg)
> Please refer to Grafana [documents](https://grafana.com/docs/) for how to use Grafana to create the corresponding monitoring interface and for more about Grafana usage.
......@@ -68,11 +68,11 @@ A `tdengine-grafana.json` importable dashboard is provided under the Grafana plu
Click the `Import` button on the left panel and upload the `tdengine-grafana.json` file:
![img](page://images/connections/import_dashboard1.jpg)
![img](../images/connections/import_dashboard1.jpg)
You can see as follows after Dashboard imported.
![img](page://images/connections/import_dashboard2.jpg)
![img](../images/connections/import_dashboard2.jpg)
## <a class="anchor" id="matlab"></a> MATLAB
......
......@@ -45,6 +45,7 @@ echo "version=${version}"
#docker manifest rm tdengine/tdengine:${version}
if [ "$verType" == "beta" ]; then
docker manifest inspect tdengine/tdengine-beta:latest
docker manifest create -a tdengine/tdengine-beta:latest tdengine/tdengine-amd64-beta:latest tdengine/tdengine-aarch64-beta:latest tdengine/tdengine-aarch32-beta:latest
docker manifest rm tdengine/tdengine-beta:latest
docker manifest create -a tdengine/tdengine-beta:${version} tdengine/tdengine-amd64-beta:${version} tdengine/tdengine-aarch64-beta:${version} tdengine/tdengine-aarch32-beta:${version}
docker manifest create -a tdengine/tdengine-beta:latest tdengine/tdengine-amd64-beta:latest tdengine/tdengine-aarch64-beta:latest tdengine/tdengine-aarch32-beta:latest
......@@ -54,6 +55,7 @@ if [ "$verType" == "beta" ]; then
elif [ "$verType" == "stable" ]; then
docker manifest inspect tdengine/tdengine:latest
docker manifest create -a tdengine/tdengine:latest tdengine/tdengine-amd64:latest tdengine/tdengine-aarch64:latest tdengine/tdengine-aarch32:latest
docker manifest rm tdengine/tdengine:latest
docker manifest create -a tdengine/tdengine:${version} tdengine/tdengine-amd64:${version} tdengine/tdengine-aarch64:${version} tdengine/tdengine-aarch32:${version}
docker manifest create -a tdengine/tdengine:latest tdengine/tdengine-amd64:latest tdengine/tdengine-aarch64:latest tdengine/tdengine-aarch32:latest
......
......@@ -22,7 +22,7 @@ cpuType=x64 # [aarch32 | aarch64 | x64 | x86 | mips64 ...]
osType=Linux # [Linux | Kylin | Alpine | Raspberrypi | Darwin | Windows | Ningsi60 | Ningsi80 |...]
pagMode=full # [full | lite]
soMode=dynamic # [static | dynamic]
dbName=taos # [taos | power | tq]
dbName=taos # [taos | power | tq | pro]
allocator=glibc # [glibc | jemalloc]
verNumber=""
verNumberComp="1.0.0.0"
......@@ -78,7 +78,7 @@ do
echo " -l [full | lite] "
echo " -a [glibc | jemalloc] "
echo " -s [static | dynamic] "
echo " -d [taos | power | tq ] "
echo " -d [taos | power | tq | pro] "
echo " -n [version number] "
echo " -m [compatible version number] "
exit 0
......@@ -253,6 +253,10 @@ if [ "$osType" != "Darwin" ]; then
${csudo} ./makepkg_tq.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName} ${verNumberComp}
${csudo} ./makeclient_tq.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName}
${csudo} ./makearbi_tq.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode}
elif [[ "$dbName" == "pro" ]]; then
${csudo} ./makepkg_pro.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName} ${verNumberComp}
${csudo} ./makeclient_pro.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName}
${csudo} ./makearbi_pro.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode}
else
${csudo} ./makepkg_power.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName} ${verNumberComp}
${csudo} ./makeclient_power.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${pagMode} ${dbName}
......@@ -262,4 +266,3 @@ else
cd ${script_dir}/tools
./makeclient.sh ${compile_dir} ${verNumber} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType} ${dbName}
fi
#!/bin/bash
#
# This file is used to install database on linux systems. The operating system
# is required to use systemd to manage services at boot
set -e
#set -x
# -----------------------Variables definition---------------------
script_dir=$(dirname $(readlink -f "$0"))
bin_link_dir="/usr/bin"
#inc_link_dir="/usr/include"
#install main path
install_main_dir="/usr/local/tarbitrator"
# old bin dir
bin_dir="/usr/local/tarbitrator/bin"
service_config_dir="/etc/systemd/system"
# Color setting
RED='\033[0;31m'
GREEN='\033[1;32m'
GREEN_DARK='\033[0;32m'
GREEN_UNDERLINE='\033[4;32m'
NC='\033[0m'
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
update_flag=0
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
service_mod=0
elif $(which service &> /dev/null); then
service_mod=1
service_config_dir="/etc/init.d"
if $(which chkconfig &> /dev/null); then
initd_mod=1
elif $(which insserv &> /dev/null); then
initd_mod=2
elif $(which update-rc.d &> /dev/null); then
initd_mod=3
else
service_mod=2
fi
else
service_mod=2
fi
# get the operating system type for using the corresponding init file
# ubuntu/debian(deb), centos/fedora(rpm), others: opensuse, redhat, ..., no verification
#osinfo=$(awk -F= '/^NAME/{print $2}' /etc/os-release)
if [[ -e /etc/os-release ]]; then
osinfo=$(cat /etc/os-release | grep "NAME" | cut -d '"' -f2) ||:
else
osinfo=""
fi
#echo "osinfo: ${osinfo}"
os_type=0
if echo $osinfo | grep -qwi "ubuntu" ; then
# echo "This is ubuntu system"
os_type=1
elif echo $osinfo | grep -qwi "debian" ; then
# echo "This is debian system"
os_type=1
elif echo $osinfo | grep -qwi "Kylin" ; then
# echo "This is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "centos" ; then
# echo "This is centos system"
os_type=2
elif echo $osinfo | grep -qwi "fedora" ; then
# echo "This is fedora system"
os_type=2
else
echo " osinfo: ${osinfo}"
echo " This is an officially unverified linux system,"
echo " if there are any problems with the installation and operation, "
echo " please feel free to contact hanatech.com.cn for support."
os_type=1
fi
function kill_tarbitrator() {
pid=$(ps -ef | grep "tarbitrator" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function install_main_path() {
#create install main dir and all sub dir
${csudo} rm -rf ${install_main_dir} || :
${csudo} mkdir -p ${install_main_dir}
${csudo} mkdir -p ${install_main_dir}/bin
#${csudo} mkdir -p ${install_main_dir}/include
${csudo} mkdir -p ${install_main_dir}/init.d
}
function install_bin() {
# Remove links
${csudo} rm -f ${bin_link_dir}/rmtarbitrator || :
${csudo} rm -f ${bin_link_dir}/tarbitrator || :
${csudo} cp -r ${script_dir}/bin/* ${install_main_dir}/bin && ${csudo} chmod 0555 ${install_main_dir}/bin/*
#Make link
[ -x ${install_main_dir}/bin/remove_arbi_prodb.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_arbi_prodb.sh ${bin_link_dir}/rmtarbitrator || :
[ -x ${install_main_dir}/bin/tarbitrator ] && ${csudo} ln -s ${install_main_dir}/bin/tarbitrator ${bin_link_dir}/tarbitrator || :
}
function install_header() {
${csudo} rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taoserror.h || :
${csudo} cp -f ${script_dir}/inc/* ${install_main_dir}/include && ${csudo} chmod 644 ${install_main_dir}/include/*
${csudo} ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h
${csudo} ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h
}
function clean_service_on_sysvinit() {
#restart_config_str="taos:2345:respawn:${service_config_dir}/taosd start"
#${csudo} sed -i "\|${restart_config_str}|d" /etc/inittab || :
if pidof tarbitrator &> /dev/null; then
${csudo} service tarbitratord stop || :
fi
if ((${initd_mod}==1)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} chkconfig --del tarbitratord || :
fi
elif ((${initd_mod}==2)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} insserv -r tarbitratord || :
fi
elif ((${initd_mod}==3)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} update-rc.d -f tarbitratord remove || :
fi
fi
${csudo} rm -f ${service_config_dir}/tarbitratord || :
if $(which init &> /dev/null); then
${csudo} init q || :
fi
}
function install_service_on_sysvinit() {
clean_service_on_sysvinit
sleep 1
# Install prodbs service
if ((${os_type}==1)); then
${csudo} cp -f ${script_dir}/init.d/tarbitratord.deb ${install_main_dir}/init.d/tarbitratord
${csudo} cp ${script_dir}/init.d/tarbitratord.deb ${service_config_dir}/tarbitratord && ${csudo} chmod a+x ${service_config_dir}/tarbitratord
elif ((${os_type}==2)); then
${csudo} cp -f ${script_dir}/init.d/tarbitratord.rpm ${install_main_dir}/init.d/tarbitratord
${csudo} cp ${script_dir}/init.d/tarbitratord.rpm ${service_config_dir}/tarbitratord && ${csudo} chmod a+x ${service_config_dir}/tarbitratord
fi
if ((${initd_mod}==1)); then
${csudo} chkconfig --add tarbitratord || :
${csudo} chkconfig --level 2345 tarbitratord on || :
elif ((${initd_mod}==2)); then
${csudo} insserv tarbitratord || :
${csudo} insserv -d tarbitratord || :
elif ((${initd_mod}==3)); then
${csudo} update-rc.d tarbitratord defaults || :
fi
}
function clean_service_on_systemd() {
tarbitratord_service_config="${service_config_dir}/tarbitratord.service"
if systemctl is-active --quiet tarbitratord; then
echo "tarbitrator is running, stopping it..."
${csudo} systemctl stop tarbitratord &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable tarbitratord &> /dev/null || echo &> /dev/null
${csudo} rm -f ${tarbitratord_service_config}
}
function install_service_on_systemd() {
clean_service_on_systemd
tarbitratord_service_config="${service_config_dir}/tarbitratord.service"
${csudo} bash -c "echo '[Unit]' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'Description=ProDB arbitrator service' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'After=network-online.target' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'Wants=network-online.target' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo >> ${tarbitratord_service_config}"
${csudo} bash -c "echo '[Service]' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'Type=simple' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'ExecStart=/usr/bin/tarbitrator' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'TimeoutStopSec=1000000s' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'LimitNOFILE=infinity' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'LimitNPROC=infinity' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'LimitCORE=infinity' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'TimeoutStartSec=0' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'StandardOutput=null' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'Restart=always' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'StartLimitBurst=3' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'StartLimitInterval=60s' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo >> ${tarbitratord_service_config}"
${csudo} bash -c "echo '[Install]' >> ${tarbitratord_service_config}"
${csudo} bash -c "echo 'WantedBy=multi-user.target' >> ${tarbitratord_service_config}"
${csudo} systemctl enable tarbitratord
}
function install_service() {
if ((${service_mod}==0)); then
install_service_on_systemd
elif ((${service_mod}==1)); then
install_service_on_sysvinit
else
# must manual stop taosd
kill_tarbitrator
fi
}
function update_prodb() {
# Start to update
echo -e "${GREEN}Start to update ProDB's arbitrator ...${NC}"
# Stop the service if running
if pidof tarbitrator &> /dev/null; then
if ((${service_mod}==0)); then
${csudo} systemctl stop tarbitratord || :
elif ((${service_mod}==1)); then
${csudo} service tarbitratord stop || :
else
kill_tarbitrator
fi
sleep 1
fi
install_main_path
#install_header
install_bin
install_service
echo
#echo -e "${GREEN_DARK}To configure ProDB ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start arbitrator ${NC}: ${csudo} systemctl start tarbitratord${NC}"
elif ((${service_mod}==1)); then
echo -e "${GREEN_DARK}To start arbitrator ${NC}: ${csudo} service tarbitratord start${NC}"
else
echo -e "${GREEN_DARK}To start arbitrator ${NC}: ./tarbitrator${NC}"
fi
echo
echo -e "\033[44;32;1mProDB's arbitrator is updated successfully!${NC}"
}
function install_prodb() {
# Start to install
echo -e "${GREEN}Start to install ProDB's arbitrator ...${NC}"
install_main_path
#install_header
install_bin
install_service
echo
#echo -e "${GREEN_DARK}To configure ProDB ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start arbitrator ${NC}: ${csudo} systemctl start tarbitratord${NC}"
elif ((${service_mod}==1)); then
echo -e "${GREEN_DARK}To start arbitrator ${NC}: ${csudo} service tarbitratord start${NC}"
else
echo -e "${GREEN_DARK}To start arbitrator ${NC}: tarbitrator${NC}"
fi
echo -e "\033[44;32;1mProDB's arbitrator is installed successfully!${NC}"
echo
}
## ==============================Main program starts from here============================
# Install server and client
if [ -x ${bin_dir}/tarbitrator ]; then
update_flag=1
update_prodb
else
install_prodb
fi
#!/bin/bash
#
# This file is used to install ProDB client on linux systems. The operating system
# is required to use systemd to manage services at boot
set -e
#set -x
# -----------------------Variables definition---------------------
osType=Linux
pagMode=full
if [ "$osType" != "Darwin" ]; then
script_dir=$(dirname $(readlink -f "$0"))
# Dynamic directory
data_dir="/var/lib/ProDB"
log_dir="/var/log/ProDB"
else
script_dir=`dirname $0`
cd ${script_dir}
script_dir="$(pwd)"
data_dir="/var/lib/ProDB"
log_dir="~/ProDB/log"
fi
log_link_dir="/usr/local/ProDB/log"
cfg_install_dir="/etc/ProDB"
if [ "$osType" != "Darwin" ]; then
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
lib64_link_dir="/usr/lib64"
inc_link_dir="/usr/include"
else
bin_link_dir="/usr/local/bin"
lib_link_dir="/usr/local/lib"
inc_link_dir="/usr/local/include"
fi
#install main path
install_main_dir="/usr/local/ProDB"
# old bin dir
bin_dir="/usr/local/ProDB/bin"
# Color setting
RED='\033[0;31m'
GREEN='\033[1;32m'
GREEN_DARK='\033[0;32m'
GREEN_UNDERLINE='\033[4;32m'
NC='\033[0m'
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
update_flag=0
function kill_client() {
pid=$(ps -ef | grep "prodbc" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function install_main_path() {
#create install main dir and all sub dir
${csudo} rm -rf ${install_main_dir} || :
${csudo} mkdir -p ${install_main_dir}
${csudo} mkdir -p ${install_main_dir}/cfg
${csudo} mkdir -p ${install_main_dir}/bin
${csudo} mkdir -p ${install_main_dir}/connector
${csudo} mkdir -p ${install_main_dir}/driver
${csudo} mkdir -p ${install_main_dir}/examples
${csudo} mkdir -p ${install_main_dir}/include
}
function install_bin() {
# Remove links
${csudo} rm -f ${bin_link_dir}/prodbc || :
if [ "$osType" != "Darwin" ]; then
${csudo} rm -f ${bin_link_dir}/prodemo || :
${csudo} rm -f ${bin_link_dir}/prodump || :
fi
${csudo} rm -f ${bin_link_dir}/rmprodb || :
${csudo} rm -f ${bin_link_dir}/set_core || :
${csudo} cp -r ${script_dir}/bin/* ${install_main_dir}/bin && ${csudo} chmod 0555 ${install_main_dir}/bin/*
#Make link
[ -x ${install_main_dir}/bin/prodbc ] && ${csudo} ln -s ${install_main_dir}/bin/prodbc ${bin_link_dir}/prodbc || :
if [ "$osType" != "Darwin" ]; then
[ -x ${install_main_dir}/bin/prodemo ] && ${csudo} ln -s ${install_main_dir}/bin/prodemo ${bin_link_dir}/prodemo || :
[ -x ${install_main_dir}/bin/prodump ] && ${csudo} ln -s ${install_main_dir}/bin/prodump ${bin_link_dir}/prodump || :
fi
[ -x ${install_main_dir}/bin/remove_client_prodb.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_client_prodb.sh ${bin_link_dir}/rmprodb || :
[ -x ${install_main_dir}/bin/set_core.sh ] && ${csudo} ln -s ${install_main_dir}/bin/set_core.sh ${bin_link_dir}/set_core || :
}
function clean_lib() {
sudo rm -f /usr/lib/libtaos.* || :
sudo rm -rf ${lib_dir} || :
}
function install_lib() {
# Remove links
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -f ${lib64_link_dir}/libtaos.* || :
#${csudo} rm -rf ${v15_java_app_dir} || :
${csudo} cp -rf ${script_dir}/driver/* ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
if [ "$osType" != "Darwin" ]; then
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo} ln -s ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
if [ -d "${lib64_link_dir}" ]; then
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib64_link_dir}/libtaos.so.1 || :
${csudo} ln -s ${lib64_link_dir}/libtaos.so.1 ${lib64_link_dir}/libtaos.so || :
fi
else
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.1.dylib
${csudo} ln -s ${lib_link_dir}/libtaos.1.dylib ${lib_link_dir}/libtaos.dylib
fi
${csudo} ldconfig
}
function install_header() {
${csudo} rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taoserror.h || :
${csudo} cp -f ${script_dir}/inc/* ${install_main_dir}/include && ${csudo} chmod 644 ${install_main_dir}/include/*
${csudo} ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h
${csudo} ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h
}
function install_config() {
#${csudo} rm -f ${install_main_dir}/cfg/taos.cfg || :
if [ ! -f ${cfg_install_dir}/taos.cfg ]; then
${csudo} mkdir -p ${cfg_install_dir}
[ -f ${script_dir}/cfg/taos.cfg ] && ${csudo} cp ${script_dir}/cfg/taos.cfg ${cfg_install_dir}
${csudo} chmod 644 ${cfg_install_dir}/*
fi
${csudo} cp -f ${script_dir}/cfg/taos.cfg ${install_main_dir}/cfg/taos.cfg.org
${csudo} ln -s ${cfg_install_dir}/taos.cfg ${install_main_dir}/cfg
}
function install_log() {
${csudo} rm -rf ${log_dir} || :
if [ "$osType" != "Darwin" ]; then
${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
else
mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
fi
${csudo} ln -s ${log_dir} ${install_main_dir}/log
}
function install_connector() {
${csudo} cp -rf ${script_dir}/connector/* ${install_main_dir}/connector
}
function install_examples() {
if [ -d ${script_dir}/examples ]; then
${csudo} cp -rf ${script_dir}/examples/* ${install_main_dir}/examples
fi
}
function update_prodb() {
# Start to update
if [ ! -e prodb.tar.gz ]; then
echo "File prodb.tar.gz does not exist"
exit 1
fi
tar -zxf prodb.tar.gz
echo -e "${GREEN}Start to update ProDB client...${NC}"
# Stop the client shell if running
if pidof prodbc &> /dev/null; then
kill_client
sleep 1
fi
install_main_path
install_log
install_header
install_lib
if [ "$pagMode" != "lite" ]; then
install_connector
fi
install_examples
install_bin
install_config
echo
echo -e "\033[44;32;1mProDB client is updated successfully!${NC}"
rm -rf $(tar -tf prodb.tar.gz)
}
function install_prodb() {
# Start to install
if [ ! -e prodb.tar.gz ]; then
echo "File prodb.tar.gz does not exist"
exit 1
fi
tar -zxf prodb.tar.gz
echo -e "${GREEN}Start to install ProDB client...${NC}"
install_main_path
install_log
install_header
install_lib
if [ "$pagMode" != "lite" ]; then
install_connector
fi
install_examples
install_bin
install_config
echo
echo -e "\033[44;32;1mProDB client is installed successfully!${NC}"
rm -rf $(tar -tf prodb.tar.gz)
}
## ==============================Main program starts from here============================
# Install or updata client and client
# if server is already install, don't install client
if [ -e ${bin_dir}/prodbs ]; then
echo -e "\033[44;32;1mThere are already installed ProDB server, so don't need install client!${NC}"
exit 0
fi
if [ -x ${bin_dir}/prodbc ]; then
update_flag=1
update_prodb
else
install_prodb
fi
此差异已折叠。
#!/bin/bash
#
# Generate arbitrator's tar.gz setup package for all os system
set -e
#set -x
curr_dir=$(pwd)
compile_dir=$1
version=$2
build_time=$3
cpuType=$4
osType=$5
verMode=$6
verType=$7
pagMode=$8
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/../..)"
# create compressed install file.
build_dir="${compile_dir}/build"
code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/ProDB-enterprise-arbitrator-${version}"
else
install_dir="${release_dir}/ProDB-arbitrator-${version}"
fi
# Directories and files.
bin_files="${build_dir}/bin/tarbitrator ${script_dir}/remove_arbi_pro.sh"
install_files="${script_dir}/install_arbi_pro.sh"
#header_files="${code_dir}/inc/taos.h ${code_dir}/inc/taoserror.h"
init_file_tarbitrator_deb=${script_dir}/../deb/tarbitratord
init_file_tarbitrator_rpm=${script_dir}/../rpm/tarbitratord
# make directories.
mkdir -p ${install_dir} && cp ${install_files} ${install_dir} && chmod a+x ${install_dir}/install_arbi_pro.sh || :
#mkdir -p ${install_dir}/inc && cp ${header_files} ${install_dir}/inc || :
mkdir -p ${install_dir}/bin && cp ${bin_files} ${install_dir}/bin && chmod a+x ${install_dir}/bin/* || :
mkdir -p ${install_dir}/init.d && cp ${init_file_tarbitrator_deb} ${install_dir}/init.d/tarbitratord.deb || :
mkdir -p ${install_dir}/init.d && cp ${init_file_tarbitrator_rpm} ${install_dir}/init.d/tarbitratord.rpm || :
cd ${release_dir}
if [ "$verMode" == "cluster" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
elif [ "$verMode" == "edge" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or edge"
exit 1
fi
if [ "$verType" == "beta" ]; then
pkg_name=${pkg_name}-${verType}
elif [ "$verType" == "stable" ]; then
pkg_name=${pkg_name}
else
echo "unknow verType, nor stabel or beta"
exit 1
fi
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
exitcode=$?
if [ "$exitcode" != "0" ]; then
echo "tar ${pkg_name}.tar.gz error !!!"
exit $exitcode
fi
cd ${curr_dir}
#!/bin/bash
#
# Generate tar.gz package for linux client in all os system
set -e
#set -x
curr_dir=$(pwd)
compile_dir=$1
version=$2
build_time=$3
cpuType=$4
osType=$5
verMode=$6
verType=$7
pagMode=$8
if [ "$osType" != "Darwin" ]; then
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/../..)"
else
script_dir=`dirname $0`
cd ${script_dir}
script_dir="$(pwd)"
top_dir=${script_dir}/../..
fi
# create compressed install file.
build_dir="${compile_dir}/build"
code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/ProDB-enterprise-client-${version}"
else
install_dir="${release_dir}/ProDB-client-${version}"
fi
# Directories and files.
if [ "$osType" != "Darwin" ]; then
lib_files="${build_dir}/lib/libtaos.so.${version}"
else
bin_files="${build_dir}/bin/taos ${script_dir}/remove_client_pro.sh"
lib_files="${build_dir}/lib/libtaos.${version}.dylib"
fi
header_files="${code_dir}/inc/taos.h ${code_dir}/inc/taoserror.h"
if [ "$verMode" == "cluster" ]; then
cfg_dir="${top_dir}/../enterprise/packaging/cfg"
else
cfg_dir="${top_dir}/packaging/cfg"
fi
install_files="${script_dir}/install_client_pro.sh"
# make directories.
mkdir -p ${install_dir}
mkdir -p ${install_dir}/inc && cp ${header_files} ${install_dir}/inc
mkdir -p ${install_dir}/cfg && cp ${cfg_dir}/taos.cfg ${install_dir}/cfg/taos.cfg
sed -i '/dataDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i '/logDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i "s/TDengine/ProDB/g" ${install_dir}/cfg/taos.cfg
mkdir -p ${install_dir}/bin
if [ "$osType" != "Darwin" ]; then
if [ "$pagMode" == "lite" ]; then
strip ${build_dir}/bin/taos
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
else
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
cp ${build_dir}/bin/taosdemo ${install_dir}/bin/prodemo
cp ${build_dir}/bin/taosdump ${install_dir}/bin/prodump
cp ${script_dir}/set_core.sh ${install_dir}/bin
cp ${script_dir}/get_client.sh ${install_dir}/bin
cp ${script_dir}/taosd-dump-cfg.gdb ${install_dir}/bin
fi
else
cp ${bin_files} ${install_dir}/bin
fi
chmod a+x ${install_dir}/bin/* || :
if [ -f ${build_dir}/bin/jemalloc-config ]; then
mkdir -p ${install_dir}/jemalloc/{bin,lib,lib/pkgconfig,include/jemalloc,share/doc/jemalloc,share/man/man3}
cp ${build_dir}/bin/jemalloc-config ${install_dir}/jemalloc/bin
if [ -f ${build_dir}/bin/jemalloc.sh ]; then
cp ${build_dir}/bin/jemalloc.sh ${install_dir}/jemalloc/bin
fi
if [ -f ${build_dir}/bin/jeprof ]; then
cp ${build_dir}/bin/jeprof ${install_dir}/jemalloc/bin
fi
if [ -f ${build_dir}/include/jemalloc/jemalloc.h ]; then
cp ${build_dir}/include/jemalloc/jemalloc.h ${install_dir}/jemalloc/include/jemalloc
fi
if [ -f ${build_dir}/lib/libjemalloc.so.2 ]; then
cp ${build_dir}/lib/libjemalloc.so.2 ${install_dir}/jemalloc/lib
ln -sf libjemalloc.so.2 ${install_dir}/jemalloc/lib/libjemalloc.so
fi
if [ -f ${build_dir}/lib/libjemalloc.a ]; then
cp ${build_dir}/lib/libjemalloc.a ${install_dir}/jemalloc/lib
fi
if [ -f ${build_dir}/lib/libjemalloc_pic.a ]; then
cp ${build_dir}/lib/libjemalloc_pic.a ${install_dir}/jemalloc/lib
fi
if [ -f ${build_dir}/lib/pkgconfig/jemalloc.pc ]; then
cp ${build_dir}/lib/pkgconfig/jemalloc.pc ${install_dir}/jemalloc/lib/pkgconfig
fi
if [ -f ${build_dir}/share/doc/jemalloc/jemalloc.html ]; then
cp ${build_dir}/share/doc/jemalloc/jemalloc.html ${install_dir}/jemalloc/share/doc/jemalloc
fi
if [ -f ${build_dir}/share/man/man3/jemalloc.3 ]; then
cp ${build_dir}/share/man/man3/jemalloc.3 ${install_dir}/jemalloc/share/man/man3
fi
fi
cd ${install_dir}
if [ "$osType" != "Darwin" ]; then
tar -zcv -f prodb.tar.gz * --remove-files || :
else
tar -zcv -f prodb.tar.gz * || :
mv prodb.tar.gz ..
rm -rf ./*
mv ../prodb.tar.gz .
fi
cd ${curr_dir}
cp ${install_files} ${install_dir}
if [ "$osType" == "Darwin" ]; then
sed 's/osType=Linux/osType=Darwin/g' ${install_dir}/install_client_pro.sh >> install_client_prodb_temp.sh
mv install_client_prodb_temp.sh ${install_dir}/install_client_pro.sh
fi
if [ "$pagMode" == "lite" ]; then
sed 's/pagMode=full/pagMode=lite/g' ${install_dir}/install_client_pro.sh >> install_client_prodb_temp.sh
mv install_client_prodb_temp.sh ${install_dir}/install_client_pro.sh
fi
chmod a+x ${install_dir}/install_client_pro.sh
# Copy example code
mkdir -p ${install_dir}/examples
examples_dir="${top_dir}/tests/examples"
cp -r ${examples_dir}/c ${install_dir}/examples
sed -i '/passwd/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp -r ${examples_dir}/JDBC ${install_dir}/examples
cp -r ${examples_dir}/matlab ${install_dir}/examples
mv ${install_dir}/examples/matlab/TDengineDemo.m ${install_dir}/examples/matlab/ProDBDemo.m
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/matlab/ProDBDemo.m
cp -r ${examples_dir}/python ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/python/read_example.py
cp -r ${examples_dir}/R ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/R/command.txt
cp -r ${examples_dir}/go ${install_dir}/examples
mv ${install_dir}/examples/go/taosdemo.go ${install_dir}/examples/go/prodemo.go
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/go/prodemo.go
fi
# Copy driver
mkdir -p ${install_dir}/driver
cp ${lib_files} ${install_dir}/driver
# Copy connector
connector_dir="${code_dir}/connector"
mkdir -p ${install_dir}/connector
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
if [ "$osType" != "Darwin" ]; then
cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
fi
if [ -d "${connector_dir}/grafanaplugin/dist" ]; then
cp -r ${connector_dir}/grafanaplugin/dist ${install_dir}/connector/grafanaplugin
else
echo "WARNING: grafanaplugin bunlded dir not found, please check if want to use it!"
fi
if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
cp -r ${connector_dir}/go ${install_dir}/connector
else
echo "WARNING: go connector not found, please check if want to use it!"
fi
cp -r ${connector_dir}/python ${install_dir}/connector
mv ${install_dir}/connector/python/taos ${install_dir}/connector/python/prodb
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/cinterface.py
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/subscription.py
sed -i '/self._password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/connection.py
fi
cd ${release_dir}
if [ "$verMode" == "cluster" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
elif [ "$verMode" == "edge" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or edge"
exit 1
fi
if [ "$pagMode" == "lite" ]; then
pkg_name=${pkg_name}-Lite
fi
if [ "$verType" == "beta" ]; then
pkg_name=${pkg_name}-${verType}
elif [ "$verType" == "stable" ]; then
pkg_name=${pkg_name}
else
echo "unknow verType, nor stable or beta"
exit 1
fi
if [ "$osType" != "Darwin" ]; then
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
else
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) || :
mv "$(basename ${pkg_name}).tar.gz" ..
rm -rf ./*
mv ../"$(basename ${pkg_name}).tar.gz" .
fi
cd ${curr_dir}
#!/bin/bash
#
# Generate tar.gz package for all os system
set -e
#set -x
curr_dir=$(pwd)
compile_dir=$1
version=$2
build_time=$3
cpuType=$4
osType=$5
verMode=$6
verType=$7
pagMode=$8
versionComp=$9
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/../..)"
# create compressed install file.
build_dir="${compile_dir}/build"
code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/ProDB-enterprise-server-${version}"
else
install_dir="${release_dir}/ProDB-server-${version}"
fi
lib_files="${build_dir}/lib/libtaos.so.${version}"
header_files="${code_dir}/inc/taos.h ${code_dir}/inc/taoserror.h"
if [ "$verMode" == "cluster" ]; then
cfg_dir="${top_dir}/../enterprise/packaging/cfg"
else
cfg_dir="${top_dir}/packaging/cfg"
fi
install_files="${script_dir}/install_pro.sh"
nginx_dir="${code_dir}/../../enterprise/src/plugins/web"
# make directories.
mkdir -p ${install_dir}
mkdir -p ${install_dir}/inc && cp ${header_files} ${install_dir}/inc
mkdir -p ${install_dir}/cfg && cp ${cfg_dir}/taos.cfg ${install_dir}/cfg/taos.cfg
#mkdir -p ${install_dir}/bin && cp ${bin_files} ${install_dir}/bin && chmod a+x ${install_dir}/bin/* || :
mkdir -p ${install_dir}/bin
if [ "$pagMode" == "lite" ]; then
strip ${build_dir}/bin/taosd
strip ${build_dir}/bin/taos
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${build_dir}/bin/taosd ${install_dir}/bin/prodbs
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
else
cp ${build_dir}/bin/taos ${install_dir}/bin/prodbc
cp ${build_dir}/bin/taosd ${install_dir}/bin/prodbs
cp ${script_dir}/remove_pro.sh ${install_dir}/bin
cp ${build_dir}/bin/taosdemo ${install_dir}/bin/prodemo
cp ${build_dir}/bin/taosdump ${install_dir}/bin/prodump
cp ${build_dir}/bin/tarbitrator ${install_dir}/bin
cp ${script_dir}/set_core.sh ${install_dir}/bin
cp ${script_dir}/get_client.sh ${install_dir}/bin
cp ${script_dir}/startPre.sh ${install_dir}/bin
cp ${script_dir}/taosd-dump-cfg.gdb ${install_dir}/bin
fi
chmod a+x ${install_dir}/bin/* || :
if [ "$verMode" == "cluster" ]; then
sed 's/verMode=edge/verMode=cluster/g' ${install_dir}/bin/remove_pro.sh >> remove_prodb_temp.sh
mv remove_prodb_temp.sh ${install_dir}/bin/remove_pro.sh
mkdir -p ${install_dir}/nginxd && cp -r ${nginx_dir}/* ${install_dir}/nginxd
cp ${nginx_dir}/png/taos.png ${install_dir}/nginxd/admin/images/taos.png
rm -rf ${install_dir}/nginxd/png
sed -i "s/TDengine/ProDB/g" ${install_dir}/nginxd/admin/*.html
sed -i "s/TDengine/ProDB/g" ${install_dir}/nginxd/admin/js/*.js
sed -i '/dataDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i '/logDir/ {s/taos/ProDB/g}' ${install_dir}/cfg/taos.cfg
sed -i "s/TDengine/ProDB/g" ${install_dir}/cfg/taos.cfg
if [ "$cpuType" == "aarch64" ]; then
cp -f ${install_dir}/nginxd/sbin/arm/64bit/nginx ${install_dir}/nginxd/sbin/
elif [ "$cpuType" == "aarch32" ]; then
cp -f ${install_dir}/nginxd/sbin/arm/32bit/nginx ${install_dir}/nginxd/sbin/
fi
rm -rf ${install_dir}/nginxd/sbin/arm
fi
cd ${install_dir}
tar -zcv -f prodb.tar.gz * --remove-files || :
exitcode=$?
if [ "$exitcode" != "0" ]; then
echo "tar prodb.tar.gz error !!!"
exit $exitcode
fi
cd ${curr_dir}
cp ${install_files} ${install_dir}
if [ "$verMode" == "cluster" ]; then
sed 's/verMode=edge/verMode=cluster/g' ${install_dir}/install_pro.sh >> install_prodb_temp.sh
mv install_prodb_temp.sh ${install_dir}/install_pro.sh
fi
if [ "$pagMode" == "lite" ]; then
sed 's/pagMode=full/pagMode=lite/g' ${install_dir}/install.sh >> install_prodb_temp.sh
mv install_prodb_temp.sh ${install_dir}/install_pro.sh
fi
chmod a+x ${install_dir}/install_pro.sh
# Copy example code
mkdir -p ${install_dir}/examples
examples_dir="${top_dir}/tests/examples"
cp -r ${examples_dir}/c ${install_dir}/examples
sed -i '/passwd/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/c/*.c
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp -r ${examples_dir}/JDBC ${install_dir}/examples
cp -r ${examples_dir}/matlab ${install_dir}/examples
mv ${install_dir}/examples/matlab/TDengineDemo.m ${install_dir}/examples/matlab/ProDBDemo.m
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/matlab/ProDBDemo.m
cp -r ${examples_dir}/python ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/python/read_example.py
cp -r ${examples_dir}/R ${install_dir}/examples
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/examples/R/command.txt
cp -r ${examples_dir}/go ${install_dir}/examples
mv ${install_dir}/examples/go/taosdemo.go ${install_dir}/examples/go/prodemo.go
sed -i '/root/ {s/taosdata/prodb/g}' ${install_dir}/examples/go/prodemo.go
fi
# Copy driver
mkdir -p ${install_dir}/driver && cp ${lib_files} ${install_dir}/driver && echo "${versionComp}" > ${install_dir}/driver/vercomp.txt
# Copy connector
connector_dir="${code_dir}/connector"
mkdir -p ${install_dir}/connector
if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then
cp ${build_dir}/lib/*.jar ${install_dir}/connector ||:
if [ -d "${connector_dir}/grafanaplugin/dist" ]; then
cp -r ${connector_dir}/grafanaplugin/dist ${install_dir}/connector/grafanaplugin
else
echo "WARNING: grafanaplugin bundled dir not found, please check if want to use it!"
fi
if find ${connector_dir}/go -mindepth 1 -maxdepth 1 | read; then
cp -r ${connector_dir}/go ${install_dir}/connector
else
echo "WARNING: go connector not found, please check if want to use it!"
fi
cp -r ${connector_dir}/python ${install_dir}/connector/
mv ${install_dir}/connector/python/taos ${install_dir}/connector/python/prodb
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/cinterface.py
sed -i '/password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/subscription.py
sed -i '/self._password/ {s/taosdata/prodb/g}' ${install_dir}/connector/python/prodb/connection.py
fi
cd ${release_dir}
if [ "$verMode" == "cluster" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
elif [ "$verMode" == "edge" ]; then
pkg_name=${install_dir}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or edge"
exit 1
fi
if [ "$pagMode" == "lite" ]; then
pkg_name=${pkg_name}-Lite
fi
if [ "$verType" == "beta" ]; then
pkg_name=${pkg_name}-${verType}
elif [ "$verType" == "stable" ]; then
pkg_name=${pkg_name}
else
echo "unknow verType, nor stabel or beta"
exit 1
fi
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
exitcode=$?
if [ "$exitcode" != "0" ]; then
echo "tar ${pkg_name}.tar.gz error !!!"
exit $exitcode
fi
cd ${curr_dir}
#!/bin/bash
#
# Script to stop the service and uninstall ProDB's arbitrator
set -e
#set -x
verMode=edge
RED='\033[0;31m'
GREEN='\033[1;32m'
NC='\033[0m'
#install main path
install_main_dir="/usr/local/tarbitrator"
bin_link_dir="/usr/bin"
service_config_dir="/etc/systemd/system"
tarbitrator_service_name="tarbitratord"
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
service_mod=0
elif $(which service &> /dev/null); then
service_mod=1
service_config_dir="/etc/init.d"
if $(which chkconfig &> /dev/null); then
initd_mod=1
elif $(which insserv &> /dev/null); then
initd_mod=2
elif $(which update-rc.d &> /dev/null); then
initd_mod=3
else
service_mod=2
fi
else
service_mod=2
fi
function kill_tarbitrator() {
pid=$(ps -ef | grep "tarbitrator" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function clean_bin() {
# Remove link
${csudo} rm -f ${bin_link_dir}/tarbitrator || :
}
function clean_header() {
# Remove link
${csudo} rm -f ${inc_link_dir}/taos.h || :
${csudo} rm -f ${inc_link_dir}/taoserror.h || :
}
function clean_log() {
# Remove link
${csudo} rm -rf /arbitrator.log || :
}
function clean_service_on_systemd() {
tarbitratord_service_config="${service_config_dir}/${tarbitrator_service_name}.service"
if systemctl is-active --quiet ${tarbitrator_service_name}; then
echo "ProDB tarbitrator is running, stopping it..."
${csudo} systemctl stop ${tarbitrator_service_name} &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable ${tarbitrator_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${tarbitratord_service_config}
}
function clean_service_on_sysvinit() {
if pidof tarbitrator &> /dev/null; then
echo "ProDB's tarbitrator is running, stopping it..."
${csudo} service tarbitratord stop || :
fi
if ((${initd_mod}==1)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} chkconfig --del tarbitratord || :
fi
elif ((${initd_mod}==2)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} insserv -r tarbitratord || :
fi
elif ((${initd_mod}==3)); then
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} update-rc.d -f tarbitratord remove || :
fi
fi
${csudo} rm -f ${service_config_dir}/tarbitratord || :
if $(which init &> /dev/null); then
${csudo} init q || :
fi
}
function clean_service() {
if ((${service_mod}==0)); then
clean_service_on_systemd
elif ((${service_mod}==1)); then
clean_service_on_sysvinit
else
# must manual stop
kill_tarbitrator
fi
}
# Stop service and disable booting start.
clean_service
# Remove binary file and links
clean_bin
# Remove header file.
##clean_header
# Remove log file
clean_log
${csudo} rm -rf ${install_main_dir}
echo -e "${GREEN}ProDB's arbitrator is removed successfully!${NC}"
echo
#!/bin/bash
#
# Script to stop the client and uninstall database, but retain the config and log files.
set -e
# set -x
RED='\033[0;31m'
GREEN='\033[1;32m'
NC='\033[0m'
#install main path
install_main_dir="/usr/local/ProDB"
log_link_dir="/usr/local/ProDB/log"
cfg_link_dir="/usr/local/ProDB/cfg"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
lib64_link_dir="/usr/lib64"
inc_link_dir="/usr/include"
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
function kill_client() {
if [ -n "$(pidof prodbc)" ]; then
${csudo} kill -9 $pid || :
fi
}
function clean_bin() {
# Remove link
${csudo} rm -f ${bin_link_dir}/prodbc || :
${csudo} rm -f ${bin_link_dir}/prodemo || :
${csudo} rm -f ${bin_link_dir}/prodump || :
${csudo} rm -f ${bin_link_dir}/rmprodb || :
${csudo} rm -f ${bin_link_dir}/set_core || :
}
function clean_lib() {
# Remove link
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -f ${lib64_link_dir}/libtaos.* || :
}
function clean_header() {
# Remove link
${csudo} rm -f ${inc_link_dir}/taos.h || :
${csudo} rm -f ${inc_link_dir}/taoserror.h || :
}
function clean_config() {
# Remove link
${csudo} rm -f ${cfg_link_dir}/* || :
}
function clean_log() {
# Remove link
${csudo} rm -rf ${log_link_dir} || :
}
# Stop client.
kill_client
# Remove binary file and links
clean_bin
# Remove header file.
clean_header
# Remove lib file
clean_lib
# Remove link log directory
clean_log
# Remove link configuration file
clean_config
${csudo} rm -rf ${install_main_dir}
echo -e "${GREEN}ProDB client is removed successfully!${NC}"
echo
#!/bin/bash
#
# Script to stop the service and uninstall ProDB, but retain the config, data and log files.
set -e
#set -x
verMode=edge
RED='\033[0;31m'
GREEN='\033[1;32m'
NC='\033[0m'
#install main path
install_main_dir="/usr/local/ProDB"
data_link_dir="/usr/local/ProDB/data"
log_link_dir="/usr/local/ProDB/log"
cfg_link_dir="/usr/local/ProDB/cfg"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
lib64_link_dir="/usr/lib64"
inc_link_dir="/usr/include"
install_nginxd_dir="/usr/local/nginxd"
service_config_dir="/etc/systemd/system"
prodb_service_name="prodbs"
tarbitrator_service_name="tarbitratord"
nginx_service_name="nginxd"
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
fi
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
service_mod=0
elif $(which service &> /dev/null); then
service_mod=1
service_config_dir="/etc/init.d"
if $(which chkconfig &> /dev/null); then
initd_mod=1
elif $(which insserv &> /dev/null); then
initd_mod=2
elif $(which update-rc.d &> /dev/null); then
initd_mod=3
else
service_mod=2
fi
else
service_mod=2
fi
function kill_prodbs() {
pid=$(ps -ef | grep "prodbs" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function kill_tarbitrator() {
pid=$(ps -ef | grep "tarbitrator" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function clean_bin() {
# Remove link
${csudo} rm -f ${bin_link_dir}/prodbc || :
${csudo} rm -f ${bin_link_dir}/prodbs || :
${csudo} rm -f ${bin_link_dir}/prodemo || :
${csudo} rm -f ${bin_link_dir}/prodump || :
${csudo} rm -f ${bin_link_dir}/rmprodb || :
${csudo} rm -f ${bin_link_dir}/tarbitrator || :
${csudo} rm -f ${bin_link_dir}/set_core || :
}
function clean_lib() {
# Remove link
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -f ${lib64_link_dir}/libtaos.* || :
}
function clean_header() {
# Remove link
${csudo} rm -f ${inc_link_dir}/taos.h || :
${csudo} rm -f ${inc_link_dir}/taoserror.h || :
}
function clean_config() {
# Remove link
${csudo} rm -f ${cfg_link_dir}/* || :
}
function clean_log() {
# Remove link
${csudo} rm -rf ${log_link_dir} || :
}
function clean_service_on_systemd() {
prodb_service_config="${service_config_dir}/${prodb_service_name}.service"
if systemctl is-active --quiet ${prodb_service_name}; then
echo "ProDB prodbs is running, stopping it..."
${csudo} systemctl stop ${prodb_service_name} &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable ${prodb_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${prodb_service_config}
tarbitratord_service_config="${service_config_dir}/${tarbitrator_service_name}.service"
if systemctl is-active --quiet ${tarbitrator_service_name}; then
echo "ProDB tarbitrator is running, stopping it..."
${csudo} systemctl stop ${tarbitrator_service_name} &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable ${tarbitrator_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${tarbitratord_service_config}
if [ "$verMode" == "cluster" ]; then
nginx_service_config="${service_config_dir}/${nginx_service_name}.service"
if [ -d ${bin_dir}/web ]; then
if systemctl is-active --quiet ${nginx_service_name}; then
echo "Nginx for ProDB is running, stopping it..."
${csudo} systemctl stop ${nginx_service_name} &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable ${nginx_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${nginx_service_config}
fi
fi
}
function clean_service_on_sysvinit() {
if pidof prodbs &> /dev/null; then
echo "ProDB prodbs is running, stopping it..."
${csudo} service prodbs stop || :
fi
if pidof tarbitrator &> /dev/null; then
echo "ProDB tarbitrator is running, stopping it..."
${csudo} service tarbitratord stop || :
fi
if ((${initd_mod}==1)); then
if [ -e ${service_config_dir}/prodbs ]; then
${csudo} chkconfig --del prodbs || :
fi
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} chkconfig --del tarbitratord || :
fi
elif ((${initd_mod}==2)); then
if [ -e ${service_config_dir}/prodbs ]; then
${csudo} insserv -r prodbs || :
fi
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} insserv -r tarbitratord || :
fi
elif ((${initd_mod}==3)); then
if [ -e ${service_config_dir}/prodbs ]; then
${csudo} update-rc.d -f prodbs remove || :
fi
if [ -e ${service_config_dir}/tarbitratord ]; then
${csudo} update-rc.d -f tarbitratord remove || :
fi
fi
${csudo} rm -f ${service_config_dir}/prodbs || :
${csudo} rm -f ${service_config_dir}/tarbitratord || :
if $(which init &> /dev/null); then
${csudo} init q || :
fi
}
function clean_service() {
if ((${service_mod}==0)); then
clean_service_on_systemd
elif ((${service_mod}==1)); then
clean_service_on_sysvinit
else
# must manual stop taosd
kill_prodbs
kill_tarbitrator
fi
}
# Stop service and disable booting start.
clean_service
# Remove binary file and links
clean_bin
# Remove header file.
clean_header
# Remove lib file
clean_lib
# Remove link log directory
clean_log
# Remove link configuration file
clean_config
# Remove data link directory
${csudo} rm -rf ${data_link_dir} || :
${csudo} rm -rf ${install_main_dir}
${csudo} rm -rf ${install_nginxd_dir}
if [[ -e /etc/os-release ]]; then
osinfo=$(awk -F= '/^NAME/{print $2}' /etc/os-release)
else
osinfo=""
fi
echo -e "${GREEN}ProDB is removed successfully!${NC}"
echo
......@@ -10,6 +10,7 @@ AUX_SOURCE_DIRECTORY(src SRC)
IF (TD_LINUX)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/linux)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/cJson/inc)
# set the static lib name
ADD_LIBRARY(taos_static STATIC ${SRC})
......@@ -21,7 +22,7 @@ IF (TD_LINUX)
ADD_LIBRARY(taos SHARED ${SRC})
TARGET_LINK_LIBRARIES(taos common query trpc tutil pthread m rt cJson)
IF (TD_LINUX_64)
TARGET_LINK_LIBRARIES(taos lua)
TARGET_LINK_LIBRARIES(taos lua cJson)
ENDIF ()
SET_TARGET_PROPERTIES(taos PROPERTIES CLEAN_DIRECT_OUTPUT 1)
......@@ -36,6 +37,7 @@ IF (TD_LINUX)
ELSEIF (TD_DARWIN)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/linux)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/cJson/inc)
# set the static lib name
ADD_LIBRARY(taos_static STATIC ${SRC})
......@@ -59,6 +61,7 @@ ELSEIF (TD_DARWIN)
ELSEIF (TD_WINDOWS)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/windows)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/windows/win32)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/cJson/inc)
CONFIGURE_FILE("${TD_COMMUNITY_DIR}/src/client/src/taos.rc.in" "${TD_COMMUNITY_DIR}/src/client/src/taos.rc")
......@@ -75,6 +78,7 @@ ELSEIF (TD_WINDOWS)
ELSEIF (TD_DARWIN)
SET(CMAKE_MACOSX_RPATH 1)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/linux)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/cJson/inc)
ADD_LIBRARY(taos_static STATIC ${SRC})
TARGET_LINK_LIBRARIES(taos_static query trpc tutil pthread m lua cJson)
......
......@@ -54,6 +54,9 @@ typedef struct {
int tscSmlInsert(TAOS* taos, TAOS_SML_DATA_POINT* points, int numPoint, SSmlLinesInfo* info);
bool checkDuplicateKey(char *key, SHashObj *pHash, SSmlLinesInfo* info);
bool isValidInteger(char *str);
bool isValidFloat(char *str);
int32_t isValidChildTableName(const char *pTbName, int16_t len);
bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
......
......@@ -41,6 +41,14 @@ JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_initImp
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setOptions
(JNIEnv *, jclass, jint, jstring);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: setConfigImp
* Signature: (Ljava/lang/String;)Lcom/taosdata/jdbc/TSDBException;
*/
JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setConfigImp
(JNIEnv *, jclass, jstring);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: getTsCharset
......
......@@ -200,6 +200,64 @@ JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_initImp(JNIEnv *e
jniDebug("jni initialized successfully, config directory: %s", configDir);
}
JNIEXPORT jobject createTSDBException(JNIEnv *env, int code, char *msg) {
// find class
jclass exception_clazz = (*env)->FindClass(env, "com/taosdata/jdbc/TSDBException");
// find methods
jmethodID init_method = (*env)->GetMethodID(env, exception_clazz, "<init>", "()V");
jmethodID setCode_method = (*env)->GetMethodID(env, exception_clazz, "setCode", "(I)V");
jmethodID setMessage_method = (*env)->GetMethodID(env, exception_clazz, "setMessage", "(Ljava/lang/String;)V");
// new exception
jobject exception_obj = (*env)->NewObject(env, exception_clazz, init_method);
// set code
(*env)->CallVoidMethod(env, exception_obj, setCode_method, code);
// set message
jstring message = (*env)->NewStringUTF(env, msg);
(*env)->CallVoidMethod(env, exception_obj, setMessage_method, message);
return exception_obj;
}
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
* Method: setConfigImp
* Signature: (Ljava/lang/String;)Lcom/taosdata/jdbc/TSDBException;
*/
JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setConfigImp(JNIEnv *env, jclass jobj,
jstring config) {
/*
if (config == NULL) {
jniDebug("config value is null");
return -1;
}
const char *cfg = (*env)->GetStringUTFChars(env, config, NULL);
if (!cfg) {
return -1;
}
return 0;
*/
if (config == NULL) {
char *msg = "config value is null";
jniDebug("config value is null");
return createTSDBException(env, -1, msg);
}
const char *cfg = (*env)->GetStringUTFChars(env, config, NULL);
if (!cfg) {
char *msg = "config value is null";
jniDebug("config value is null");
return createTSDBException(env, -1, msg);
}
setConfRet result = taos_set_config(cfg);
int code = result.retCode;
char * msg = result.retMsg;
return createTSDBException(env, code, msg);
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_setOptions(JNIEnv *env, jobject jobj, jint optionIndex,
jstring optionValue) {
if (optionValue == NULL) {
......
......@@ -2,6 +2,7 @@ EXPORTS
taos_init
taos_cleanup
taos_options
taos_set_config
taos_connect
taos_connect_auth
taos_close
......
......@@ -1137,7 +1137,7 @@ static void escapeSpecialCharacter(uint8_t field, const char **pos) {
*pos = cur;
}
static bool isValidInteger(char *str) {
bool isValidInteger(char *str) {
char *c = str;
if (*c != '+' && *c != '-' && !isdigit(*c)) {
return false;
......@@ -1152,7 +1152,7 @@ static bool isValidInteger(char *str) {
return true;
}
static bool isValidFloat(char *str) {
bool isValidFloat(char *str) {
char *c = str;
uint8_t has_dot, has_exp, has_sign;
has_dot = 0;
......@@ -1212,7 +1212,7 @@ static bool isTinyInt(char *pVal, uint16_t len) {
if (len <= 2) {
return false;
}
if (!strcmp(&pVal[len - 2], "i8")) {
if (!strcasecmp(&pVal[len - 2], "i8")) {
//printf("Type is int8(%s)\n", pVal);
return true;
}
......@@ -1226,7 +1226,7 @@ static bool isTinyUint(char *pVal, uint16_t len) {
if (pVal[0] == '-') {
return false;
}
if (!strcmp(&pVal[len - 2], "u8")) {
if (!strcasecmp(&pVal[len - 2], "u8")) {
//printf("Type is uint8(%s)\n", pVal);
return true;
}
......@@ -1237,7 +1237,7 @@ static bool isSmallInt(char *pVal, uint16_t len) {
if (len <= 3) {
return false;
}
if (!strcmp(&pVal[len - 3], "i16")) {
if (!strcasecmp(&pVal[len - 3], "i16")) {
//printf("Type is int16(%s)\n", pVal);
return true;
}
......@@ -1251,7 +1251,7 @@ static bool isSmallUint(char *pVal, uint16_t len) {
if (pVal[0] == '-') {
return false;
}
if (strcmp(&pVal[len - 3], "u16") == 0) {
if (strcasecmp(&pVal[len - 3], "u16") == 0) {
//printf("Type is uint16(%s)\n", pVal);
return true;
}
......@@ -1262,7 +1262,7 @@ static bool isInt(char *pVal, uint16_t len) {
if (len <= 3) {
return false;
}
if (strcmp(&pVal[len - 3], "i32") == 0) {
if (strcasecmp(&pVal[len - 3], "i32") == 0) {
//printf("Type is int32(%s)\n", pVal);
return true;
}
......@@ -1276,7 +1276,7 @@ static bool isUint(char *pVal, uint16_t len) {
if (pVal[0] == '-') {
return false;
}
if (strcmp(&pVal[len - 3], "u32") == 0) {
if (strcasecmp(&pVal[len - 3], "u32") == 0) {
//printf("Type is uint32(%s)\n", pVal);
return true;
}
......@@ -1287,7 +1287,7 @@ static bool isBigInt(char *pVal, uint16_t len) {
if (len <= 3) {
return false;
}
if (strcmp(&pVal[len - 3], "i64") == 0) {
if (strcasecmp(&pVal[len - 3], "i64") == 0) {
//printf("Type is int64(%s)\n", pVal);
return true;
}
......@@ -1301,7 +1301,7 @@ static bool isBigUint(char *pVal, uint16_t len) {
if (pVal[0] == '-') {
return false;
}
if (strcmp(&pVal[len - 3], "u64") == 0) {
if (strcasecmp(&pVal[len - 3], "u64") == 0) {
//printf("Type is uint64(%s)\n", pVal);
return true;
}
......@@ -1312,7 +1312,7 @@ static bool isFloat(char *pVal, uint16_t len) {
if (len <= 3) {
return false;
}
if (strcmp(&pVal[len - 3], "f32") == 0) {
if (strcasecmp(&pVal[len - 3], "f32") == 0) {
//printf("Type is float(%s)\n", pVal);
return true;
}
......@@ -1323,7 +1323,7 @@ static bool isDouble(char *pVal, uint16_t len) {
if (len <= 3) {
return false;
}
if (strcmp(&pVal[len - 3], "f64") == 0) {
if (strcasecmp(&pVal[len - 3], "f64") == 0) {
//printf("Type is double(%s)\n", pVal);
return true;
}
......@@ -1331,34 +1331,24 @@ static bool isDouble(char *pVal, uint16_t len) {
}
static bool isBool(char *pVal, uint16_t len, bool *bVal) {
if ((len == 1) &&
(pVal[len - 1] == 't' ||
pVal[len - 1] == 'T')) {
if ((len == 1) && !strcasecmp(&pVal[len - 1], "t")) {
//printf("Type is bool(%c)\n", pVal[len - 1]);
*bVal = true;
return true;
}
if ((len == 1) &&
(pVal[len - 1] == 'f' ||
pVal[len - 1] == 'F')) {
if ((len == 1) && !strcasecmp(&pVal[len - 1], "f")) {
//printf("Type is bool(%c)\n", pVal[len - 1]);
*bVal = false;
return true;
}
if((len == 4) &&
(!strcmp(&pVal[len - 4], "true") ||
!strcmp(&pVal[len - 4], "True") ||
!strcmp(&pVal[len - 4], "TRUE"))) {
if((len == 4) && !strcasecmp(&pVal[len - 4], "true")) {
//printf("Type is bool(%s)\n", &pVal[len - 4]);
*bVal = true;
return true;
}
if((len == 5) &&
(!strcmp(&pVal[len - 5], "false") ||
!strcmp(&pVal[len - 5], "False") ||
!strcmp(&pVal[len - 5], "FALSE"))) {
if((len == 5) && !strcasecmp(&pVal[len - 5], "false")) {
//printf("Type is bool(%s)\n", &pVal[len - 5]);
*bVal = false;
return true;
......@@ -1384,7 +1374,7 @@ static bool isNchar(char *pVal, uint16_t len) {
if (len < 3) {
return false;
}
if (pVal[0] == 'L' && pVal[1] == '"' && pVal[len - 1] == '"') {
if ((pVal[0] == 'l' || pVal[0] == 'L')&& pVal[1] == '"' && pVal[len - 1] == '"') {
//printf("Type is nchar(%s)\n", pVal);
return true;
}
......@@ -1434,7 +1424,7 @@ static bool isTimeStamp(char *pVal, uint16_t len, SMLTimeStampType *tsType) {
return false;
}
static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str, SSmlLinesInfo* info) {
static bool convertStrToNumber(TAOS_SML_KV *pVal, char *str, SSmlLinesInfo* info) {
errno = 0;
uint8_t type = pVal->type;
int16_t length = pVal->length;
......@@ -1442,6 +1432,7 @@ static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str, SSmlLinesInfo* info)
uint64_t val_u;
double val_d;
strntolower_s(str, str, (int32_t)strlen(str));
if (IS_FLOAT_TYPE(type)) {
val_d = strtod(str, NULL);
} else {
......@@ -1659,9 +1650,19 @@ bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
memcpy(pVal->value, &bVal, pVal->length);
return true;
}
//Handle default(no appendix) as float
if (isValidInteger(value) || isValidFloat(value)) {
pVal->type = TSDB_DATA_TYPE_FLOAT;
//Handle default(no appendix) interger type as BIGINT
if (isValidInteger(value)) {
pVal->type = TSDB_DATA_TYPE_BIGINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
if (!convertStrToNumber(pVal, value, info)) {
return false;
}
return true;
}
//Handle default(no appendix) floating number type as DOUBLE
if (isValidFloat(value)) {
pVal->type = TSDB_DATA_TYPE_DOUBLE;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
if (!convertStrToNumber(pVal, value, info)) {
return false;
......@@ -1724,6 +1725,7 @@ int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
SMLTimeStampType type;
int64_t tsVal;
strntolower_s(value, value, len);
if (!isTimeStamp(value, len, &type)) {
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
}
......
......@@ -38,7 +38,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **index,
uint16_t len = 0;
pSml->stableName = tcalloc(TSDB_TABLE_NAME_LEN + 1, 1); // +1 to avoid 1772 line over write
if (pSml->stableName == NULL){
if (pSml->stableName == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
if (isdigit(*cur)) {
......@@ -58,7 +58,13 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **index,
break;
}
pSml->stableName[len] = *cur;
//convert dot to underscore for now, will be removed once dot is allowed in tbname.
if (*cur == '.') {
pSml->stableName[len] = '_';
} else {
pSml->stableName[len] = *cur;
}
cur++;
len++;
}
......@@ -178,6 +184,9 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **index, SHashObj
tscError("OTD:0x%"PRIx64" Tag key cannot exceeds 65 characters", info->id);
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
}
if (*cur == ' ') {
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
}
if (*cur == '=') {
break;
}
......@@ -211,8 +220,8 @@ static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **index,
start = cur = *index;
while (1) {
// ',' or '\0' identifies a value
if (*cur == ',' || *cur == '\0') {
// whitespace or '\0' identifies a value
if (*cur == ' ' || *cur == '\0') {
// '\0' indicates end of value
*is_last_kv = (*cur == '\0') ? true : false;
break;
......@@ -452,6 +461,13 @@ int32_t parseMetricFromJSON(cJSON *root, TAOS_SML_DATA_POINT* pSml, SSmlLinesInf
return TSDB_CODE_TSC_INVALID_JSON;
}
//convert dot to underscore for now, will be removed once dot is allowed in tbname.
for (int i = 0; i < strlen(metric->valuestring); ++i) {
if (metric->valuestring[i] == '.') {
metric->valuestring[i] = '_';
}
}
tstrncpy(pSml->stableName, metric->valuestring, stableLen + 1);
return TSDB_CODE_SUCCESS;
......@@ -482,6 +498,7 @@ int32_t parseTimestampFromJSONObj(cJSON *root, int64_t *tsVal, SSmlLinesInfo* in
}
size_t typeLen = strlen(type->valuestring);
strntolower_s(type->valuestring, type->valuestring, (int32_t)typeLen);
if (typeLen == 1 && type->valuestring[0] == 's') {
//seconds
*tsVal = (int64_t)(*tsVal * 1e9);
......@@ -502,6 +519,8 @@ int32_t parseTimestampFromJSONObj(cJSON *root, int64_t *tsVal, SSmlLinesInfo* in
default:
return TSDB_CODE_TSC_INVALID_JSON;
}
} else {
return TSDB_CODE_TSC_INVALID_JSON;
}
return TSDB_CODE_SUCCESS;
......@@ -722,16 +741,34 @@ int32_t parseValueFromJSON(cJSON *root, TAOS_SML_KV *pVal, SSmlLinesInfo* info)
break;
}
case cJSON_Number: {
//convert default JSON Number type to float
pVal->type = TSDB_DATA_TYPE_FLOAT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
pVal->value = tcalloc(pVal->length, 1);
*(float *)(pVal->value) = (float)(root->valuedouble);
//convert default JSON Number type to BIGINT/DOUBLE
if (isValidInteger(root->numberstring)) {
pVal->type = TSDB_DATA_TYPE_BIGINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
pVal->value = tcalloc(pVal->length, 1);
*(int64_t *)(pVal->value) = (int64_t)(root->valuedouble);
} else if (isValidFloat(root->numberstring)) {
pVal->type = TSDB_DATA_TYPE_DOUBLE;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
pVal->value = tcalloc(pVal->length, 1);
*(double *)(pVal->value) = (double)(root->valuedouble);
} else {
return TSDB_CODE_TSC_INVALID_JSON_TYPE;
}
break;
}
case cJSON_String: {
//convert default JSON String type to nchar
pVal->type = TSDB_DATA_TYPE_NCHAR;
/* set default JSON type to binary/nchar according to
* user configured parameter tsDefaultJSONStrType
*/
if (strcasecmp(tsDefaultJSONStrType, "binary") == 0) {
pVal->type = TSDB_DATA_TYPE_BINARY;
} else if (strcasecmp(tsDefaultJSONStrType, "nchar") == 0) {
pVal->type = TSDB_DATA_TYPE_NCHAR;
} else {
tscError("OTD:0x%"PRIx64" Invalid default JSON string type set from config %s", info->id, tsDefaultJSONStrType);
return TSDB_CODE_TSC_INVALID_JSON_CONFIG;
}
//pVal->length = wcslen((wchar_t *)root->valuestring) * TSDB_NCHAR_SIZE;
pVal->length = (int16_t)strlen(root->valuestring);
pVal->value = tcalloc(pVal->length + 1, 1);
......
此差异已折叠。
......@@ -703,11 +703,6 @@ static int32_t tscEstimateQueryMsgSize(SSqlObj *pSql) {
}
}
SCond* pCond = &pQueryInfo->tagCond.tbnameCond;
if (pCond->len > 0) {
srcColListSize += pCond->len;
}
return MIN_QUERY_MSG_PKT_SIZE + minMsgSize() + sizeof(SQueryTableMsg) + srcColListSize + srcColFilterSize + srcTagFilterSize +
exprSize + tsBufSize + tableSerialize + sqlLen + 4096 + pQueryInfo->bufLen;
}
......@@ -956,8 +951,6 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->numOfOutput = htons((int16_t)query.numOfOutput); // this is the stage one output column number
pQueryMsg->numOfGroupCols = htons(pQueryInfo->groupbyExpr.numOfGroupCols);
pQueryMsg->tagNameRelType = htons(pQueryInfo->tagCond.relType);
pQueryMsg->tbnameCondLen = htonl(pQueryInfo->tagCond.tbnameCond.len);
pQueryMsg->queryType = htonl(pQueryInfo->type);
pQueryMsg->prevResultLen = htonl(pQueryInfo->bufLen);
......@@ -1076,12 +1069,6 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pMsg += pQueryInfo->bufLen;
}
SCond* pCond = &pQueryInfo->tagCond.tbnameCond;
if (pCond->len > 0) {
strncpy(pMsg, pCond->cond, pCond->len);
pMsg += pCond->len;
}
// compressed ts block
pQueryMsg->tsBuf.tsOffset = htonl((int32_t)(pMsg - pCmd->payload));
......
......@@ -1347,7 +1347,11 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
pCmd->command = TSDB_SQL_SELECT;
tscResetForNextRetrieve(&pSql->res);
assert(pSupporter->f == NULL);
if (pSupporter->f != NULL) {
fclose(pSupporter->f);
pSupporter->f = NULL;
}
taosGetTmpfilePath("ts-join", pSupporter->path);
// TODO check for failure
......
......@@ -50,6 +50,7 @@ int tscLogFileNum = 10;
static pthread_mutex_t rpcObjMutex; // mutex to protect open the rpc obj concurrently
static pthread_once_t tscinit = PTHREAD_ONCE_INIT;
static pthread_mutex_t setConfMutex = PTHREAD_MUTEX_INITIALIZER;
// pthread_once can not return result code, so result code is set to a global variable.
static volatile int tscInitRes = 0;
......@@ -249,6 +250,7 @@ void taos_cleanup(void) {
pthread_mutex_destroy(&rpcObjMutex);
}
pthread_mutex_destroy(&setConfMutex);
taosCacheCleanup(tscVgroupListBuf);
tscVgroupListBuf = NULL;
......@@ -437,3 +439,66 @@ int taos_options(TSDB_OPTION option, const void *arg, ...) {
atomic_store_32(&lock, 0);
return ret;
}
#include "cJSON.h"
static setConfRet taos_set_config_imp(const char *config){
setConfRet ret = {SET_CONF_RET_SUCC, {0}};
static bool setConfFlag = false;
if (setConfFlag) {
ret.retCode = SET_CONF_RET_ERR_ONLY_ONCE;
strcpy(ret.retMsg, "configuration can only set once");
return ret;
}
taosInitGlobalCfg();
cJSON *root = cJSON_Parse(config);
if (root == NULL){
ret.retCode = SET_CONF_RET_ERR_JSON_PARSE;
strcpy(ret.retMsg, "parse json error");
return ret;
}
int size = cJSON_GetArraySize(root);
if(!cJSON_IsObject(root) || size == 0) {
ret.retCode = SET_CONF_RET_ERR_JSON_INVALID;
strcpy(ret.retMsg, "json content is invalid, must be not empty object");
return ret;
}
if(size >= 1000) {
ret.retCode = SET_CONF_RET_ERR_TOO_LONG;
strcpy(ret.retMsg, "json object size is too long");
return ret;
}
for(int i = 0; i < size; i++){
cJSON *item = cJSON_GetArrayItem(root, i);
if(!item) {
ret.retCode = SET_CONF_RET_ERR_INNER;
strcpy(ret.retMsg, "inner error");
return ret;
}
if(!taosReadConfigOption(item->string, item->valuestring, NULL, NULL, TAOS_CFG_CSTATUS_OPTION, TSDB_CFG_CTYPE_B_CLIENT)){
ret.retCode = SET_CONF_RET_ERR_PART;
if (strlen(ret.retMsg) == 0){
snprintf(ret.retMsg, RET_MSG_LENGTH, "part error|%s", item->string);
}else{
int tmp = RET_MSG_LENGTH - 1 - (int)strlen(ret.retMsg);
size_t leftSize = tmp >= 0 ? tmp : 0;
strncat(ret.retMsg, "|", leftSize);
tmp = RET_MSG_LENGTH - 1 - (int)strlen(ret.retMsg);
leftSize = tmp >= 0 ? tmp : 0;
strncat(ret.retMsg, item->string, leftSize);
}
}
}
cJSON_Delete(root);
setConfFlag = true;
return ret;
}
setConfRet taos_set_config(const char *config){
pthread_mutex_lock(&setConfMutex);
setConfRet ret = taos_set_config_imp(config);
pthread_mutex_unlock(&setConfMutex);
return ret;
}
......@@ -269,7 +269,10 @@ bool tscIsProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex) {
functionId != TSDB_FUNC_DIFF &&
functionId != TSDB_FUNC_DERIVATIVE &&
functionId != TSDB_FUNC_TS_DUMMY &&
functionId != TSDB_FUNC_TID_TAG) {
functionId != TSDB_FUNC_TID_TAG &&
functionId != TSDB_FUNC_CEIL &&
functionId != TSDB_FUNC_FLOOR &&
functionId != TSDB_FUNC_ROUND) {
return false;
}
}
......@@ -811,7 +814,7 @@ typedef struct SDummyInputInfo {
SSDataBlock *block;
STableQueryInfo *pTableQueryInfo;
SSqlObj *pSql; // refactor: remove it
SFilterInfo *pFilterInfo;
void *pFilterInfo;
} SDummyInputInfo;
typedef struct SJoinStatus {
......@@ -827,7 +830,7 @@ typedef struct SJoinOperatorInfo {
SRspResultInfo resultInfo; // todo refactor, add this info for each operator
} SJoinOperatorInfo;
static void doSetupSDataBlock(SSqlRes* pRes, SSDataBlock* pBlock, SFilterInfo* pFilterInfo) {
static void doSetupSDataBlock(SSqlRes* pRes, SSDataBlock* pBlock, void* pFilterInfo) {
int32_t offset = 0;
char* pData = pRes->data;
......@@ -844,8 +847,9 @@ static void doSetupSDataBlock(SSqlRes* pRes, SSDataBlock* pBlock, SFilterInfo* p
// filter data if needed
if (pFilterInfo) {
//doSetFilterColumnInfo(pFilterInfo, numOfFilterCols, pBlock);
filterSetColFieldData(pFilterInfo, pBlock->info.numOfCols, pBlock->pDataBlock);
SColumnDataParam param = {.numOfCols = pBlock->info.numOfCols, .pDataBlock = pBlock->pDataBlock};
filterSetColFieldData(pFilterInfo, &param, getColumnDataFromId);
bool gotNchar = false;
filterConverNcharColumns(pFilterInfo, pBlock->info.rows, &gotNchar);
int8_t* p = NULL;
......@@ -1108,7 +1112,7 @@ static void destroyDummyInputOperator(void* param, int32_t numOfOutput) {
}
// todo this operator servers as the adapter for Operator tree and SqlRes result, remove it later
SOperatorInfo* createDummyInputOperator(SSqlObj* pSql, SSchema* pSchema, int32_t numOfCols, SFilterInfo* pFilters) {
SOperatorInfo* createDummyInputOperator(SSqlObj* pSql, SSchema* pSchema, int32_t numOfCols, void* pFilters) {
assert(numOfCols > 0);
STimeWindow win = {.skey = INT64_MIN, .ekey = INT64_MAX};
......@@ -1250,7 +1254,7 @@ void handleDownstreamOperator(SSqlObj** pSqlObjList, int32_t numOfUpstream, SQue
// if it is a join query, create join operator here
int32_t numOfCol1 = pTableMeta->tableInfo.numOfColumns;
SFilterInfo *pFilters = NULL;
void *pFilters = NULL;
STblCond *pCond = NULL;
if (px->colCond) {
pCond = tsGetTableFilter(px->colCond, pTableMeta->id.uid, 0);
......@@ -1277,7 +1281,7 @@ void handleDownstreamOperator(SSqlObj** pSqlObjList, int32_t numOfUpstream, SQue
for(int32_t i = 1; i < px->numOfTables; ++i) {
STableMeta* pTableMeta1 = tscGetMetaInfo(px, i)->pTableMeta;
numOfCol1 = pTableMeta1->tableInfo.numOfColumns;
SFilterInfo *pFilters1 = NULL;
void *pFilters1 = NULL;
SSchema* pSchema1 = tscGetTableSchema(pTableMeta1);
int32_t n = pTableMeta1->tableInfo.numOfColumns;
......@@ -1465,7 +1469,12 @@ void tscFreeSubobj(SSqlObj* pSql) {
tscDebug("0x%"PRIx64" start to free sub SqlObj, numOfSub:%d", pSql->self, pSql->subState.numOfSub);
for(int32_t i = 0; i < pSql->subState.numOfSub; ++i) {
tscDebug("0x%"PRIx64" free sub SqlObj:0x%"PRIx64", index:%d", pSql->self, pSql->pSubs[i]->self, i);
if (pSql->pSubs[i] != NULL) {
tscDebug("0x%"PRIx64" free sub SqlObj:0x%"PRIx64", index:%d", pSql->self, pSql->pSubs[i]->self, i);
} else {
/* just for python error test case */
tscDebug("0x%"PRIx64" free sub SqlObj:0x0, index:%d", pSql->self, i);
}
taos_free_result(pSql->pSubs[i]);
pSql->pSubs[i] = NULL;
}
......@@ -2898,16 +2907,6 @@ bool tscValidateColumnId(STableMetaInfo* pTableMetaInfo, int32_t colId, int32_t
int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
memset(dest, 0, sizeof(STagCond));
if (src->tbnameCond.cond != NULL) {
dest->tbnameCond.cond = strdup(src->tbnameCond.cond);
if (dest->tbnameCond.cond == NULL) {
return -1;
}
}
dest->tbnameCond.uid = src->tbnameCond.uid;
dest->tbnameCond.len = src->tbnameCond.len;
dest->joinInfo.hasJoin = src->joinInfo.hasJoin;
for (int32_t i = 0; i < TSDB_MAX_JOIN_TABLE_NUM; ++i) {
......@@ -2926,9 +2925,6 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
}
}
dest->relType = src->relType;
if (src->pCond == NULL) {
return 0;
}
......@@ -3018,8 +3014,6 @@ void tscColCondRelease(SArray** pCond) {
void tscTagCondRelease(STagCond* pTagCond) {
free(pTagCond->tbnameCond.cond);
if (pTagCond->pCond != NULL) {
size_t s = taosArrayGetSize(pTagCond->pCond);
for (int32_t i = 0; i < s; ++i) {
......
......@@ -17,5 +17,5 @@ IF (HEADER_GTEST_INCLUDE_DIR AND (LIB_GTEST_STATIC_DIR OR LIB_GTEST_SHARED_DIR))
AUX_SOURCE_DIRECTORY(${CMAKE_CURRENT_SOURCE_DIR} SOURCE_LIST)
ADD_EXECUTABLE(cliTest ${SOURCE_LIST})
TARGET_LINK_LIBRARIES(cliTest taos tutil common gtest pthread)
TARGET_LINK_LIBRARIES(cliTest taos cJson tutil common gtest pthread)
ENDIF()
#include <gtest/gtest.h>
#include <inttypes.h>
#include "taos.h"
#include "tglobal.h"
#include "tconfig.h"
/* test set config function */
TEST(testCase, set_config_test1) {
const char *config = "{\"debugFlag\":\"131\"}";
setConfRet ret = taos_set_config(config);
ASSERT_EQ(ret.retCode, 0);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
const char *config2 = "{\"debugFlag\":\"199\"}";
ret = taos_set_config(config2); // not take effect
ASSERT_EQ(ret.retCode, -5);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
bool readResult = taosReadGlobalCfg(); // load file config, debugFlag not take effect
ASSERT_TRUE(readResult);
int32_t checkResult = taosCheckGlobalCfg();
ASSERT_EQ(checkResult, 0);
SGlobalCfg *cfg = taosGetConfigOption("debugFlag");
ASSERT_EQ(cfg->cfgStatus, TAOS_CFG_CSTATUS_OPTION);
int32_t result = *(int32_t *)cfg->ptr;
ASSERT_EQ(result, 131);
}
TEST(testCase, set_config_test2) {
const char *config = "{\"numOfCommitThreads\":\"10\"}";
taos_set_config(config);
bool readResult = taosReadGlobalCfg(); // load file config, debugFlag not take effect
ASSERT_TRUE(readResult);
int32_t checkResult = taosCheckGlobalCfg();
ASSERT_EQ(checkResult, 0);
SGlobalCfg *cfg = taosGetConfigOption("numOfCommitThreads");
int32_t result = *(int32_t*)cfg->ptr;
ASSERT_NE(result, 10); // numOfCommitThreads not type of TSDB_CFG_CTYPE_B_CLIENT
}
TEST(testCase, set_config_test3) {
const char *config = "{\"numOfCoitThreads\":\"10\", \"esdfa\":\"10\"}";
setConfRet ret = taos_set_config(config);
ASSERT_EQ(ret.retCode, -1);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
}
TEST(testCase, set_config_test4) {
const char *config = "{null}";
setConfRet ret = taos_set_config(config);
ASSERT_EQ(ret.retCode, -4);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
}
TEST(testCase, set_config_test5) {
const char *config = "\"ddd\"";
setConfRet ret = taos_set_config(config);
ASSERT_EQ(ret.retCode, -3);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
}
TEST(testCase, set_config_test6) {
const char *config = "{\"numOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitThreadsnumOfCoitT3333dd\":\"10\", \"esdfa\":\"10\"}";
setConfRet ret = taos_set_config(config);
ASSERT_EQ(ret.retCode, -1);
printf("msg:%d->%s", ret.retCode, ret.retMsg);
}
......@@ -88,7 +88,6 @@ void tExprTreeDestroy(tExprNode *pNode, void (*fp)(void *));
void exprTreeToBinary(SBufferWriter* bw, tExprNode* pExprTree);
tExprNode* exprTreeFromBinary(const void* data, size_t size);
tExprNode* exprTreeFromTableName(const char* tbnameCond);
tExprNode* exprdup(tExprNode* pTree);
void exprTreeToBinary(SBufferWriter* bw, tExprNode* pExprTree);
......
......@@ -218,7 +218,7 @@ extern int32_t cqDebugFlag;
extern int32_t debugFlag;
#ifdef TD_TSZ
// lossy
// lossy
extern char lossyColumns[];
extern double fPrecision;
extern double dPrecision;
......@@ -226,9 +226,12 @@ extern uint32_t maxRange;
extern uint32_t curRange;
extern char Compressor[];
#endif
// long query
// long query
extern int8_t tsDeadLockKillQuery;
// schemaless
extern char tsDefaultJSONStrType[];
typedef struct {
char dir[TSDB_FILENAME_LEN];
int level;
......
......@@ -325,14 +325,6 @@ static void* exception_calloc(size_t nmemb, size_t size) {
return p;
}
static void* exception_malloc(size_t size) {
void* p = malloc(size);
if (p == NULL) {
THROW(TSDB_CODE_QRY_OUT_OF_MEMORY);
}
return p;
}
static UNUSED_FUNC char* exception_strdup(const char* str) {
char* p = strdup(str);
if (p == NULL) {
......@@ -395,97 +387,6 @@ tExprNode* exprTreeFromBinary(const void* data, size_t size) {
return exprTreeFromBinaryImpl(&br);
}
tExprNode* exprTreeFromTableName(const char* tbnameCond) {
if (!tbnameCond) {
return NULL;
}
int32_t anchor = CLEANUP_GET_ANCHOR();
tExprNode* expr = exception_calloc(1, sizeof(tExprNode));
CLEANUP_PUSH_VOID_PTR_PTR(true, tExprTreeDestroy, expr, NULL);
expr->nodeType = TSQL_NODE_EXPR;
tExprNode* left = exception_calloc(1, sizeof(tExprNode));
expr->_node.pLeft = left;
left->nodeType = TSQL_NODE_COL;
SSchema* pSchema = exception_calloc(1, sizeof(SSchema));
left->pSchema = pSchema;
*pSchema = *tGetTbnameColumnSchema();
tExprNode* right = exception_calloc(1, sizeof(tExprNode));
expr->_node.pRight = right;
if (strncmp(tbnameCond, QUERY_COND_REL_PREFIX_LIKE, QUERY_COND_REL_PREFIX_LIKE_LEN) == 0) {
right->nodeType = TSQL_NODE_VALUE;
expr->_node.optr = TSDB_RELATION_LIKE;
tVariant* pVal = exception_calloc(1, sizeof(tVariant));
right->pVal = pVal;
size_t len = strlen(tbnameCond + QUERY_COND_REL_PREFIX_LIKE_LEN) + 1;
pVal->pz = exception_malloc(len);
memcpy(pVal->pz, tbnameCond + QUERY_COND_REL_PREFIX_LIKE_LEN, len);
pVal->nType = TSDB_DATA_TYPE_BINARY;
pVal->nLen = (int32_t)len;
} else if (strncmp(tbnameCond, QUERY_COND_REL_PREFIX_MATCH, QUERY_COND_REL_PREFIX_MATCH_LEN) == 0) {
right->nodeType = TSQL_NODE_VALUE;
expr->_node.optr = TSDB_RELATION_MATCH;
tVariant* pVal = exception_calloc(1, sizeof(tVariant));
right->pVal = pVal;
size_t len = strlen(tbnameCond + QUERY_COND_REL_PREFIX_MATCH_LEN) + 1;
pVal->pz = exception_malloc(len);
memcpy(pVal->pz, tbnameCond + QUERY_COND_REL_PREFIX_MATCH_LEN, len);
pVal->nType = TSDB_DATA_TYPE_BINARY;
pVal->nLen = (int32_t)len;
} else if (strncmp(tbnameCond, QUERY_COND_REL_PREFIX_NMATCH, QUERY_COND_REL_PREFIX_NMATCH_LEN) == 0) {
right->nodeType = TSQL_NODE_VALUE;
expr->_node.optr = TSDB_RELATION_NMATCH;
tVariant* pVal = exception_calloc(1, sizeof(tVariant));
right->pVal = pVal;
size_t len = strlen(tbnameCond + QUERY_COND_REL_PREFIX_NMATCH_LEN) + 1;
pVal->pz = exception_malloc(len);
memcpy(pVal->pz, tbnameCond + QUERY_COND_REL_PREFIX_NMATCH_LEN, len);
pVal->nType = TSDB_DATA_TYPE_BINARY;
pVal->nLen = (int32_t)len;
} else if (strncmp(tbnameCond, QUERY_COND_REL_PREFIX_IN, QUERY_COND_REL_PREFIX_IN_LEN) == 0) {
right->nodeType = TSQL_NODE_VALUE;
expr->_node.optr = TSDB_RELATION_IN;
tVariant* pVal = exception_calloc(1, sizeof(tVariant));
right->pVal = pVal;
pVal->nType = TSDB_DATA_TYPE_POINTER_ARRAY;
pVal->arr = taosArrayInit(2, POINTER_BYTES);
const char* cond = tbnameCond + QUERY_COND_REL_PREFIX_IN_LEN;
for (const char *e = cond; *e != 0; e++) {
if (*e == TS_PATH_DELIMITER[0]) {
cond = e + 1;
} else if (*e == ',') {
size_t len = e - cond;
char* p = exception_malloc(len + VARSTR_HEADER_SIZE);
STR_WITH_SIZE_TO_VARSTR(p, cond, (VarDataLenT)len);
cond += len;
taosArrayPush(pVal->arr, &p);
}
}
if (*cond != 0) {
size_t len = strlen(cond) + VARSTR_HEADER_SIZE;
char* p = exception_malloc(len);
STR_WITH_SIZE_TO_VARSTR(p, cond, (VarDataLenT)(len - VARSTR_HEADER_SIZE));
taosArrayPush(pVal->arr, &p);
}
taosArraySortString(pVal->arr, taosArrayCompareString);
}
CLEANUP_EXECUTE_TO(anchor, false);
return expr;
}
void buildFilterSetFromBinary(void **q, const char *buf, int32_t len) {
SBufferReader br = tbufInitReader(buf, len, false);
uint32_t type = tbufReadUint32(&br);
......
......@@ -284,6 +284,9 @@ char Compressor[32] = "ZSTD_COMPRESSOR"; // ZSTD_COMPRESSOR or GZIP_COMPRESS
// long query death-lock
int8_t tsDeadLockKillQuery = 0;
// default JSON string type
char tsDefaultJSONStrType[7] = "binary";
int32_t (*monStartSystemFp)() = NULL;
void (*monStopSystemFp)() = NULL;
void (*monExecuteSQLFp)(char *sql) = NULL;
......@@ -1639,6 +1642,17 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
// default JSON string type option "binary"/"nchar"
cfg.option = "defaultJSONStrType";
cfg.ptr = tsDefaultJSONStrType;
cfg.valType = TAOS_CFG_VTYPE_STRING;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 0;
cfg.maxValue = 0;
cfg.ptrLength = tListLen(tsDefaultJSONStrType);
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
#ifdef TD_TSZ
// lossy compress
cfg.option = "lossyColumns";
......
......@@ -8,7 +8,7 @@ IF (TD_MVN_INSTALLED)
ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME}
POST_BUILD
COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.34-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.35-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMENT "build jdbc driver")
ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME})
......
......@@ -5,7 +5,7 @@
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.34</version>
<version>2.0.35</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
......
......@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.34</version>
<version>2.0.35</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
......
......@@ -118,9 +118,6 @@ public class TSDBDriver extends AbstractDriver {
}
public Connection connect(String url, Properties info) throws SQLException {
if (url == null)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_URL_NOT_SET);
if (!acceptsURL(url))
return null;
......@@ -135,8 +132,7 @@ public class TSDBDriver extends AbstractDriver {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_PASSWORD_IS_REQUIRED);
try {
TSDBJNIConnector.init((String) props.get(PROPERTY_KEY_CONFIG_DIR), (String) props.get(PROPERTY_KEY_LOCALE),
(String) props.get(PROPERTY_KEY_CHARSET), (String) props.get(PROPERTY_KEY_TIME_ZONE));
TSDBJNIConnector.init(props);
return new TSDBConnection(props, this.dbMetaData);
} catch (SQLWarning sqlWarning) {
sqlWarning.printStackTrace();
......@@ -205,6 +201,7 @@ public class TSDBDriver extends AbstractDriver {
String dbProductName = url.substring(0, beginningOfSlashes);
dbProductName = dbProductName.substring(dbProductName.indexOf(":") + 1);
dbProductName = dbProductName.substring(0, dbProductName.indexOf(":"));
urlProps.setProperty(TSDBDriver.PROPERTY_KEY_PRODUCT_NAME, dbProductName);
// parse database name
url = url.substring(beginningOfSlashes + 2);
......
......@@ -35,6 +35,7 @@ public class TSDBError {
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN_TIMESTAMP_PRECISION, "unknown timestamp precision");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_USER_IS_REQUIRED, "user is required");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_PASSWORD_IS_REQUIRED, "password is required");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_INVALID_JSON_FORMAT, "invalid json format");
TSDBErrorMap.put(TSDBErrorNumbers.ERROR_UNKNOWN, "unknown error");
......
......@@ -31,6 +31,7 @@ public class TSDBErrorNumbers {
public static final int ERROR_RESTFul_Client_IOException = 0x2318;
public static final int ERROR_USER_IS_REQUIRED = 0x2319; // user is required
public static final int ERROR_PASSWORD_IS_REQUIRED = 0x231a; // password is required
public static final int ERROR_INVALID_JSON_FORMAT = 0x231b;
public static final int ERROR_UNKNOWN = 0x2350; //unknown error
......@@ -72,6 +73,7 @@ public class TSDBErrorNumbers {
errorNumbers.add(ERROR_RESTFul_Client_IOException);
errorNumbers.add(ERROR_USER_IS_REQUIRED);
errorNumbers.add(ERROR_PASSWORD_IS_REQUIRED);
errorNumbers.add(ERROR_INVALID_JSON_FORMAT);
errorNumbers.add(ERROR_RESTFul_Client_Protocol_Exception);
......
package com.taosdata.jdbc;
public class TSDBException {
private int code;
private String message;
public int getCode() {
return code;
}
public void setCode(int code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
}
\ No newline at end of file
......@@ -16,18 +16,21 @@
*/
package com.taosdata.jdbc;
import com.alibaba.fastjson.JSONObject;
import com.taosdata.jdbc.utils.TaosInfo;
import java.nio.ByteBuffer;
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.List;
import java.util.Properties;
/**
* JNI connector
*/
public class TSDBJNIConnector {
private static volatile Boolean isInitialized = false;
private static final Object LOCK = new Object();
private static volatile boolean isInitialized;
private final TaosInfo taosInfo = TaosInfo.getInstance();
private long taos = TSDBConstants.JNI_NULL_POINTER; // Connection pointer used in C
......@@ -38,24 +41,27 @@ public class TSDBJNIConnector {
System.loadLibrary("taos");
}
public boolean isClosed() {
return this.taos == TSDBConstants.JNI_NULL_POINTER;
}
public static void init(Properties props) throws SQLWarning {
synchronized (LOCK) {
if (!isInitialized) {
public boolean isResultsetClosed() {
return this.isResultsetClosed;
}
JSONObject configJSON = new JSONObject();
for (String key : props.stringPropertyNames()) {
configJSON.put(key, props.getProperty(key));
}
setConfigImp(configJSON.toJSONString());
public static void init(String configDir, String locale, String charset, String timezone) throws SQLWarning {
synchronized (isInitialized) {
if (!isInitialized) {
initImp(configDir);
initImp(props.getProperty(TSDBDriver.PROPERTY_KEY_CONFIG_DIR, null));
String locale = props.getProperty(TSDBDriver.PROPERTY_KEY_LOCALE);
if (setOptions(0, locale) < 0) {
throw TSDBError.createSQLWarning("Failed to set locale: " + locale + ". System default will be used.");
}
String charset = props.getProperty(TSDBDriver.PROPERTY_KEY_CHARSET);
if (setOptions(1, charset) < 0) {
throw TSDBError.createSQLWarning("Failed to set charset: " + charset + ". System default will be used.");
}
String timezone = props.getProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE);
if (setOptions(2, timezone) < 0) {
throw TSDBError.createSQLWarning("Failed to set timezone: " + timezone + ". System default will be used.");
}
......@@ -65,11 +71,13 @@ public class TSDBJNIConnector {
}
}
public static native void initImp(String configDir);
private static native void initImp(String configDir);
private static native int setOptions(int optionIndex, String optionValue);
public static native int setOptions(int optionIndex, String optionValue);
private static native String getTsCharset();
public static native String getTsCharset();
private static native TSDBException setConfigImp(String config);
public boolean connect(String host, int port, String dbName, String user, String password) throws SQLException {
if (this.taos != TSDBConstants.JNI_NULL_POINTER) {
......@@ -159,6 +167,14 @@ public class TSDBJNIConnector {
private native long isUpdateQueryImp(long connection, long pSql);
public boolean isClosed() {
return this.taos == TSDBConstants.JNI_NULL_POINTER;
}
public boolean isResultsetClosed() {
return this.isResultsetClosed;
}
/**
* Free result set operation from C to release result set pointer by JNI
*/
......@@ -351,4 +367,6 @@ public class TSDBJNIConnector {
}
private native int insertLinesImp(String[] lines, long conn);
}
......@@ -5,9 +5,9 @@ import org.junit.Test;
import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
......@@ -19,25 +19,25 @@ public class TSDBJNIConnectorTest {
@Test
public void test() {
try {
try {
//change sleepSeconds when debugging with attach to process to find PID
int sleepSeconds = -1;
if (sleepSeconds>0) {
if (sleepSeconds > 0) {
RuntimeMXBean runtimeBean = ManagementFactory.getRuntimeMXBean();
String jvmName = runtimeBean.getName();
long pid = Long.valueOf(jvmName.split("@")[0]);
System.out.println("JVM PID = " + pid);
Thread.sleep(sleepSeconds*1000);
Thread.sleep(sleepSeconds * 1000);
}
}
catch (Exception e) {
} catch (Exception e) {
e.printStackTrace();
}
// init
TSDBJNIConnector.init("/etc/taos", null, null, null);
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_CONFIG_DIR, "/etc/taos");
TSDBJNIConnector.init(properties);
// connect
TSDBJNIConnector connector = new TSDBJNIConnector();
......@@ -45,12 +45,12 @@ public class TSDBJNIConnectorTest {
// setup
String setupSqlStrs[] = {"create database if not exists d precision \"us\"",
"create table if not exists d.t(ts timestamp, f int)",
"create database if not exists d2",
"create table if not exists d2.t2(ts timestamp, f int)",
"insert into d.t values(now+100s, 100)",
"insert into d2.t2 values(now+200s, 200)"
};
"create table if not exists d.t(ts timestamp, f int)",
"create database if not exists d2",
"create table if not exists d2.t2(ts timestamp, f int)",
"insert into d.t values(now+100s, 100)",
"insert into d2.t2 values(now+200s, 200)"
};
for (String setupSqlStr : setupSqlStrs) {
long setupSql = connector.executeQuery(setupSqlStr);
......@@ -115,15 +115,13 @@ public class TSDBJNIConnectorTest {
}
// close statement
connector.executeQuery("use d");
String[] lines = new String[] {"st,t1=3i64,t2=4f64,t3=\"t3\" c1=3i64,c3=L\"passit\",c2=false,c4=4f64 1626006833639000000ns",
"st,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin\",c2=true,c4=5f64,c5=5f64 1626006833640000000ns"};
String[] lines = new String[]{"st,t1=3i64,t2=4f64,t3=\"t3\" c1=3i64,c3=L\"passit\",c2=false,c4=4f64 1626006833639000000ns",
"st,t1=4i64,t3=\"t4\",t2=5f64,t4=5f64 c1=3i64,c3=L\"passitagin\",c2=true,c4=5f64,c5=5f64 1626006833640000000ns"};
connector.insertLines(lines);
// close connection
connector.closeConnection();
} catch (SQLWarning throwables) {
throwables.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
......@@ -140,11 +138,7 @@ public class TSDBJNIConnectorTest {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} else if (code == TSDBConstants.JNI_FETCH_END) {
return false;
} else {
return true;
}
} else return code != TSDBConstants.JNI_FETCH_END;
}
}
......@@ -165,12 +165,14 @@ def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, precision=Field
assert nbytes is not None
res = []
for i in range(abs(num_of_rows)):
try:
rbyte = ctypes.cast(data + nbytes * i, ctypes.POINTER(ctypes.c_short))[:1].pop()
tmpstr = ctypes.c_char_p(data + nbytes * i + 2)
res.append(tmpstr.value.decode()[0:rbyte])
except ValueError:
rbyte = ctypes.cast(data + nbytes * i, ctypes.POINTER(ctypes.c_short))[:1].pop()
chars = ctypes.cast(c_char_p(data + nbytes * i + 2), ctypes.POINTER(c_char * rbyte))
buffer = create_string_buffer(rbyte + 1)
buffer[:rbyte] = chars[0][:rbyte]
if rbyte == 1 and buffer[0] == b'\xff':
res.append(None)
else:
res.append(cast(buffer, c_char_p).value.decode())
return res
......@@ -179,11 +181,14 @@ def _crow_nchar_to_python_block(data, num_of_rows, nbytes=None, precision=FieldT
assert nbytes is not None
res = []
for i in range(abs(num_of_rows)):
try:
tmpstr = ctypes.c_char_p(data + nbytes * i + 2)
res.append(tmpstr.value.decode())
except ValueError:
rbyte = ctypes.cast(data + nbytes * i, ctypes.POINTER(ctypes.c_short))[:1].pop()
chars = ctypes.cast(c_char_p(data + nbytes * i + 2), ctypes.POINTER(c_char * rbyte))
buffer = create_string_buffer(rbyte + 1)
buffer[:rbyte] = chars[0][:rbyte]
if rbyte == 4 and buffer[:4] == b'\xff'*4:
res.append(None)
else:
res.append(cast(buffer, c_char_p).value.decode())
return res
......
......@@ -62,6 +62,22 @@ typedef struct taosField {
int16_t bytes;
} TAOS_FIELD;
typedef enum {
SET_CONF_RET_SUCC = 0,
SET_CONF_RET_ERR_PART = -1,
SET_CONF_RET_ERR_INNER = -2,
SET_CONF_RET_ERR_JSON_INVALID = -3,
SET_CONF_RET_ERR_JSON_PARSE = -4,
SET_CONF_RET_ERR_ONLY_ONCE = -5,
SET_CONF_RET_ERR_TOO_LONG = -6
} SET_CONF_RET_CODE;
#define RET_MSG_LENGTH 1024
typedef struct setConfRet {
SET_CONF_RET_CODE retCode;
char retMsg[RET_MSG_LENGTH];
} setConfRet;
#ifdef _TD_GO_DLL_
#define DLL_EXPORT __declspec(dllexport)
#else
......@@ -71,6 +87,7 @@ typedef struct taosField {
DLL_EXPORT int taos_init();
DLL_EXPORT void taos_cleanup(void);
DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...);
DLL_EXPORT setConfRet taos_set_config(const char *config);
DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port);
DLL_EXPORT TAOS *taos_connect_auth(const char *ip, const char *user, const char *auth, const char *db, uint16_t port);
DLL_EXPORT void taos_close(TAOS *taos);
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册