Skip to content

T
TDengine

怡红公子⭕ / TDengine
与 Fork 源项目一致

Fork自 taosdata / TDengine

通知 5
Star 0
Fork 0
T
TDengine

前往新版Gitcode，体验更适合开发者的 AI 搜索 >>

未验证 提交 69a5dcda 编写于 作者: S slguan 提交者： GitHub
浏览文件
操作

Merge branch 'develop' into feature/#1180

上级 26802eb7 fdfcc4aa
展开全部 隐藏空白更改
内联 并排
Showing with 3651 addition and 1476 deletion
CMakeLists.txt
浏览文件 @ 69a5dcda
......@@ -280,6 +280,11 @@ IF (NOT DEFINED TD_CLUSTER)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/libtaos.dll DESTINATION driver)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/libtaos.dll.a DESTINATION driver)
ENDIF ()
ELSEIF (TD_DARWIN_64)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
INSTALL(CODE "MESSAGE(\"make install script: ${TD_MAKE_INSTALL_SH}\")")
INSTALL(CODE "execute_process(COMMAND chmod 777 ${TD_MAKE_INSTALL_SH})")
INSTALL(CODE "execute_process(COMMAND ${TD_MAKE_INSTALL_SH} ${TD_COMMUNITY_DIR} ${PROJECT_BINARY_DIR} Darwin)")
ENDIF ()
ENDIF ()
......
documentation/webdocs/markdowndocs/Connector.md
浏览文件 @ 69a5dcda
......@@ -175,26 +175,34 @@ TDengine provides APIs for continuous query driven by time, which run queries pe
### C/C++ subscription API
For the time being, TDengine supports subscription on one table. It is implemented through periodic pulling from a TDengine server.
For the time being, TDengine supports subscription on one or multiple tables. It is implemented through periodic pulling from a TDengine server.
- `TAOS_SUB *taos_subscribe(char *host, char *user, char *pass, char *db, char *table, int64_t time, int mseconds)`
The API is used to start a subscription session by given a handle. The parameters required are _host_ (IP address of a TDenginer server), _user_ (username), _pass_ (password), _db_ (database to use), _table_ (table name to subscribe), _time_ (start time to subscribe, 0 for now), _mseconds_ (pulling period). If failed to open a subscription session, a _NULL_ pointer is returned.
* `TAOS_SUB *taos_subscribe(TAOS* taos, int restart, const char* topic, const char *sql, TAOS_SUBSCRIBE_CALLBACK fp, void *param, int interval)`
The API is used to start a subscription session, it returns the subscription object on success and `NULL` in case of failure, the parameters are:
* **taos**: The database connnection, which must be established already.
* **restart**: `Zero` to continue a subscription if it already exits, other value to start from the beginning.
* **topic**: The unique identifier of a subscription.
* **sql**: A sql statement for data query, it can only be a `select` statement, can only query for raw data, and can only query data in ascending order of the timestamp field.
* **fp**: A callback function to receive query result, only used in asynchronization mode and should be `NULL` in synchronization mode, please refer below for its prototype.
* **param**: User provided additional parameter for the callback function.
* **interval**: Pulling interval in millisecond. Under asynchronization mode, API will call the callback function `fp` in this interval, system performance will be impacted if this interval is too short. Under synchronization mode, if the duration between two call to `taos_consume` is less than this interval, the second call blocks until the duration exceed this interval.
- `TAOS_ROW taos_consume(TAOS_SUB *tsub)`
The API used to get the new data from a TDengine server. It should be put in an infinite loop. The parameter _tsub_ is the handle returned by _taos_subscribe_. If new data are updated, the API will return a row of the result. Otherwise, the API is blocked until new data arrives. If _NULL_ pointer is returned, it means an error occurs.
* `typedef void (*TAOS_SUBSCRIBE_CALLBACK)(TAOS_SUB* tsub, TAOS_RES *res, void* param, int code)`
Prototype of the callback function, the parameters are:
* tsub: The subscription object.
* res: The query result.
* param: User provided additional parameter when calling `taos_subscribe`.
* code: Error code in case of failures.
- `void taos_unsubscribe(TAOS_SUB *tsub)`
Stop a subscription session by the handle returned by _taos_subscribe_.
- `int taos_num_subfields(TAOS_SUB *tsub)`
The API used to get the number of fields in a row.
* `TAOS_RES *taos_consume(TAOS_SUB *tsub)`
The API used to get the new data from a TDengine server. It should be put in an loop. The parameter `tsub` is the handle returned by `taos_subscribe`. This API should only be called in synchronization mode. If the duration between two call to `taos_consume` is less than pulling interval, the second call blocks until the duration exceed the interval. The API returns the new rows if new data arrives, or empty rowset otherwise, and if there's an error, it returns `NULL`.
* `void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress)`
- `TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub)`
The API used to get the description of each column.
Stop a subscription session by the handle returned by `taos_subscribe`. If `keepProgress` is **not** zero, the subscription progress information is kept and can be reused in later call to `taos_subscribe`, the information is removed otherwise.
## Java Connector
......@@ -208,7 +216,7 @@ Since the native language of TDengine is C, the necessary TDengine library shoul
* taos.dll (Windows)
After TDengine client is installed, the library `taos.dll` will be automatically copied to the `C:/Windows/System32`, which is the system's default search path.
> Note: Please make sure that TDengine Windows client has been installed if developing on Windows.
> Note: Please make sure that [TDengine Windows client][14] has been installed if developing on Windows. Now although TDengine client would be defaultly installed together with TDengine server, it can also be installed [alone][15].
Since TDengine is time-series database, there are still some differences compared with traditional databases in using TDengine JDBC driver:
* TDengine doesn't allow to delete/modify a single record, and thus JDBC driver also has no such method.
......@@ -583,13 +591,35 @@ data = c1.fetchall()
numOfRows = c1.rowcount
numOfCols = len(c1.description)
for irow in range(numOfRows):
print("Row%d: ts=%s, temperature=%d, humidity=%f" %(irow, data[irow][0], data[irow][1],data[irow][2])
print("Row%d: ts=%s, temperature=%d, humidity=%f" %(irow, data[irow][0], data[irow][1],data[irow][2]))
# use the cursor as an iterator to retrieve all returned results
c1.execute('select * from tb')
for data in c1:
print("ts=%s, temperature=%d, humidity=%f" %(data[0], data[1],data[2])
```
* create a subscription
```python
# Create a subscription with topic 'test' and consumption interval 1000ms.
# The first argument is True means to restart the subscription;
# if the subscription with topic 'test' has already been created, then pass
# False to this argument means to continue the existing subscription.
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
```
* consume a subscription
```python
data = sub.consume()
for d in data:
print(d)
```
* close the subscription
```python
sub.close()
```
* close the connection
```python
c1.close()
......@@ -882,4 +912,5 @@ An example of using the NodeJS connector to achieve the same things but without
[11]: https://github.com/taosdata/TDengine/tree/develop/tests/examples/JDBC/SpringJdbcTemplate
[12]: https://github.com/taosdata/TDengine/tree/develop/tests/examples/JDBC/springbootdemo
[13]: https://www.taosdata.com/cn/documentation/administrator/#%E5%AE%A2%E6%88%B7%E7%AB%AF%E9%85%8D%E7%BD%AE
[14]: https://www.taosdata.com/cn/documentation/connector/#Windows%E5%AE%A2%E6%88%B7%E7%AB%AF%E5%8F%8A%E7%A8%8B%E5%BA%8F%E6%8E%A5%E5%8F%A3
[15]: https://www.taosdata.com/cn/getting-started/#%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B
documentation/webdocs/markdowndocs/TAOS SQL-ch.md
浏览文件 @ 69a5dcda
......@@ -423,7 +423,7 @@ count(tbname) |
###聚合函数
TDengine支持针对数据的聚合查询。提供支持的聚合和提取函数如下表：
TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数如下：
- **COUNT**
```mysql
......@@ -446,13 +446,14 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和提取函数
适用于：表、超级表。
- **WAVG**
- **TWA**
```mysql
SELECT WAVG(field_name) FROM tb_name WHERE clause
SELECT TWA(field_name) FROM tb_name WHERE clause
```
功能说明：统计表/超级表中某列在一段时间内的时间加权平均。
功能说明：时间加权平均函数。统计表/超级表中某列在一段时间内的时间加权平均。
返回结果数据类型：双精度浮点数Double。
应用字段：不能应用在timestamp、binary、nchar、bool类型字段。
应用字段：不能应用在timestamp、binary、nchar、bool类型字段。
说明：时间加权平均(time weighted average, TWA）查询需要指定查询时间段的 _开始时间_ 和 _结束时间_ 。
适用于：表、超级表。
......@@ -556,7 +557,15 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和提取函数
应用字段：不能应用在timestamp、binary、nchar、bool类型字段。
说明：*k*值取值范围0≤*k*≤100，为0的时候等同于MIN，为100的时候等同于MAX。
- **APERCENTILE**
```mysql
SELECT APERCENTILE(field_name, P) FROM { tb_name | stb_name } [WHERE clause]
```
功能说明：统计表中某列的值百分比分位数，与PERCENTILE函数相似，但是返回近似结果。
返回结果数据类型： 双精度浮点数Double。
应用字段：不能应用在timestamp、binary、nchar、bool类型字段。
说明：*k*值取值范围0≤*k*≤100，为0的时候等同于MIN，为100的时候等同于MAX。推荐使用```APERCENTILE```函数，该函数性能远胜于```PERCENTILE```函数
- **LAST_ROW**
```mysql
SELECT LAST_ROW(field_name) FROM { tb_name | stb_name }
......
documentation/webdocs/markdowndocs/administrator-ch.md
浏览文件 @ 69a5dcda
......@@ -337,6 +337,8 @@ TDengine也支持在shell对已存在的表从CSV文件中进行数据导入。
insert into tb1 file a.csv b.csv tb2 c.csv …
import into tb1 file a.csv b.csv tb2 c.csv …
```
> 注意：导入的CSV文件不能够带表头, 且表的列与CSV文件的列需要严格对应。
> 同样还可以使用[样例数据导入工具][1]对数据进行横向和纵向扩展导入。
## 数据导出
......@@ -407,3 +409,6 @@ KILL STREAM <stream-id>
TDengine启动后，会自动创建一个监测数据库`LOG`，并自动将服务器的CPU、内存、硬盘空间、带宽、请求数、磁盘读写速度、慢查询等信息定时写入该数据库。TDengine还将重要的系统操作（比如登录、创建、删除数据库等）日志以及各种错误报警信息记录下来存放在`LOG`库里。系统管理员可以通过客户端程序查看记录库中的运行负载信息，（在企业版中）还可以通过浏览器查看数据的图标可视化结果。
这些监测信息的采集缺省是打开的，但可以修改配置文件里的选项`monitor`将其关闭或打开。
[1]: https://github.com/taosdata/TDengine/tree/develop/importSampleData
\ No newline at end of file
documentation/webdocs/markdowndocs/advanced features-ch.md
浏览文件 @ 69a5dcda
......@@ -63,28 +63,11 @@ CREATE TABLE QUERY_RES
## 数据订阅(Publisher/Subscriber)
基于数据天然的时间序列特性，TDengine的数据写入（insert）与消息系统的数据发布（pub）逻辑上一致，均可视为系统中插入一条带时间戳的新记录。同时，TDengine在内部严格按照数据时间序列单调递增的方式保存数据。本质上来说，TDengine中里每一张表均可视为一个标准的消息队列。
TDengine内嵌支持轻量级的消息订阅与推送服务。使用系统提供的API，用户可订阅数据库中的某一张表（或超级表）。订阅的逻辑和操作状态的维护均是由客户端完成，客户端定时轮询服务器是否有新的记录到达，有新的记录到达就会将结果反馈到客户。
TDengine内嵌支持轻量级的消息订阅与推送服务。使用系统提供的API，用户可使用普通查询语句订阅数据库中的一张或多张表。订阅的逻辑和操作状态的维护均是由客户端完成，客户端定时轮询服务器是否有新的记录到达，有新的记录到达就会将结果反馈到客户。
TDengine的订阅与推送服务的状态是客户端维持，TDengine服务器并不维持。因此如果应用重启，从哪个时间点开始获取最新数据，由应用决定。
#### API说明
使用订阅的功能，主要API如下：
<ul>
<li><p><code>TAOS_SUB *taos_subscribe(char *host, char *user, char *pass, char *db, char *table, int64_t time, int mseconds)</code></p><p>该函数负责启动订阅服务。其中参数说明：</p></li><ul>
<li><p>host：主机IP地址</p></li>
<li><p>user：数据库登录用户名</p></li>
<li><p>pass：密码</p></li>
<li><p>db：数据库名称</p></li>
<li><p>table：(超级) 表的名称</p></li>
<li><p>time：启动时间，Unix Epoch时间，单位为毫秒。从1970年1月1日起计算的毫秒数。如果设为0，表示从当前时间开始订阅</p></li>
<li><p>mseconds：查询数据库更新的时间间隔，单位为毫秒。一般设置为1000毫秒。返回值为指向TDengine_SUB 结构的指针，如果返回为空，表示失败。</p></li>
</ul><li><p><code>TAOS_ROW taos_consume(TAOS_SUB *tsub)</code>
</p><p>该函数用来获取订阅的结果，用户应用程序将其置于一个无限循环语句。如果数据库有新记录到达，该API将返回该最新的记录。如果没有新的记录，该API将阻塞。如果返回值为空，说明系统出错。参数说明：</p></li><ul><li><p>tsub：taos_subscribe的结构体指针。</p></li></ul><li><p><code>void taos_unsubscribe(TAOS_SUB *tsub)</code></p><p>取消订阅。应用程序退出时，务必调用该函数以避免资源泄露。</p></li>
<li><p><code>int taos_num_subfields(TAOS_SUB *tsub)</code></p><p>获取返回的一行记录中数据包含多少列。</p></li>
<li><p><code>TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub)</code></p><p>获取每列数据的属性（数据类型、名字、长度），与taos_num_subfileds配合使用，可解析返回的每行数据。</p></li></ul>
示例代码：请看安装包中的的示范程序
订阅相关API请见 [连接器](https://www.taosdata.com/cn/documentation/connector/)。
## 缓存 (Cache)
TDengine采用时间驱动缓存管理策略（First-In-First-Out，FIFO），又称为写驱动的缓存管理机制。这种策略有别于读驱动的数据缓存模式（Least-Recent-Use，LRU），直接将最近写入的数据保存在系统的缓存中。当缓存达到临界值的时候，将最早的数据批量写入磁盘。一般意义上来说，对于物联网数据的使用，用户最为关心最近产生的数据，即当前状态。TDengine充分利用了这一特性，将最近到达的（当前状态）数据保存在缓存中。
......
documentation/webdocs/markdowndocs/connector-ch.md
浏览文件 @ 69a5dcda
......@@ -164,27 +164,36 @@ TDengine提供时间驱动的实时流式计算API。可以每隔一指定的时
### C/C++ 数据订阅接口
订阅API目前支持订阅一张表，并通过定期轮询的方式不断获取写入表中的最新数据。
订阅API目前支持订阅一张或多张表，并通过定期轮询的方式不断获取写入表中的最新数据。
- `TAOS_SUB *taos_subscribe(char *host, char *user, char *pass, char *db, char *table, int64_t time, int mseconds)`
* `TAOS_SUB *taos_subscribe(TAOS* taos, int restart, const char* topic, const char *sql, TAOS_SUBSCRIBE_CALLBACK fp, void *param, int interval)`
该API用来启动订阅，需要提供的参数包含：TDengine管理主节点的IP地址、用户名、密码、数据库、数据库表的名字；time是开始订阅消息的时间，是从1970年1月1日起计算的毫秒数，为长整型, 如果设为0，表示从当前时间开始订阅；mseconds为查询数据库更新的时间间隔，单位为毫秒，建议设为1000毫秒。返回值为一指向TDengine_SUB结构的指针，如果返回为空，表示失败。
该函数负责启动订阅服务，成功时返回订阅对象，失败时返回 `NULL`，其参数为：
* taos：已经建立好的数据库连接
* restart：如果订阅已经存在，是重新开始，还是继续之前的订阅
* topic：订阅的主题（即名称），此参数是订阅的唯一标识
* sql：订阅的查询语句，此语句只能是 `select` 语句，只应查询原始数据，只能按时间正序查询数据
* fp：收到查询结果时的回调函数（稍后介绍函数原型），只在异步调用时使用，同步调用时此参数应该传 `NULL`
* param：调用回调函数时的附加参数，系统API将其原样传递到回调函数，不进行任何处理
* interval：轮询周期，单位为毫秒。异步调用时，将根据此参数周期性的调用回调函数，为避免对系统性能造成影响，不建议将此参数设置的过小；同步调用时，如两次调用`taos_consume`的间隔小于此周期，API将会阻塞，直到时间间隔超过此周期。
- `TAOS_ROW taos_consume(TAOS_SUB *tsub)`
* `typedef void (*TAOS_SUBSCRIBE_CALLBACK)(TAOS_SUB* tsub, TAOS_RES *res, void* param, int code)`
该API用来获取最新消息，应用程序一般会将其置于一个无限循环语句中。其中参数tsub是taos_subscribe的返回值。如果数据库有新的记录，该API将返回，返回参数是一行记录。如果没有新的记录，该API将阻塞。如果返回值为空，说明系统出错，需要检查系统是否还在正常运行。
异步模式下，回调函数的原型，其参数为：
* tsub：订阅对象
* res：查询结果集，注意结果集中可能没有记录
* param：调用 `taos_subscribe`时客户程序提供的附加参数
* code：错误码
- `void taos_unsubscribe(TAOS_SUB *tsub)`
该API用于取消订阅，参数tsub是taos_subscribe的返回值。应用程序退出时，需要调用该API，否则有资源泄露。
* `TAOS_RES *taos_consume(TAOS_SUB *tsub)`
- `int taos_num_subfields(TAOS_SUB *tsub)`
同步模式下，该函数用来获取订阅的结果。 用户应用程序将其置于一个循环之中。 如两次调用`taos_consume`的间隔小于订阅的轮询周期，API将会阻塞，直到时间间隔超过此周期。 如果数据库有新记录到达，该API将返回该最新的记录，否则返回一个没有记录的空结果集。 如果返回值为 `NULL`，说明系统出错。 异步模式下，用户程序不应调用此API。
该API用来获取返回的一排数据中数据的列数
* `void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress)`
- `TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub)`
取消订阅。 如参数 `keepProgress` 不为0，API会保留订阅的进度信息，后续调用 `taos_subscribe` 时可以基于此进度继续；否则将删除进度信息，后续只能重新开始读取数据。
该API用来获取每列数据的属性（数据类型、名字、字节数），与taos_num_subfields配合使用，可用来解析返回的一排数据。
## Java Connector
......@@ -198,38 +207,38 @@ TDengine 为了方便 Java 应用使用，提供了遵循 JDBC 标准(3.0)API
* taos.dll
在 windows 系统中安装完客户端之后，驱动包依赖的 taos.dll 文件会自动拷贝到系统默认搜索路径 C:/Windows/System32 下，同样无需要单独指定。
> 注意：在 windows 环境开发时需要安装 TDengine 对应的 windows 版本客户端，由于目前没有提供 Linux 环境单独的客户端，需要安装 TDengine 才能使用。
> 注意：在 windows 环境开发时需要安装 TDengine 对应的 [windows 客户端][14]，Linux 服务器安装完 TDengine 之后默认已安装 client，也可以单独安装 [Linux 客户端][15] 连接远程 TDengine Server。
TDengine 的 JDBC 驱动实现尽可能的与关系型数据库驱动保持一致，但时序空间数据库与关系对象型数据库服务的对象和技术特征的差异导致 taos-jdbcdriver 并未完全实现 JDBC 标准规范。在使用时需要注意以下几点：
* TDengine 不提供针对单条数据记录的删除和修改的操作，驱动中也没有支持相关方法。
* 由于不支持删除和修改，所以也不支持事务操作。
* 目前不支持表间的 union 操作。
* 目前不支持嵌套查询(nested query)，`对每个 Connection 的实例，至多只能有一个打开的 ResultSet 实例；如果在 ResultSet还没关闭的情况下执行了新的查询，TSDBJDBCDriver 则会自动关闭上一个 ResultSet`。
* 目前不支持嵌套查询(nested query)，对每个 Connection 的实例，至多只能有一个打开的 ResultSet 实例；如果在 ResultSet还没关闭的情况下执行了新的查询，TSDBJDBCDriver 则会自动关闭上一个 ResultSet。
## TAOS-JDBCDriver 版本以及支持的 TDengine 版本和 JDK 版本
| taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 |
| --- | --- | --- |
| taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 |
| --- | --- | --- |
| 1.0.3 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.2 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.1 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.2 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.1 | 1.6.1.x 及以上 | 1.8.x |
## TDengine DataType 和 Java DataType
TDengine 目前支持时间戳、数字、字符、布尔类型，与 Java 对应类型转换如下：
| TDengine DataType | Java DataType |
| --- | --- |
| TIMESTAMP | java.sql.Timestamp |
| INT | java.lang.Integer |
| BIGINT | java.lang.Long |
| FLOAT | java.lang.Float |
| DOUBLE | java.lang.Double |
| TDengine DataType | Java DataType |
| --- | --- |
| TIMESTAMP | java.sql.Timestamp |
| INT | java.lang.Integer |
| BIGINT | java.lang.Long |
| FLOAT | java.lang.Float |
| DOUBLE | java.lang.Double |
| SMALLINT, TINYINT |java.lang.Short |
| BOOL | java.lang.Boolean |
| BINARY, NCHAR | java.lang.String |
| BOOL | java.lang.Boolean |
| BINARY, NCHAR | java.lang.String |
## 如何获取 TAOS-JDBCDriver
......@@ -579,13 +588,34 @@ data = c1.fetchall()
numOfRows = c1.rowcount
numOfCols = len(c1.description)
for irow in range(numOfRows):
print("Row%d: ts=%s, temperature=%d, humidity=%f" %(irow, data[irow][0], data[irow][1],data[irow][2])
print("Row%d: ts=%s, temperature=%d, humidity=%f" %(irow, data[irow][0], data[irow][1],data[irow][2]))
# 直接使用cursor 循环拉取查询结果
c1.execute('select * from tb')
for data in c1:
print("ts=%s, temperature=%d, humidity=%f" %(data[0], data[1],data[2])
```
* 创建订阅
```python
# 创建一个主题为 'test' 消费周期为1000毫秒的订阅
# 第一个参数为 True 表示重新开始订阅，如为 False 且之前创建过主题为 'test' 的订阅，则表示继续消费此订阅的数据，而不是重新开始消费所有数据
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
```
* 消费订阅的数据
```python
data = sub.consume()
for d in data:
print(d)
```
* 取消订阅
```python
sub.close()
```
* 关闭连接
```python
c1.close()
......@@ -807,6 +837,8 @@ HTTP请求URL采用`sqlutc`时，返回结果集的时间戳将采用UTC时间
## Go Connector
### linux环境
#### 安装TDengine
Go的连接器使用到了 libtaos.so 和taos.h，因此，在使用Go连接器之前，需要在程序运行的机器上安装TDengine以获得相关的驱动文件。
......@@ -867,7 +899,14 @@ taosSql驱动包内采用cgo模式，调用了TDengine的C/C++同步接口，与
3. 创建表、写入和查询数据
在创建好了数据库后，就可以开始创建表和写入查询数据了。这些操作的基本思路都是首先组装SQL语句，然后调用db.Exec执行，并检查错误信息和执行相应的处理。可以参考上面的样例代码
在创建好了数据库后，就可以开始创建表和写入查询数据了。这些操作的基本思路都是首先组装SQL语句，然后调用db.Exec执行，并检查错误信息和执行相应的处理。可以参考上面的样例代码。
### windows环境
在windows上使用Go，请参考 
[TDengine GO windows驱动的编译和使用](https://www.taosdata.com/blog/2020/01/06/tdengine-go-windows%E9%A9%B1%E5%8A%A8%E7%9A%84%E7%BC%96%E8%AF%91/)
## Node.js Connector
......@@ -1054,6 +1093,8 @@ https://gitee.com/maikebing/Maikebing.EntityFrameworkCore.Taos
├───├── jdbc
├───└── python
├── driver
├───├── libtaos.dll
├───├── libtaos.dll.a
├───├── taos.dll
├───├── taos.exp
├───└── taos.lib
......@@ -1078,8 +1119,8 @@ https://gitee.com/maikebing/Maikebing.EntityFrameworkCore.Taos
+ Client可执行文件: C:/TDengine/taos.exe
+ 配置文件: C:/TDengine/cfg/taos.cfg
+ C驱动程序目录: C:/TDengine/driver
+ C驱动程序头文件: C:/TDengine/include
+ 驱动程序目录: C:/TDengine/driver
+ 驱动程序头文件: C:/TDengine/include
+ JDBC驱动程序目录: C:/TDengine/connector/jdbc
+ GO驱动程序目录：C:/TDengine/connector/go
+ Python驱动程序目录：C:/TDengine/connector/python
......@@ -1106,6 +1147,14 @@ taos -h <ServerIP>
TDengine在Window系统上提供的API与Linux系统是相同的， 应用程序使用时，需要包含TDengine头文件taos.h，连接时需要链接TDengine库taos.lib，运行时将taos.dll放到可执行文件目录下。
#### Go接口注意事项
TDengine在Window系统上提供的API与Linux系统是相同的， 应用程序使用时，除了需要Go的驱动包（C:\TDengine\connector\go）外，还需要包含TDengine头文件taos.h，连接时需要链接TDengine库libtaos.dll、libtaos.dll.a（C:\TDengine\driver），运行时将libtaos.dll、libtaos.dll.a放到可执行文件目录下。
使用参考请见：
[TDengine GO windows驱动的编译和使用](https://www.taosdata.com/blog/2020/01/06/tdengine-go-windows%E9%A9%B1%E5%8A%A8%E7%9A%84%E7%BC%96%E8%AF%91/)
#### JDBC接口注意事项
在Windows系统上，应用程序可以使用JDBC接口来操纵数据库，使用JDBC接口的注意事项如下：
......@@ -1121,6 +1170,49 @@ TDengine在Window系统上提供的API与Linux系统是相同的， 应用程序
+ 将Windows开发包(taos.dll)放置到system32目录下。
## Mac客户端及程序接口
### 客户端安装
在Mac操作系统下，TDengine提供64位的Mac客户端([2月10日起提供下载](https://www.taosdata.com/cn/all-downloads/#tdengine_mac-list))，客户端安装程序为.tar.gz文件，解压并运行其中的install_client.sh后即可完成安装，安装路径为/usr/loca/taos。客户端目录结构如下：
```
├── cfg
├───└── taos.cfg
├── connector
├───├── go
├───├── grafana
├───├── jdbc
├───└── python
├── driver
├───├── libtaos.1.6.5.1.dylib
├── examples
├───├── bash
├───├── c
├───├── C#
├───├── go
├───├── JDBC
├───├── lua
├───├── matlab
├───├── nodejs
├───├── python
├───├── R
├───└── rust
├── include
├───└── taos.h
└── bin
├───└── taos
```
其中，最常用的文件列出如下：
+ Client可执行文件: /usr/local/taos/bin/taos 软连接到 /usr/local/bin/taos
+ 配置文件: /usr/local/taos/cfg/taos.cfg 软连接到 /etc/taos/taos.cfg
+ 驱动程序目录: /usr/local/taos/driver/libtaos.1.6.5.1.dylib 软连接到 /usr/local/lib/libtaos.dylib
+ 驱动程序头文件: /usr/local/taos/include/taos.h 软连接到 /usr/local/include/taos.h
+ 日志目录（第一次运行程序时生成）：~/TDengineLog
[1]: https://search.maven.org/artifact/com.taosdata.jdbc/taos-jdbcdriver
[2]: https://mvnrepository.com/artifact/com.taosdata.jdbc/taos-jdbcdriver
......@@ -1135,3 +1227,5 @@ TDengine在Window系统上提供的API与Linux系统是相同的， 应用程序
[11]: https://github.com/taosdata/TDengine/tree/develop/tests/examples/JDBC/SpringJdbcTemplate
[12]: https://github.com/taosdata/TDengine/tree/develop/tests/examples/JDBC/springbootdemo
[13]: https://www.taosdata.com/cn/documentation/administrator/#%E5%AE%A2%E6%88%B7%E7%AB%AF%E9%85%8D%E7%BD%AE
[14]: https://www.taosdata.com/cn/documentation/connector/#Windows%E5%AE%A2%E6%88%B7%E7%AB%AF%E5%8F%8A%E7%A8%8B%E5%BA%8F%E6%8E%A5%E5%8F%A3
[15]: https://www.taosdata.com/cn/getting-started/#%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B
packaging/deb/makedeb.sh
浏览文件 @ 69a5dcda
......@@ -2,7 +2,7 @@
#
# Generate deb package for ubuntu
set -e
#set -x
# set -x
#curr_dir=$(pwd)
compile_dir=$1
......
packaging/release.sh
浏览文件 @ 69a5dcda
......@@ -3,7 +3,7 @@
# Generate the deb package for ubunt, or rpm package for centos, or tar.gz package for other linux os
set -e
# set -x
#set -x
# releash.sh -v [cluster | lite] -c [aarch32 | aarch64 | x64 | x86 | mips64 ...] -o [Linux | Kylin | Alpine | Raspberrypi | Darwin | Windows | ...] -V [stable | beta]
......@@ -46,8 +46,17 @@ done
echo "verMode=${verMode} verType=${verType} cpuType=${cpuType} osType=${osType}"
curr_dir=$(pwd)
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/..)"
if [ "$osType" != "Darwin" ]; then
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/..)"
else
script_dir=`dirname $0`
cd ${script_dir}
script_dir="$(pwd)"
top_dir=${script_dir}/..
fi
versioninfo="${top_dir}/src/util/src/version.c"
csudo=""
......@@ -147,7 +156,14 @@ build_time=$(date +"%F %R")
echo "char version[64] = \"${version}\";" > ${versioninfo}
echo "char compatible_version[64] = \"${compatible_version}\";" >> ${versioninfo}
echo "char gitinfo[128] = \"$(git rev-parse --verify HEAD)\";" >> ${versioninfo}
echo "char gitinfoOfInternal[128] = \"\";" >> ${versioninfo}
if [ "$verMode" != "cluster" ]; then
echo "char gitinfoOfInternal[128] = \"\";" >> ${versioninfo}
else
enterprise_dir="${top_dir}/../enterprise"
cd ${enterprise_dir}
echo "char gitinfoOfInternal[128] = \"$(git rev-parse --verify HEAD)\";" >> ${versioninfo}
cd ${curr_dir}
fi
echo "char buildinfo[512] = \"Built by ${USER} at ${build_time}\";" >> ${versioninfo}
echo "" >> ${versioninfo}
tmp_version=$(echo $version | tr -s "." "_")
......@@ -167,15 +183,23 @@ if [ -d ${compile_dir} ]; then
${csudo} rm -rf ${compile_dir}
fi
${csudo} mkdir -p ${compile_dir}
if [ "$osType" != "Darwin" ]; then
${csudo} mkdir -p ${compile_dir}
else
mkdir -p ${compile_dir}
fi
cd ${compile_dir}
# check support cpu type
if [[ "$cpuType" == "x64" ]] || [[ "$cpuType" == "aarch64" ]] || [[ "$cpuType" == "aarch32" ]] || [[ "$cpuType" == "mips64" ]] ; then
cmake ../ -DCPUTYPE=${cpuType}
if [ "$verMode" != "cluster" ]; then
cmake ../ -DCPUTYPE=${cpuType}
else
cmake ../../ -DCPUTYPE=${cpuType}
fi
else
echo "input cpuType=${cpuType} error!!!"
exit 1
echo "input cpuType=${cpuType} error!!!"
exit 1
fi
make
......@@ -187,28 +211,36 @@ cd ${curr_dir}
#osinfo=$(cat /etc/os-release | grep "NAME" | cut -d '"' -f2)
#echo "osinfo: ${osinfo}"
echo "====do deb package for the ubuntu system===="
output_dir="${top_dir}/debs"
if [ -d ${output_dir} ]; then
${csudo} rm -rf ${output_dir}
fi
${csudo} mkdir -p ${output_dir}
cd ${script_dir}/deb
${csudo} ./makedeb.sh ${compile_dir} ${output_dir} ${version} ${cpuType} ${osType} ${verMode} ${verType}
echo "====do rpm package for the centos system===="
output_dir="${top_dir}/rpms"
if [ -d ${output_dir} ]; then
${csudo} rm -rf ${output_dir}
if [ "$osType" != "Darwin" ]; then
if [ "$verMode" != "cluster" ]; then
echo "====do deb package for the ubuntu system===="
output_dir="${top_dir}/debs"
if [ -d ${output_dir} ]; then
${csudo} rm -rf ${output_dir}
fi
${csudo} mkdir -p ${output_dir}
cd ${script_dir}/deb
${csudo} ./makedeb.sh ${compile_dir} ${output_dir} ${version} ${cpuType} ${osType} ${verMode} ${verType}
echo "====do rpm package for the centos system===="
output_dir="${top_dir}/rpms"
if [ -d ${output_dir} ]; then
${csudo} rm -rf ${output_dir}
fi
${csudo} mkdir -p ${output_dir}
cd ${script_dir}/rpm
${csudo} ./makerpm.sh ${compile_dir} ${output_dir} ${version} ${cpuType} ${osType} ${verMode} ${verType}
fi
echo "====do tar.gz package for all systems===="
cd ${script_dir}/tools
${csudo} ./makepkg.sh ${compile_dir} ${version} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType}
${csudo} ./makeclient.sh ${compile_dir} ${version} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType}
else
cd ${script_dir}/tools
./makeclient.sh ${compile_dir} ${version} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType}
fi
${csudo} mkdir -p ${output_dir}
cd ${script_dir}/rpm
${csudo} ./makerpm.sh ${compile_dir} ${output_dir} ${version} ${cpuType} ${osType} ${verMode} ${verType}
echo "====do tar.gz package for all systems===="
cd ${script_dir}/tools
${csudo} ./makepkg.sh ${compile_dir} ${version} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType}
${csudo} ./makeclient.sh ${compile_dir} ${version} "${build_time}" ${cpuType} ${osType} ${verMode} ${verType}
# 4. Clean up temporary compile directories
#${csudo} rm -rf ${compile_dir}
......
packaging/rpm/makerpm.sh
浏览文件 @ 69a5dcda
......@@ -2,8 +2,8 @@
#
# Generate rpm package for centos
#set -e
#set -x
set -e
# set -x
#curr_dir=$(pwd)
compile_dir=$1
......
packaging/rpm/taosd
浏览文件 @ 69a5dcda
......@@ -26,7 +26,7 @@ MAX_OPEN_FILES=65535
# Default program options
NAME=taosd
PROG=/usr/local/bin/taos/taosd
PROG=/usr/local/taos/bin/taosd
USER=root
GROUP=root
......
packaging/tools/install.sh 100755 → 100644
浏览文件 @ 69a5dcda
......@@ -6,6 +6,8 @@
set -e
#set -x
verMode=lite
# -----------------------Variables definition---------------------
script_dir=$(dirname $(readlink -f "$0"))
# Dynamic directory
......@@ -27,7 +29,12 @@ install_main_dir="/usr/local/taos"
# old bin dir
bin_dir="/usr/local/taos/bin"
# v1.5 jar dir
v15_java_app_dir="/usr/local/lib/taos"
service_config_dir="/etc/systemd/system"
nginx_port=6060
nginx_dir="/usr/local/nginxd"
# Color setting
RED='\033[0;31m'
......@@ -41,6 +48,8 @@ if command -v sudo > /dev/null; then
csudo="sudo"
fi
update_flag=0
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
......@@ -69,23 +78,24 @@ osinfo=$(cat /etc/os-release | grep "NAME" | cut -d '"' -f2)
#echo "osinfo: ${osinfo}"
os_type=0
if echo $osinfo | grep -qwi "ubuntu" ; then
echo "this is ubuntu system"
echo "This is ubuntu system"
os_type=1
elif echo $osinfo | grep -qwi "debian" ; then
echo "this is debian system"
echo "This is debian system"
os_type=1
elif echo $osinfo | grep -qwi "Kylin" ; then
echo "this is Kylin system"
echo "This is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "centos" ; then
echo "this is centos system"
echo "This is centos system"
os_type=2
elif echo $osinfo | grep -qwi "fedora" ; then
echo "this is fedora system"
echo "This is fedora system"
os_type=2
else
echo "this is other linux system"
os_type=0
echo "${osinfo}: This is an officially unverified linux system, If there are any problems with the installation and operation, "
echo "please feel free to contact taosdata.com for support."
os_type=1
fi
function kill_taosd() {
......@@ -106,6 +116,9 @@ function install_main_path() {
${csudo} mkdir -p ${install_main_dir}/examples
${csudo} mkdir -p ${install_main_dir}/include
${csudo} mkdir -p ${install_main_dir}/init.d
if [ "$verMode" == "cluster" ]; then
${csudo} mkdir -p ${nginx_dir}
fi
}
function install_bin() {
......@@ -124,16 +137,30 @@ function install_bin() {
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo} ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || :
if [ "$verMode" == "cluster" ]; then
${csudo} cp -r ${script_dir}/nginxd/* ${nginx_dir} && ${csudo} chmod 0555 ${nginx_dir}/*
${csudo} mkdir -p ${nginx_dir}/logs
${csudo} chmod 777 ${nginx_dir}/sbin/nginx
fi
}
function install_lib() {
# Remove links
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -rf ${v15_java_app_dir} || :
${csudo} cp -rf ${script_dir}/driver/* ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo} ln -s ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
if [ "$verMode" == "cluster" ]; then
# Compatible with version 1.5
${csudo} mkdir -p ${v15_java_app_dir}
${csudo} ln -s ${install_main_dir}/connector/taos-jdbcdriver-1.0.2-dist.jar ${v15_java_app_dir}/JDBCDriver-1.0.2-dist.jar
${csudo} chmod 777 ${v15_java_app_dir} || :
fi
}
function install_header() {
......@@ -154,6 +181,57 @@ function install_config() {
${csudo} cp -f ${script_dir}/cfg/taos.cfg ${install_main_dir}/cfg/taos.cfg.org
${csudo} ln -s ${cfg_install_dir}/taos.cfg ${install_main_dir}/cfg
if [ "$verMode" == "cluster" ]; then
[ ! -z $1 ] && return 0 || : # only install client
if ((${update_flag}==1)); then
return 0
fi
IP_FORMAT="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
IP_PATTERN="\b$IP_FORMAT\.$IP_FORMAT\.$IP_FORMAT\.$IP_FORMAT\b"
echo
echo -e -n "${GREEN}Enter the IP address of an existing TDengine cluster node to join${NC} OR ${GREEN}leave it blank to build one${NC} :"
read masterIp
while true; do
if [ ! -z "$masterIp" ]; then
# check the format of the masterIp
if [[ $masterIp =~ $IP_PATTERN ]]; then
# Write the first IP to configuration file
sudo sed -i -r "s/#*\s*(masterIp\s*).*/\1$masterIp/" ${cfg_dir}/taos.cfg
# Get the second IP address
echo
echo -e -n "${GREEN}Enter the IP address of another node in cluster${NC} OR ${GREEN}leave it blank to skip${NC}: "
read secondIp
while true; do
if [ ! -z "$secondIp" ]; then
if [[ $secondIp =~ $IP_PATTERN ]]; then
# Write the second IP to configuration file
sudo sed -i -r "s/#*\s*(secondIp\s*).*/\1$secondIp/" ${cfg_dir}/taos.cfg
break
else
read -p "Please enter the correct IP address: " secondIp
fi
else
break
fi
done
break
else
read -p "Please enter the correct IP address: " masterIp
fi
else
break
fi
done
fi
}
......@@ -175,7 +253,9 @@ function install_connector() {
}
function install_examples() {
${csudo} cp -rf ${script_dir}/examples/* ${install_main_dir}/examples
if [ -d ${script_dir}/examples ]; then
${csudo} cp -rf ${script_dir}/examples/* ${install_main_dir}/examples
fi
}
function clean_service_on_sysvinit() {
......@@ -240,7 +320,19 @@ function clean_service_on_systemd() {
${csudo} systemctl disable taosd &> /dev/null || echo &> /dev/null
${csudo} rm -f ${taosd_service_config}
}
if [ "$verMode" == "cluster" ]; then
nginx_service_config="${service_config_dir}/nginxd.service"
if systemctl is-active --quiet nginxd; then
echo "Nginx for TDengine is running, stopping it..."
${csudo} systemctl stop nginxd &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable nginxd &> /dev/null || echo &> /dev/null
${csudo} rm -f ${nginx_service_config}
fi
}
# taos:2345:respawn:/etc/init.d/taosd start
......@@ -269,6 +361,36 @@ function install_service_on_systemd() {
${csudo} bash -c "echo '[Install]' >> ${taosd_service_config}"
${csudo} bash -c "echo 'WantedBy=multi-user.target' >> ${taosd_service_config}"
${csudo} systemctl enable taosd
if [ "$verMode" == "cluster" ]; then
nginx_service_config="${service_config_dir}/nginxd.service"
${csudo} bash -c "echo '[Unit]' >> ${nginx_service_config}"
${csudo} bash -c "echo 'Description=Nginx For TDengine Service' >> ${nginx_service_config}"
${csudo} bash -c "echo 'After=network-online.target' >> ${nginx_service_config}"
${csudo} bash -c "echo 'Wants=network-online.target' >> ${nginx_service_config}"
${csudo} bash -c "echo >> ${nginx_service_config}"
${csudo} bash -c "echo '[Service]' >> ${nginx_service_config}"
${csudo} bash -c "echo 'Type=forking' >> ${nginx_service_config}"
${csudo} bash -c "echo 'PIDFile=/usr/local/nginxd/logs/nginx.pid' >> ${nginx_service_config}"
${csudo} bash -c "echo 'ExecStart=/usr/local/nginxd/sbin/nginx' >> ${nginx_service_config}"
${csudo} bash -c "echo 'ExecStop=/usr/local/nginxd/sbin/nginx -s stop' >> ${nginx_service_config}"
${csudo} bash -c "echo 'LimitNOFILE=infinity' >> ${nginx_service_config}"
${csudo} bash -c "echo 'LimitNPROC=infinity' >> ${nginx_service_config}"
${csudo} bash -c "echo 'LimitCORE=infinity' >> ${nginx_service_config}"
${csudo} bash -c "echo 'TimeoutStartSec=0' >> ${nginx_service_config}"
${csudo} bash -c "echo 'StandardOutput=null' >> ${nginx_service_config}"
${csudo} bash -c "echo 'Restart=always' >> ${nginx_service_config}"
${csudo} bash -c "echo 'StartLimitBurst=3' >> ${nginx_service_config}"
${csudo} bash -c "echo 'StartLimitInterval=60s' >> ${nginx_service_config}"
${csudo} bash -c "echo >> ${nginx_service_config}"
${csudo} bash -c "echo '[Install]' >> ${nginx_service_config}"
${csudo} bash -c "echo 'WantedBy=multi-user.target' >> ${nginx_service_config}"
if ! ${csudo} systemctl enable nginxd &> /dev/null; then
${csudo} systemctl daemon-reexec
${csudo} systemctl enable nginxd
fi
${csudo} systemctl start nginxd
fi
}
function install_service() {
......@@ -363,6 +485,21 @@ function update_TDengine() {
install_bin
install_service
install_config
if [ "$verMode" == "cluster" ]; then
# Check if openresty is installed
openresty_work=false
# Check if nginx is installed successfully
if type curl &> /dev/null; then
if curl -sSf http://127.0.0.1:${nginx_port} &> /dev/null; then
echo -e "\033[44;32;1mNginx for TDengine is updated successfully!${NC}"
openresty_work=true
else
echo -e "\033[44;31;5mNginx for TDengine does not work! Please try again!\033[0m"
fi
fi
fi
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
......@@ -376,7 +513,15 @@ function update_TDengine() {
echo -e "${GREEN_DARK}To start TDengine ${NC}: ./taosd${NC}"
fi
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
if [ "$verMode" == "cluster" ]; then
if [ ${openresty_work} = 'true' ]; then
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell OR from ${GREEN_UNDERLINE}http://127.0.0.1:${nginx_port}${NC}"
else
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
fi
else
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
fi
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
else
......@@ -416,6 +561,20 @@ function install_TDengine() {
# For installing new
install_bin
install_service
if [ "$verMode" == "cluster" ]; then
openresty_work=false
# Check if nginx is installed successfully
if type curl &> /dev/null; then
if curl -sSf http://127.0.0.1:${nginx_port} &> /dev/null; then
echo -e "\033[44;32;1mNginx for TDengine is installed successfully!${NC}"
openresty_work=true
else
echo -e "\033[44;31;5mNginx for TDengine does not work! Please try again!\033[0m"
fi
fi
fi
install_config
# Ask if to start the service
......@@ -430,8 +589,17 @@ function install_TDengine() {
else
echo -e "${GREEN_DARK}To start TDengine ${NC}: taosd${NC}"
fi
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
if [ "$verMode" == "cluster" ]; then
if [ ${openresty_work} = 'true' ]; then
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell OR from ${GREEN_UNDERLINE}http://127.0.0.1:${nginx_port}${NC}"
else
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
fi
else
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
fi
echo
echo -e "\033[44;32;1mTDengine is installed successfully!${NC}"
else # Only install client
......@@ -450,6 +618,7 @@ function install_TDengine() {
if [ -z $1 ]; then
# Install server and client
if [ -x ${bin_dir}/taosd ]; then
update_flag=1
update_TDengine
else
install_TDengine
......@@ -457,6 +626,7 @@ if [ -z $1 ]; then
else
# Only install client
if [ -x ${bin_dir}/taos ]; then
update_flag=1
update_TDengine client
else
install_TDengine client
......
packaging/tools/install_client.sh
浏览文件 @ 69a5dcda
......@@ -7,18 +7,35 @@ set -e
#set -x
# -----------------------Variables definition---------------------
script_dir=$(dirname $(readlink -f "$0"))
# Dynamic directory
data_dir="/var/lib/taos"
log_dir="/var/log/taos"
osType=Linux
if [ "$osType" != "Darwin" ]; then
script_dir=$(dirname $(readlink -f "$0"))
# Dynamic directory
data_dir="/var/lib/taos"
log_dir="/var/log/taos"
else
script_dir=`dirname $0`
cd ${script_dir}
script_dir="$(pwd)"
data_dir="/var/lib/taos"
log_dir="~/TDengineLog"
fi
log_link_dir="/usr/local/taos/log"
cfg_install_dir="/etc/taos"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
if [ "$osType" != "Darwin" ]; then
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
else
bin_link_dir="/usr/local/bin"
lib_link_dir="/usr/local/lib"
inc_link_dir="/usr/local/include"
fi
#install main path
install_main_dir="/usr/local/taos"
......@@ -26,6 +43,8 @@ install_main_dir="/usr/local/taos"
# old bin dir
bin_dir="/usr/local/taos/bin"
# v1.5 jar dir
v15_java_app_dir="/usr/local/lib/taos"
# Color setting
RED='\033[0;31m'
......@@ -51,9 +70,9 @@ function kill_client() {
function install_main_path() {
#create install main dir and all sub dir
${csudo} rm -rf ${install_main_dir} || :
${csudo} mkdir -p ${install_main_dir}
${csudo} mkdir -p ${install_main_dir}
${csudo} mkdir -p ${install_main_dir}/cfg
${csudo} mkdir -p ${install_main_dir}/bin
${csudo} mkdir -p ${install_main_dir}/bin
${csudo} mkdir -p ${install_main_dir}/connector
${csudo} mkdir -p ${install_main_dir}/driver
${csudo} mkdir -p ${install_main_dir}/examples
......@@ -61,51 +80,60 @@ function install_main_path() {
}
function install_bin() {
# Remove links
${csudo} rm -f ${bin_link_dir}/taos || :
${csudo} rm -f ${bin_link_dir}/taosdump || :
${csudo} rm -f ${bin_link_dir}/rmtaos || :
# Remove links
${csudo} rm -f ${bin_link_dir}/taos || :
if [ "$osType" == "Darwin" ]; then
${csudo} rm -f ${bin_link_dir}/taosdump || :
fi
${csudo} rm -f ${bin_link_dir}/rmtaos || :
${csudo} cp -r ${script_dir}/bin/* ${install_main_dir}/bin && ${csudo} chmod 0555 ${install_main_dir}/bin/*
${csudo} cp -r ${script_dir}/bin/* ${install_main_dir}/bin && ${csudo} chmod 0555 ${install_main_dir}/bin/*
#Make link
[ -x ${install_main_dir}/bin/taos ] && ${csudo} ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/remove_client.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_client.sh ${bin_link_dir}/rmtaos || :
[ -x ${install_main_dir}/bin/taos ] && ${csudo} ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || :
if [ "$osType" == "Darwin" ]; then
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
fi
[ -x ${install_main_dir}/bin/remove_client.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_client.sh ${bin_link_dir}/rmtaos || :
}
function clean_lib() {
sudo rm -f /usr/lib/libtaos.so || :
sudo rm -f /usr/lib/libtaos.* || :
sudo rm -rf ${lib_dir} || :
}
function install_lib() {
# Remove links
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -rf ${v15_java_app_dir} || :
${csudo} cp -rf ${script_dir}/driver/* ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} cp -rf ${script_dir}/driver/* ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo} ln -s ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
if [ "$osType" != "Darwin" ]; then
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.so.1
${csudo} ln -s ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
else
${csudo} ln -s ${install_main_dir}/driver/libtaos.* ${lib_link_dir}/libtaos.1.dylib
${csudo} ln -s ${lib_link_dir}/libtaos.1.dylib ${lib_link_dir}/libtaos.dylib
fi
}
function install_header() {
${csudo} rm -f ${inc_link_dir}/taos.h ${inc_link_dir}/taoserror.h || :
${csudo} cp -f ${script_dir}/inc/* ${install_main_dir}/include && ${csudo} chmod 644 ${install_main_dir}/include/*
${csudo} cp -f ${script_dir}/inc/* ${install_main_dir}/include && ${csudo} chmod 644 ${install_main_dir}/include/*
${csudo} ln -s ${install_main_dir}/include/taos.h ${inc_link_dir}/taos.h
${csudo} ln -s ${install_main_dir}/include/taoserror.h ${inc_link_dir}/taoserror.h
}
function install_config() {
#${csudo} rm -f ${install_main_dir}/cfg/taos.cfg || :
if [ ! -f ${cfg_install_dir}/taos.cfg ]; then
${csudo} mkdir -p ${cfg_install_dir}
[ -f ${script_dir}/cfg/taos.cfg ] && ${csudo} cp ${script_dir}/cfg/taos.cfg ${cfg_install_dir}
${csudo} chmod 644 ${cfg_install_dir}/*
fi
fi
${csudo} cp -f ${script_dir}/cfg/taos.cfg ${install_main_dir}/cfg/taos.cfg.org
${csudo} ln -s ${cfg_install_dir}/taos.cfg ${install_main_dir}/cfg
}
......@@ -113,8 +141,12 @@ function install_config() {
function install_log() {
${csudo} rm -rf ${log_dir} || :
${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
if [ "$osType" != "Darwin" ]; then
${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
else
mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
fi
${csudo} ln -s ${log_dir} ${install_main_dir}/log
}
......@@ -142,7 +174,7 @@ function update_TDengine() {
kill_client
sleep 1
fi
install_main_path
install_log
......@@ -152,7 +184,7 @@ function update_TDengine() {
install_examples
install_bin
install_config
echo
echo -e "\033[44;32;1mTDengine client is updated successfully!${NC}"
......@@ -168,16 +200,16 @@ function install_TDengine() {
tar -zxf taos.tar.gz
echo -e "${GREEN}Start to install TDengine client...${NC}"
install_main_path
install_log
install_main_path
install_log
install_header
install_lib
install_connector
install_examples
install_bin
install_config
echo
echo -e "\033[44;32;1mTDengine client is installed successfully!${NC}"
......@@ -191,8 +223,8 @@ function install_TDengine() {
if [ -e ${bin_dir}/taosd ]; then
echo -e "\033[44;32;1mThere are already installed TDengine server, so don't need install client!${NC}"
exit 0
fi
fi
if [ -x ${bin_dir}/taos ]; then
update_flag=1
update_TDengine
......
packaging/tools/make_install.sh
浏览文件 @ 69a5dcda
......@@ -9,19 +9,37 @@ set -e
# -----------------------Variables definition---------------------
source_dir=$1
binary_dir=$2
script_dir=$(dirname $(readlink -f "$0"))
osType=$3
if [ "$osType" != "Darwin" ]; then
script_dir=$(dirname $(readlink -f "$0"))
else
script_dir=${source_dir}/packaging/tools
fi
# Dynamic directory
data_dir="/var/lib/taos"
log_dir="/var/log/taos"
if [ "$osType" != "Darwin" ]; then
log_dir="/var/log/taos"
else
log_dir="~/TDengineLog"
fi
data_link_dir="/usr/local/taos/data"
log_link_dir="/usr/local/taos/log"
cfg_install_dir="/etc/taos"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
if [ "$osType" != "Darwin" ]; then
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
else
bin_link_dir="/usr/local/bin"
lib_link_dir="/usr/local/lib"
inc_link_dir="/usr/local/include"
fi
#install main path
install_main_dir="/usr/local/taos"
......@@ -43,58 +61,61 @@ if command -v sudo > /dev/null; then
csudo="sudo"
fi
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
service_mod=0
elif $(which service &> /dev/null); then
service_mod=1
service_config_dir="/etc/init.d"
if $(which chkconfig &> /dev/null); then
initd_mod=1
elif $(which insserv &> /dev/null); then
initd_mod=2
elif $(which update-rc.d &> /dev/null); then
initd_mod=3
if [ "$osType" != "Darwin" ]; then
initd_mod=0
service_mod=2
if pidof systemd &> /dev/null; then
service_mod=0
elif $(which service &> /dev/null); then
service_mod=1
service_config_dir="/etc/init.d"
if $(which chkconfig &> /dev/null); then
initd_mod=1
elif $(which insserv &> /dev/null); then
initd_mod=2
elif $(which update-rc.d &> /dev/null); then
initd_mod=3
else
service_mod=2
fi
else
service_mod=2
fi
else
service_mod=2
fi
# get the operating system type for using the corresponding init file
# ubuntu/debian(deb), centos/fedora(rpm), others: opensuse, redhat, ..., no verification
#osinfo=$(awk -F= '/^NAME/{print $2}' /etc/os-release)
osinfo=$(cat /etc/os-release | grep "NAME" | cut -d '"' -f2)
#echo "osinfo: ${osinfo}"
os_type=0
if echo $osinfo | grep -qwi "ubuntu" ; then
echo "this is ubuntu system"
os_type=1
elif echo $osinfo | grep -qwi "debian" ; then
echo "this is debian system"
os_type=1
elif echo $osinfo | grep -qwi "Kylin" ; then
echo "this is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "centos" ; then
echo "this is centos system"
os_type=2
elif echo $osinfo | grep -qwi "fedora" ; then
echo "this is fedora system"
os_type=2
else
echo "this is other linux system"
os_type=0
# get the operating system type for using the corresponding init file
# ubuntu/debian(deb), centos/fedora(rpm), others: opensuse, redhat, ..., no verification
#osinfo=$(awk -F= '/^NAME/{print $2}' /etc/os-release)
osinfo=$(cat /etc/os-release | grep "NAME" | cut -d '"' -f2)
#echo "osinfo: ${osinfo}"
os_type=0
if echo $osinfo | grep -qwi "ubuntu" ; then
echo "this is ubuntu system"
os_type=1
elif echo $osinfo | grep -qwi "debian" ; then
echo "this is debian system"
os_type=1
elif echo $osinfo | grep -qwi "Kylin" ; then
echo "this is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "centos" ; then
echo "this is centos system"
os_type=2
elif echo $osinfo | grep -qwi "fedora" ; then
echo "this is fedora system"
os_type=2
else
echo "${osinfo}: This is an officially unverified linux system, If there are any problems with the installation and operation, "
echo "please feel free to contact taosdata.com for support."
os_type=1
fi
fi
function kill_taosd() {
pid=$(ps -ef | grep "taosd" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
pid=$(ps -ef | grep "taosd" | grep -v "grep" | awk '{print $2}')
if [ -n "$pid" ]; then
${csudo} kill -9 $pid || :
fi
}
function install_main_path() {
......@@ -107,37 +128,62 @@ function install_main_path() {
${csudo} mkdir -p ${install_main_dir}/driver
${csudo} mkdir -p ${install_main_dir}/examples
${csudo} mkdir -p ${install_main_dir}/include
${csudo} mkdir -p ${install_main_dir}/init.d
if [ "$osType" != "Darwin" ]; then
${csudo} mkdir -p ${install_main_dir}/init.d
fi
}
function install_bin() {
# Remove links
${csudo} rm -f ${bin_link_dir}/taos || :
${csudo} rm -f ${bin_link_dir}/taosd || :
${csudo} rm -f ${bin_link_dir}/taosdemo || :
${csudo} rm -f ${bin_link_dir}/taosdump || :
${csudo} rm -f ${bin_link_dir}/rmtaos || :
${csudo} cp -r ${binary_dir}/build/bin/* ${install_main_dir}/bin
${csudo} cp -r ${script_dir}/remove.sh ${install_main_dir}/bin
${csudo} rm -f ${bin_link_dir}/taos || :
if [ "$osType" != "Darwin" ]; then
${csudo} rm -f ${bin_link_dir}/taosd || :
${csudo} rm -f ${bin_link_dir}/taosdemo || :
${csudo} rm -f ${bin_link_dir}/taosdump || :
fi
${csudo} rm -f ${bin_link_dir}/rmtaos || :
${csudo} cp -r ${binary_dir}/build/bin/* ${install_main_dir}/bin
if [ "$osType" != "Darwin" ]; then
${csudo} cp -r ${script_dir}/remove.sh ${install_main_dir}/bin
else
${csudo} cp -r ${script_dir}/remove_client.sh ${install_main_dir}/bin
fi
${csudo} chmod 0555 ${install_main_dir}/bin/*
#Make link
[ -x ${install_main_dir}/bin/taos ] && ${csudo} ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || :
[ -x ${install_main_dir}/bin/taosd ] && ${csudo} ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo} ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || :
if [ "$osType" != "Darwin" ]; then
[ -x ${install_main_dir}/bin/taosd ] && ${csudo} ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo} ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo} ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || :
fi
if [ "$osType" != "Darwin" ]; then
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || :
else
[ -x ${install_main_dir}/bin/remove_client.sh ] && ${csudo} ln -s ${install_main_dir}/bin/remove_client.sh ${bin_link_dir}/rmtaos || :
fi
}
function install_lib() {
# Remove links
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
versioninfo=$(${script_dir}/get_version.sh ${source_dir}/src/util/src/version.c)
${csudo} cp ${binary_dir}/build/lib/libtaos.so.${versioninfo} ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} ln -sf ${install_main_dir}/driver/libtaos.so.${versioninfo} ${lib_link_dir}/libtaos.so.1
${csudo} ln -sf ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
if [ "$osType" != "Darwin" ]; then
${csudo} cp ${binary_dir}/build/lib/libtaos.so.${versioninfo} ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} ln -sf ${install_main_dir}/driver/libtaos.so.${versioninfo} ${lib_link_dir}/libtaos.so.1
${csudo} ln -sf ${lib_link_dir}/libtaos.so.1 ${lib_link_dir}/libtaos.so
else
${csudo} cp ${binary_dir}/build/lib/libtaos.${versioninfo}.dylib ${install_main_dir}/driver && ${csudo} chmod 777 ${install_main_dir}/driver/*
${csudo} ln -sf ${install_main_dir}/driver/libtaos.${versioninfo}.dylib ${lib_link_dir}/libtaos.1.dylib
${csudo} ln -sf ${lib_link_dir}/libtaos.1.dylib ${lib_link_dir}/libtaos.dylib
fi
}
function install_header() {
......@@ -163,8 +209,13 @@ function install_config() {
function install_log() {
${csudo} rm -rf ${log_dir} || :
${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
if [ "$osType" != "Darwin" ]; then
${csudo} mkdir -p ${log_dir} && ${csudo} chmod 777 ${log_dir}
else
mkdir -p ${log_dir} && chmod 777 ${log_dir}
fi
${csudo} ln -s ${log_dir} ${install_main_dir}/log
}
......@@ -291,7 +342,9 @@ function install_service() {
function update_TDengine() {
echo -e "${GREEN}Start to update TDEngine...${NC}"
# Stop the service if running
if pidof taosd &> /dev/null; then
if [ "$osType" != "Darwin" ]; then
if pidof taosd &> /dev/null; then
if ((${service_mod}==0)); then
${csudo} systemctl stop taosd || :
elif ((${service_mod}==1)); then
......@@ -300,6 +353,7 @@ function update_TDengine() {
kill_taosd
fi
sleep 1
fi
fi
install_main_path
......@@ -310,32 +364,54 @@ function update_TDengine() {
install_connector
install_examples
install_bin
install_service
if [ "$osType" != "Darwin" ]; then
install_service
fi
install_config
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
echo
echo -e "${GREEN_DARK}To configure TDengine ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} systemctl start taosd${NC}"
elif ((${service_mod}==1)); then
if [ "$osType" != "Darwin" ]; then
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
echo
echo -e "${GREEN_DARK}To configure TDengine ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} systemctl start taosd${NC}"
elif ((${service_mod}==1)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} service taosd start${NC}"
else
echo -e "${GREEN_DARK}To start TDengine ${NC}: ./taosd${NC}"
fi
else
echo -e "${GREEN_DARK}To start TDengine ${NC}: ./taosd${NC}"
fi
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
else
echo
echo -e "\033[44;32;1mTDengine Client is updated successfully!${NC}"
echo
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine is updated successfully!${NC}"
echo -e "${GREEN_DARK}To access TDengine Client ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine Client is updated successfully!${NC}"
fi
}
function install_TDengine() {
# Start to install
echo -e "${GREEN}Start to install TDEngine...${NC}"
if [ "$osType" != "Darwin" ]; then
echo -e "${GREEN}Start to install TDEngine...${NC}"
else
echo -e "${GREEN}Start to install TDEngine Client ...${NC}"
fi
install_main_path
install_data
if [ "$osType" != "Darwin" ]; then
install_data
fi
install_log
install_header
install_lib
......@@ -343,30 +419,41 @@ function install_TDengine() {
install_examples
install_bin
install_service
if [ "$osType" != "Darwin" ]; then
install_service
fi
install_config
# Ask if to start the service
echo
echo -e "\033[44;32;1mTDengine is installed successfully!${NC}"
echo
echo -e "${GREEN_DARK}To configure TDengine ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} systemctl start taosd${NC}"
elif ((${service_mod}==1)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} service taosd start${NC}"
if [ "$osType" != "Darwin" ]; then
# Ask if to start the service
echo
echo -e "\033[44;32;1mTDengine is installed successfully!${NC}"
echo
echo -e "${GREEN_DARK}To configure TDengine ${NC}: edit /etc/taos/taos.cfg"
if ((${service_mod}==0)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} systemctl start taosd${NC}"
elif ((${service_mod}==1)); then
echo -e "${GREEN_DARK}To start TDengine ${NC}: ${csudo} service taosd start${NC}"
else
echo -e "${GREEN_DARK}To start TDengine ${NC}: ./taosd${NC}"
fi
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine is installed successfully!${NC}"
else
echo -e "${GREEN_DARK}To start TDengine ${NC}: ./taosd${NC}"
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine Client is installed successfully!${NC}"
fi
echo -e "${GREEN_DARK}To access TDengine ${NC}: use ${GREEN_UNDERLINE}taos${NC} in shell${NC}"
echo
echo -e "\033[44;32;1mTDengine is installed successfully!${NC}"
}
## ==============================Main program starts from here============================
echo source directory: $1
echo binary directory: $2
if [ -x ${bin_dir}/taosd ]; then
if [ -x ${bin_dir}/taos ]; then
update_TDengine
else
install_TDengine
......
packaging/tools/makeclient.sh
浏览文件 @ 69a5dcda
......@@ -13,8 +13,15 @@ osType=$5
verMode=$6
verType=$7
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/../..)"
if [ "$osType" != "Darwin" ]; then
script_dir="$(dirname $(readlink -f $0))"
top_dir="$(readlink -f ${script_dir}/../..)"
else
script_dir=`dirname $0`
cd ${script_dir}
script_dir="$(pwd)"
top_dir=${script_dir}/../..
fi
# create compressed install file.
build_dir="${compile_dir}/build"
......@@ -22,13 +29,26 @@ code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
install_dir="${release_dir}/TDengine-client"
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/TDengine-enterprise-client"
else
install_dir="${release_dir}/TDengine-client"
fi
# Directories and files.
bin_files="${build_dir}/bin/taos ${build_dir}/bin/taosdump ${script_dir}/remove_client.sh"
lib_files="${build_dir}/lib/libtaos.so.${version}"
if [ "$osType" != "Darwin" ]; then
bin_files="${build_dir}/bin/taos ${build_dir}/bin/taosdump ${script_dir}/remove_client.sh"
lib_files="${build_dir}/lib/libtaos.so.${version}"
else
bin_files="${build_dir}/bin/taos ${script_dir}/remove_client.sh"
lib_files="${build_dir}/lib/libtaos.${version}.dylib"
fi
header_files="${code_dir}/inc/taos.h ${code_dir}/inc/taoserror.h"
cfg_dir="${top_dir}/packaging/cfg"
install_files="${script_dir}/install_client.sh"
# make directories.
......@@ -38,10 +58,23 @@ mkdir -p ${install_dir}/cfg && cp ${cfg_dir}/taos.cfg ${install_dir}/cfg/taos.cf
mkdir -p ${install_dir}/bin && cp ${bin_files} ${install_dir}/bin && chmod a+x ${install_dir}/bin/*
cd ${install_dir}
tar -zcv -f taos.tar.gz * --remove-files || :
if [ "$osType" != "Darwin" ]; then
tar -zcv -f taos.tar.gz * --remove-files || :
else
tar -zcv -f taos.tar.gz * || :
mv taos.tar.gz ..
rm -rf ./*
mv ../taos.tar.gz .
fi
cd ${curr_dir}
cp ${install_files} ${install_dir} && chmod a+x ${install_dir}/install*
cp ${install_files} ${install_dir}
if [ "$osType" == "Darwin" ]; then
sed 's/osType=Linux/osType=Darwin/g' ${install_dir}/install_client.sh >> install_client_temp.sh
mv install_client_temp.sh ${install_dir}/install_client.sh
fi
chmod a+x ${install_dir}/install_client.sh
# Copy example code
mkdir -p ${install_dir}/examples
......@@ -60,7 +93,10 @@ cp ${lib_files} ${install_dir}/driver
# Copy connector
connector_dir="${code_dir}/connector"
mkdir -p ${install_dir}/connector
cp ${build_dir}/lib/*.jar ${install_dir}/connector
if [ "$osType" != "Darwin" ]; then
cp ${build_dir}/lib/*.jar ${install_dir}/connector
fi
cp -r ${connector_dir}/grafana ${install_dir}/connector/
cp -r ${connector_dir}/python ${install_dir}/connector/
cp -r ${connector_dir}/go ${install_dir}/connector
......@@ -90,6 +126,13 @@ else
exit 1
fi
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files
if [ "$osType" != "Darwin" ]; then
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
else
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) || :
mv "$(basename ${pkg_name}).tar.gz" ..
rm -rf ./*
mv ../"$(basename ${pkg_name}).tar.gz" .
fi
cd ${curr_dir}
packaging/tools/makepkg.sh
浏览文件 @ 69a5dcda
......@@ -23,7 +23,11 @@ code_dir="${top_dir}/src"
release_dir="${top_dir}/release"
#package_name='linux'
install_dir="${release_dir}/TDengine-server"
if [ "$verMode" == "cluster" ]; then
install_dir="${release_dir}/TDengine-enterprise-server"
else
install_dir="${release_dir}/TDengine-server"
fi
# Directories and files.
bin_files="${build_dir}/bin/taosd ${build_dir}/bin/taos ${build_dir}/bin/taosdemo ${build_dir}/bin/taosdump ${script_dir}/remove.sh"
......@@ -31,6 +35,7 @@ lib_files="${build_dir}/lib/libtaos.so.${version}"
header_files="${code_dir}/inc/taos.h ${code_dir}/inc/taoserror.h"
cfg_dir="${top_dir}/packaging/cfg"
install_files="${script_dir}/install.sh"
nginx_dir="${code_dir}/../../enterprise/src/modules/web"
# Init file
#init_dir=${script_dir}/deb
......@@ -50,11 +55,29 @@ mkdir -p ${install_dir}/bin && cp ${bin_files} ${install_dir}/bin && chmod a+x $
mkdir -p ${install_dir}/init.d && cp ${init_file_deb} ${install_dir}/init.d/taosd.deb
mkdir -p ${install_dir}/init.d && cp ${init_file_rpm} ${install_dir}/init.d/taosd.rpm
if [ "$verMode" == "cluster" ]; then
mkdir -p ${install_dir}/nginxd && cp -r ${nginx_dir}/* ${install_dir}/nginxd
cp ${nginx_dir}/png/taos.png ${install_dir}/nginxd/admin/images/taos.png
rm -rf ${install_dir}/nginxd/png
if [ "$cpuType" == "aarch64" ]; then
cp -f ${install_dir}/nginxd/sbin/arm/64bit/nginx ${install_dir}/nginxd/sbin/
elif [ "$cpuType" == "aarch32" ]; then
cp -f ${install_dir}/nginxd/sbin/arm/32bit/nginx ${install_dir}/nginxd/sbin/
fi
rm -rf ${install_dir}/nginxd/sbin/arm
fi
cd ${install_dir}
tar -zcv -f taos.tar.gz * --remove-files || :
tar -zcv -f taos.tar.gz * --remove-files || :
cd ${curr_dir}
cp ${install_files} ${install_dir} && chmod a+x ${install_dir}/install*
cp ${install_files} ${install_dir}
if [ "$verMode" == "cluster" ]; then
sed 's/verMode=lite/verMode=cluster/g' ${install_dir}/install.sh >> install_temp.sh
mv install_temp.sh ${install_dir}/install.sh
fi
chmod a+x ${install_dir}/install.sh
# Copy example code
mkdir -p ${install_dir}/examples
......@@ -103,6 +126,6 @@ else
exit 1
fi
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
cd ${curr_dir}
cd ${curr_dir}
\ No newline at end of file
packaging/tools/remove.sh
浏览文件 @ 69a5dcda
......@@ -2,6 +2,11 @@
#
# Script to stop the service and uninstall TDengine, but retain the config, data and log files.
set -e
#set -x
verMode=lite
RED='\033[0;31m'
GREEN='\033[1;32m'
NC='\033[0m'
......@@ -14,10 +19,14 @@ cfg_link_dir="/usr/local/taos/cfg"
bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
install_nginxd_dir="/usr/local/nginxd"
# v1.5 jar dir
v15_java_app_dir="/usr/local/lib/taos"
service_config_dir="/etc/systemd/system"
taos_service_name="taosd"
nginx_service_name="nginxd"
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
......@@ -62,6 +71,7 @@ function clean_bin() {
function clean_lib() {
# Remove link
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -rf ${v15_java_app_dir} || :
}
function clean_header() {
......@@ -90,6 +100,20 @@ function clean_service_on_systemd() {
${csudo} systemctl disable ${taos_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${taosd_service_config}
if [ "$verMode" == "cluster" ]; then
nginx_service_config="${service_config_dir}/${nginx_service_name}.service"
if [ -d ${bin_dir}/web ]; then
if systemctl is-active --quiet ${nginx_service_name}; then
echo "Nginx for TDengine is running, stopping it..."
${csudo} systemctl stop ${nginx_service_name} &> /dev/null || echo &> /dev/null
fi
${csudo} systemctl disable ${nginx_service_name} &> /dev/null || echo &> /dev/null
${csudo} rm -f ${nginx_service_config}
fi
fi
}
function clean_service_on_sysvinit() {
......@@ -143,6 +167,7 @@ clean_config
${csudo} rm -rf ${data_link_dir} || :
${csudo} rm -rf ${install_main_dir}
${csudo} rm -rf ${install_nginxd_dir}
osinfo=$(awk -F= '/^NAME/{print $2}' /etc/os-release)
if echo $osinfo | grep -qwi "ubuntu" ; then
......
packaging/tools/remove_client.sh
浏览文件 @ 69a5dcda
......@@ -17,6 +17,10 @@ bin_link_dir="/usr/bin"
lib_link_dir="/usr/lib"
inc_link_dir="/usr/include"
# v1.5 jar dir
v15_java_app_dir="/usr/local/lib/taos"
csudo=""
if command -v sudo > /dev/null; then
csudo="sudo"
......@@ -39,6 +43,7 @@ function clean_bin() {
function clean_lib() {
# Remove link
${csudo} rm -f ${lib_link_dir}/libtaos.* || :
${csudo} rm -rf ${v15_java_app_dir} || :
}
function clean_header() {
......
src/client/CMakeLists.txt
浏览文件 @ 69a5dcda
......@@ -24,20 +24,10 @@ IF ((TD_LINUX_64) OR (TD_LINUX_32 AND TD_ARM))
#set version of .so
#VERSION so version
#SOVERSION api version
IF (TD_LITE)
execute_process(COMMAND chmod 777 ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh)
execute_process(COMMAND ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh ${TD_COMMUNITY_DIR}/src/util/src/version.c
OUTPUT_VARIABLE
VERSION_INFO)
MESSAGE(STATUS "build lite version ${VERSION_INFO}")
ELSE ()
execute_process(COMMAND chmod 777 ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh)
execute_process(COMMAND ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh ${TD_COMMUNITY_DIR}/src/util/src/version.c
OUTPUT_VARIABLE
VERSION_INFO)
MESSAGE(STATUS "build cluster version ${VERSION_INFO}")
ENDIF ()
execute_process(COMMAND chmod 777 ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh)
execute_process(COMMAND ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh ${TD_COMMUNITY_DIR}/src/util/src/version.c
OUTPUT_VARIABLE
VERSION_INFO)
MESSAGE(STATUS "build version ${VERSION_INFO}")
SET_TARGET_PROPERTIES(taos PROPERTIES VERSION ${VERSION_INFO} SOVERSION 1)
......@@ -57,6 +47,7 @@ ELSEIF (TD_WINDOWS_64)
TARGET_LINK_LIBRARIES(taos trpc)
ELSEIF (TD_DARWIN_64)
SET(CMAKE_MACOSX_RPATH 1)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/jni/linux)
ADD_LIBRARY(taos_static STATIC ${SRC})
......@@ -66,6 +57,17 @@ ELSEIF (TD_DARWIN_64)
# generate dynamic library (*.dylib)
ADD_LIBRARY(taos SHARED ${SRC})
TARGET_LINK_LIBRARIES(taos trpc tutil pthread m)
SET_TARGET_PROPERTIES(taos PROPERTIES CLEAN_DIRECT_OUTPUT 1)
#set version of .so
#VERSION so version
#SOVERSION api version
execute_process(COMMAND chmod 777 ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh)
execute_process(COMMAND ${TD_COMMUNITY_DIR}/packaging/tools/get_version.sh ${TD_COMMUNITY_DIR}/src/util/src/version.c
OUTPUT_VARIABLE
VERSION_INFO)
MESSAGE(STATUS "build version ${VERSION_INFO}")
SET_TARGET_PROPERTIES(taos PROPERTIES VERSION ${VERSION_INFO} SOVERSION 1)
ENDIF ()
src/client/inc/tsclient.h
浏览文件 @ 69a5dcda
......@@ -336,6 +336,7 @@ typedef struct {
int rspType;
int rspLen;
uint64_t qhandle;
int64_t uid;
int64_t useconds;
int64_t offset; // offset value from vnode during projection query of stable
int row;
......@@ -382,6 +383,7 @@ typedef struct _sql_obj {
uint32_t queryId;
void * thandle;
void * pStream;
void * pSubscription;
char * sqlstr;
char retry;
char maxRetry;
......@@ -465,6 +467,12 @@ void tscDestroyResPointerInfo(SSqlRes *pRes);
void tscFreeSqlCmdData(SSqlCmd *pCmd);
/**
* free query result of the sql object
* @param pObj
*/
void tscFreeSqlResult(SSqlObj* pSql);
/**
* only free part of resources allocated during query.
* Note: this function is multi-thread safe.
......
src/client/jni/com_taosdata_jdbc_TSDBJNIConnector.h
浏览文件 @ 69a5dcda
......@@ -135,7 +135,7 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeConnectionIm
* Signature: (Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;JI)J
*/
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp
(JNIEnv *, jobject, jstring, jstring, jstring, jstring, jstring, jlong, jint);
(JNIEnv *, jobject, jlong, jboolean, jstring, jstring, jint);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
......@@ -143,7 +143,7 @@ JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp
* Signature: (J)Lcom/taosdata/jdbc/TSDBResultSetRowData;
*/
JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp
(JNIEnv *, jobject, jlong);
(JNIEnv *, jobject, jlong, jint);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
......@@ -151,7 +151,7 @@ JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp
* Signature: (J)V
*/
JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_unsubscribeImp
(JNIEnv *, jobject, jlong);
(JNIEnv *, jobject, jlong, jboolean);
/*
* Class: com_taosdata_jdbc_TSDBJNIConnector
......
src/client/src/TSDBJNIConnector.c
浏览文件 @ 69a5dcda
......@@ -20,6 +20,7 @@
#include "tscJoinProcess.h"
#include "tsclient.h"
#include "tscUtil.h"
#include "ttime.h"
int __init = 0;
......@@ -514,92 +515,42 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_closeConnectionIm
}
}
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp(JNIEnv *env, jobject jobj, jstring jhost,
jstring juser, jstring jpass, jstring jdb,
jstring jtable, jlong jtime,
jint jperiod) {
TAOS_SUB *tsub;
jlong sub = 0;
char * host = NULL;
char * user = NULL;
char * pass = NULL;
char * db = NULL;
char * table = NULL;
int64_t time = 0;
int period = 0;
JNIEXPORT jlong JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_subscribeImp(JNIEnv *env, jobject jobj, jlong con,
jboolean restart, jstring jtopic, jstring jsql, jint jinterval) {
jlong sub = 0;
TAOS *taos = (TAOS *)con;
char *topic = NULL;
char *sql = NULL;
jniGetGlobalMethod(env);
jniTrace("jobj:%p, in TSDBJNIConnector_subscribeImp", jobj);
if (jhost != NULL) {
host = (char *)(*env)->GetStringUTFChars(env, jhost, NULL);
}
if (juser != NULL) {
user = (char *)(*env)->GetStringUTFChars(env, juser, NULL);
}
if (jpass != NULL) {
pass = (char *)(*env)->GetStringUTFChars(env, jpass, NULL);
}
if (jdb != NULL) {
db = (char *)(*env)->GetStringUTFChars(env, jdb, NULL);
}
if (jtable != NULL) {
table = (char *)(*env)->GetStringUTFChars(env, jtable, NULL);
if (jtopic != NULL) {
topic = (char *)(*env)->GetStringUTFChars(env, jtopic, NULL);
}
time = (int64_t)jtime;
period = (int)jperiod;
if (user == NULL) {
jniTrace("jobj:%p, user is null, use tsDefaultUser", jobj);
user = tsDefaultUser;
}
if (pass == NULL) {
jniTrace("jobj:%p, pass is null, use tsDefaultPass", jobj);
pass = tsDefaultPass;
if (jsql != NULL) {
sql = (char *)(*env)->GetStringUTFChars(env, jsql, NULL);
}
jniTrace("jobj:%p, host:%s, user:%s, pass:%s, db:%s, table:%s, time:%d, period:%d", jobj, host, user, pass, db, table,
time, period);
tsub = taos_subscribe(host, user, pass, db, table, time, period);
TAOS_SUB *tsub = taos_subscribe(taos, (int)restart, topic, sql, NULL, NULL, jinterval);
sub = (jlong)tsub;
if (sub == 0) {
jniTrace("jobj:%p, failed to subscribe to db:%s, table:%s", jobj, db, table);
jniTrace("jobj:%p, failed to subscribe: topic:%s", jobj, jtopic);
} else {
jniTrace("jobj:%p, successfully subscribe to db:%s, table:%s, sub:%ld, tsub:%p", jobj, db, table, sub, tsub);
jniTrace("jobj:%p, successfully subscribe: topic: %s", jobj, jtopic);
}
if (host != NULL) (*env)->ReleaseStringUTFChars(env, jhost, host);
if (user != NULL && user != tsDefaultUser) (*env)->ReleaseStringUTFChars(env, juser, user);
if (pass != NULL && pass != tsDefaultPass) (*env)->ReleaseStringUTFChars(env, jpass, pass);
if (db != NULL) (*env)->ReleaseStringUTFChars(env, jdb, db);
if (table != NULL) (*env)->ReleaseStringUTFChars(env, jtable, table);
if (topic != NULL) (*env)->ReleaseStringUTFChars(env, jtopic, topic);
if (sql != NULL) (*env)->ReleaseStringUTFChars(env, jsql, sql);
return sub;
}
JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNIEnv *env, jobject jobj, jlong sub) {
jniTrace("jobj:%p, in TSDBJNIConnector_consumeImp, sub:%ld", jobj, sub);
TAOS_SUB * tsub = (TAOS_SUB *)sub;
TAOS_ROW row = taos_consume(tsub);
TAOS_FIELD *fields = taos_fetch_subfields(tsub);
int num_fields = taos_subfields_count(tsub);
jniGetGlobalMethod(env);
jniTrace("jobj:%p, check fields:%p, num_fields=%d", jobj, fields, num_fields);
static jobject convert_one_row(JNIEnv *env, TAOS_ROW row, TAOS_FIELD* fields, int num_fields) {
jobject rowobj = (*env)->NewObject(env, g_rowdataClass, g_rowdataConstructor, num_fields);
jniTrace("created a rowdata object, rowobj:%p", rowobj);
if (row == NULL) {
jniTrace("jobj:%p, tsub:%p, fields size is %d, fetch row to the end", jobj, tsub, num_fields);
return NULL;
}
char tmp[TSDB_MAX_BYTES_PER_ROW] = {0};
for (int i = 0; i < num_fields; i++) {
if (row[i] == NULL) {
continue;
......@@ -634,6 +585,7 @@ JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNI
}
break;
case TSDB_DATA_TYPE_BINARY: {
char tmp[TSDB_MAX_BYTES_PER_ROW] = {0};
strncpy(tmp, row[i], (size_t) fields[i].bytes); // handle the case that terminated does not exist
(*env)->CallVoidMethod(env, rowobj, g_rowdataSetStringFp, i, (*env)->NewStringUTF(env, tmp));
......@@ -642,7 +594,7 @@ JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNI
}
case TSDB_DATA_TYPE_NCHAR:
(*env)->CallVoidMethod(env, rowobj, g_rowdataSetByteArrayFp, i,
jniFromNCharToByteArray(env, (char*)row[i], fields[i].bytes));
jniFromNCharToByteArray(env, (char*)row[i], fields[i].bytes));
break;
case TSDB_DATA_TYPE_TIMESTAMP:
(*env)->CallVoidMethod(env, rowobj, g_rowdataSetTimestampFp, i, (jlong) * ((int64_t *)row[i]));
......@@ -651,13 +603,56 @@ JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNI
break;
}
}
jniTrace("jobj:%p, rowdata retrieved, rowobj:%p", jobj, rowobj);
return rowobj;
}
JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_unsubscribeImp(JNIEnv *env, jobject jobj, jlong sub) {
JNIEXPORT jobject JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_consumeImp(JNIEnv *env, jobject jobj, jlong sub, jint timeout) {
jniTrace("jobj:%p, in TSDBJNIConnector_consumeImp, sub:%ld", jobj, sub);
jniGetGlobalMethod(env);
TAOS_SUB *tsub = (TAOS_SUB *)sub;
jobject rows = (*env)->NewObject(env, g_arrayListClass, g_arrayListConstructFp);
int64_t start = taosGetTimestampMs();
int count = 0;
while (true) {
TAOS_RES * res = taos_consume(tsub);
if (res == NULL) {
jniError("jobj:%p, tsub:%p, taos_consume returns NULL", jobj, tsub);
return NULL;
}
TAOS_FIELD *fields = taos_fetch_fields(res);
int num_fields = taos_num_fields(res);
while (true) {
TAOS_ROW row = taos_fetch_row(res);
if (row == NULL) {
break;
}
jobject rowobj = convert_one_row(env, row, fields, num_fields);
(*env)->CallBooleanMethod(env, rows, g_arrayListAddFp, rowobj);
count++;
}
if (count > 0) {
break;
}
if (timeout == -1) {
continue;
}
if (((int)(taosGetTimestampMs() - start)) >= timeout) {
jniTrace("jobj:%p, sub:%ld, timeout", jobj, sub);
break;
}
}
return rows;
}
JNIEXPORT void JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_unsubscribeImp(JNIEnv *env, jobject jobj, jlong sub, jboolean keepProgress) {
TAOS_SUB *tsub = (TAOS_SUB *)sub;
taos_unsubscribe(tsub);
taos_unsubscribe(tsub, keepProgress);
}
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_TSDBJNIConnector_validateCreateTableSqlImp(JNIEnv *env, jobject jobj,
......
src/client/src/tscAst.c
浏览文件 @ 69a5dcda
......@@ -833,7 +833,7 @@ void tSQLBinaryExprCalcTraverse(tSQLBinaryExpr *pExprs, int32_t numOfRows, char
tSQLSyntaxNode *pRight = pExprs->pRight;
/* the left output has result from the left child syntax tree */
char *pLeftOutput = malloc(sizeof(int64_t) * numOfRows);
char *pLeftOutput = (char*)malloc(sizeof(int64_t) * numOfRows);
if (pLeft->nodeType == TSQL_NODE_EXPR) {
tSQLBinaryExprCalcTraverse(pLeft->pExpr, numOfRows, pLeftOutput, param, order, getSourceDataBlock);
}
......
src/client/src/tscCache.c
浏览文件 @ 69a5dcda
......@@ -96,11 +96,7 @@ void *taosAddConnIntoCache(void *handle, void *data, uint32_t ip, uint16_t port,
pObj = (SConnCache *)handle;
if (pObj == NULL || pObj->maxSessions == 0) return NULL;
#ifdef CLUSTER
if (data == NULL || ip == 0) {
#else
if (data == NULL) {
#endif
tscTrace("data:%p ip:%p:%d not valid, not added in cache", data, ip, port);
return NULL;
}
......
src/client/src/tscJoinProcess.c
浏览文件 @ 69a5dcda
......@@ -793,7 +793,9 @@ STSBuf* tsBufCreate(bool autoDelete) {
return NULL;
}
allocResForTSBuf(pTSBuf);
if (NULL == allocResForTSBuf(pTSBuf)) {
return NULL;
}
// update the header info
STSBufFileHeader header = {.magic = TS_COMP_FILE_MAGIC, .numOfVnode = pTSBuf->numOfVnodes, .tsOrder = TSQL_SO_ASC};
......
src/client/src/tscProfile.c
浏览文件 @ 69a5dcda
......@@ -202,10 +202,10 @@ void tscKillStream(STscObj *pObj, uint32_t killId) {
tscTrace("%p stream:%p is killed, streamId:%d", pStream->pSql, pStream, killId);
}
taos_close_stream(pStream);
if (pStream->callback) {
pStream->callback(pStream->param);
}
taos_close_stream(pStream);
}
char *tscBuildQueryStreamDesc(char *pMsg, STscObj *pObj) {
......@@ -285,8 +285,9 @@ void tscKillConnection(STscObj *pObj) {
SSqlStream *pStream = pObj->streamList;
while (pStream) {
SSqlStream *tmp = pStream->next;
taos_close_stream(pStream);
pStream = pStream->next;
pStream = tmp;
}
pthread_mutex_unlock(&pObj->mutex);
......
src/client/src/tscSQLParser.c
浏览文件 @ 69a5dcda
......@@ -2918,7 +2918,7 @@ static SColumnFilterInfo* addColumnFilterInfo(SColumnBase* pColumn) {
}
int32_t size = pColumn->numOfFilters + 1;
char* tmp = realloc(pColumn->filterInfo, sizeof(SColumnFilterInfo) * (size));
char* tmp = (char*)realloc((void*)(pColumn->filterInfo), sizeof(SColumnFilterInfo) * (size));
if (tmp != NULL) {
pColumn->filterInfo = (SColumnFilterInfo*)tmp;
}
......
src/client/src/tscSQLParserImpl.c
浏览文件 @ 69a5dcda
......@@ -706,14 +706,14 @@ void setDCLSQLElems(SSqlInfo *pInfo, int32_t type, int32_t nParam, ...) {
pInfo->sqlType = type;
if (nParam == 0) return;
if (pInfo->pDCLInfo == NULL) pInfo->pDCLInfo = calloc(1, sizeof(tDCLSQL));
if (pInfo->pDCLInfo == NULL) pInfo->pDCLInfo = (tDCLSQL *)calloc(1, sizeof(tDCLSQL));
va_list va;
va_start(va, nParam);
while (nParam-- > 0) {
SSQLToken *pToken = va_arg(va, SSQLToken *);
tTokenListAppend(pInfo->pDCLInfo, pToken);
(void)tTokenListAppend(pInfo->pDCLInfo, pToken);
}
va_end(va);
}
......
src/client/src/tscSecondaryMerge.c
浏览文件 @ 69a5dcda
......@@ -212,7 +212,7 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
tscTrace("%p load data from disk into memory, orderOfVnode:%d, total:%d", pSqlObjAddr, i + 1, idx + 1);
tExtMemBufferLoadData(pMemBuffer[i], &(pDS->filePage), j, 0);
#ifdef _DEBUG_VIEW
printf("load data page into mem for build loser tree: %ld rows\n", pDS->filePage.numOfElems);
printf("load data page into mem for build loser tree: %" PRIu64 " rows\n", pDS->filePage.numOfElems);
SSrcColumnInfo colInfo[256] = {0};
tscGetSrcColumnInfo(colInfo, pCmd);
......@@ -342,7 +342,7 @@ static int32_t tscFlushTmpBufferImpl(tExtMemBuffer *pMemoryBuf, tOrderDescriptor
}
#ifdef _DEBUG_VIEW
printf("%ld rows data flushed to disk after been sorted:\n", pPage->numOfElems);
printf("%" PRIu64 " rows data flushed to disk after been sorted:\n", pPage->numOfElems);
tColModelDisplay(pDesc->pSchema, pPage->data, pPage->numOfElems, pPage->numOfElems);
#endif
......@@ -601,7 +601,9 @@ int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOr
rlen += pExpr->resBytes;
}
int32_t capacity = nBufferSizes / rlen;
int32_t capacity = 0;
if (0 != rlen) capacity = nBufferSizes / rlen;
pModel = tColModelCreate(pSchema, pCmd->fieldsInfo.numOfOutputCols, capacity);
for (int32_t i = 0; i < pMeterMetaInfo->pMetricMeta->numOfVnodes; ++i) {
......
src/client/src/tscServer.c
浏览文件 @ 69a5dcda
......@@ -31,33 +31,66 @@
#define TSC_MGMT_VNODE 999
#ifdef CLUSTER
SIpStrList tscMgmtIpList;
int tsMasterIndex = 0;
int tsSlaveIndex = 1;
#else
int tsMasterIndex = 0;
int tsSlaveIndex = 0; // slave == master for single node edition
uint32_t tsServerIp;
#endif
SIpStrList tscMgmtIpList;
int tsMasterIndex = 0;
int tsSlaveIndex = 1;
int (*tscBuildMsg[TSDB_SQL_MAX])(SSqlObj *pSql);
int (*tscProcessMsgRsp[TSDB_SQL_MAX])(SSqlObj *pSql);
void (*tscUpdateVnodeMsg[TSDB_SQL_MAX])(SSqlObj *pSql, char *buf);
void tscProcessActivityTimer(void *handle, void *tmrId);
int tscKeepConn[TSDB_SQL_MAX] = {0};
TSKEY tscGetSubscriptionProgress(void* sub, int64_t uid);
void tscUpdateSubscriptionProgress(void* sub, int64_t uid, TSKEY ts);
void tscSaveSubscriptionProgress(void* sub);
static int32_t minMsgSize() { return tsRpcHeadSize + sizeof(STaosDigest); }
#ifdef CLUSTER
void tscPrintMgmtIp() {
if (tscMgmtIpList.numOfIps <= 0) {
tscError("invalid IP list:%d", tscMgmtIpList.numOfIps);
tscError("invalid mgmt IP list:%d", tscMgmtIpList.numOfIps);
} else {
for (int i = 0; i < tscMgmtIpList.numOfIps; ++i) tscTrace("mgmt index:%d ip:%s", i, tscMgmtIpList.ipstr[i]);
for (int i = 0; i < tscMgmtIpList.numOfIps; ++i) {
tscTrace("mgmt index:%d ip:%s", i, tscMgmtIpList.ipstr[i]);
}
}
}
void tscSetMgmtIpListFromCluster(SIpList *pIpList) {
tscMgmtIpList.numOfIps = pIpList->numOfIps;
if (memcmp(tscMgmtIpList.ip, pIpList->ip, pIpList->numOfIps * 4) != 0) {
for (int i = 0; i < pIpList->numOfIps; ++i) {
tinet_ntoa(tscMgmtIpList.ipstr[i], pIpList->ip[i]);
tscMgmtIpList.ip[i] = pIpList->ip[i];
}
tscTrace("cluster mgmt IP list:");
tscPrintMgmtIp();
}
}
void tscSetMgmtIpListFromEdge() {
if (tscMgmtIpList.numOfIps != 2) {
tscMgmtIpList.numOfIps = 2;
strcpy(tscMgmtIpList.ipstr[0], tsMasterIp);
tscMgmtIpList.ip[0] = inet_addr(tsMasterIp);
strcpy(tscMgmtIpList.ipstr[1], tsMasterIp);
tscMgmtIpList.ip[1] = inet_addr(tsMasterIp);
tscTrace("edge mgmt IP list:");
tscPrintMgmtIp();
}
}
void tscSetMgmtIpList(SIpList *pIpList) {
/*
* The iplist returned by the cluster edition is the current management nodes
* and the iplist returned by the edge edition is empty
*/
if (pIpList->numOfIps != 0) {
tscSetMgmtIpListFromCluster(pIpList);
} else {
tscSetMgmtIpListFromEdge();
}
}
#endif
/*
* For each management node, try twice at least in case of poor network situation.
......@@ -68,11 +101,7 @@ void tscPrintMgmtIp() {
*/
static int32_t tscGetMgmtConnMaxRetryTimes() {
int32_t factor = 2;
#ifdef CLUSTER
return tscMgmtIpList.numOfIps * factor;
#else
return 1*factor;
#endif
}
void tscProcessHeartBeatRsp(void *param, TAOS_RES *tres, int code) {
......@@ -88,18 +117,9 @@ void tscProcessHeartBeatRsp(void *param, TAOS_RES *tres, int code) {
if (code == 0) {
SHeartBeatRsp *pRsp = (SHeartBeatRsp *)pRes->pRsp;
#ifdef CLUSTER
SIpList * pIpList = &pRsp->ipList;
tscMgmtIpList.numOfIps = pIpList->numOfIps;
if (memcmp(tscMgmtIpList.ip, pIpList->ip, pIpList->numOfIps * 4) != 0) {
for (int i = 0; i < pIpList->numOfIps; ++i) {
tinet_ntoa(tscMgmtIpList.ipstr[i], pIpList->ip[i]);
tscMgmtIpList.ip[i] = pIpList->ip[i];
}
tscTrace("new mgmt IP list:");
tscPrintMgmtIp();
}
#endif
tscSetMgmtIpList(pIpList);
if (pRsp->killConnection) {
tscKillConnection(pObj);
} else {
......@@ -152,19 +172,12 @@ void tscProcessActivityTimer(void *handle, void *tmrId) {
void tscGetConnToMgmt(SSqlObj *pSql, uint8_t *pCode) {
STscObj *pTscObj = pSql->pTscObj;
#ifdef CLUSTER
if (pSql->retry < tscGetMgmtConnMaxRetryTimes()) {
*pCode = 0;
pSql->retry++;
pSql->index = pSql->index % tscMgmtIpList.numOfIps;
if (pSql->cmd.command > TSDB_SQL_READ && pSql->index == 0) pSql->index = 1;
void *thandle = taosGetConnFromCache(tscConnCache, tscMgmtIpList.ip[pSql->index], TSC_MGMT_VNODE, pTscObj->user);
#else
if (pSql->retry < tscGetMgmtConnMaxRetryTimes()) {
*pCode = 0;
pSql->retry++;
void *thandle = taosGetConnFromCache(tscConnCache, tsServerIp, TSC_MGMT_VNODE, pTscObj->user);
#endif
if (thandle == NULL) {
SRpcConnInit connInit;
......@@ -180,24 +193,15 @@ void tscGetConnToMgmt(SSqlObj *pSql, uint8_t *pCode) {
connInit.encrypt = 0;
connInit.secret = pSql->pTscObj->pass;
#ifdef CLUSTER
connInit.peerIp = tscMgmtIpList.ipstr[pSql->index];
#else
connInit.peerIp = tsMasterIp;
#endif
thandle = taosOpenRpcConn(&connInit, pCode);
}
pSql->thandle = thandle;
#ifdef CLUSTER
pSql->ip = tscMgmtIpList.ip[pSql->index];
pSql->vnode = TSC_MGMT_VNODE;
tscTrace("%p mgmt index:%d ip:0x%x is picked up, pConn:%p", pSql, pSql->index, tscMgmtIpList.ip[pSql->index],
pSql->thandle);
#else
pSql->ip = tsServerIp;
pSql->vnode = TSC_MGMT_VNODE;
#endif
}
// the pSql->res.code is the previous error(status) code.
......@@ -242,11 +246,16 @@ void tscGetConnToVnode(SSqlObj *pSql, uint8_t *pCode) {
while (pSql->retry < pSql->maxRetry) {
(pSql->retry)++;
#ifdef CLUSTER
char ipstr[40] = {0};
if (pVPeersDesc[pSql->index].ip == 0) {
(pSql->index) = (pSql->index + 1) % TSDB_VNODES_SUPPORT;
continue;
/*
* in the edge edition, ip is 0, and at this time we use masterIp instead
* in the cluster edition, ip is vnode ip
*/
//(pSql->index) = (pSql->index + 1) % TSDB_VNODES_SUPPORT;
//continue;
pVPeersDesc[pSql->index].ip = tscMgmtIpList.ip[0];
}
*pCode = TSDB_CODE_SUCCESS;
......@@ -276,31 +285,6 @@ void tscGetConnToVnode(SSqlObj *pSql, uint8_t *pCode) {
pSql->vnode = pVPeersDesc[pSql->index].vnode;
tscTrace("%p vnode:%d ip:%p index:%d is picked up, pConn:%p", pSql, pVPeersDesc[pSql->index].vnode,
pVPeersDesc[pSql->index].ip, pSql->index, pSql->thandle);
#else
*pCode = 0;
void *thandle = taosGetConnFromCache(tscConnCache, tsServerIp, pVPeersDesc[0].vnode, pTscObj->user);
if (thandle == NULL) {
SRpcConnInit connInit;
memset(&connInit, 0, sizeof(connInit));
connInit.cid = vidIndex;
connInit.sid = 0;
connInit.spi = 0;
connInit.encrypt = 0;
connInit.meterId = pSql->pTscObj->user;
connInit.peerId = htonl((pVPeersDesc[0].vnode << TSDB_SHELL_VNODE_BITS));
connInit.shandle = pVnodeConn;
connInit.ahandle = pSql;
connInit.peerIp = tsMasterIp;
connInit.peerPort = tsVnodeShellPort;
thandle = taosOpenRpcConn(&connInit, pCode);
vidIndex = (vidIndex + 1) % tscNumOfThreads;
}
pSql->thandle = thandle;
pSql->ip = tsServerIp;
pSql->vnode = pVPeersDesc[0].vnode;
#endif
break;
}
......@@ -367,15 +351,9 @@ int tscSendMsgToServer(SSqlObj *pSql) {
return code;
}
#ifdef CLUSTER
void tscProcessMgmtRedirect(SSqlObj *pSql, uint8_t *cont) {
SIpList *pIpList = (SIpList *)(cont);
tscMgmtIpList.numOfIps = pIpList->numOfIps;
for (int i = 0; i < pIpList->numOfIps; ++i) {
tinet_ntoa(tscMgmtIpList.ipstr[i], pIpList->ip[i]);
tscMgmtIpList.ip[i] = pIpList->ip[i];
tscTrace("Update mgmt Ip, index:%d ip:%s", i, tscMgmtIpList.ipstr[i]);
}
tscSetMgmtIpList(pIpList);
if (pSql->cmd.command < TSDB_SQL_READ) {
tsMasterIndex = 0;
......@@ -386,7 +364,6 @@ void tscProcessMgmtRedirect(SSqlObj *pSql, uint8_t *cont) {
tscPrintMgmtIp();
}
#endif
void *tscProcessMsgFromServer(char *msg, void *ahandle, void *thandle) {
if (ahandle == NULL) return NULL;
......@@ -420,13 +397,9 @@ void *tscProcessMsgFromServer(char *msg, void *ahandle, void *thandle) {
SMeterMetaInfo *pMeterMetaInfo = tscGetMeterMetaInfo(pCmd, 0);
if (msg == NULL) {
tscTrace("%p no response from ip:0x%x", pSql, pSql->ip);
#ifdef CLUSTER
tscTrace("%p no response from ip:%s", pSql, taosIpStr(pSql->ip));
pSql->index++;
#else
// for single node situation, do NOT try next index
#endif
pSql->thandle = NULL;
// todo taos_stop_query() in async model
/*
......@@ -442,12 +415,7 @@ void *tscProcessMsgFromServer(char *msg, void *ahandle, void *thandle) {
// renew meter meta in case it is changed
if (pCmd->command < TSDB_SQL_FETCH && pRes->code != TSDB_CODE_QUERY_CANCELLED) {
#ifdef CLUSTER
pSql->maxRetry = TSDB_VNODES_SUPPORT * 2;
#else
// for fetch, it shall not renew meter meta
pSql->maxRetry = 2;
#endif
code = tscRenewMeterMeta(pSql, pMeterMetaInfo->name);
pRes->code = code;
if (code == TSDB_CODE_ACTION_IN_PROGRESS) return pSql;
......@@ -460,8 +428,6 @@ void *tscProcessMsgFromServer(char *msg, void *ahandle, void *thandle) {
} else {
uint16_t rspCode = pMsg->content[0];
#ifdef CLUSTER
if (rspCode == TSDB_CODE_REDIRECT) {
tscTrace("%p it shall be redirected!", pSql);
taosAddConnIntoCache(tscConnCache, thandle, pSql->ip, pSql->vnode, pObj->user);
......@@ -493,12 +459,7 @@ void *tscProcessMsgFromServer(char *msg, void *ahandle, void *thandle) {
* removed. So, renew metermeta and try again.
* not_active_session: db has been move to other node, the vnode does not exist on this dnode anymore.
*/
#else
if (rspCode == TSDB_CODE_NOT_ACTIVE_TABLE || rspCode == TSDB_CODE_INVALID_TABLE_ID ||
rspCode == TSDB_CODE_NOT_ACTIVE_VNODE || rspCode == TSDB_CODE_INVALID_VNODE_ID ||
rspCode == TSDB_CODE_TABLE_ID_MISMATCH || rspCode == TSDB_CODE_NETWORK_UNAVAIL) {
#endif
pSql->thandle = NULL;
pSql->thandle = NULL;
taosAddConnIntoCache(tscConnCache, thandle, pSql->ip, pSql->vnode, pObj->user);
if (pCmd->command == TSDB_SQL_CONNECT) {
......@@ -767,12 +728,8 @@ int tscProcessSql(SSqlObj *pSql) {
tscTrace("%p SQL cmd:%d will be processed, name:%s, type:%d", pSql, pSql->cmd.command, name, pSql->cmd.type);
pSql->retry = 0;
if (pSql->cmd.command < TSDB_SQL_MGMT) {
#ifdef CLUSTER
pSql->maxRetry = TSDB_VNODES_SUPPORT;
#else
pSql->maxRetry = 2;
#endif
// the pMeterMetaInfo cannot be NULL
if (pMeterMetaInfo == NULL) {
pSql->res.code = TSDB_CODE_OTHERS;
......@@ -820,10 +777,10 @@ int tscProcessSql(SSqlObj *pSql) {
}
}
sem_post(&pSql->emptyRspSem);
sem_wait(&pSql->rspSem);
tsem_post(&pSql->emptyRspSem);
tsem_wait(&pSql->rspSem);
sem_post(&pSql->emptyRspSem);
tsem_post(&pSql->emptyRspSem);
if (pSql->numOfSubs <= 0) {
pSql->cmd.command = TSDB_SQL_RETRIEVE_EMPTY_RESULT;
......@@ -856,9 +813,9 @@ int tscProcessSql(SSqlObj *pSql) {
}
if (fp == NULL) {
sem_post(&pSql->emptyRspSem);
sem_wait(&pSql->rspSem);
sem_post(&pSql->emptyRspSem);
tsem_post(&pSql->emptyRspSem);
tsem_wait(&pSql->rspSem);
tsem_post(&pSql->emptyRspSem);
// set the command flag must be after the semaphore been correctly set.
pSql->cmd.command = TSDB_SQL_RETRIEVE_METRIC;
......@@ -1211,7 +1168,7 @@ void tscRetrieveFromVnodeCallBack(void *param, TAOS_RES *tres, int numOfRows) {
tColModelCompact(pDesc->pSchema, trsupport->localBuffer, pDesc->pSchema->maxCapacity);
#ifdef _DEBUG_VIEW
printf("%ld rows data flushed to disk:\n", trsupport->localBuffer->numOfElems);
printf("%" PRIu64 " rows data flushed to disk:\n", trsupport->localBuffer->numOfElems);
SSrcColumnInfo colInfo[256] = {0};
tscGetSrcColumnInfo(colInfo, &pPObj->cmd);
tColModelDisplayEx(pDesc->pSchema, trsupport->localBuffer->data, trsupport->localBuffer->numOfElems,
......@@ -1387,7 +1344,7 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) {
SSqlObj *pNew = tscCreateSqlObjForSubquery(trsupport->pParentSqlObj, trsupport, pSql);
if (pNew == NULL) {
tscError("%p sub:%p failed to create new subquery due to out of memory, abort retry, vid:%d, orderOfSub:%d",
trsupport->pParentSqlObj, pSql, pSvd->vnode, trsupport->subqueryIndex);
trsupport->pParentSqlObj, pSql, pSvd != NULL ? pSvd->vnode : -1, trsupport->subqueryIndex);
pState->code = -TSDB_CODE_CLI_OUT_OF_MEMORY;
trsupport->numOfRetry = MAX_NUM_OF_SUBQUERY_RETRY;
......@@ -1411,9 +1368,14 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) {
tscRetrieveFromVnodeCallBack(param, tres, pState->code);
} else { // success, proceed to retrieve data from dnode
tscTrace("%p sub:%p query complete,ip:%u,vid:%d,orderOfSub:%d,retrieve data", trsupport->pParentSqlObj, pSql,
if (vnodeInfo != NULL) {
tscTrace("%p sub:%p query complete,ip:%u,vid:%d,orderOfSub:%d,retrieve data", trsupport->pParentSqlObj, pSql,
vnodeInfo->vpeerDesc[vnodeInfo->index].ip, vnodeInfo->vpeerDesc[vnodeInfo->index].vnode,
trsupport->subqueryIndex);
} else {
tscTrace("%p sub:%p query complete, orderOfSub:%d,retrieve data", trsupport->pParentSqlObj, pSql,
trsupport->subqueryIndex);
}
taos_fetch_rows_a(tres, tscRetrieveFromVnodeCallBack, param);
}
......@@ -1533,12 +1495,12 @@ static char* doSerializeTableInfo(SSqlObj* pSql, int32_t numOfMeters, int32_t vn
tscTrace("%p vid:%d, query on %d meters", pSql, htons(vnodeId), numOfMeters);
if (UTIL_METER_IS_NOMRAL_METER(pMeterMetaInfo)) {
#ifdef _DEBUG_VIEW
tscTrace("%p sid:%d, uid:%lld", pSql, pMeterMetaInfo->pMeterMeta->sid, pMeterMetaInfo->pMeterMeta->uid);
tscTrace("%p sid:%d, uid:%" PRIu64, pSql, pMeterMetaInfo->pMeterMeta->sid, pMeterMetaInfo->pMeterMeta->uid);
#endif
SMeterSidExtInfo *pMeterInfo = (SMeterSidExtInfo *)pMsg;
pMeterInfo->sid = htonl(pMeterMeta->sid);
pMeterInfo->uid = htobe64(pMeterMeta->uid);
pMeterInfo->key = htobe64(tscGetSubscriptionProgress(pSql->pSubscription, pMeterMeta->uid));
pMsg += sizeof(SMeterSidExtInfo);
} else {
SVnodeSidList *pVnodeSidList = tscGetVnodeSidList(pMetricMeta, pMeterMetaInfo->vnodeIndex);
......@@ -1549,6 +1511,7 @@ static char* doSerializeTableInfo(SSqlObj* pSql, int32_t numOfMeters, int32_t vn
pMeterInfo->sid = htonl(pQueryMeterInfo->sid);
pMeterInfo->uid = htobe64(pQueryMeterInfo->uid);
pMeterInfo->key = htobe64(tscGetSubscriptionProgress(pSql->pSubscription, pQueryMeterInfo->uid));
pMsg += sizeof(SMeterSidExtInfo);
......@@ -1556,7 +1519,7 @@ static char* doSerializeTableInfo(SSqlObj* pSql, int32_t numOfMeters, int32_t vn
pMsg += pMetricMeta->tagLen;
#ifdef _DEBUG_VIEW
tscTrace("%p sid:%d, uid:%lld", pSql, pQueryMeterInfo->sid, pQueryMeterInfo->uid);
tscTrace("%p sid:%d, uid:%" PRId64, pSql, pQueryMeterInfo->sid, pQueryMeterInfo->uid);
#endif
}
}
......@@ -1648,7 +1611,7 @@ int tscBuildQueryMsg(SSqlObj *pSql) {
pQueryMsg->nAggTimeInterval = htobe64(pCmd->nAggTimeInterval);
pQueryMsg->intervalTimeUnit = pCmd->intervalTimeUnit;
if (pCmd->nAggTimeInterval < 0) {
tscError("%p illegal value of aggregation time interval in query msg: %ld", pSql, pCmd->nAggTimeInterval);
tscError("%p illegal value of aggregation time interval in query msg: %" PRId64, pSql, pCmd->nAggTimeInterval);
return -1;
}
......@@ -3430,20 +3393,11 @@ int tscProcessConnectRsp(SSqlObj *pSql) {
assert(len <= tListLen(pObj->db));
strncpy(pObj->db, temp, tListLen(pObj->db));
#ifdef CLUSTER
SIpList * pIpList;
char *rsp = pRes->pRsp + sizeof(SConnectRsp);
pIpList = (SIpList *)rsp;
tscMgmtIpList.numOfIps = pIpList->numOfIps;
for (int i = 0; i < pIpList->numOfIps; ++i) {
tinet_ntoa(tscMgmtIpList.ipstr[i], pIpList->ip[i]);
tscMgmtIpList.ip[i] = pIpList->ip[i];
}
rsp += sizeof(SIpList) + sizeof(int32_t) * pIpList->numOfIps;
tscSetMgmtIpList(pIpList);
tscPrintMgmtIp();
#endif
strcpy(pObj->sversion, pConnect->version);
pObj->writeAuth = pConnect->writeAuth;
pObj->superAuth = pConnect->superAuth;
......@@ -3542,11 +3496,27 @@ int tscProcessRetrieveRspFromVnode(SSqlObj *pSql) {
pRes->numOfRows = htonl(pRetrieve->numOfRows);
pRes->precision = htons(pRetrieve->precision);
pRes->offset = htobe64(pRetrieve->offset);
pRes->useconds = htobe64(pRetrieve->useconds);
pRes->data = pRetrieve->data;
tscSetResultPointer(pCmd, pRes);
if (pSql->pSubscription != NULL) {
TAOS_FIELD *pField = tscFieldInfoGetField(pCmd, pCmd->fieldsInfo.numOfOutputCols - 1);
int16_t offset = tscFieldInfoGetOffset(pCmd, pCmd->fieldsInfo.numOfOutputCols - 1);
char* p = pRes->data + (pField->bytes + offset) * pRes->numOfRows;
int32_t numOfMeters = htonl(*(int32_t*)p);
p += sizeof(int32_t);
for (int i = 0; i < numOfMeters; i++) {
int64_t uid = htobe64(*(int64_t*)p);
p += sizeof(int64_t);
TSKEY key = htobe64(*(TSKEY*)p);
p += sizeof(TSKEY);
tscUpdateSubscriptionProgress(pSql->pSubscription, uid, key);
}
}
pRes->row = 0;
/**
......
src/client/src/tscSql.c
浏览文件 @ 69a5dcda
......@@ -63,19 +63,17 @@ TAOS *taos_connect_imp(const char *ip, const char *user, const char *pass, const
}
}
#ifdef CLUSTER
if (ip && ip[0]) {
tscMgmtIpList.numOfIps = 4;
strcpy(tscMgmtIpList.ipstr[0], ip);
tscMgmtIpList.ip[0] = inet_addr(ip);
strcpy(tscMgmtIpList.ipstr[1], ip);
tscMgmtIpList.ip[1] = inet_addr(ip);
strcpy(tscMgmtIpList.ipstr[2], tsMasterIp);
tscMgmtIpList.ip[2] = inet_addr(tsMasterIp);
strcpy(tscMgmtIpList.ipstr[3], tsSecondIp);
tscMgmtIpList.ip[3] = inet_addr(tsSecondIp);
}
#else
if (ip && ip[0]) {
if (ip != tsMasterIp) {
strcpy(tsMasterIp, ip);
}
tsServerIp = inet_addr(ip);
}
#endif
pObj = (STscObj *)malloc(sizeof(STscObj));
if (NULL == pObj) {
......@@ -175,11 +173,6 @@ TAOS *taos_connect(const char *ip, const char *user, const char *pass, const cha
TAOS *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int),
void *param, void **taos) {
#ifndef CLUSTER
if (ip == NULL) {
ip = tsMasterIp;
}
#endif
return taos_connect_imp(ip, user, pass, db, port, fp, param, taos);
}
......@@ -344,7 +337,7 @@ int taos_fetch_block_impl(TAOS_RES *res, TAOS_ROW *rows) {
SSqlRes *pRes = &pSql->res;
STscObj *pObj = pSql->pTscObj;
if (pRes->qhandle == 0 || pObj->pSql != pSql) {
if (pRes->qhandle == 0) {
*rows = NULL;
return 0;
}
......@@ -694,7 +687,7 @@ int taos_select_db(TAOS *taos, const char *db) {
return taos_query(taos, sql);
}
void taos_free_result(TAOS_RES *res) {
void taos_free_result_imp(TAOS_RES* res, int keepCmd) {
if (res == NULL) return;
SSqlObj *pSql = (SSqlObj *)res;
......@@ -712,6 +705,8 @@ void taos_free_result(TAOS_RES *res) {
pSql->thandle = NULL;
tscFreeSqlObj(pSql);
tscTrace("%p Async SqlObj is freed by app", pSql);
} else if (keepCmd) {
tscFreeSqlResult(pSql);
} else {
tscFreeSqlObjPartial(pSql);
}
......@@ -761,8 +756,13 @@ void taos_free_result(TAOS_RES *res) {
* Then this object will be reused and no free operation is required.
*/
pSql->thandle = NULL;
tscFreeSqlObjPartial(pSql);
tscTrace("%p sql result is freed by app", pSql);
if (keepCmd) {
tscFreeSqlResult(pSql);
tscTrace("%p sql result is freed by app while sql command is kept", pSql);
} else {
tscFreeSqlObjPartial(pSql);
tscTrace("%p sql result is freed by app", pSql);
}
}
} else {
// if no free resource msg is sent to vnode, we free this object immediately.
......@@ -772,6 +772,9 @@ void taos_free_result(TAOS_RES *res) {
assert(pRes->numOfRows == 0 || (pCmd->command > TSDB_SQL_LOCAL));
tscFreeSqlObj(pSql);
tscTrace("%p Async sql result is freed by app", pSql);
} else if (keepCmd) {
tscFreeSqlResult(pSql);
tscTrace("%p sql result is freed while sql command is kept", pSql);
} else {
tscFreeSqlObjPartial(pSql);
tscTrace("%p sql result is freed", pSql);
......@@ -779,6 +782,10 @@ void taos_free_result(TAOS_RES *res) {
}
}
void taos_free_result(TAOS_RES *res) {
taos_free_result_imp(res, 0);
}
int taos_errno(TAOS *taos) {
STscObj *pObj = (STscObj *)taos;
int code;
......@@ -861,61 +868,63 @@ void taos_stop_query(TAOS_RES *res) {
int taos_print_row(char *str, TAOS_ROW row, TAOS_FIELD *fields, int num_fields) {
int len = 0;
for (int i = 0; i < num_fields; ++i) {
if (i > 0) {
str[len++] = ' ';
}
if (row[i] == NULL) {
len += sprintf(str + len, "%s ", TSDB_DATA_NULL_STR);
len += sprintf(str + len, "%s", TSDB_DATA_NULL_STR);
continue;
}
switch (fields[i].type) {
case TSDB_DATA_TYPE_TINYINT:
len += sprintf(str + len, "%d ", *((char *)row[i]));
len += sprintf(str + len, "%d", *((char *)row[i]));
break;
case TSDB_DATA_TYPE_SMALLINT:
len += sprintf(str + len, "%d ", *((short *)row[i]));
len += sprintf(str + len, "%d", *((short *)row[i]));
break;
case TSDB_DATA_TYPE_INT:
len += sprintf(str + len, "%d ", *((int *)row[i]));
len += sprintf(str + len, "%d", *((int *)row[i]));
break;
case TSDB_DATA_TYPE_BIGINT:
len += sprintf(str + len, "%" PRId64 " ", *((int64_t *)row[i]));
len += sprintf(str + len, "%" PRId64, *((int64_t *)row[i]));
break;
case TSDB_DATA_TYPE_FLOAT: {
float fv = 0;
fv = GET_FLOAT_VAL(row[i]);
len += sprintf(str + len, "%f ", fv);
len += sprintf(str + len, "%f", fv);
}
break;
case TSDB_DATA_TYPE_DOUBLE:{
double dv = 0;
dv = GET_DOUBLE_VAL(row[i]);
len += sprintf(str + len, "%lf ", dv);
len += sprintf(str + len, "%lf", dv);
}
break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
/* limit the max length of string to no greater than the maximum length,
* in case of not null-terminated string */
size_t xlen = strlen(row[i]);
size_t trueLen = MIN(xlen, fields[i].bytes);
memcpy(str + len, (char *)row[i], trueLen);
str[len + trueLen] = ' ';
len += (trueLen + 1);
} break;
size_t xlen = 0;
for (xlen = 0; xlen <= fields[i].bytes; xlen++) {
char c = ((char*)row[i])[xlen];
if (c == 0) break;
str[len++] = c;
}
str[len] = 0;
}
break;
case TSDB_DATA_TYPE_TIMESTAMP:
len += sprintf(str + len, "%" PRId64 " ", *((int64_t *)row[i]));
len += sprintf(str + len, "%" PRId64, *((int64_t *)row[i]));
break;
case TSDB_DATA_TYPE_BOOL:
len += sprintf(str + len, "%d ", *((int8_t *)row[i]));
len += sprintf(str + len, "%d", *((int8_t *)row[i]));
default:
break;
}
......
src/client/src/tscStream.c
浏览文件 @ 69a5dcda
......@@ -268,11 +268,11 @@ static void tscSetRetryTimer(SSqlStream *pStream, SSqlObj *pSql, int64_t timer)
tscTrace("%p stream:%p, etime:%" PRId64 " is too old, exceeds the max retention time window:%" PRId64 ", stop the stream",
pStream->pSql, pStream, pStream->stime, pStream->etime);
// TODO : How to terminate stream here
taos_close_stream(pStream);
if (pStream->callback) {
// Callback function from upper level
pStream->callback(pStream->param);
}
taos_close_stream(pStream);
return;
}
......@@ -302,24 +302,24 @@ static void tscSetNextLaunchTimer(SSqlStream *pStream, SSqlObj *pSql) {
tscTrace("%p stream:%p, stime:%" PRId64 " is larger than end time: %" PRId64 ", stop the stream", pStream->pSql, pStream,
pStream->stime, pStream->etime);
// TODO : How to terminate stream here
taos_close_stream(pStream);
if (pStream->callback) {
// Callback function from upper level
pStream->callback(pStream->param);
}
taos_close_stream(pStream);
return;
}
} else {
pStream->stime += pStream->slidingTime;
if ((pStream->stime - pStream->interval) >= pStream->etime) {
tscTrace("%p stream:%p, stime:%ld is larger than end time: %ld, stop the stream", pStream->pSql, pStream,
tscTrace("%p stream:%p, stime:%" PRId64 " is larger than end time: %" PRId64 ", stop the stream", pStream->pSql, pStream,
pStream->stime, pStream->etime);
// TODO : How to terminate stream here
taos_close_stream(pStream);
if (pStream->callback) {
// Callback function from upper level
pStream->callback(pStream->param);
}
taos_close_stream(pStream);
return;
}
......@@ -353,7 +353,7 @@ static void tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) {
int64_t minIntervalTime =
(pStream->precision == TSDB_TIME_PRECISION_MICRO) ? tsMinIntervalTime * 1000L : tsMinIntervalTime;
if (pCmd->nAggTimeInterval < minIntervalTime) {
tscWarn("%p stream:%p, original sample interval:%ld too small, reset to:%" PRId64 "", pSql, pStream,
tscWarn("%p stream:%p, original sample interval:%" PRId64 " too small, reset to:%" PRId64, pSql, pStream,
pCmd->nAggTimeInterval, minIntervalTime);
pCmd->nAggTimeInterval = minIntervalTime;
}
......
src/client/src/tscSub.c
浏览文件 @ 69a5dcda
......@@ -22,125 +22,407 @@
#include "tsclient.h"
#include "tsocket.h"
#include "ttime.h"
#include "ttimer.h"
#include "tutil.h"
#include "tscUtil.h"
#include "tcache.h"
typedef struct {
void * signature;
char name[TSDB_METER_ID_LEN];
int mseconds;
TSKEY lastKey;
uint64_t stime;
TAOS_FIELD fields[TSDB_MAX_COLUMNS];
int numOfFields;
TAOS * taos;
TAOS_RES * result;
typedef struct SSubscriptionProgress {
int64_t uid;
TSKEY key;
} SSubscriptionProgress;
typedef struct SSub {
void * signature;
char topic[32];
int64_t lastSyncTime;
int64_t lastConsumeTime;
TAOS * taos;
void * pTimer;
SSqlObj * pSql;
int interval;
TAOS_SUBSCRIBE_CALLBACK fp;
void * param;
int numOfMeters;
SSubscriptionProgress * progress;
} SSub;
TAOS_SUB *taos_subscribe(const char *host, const char *user, const char *pass, const char *db, const char *name, int64_t time, int mseconds) {
SSub *pSub;
pSub = (SSub *)malloc(sizeof(SSub));
if (pSub == NULL) return NULL;
memset(pSub, 0, sizeof(SSub));
static int tscCompareSubscriptionProgress(const void* a, const void* b) {
const SSubscriptionProgress* x = (const SSubscriptionProgress*)a;
const SSubscriptionProgress* y = (const SSubscriptionProgress*)b;
if (x->uid > y->uid) return 1;
if (x->uid < y->uid) return -1;
return 0;
}
TSKEY tscGetSubscriptionProgress(void* sub, int64_t uid) {
if (sub == NULL)
return 0;
SSub* pSub = (SSub*)sub;
for (int s = 0, e = pSub->numOfMeters; s < e;) {
int m = (s + e) / 2;
SSubscriptionProgress* p = pSub->progress + m;
if (p->uid > uid)
e = m;
else if (p->uid < uid)
s = m + 1;
else
return p->key;
}
return 0;
}
void tscUpdateSubscriptionProgress(void* sub, int64_t uid, TSKEY ts) {
if( sub == NULL)
return;
SSub* pSub = (SSub*)sub;
for (int s = 0, e = pSub->numOfMeters; s < e;) {
int m = (s + e) / 2;
SSubscriptionProgress* p = pSub->progress + m;
if (p->uid > uid)
e = m;
else if (p->uid < uid)
s = m + 1;
else {
if (ts >= p->key) p->key = ts;
break;
}
}
}
static SSub* tscCreateSubscription(STscObj* pObj, const char* topic, const char* sql) {
SSub* pSub = calloc(1, sizeof(SSub));
if (pSub == NULL) {
globalCode = TSDB_CODE_CLI_OUT_OF_MEMORY;
tscError("failed to allocate memory for subscription");
return NULL;
}
SSqlObj* pSql = calloc(1, sizeof(SSqlObj));
if (pSql == NULL) {
globalCode = TSDB_CODE_CLI_OUT_OF_MEMORY;
tscError("failed to allocate SSqlObj for subscription");
goto failed;
}
pSql->signature = pSql;
pSql->pTscObj = pObj;
char* sqlstr = (char*)malloc(strlen(sql) + 1);
if (sqlstr == NULL) {
tscError("failed to allocate sql string for subscription");
goto failed;
}
strcpy(sqlstr, sql);
strtolower(sqlstr, sqlstr);
pSql->sqlstr = sqlstr;
tsem_init(&pSql->rspSem, 0, 0);
tsem_init(&pSql->emptyRspSem, 0, 1);
SSqlRes *pRes = &pSql->res;
pRes->numOfRows = 1;
pRes->numOfTotal = 0;
pSql->pSubscription = pSub;
pSub->pSql = pSql;
pSub->signature = pSub;
strcpy(pSub->name, name);
pSub->mseconds = mseconds;
pSub->lastKey = time;
if (pSub->lastKey == 0) {
pSub->lastKey = taosGetTimestampMs();
strncpy(pSub->topic, topic, sizeof(pSub->topic));
pSub->topic[sizeof(pSub->topic) - 1] = 0;
return pSub;
failed:
if (sqlstr != NULL) {
free(sqlstr);
}
if (pSql != NULL) {
free(pSql);
}
free(pSub);
return NULL;
}
static void tscProcessSubscriptionTimer(void *handle, void *tmrId) {
SSub *pSub = (SSub *)handle;
if (pSub == NULL || pSub->pTimer != tmrId) return;
TAOS_RES* res = taos_consume(pSub);
if (res != NULL) {
pSub->fp(pSub, res, pSub->param, 0);
}
taosTmrReset(tscProcessSubscriptionTimer, pSub->interval, pSub, tscTmr, &pSub->pTimer);
}
int tscUpdateSubscription(STscObj* pObj, SSub* pSub) {
int code = (uint8_t)tsParseSql(pSub->pSql, pObj->acctId, pObj->db, false);
if (code != TSDB_CODE_SUCCESS) {
tscError("failed to parse sql statement: %s", pSub->topic);
return 0;
}
SSqlCmd* pCmd = &pSub->pSql->cmd;
if (pCmd->command != TSDB_SQL_SELECT) {
tscError("only 'select' statement is allowed in subscription: %s", pSub->topic);
return 0;
}
SMeterMetaInfo *pMeterMetaInfo = tscGetMeterMetaInfo(pCmd, 0);
int numOfMeters = 0;
if (!UTIL_METER_IS_NOMRAL_METER(pMeterMetaInfo)) {
SMetricMeta* pMetricMeta = pMeterMetaInfo->pMetricMeta;
for (int32_t i = 0; i < pMetricMeta->numOfVnodes; i++) {
SVnodeSidList *pVnodeSidList = tscGetVnodeSidList(pMetricMeta, i);
numOfMeters += pVnodeSidList->numOfSids;
}
}
taos_init();
pSub->taos = taos_connect(host, user, pass, NULL, 0);
if (pSub->taos == NULL) {
tfree(pSub);
SSubscriptionProgress* progress = (SSubscriptionProgress*)calloc(numOfMeters, sizeof(SSubscriptionProgress));
if (progress == NULL) {
tscError("failed to allocate memory for progress: %s", pSub->topic);
return 0;
}
if (UTIL_METER_IS_NOMRAL_METER(pMeterMetaInfo)) {
numOfMeters = 1;
int64_t uid = pMeterMetaInfo->pMeterMeta->uid;
progress[0].uid = uid;
progress[0].key = tscGetSubscriptionProgress(pSub, uid);
} else {
char qstr[256] = {0};
sprintf(qstr, "use %s", db);
int res = taos_query(pSub->taos, qstr);
if (res != 0) {
tscError("failed to open DB:%s", db);
taos_close(pSub->taos);
tfree(pSub);
} else {
snprintf(qstr, tListLen(qstr), "select * from %s where _c0 > now+1000d", pSub->name);
if (taos_query(pSub->taos, qstr)) {
tscTrace("failed to select, reason:%s", taos_errstr(pSub->taos));
taos_close(pSub->taos);
tfree(pSub);
return NULL;
SMetricMeta* pMetricMeta = pMeterMetaInfo->pMetricMeta;
numOfMeters = 0;
for (int32_t i = 0; i < pMetricMeta->numOfVnodes; i++) {
SVnodeSidList *pVnodeSidList = tscGetVnodeSidList(pMetricMeta, i);
for (int32_t j = 0; j < pVnodeSidList->numOfSids; j++) {
SMeterSidExtInfo *pMeterInfo = tscGetMeterSidInfo(pVnodeSidList, j);
int64_t uid = pMeterInfo->uid;
progress[numOfMeters].uid = uid;
progress[numOfMeters++].key = tscGetSubscriptionProgress(pSub, uid);
}
pSub->result = taos_use_result(pSub->taos);
pSub->numOfFields = taos_num_fields(pSub->result);
memcpy(pSub->fields, taos_fetch_fields(pSub->result), sizeof(TAOS_FIELD) * pSub->numOfFields);
}
qsort(progress, numOfMeters, sizeof(SSubscriptionProgress), tscCompareSubscriptionProgress);
}
return pSub;
free(pSub->progress);
pSub->numOfMeters = numOfMeters;
pSub->progress = progress;
pSub->lastSyncTime = taosGetTimestampMs();
return 1;
}
TAOS_ROW taos_consume(TAOS_SUB *tsub) {
SSub * pSub = (SSub *)tsub;
TAOS_ROW row;
char qstr[256];
if (pSub == NULL) return NULL;
if (pSub->signature != pSub) return NULL;
while (1) {
if (pSub->result != NULL) {
row = taos_fetch_row(pSub->result);
if (row != NULL) {
pSub->lastKey = *((uint64_t *)row[0]);
return row;
}
static int tscLoadSubscriptionProgress(SSub* pSub) {
char buf[TSDB_MAX_SQL_LEN];
sprintf(buf, "%s/subscribe/%s", dataDir, pSub->topic);
taos_free_result(pSub->result);
pSub->result = NULL;
uint64_t etime = taosGetTimestampMs();
int64_t mseconds = pSub->mseconds - etime + pSub->stime;
if (mseconds < 0) mseconds = 0;
taosMsleep((int)mseconds);
}
FILE* fp = fopen(buf, "r");
if (fp == NULL) {
tscTrace("subscription progress file does not exist: %s", pSub->topic);
return 1;
}
pSub->stime = taosGetTimestampMs();
if (fgets(buf, sizeof(buf), fp) == NULL) {
tscTrace("invalid subscription progress file: %s", pSub->topic);
fclose(fp);
return 0;
}
sprintf(qstr, "select * from %s where _c0 > %" PRId64 " order by _c0 asc", pSub->name, pSub->lastKey);
if (taos_query(pSub->taos, qstr)) {
tscTrace("failed to select, reason:%s", taos_errstr(pSub->taos));
return NULL;
for (int i = 0; i < sizeof(buf); i++) {
if (buf[i] == 0)
break;
if (buf[i] == '\r' || buf[i] == '\n') {
buf[i] = 0;
break;
}
}
if (strcmp(buf, pSub->pSql->sqlstr) != 0) {
tscTrace("subscription sql statement mismatch: %s", pSub->topic);
fclose(fp);
return 0;
}
pSub->result = taos_use_result(pSub->taos);
if (fgets(buf, sizeof(buf), fp) == NULL || atoi(buf) < 0) {
tscTrace("invalid subscription progress file: %s", pSub->topic);
fclose(fp);
return 0;
}
if (pSub->result == NULL) {
tscTrace("failed to get result, reason:%s", taos_errstr(pSub->taos));
return NULL;
int numOfMeters = atoi(buf);
SSubscriptionProgress* progress = calloc(numOfMeters, sizeof(SSubscriptionProgress));
for (int i = 0; i < numOfMeters; i++) {
if (fgets(buf, sizeof(buf), fp) == NULL) {
fclose(fp);
free(progress);
return 0;
}
int64_t uid, key;
sscanf(buf, "%" SCNd64 ":%" SCNd64, &uid, &key);
progress[i].uid = uid;
progress[i].key = key;
}
return NULL;
fclose(fp);
qsort(progress, numOfMeters, sizeof(SSubscriptionProgress), tscCompareSubscriptionProgress);
pSub->numOfMeters = numOfMeters;
pSub->progress = progress;
tscTrace("subscription progress loaded, %d tables: %s", numOfMeters, pSub->topic);
return 1;
}
void taos_unsubscribe(TAOS_SUB *tsub) {
SSub *pSub = (SSub *)tsub;
void tscSaveSubscriptionProgress(void* sub) {
SSub* pSub = (SSub*)sub;
if (pSub == NULL) return;
if (pSub->signature != pSub) return;
char path[256];
sprintf(path, "%s/subscribe", dataDir);
if (access(path, 0) != 0) {
mkdir(path, 0777);
}
taos_close(pSub->taos);
free(pSub);
sprintf(path, "%s/subscribe/%s", dataDir, pSub->topic);
FILE* fp = fopen(path, "w+");
if (fp == NULL) {
tscError("failed to create progress file for subscription: %s", pSub->topic);
return;
}
fputs(pSub->pSql->sqlstr, fp);
fprintf(fp, "\n%d\n", pSub->numOfMeters);
for (int i = 0; i < pSub->numOfMeters; i++) {
int64_t uid = pSub->progress[i].uid;
TSKEY key = pSub->progress[i].key;
fprintf(fp, "%" PRId64 ":%" PRId64 "\n", uid, key);
}
fclose(fp);
}
int taos_subfields_count(TAOS_SUB *tsub) {
TAOS_SUB *taos_subscribe(TAOS *taos, int restart, const char* topic, const char *sql, TAOS_SUBSCRIBE_CALLBACK fp, void *param, int interval) {
STscObj* pObj = (STscObj*)taos;
if (pObj == NULL || pObj->signature != pObj) {
globalCode = TSDB_CODE_DISCONNECTED;
tscError("connection disconnected");
return NULL;
}
SSub* pSub = tscCreateSubscription(pObj, topic, sql);
if (pSub == NULL) {
return NULL;
}
pSub->taos = taos;
if (restart) {
tscTrace("restart subscription: %s", topic);
} else {
tscLoadSubscriptionProgress(pSub);
}
if (!tscUpdateSubscription(pObj, pSub)) {
taos_unsubscribe(pSub, 1);
return NULL;
}
pSub->interval = interval;
if (fp != NULL) {
tscTrace("asynchronize subscription, create new timer", topic);
pSub->fp = fp;
pSub->param = param;
taosTmrReset(tscProcessSubscriptionTimer, interval, pSub, tscTmr, &pSub->pTimer);
}
return pSub;
}
void taos_free_result_imp(SSqlObj* pSql, int keepCmd);
TAOS_RES *taos_consume(TAOS_SUB *tsub) {
SSub *pSub = (SSub *)tsub;
if (pSub == NULL) return NULL;
return pSub->numOfFields;
tscSaveSubscriptionProgress(pSub);
SSqlObj* pSql = pSub->pSql;
SSqlRes *pRes = &pSql->res;
if (pSub->pTimer == NULL) {
int64_t duration = taosGetTimestampMs() - pSub->lastConsumeTime;
if (duration < (int64_t)(pSub->interval)) {
tscTrace("subscription consume too frequently, blocking...");
taosMsleep(pSub->interval - (int32_t)duration);
}
}
for (int retry = 0; retry < 3; retry++) {
tscRemoveFromSqlList(pSql);
if (taosGetTimestampMs() - pSub->lastSyncTime > 10 * 60 * 1000) {
tscTrace("begin meter synchronization");
char* sqlstr = pSql->sqlstr;
pSql->sqlstr = NULL;
taos_free_result_imp(pSql, 0);
pSql->sqlstr = sqlstr;
taosClearDataCache(tscCacheHandle);
if (!tscUpdateSubscription(pSub->taos, pSub)) return NULL;
tscTrace("meter synchronization completed");
} else {
uint16_t type = pSql->cmd.type;
taos_free_result_imp(pSql, 1);
pRes->numOfRows = 1;
pRes->numOfTotal = 0;
pRes->qhandle = 0;
pSql->thandle = NULL;
pSql->cmd.command = TSDB_SQL_SELECT;
pSql->cmd.type = type;
tscGetMeterMetaInfo(&pSql->cmd, 0)->vnodeIndex = 0;
}
tscDoQuery(pSql);
if (pRes->code != TSDB_CODE_NOT_ACTIVE_TABLE) {
break;
}
// meter was removed, make sync time zero, so that next retry will
// do synchronization first
pSub->lastSyncTime = 0;
}
if (pRes->code != TSDB_CODE_SUCCESS) {
tscError("failed to query data, error code=%d", pRes->code);
tscRemoveFromSqlList(pSql);
return NULL;
}
pSub->lastConsumeTime = taosGetTimestampMs();
return pSql;
}
TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub) {
void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress) {
SSub *pSub = (SSub *)tsub;
if (pSub == NULL || pSub->signature != pSub) return;
return pSub->fields;
if (pSub->pTimer != NULL) {
taosTmrStop(pSub->pTimer);
}
if (keepProgress) {
tscSaveSubscriptionProgress(pSub);
} else {
char path[256];
sprintf(path, "%s/subscribe/%s", dataDir, pSub->topic);
remove(path);
}
tscFreeSqlObj(pSub->pSql);
free(pSub->progress);
memset(pSub, 0, sizeof(*pSub));
free(pSub);
}
src/client/src/tscSystem.c
浏览文件 @ 69a5dcda
......@@ -48,6 +48,7 @@ static pthread_once_t tscinit = PTHREAD_ONCE_INIT;
extern int tsTscEnableRecordSql;
extern int tsNumOfLogLines;
void taosInitNote(int numOfNoteLines, int maxNotes, char* lable);
void deltaToUtcInitOnce();
void tscCheckDiskUsage(void *para, void *unused) {
taosGetDisk();
......@@ -60,6 +61,7 @@ void taos_init_imp() {
SRpcInit rpcInit;
srand(taosGetTimestampSec());
deltaToUtcInitOnce();
if (tscEmbedded == 0) {
/*
......@@ -93,7 +95,6 @@ void taos_init_imp() {
taosInitNote(tsNumOfLogLines / 10, 1, (char*)"tsc_note");
}
#ifdef CLUSTER
tscMgmtIpList.numOfIps = 2;
strcpy(tscMgmtIpList.ipstr[0], tsMasterIp);
tscMgmtIpList.ip[0] = inet_addr(tsMasterIp);
......@@ -106,7 +107,6 @@ void taos_init_imp() {
strcpy(tscMgmtIpList.ipstr[2], tsSecondIp);
tscMgmtIpList.ip[2] = inet_addr(tsSecondIp);
}
#endif
tscInitMsgs();
slaveIndex = rand();
......
src/client/src/tscUtil.c
浏览文件 @ 69a5dcda
......@@ -376,6 +376,21 @@ void tscFreeSqlCmdData(SSqlCmd* pCmd) {
}
}
void tscFreeSqlResult(SSqlObj* pSql) {
tfree(pSql->res.pRsp);
pSql->res.row = 0;
pSql->res.numOfRows = 0;
pSql->res.numOfTotal = 0;
pSql->res.numOfGroups = 0;
tfree(pSql->res.pGroupRec);
tscDestroyLocalReducer(pSql);
tscDestroyResPointerInfo(&pSql->res);
tfree(pSql->res.pColumnIndex);
}
void tscFreeSqlObjPartial(SSqlObj* pSql) {
if (pSql == NULL || pSql->signature != pSql) {
return;
......@@ -399,20 +414,9 @@ void tscFreeSqlObjPartial(SSqlObj* pSql) {
tfree(pSql->sqlstr);
pthread_mutex_unlock(&pObj->mutex);
tfree(pSql->res.pRsp);
pSql->res.row = 0;
pSql->res.numOfRows = 0;
pSql->res.numOfTotal = 0;
pSql->res.numOfGroups = 0;
tfree(pSql->res.pGroupRec);
tscDestroyLocalReducer(pSql);
tscFreeSqlResult(pSql);
tfree(pSql->pSubs);
pSql->numOfSubs = 0;
tscDestroyResPointerInfo(pRes);
tfree(pSql->res.pColumnIndex);
tscFreeSqlCmdData(pCmd);
tscRemoveAllMeterMetaInfo(pCmd, false);
......@@ -822,7 +826,7 @@ void tscFieldInfoSetValFromField(SFieldInfo* pFieldInfo, int32_t index, TAOS_FIE
}
void tscFieldInfoUpdateVisible(SFieldInfo* pFieldInfo, int32_t index, bool visible) {
if (index < 0 || index > pFieldInfo->numOfOutputCols) {
if (index < 0 || index >= pFieldInfo->numOfOutputCols) {
return;
}
......
src/connector/grafana/tdengine/dist/plugin.json
浏览文件 @ 69a5dcda
{
"name": "TDengine",
"id": "tdengine",
"id": "taosdata-tdengine-datasource",
"type": "datasource",
"partials": {
......@@ -24,8 +24,8 @@
{"name": "GitHub", "url": "https://github.com/taosdata/TDengine/tree/develop/src/connector/grafana/tdengine"},
{"name": "AGPL 3.0", "url": "https://github.com/taosdata/TDengine/tree/develop/src/connector/grafana/tdengine/LICENSE"}
],
"version": "1.6.0",
"updated": "2019-11-12"
"version": "1.0.0",
"updated": "2020-01-13"
},
"dependencies": {
......
src/connector/grafana/tdengine/package.json
浏览文件 @ 69a5dcda
{
"name": "TDengine",
"private": true,
"private": false,
"version": "1.0.0",
"description": "grafana datasource plugin for tdengine",
"main": "index.js",
"scripts": {
"build": "./node_modules/grunt-cli/bin/grunt",
"test": "./node_modules/grunt-cli/bin/grunt mochaTest"
......@@ -12,7 +11,7 @@
"type": "git",
"url": "git+https://github.com/taosdata/TDengine.git"
},
"author": "",
"author": "https://www.taosdata.com",
"license": "AGPL 3.0",
"bugs": {
"url": "https://github.com/taosdata/TDengine/issues"
......
src/connector/grafana/tdengine/src/plugin.json
浏览文件 @ 69a5dcda
{
"name": "TDengine",
"id": "tdengine",
"id": "taosdata-tdengine-datasource",
"type": "datasource",
"partials": {
......@@ -24,8 +24,8 @@
{"name": "GitHub", "url": "https://github.com/taosdata/TDengine/tree/develop/src/connector/grafana/tdengine"},
{"name": "AGPL 3.0", "url": "https://github.com/taosdata/TDengine/tree/develop/src/connector/grafana/tdengine/LICENSE"}
],
"version": "1.6.0",
"updated": "2019-11-12"
"version": "1.0.0",
"updated": "2020-01-13"
},
"dependencies": {
......
src/connector/python/linux/python2/taos/cinterface.py
浏览文件 @ 69a5dcda
......@@ -13,14 +13,14 @@ def _convert_microsecond_to_datetime(micro):
def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
"""Function to convert C bool row to python row
"""
_timstamp_converter = _convert_millisecond_to_datetime
_timestamp_converter = _convert_millisecond_to_datetime
if micro:
_timstamp_converter = _convert_microsecond_to_datetime
_timestamp_converter = _convert_microsecond_to_datetime
if num_of_rows > 0:
return list(map(_timstamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)][::-1]))
return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)][::-1]))
else:
return list(map(_timstamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]))
return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
"""Function to convert C bool row to python row
......@@ -144,6 +144,8 @@ class CTaosInterface(object):
libtaos.taos_use_result.restype = ctypes.c_void_p
libtaos.taos_fetch_row.restype = ctypes.POINTER(ctypes.c_void_p)
libtaos.taos_errstr.restype = ctypes.c_char_p
libtaos.taos_subscribe.restype = ctypes.c_void_p
libtaos.taos_consume.restype = ctypes.c_void_p
def __init__(self, config=None):
'''
......@@ -252,6 +254,41 @@ class CTaosInterface(object):
"""
return CTaosInterface.libtaos.taos_affected_rows(connection)
@staticmethod
def subscribe(connection, restart, topic, sql, interval):
"""Create a subscription
@restart boolean,
@sql string, sql statement for data query, must be a 'select' statement.
@topic string, name of this subscription
"""
return ctypes.c_void_p(CTaosInterface.libtaos.taos_subscribe(
connection,
1 if restart else 0,
ctypes.c_char_p(topic.encode('utf-8')),
ctypes.c_char_p(sql.encode('utf-8')),
None,
None,
interval))
@staticmethod
def consume(sub):
"""Consume data of a subscription
"""
result = ctypes.c_void_p(CTaosInterface.libtaos.taos_consume(sub))
fields = []
pfields = CTaosInterface.fetchFields(result)
for i in range(CTaosInterface.libtaos.taos_num_fields(result)):
fields.append({'name': pfields[i].name.decode('utf-8'),
'bytes': pfields[i].bytes,
'type': ord(pfields[i].type)})
return result, fields
@staticmethod
def unsubscribe(sub, keepProgress):
"""Cancel a subscription
"""
CTaosInterface.libtaos.taos_unsubscribe(sub, 1 if keepProgress else 0)
@staticmethod
def useResult(connection):
'''Use result after calling self.query
......@@ -275,8 +312,8 @@ class CTaosInterface(object):
if num_of_rows == 0:
return None, 0
blocks = [None] * len(fields)
isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO)
blocks = [None] * len(fields)
for i in range(len(fields)):
data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i]
......@@ -351,4 +388,20 @@ class CTaosInterface(object):
def errStr(connection):
"""Return the error styring
"""
return CTaosInterface.libtaos.taos_errstr(connection)
\ No newline at end of file
return CTaosInterface.libtaos.taos_errstr(connection)
if __name__ == '__main__':
cinter = CTaosInterface()
conn = cinter.connect()
print('Query return value: {}'.format(cinter.query(conn, 'show databases')))
print('Affected rows: {}'.format(cinter.affectedRows(conn)))
result, des = CTaosInterface.useResult(conn)
data, num_of_rows = CTaosInterface.fetchBlock(result, des)
print(data)
cinter.close(conn)
\ No newline at end of file
src/connector/python/linux/python2/taos/connection.py
浏览文件 @ 69a5dcda
# from .cursor import TDengineCursor
from .cursor import TDengineCursor
from .subscription import TDengineSubscription
from .cinterface import CTaosInterface
class TDengineConnection(object):
......@@ -50,6 +50,14 @@ class TDengineConnection(object):
"""
return CTaosInterface.close(self._conn)
def subscribe(self, restart, topic, sql, interval):
"""Create a subscription.
"""
if self._conn is None:
return None
sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval)
return TDengineSubscription(sub)
def cursor(self):
"""Return a new Cursor object using the connection.
"""
......
src/connector/python/linux/python2/taos/subscription.py 0 → 100644
浏览文件 @ 69a5dcda
from .cinterface import CTaosInterface
from .error import *
class TDengineSubscription(object):
"""TDengine subscription object
"""
def __init__(self, sub):
self._sub = sub
def consume(self):
"""Consume rows of a subscription
"""
if self._sub is None:
raise OperationalError("Invalid use of consume")
result, fields = CTaosInterface.consume(self._sub)
buffer = [[] for i in range(len(fields))]
while True:
block, num_of_fields = CTaosInterface.fetchBlock(result, fields)
if num_of_fields == 0: break
for i in range(len(fields)):
buffer[i].extend(block[i])
self.fields = fields
return list(map(tuple, zip(*buffer)))
def close(self, keepProgress = True):
"""Close the Subscription.
"""
if self._sub is None:
return False
CTaosInterface.unsubscribe(self._sub, keepProgress)
return True
if __name__ == '__main__':
from .connection import TDengineConnection
conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test")
# Generate a cursor object to run SQL commands
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
for i in range(0,10):
data = sub.consume()
for d in data:
print(d)
sub.close()
conn.close()
\ No newline at end of file
src/connector/python/linux/python3/taos/cinterface.py
浏览文件 @ 69a5dcda
......@@ -144,6 +144,8 @@ class CTaosInterface(object):
libtaos.taos_use_result.restype = ctypes.c_void_p
libtaos.taos_fetch_row.restype = ctypes.POINTER(ctypes.c_void_p)
libtaos.taos_errstr.restype = ctypes.c_char_p
libtaos.taos_subscribe.restype = ctypes.c_void_p
libtaos.taos_consume.restype = ctypes.c_void_p
def __init__(self, config=None):
'''
......@@ -252,6 +254,41 @@ class CTaosInterface(object):
"""
return CTaosInterface.libtaos.taos_affected_rows(connection)
@staticmethod
def subscribe(connection, restart, topic, sql, interval):
"""Create a subscription
@restart boolean,
@sql string, sql statement for data query, must be a 'select' statement.
@topic string, name of this subscription
"""
return ctypes.c_void_p(CTaosInterface.libtaos.taos_subscribe(
connection,
1 if restart else 0,
ctypes.c_char_p(topic.encode('utf-8')),
ctypes.c_char_p(sql.encode('utf-8')),
None,
None,
interval))
@staticmethod
def consume(sub):
"""Consume data of a subscription
"""
result = ctypes.c_void_p(CTaosInterface.libtaos.taos_consume(sub))
fields = []
pfields = CTaosInterface.fetchFields(result)
for i in range(CTaosInterface.libtaos.taos_num_fields(result)):
fields.append({'name': pfields[i].name.decode('utf-8'),
'bytes': pfields[i].bytes,
'type': ord(pfields[i].type)})
return result, fields
@staticmethod
def unsubscribe(sub, keepProgress):
"""Cancel a subscription
"""
CTaosInterface.libtaos.taos_unsubscribe(sub, 1 if keepProgress else 0)
@staticmethod
def useResult(connection):
'''Use result after calling self.query
......
src/connector/python/linux/python3/taos/connection.py
浏览文件 @ 69a5dcda
# from .cursor import TDengineCursor
from .cursor import TDengineCursor
from .subscription import TDengineSubscription
from .cinterface import CTaosInterface
class TDengineConnection(object):
......@@ -50,6 +50,14 @@ class TDengineConnection(object):
"""
return CTaosInterface.close(self._conn)
def subscribe(self, restart, topic, sql, interval):
"""Create a subscription.
"""
if self._conn is None:
return None
sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval)
return TDengineSubscription(sub)
def cursor(self):
"""Return a new Cursor object using the connection.
"""
......
src/connector/python/linux/python3/taos/subscription.py 0 → 100644
浏览文件 @ 69a5dcda
from .cinterface import CTaosInterface
from .error import *
class TDengineSubscription(object):
"""TDengine subscription object
"""
def __init__(self, sub):
self._sub = sub
def consume(self):
"""Consume rows of a subscription
"""
if self._sub is None:
raise OperationalError("Invalid use of consume")
result, fields = CTaosInterface.consume(self._sub)
buffer = [[] for i in range(len(fields))]
while True:
block, num_of_fields = CTaosInterface.fetchBlock(result, fields)
if num_of_fields == 0: break
for i in range(len(fields)):
buffer[i].extend(block[i])
self.fields = fields
return list(map(tuple, zip(*buffer)))
def close(self, keepProgress = True):
"""Close the Subscription.
"""
if self._sub is None:
return False
CTaosInterface.unsubscribe(self._sub, keepProgress)
return True
if __name__ == '__main__':
from .connection import TDengineConnection
conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test")
# Generate a cursor object to run SQL commands
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
for i in range(0,10):
data = sub.consume()
for d in data:
print(d)
sub.close()
conn.close()
\ No newline at end of file
src/connector/python/windows/python2/taos/cinterface.py
浏览文件 @ 69a5dcda
......@@ -13,14 +13,14 @@ def _convert_microsecond_to_datetime(micro):
def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
"""Function to convert C bool row to python row
"""
_timstamp_converter = _convert_millisecond_to_datetime
_timestamp_converter = _convert_millisecond_to_datetime
if micro:
_timstamp_converter = _convert_microsecond_to_datetime
_timestamp_converter = _convert_microsecond_to_datetime
if num_of_rows > 0:
return list(map(_timstamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)][::-1]))
return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)][::-1]))
else:
return list(map(_timstamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)]))
return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
"""Function to convert C bool row to python row
......@@ -144,6 +144,8 @@ class CTaosInterface(object):
libtaos.taos_use_result.restype = ctypes.c_void_p
libtaos.taos_fetch_row.restype = ctypes.POINTER(ctypes.c_void_p)
libtaos.taos_errstr.restype = ctypes.c_char_p
libtaos.taos_subscribe.restype = ctypes.c_void_p
libtaos.taos_consume.restype = ctypes.c_void_p
def __init__(self, config=None):
'''
......@@ -252,6 +254,41 @@ class CTaosInterface(object):
"""
return CTaosInterface.libtaos.taos_affected_rows(connection)
@staticmethod
def subscribe(connection, restart, topic, sql, interval):
"""Create a subscription
@restart boolean,
@sql string, sql statement for data query, must be a 'select' statement.
@topic string, name of this subscription
"""
return ctypes.c_void_p(CTaosInterface.libtaos.taos_subscribe(
connection,
1 if restart else 0,
ctypes.c_char_p(topic.encode('utf-8')),
ctypes.c_char_p(sql.encode('utf-8')),
None,
None,
interval))
@staticmethod
def consume(sub):
"""Consume data of a subscription
"""
result = ctypes.c_void_p(CTaosInterface.libtaos.taos_consume(sub))
fields = []
pfields = CTaosInterface.fetchFields(result)
for i in range(CTaosInterface.libtaos.taos_num_fields(result)):
fields.append({'name': pfields[i].name.decode('utf-8'),
'bytes': pfields[i].bytes,
'type': ord(pfields[i].type)})
return result, fields
@staticmethod
def unsubscribe(sub, keepProgress):
"""Cancel a subscription
"""
CTaosInterface.libtaos.taos_unsubscribe(sub, 1 if keepProgress else 0)
@staticmethod
def useResult(connection):
'''Use result after calling self.query
......@@ -275,8 +312,8 @@ class CTaosInterface(object):
if num_of_rows == 0:
return None, 0
blocks = [None] * len(fields)
isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO)
blocks = [None] * len(fields)
for i in range(len(fields)):
data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i]
......@@ -351,4 +388,20 @@ class CTaosInterface(object):
def errStr(connection):
"""Return the error styring
"""
return CTaosInterface.libtaos.taos_errstr(connection)
\ No newline at end of file
return CTaosInterface.libtaos.taos_errstr(connection)
if __name__ == '__main__':
cinter = CTaosInterface()
conn = cinter.connect()
print('Query return value: {}'.format(cinter.query(conn, 'show databases')))
print('Affected rows: {}'.format(cinter.affectedRows(conn)))
result, des = CTaosInterface.useResult(conn)
data, num_of_rows = CTaosInterface.fetchBlock(result, des)
print(data)
cinter.close(conn)
\ No newline at end of file
src/connector/python/windows/python2/taos/connection.py
浏览文件 @ 69a5dcda
# from .cursor import TDengineCursor
from .cursor import TDengineCursor
from .subscription import TDengineSubscription
from .cinterface import CTaosInterface
class TDengineConnection(object):
......@@ -15,7 +15,8 @@ class TDengineConnection(object):
self._config = None
self._chandle = None
self.config(**kwargs)
if len(kwargs) > 0:
self.config(**kwargs)
def config(self, **kwargs):
# host
......@@ -50,6 +51,14 @@ class TDengineConnection(object):
"""
return CTaosInterface.close(self._conn)
def subscribe(self, restart, topic, sql, interval):
"""Create a subscription.
"""
if self._conn is None:
return None
sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval)
return TDengineSubscription(sub)
def cursor(self):
"""Return a new Cursor object using the connection.
"""
......
src/connector/python/windows/python2/taos/subscription.py 0 → 100644
浏览文件 @ 69a5dcda
from .cinterface import CTaosInterface
from .error import *
class TDengineSubscription(object):
"""TDengine subscription object
"""
def __init__(self, sub):
self._sub = sub
def consume(self):
"""Consume rows of a subscription
"""
if self._sub is None:
raise OperationalError("Invalid use of consume")
result, fields = CTaosInterface.consume(self._sub)
buffer = [[] for i in range(len(fields))]
while True:
block, num_of_fields = CTaosInterface.fetchBlock(result, fields)
if num_of_fields == 0: break
for i in range(len(fields)):
buffer[i].extend(block[i])
self.fields = fields
return list(map(tuple, zip(*buffer)))
def close(self, keepProgress = True):
"""Close the Subscription.
"""
if self._sub is None:
return False
CTaosInterface.unsubscribe(self._sub, keepProgress)
return True
if __name__ == '__main__':
from .connection import TDengineConnection
conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test")
# Generate a cursor object to run SQL commands
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
for i in range(0,10):
data = sub.consume()
for d in data:
print(d)
sub.close()
conn.close()
\ No newline at end of file
src/connector/python/windows/python3/taos/connection.py
浏览文件 @ 69a5dcda
# from .cursor import TDengineCursor
from .cursor import TDengineCursor
from .cinterface import CTaosInterface
class TDengineConnection(object):
""" TDengine connection object
"""
def __init__(self, *args, **kwargs):
self._conn = None
self._host = None
self._user = "root"
self._password = "taosdata"
self._database = None
self._port = 0
self._config = None
self._chandle = None
if len(kwargs) > 0:
self.config(**kwargs)
def config(self, **kwargs):
# host
if 'host' in kwargs:
self._host = kwargs['host']
# user
if 'user' in kwargs:
self._user = kwargs['user']
# password
if 'password' in kwargs:
self._password = kwargs['password']
# database
if 'database' in kwargs:
self._database = kwargs['database']
# port
if 'port' in kwargs:
self._port = kwargs['port']
# config
if 'config' in kwargs:
self._config = kwargs['config']
self._chandle = CTaosInterface(self._config)
self._conn = self._chandle.connect(self._host, self._user, self._password, self._database, self._port)
def close(self):
"""Close current connection.
"""
return CTaosInterface.close(self._conn)
def cursor(self):
"""Return a new Cursor object using the connection.
"""
return TDengineCursor(self)
def commit(self):
"""Commit any pending transaction to the database.
Since TDengine do not support transactions, the implement is void functionality.
"""
pass
def rollback(self):
"""Void functionality
"""
pass
def clear_result_set(self):
"""Clear unused result set on this connection.
"""
result = self._chandle.useResult(self._conn)[0]
if result:
self._chandle.freeResult(result)
if __name__ == "__main__":
conn = TDengineConnection(host='192.168.1.107')
conn.close()
from .cursor import TDengineCursor
from .subscription import TDengineSubscription
from .cinterface import CTaosInterface
class TDengineConnection(object):
""" TDengine connection object
"""
def __init__(self, *args, **kwargs):
self._conn = None
self._host = None
self._user = "root"
self._password = "taosdata"
self._database = None
self._port = 0
self._config = None
self._chandle = None
if len(kwargs) > 0:
self.config(**kwargs)
def config(self, **kwargs):
# host
if 'host' in kwargs:
self._host = kwargs['host']
# user
if 'user' in kwargs:
self._user = kwargs['user']
# password
if 'password' in kwargs:
self._password = kwargs['password']
# database
if 'database' in kwargs:
self._database = kwargs['database']
# port
if 'port' in kwargs:
self._port = kwargs['port']
# config
if 'config' in kwargs:
self._config = kwargs['config']
self._chandle = CTaosInterface(self._config)
self._conn = self._chandle.connect(self._host, self._user, self._password, self._database, self._port)
def close(self):
"""Close current connection.
"""
return CTaosInterface.close(self._conn)
def subscribe(self, restart, topic, sql, interval):
"""Create a subscription.
"""
if self._conn is None:
return None
sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval)
return TDengineSubscription(sub)
def cursor(self):
"""Return a new Cursor object using the connection.
"""
return TDengineCursor(self)
def commit(self):
"""Commit any pending transaction to the database.
Since TDengine do not support transactions, the implement is void functionality.
"""
pass
def rollback(self):
"""Void functionality
"""
pass
def clear_result_set(self):
"""Clear unused result set on this connection.
"""
result = self._chandle.useResult(self._conn)[0]
if result:
self._chandle.freeResult(result)
if __name__ == "__main__":
conn = TDengineConnection(host='192.168.1.107')
conn.close()
print("Hello world")
\ No newline at end of file
src/connector/python/windows/python3/taos/subscription.py 0 → 100644
浏览文件 @ 69a5dcda
from .cinterface import CTaosInterface
from .error import *
class TDengineSubscription(object):
"""TDengine subscription object
"""
def __init__(self, sub):
self._sub = sub
def consume(self):
"""Consume rows of a subscription
"""
if self._sub is None:
raise OperationalError("Invalid use of consume")
result, fields = CTaosInterface.consume(self._sub)
buffer = [[] for i in range(len(fields))]
while True:
block, num_of_fields = CTaosInterface.fetchBlock(result, fields)
if num_of_fields == 0: break
for i in range(len(fields)):
buffer[i].extend(block[i])
self.fields = fields
return list(map(tuple, zip(*buffer)))
def close(self, keepProgress = True):
"""Close the Subscription.
"""
if self._sub is None:
return False
CTaosInterface.unsubscribe(self._sub, keepProgress)
return True
if __name__ == '__main__':
from .connection import TDengineConnection
conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test")
# Generate a cursor object to run SQL commands
sub = conn.subscribe(True, "test", "select * from meters;", 1000)
for i in range(0,10):
data = sub.consume()
for d in data:
print(d)
sub.close()
conn.close()
\ No newline at end of file
src/inc/taos.h
浏览文件 @ 69a5dcda
......@@ -116,11 +116,10 @@ DLL_EXPORT void taos_query_a(TAOS *taos, const char *sql, void (*fp)(void *param
DLL_EXPORT void taos_fetch_rows_a(TAOS_RES *res, void (*fp)(void *param, TAOS_RES *, int numOfRows), void *param);
DLL_EXPORT void taos_fetch_row_a(TAOS_RES *res, void (*fp)(void *param, TAOS_RES *, TAOS_ROW row), void *param);
DLL_EXPORT TAOS_SUB *taos_subscribe(const char *host, const char *user, const char *pass, const char *db, const char *table, int64_t time, int mseconds);
DLL_EXPORT TAOS_ROW taos_consume(TAOS_SUB *tsub);
DLL_EXPORT void taos_unsubscribe(TAOS_SUB *tsub);
DLL_EXPORT int taos_subfields_count(TAOS_SUB *tsub);
DLL_EXPORT TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub);
typedef void (*TAOS_SUBSCRIBE_CALLBACK)(TAOS_SUB* tsub, TAOS_RES *res, void* param, int code);
DLL_EXPORT TAOS_SUB *taos_subscribe(TAOS* taos, int restart, const char* topic, const char *sql, TAOS_SUBSCRIBE_CALLBACK fp, void *param, int interval);
DLL_EXPORT TAOS_RES *taos_consume(TAOS_SUB *tsub);
DLL_EXPORT void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress);
DLL_EXPORT TAOS_STREAM *taos_open_stream(TAOS *taos, const char *sql, void (*fp)(void *param, TAOS_RES *, TAOS_ROW row),
int64_t stime, void *param, void (*callback)(void *));
......
src/inc/taosmsg.h
浏览文件 @ 69a5dcda
......@@ -490,6 +490,7 @@ typedef struct SColumnInfo {
typedef struct SMeterSidExtInfo {
int32_t sid;
int64_t uid;
TSKEY key; // key for subscription
char tags[];
} SMeterSidExtInfo;
......
src/inc/tglobalcfg.h
浏览文件 @ 69a5dcda
......@@ -54,6 +54,7 @@ extern char tsDirectory[];
extern char dataDir[];
extern char logDir[];
extern char scriptDir[];
extern char osName[];
extern char tsMasterIp[];
extern char tsSecondIp[];
......@@ -78,7 +79,6 @@ extern char tsPrivateIp[];
extern short tsNumOfVnodesPerCore;
extern short tsNumOfTotalVnodes;
extern short tsCheckHeaderFile;
extern uint32_t tsServerIp;
extern uint32_t tsPublicIpInt;
extern short tsAffectedRowsMod;
......
src/inc/ttime.h
浏览文件 @ 69a5dcda
......@@ -42,6 +42,7 @@ int64_t taosGetTimestamp(int32_t precision);
int32_t getTimestampInUsFromStr(char* token, int32_t tokenlen, int64_t* ts);
int32_t taosParseTime(char* timestr, int64_t* time, int32_t len, int32_t timePrec);
void deltaToUtcInitOnce();
#ifdef __cplusplus
}
......
src/kit/shell/CMakeLists.txt
浏览文件 @ 69a5dcda
......@@ -9,6 +9,7 @@ INCLUDE_DIRECTORIES(inc)
IF ((TD_LINUX_64) OR (TD_LINUX_32 AND TD_ARM))
AUX_SOURCE_DIRECTORY(./src SRC)
LIST(REMOVE_ITEM SRC ./src/shellWindows.c)
LIST(REMOVE_ITEM SRC ./src/shellDarwin.c)
ADD_EXECUTABLE(shell ${SRC})
TARGET_LINK_LIBRARIES(shell taos_static)
SET_TARGET_PROPERTIES(shell PROPERTIES OUTPUT_NAME taos)
......@@ -24,7 +25,9 @@ ELSEIF (TD_WINDOWS_64)
ELSEIF (TD_DARWIN_64)
LIST(APPEND SRC ./src/shellEngine.c)
LIST(APPEND SRC ./src/shellMain.c)
LIST(APPEND SRC ./src/shellWindows.c)
LIST(APPEND SRC ./src/shellDarwin.c)
LIST(APPEND SRC ./src/shellCommand.c)
LIST(APPEND SRC ./src/shellImport.c)
ADD_EXECUTABLE(shell ${SRC})
TARGET_LINK_LIBRARIES(shell taos_static)
SET_TARGET_PROPERTIES(shell PROPERTIES OUTPUT_NAME taos)
......
src/kit/shell/src/shellDarwin.c 0 → 100644
浏览文件 @ 69a5dcda
此差异已折叠。
src/kit/shell/src/shellEngine.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/kit/shell/src/shellImport.c
浏览文件 @ 69a5dcda
......@@ -90,20 +90,12 @@ static void shellParseDirectory(const char *directoryName, const char *prefix, c
static void shellCheckTablesSQLFile(const char *directoryName)
{
char cmd[1024] = { 0 };
sprintf(cmd, "ls %s/tables.sql", directoryName);
sprintf(shellTablesSQLFile, "%s/tables.sql", directoryName);
FILE *fp = popen(cmd, "r");
if (fp == NULL) {
fprintf(stderr, "ERROR: failed to execute:%s, error:%s\n", cmd, strerror(errno));
exit(0);
struct stat fstat;
if (stat(shellTablesSQLFile, &fstat) < 0) {
shellTablesSQLFile[0] = 0;
}
while (fscanf(fp, "%s", shellTablesSQLFile)) {
break;
}
pclose(fp);
}
static void shellMallocSQLFiles()
......
src/kit/shell/src/shellLinux.c
浏览文件 @ 69a5dcda
......@@ -119,13 +119,9 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
static struct argp argp = {options, parse_opt, args_doc, doc};
void shellParseArgument(int argc, char *argv[], struct arguments *arguments) {
char verType[32] = {0};
#ifdef CLUSTER
sprintf(verType, "enterprise version: %s\n", version);
#else
sprintf(verType, "community version: %s\n", version);
#endif
static char verType[32] = {0};
sprintf(verType, "version: %s\n", version);
argp_program_version = verType;
argp_parse(&argp, argc, argv, 0, 0, arguments);
......
src/kit/taosdemo/taosdemo.c
浏览文件 @ 69a5dcda
......@@ -17,6 +17,7 @@
#include <argp.h>
#include <assert.h>
#include <inttypes.h>
#ifndef _ALPINE
#include <error.h>
......@@ -575,7 +576,7 @@ void *readTable(void *sarg) {
double totalT = 0;
int count = 0;
for (int i = 0; i < num_of_tables; i++) {
sprintf(command, "select %s from %s%d where ts>= %ld", aggreFunc[j], tb_prefix, i, sTime);
sprintf(command, "select %s from %s%d where ts>= %" PRId64, aggreFunc[j], tb_prefix, i, sTime);
double t = getCurrentTime();
if (taos_query(taos, command) != 0) {
......@@ -818,7 +819,7 @@ double getCurrentTime() {
void generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary) {
memset(res, 0, MAX_DATA_SIZE);
char *pstr = res;
pstr += sprintf(pstr, "(%ld", timestamp);
pstr += sprintf(pstr, "(%" PRId64, timestamp);
int c = 0;
for (; c < MAX_NUM_DATATYPE; c++) {
......@@ -835,7 +836,7 @@ void generateData(char *res, char **data_type, int num_of_cols, int64_t timestam
} else if (strcasecmp(data_type[i % c], "int") == 0) {
pstr += sprintf(pstr, ", %d", (int)(rand() % 10));
} else if (strcasecmp(data_type[i % c], "bigint") == 0) {
pstr += sprintf(pstr, ", %ld", rand() % 2147483648);
pstr += sprintf(pstr, ", %" PRId64, rand() % 2147483648);
} else if (strcasecmp(data_type[i % c], "float") == 0) {
pstr += sprintf(pstr, ", %10.4f", (float)(rand() / 1000));
} else if (strcasecmp(data_type[i % c], "double") == 0) {
......
src/kit/taosdump/taosdump.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/modules/http/src/httpJson.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/modules/http/src/restJson.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/modules/http/src/tgHandle.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/modules/monitor/src/monitorSystem.c
浏览文件 @ 69a5dcda
......@@ -217,9 +217,7 @@ void monitorInitDatabaseCb(void *param, TAOS_RES *result, int code) {
if (monitor->cmdIndex == MONITOR_CMD_CREATE_TB_LOG) {
taosLogFp = monitorSaveLog;
taosLogSqlFp = monitorExecuteSQL;
#ifdef CLUSTER
taosLogAcctFp = monitorSaveAcctLog;
#endif
monitorLPrint("dnode:%s is started", tsPrivateIp);
}
monitor->cmdIndex++;
......
src/os/darwin/inc/os.h
浏览文件 @ 69a5dcda
此差异已折叠。
src/os/darwin/src/tdarwin.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/os/linux/src/tlinux.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/os/linux/src/tsystem.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/os/windows/src/twindows.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/rpc/src/trpc.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/rpc/src/tudp.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/sdb/src/sdbEngine.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/dnodeMgmt.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/mgmtMeter.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/mgmtShell.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeCache.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeCommit.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeFile.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeImport.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeMeter.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeRead.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeShell.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeStore.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/system/detail/src/vnodeUtil.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/CMakeLists.txt
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/tbase64.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/tcache.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/textbuffer.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/tglobalcfg.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/thistogram.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/tnote.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/tsocket.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/ttime.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/ttypes.c
浏览文件 @ 69a5dcda
此差异已折叠。
src/util/src/version.c
浏览文件 @ 69a5dcda
此差异已折叠。
tests/examples/c/CMakeLists.txt
浏览文件 @ 69a5dcda
此差异已折叠。
tests/examples/c/makefile
浏览文件 @ 69a5dcda
此差异已折叠。
tests/examples/c/subscribe.c
浏览文件 @ 69a5dcda
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或