diff --git a/alert/cmd/alert/main.go b/alert/cmd/alert/main.go index f934a11d42237546cd3bbd6f91b48d3af5997e0f..f4c30e156a742c5edc5399dfaf318bedd3ceb1ce 100644 --- a/alert/cmd/alert/main.go +++ b/alert/cmd/alert/main.go @@ -119,7 +119,7 @@ WantedBy=multi-user.target return nil } -const version = "TDengine alert v2.0.0.0" +const version = "TDengine alert v2.0.0.1" func main() { var ( diff --git a/documentation20/webdocs/markdowndocs/cluster-ch.md b/documentation20/webdocs/markdowndocs/cluster-ch.md index 097433a18a900779d8bc25abdcb5afa0a30f4a3f..a1ac1d6fd6047aa6b4b80f41945d67fe505ae9db 100644 --- a/documentation20/webdocs/markdowndocs/cluster-ch.md +++ b/documentation20/webdocs/markdowndocs/cluster-ch.md @@ -2,13 +2,15 @@ 多个taosd的运行实例可以组成一个集群,以保证TDengine的高可靠运行,并提供水平扩展能力。要了解TDengine 2.0的集群管理,需要对集群的基本概念有所了解,请看TDengine 2.0整体架构一章。而且在安装集群之前,请按照[《立即开始》](https://www.taosdata.com/cn/getting-started20/)一章安装并体验过单节点功能。 -集群的每个节点是由End Point来唯一标识的,End Point是由FQDN(Fully Qualified Domain Name)外加Port组成,比如 h1.taosdata.com:6030。一般FQDN就是服务器的hostname,可通过Linux命令`hostname -f`获取。端口是这个节点对外服务的端口号,缺省是6030,但可以通过taos.cfg里配置参数serverPort进行修改。一个节点可能配置了多个hostname, TDengine会自动获取第一个,但也可以通过taos.cfg里配置参数fqdn进行指定。如果习惯IP地址直接访问,可以将参数fqdn设置为本节点的IP地址。 +集群的每个节点是由End Point来唯一标识的,End Point是由FQDN(Fully Qualified Domain Name)外加Port组成,比如 h1.taosdata.com:6030。一般FQDN就是服务器的hostname,可通过Linux命令`hostname -f`获取,FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)。端口是这个节点对外服务的端口号,缺省是6030,但可以通过taos.cfg里配置参数serverPort进行修改。一个节点可能配置了多个hostname, TDengine会自动获取第一个,但也可以通过taos.cfg里配置参数fqdn进行指定。如果习惯IP地址直接访问,可以将参数fqdn设置为本节点的IP地址。 TDengine的集群管理极其简单,除添加和删除节点需要人工干预之外,其他全部是自动完成,最大程度的降低了运维的工作量。本章对集群管理的操作做详细的描述。 ## 准备工作 **第一步**:如果搭建集群的节点中,存有之前的测试数据、装过1.X的版本,或者装过其他版本的TDengine,请先将其删除,并清空所有数据,具体步骤请参考博客[《TDengine多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html ) +**注意1:**因为FQDN的信息会写进文件,如果之前没有配置或者更改FQDN,且启动了TDengine。请一定在确保数据无用或者备份的前提下,清理一下之前的数据(rm -rf /var/lib/taos/); +**注意2:**客户端也需要配置,确保它可以正确解析每个节点的FQDN配置,不管是通过DNS服务,还是 Host 文件。 **第二步**:建议关闭防火墙,至少保证端口:6030 - 6042的TCP和UDP端口都是开放的。**强烈建议**先关闭防火墙,集群搭建完毕之后,再来配置端口; @@ -23,7 +25,7 @@ TDengine的集群管理极其简单,除添加和删除节点需要人工干预 **第五步**:修改TDengine的配置文件(所有节点的文件/etc/taos/taos.cfg都需要修改)。假设准备启动的第一个节点End Point为 h1.taosdata.com:6030, 那么以下几个参数与集群相关: ``` -// firstEp 是每个节点启动后连接的第一个节点 +// firstEp 集群中所有节点的配置都是一致的,对其第一次访问后,就获得了整个集群的信息 firstEp h1.taosdata.com:6030 // 配置本节点的FQDN,如果本机只有一个hostname, 无需配置 @@ -32,7 +34,7 @@ fqdn h1.taosdata.com // 配置本节点的端口号,缺省是6030 serverPort 6030 -// 副本数为偶数的时候,需要配置,请参考《Arbitrator的使用》的部分 +// 服务端节点数为偶数的时候,需要配置,请参考《Arbitrator的使用》的部分 arbitrator ha.taosdata.com:6042 ``` diff --git a/documentation20/webdocs/markdowndocs/faq-ch.md b/documentation20/webdocs/markdowndocs/faq-ch.md index e4bb920dd3758174f0adc6313b9c824639c02323..7bbf7531c805eed1b3711cbe67a6dcfbc7fae3a1 100644 --- a/documentation20/webdocs/markdowndocs/faq-ch.md +++ b/documentation20/webdocs/markdowndocs/faq-ch.md @@ -32,7 +32,7 @@ 3. 在服务器,执行 `systemctl status taosd` 检查*taosd*运行状态。如果没有运行,启动*taosd* -4. 确认客户端连接时指定了正确的服务器FQDN (Fully Qualified Domain Name(可在服务器上执行Linux命令hostname -f获得) +4. 确认客户端连接时指定了正确的服务器FQDN (Fully Qualified Domain Name(可在服务器上执行Linux命令hostname -f获得)),FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)。 5. ping服务器FQDN,如果没有反应,请检查你的网络,DNS设置,或客户端所在计算机的系统hosts文件 @@ -54,7 +54,7 @@ ## 6. 遇到错误“Unexpected generic error in RPC”或者"TDengine Error: Unable to resolve FQDN", 我怎么办? 产生这个错误,是由于客户端或数据节点无法解析FQDN(Fully Qualified Domain Name)导致。对于TAOS Shell或客户端应用,请做如下检查: -1. 请检查连接的服务器的FQDN是否正确 +1. 请检查连接的服务器的FQDN是否正确,FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)。 2. 如果网络配置有DNS server, 请检查是否正常工作 3. 如果网络没有配置DNS server, 请检查客户端所在机器的hosts文件,查看该FQDN是否配置,并是否有正确的IP地址。 4. 如果网络配置OK,从客户端所在机器,你需要能Ping该连接的FQDN,否则客户端是无法链接服务器的 diff --git a/src/client/src/tscServer.c b/src/client/src/tscServer.c index ae2013cd2b2abe3de92b09c4fedc3bc86c9f2fbf..c9fe5fbd2cd373749147fb05071335cfa8c7149d 100644 --- a/src/client/src/tscServer.c +++ b/src/client/src/tscServer.c @@ -2090,6 +2090,10 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) { SSqlCmd *pCmd = &pSql->cmd; SRetrieveTableRsp *pRetrieve = (SRetrieveTableRsp *)pRes->pRsp; + if (pRetrieve == NULL) { + pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; + return pRes->code; + } pRes->numOfRows = htonl(pRetrieve->numOfRows); pRes->precision = htons(pRetrieve->precision); diff --git a/src/connector/jdbc/src/test/java/com/taosdata/jdbc/SubscribeTest.java b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/SubscribeTest.java index 07b43d1227a55538026e3c7d8d35d1e6f5c9a416..0a71c77d1d9dc718cc029c6a0b11ed63cd3034c1 100644 --- a/src/connector/jdbc/src/test/java/com/taosdata/jdbc/SubscribeTest.java +++ b/src/connector/jdbc/src/test/java/com/taosdata/jdbc/SubscribeTest.java @@ -10,8 +10,6 @@ import java.sql.SQLException; import java.sql.Statement; import java.util.Properties; -import static org.junit.Assert.assertTrue; - public class SubscribeTest extends BaseTest { Connection connection = null; Statement statement = null; diff --git a/src/mnode/src/mnodeSdb.c b/src/mnode/src/mnodeSdb.c index 60f1e8e80bb9e364884c1c40ebf3bbb4fee5b871..646c17b2b8b64344afb7c5117f2bd83b3da89dab 100644 --- a/src/mnode/src/mnodeSdb.c +++ b/src/mnode/src/mnodeSdb.c @@ -65,6 +65,7 @@ typedef struct _SSdbTable { int32_t (*encodeFp)(SSdbOper *pOper); int32_t (*destroyFp)(SSdbOper *pOper); int32_t (*restoredFp)(); + pthread_mutex_t mutex; } SSdbTable; typedef struct { @@ -455,8 +456,9 @@ static void *sdbGetRowMeta(SSdbTable *pTable, void *key) { } void **ppRow = (void **)taosHashGet(pTable->iHandle, key, keySize); - if (ppRow == NULL) return NULL; - return *ppRow; + if (ppRow != NULL) return *ppRow; + + return NULL; } static void *sdbGetRowMetaFromObj(SSdbTable *pTable, void *key) { @@ -464,13 +466,14 @@ static void *sdbGetRowMetaFromObj(SSdbTable *pTable, void *key) { } void *sdbGetRow(void *handle, void *key) { + SSdbTable *pTable = handle; + + pthread_mutex_lock(&pTable->mutex); void *pRow = sdbGetRowMeta(handle, key); - if (pRow) { - sdbIncRef(handle, pRow); - return pRow; - } else { - return NULL; - } + if (pRow) sdbIncRef(handle, pRow); + pthread_mutex_unlock(&pTable->mutex); + + return pRow; } static void *sdbGetRowFromObj(SSdbTable *pTable, void *key) { @@ -485,7 +488,9 @@ static int32_t sdbInsertHash(SSdbTable *pTable, SSdbOper *pOper) { keySize = strlen((char *)key); } + pthread_mutex_lock(&pTable->mutex); taosHashPut(pTable->iHandle, key, keySize, &pOper->pObj, sizeof(int64_t)); + pthread_mutex_unlock(&pTable->mutex); sdbIncRef(pTable, pOper->pObj); atomic_add_fetch_32(&pTable->numOfRows, 1); @@ -526,7 +531,10 @@ static int32_t sdbDeleteHash(SSdbTable *pTable, SSdbOper *pOper) { keySize = strlen((char *)key); } + pthread_mutex_lock(&pTable->mutex); taosHashRemove(pTable->iHandle, key, keySize); + pthread_mutex_unlock(&pTable->mutex); + atomic_sub_fetch_32(&pTable->numOfRows, 1); sdbDebug("table:%s, delete record:%s from hash, numOfRows:%" PRId64 ", msg:%p", pTable->tableName, @@ -868,6 +876,7 @@ void *sdbOpenTable(SSdbTableDesc *pDesc) { if (pTable == NULL) return NULL; + pthread_mutex_init(&pTable->mutex, NULL); tstrncpy(pTable->tableName, pDesc->tableName, SDB_TABLE_LEN); pTable->keyType = pDesc->keyType; pTable->tableId = pDesc->tableId; @@ -915,6 +924,7 @@ void sdbCloseTable(void *handle) { taosHashDestroyIter(pIter); taosHashCleanup(pTable->iHandle); + pthread_mutex_destroy(&pTable->mutex); sdbDebug("table:%s, is closed, numOfTables:%d", pTable->tableName, tsSdbObj.numOfTables); free(pTable); diff --git a/src/vnode/src/vnodeRead.c b/src/vnode/src/vnodeRead.c index 11315f5321f674fa11faeef98d28a1776f42468c..017eeaf4269aa2de78a8167c73a9e9f3596f7096 100644 --- a/src/vnode/src/vnodeRead.c +++ b/src/vnode/src/vnodeRead.c @@ -93,11 +93,12 @@ static int32_t vnodeDumpQueryResult(SRspRet *pRet, void* pVnode, void** handle, vDebug("QInfo:%p exec completed, free handle:%d", *handle, *freeHandle); } } else { - SRetrieveTableRsp* pRsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp)); + SRetrieveTableRsp *pRsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp)); memset(pRsp, 0, sizeof(SRetrieveTableRsp)); pRsp->completed = true; pRet->rsp = pRsp; + pRet->len = sizeof(SRetrieveTableRsp); *freeHandle = true; } @@ -270,6 +271,7 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) { if (code != TSDB_CODE_SUCCESS) { //TODO handle malloc failure pRet->rsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp)); + pRet->len = sizeof(SRetrieveTableRsp); memset(pRet->rsp, 0, sizeof(SRetrieveTableRsp)); freeHandle = true; } else { // result is not ready, return immediately diff --git a/tests/examples/JDBC/JDBCDemo/pom.xml b/tests/examples/JDBC/JDBCDemo/pom.xml index f0234f2bd762bd56fa0668af0d2b345d25320fea..92d757edfdea304084b5971613da231db2b9daef 100644 --- a/tests/examples/JDBC/JDBCDemo/pom.xml +++ b/tests/examples/JDBC/JDBCDemo/pom.xml @@ -9,21 +9,20 @@ 1.0-SNAPSHOT jar - - - - org.apache.maven.plugins - maven-plugins - 30 - - - org.apache.maven.plugins - maven-assembly-plugin - 3.0.0 - - - + + + org.apache.maven.plugins + maven-plugins + 30 + + + + org.apache.maven.plugins + maven-assembly-plugin + 3.0.0 + + org.apache.maven.plugins maven-assembly-plugin @@ -48,6 +47,7 @@ + org.apache.maven.plugins maven-compiler-plugin diff --git a/tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JDBCConnectorChecker.java b/tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JDBCConnectorChecker.java new file mode 100644 index 0000000000000000000000000000000000000000..1e801bc65894ef437ee41d692f9ef725b4f639ef --- /dev/null +++ b/tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JDBCConnectorChecker.java @@ -0,0 +1,174 @@ +package com.taosdata.example; + +import com.taosdata.jdbc.TSDBDriver; + +import java.sql.*; +import java.util.Properties; + +public class JDBCConnectorChecker { + private static String host; + private static String dbName = "test"; + private static String tbName = "weather"; + private Connection connection; + + + /** + * get connection + **/ + private void init() { + try { + Class.forName("com.taosdata.jdbc.TSDBDriver"); + Properties properties = new Properties(); + properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host); + properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); + properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); + properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); + System.out.println("get connection starting..."); + connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/", properties); + if (connection != null) + System.out.println("[ OK ] Connection established."); + } catch (ClassNotFoundException | SQLException e) { + throw new RuntimeException("connection failed: " + host); + } + } + + /** + * create database + */ + private void createDatabase() { + String sql = "create database if not exists " + dbName; + exuete(sql); + } + + /** + * use database + */ + private void useDatabase() { + String sql = "use " + dbName; + exuete(sql); + } + + /** + * select + */ + private void checkSelect() { + final String sql = "select * from test.weather"; + executeQuery(sql); + } + + private void executeQuery(String sql) { + try (Statement statement = connection.createStatement()) { + long start = System.currentTimeMillis(); + ResultSet resultSet = statement.executeQuery(sql); + long end = System.currentTimeMillis(); + printSql(sql, true, (end - start)); + printResult(resultSet); + } catch (SQLException e) { + e.printStackTrace(); + } + } + + private void printResult(ResultSet resultSet) throws SQLException { + ResultSetMetaData metaData = resultSet.getMetaData(); + while (resultSet.next()) { + for (int i = 1; i <= metaData.getColumnCount(); i++) { + String columnLabel = metaData.getColumnLabel(i); + String value = resultSet.getString(i); + System.out.printf("%s: %s\t", columnLabel, value); + } + System.out.println(); + } + } + + private String formatString(String str) { + StringBuilder sb = new StringBuilder(); + int blankCnt = (26 - str.length()) / 2; + for (int j = 0; j < blankCnt; j++) + sb.append(" "); + sb.append(str); + for (int j = 0; j < blankCnt; j++) + sb.append(" "); + sb.append("|"); + return sb.toString(); + } + + + /** + * insert + */ + private void checkInsert() { + final String sql = "insert into test.weather (ts, temperature, humidity) values(now, 20.5, 34)"; + exuete(sql); + } + + /** + * create table + */ + private void createTable() { + final String sql = "create table if not exists " + dbName + "." + tbName + " (ts timestamp, temperature float, humidity int)"; + exuete(sql); + } + + private final void printSql(String sql, boolean succeed, long cost) { + System.out.println("[ " + (succeed ? "OK" : "ERROR!") + " ] time cost: " + cost + " ms, execute statement ====> " + sql); + } + + private final void exuete(String sql) { + try (Statement statement = connection.createStatement()) { + long start = System.currentTimeMillis(); + boolean execute = statement.execute(sql); + long end = System.currentTimeMillis(); + printSql(sql, execute, (end - start)); + } catch (SQLException e) { + e.printStackTrace(); + } + } + + private void close() { + try { + if (connection != null) { + this.connection.close(); + System.out.println("connection closed."); + } + } catch (SQLException e) { + e.printStackTrace(); + } + } + + private void checkDropTable() { + final String sql = "drop table if exists " + dbName + "." + tbName + ""; + exuete(sql); + } + + public static void main(String[] args) { + for (int i = 0; i < args.length; i++) { + if ("-host".equalsIgnoreCase(args[i]) && i < args.length - 1) { + host = args[++i]; + } + if ("-db".equalsIgnoreCase(args[i]) && i < args.length - 1) { + dbName = args[++i]; + } + if ("-t".equalsIgnoreCase(args[i]) && i < args.length - 1) { + tbName = args[++i]; + } + } + + if (host == null) { + System.out.println("Usage: java -jar JDBCConnectorChecker.jar -host "); + return; + } + + JDBCConnectorChecker checker = new JDBCConnectorChecker(); + checker.init(); + checker.createDatabase(); + checker.useDatabase(); + checker.checkDropTable(); + checker.createTable(); + checker.checkInsert(); + checker.checkSelect(); + checker.checkDropTable(); + checker.close(); + } + + +} diff --git a/tests/pytest/fulltest.sh b/tests/pytest/fulltest.sh index d600a003b08ab296245c06c0193552e628b2710d..f6c68378eaac754bf8fe748f4a65737cadc66e57 100755 --- a/tests/pytest/fulltest.sh +++ b/tests/pytest/fulltest.sh @@ -185,6 +185,7 @@ python3 ./test.py -f functions/function_stddev.py python3 ./test.py -f functions/function_sum.py python3 ./test.py -f functions/function_top.py #python3 ./test.py -f functions/function_twa.py +python3 queryCount.py # tools python3 test.py -f tools/taosdemo.py diff --git a/tests/pytest/queryCount.py b/tests/pytest/queryCount.py index 3281c0926964e282cd8254f4ec9b975c601ef654..7cc8f61f4dd4790cfc8c2229675eab87b440fadf 100644 --- a/tests/pytest/queryCount.py +++ b/tests/pytest/queryCount.py @@ -80,8 +80,8 @@ class QueryCountMultiThread: if(data[0][0] == 10000000): tdLog.info("sql:%s, row:%d col:%d data:%d == expect:%d" % (sql, 0, 0, data[0][0], 10000000)) - else: - tdLog.exit("sql:%s, row:%d col:%d data:%d == expect:%d" % (sql, 0, 0, data[0][0], 10000000)) + else: + tdLog.exit("queryCount.py failed: sql:%s failed, row:%d col:%d data:%d != expect:%d" % (sql, 0, 0, data[0][0], 10000000)) cursor.close() self.conn.close()